ZenIp: 42400 - Modernizing Horizen Technology via a Modular Proof Verification Layer

ZenIp: 42400
Title: “Modernizing Horizen Technology via a Modular Proof Verification Layer”
Owner: Diego Alvarez, dalvarez@horizenlabs.io
Comments-URI: ZenIp: 42400 - Modernizing Horizen Technology via a Modular Proof Verification Layer
Status: Draft
Type: Standard
Created: 02-07-2024

Abstract

The objective of this ZenIP is to offer a comprehensive understanding of the “New Horizen” initiative, as initially presented on Discourse. This proposal details our driving motives behind creating the system, outlines the strategic approach we plan to adopt, and delves into the technical aspects of its first stage, a modular proof verification layer.

Like any significant technological breakthrough, the “New Horizen” initiative will be rolled out in several stages. However, our immediate priority is to facilitate a decision on whether Horizen Labs should begin the development activities for Phase 1, as detailed below. Comprehensive information regarding the later phases will be presented in a step-by-step manner, aligning with the progression of the development.

In its final form, the “New Horizen” system will encompass two separate blockchains:
1. A Modular Proof Verification Layer: This blockchain will be solely dedicated to the verification of proofs, with the capacity to support any settlement layer. This represents the focus of this proposal.
2. A Modular Settlement Layer: This separate, second blockchain will be exclusively tasked with settling proofs and managing the states of other blockchains. Designed as an alternative to traditional, monolithic chains, its focus is solely on settlement processes, aiming for reduced and more predictable costs. This innovative approach will facilitate the creation of Layer 2 solutions and other networks directly within the Horizen ecosystem.

This ZenIP focuses exclusively on the first point, which entails the establishment of a modular Proof Verification Layer. Throughout this proposal, the terms “Proof Verification Layer” and “Proof Verification Chain” will be used interchangeably.

Before exploring further details, it is important to address several queries and comments raised by community members on Discourse and Discord:

  • Name of the Initiative: The term “New Horizen” is currently a placeholder for the system and its individual component blockchains. The designation is being used in this document to conceptually describe the initiative until an official name is determined.
  • EON Integration: A key aspect of our development strategy is the integration of EON into the new Proof Verification Chain. Our teams are actively involved in exploring viable pathways for this migration, ensuring that EON remains an integral part of the Horizen ecosystem after the implementation of the blockchains that comprise the “New Horizen” initiative.
  • $ZEN Tokenomics: The modernization process will preserve the balances and tokenomics of $ZEN. Currently, there are no plans to modify $ZEN’s tokenomics, and any such change would be subject to community discussion and DAO approval. These matters are not within the scope of this proposal.
  • Market and User Research: Our approach is informed by comprehensive market and user research, aiming to design a user-friendly system capable of widespread adoption. We have shifted from a solely technology-focused paradigm, learning from previous experiences. Our new philosophy is not just to create excellent technology that attracts developers, but to build platforms that meet the actual needs and preferences of their users.

Our proposals to the DAO encompass the following key permission and recognition:

  • Branding Authorization: We seek permission to utilize the Horizen name and all associated Horizen trademarks for the branding of technical components like the Proof Verification Layer and the future Settlement Layer. This authorization will allow us to align closely with the Horizen brand identity and leverage its established reputation in our communications and marketing efforts.

Motivation

1. To Modernize Horizen Tech To Be Aligned With The Broader Industry
While Horizen’s past releases have been ambitious, they often faced challenges due to their complexity and reliance on technologies that much of the industry has since moved away from, such as Proof of Work (PoW) and the UTXO model. Despite these hurdles, the fundamental goals they aimed to tackle—enhancing scalability, bolstering security, and providing customizability—remain highly relevant in today’s blockchain landscape. The “New Horizen” initiative is a strategic evolution of these goals, building on the legacy of the Horizen Mainchain and Zendoo.

At Horizen Labs, as a key technology implementation partner for Horizen, we are adopting a renewed approach in our technology development. This initiative aims to recalibrate Horizen’s technology, ensuring it meets modern industry standards and is informed by our extensive research and the latest market trends.

2. To Future-Proof Horizen Technology And Develop It Based On Market Needs
In recent years, Layer 2 solutions (L2s) like optimistic rollups and ZK-Rollups have emerged to address Ethereum’s scaling challenges, gaining significant traction. These innovations have eased some of Ethereum’s load by offloading execution layer tasks. However, their heavy reliance on Ethereum for settlement has paradoxically contributed to the very congestion they sought to alleviate. The growing popularity of L2s not only highlights the evolving demands of the blockchain space but also underscores the limitations of their dependence on monolithic chains like Ethereum for settlement tasks.

The rise of L2s and the challenge posed by their reliance on Ethereum for Data Availability (DA) have led to the development of solutions that separate DA from the other settlement functions. This move towards greater modularization in the blockchain architecture presents a unique opportunity for Horizen. We propose to capitalize on this trend by building a Proof Verification network that is both proof and settlement layer agnostic. This is akin to how DA solutions carved out a new layer by isolating a function traditionally seen as part of settlement, and we envision a similar approach by creating a chain optimized specifically for Proof Verification.

Our proposed dedicated Proof Verification Layer aims to substantially reduce security costs for L2s, particularly ZK-Rollups, as well as for any applications that generate proofs. This initiative addresses current market challenges and anticipates future demands. Considering the rapid development in this field, where over 15 ZK-rollups on Ethereum are already incurring substantial costs (~$47m) in Proof Verification, with more in development, the need for cost-effective solutions is clear.

Our preliminary research suggests the potential for a dramatic reduction in proof verification costs for ZK-Rollups, by as much as 90-95%. This is especially pertinent given the predictions of Ethereum’s co-founder, Vitalik Buterin, that ZK-Rollups will likely become the preferred scaling solution in the future (Source: The Block). A dedicated Proof Verification Chain would not only make running a ZK-Rollup more economical but also aligns with the anticipated shift towards these technologies becoming the dominant scaling solution.

3. To Foster Collaboration Across Ecosystems And Expand The Reach Of Horizen
In line with our strategic vision, the Proof Verification Chain is being meticulously designed to accommodate a variety of proof types and settlement layers. This approach is pivotal as it underscores our commitment to collaboration and seamless integration across diverse ecosystems, marking a significant evolution from Horizen’s previous solutions that were predominantly focused on serving the internal Horizen ecosystem.

By developing a solution that is versatile enough to be compatible with multiple development frameworks, proving systems, and settlement layers, Horizen is positioning itself as a hub of collaborative innovation. This broader compatibility not only allows Horizen to engage with leading protocols in the industry for collaborative ventures rather than competition, but it also opens avenues for the Horizen brand to reach new audiences beyond its current community. Such an inclusive and adaptable approach is vital for positioning Horizen for sustained growth in an industry that is progressively leaning towards composability and collaborative dynamics. This strategic shift is not just about expanding our technological capabilities; it’s about embracing and leading the change towards a more interconnected and cooperative blockchain environment.

Specification

The “New Horizen” initiative is set to revolutionize blockchain technology by adopting a new modular approach. It will deliver multiple components that address the market’s current and future needs. Central to “New Horizen” is the Proof Verification Chain, which will focus on verifying proofs originating from other blockchains, such as rollups or appchains.

At this initial stage, we broadly refer to “proofs” to highlight our vision for the versatility of the Proof Verification Chain. Initially, our focus will be on ZK SNARK proofs, addressing the immediate market need and ease of integration, particularly for ZK-Rollups. However, this is just the starting point, as we plan to expand its capabilities to accommodate a wide range of proof types in the future.

The Proof Verification Layer aims to streamline the process of proving state validity for ZK-Rollups and other proof generating applications, offering a more efficient, scalable solution to handle these operations, thereby enhancing the overall performance and capability of blockchain networks.

The current process for verifying the proofs and settling the states for ZK-Rollups today, is as follows:

  1. Transaction Initiation: Users on the rollup platform carry out their transactions.
  2. State Update Preparation: After a set period, the rollup system aggregates the recent transactions and proposes a new candidate state to the settlement layer.
  3. Proof Submission: Subsequently, the rollup submits proof to the settlement layer. This proof is crucial as it validates that the state transition adheres to specific rules and is legitimate.
  4. Proof Verification by Validators: The validators within the settlement layer check the proof. Their role is to ensure its validity and adherence to the defined rules. This step is computationally intense and therefore costly, especially on networks not explicitly designed for this functionality.
  5. State Consolidation: Once the proof is verified and deemed valid, the settlement layer finalizes and integrates the rollup state. This integrated state then becomes immutable, signifying a successful update.

This process highlights the critical steps involved in the rollup mechanism, from transaction initiation to the final consolidation of the updated state, ensuring a secure and verifiable transition within the blockchain framework.

Phase 1 of the “New Horizen” initiative is designed to introduce a layer specifically optimized for proof verification. This can be integrated into processes like the one described previously to alleviate the load on the settlement layer and reduce operational costs:

  1. User Transactions on the Rollup: Users engage in transactions within the rollup environment.
  2. State Update Preparation: At regular intervals, the rollup system compiles the latest transactions and proposes a new candidate state to the settlement layer.
  3. [New] Proof Submission to the Proof Verification Chain: The rollup publishes the proof on the Proof Verification Chain. The objective here is to validate the state transition as per the established rules.
  4. [New] Proof Verification by the Proof Verification Chain’s Validators: Validators within the Proof Verification Chain’s network undertake the responsibility of verifying this proof.
  5. [New] Attestation by the Proof Verification Chain: Upon successful verification, the Proof Verification Chain issues an attestation to the original settlement layer (like Ethereum) where the rollup is deployed. This attestation confirms the validity of the latest batch of transactions or state.
  6. [New] Final Consolidation by Settlement Layer Validation: The validators of the original settlement layer then review the Proof Verification Chain’s attestation. Upon confirmation, they finalize and integrate the rollup state.

This integration of the Proof Verification Chain streamlines the process, offloading the settlement layer from the intensive task of verifying proofs. This not only enhances efficiency but also significantly reduces the associated costs and resource utilization, leading to a more scalable and economical blockchain ecosystem.

The Proof Verification Chain will be established on an entirely new technological framework. A key component of this framework is the use of the Substrate SDK. Substrate is recognized as a leading platform for blockchain development, owing to its exceptional flexibility. It comes equipped with a variety of secure, battle-tested components, enabling developers to concentrate primarily on crafting the specific application logic. In our case, this pertains to the development of the verification layer. Notably, the new blockchain developed under New Horizen will operate independently of the Polkadot ecosystem, meaning it won’t function as a parachain linked to the Polkadot relay chain. Additionally, it will adopt a Proof of Stake (PoS) consensus algorithm to ensure robust network security.

A pivotal aspect of the Proof Verification Chain’s success hinges on its ease of use and the simplicity of its integration by third parties. For this reason, it is designed to be as universally adaptable as possible. It aims to accommodate proofs from any rollup or appchain, irrespective of their specific proving system or the intricacies of their implementation. This broad compatibility underscores our commitment to making the Proof Verification Chain a versatile and user-friendly technology that can seamlessly interface with a wide array of blockchain systems.

To showcase the versatility and strengths of the verification layer, our first step will involve integrating a singular ZK-Rollup development framework from an execution layer partner, whose identity will be revealed in due course. Our selection will be meticulously strategic, prioritizing not just technological compatibility but also additional factors like the potential partner’s commitment to modularity and composability. The initial collaboration with our chosen execution layer partner is designed to highlight the Proof Verification Chain’s adaptability and capacity to cater to a broad spectrum of blockchain applications, demonstrating its potential to revolutionize blockchain technology with enhanced efficiency and scalability.

In contrast to the standard operation of a L2 developed using a ZK-Rollup development kit and implemented on Ethereum, the workflow integrating with the Proof Verification Chain will proceed as follows:

  1. Standard Rollup Deployment: The rollup initially deploys the required smart contracts and Layer 2 (L2) components in line with the typical development kit’s methodology.
    • [New] Smart Contract Modification: Just one smart contract needs to be modified to facilitate SNARK proof verification through the Proof Verification Chain, rather than relying on Ethereum validators.
  2. Transaction Batching: The rollup aggregates transactions into batches.
  3. Proof Generation: The rollup produces a proof to authenticate the batch of transactions.
  4. [New] Proof Submission to the Proof Verification Chain: Instead of sending the SNARK proof directly to an Ethereum smart contract, the rollup forwards it to the Proof Verification Chain.
  5. [New] Proof Verification by the Proof Verification Chain: The Proof Verification Chain is responsible for the verification of the submitted SNARK proof.
  6. [New] Attestation on Ethereum: Following a successful verification, the Proof Verification Chain publishes an attestation on Ethereum. This serves as a validation receipt, confirming the legitimacy of the SNARK proof associated with the latest transaction batch.
  7. Ethereum State Acceptance: Subsequently, Ethereum acknowledges and accepts the updated rollup state as valid.

This modified workflow illustrates how the Proof Verification Chain can streamline the verification process for ZK-Rollups, enhancing efficiency and security while offloading significant computational demands from the Ethereum network.

Rationale

  • Addressing Market Needs: The current usage of our mainchain and Zendoo are somewhat limited. By introducing new technology that is attuned to both existing and emerging market demands, we anticipate stimulating significant activity within the Horizen ecosystem—a feature that has been notably absent historically. In the past, Horizen was primarily technology-led rather than guided by addressing the current and evolving needs of the market. This new direction aims to bridge that gap.
  • Embracing Modularization In The Industry: The blockchain industry is progressively moving away from monolithic, “one-size-fits-all” models towards a more dynamic and flexible modular approach. It’s crucial for Horizen to seize this pivotal opportunity now, to stay ahead in the rapidly evolving blockchain landscape. By doing so, we can ensure that we are not just keeping pace but leading the way in this industry transformation.
  • Promoting Inter-Ecosystem Collaboration: The trend towards collaboration across different ecosystems is accelerating. Adopting a settlement layer agnostic solution aligns perfectly with this trend. It will enable us to collaborate more effectively with industry leaders and foster an unprecedented level of interconnectedness between Horizen and other ecosystems. This approach is not just about technological innovation; it’s about integrating Horizen more deeply into the broader blockchain community and forging alliances that can lead to shared growth and success.

Challenges and Considerations

Migrating the existing Horizen ecosystem to the innovative Proof Verification Chain presents several challenges that require meticulous attention and strategic planning.

  1. Transitioning from PoW to PoS:
  • In the past, Horizen technologies have largely relied on Proof-of-Work consensus mechanisms, which were integral to the Horizen identity at its inception. However, with the release of EON, Horizen has started incorporating Proof-of-Stake consensus mechanisms into its most recent releases. The implementation of the Proof Verification Chain as a PoS-based chain advances Horizen’s transition towards a predominantly PoS ecosystem, signaling a further departure from the historical precedence given to PoW by the Horizen community.
  1. Possibility of Limited Community Adoption:
  • Given the proposed significant technological departure from the technologies that the Horizen community has used in the past, we must acknowledge the possibility of existing users potentially not adopting the new Proof Verification Chain. Therefore, engaging the existing community and facilitating the migration to and utilization of this new chain is crucial.
  1. Initial Security of the New PoS Chain:
  • PoS chains derive their security level from the value and decentralization of total staked assets by validators. Therefore, there is a need for mechanisms to bolster network security, especially during the initial phase following token creation or, as in Horizen’s case, during the transition of funds from one chain to another.
  1. Secure Fund Migration:
  • Transferring funds from one blockchain to another is a highly sensitive operation that demands the utmost security. This is to ensure that users’ assets are not at risk of being locked, inaccessible, or double-spendable

These preliminary security considerations are under thorough examination to ensure the smoothest and safest transition possible for our users. While navigating these challenges is inevitable, we are committed to addressing them effectively, always relying on the decentralized governance principles of the Horizen DAO. This process is not just a technical upgrade; it’s a step towards a more robust, adaptable, and future-proof blockchain ecosystem.

13 Likes

Exciting stuff and I’m personally excited to hear any feedback or questions from the community on this one. Feel free to tag me in any questions, comments or concerns!

4 Likes

So excited to see this moving ahead - I see it as the culmination of the expertise in cryptography and blockchain tech that the Horizen team has been developing since the start of the project, and applying it to solving a major problem in the space today. Let’s do this!!

6 Likes

How would the newly issued Zen coins be allocated in this new system. What is the role of super nodes, EON forger nodes / secure nodes? Are they going to be obsolete?
It used to be:
20% DAO/HL funds
10% super nodes
10% secure nodes / EON Forger nodes
60% miners.

Can you tell me the new rules of coin issuance in this new system?

See here @Trololino.

EON forger nodes are completely separate, and pertain to EON.

This proposal is for a proof-of-stake system. Therefore Proof of Stake Validators will receive a portion of the block reward.

All of this is up for discussion in a future ZenIP as the quote states.

2 Likes

Hi John

Am I right in thinking that Proof of Stakr validators are different from forger nodes?
If so, what can holders do to become validators?

What are the anticipated volume and speed capabilities of the proof verification chain? I ask that question specifically wondering if we have calculated the computing power and other specifics that will be necessary to preform these proofs in a satisfactory time and way in order to capture this segment of the market? For instance, at what proof verification volume will the network through-put be slowed down? Also, I am assuming the 4,000 super nodes will be processing the proof verification and would need better hardware spec’s. Is that a correct assumption?
If it’s too early in development to have answers to these questions I understand.

1 Like

Totally agree that user-friendliness should be taken into account
Being technically good is one thing, being used by many is another.
I think that no matter how good the technology is, if users don’t use it, it’s as good as nothing.
Timing-wise, I think this is a very good time for a scalability update, and I’m very supportive of it.

5 Likes

Thank you for documenting this idea - I enjoyed reading it.

As I understand the idea, the main-chain will be replaced with a new chain, the “Proof Verification Chain” based on the Substrate SDK.

Rather than replace the main-chain, would it be possible to implement these two new layers (Proof Verification, Settlement) as side-chains using the existing technology (ie sister chains to EON)? Would that reduce development time and bring a viable product to market faster?

Once the new side-chains are in place, could the main-chain then be shifted from PoW to PoS?

1 Like

They’re equivalent. There will be a different set of validator nodes for EON and New Horizen Proof Verification chain. Different term referring to the same thing.

1 Like

It is too early to give details on this. However, we’re envisioning that the validators / forgers will be performing the proof verification. This is actually what happens with Horizen today when it receives proofs from EON / other sidechains.

1 Like

Our assessment is that it will not speed up development. There’s plenty of tech debt on the current Horizen mainchain that needs paying down. This is one of the motivations for the new chain.

Additionally, the sidechains SDK was custom built. This is something that hasn’t been adopted by the market, hence the desire to use other protocols’ SDKs.

Additionally, the migration of a chain from POW to POS is not trivial. In fact, it’s quite challenging. Further reason to create a new chain rather than work further to improve the existing one.

1 Like

I’m incredibly excited! With New Horizen, we’re diving deep into our zero-knowledge cryptography expertise and truly owning one of the key layers in the world of modular blockchains. We’re pushing boundaries and aiming to be leaders in the space.

I firmly believe in a future where interoperability and collaboration are paramount, with modularity playing a crucial role. New Horizen perfectly aligns with this narrative.

Also, I must highlight our engineering team’s talent for thinking outside the box. They’re doing exceptional work but sometimes don’t get enough credit.

3 Likes

We should definitely start crunching some numbers, because we might not have the capability to handle the numbers of proofs needed to build “new Horizen” the way we envisioned.

1 Like

it’s on our radar - there are some other higher priority items that have been prioritized ahead of this.

I am curious though, how many proofs do you think are generated by rollups? and how resource intensive do you think that it is to verify a proof?

1 Like

It sounds like a lot if we want to capture the whole ETH market.

I love the idea of helping other chains perform their rollups to bring adoption to Horizen. Just have a few questions:

  • Is the long term goal to fully deprecate the existing mainchain? Or will miners and super nodes continue to operate as they are, and support the operation of EON?

  • How does this Proof Verification chain make settlement layer transactions cheaper? Settlement layer computational costs will go down, but validators on the verification layers need to be compensated, so there are now 2 sets of validators that need to split rewards from each block. Will ZEN inflation be used to compensate these verification layer validators? If so, where is that ZEN coming from? And what is stopping settlement chains from abusing the fact that none of their own inflation has to go towards verifying rollups?

3 Likes

Filter for zk-rollups. It’s fewer proofs than you think.

We should be fine, but of course will perform the necessary analysis and testing.

Neither Horizen, nor Horizen Labs has the ability to deprecate a technology / solution that is decentralized. However, there will likely be a point at which there is no further support provided by either of the companies (this is pretty far in the future).

I don’t fully understand how you think miners / secure nodes support the operation of EON. It has it’s own set of validators. It merely sends a certificate for verification on the mainchain, which miners and super nodes do verify.

I also don’t understand your second point. To answer your questions first, Ethereum was not built with the primitives necessary to verfiy zk-proofs coming from rollups. Ethereum also is congested. Proofs being submitted to Ethereum for validation therefore are costly. New Horizen Proof Verification chain will be purpose-built (with the necessary primitives) to validate zk-proofs efficiently. It does not make settlement layer transactions cheaper. It makes proof verification costs cheaper. The cost savings come in the difference between what it costs for rollups to submit and verify their proofs on Ethereum and what it will cost on New Horizen. The difference is significant.

What are the two sets of validators you’re talking about?

2 Likes

So the long term goal is deprecation of support for miners and super nodes? If that is the case, how can EON be a sidechain that is verified by the mainchain?

As for the second point, I understand Ethereum proofs are very costly. However, validators on the settlement layer (i.e ETH in this case) would still need to reach consensus that a proof verified by the Verification chain is valid.
See:
( 1. [New] Final Consolidation by Settlement Layer Validation: The validators of the original settlement layer then review the Proof Verification Chain’s attestation. Upon confirmation, they finalize and integrate the rollup state.)

These settlement layer validators would still need to be compensated, along with the validators of the Verification chain. Where is this compensation coming from? The settlement layer’s transaction fee? My worry is that even though the proof verification will cost less, there are still 2 parties that need to be compensated.

Is the idea that since proof verification will cost less, that same settlement layer tx fee can be used to compensate both sets of validators?