Raspberry Pi running Bitcoin node in a home office setup, small device with status LEDs, ethernet cable, bookshelf background, casual hobbyist setup.
Protocols

High Availability Setup Guide

Reference architecture for deploying fault-tolerant zkSync Era infrastructure covering main node failover, EN load balancing, and prover redundancy. Exchanges and institutional operators require HA for service continuity.
introduction
SERVICE CONTINUITY FOR INSTITUTIONAL OPERATORS

Why High Availability for zkSync Era Matters

High availability architecture is not a luxury for zkSync Era operators—it is a core operational requirement for exchanges, RPC providers, and institutional services that depend on consistent access to the canonical chain.

For zkSync Era, a single-node deployment represents a critical point of failure. The main node is responsible for tracking the L2 state, sequencing transactions, and interacting with the L1 smart contracts. If this node fails, an exchange's ability to monitor deposits, process withdrawals, or broadcast transactions stops. The External Node (EN), while designed for read scaling, is not a simple drop-in replacement for a failed main node without a deliberate failover architecture. Operators who treat the EN as a passive replica without formalizing leadership election and state verification risk serving stale data or halting operations during an upgrade or crash.

A production-grade HA setup for zkSync Era must address three distinct failure domains. Main node failover requires a standby instance with a replicated database that can assume block production or L1 interaction duties without causing a chain fork or double-signing. EN load balancing demands a topology where multiple EN instances serve API traffic behind a layer-7 proxy, with health checks that can detect a node falling out of sync and drain it from the pool before it serves incorrect data. Prover redundancy is essential for ZK Stack deployments where proof generation is on the critical path for finality; a stalled prover can delay L1 settlement and freeze bridge withdrawals. Each layer requires distinct monitoring signals—block height lag, proof generation queue depth, and L1 gas price volatility—to trigger automated failover without manual intervention.

The operational risk is compounded during network upgrades. A mandatory protocol upgrade that changes the state transition function or system contracts can cause nodes running different versions to diverge. An HA setup without a tested rollback and re-sync procedure can turn a routine upgrade into a prolonged outage. Chainscore Labs designs and reviews HA architectures specifically for zkSync Era production deployments, ensuring that failover logic, database replication, and monitoring are validated against the protocol's actual failure modes—not generic Kubernetes patterns.

FAULT-TOLERANCE ARCHITECTURE

HA Component Overview

A reference model for the critical components of a zkSync Era deployment and the redundancy strategies required to eliminate single points of failure for institutional operators.

ComponentFailure ModeWho is affectedRedundancy StrategyValidation Step

Main Node (Sequencer)

Block production halt

Exchanges, wallets, all L2 users

Active-passive failover with shared state and leader election

Simulate sequencer crash and verify block height increment on standby

External Node (EN)

API unavailability or stale data

RPC providers, dApp backends, indexers

Load-balanced EN cluster behind a health-checking reverse proxy

Query latest block number across all instances and check for consistency

Prover

Proof generation backlog

ZkSync Era bridge, finality providers

Active-active prover pool with job queue distribution

Monitor proof submission rate on L1 and proof generation queue depth

L1 Connection (ETH Client)

Missed L1 events or failed L2 tx submission

Main node, prover

Redundant execution and consensus client pairs with failover

Verify L1 block header sync and transaction submission success rate

Database (PostgreSQL)

State corruption or connection loss

Main node, EN

Streaming replication to a hot standby with automated failover

Perform a controlled switchover and verify data integrity on the new primary

Load Balancer

Traffic routing failure

All API consumers

Redundant load balancer pair with virtual IP failover (VRRP/keepalived)

Simulate primary LB failure and confirm API requests resolve to the secondary

technical-context
HIGH AVAILABILITY RISK MODEL

Core Failure Domains in zkSync Era

A technical breakdown of the independent failure domains that must be addressed in any high-availability deployment of zkSync Era infrastructure.

Designing a high-availability (HA) setup for zkSync Era requires modeling the protocol's architecture as a set of distinct, loosely coupled failure domains. A failure in one domain must not cascade into another, and each domain requires its own redundancy strategy. The primary domains are: the L2 sequencer and block production pipeline, the state synchronization and External Node (EN) read layer, the prover subsystem responsible for generating validity proofs, and the L1 interaction layer that manages data availability and proof settlement on Ethereum.

The sequencer domain is the most critical for liveness. A failure here halts L2 block production. HA strategies involve active-passive or active-active main node configurations with automated failover based on L1 contract monitoring and health checks. The EN domain is stateless in terms of block production but stateful for serving API traffic. It scales horizontally behind a load balancer, but operators must manage WebSocket subscription affinity and ensure all instances are synchronized to within a safe lag tolerance to prevent serving stale data. The prover domain is computationally intensive and operates asynchronously. Redundancy is achieved by running multiple prover instances, but operators must implement locking or leader-election mechanisms to prevent duplicate proof submissions that waste L1 gas.

The L1 interaction domain is a shared dependency. The zkSync Era main node must maintain a reliable connection to an Ethereum execution client to monitor the bridge contract, publish state diffs, and settle proofs. An HA design must account for L1 client failover and manage nonce continuity for L1 transactions. Chainscore Labs can review a team's HA architecture against these specific failure domains, identifying single points of failure in sequencer failover logic, EN synchronization gaps, prover coordination, or L1 relay strategies before they cause a production outage.

AFFECTED ACTORS AND OPERATIONAL REQUIREMENTS

Who Needs This Architecture

Exchange & Custodian Operators

Centralized exchanges and custodians managing user deposits and withdrawals on zkSync Era require a high availability setup to prevent service disruption during node failures or network instability.

Critical requirements:

  • Main node failover to guarantee transaction submission and block monitoring continue if the primary node fails.
  • External Node (EN) load balancing to serve high-volume API requests for balance checks, deposit tracking, and withdrawal generation without a single point of failure.
  • State consistency verification to ensure no divergence between the canonical chain and internal accounting systems.

Operational checklist:

  • Deploy at least two main node instances in an active/passive or active/active configuration.
  • Implement health checks that trigger automatic failover if block production monitoring stalls.
  • Use a load-balanced EN cluster for all read-heavy RPC calls to avoid overloading the main node.
  • Test disaster recovery procedures quarterly, including forced failover and database restoration from snapshots.

Chainscore can review your HA architecture, validate failover logic, and ensure your integration remains resilient during protocol upgrades.

implementation-impact
FAULT-TOLERANT INFRASTRUCTURE DESIGN

Key Architectural Patterns

Core architectural patterns for deploying zkSync Era infrastructure with high availability. These patterns address the unique failure modes of ZK rollup components, including proof generation stalls, L1 reorg exposure, and state sync divergence.

01

Active-Passive Main Node Failover

Deploy a hot standby main node that continuously syncs state but remains in passive mode. Use a distributed lock or consensus mechanism (e.g., etcd leader election) to ensure only one instance submits blocks to L1 at any time. The failover trigger must monitor block production lag and L1 finality depth. Operators must ensure the passive node's L1 transaction nonce management does not conflict with the active node. Chainscore can review your leader election logic and failover automation to prevent double-submission scenarios.

02

External Node Load Balancing with Session Affinity

Distribute read traffic across multiple External Nodes behind a load balancer with sticky sessions based on WebSocket connection IDs. ENs are stateless for reads but WebSocket subscriptions require session affinity to avoid dropped events during reconnection. Implement health checks that verify sync lag against the main node and automatically drain nodes that fall behind a configurable threshold. This pattern prevents stale data serving during EN catch-up after restarts.

03

Prover Pool with Priority Queuing

Operate a pool of prover instances behind a job queue that prioritizes proofs for batches closest to the execution deadline. A single prover failure can stall L1 finality, so redundancy is critical. Each prover instance should run on dedicated hardware with GPU acceleration and independent L1 RPC endpoints. The queue must track proof generation timeouts and automatically reassign failed jobs. Chainscore can model your proof generation capacity against chain throughput to identify bottlenecks before they cause finality stalls.

04

L1 RPC Redundancy with Cross-Client Diversity

The main node and prover depend on L1 RPC endpoints for transaction submission and event reading. Use a pool of L1 RPC providers spanning at least two execution client implementations (e.g., Geth and Nethermind) to mitigate client-specific bugs. Implement circuit breakers that switch providers when L1 block height stalls or responses diverge. This pattern protects against L1 reorg exposure where a single RPC provider serves stale or incorrect data.

05

Database Replication with Point-in-Time Recovery

Configure PostgreSQL streaming replication from the primary main node database to a standby replica. Combine this with continuous WAL archiving to enable point-in-time recovery. During a failed upgrade or database corruption, operators can restore to a specific L2 block height rather than replaying from genesis. This pattern is essential for meeting recovery time objectives without relying solely on snapshot imports. Chainscore can validate your backup cadence and test restoration procedures in a staging environment.

06

Multi-Zone Monitoring with Independent Alerting Paths

Deploy monitoring infrastructure in a separate failure domain from the nodes being monitored. Use Prometheus federation to aggregate metrics from each availability zone into a central dashboard while maintaining local alerting rules that fire even if the central aggregator is unreachable. Critical alerts—block production stalls, proof queue depth exceeding threshold, L1 gas balance depletion—must route through a paging service independent of the infrastructure being monitored. Chainscore can design alerting thresholds tuned to zkSync Era's specific operational metrics.

HIGH AVAILABILITY RISK ASSESSMENT

Failure Mode and Mitigation Matrix

Identifies critical failure modes in a high-availability zkSync Era deployment and maps them to affected components, operational impact, and required mitigations.

Failure ModeAffected ComponentOperational ImpactMitigation Strategy

Main node process crash or hang

Main Node (Sequencer)

Complete halt of L2 block production and EN sync. Exchange deposits and withdrawals stall.

Implement active/passive failover with a standby main node. Use a shared heartbeat mechanism (e.g., Consul, etcd) to trigger automatic promotion.

Database corruption on primary main node

Main Node Database

Node cannot produce new blocks. Failover to standby may replicate corrupted state if using synchronous replication.

Use asynchronous streaming replication to a dedicated standby. Perform periodic integrity checks (e.g., pg_checksums). Maintain point-in-time recovery (PITR) backups to avoid replicating corruption.

L1 gas price spike makes batch submission uneconomical

Main Node (Sequencer)

Transactions are queued but not committed to L1, causing a growing backlog and unpredictable finality delays for users.

Configure gas price bounds in the node configuration. Implement circuit breakers that pause batch submission if L1 base fee exceeds a threshold. Monitor L1 gas markets proactively.

External Node (EN) falls behind the main node tip

External Node

EN serves stale data to RPC consumers (wallets, dApps), leading to incorrect nonces, balances, and transaction simulation failures.

Deploy multiple ENs behind a load balancer with health checks that verify en_sync_fetcher_last_l1_batch_number against the main node. Automatically drain stale ENs from the pool.

Prover fails to generate a proof within the execution deadline

Prover

Proof submission window on L1 is missed. The batch must be reverted or skipped, potentially causing a chain halt depending on protocol rules.

Deploy a redundant prover fleet. Implement a proof generation queue with priority scheduling. Monitor proof_generation_time metric and alert if it approaches the execution deadline.

Load balancer becomes a single point of failure

Load Balancer (EN/API)

All RPC requests fail, causing a complete service outage for integrated applications and users even though backend nodes are healthy.

Deploy a redundant load balancer pair (active/passive) with a floating virtual IP (VIP) or use DNS-based failover. Use independent load balancers for WebSocket and HTTP traffic.

Network partition between main node and L1 RPC

Main Node (Sequencer)

Node cannot read L1 state for bridge deposits or publish state diffs. L2 block production halts to prevent state divergence.

Configure the main node with multiple, geographically diverse L1 RPC endpoints. Implement a fallback mechanism that rotates to a healthy endpoint on connection failure.

Accidental key exposure for block production or proof submission

Key Management System

Attacker could produce malicious blocks or submit fraudulent proofs, potentially leading to slashing events or a forced chain rollback.

Store keys exclusively in HSMs or secure enclaves. Never store keys on node filesystems. Use remote signing architectures where the node requests signatures without accessing the key material directly.

PRODUCTION READINESS VERIFICATION

HA Implementation Checklist

A structured checklist to validate that a zkSync Era high availability deployment meets operational standards for fault tolerance, data integrity, and automated failover before production traffic is routed.

What to check: Verify that a standby main node can take over block production without causing a chain halt or double-signing.

Why it matters: The main node is the single sequencer. If it fails, the chain stops producing blocks, halting all L2 activity including deposits, withdrawals, and transaction processing.

Readiness signal:

  • A documented and tested runbook exists for promoting the standby node.
  • The standby node is fully synced and its database is in a consistent state.
  • The failover process includes a step to verify the active node is fully stopped and its sequencer key is disabled before the standby activates.
  • Automated health checks trigger an alert, not an automatic failover, to prevent split-brain scenarios.
Chains We Build On

Looking to build on a specific blockchain?

We build smart contracts, DeFi applications, wallets, tokenization platforms, and blockchain infrastructure across the major ecosystems teams choose today. That includes Ethereum, Arbitrum, Optimism, Polygon, Avalanche, Solana, Sui, Aptos, Hedera, Stellar, and NEAR, with support for additional EVM and non-EVM networks based on your product requirements.

EVM ecosystems

  • Ethereum logo
    Ethereum
  • Arbitrum logo
    Arbitrum
  • Optimism logo
    Optimism
  • Polygon logo
    Polygon
  • Avalanche logo
    Avalanche
  • Cronos logo
    Cronos

Non-EVM ecosystems

  • Solana logo
    Solana
  • Sui logo
    Sui
  • Aptos logo
    Aptos
  • Hedera logo
    Hedera
  • Stellar logo
    Stellar
  • NEAR logo
    NEAR

Additional ecosystems

  • Polkadot logo
    Polkadot
  • Cosmos logo
    Cosmos
  • TON logo
    TON
  • Cardano logo
    Cardano
  • Algorand logo
    Algorand
  • Tempo logo
    Tempo

Also available for Base, appchains, custom EVM networks, and cross-chain product architecture.

HIGH AVAILABILITY PLANNING

Frequently Asked Questions

Common architectural and operational questions for teams designing fault-tolerant zkSync Era infrastructure. These answers address failover sequencing, state consistency, and redundancy patterns for main nodes, external nodes, and provers.

The main node is a singleton sequencer and prover interface. A hot standby architecture is required for high availability.

What to check:

  • A passive replica is continuously syncing from the active main node's database.
  • The replica is not broadcasting transactions or submitting proofs.
  • A health-check loop monitors the active node's block production and L1 interaction.

Why it matters: Running two active main nodes simultaneously will cause equivocation and can lead to a chain halt or slashing conditions. The failover must be a strictly ordered promotion of the standby.

Readiness signal: A tested, automated failover script that stops the active node, promotes the standby, and updates DNS or load balancer targets in under 60 seconds.

Trusted by Industry Leaders

Delivering blockchain solutions for 5+ years.

We have partnered with 50+ leading DeFi protocols, NFT ecosystems, and fintech innovators to build secure, scalable, and capital-efficient blockchain products.

Selected Partners & Clients

ChainVote logo
Reax logo
Sokail logo
Swapsicle logo
SyntheX logo
Tekika logo
Telos logo
Zexe logo
ChainVote logo
Reax logo
Sokail logo
Swapsicle logo
SyntheX logo
Tekika logo
Telos logo
Zexe logo
ChainVote logo
Reax logo
Sokail logo
Swapsicle logo
SyntheX logo
Tekika logo
Telos logo
Zexe logo
ChainVote logo
Reax logo
Sokail logo
Swapsicle logo
SyntheX logo
Tekika logo
Telos logo
Zexe logo
“I've been working with Chainscore Labs for last 3+ years, they've consistently delivered with strong ownership across multiple projects. The team is reliable and detail-oriented.”
L
Lee Erswell
CEO, Telos Foundation
how to get started

How to get started?

If you're looking for blockchain integration, ChainScore Labs has 5+ years of experience helping teams build and integrate exchanges, wallets, smart contracts, tokenization solutions, and protocol-connected products, we can help you choose the right path, integrate securely, and get to production faster. Our team consists of experienced blockchain developers and architects who can help you with your blockchain integration needs.

01

Exploration & Strategy

Define your product goals and choose the right blockchain architecture for your use case.

02

Architecture & Design

Design the smart contracts, tokenomics, and security parameters of your system.

03

Development & Integration

Build and integrate with wallets, oracles, and front-end dApps for a seamless experience.

04

Security & Launch

Comprehensive audits followed by a risk-managed mainnet deployment to protect your users.

Start a build

Need a blockchain engineering team?

Send the project context and we will respond with next steps, scope questions, and a practical path to delivery.