Liquidity provider reviewing AMM pool statistics on phone, pool allocation dashboard visible, casual degen setup at home with screens.
Protocols

Smartnode Upgrade Guide: Planning and Execution

A step-by-step operational runbook for node operators to plan, execute, and validate Smartnode stack upgrades safely, minimizing attestation downtime and avoiding slashing or penalty risks.
introduction
OPERATIONAL RUNBOOK

Introduction

A structured methodology for planning and executing Smartnode stack upgrades to minimize attestation downtime and eliminate slashing risk.

Upgrading the Smartnode stack is a mandatory operational procedure for every Rocket Pool node operator. Unlike a simple application update, a Smartnode upgrade coordinates changes across the rocketpool CLI, the smartnode Docker containers, and the underlying execution and consensus clients. A failed or rushed upgrade can lead to missed attestations, sync committee penalties, or, in the worst case, a slashing event if validator keys are mishandled. This guide provides a systematic framework for planning, executing, and validating an upgrade, treating it as a high-stakes change management process rather than a routine maintenance task.

The core risk during any upgrade is the introduction of a configuration mismatch or a client incompatibility that causes the validator client to stop attesting. Operators must consult the Execution and Consensus Client Compatibility Matrix before initiating any procedure to ensure the target Smartnode version supports their specific client combination. The upgrade process itself must account for the validator's state: a node that is actively proposing a block or participating in a sync committee requires a different timing strategy than an idle attester. The pre-upgrade checklist therefore includes verifying the node's current duties, ensuring a recent fallback client state is available, and confirming that a validated backup of the wallet and validator keys exists.

Post-upgrade validation is not complete until the operator has confirmed correct attestation behavior against a beaconchain explorer and verified that the node has not entered Rocket Pool's penalty system. The Post-Upgrade Verification and Health-Check Protocol provides a standardized checklist for this critical safety net. For teams managing fleets of nodes, the upgrade procedure should be codified into an Infrastructure-as-Code pipeline to prevent configuration drift, a pattern detailed in the Multi-Node Management and Infrastructure-as-Code guide. Chainscore Labs can assist professional node operators by reviewing their upgrade runbooks, auditing their CI/CD pipelines for Smartnode deployments, and designing automated post-upgrade health-check scripts that integrate directly into existing monitoring stacks.

SMARTNODE UPGRADE PLANNING

Quick Facts

Operational facts for planning and executing a Smartnode stack upgrade to minimize downtime and slashing risk.

AreaWhat changesWho is affectedAction

Smartnode binary

New version with updated client dependencies, features, or bug fixes

Node operators

Review release notes for breaking changes; verify checksum before installation

Execution & consensus clients

Version compatibility matrix may shift; old client versions may be unsupported

Node operators

Cross-reference client compatibility matrix; plan client upgrades before or alongside Smartnode

Validator keys

No change to keys, but incorrect migration can cause double-signing

Node operators

Back up wallet and validator keys before starting; never run two instances with the same keys

MEV-Boost configuration

Relay endpoints or ordering logic may require updates

Validators using MEV-Boost

Verify relay URLs and ordering against current recommendations; test after upgrade

Fee recipient settings

Configuration format or default behavior may change

Node operators

Audit fee recipient for all minipools post-upgrade to prevent misrouted priority fees

Monitoring and alerting

New metrics or changed endpoint paths can break dashboards

Infrastructure teams

Update health-check scripts and alerting rules to match new diagnostic commands and endpoints

Fallback clients

Failover logic may need reconfiguration if client endpoints or flags change

Operators with high-availability setups

Test failover behavior in a staging environment before upgrading production nodes

Penalty state risk

Extended downtime during a mishandled upgrade can trigger the penalty loop

Node operators

Execute upgrades during low-attestation-impact windows; have a rescue node plan ready

technical-context
COMPONENT INTERACTIONS AND FAILURE MODES

The Upgrade Surface: What Changes

A Smartnode upgrade is not a single binary swap; it is a coordinated state transition across a tightly coupled stack of execution clients, consensus clients, and Rocket Pool's own contract-aware middleware.

When a Rocket Pool node operator executes a Smartnode upgrade, the change surface extends far beyond the rocketpool CLI itself. The upgrade typically modifies the Docker image composition, which can alter the bundled versions of the execution client (EL) and consensus client (CL). A version mismatch between these components—or between the Smartnode's internal API expectations and the actual client endpoints—is a primary source of post-upgrade attestation failures and sync stalls. Operators must treat the Smartnode stack as a single integrated system where the upgrade's impact is measured by the compatibility of its three core layers: the Rocket Pool service layer (which manages minipool state, RPL collateral checks, and fee distribution), the consensus client (which handles validator duties and beacon state), and the execution client (which provides the execution payload and processes Rocket Pool contract events).

The operational risk is concentrated in the transition window. During the upgrade, the Smartnode service restarts, temporarily severing the watchtower functions that monitor on-chain minipool status, RPL collateral ratios, and the penalty loop. If the new stack version introduces a breaking change to the Rocket Pool API—such as a modified response format for rocketpool node status or a new requirement for fee recipient configuration—any dependent automation scripts, monitoring dashboards, or Infrastructure-as-Code (IaC) modules will fail silently until manually updated. Furthermore, upgrades that include changes to the Smoothing Pool participation logic or the MEV-Boost relay configuration schema can alter reward routing without explicit operator action, potentially causing missed MEV revenue or incorrect fee distribution if not validated post-upgrade.

A rigorous upgrade procedure must therefore map the full change surface before execution. This includes diffing the release notes for EL/CL version bumps, verifying the new Docker Compose definitions against the operator's custom overrides, and auditing the rocketpool API changelog for any deprecated or altered commands used in health-check scripts. Chainscore Labs can provide a pre-upgrade impact assessment that models this change surface against your specific fleet configuration, identifying breaking changes in monitoring integrations, reward routing, and client compatibility before they cause downtime or revenue loss.

UPGRADE IMPACT ANALYSIS

Affected Actors and Systems

Node Operators

Node operators are the primary actors affected by any Smartnode upgrade. A failed or delayed upgrade can lead to missed attestations, reduced rewards, and entry into a penalty state if the node remains offline for an extended period.

Pre-Upgrade Actions:

  • Review the release notes for breaking changes to the rocketpool CLI, API, or Docker Compose configuration.
  • Verify the compatibility of your execution and consensus clients against the new Smartnode version's client matrix.
  • Take a snapshot or full backup of your validator keys and chain data before initiating the upgrade.
  • Schedule the upgrade during a period of low proposal likelihood to minimize potential MEV loss.

Post-Upgrade Actions:

  • Run rocketpool node sync and rocketpool node status to confirm the node is fully synced and attesting.
  • Monitor attestation effectiveness for 2-3 epochs to ensure stability.
  • Verify that your fee recipient and MEV-Boost relay configurations remain intact.
implementation-impact
UPGRADE PLANNING

Implementation Impact Areas

A Smartnode upgrade is a multi-component operation that touches the consensus client, execution client, MEV relay configuration, and validator keys. Each area below requires specific verification to avoid attestation downtime or slashing conditions.

01

Validator Key Safety and Slashing Prevention

The highest priority during any upgrade is preventing a double-signing event. Operators must ensure validator keys are not active on two machines simultaneously. Before starting the upgrade, stop the Smartnode service and verify that no validator client process is running. If migrating to new hardware, never copy the validator keystore to a new machine without first stopping and disabling the original instance. A safe migration pattern involves waiting for two epochs after shutdown before importing keys elsewhere. Chainscore can review key-handling procedures and design slashing-prevention checklists for fleet operators.

02

Execution and Consensus Client Compatibility

Smartnode releases often bundle specific client versions or introduce breaking changes to the API surface between the stack and its managed clients. Upgrading Smartnode without verifying that the target execution client (e.g., Nethermind, Besu) and consensus client (e.g., Lighthouse, Prysm) versions are compatible can lead to a failure to attest. Operators must cross-reference the release notes against the official compatibility matrix. A mismatch may require a separate client upgrade before or after the Smartnode binary update. Chainscore can validate client version alignment against the canonical matrix for a given upgrade.

03

MEV-Boost and Relay Configuration Integrity

Smartnode upgrades can reset or deprecate MEV-Boost configuration parameters, including relay URLs, min-bid settings, and fee-recipient overrides. A silent failure in MEV-Boost integration will cause the validator to produce locally built blocks instead of MEV-enhanced blocks, resulting in lost revenue without triggering a monitoring alert. After upgrading, operators must explicitly verify that the MEV-Boost service is enabled, the relay list is current, and the connection to at least one relay is healthy. Chainscore can audit post-upgrade MEV configurations to confirm relay connectivity and correct fee-recipient routing.

04

Database Migration and State Compatibility

Major consensus client upgrades occasionally require a database schema migration or a full re-sync from genesis. If the Smartnode upgrade bundles a client version that cannot read the existing database, the node will fail to start or will stall during the migration process. Operators must check client release notes for database-breaking changes and plan for the additional downtime of a re-sync, which can take hours or days. Running a fallback client during this window is a common high-availability pattern. Chainscore can help plan migration sequencing to minimize attestation gaps.

05

Post-Upgrade Attestation and Health Verification

A successful binary upgrade does not guarantee a healthy validator. Operators must follow a structured post-upgrade checklist that includes verifying the validator client is attesting to the correct chain head, checking for inclusion distance increases, and confirming the node has not entered the Rocket Pool penalty queue. A common failure mode is a node that is online but attesting to a wrong fork due to a misconfigured checkpoint sync. Chainscore can develop automated health-check scripts that run a standard verification protocol immediately after an upgrade and alert on anomalies.

06

Fallback and Rescue Node Readiness

An upgrade window is a high-risk period for a single-node setup. If the upgrade fails and the primary node cannot recover quickly, a pre-configured rescue node must be ready to take over validator duties. Before starting the upgrade, operators should verify that the rescue node's clients are synced, its validator keys are importable, and the doppelganger detection feature is enabled to prevent accidental double-signing. The rescue node should not be upgraded simultaneously with the primary. Chainscore can design and validate rescue-node architectures that support safe failover during upgrade operations.

SMARTNODE UPGRADE PLANNING

Upgrade Risk Matrix

Operational risks and required actions for node operators before, during, and after a Smartnode stack upgrade.

AreaFailure modeWho is affectedMitigation

Client compatibility

Upgraded Smartnode version enforces a new execution or consensus client version, causing the node to fail to start or attest.

Node operators

Cross-reference the Execution and Consensus Client Compatibility Matrix before upgrading. Run on a testnet node first.

Validator keys

Improper backup or migration leads to key loss or accidental double-signing during the upgrade process.

Node operators

Verify backup integrity of wallet files and mnemonic phrases. Follow the Validator Key Recovery and Migration runbook exactly.

Database corruption

Incompatible database migration between client versions forces a lengthy resync, causing extended downtime.

Node operators

Check client release notes for database breaking changes. Pre-sync a fallback client to minimize attestation gaps.

MEV-Boost configuration

Relay endpoints or MEV-Boost version becomes incompatible with the new Smartnode stack, causing missed block proposals.

Validators using MEV-Boost

Review the MEV-Boost Integration and Relay Selection Strategy guide. Test relay connectivity on a testnet node post-upgrade.

Fee recipient settings

Upgrade resets or misconfigures the fee recipient address, routing priority fees and MEV rewards to an incorrect address.

Node operators

Audit fee recipient configuration for all minipools immediately after upgrade using the Minipool Commission and Fee Recipient Configuration guide.

Smoothing pool state

Upgrade process corrupts or loses the local Smoothing Pool commitment data, invalidating reward claims.

Smoothing Pool participants

Back up the smoothing-pool data directory before upgrading. Verify participation status post-upgrade.

Monitoring and alerts

Post-upgrade, monitoring dashboards and alerting rules fail to detect critical issues like missed attestations or low RPL collateral.

Infrastructure teams

Update monitoring queries to match new metric endpoints or log formats. Validate alerts using the Essential Node Monitoring and Alerting Rules guide.

Rescue node readiness

Primary node failure during a botched upgrade leaves no operational rescue node, resulting in prolonged downtime and potential penalties.

Operators with rescue nodes

Ensure the rescue node is updated and synced before initiating the primary node upgrade. Test the failover process.

SMARTNODE UPGRADE EXECUTION

Operator Upgrade Checklist

A step-by-step operational checklist to minimize downtime and slashing risk when upgrading the Smartnode stack. Each item includes the verification signal that confirms readiness before proceeding to the next stage.

Confirm the current node is in a healthy, non-penalized state before initiating any upgrade procedure.

What to check:

  • Attestation effectiveness is above 95% for the last epoch.
  • The node is not currently in a penalty state (rocketpool node status).
  • RPL collateral remains above the minimum threshold to avoid an undercollateralized exit.
  • No active or pending validator exits are in the queue.

Why it matters: Upgrading a node that is already missing attestations or close to a penalty threshold compounds risk. A brief downtime for a healthy node is a planned event; downtime for an already degraded node can trigger slashing conditions or forced exits.

Readiness signal: The output of rocketpool node status shows a green health status, and a Beaconcha.in dashboard confirms no recent missed attestations.

Chains We Build On

Looking to build on a specific blockchain?

We build smart contracts, DeFi applications, wallets, tokenization platforms, and blockchain infrastructure across the major ecosystems teams choose today. That includes Ethereum, Arbitrum, Optimism, Polygon, Avalanche, Solana, Sui, Aptos, Hedera, Stellar, and NEAR, with support for additional EVM and non-EVM networks based on your product requirements.

EVM ecosystems

  • Ethereum logo
    Ethereum
  • Arbitrum logo
    Arbitrum
  • Optimism logo
    Optimism
  • Polygon logo
    Polygon
  • Avalanche logo
    Avalanche
  • Cronos logo
    Cronos

Non-EVM ecosystems

  • Solana logo
    Solana
  • Sui logo
    Sui
  • Aptos logo
    Aptos
  • Hedera logo
    Hedera
  • Stellar logo
    Stellar
  • NEAR logo
    NEAR

Additional ecosystems

  • Polkadot logo
    Polkadot
  • Cosmos logo
    Cosmos
  • TON logo
    TON
  • Cardano logo
    Cardano
  • Algorand logo
    Algorand
  • Tempo logo
    Tempo

Also available for Base, appchains, custom EVM networks, and cross-chain product architecture.

SMARTNODE UPGRADE FAQ

Frequently Asked Questions

Answers to common operational questions about planning and executing a Smartnode stack upgrade to minimize downtime and avoid slashing risks.

The most critical step is to ensure you never run two validator clients with the same keys simultaneously. Before upgrading, you must stop the existing Smartnode services and verify that no validator processes are active. Use docker ps to confirm all Rocket Pool containers have exited. Wait for at least two epochs (approximately 12.8 minutes) after stopping before starting the new version to guarantee your validator is not attesting on the old client. This cool-down period is your primary defense against double-signing slashable offenses.

Trusted by Industry Leaders

Delivering blockchain solutions for 5+ years.

We have partnered with 50+ leading DeFi protocols, NFT ecosystems, and fintech innovators to build secure, scalable, and capital-efficient blockchain products.

Selected Partners & Clients

ChainVote logo
Reax logo
Sokail logo
Swapsicle logo
SyntheX logo
Tekika logo
Telos logo
Zexe logo
ChainVote logo
Reax logo
Sokail logo
Swapsicle logo
SyntheX logo
Tekika logo
Telos logo
Zexe logo
ChainVote logo
Reax logo
Sokail logo
Swapsicle logo
SyntheX logo
Tekika logo
Telos logo
Zexe logo
ChainVote logo
Reax logo
Sokail logo
Swapsicle logo
SyntheX logo
Tekika logo
Telos logo
Zexe logo
“I've been working with Chainscore Labs for last 3+ years, they've consistently delivered with strong ownership across multiple projects. The team is reliable and detail-oriented.”
L
Lee Erswell
CEO, Telos Foundation
how to get started

How to get started?

If you're looking for blockchain integration, ChainScore Labs has 5+ years of experience helping teams build and integrate exchanges, wallets, smart contracts, tokenization solutions, and protocol-connected products, we can help you choose the right path, integrate securely, and get to production faster. Our team consists of experienced blockchain developers and architects who can help you with your blockchain integration needs.

01

Exploration & Strategy

Define your product goals and choose the right blockchain architecture for your use case.

02

Architecture & Design

Design the smart contracts, tokenomics, and security parameters of your system.

03

Development & Integration

Build and integrate with wallets, oracles, and front-end dApps for a seamless experience.

04

Security & Launch

Comprehensive audits followed by a risk-managed mainnet deployment to protect your users.

Start a build

Need a blockchain engineering team?

Send the project context and we will respond with next steps, scope questions, and a practical path to delivery.