Case Study

Tessera: Python JAM Client

Case Study: Implementing the Join-Accumulate Machine Protocol in Python

Web3 Foundation

web3.foundation

PythonAsyncIOQUICJamPackPyring+5 more

Project Overview

Transforming vision into reality

About This Project

A technical case study documenting the design and development considerations for Tessera, a Python-based client implementation for the JAM (Join-Accumulate Machine) protocol. This study explores how Tessera aims to interpret and implement the JAM specification outlined in the Graypaper, focusing on core protocol mechanics, state management, PVM interaction, consensus participation (client-side), networking, and cryptographic requirements necessary for a functional JAM node.

Client

Web3 Foundation

web3.foundation

Timeline

Start

April 2025

Completed

Invalid Date

Duration

Ongoing Development

INTRODUCTION

Tessera: A Python Implementation of the JAM Protocol

Tessera represents an effort by ChainScore Labs to create a functional and spec-compliant client for the JAM (Join-Accumulate Machine) protocol, as detailed in the Graypaper. Written primarily in Python, Tessera aims to provide an accessible, well-documented implementation for researchers, developers, and network participants interested in interacting with or validating the JAM network. This case study outlines the architectural considerations, implementation approach, and challenges involved in building such a client.

PROTOCOL-UNDERSTANDING

Core JAM Concepts in Tessera

Implementing Tessera requires a deep understanding and accurate translation of the JAM Graypaper's core concepts into Python data structures and logic. Key areas of focus include:

Hybrid Architecture: Modeling the distinction between on-chain (Accumulation) and in-core (Refinement) computation logic and their interaction.
Service-Oriented Model: Implementing data structures and logic for service accounts, including state (storage, preimages), code (refine/accumulate entry points), balance, and gas limits.
Coretime Concept: Handling core assignment, authorization checks (verifying authorizer logic outputs), and coretime accounting from a client perspective.
Block & State Structure: Defining Python classes/objects for blocks, headers, extrinsics (tickets, preimages, reports, etc.), and the partitioned state tuple (α, β, γ...).
State Transition Function (Y): Implementing the block-level state transition logic, ensuring deterministic updates based on prior state and block contents.

NETWORKING

Peer-to-Peer Networking Layer

Connecting to the JAM Network

Tessera requires a robust networking layer to discover peers, synchronize chain data, and propagate messages (blocks, transactions, consensus votes). This involves selecting or implementing a suitable P2P library (like LibP2P) and defining message types and protocols based on JAM's anticipated requirements.

Peer Discovery: Implementing mechanisms like Discv5 or Kademlia DHT for finding other JAM nodes.
Message Serialization: Defining and implementing serialization/deserialization formats (e.g., SSZ, RLPx, or a JAM-specific format) for network messages.
Gossip Protocols: Implementing gossip logic for propagating new blocks, transactions (if applicable directly), and consensus messages efficiently.
Request/Response Protocols: Defining protocols for requesting specific blocks, state parts, or historical data from peers during synchronization.
Transport Layer: Utilizing secure and reliable transport protocols (e.g., TCP with TLS/Noise).

CONSENSUS-CLIENT

Consensus Mechanism Interaction (Client-Side)

While Tessera might not initially act as a block producer or finality voter, it must understand and verify the outputs of JAM's consensus mechanisms: Safrole for block production/fork choice and Grandpa for finality. This involves processing and validating consensus-related messages received from the network.

Safrole Verification: Validating block author eligibility based on sealing keys derived from tickets/VRFs. Implementing the fork-choice rule based on Safrole's criteria (e.g., highest valid ticket score, chain length).
Grandpa Verification: Processing Grandpa votes (prevotes, precommits) gossiped on the network. Verifying vote signatures against the known validator set. Identifying finalized blocks based on supermajority agreement (2/3+).
Validator Set Management: Tracking the current and upcoming validator sets (κ, λ, γκ) based on state transitions and epoch changes defined in Safrole.
Epoch Processing: Handling epoch transitions, including validator set rotation and Safrole state updates (roots, sealers).

BLOCK-PROCESSING

Block Processing and State Transition

Applying the JAM State Machine

A core function of Tessera is processing incoming blocks and applying the state transition function (Y) defined in the Graypaper. This involves validating block structure, processing extrinsics, executing/verifying PVM code (Accumulation), and updating the local state representation.

Block Validation: Verifying header fields (parent hash, state root, extrinsic root), seal signature (Hs), VRF output (Hu), consensus markers (He, Hw, Ho).
Extrinsic Processing: Handling different extrinsic types (Tickets ET, Preimages EP, Reports EG, Assurances EA, Disputes ED) according to their specific validation rules and state effects.
State Transition (Y): Implementing the logic for updating each state component (α, β, γ...) based on the prior state and the processed block B, as defined in Graypaper section 4.2.1 and subsequent sections.
Accumulation Logic Verification: Executing or verifying the results of the 'Accumulate' entry point for relevant service accounts affected by the block's work reports (W*).
State Root Calculation: Calculating the posterior state root (H'r) after applying state changes, using the specified Merkle Trie structure, and comparing it to the expected root.

PVM-INTERACTION

PVM (PolkaVM/RISC-V) Integration

Executing and Verifying Service Code

JAM utilizes a PVM based on RISC-V (RV64EM) for executing service logic (Refine and Accumulate). Tessera needs to integrate with a PVM execution environment. This could involve embedding a Python-based RISC-V interpreter/JIT, or interfacing with an external PVM implementation (potentially written in a more performant language like Rust or C++ via FFI). The client must handle PVM initialization, gas metering, host function calls (Ω functions), and result/state change verification.

Execution Environment: Integrating a RISC-V (RV64EM compatible) interpreter, JIT, or external process interface.
Gas Metering: Implementing instruction-level gas counting according to PVM specifications to enforce computational limits.
Host Function Interface (Ω): Providing implementations for all host functions callable from within the PVM (e.g., state access, historical lookups, cryptographic operations, logging).
State Interaction: Allowing the PVM to securely read relevant parts of the JAM state and propose state modifications (primarily for Accumulate).
Refinement Verification: Potentially re-executing or verifying proofs of refinement computation performed in-core by guarantors.
Accumulation Execution/Verification: Executing the 'Accumulate' entry point for services based on work reports within the defined gas limits.

STATE-MANAGEMENT

State Management and Storage

Tessera needs an efficient way to store and query the JAM state, which is represented as a complex structure committed to via a Merkle Trie. This likely involves using a persistent key-value store (like LMDB or RocksDB) optimized for Merkle Trie operations. The implementation must handle state updates atomically per block, support efficient trie root calculation, and potentially implement state pruning.

Data Store: Utilizing a performant key-value database (e.g., LMDB, RocksDB) suitable for large datasets and frequent updates.
Merkle Trie Implementation: Implementing or using a library for the specific binary Patricia Merkle Trie structure defined in the JAM Graypaper (Appendix D), optimized for Python.
Atomic Updates: Ensuring state changes from a block are applied atomically; either all succeed or none do, maintaining state consistency.
State Root Calculation: Efficiently calculating the state root after each block application.
State Pruning: Implementing strategies to prune old state data that is no longer needed (beyond finality + archival requirements) to manage storage growth.
Preimage/Lookup Handling: Storing and managing preimage data (components p and l of service accounts) and implementing the historical lookup logic (Λ).

CRYPTOGRAPHY

Cryptographic Primitives

Implementing JAM's Security Foundations

Correct and performant implementation of JAM's cryptographic primitives is crucial for security and interoperability. Tessera needs to integrate Python libraries or bindings for:

Hashing: Blake2b-256 (H) and Keccak-256 (Hk) for various commitments and identifiers.
Signatures (Ed25519): Used for assurances (EA) and potentially other validator messages. Requires key generation, signing, and verification.
Signatures (BLS12-381): Used for Grandpa finality votes and BEEFY commitments. Requires key generation, signing, verification, and aggregation.
Signatures (Bandersnatch Schnorr-like): Used for block seals (Hs). Requires specific curve operations.
VRFs (Bandersnatch RingVRF): Used for ticket generation (ET) and potentially Safrole leader selection. Requires proof generation and verification, and output derivation (Y).
Serialization/Merklization: Implementing the specific encoding (Appendix C) and Merklization (Appendix D/E) rules for state and block components.

Metrics

Statistics

1M+

TPS

Block Time

~682MB/s

Data Availability Throughput

~12s

Finality Time

CHALLENGES-PYTHON

Challenges and Considerations for Python

Building a high-performance blockchain client in Python presents unique challenges:

Performance Bottlenecks: Python's Global Interpreter Lock (GIL) can limit CPU-bound parallelism. Cryptographic operations and PVM execution can be significantly slower than in compiled languages like Rust or C++.
Memory Management: Handling large state tries and network buffers efficiently requires careful memory management to avoid excessive consumption or garbage collection pauses.
Cryptographic Library Availability: Mature and performant Python libraries for newer primitives like Bandersnatch curve operations (VRFs, signatures) might be scarce or require bindings to C/Rust libraries.
Concurrency Model: Leveraging `asyncio` effectively for network I/O and concurrent task handling is crucial, but requires careful design to avoid blocking operations.
Potential Solutions: Utilizing optimized libraries (e.g., `numpy` for some math), implementing critical sections in C/Cython/Rust via FFI, aggressive caching, efficient data structures, and potentially multi-processing for CPU-bound tasks where possible.

LONG-RUN

Towards a Pythonic JAM Client

Tessera aims to provide a valuable contribution to the JAM ecosystem by offering a Python client implementation. While facing inherent performance challenges, careful architecture, leveraging optimized libraries, and potentially using FFI for critical components can lead to a functional and useful client for development, testing, and potentially certain network participation roles. This project underscores ChainScore Labs' commitment to exploring and implementing cutting-edge blockchain protocols.

BUILD

Start Your Project

Ready to build something extraordinary?

Let's discuss how we can help you achieve similar results with a custom solution tailored to your specific needs.

Built for

Web3 Foundation

Get in touch

Tekika Adventure

REAX

Core JAM Concepts in Tessera

Implementing Tessera requires a deep understanding and accurate translation of the JAM Graypaper's core concepts into Python data structures and logic. Key areas of focus include:

Hybrid Architecture: Modeling the distinction between on-chain (Accumulation) and in-core (Refinement) computation logic and their interaction.
Service-Oriented Model: Implementing data structures and logic for service accounts, including state (storage, preimages), code (refine/accumulate entry points), balance, and gas limits.
Coretime Concept: Handling core assignment, authorization checks (verifying authorizer logic outputs), and coretime accounting from a client perspective.
Block & State Structure: Defining Python classes/objects for blocks, headers, extrinsics (tickets, preimages, reports, etc.), and the partitioned state tuple (α, β, γ...).
State Transition Function (Y): Implementing the block-level state transition logic, ensuring deterministic updates based on prior state and block contents.

Consensus Mechanism Interaction (Client-Side)

Safrole Verification: Validating block author eligibility based on sealing keys derived from tickets/VRFs. Implementing the fork-choice rule based on Safrole's criteria (e.g., highest valid ticket score, chain length).
Grandpa Verification: Processing Grandpa votes (prevotes, precommits) gossiped on the network. Verifying vote signatures against the known validator set. Identifying finalized blocks based on supermajority agreement (2/3+).
Validator Set Management: Tracking the current and upcoming validator sets (κ, λ, γκ) based on state transitions and epoch changes defined in Safrole.
Epoch Processing: Handling epoch transitions, including validator set rotation and Safrole state updates (roots, sealers).

State Management and Storage

Data Store: Utilizing a performant key-value database (e.g., LMDB, RocksDB) suitable for large datasets and frequent updates.
Merkle Trie Implementation: Implementing or using a library for the specific binary Patricia Merkle Trie structure defined in the JAM Graypaper (Appendix D), optimized for Python.
Atomic Updates: Ensuring state changes from a block are applied atomically; either all succeed or none do, maintaining state consistency.
State Root Calculation: Efficiently calculating the state root after each block application.
State Pruning: Implementing strategies to prune old state data that is no longer needed (beyond finality + archival requirements) to manage storage growth.
Preimage/Lookup Handling: Storing and managing preimage data (components p and l of service accounts) and implementing the historical lookup logic (Λ).