Share Format Parsing and Decoding

Celestia blocks are organized into a matrix of shares, fixed-size data containers that are the atomic unit of the Data Availability (DA) layer. For indexers, custom light clients, and data pipeline operators, understanding the compact share format is essential to extract rollup data directly from raw blocks without relying on a full consensus node or a third-party API. This page details the binary layout, namespace association, and decoding rules required to manually parse Celestia shares.

The share format uses a compact, non-self-describing layout to minimize DA overhead. Each share belongs to a specific namespace, with the first share of a sequence containing a length-delimited beginning of the data and subsequent continuation shares packing raw bytes. Reserved namespaces, such as those for transactions and intermediate state roots, follow special encoding rules. Parsing logic must correctly handle padding shares, which fill the remainder of a row, and must distinguish between a sequence's first share and its continuations to reassemble the original blob or transaction.

Operators building custom indexers or trust-minimized light clients must implement this parsing logic to verify data availability sampling proofs against the block's data root. A failure to correctly decode the share format can lead to data corruption, missed rollup transactions, or an inability to reconstruct state. Teams integrating this logic into bridge verifiers or data availability sampling clients should subject their parsing implementation to a rigorous security review to ensure it handles all edge cases defined by the canonical specification.

Celestia blocks are composed of a sequence of fixed-size shares, each belonging to a specific namespace. The compact share format is the binary encoding that allows a block to be split into these uniform units and later reassembled. A share is not a transaction or a blob itself, but a container for a portion of one. The first share of a sequence includes a start indicator and a length field, while continuation shares carry only raw data. This layout is fundamental for any system that parses raw blocks, including indexers, custom light clients, and data availability sampling (DAS) verifiers.

Every share begins with a namespace ID (NAMESPACE_ID_BYTES), immediately followed by a share version and a sequence start indicator encoded in a single info byte. If the share is the first in a sequence, the next bytes encode a sequence_length as a variable-length unsigned integer (varint), with the remaining space filled by the raw data chunk. Continuation shares omit the length and pack data from the first byte after the info byte. The final share in a sequence is padded with zero bytes to maintain the fixed SHARE_SIZE. Reserved namespaces like TRANSACTION_NAMESPACE and PAY_FOR_BLOB_NAMESPACE have specific parsing rules that determine how data is split across shares, making correct handling of these namespaces critical for accurate data extraction.

A common parsing error is to treat all shares as independent data units, which breaks when a blob or transaction spans multiple shares. Implementations must buffer data across share boundaries, using the sequence_length from the first share to determine when a complete unit has been reconstructed. Another pitfall is ignoring the info byte's version bits, which could lead to silent misinterpretation of data if the share format is upgraded. Teams building custom indexers or light clients should verify their parsers against the celestia-app reference implementation and test edge cases like zero-length sequences, maximum-size blobs, and blocks with only padding shares.

For rollup developers and data teams extracting transaction or blob data without running a full consensus node, a correct share parser is the foundation of reliable data retrieval. Chainscore Labs can review custom share parsing implementations, assess compatibility with the latest Celestia network upgrade, and help integration teams build robust data pipelines that handle edge cases like reserved namespace rules and format version upgrades.