btclib_node.block_db package

Module contents

BlockDB: blocks and their undo data on disk.

Blocks and reverse patches (RevBlock) are appended to flat, rotating files under data_dir; BlockLocation and FileMetadata, kept in the key-value store, are what let a later read seek straight to a block instead of scanning a file for it. A reverse patch is filed against its own block only once that block’s branch connects, pending_rev_blocks holding one generated for a branch update_chain may still refuse.

prune_up_to is this store’s own half of pruning (btclib-org/btclib-node#601): it deletes a block and its reverse patch, by height rather than by file, because this store’s own rotation (__find_block_file) tracks append order and not a file’s own height range the way Core’s FlatFilePos/nHeightFirst/nHeightLast (node/blockstorage.h, at bitcoin/bitcoin@ca7162cde5) does. live is what lets a .blk/.rev file still be reclaimed once every block it ever held has been pruned this way – _release’s own docstring is where that is argued end to end, the uncomfortable half (a syncing node’s own append order tracking height only approximately) included.

class btclib_node.block_db.BlockDB(data_dir: Path, logger: Logger, blocks_dir: Path | None = None)

Bases: object

Blocks and their undo data, appended to rotating flat files on disk.

blocks and rev_patches map a block hash to the BlockLocation that finds it inside a .blk or .rev file; files tracks every such file’s own size for __find_block_file’s rotation check. _LOCAL_BOOKKEEPING_MAX above is where that layout’s own bookkeeping fields are argued against Bitcoin Core’s.

Unlike Mempool (mempool.py:7-10), this is reached from more than one thread already: update_chain, on Node’s own thread, is every production caller of every method here, but the suite calls add_block directly from a test’s own thread while Node’s thread is concurrently inside update_chain reading earlier blocks back through get_block – and get_raw_transaction (rpc/callbacks.py:535) is one call away from doing the same for real, the moment a handler is added that is not routed through Node’s own loop. open_block_file and open_rev_file are each one BinaryIO handle with one file position shared by every seek, read and write that reaches it, so a write landing between a reader’s seek and its read moves the position out from under the read and hands it back whatever is there instead (btclib-org/btclib-node#432). _lock – one RLock for the whole instance, matching KeyValueStore’s own “one connection, and a lock around every use of it” (db.py:58-73) – is held for the whole of every public method below, the write path included: the race is symmetric, and a write left unlocked could still move a reader’s position it does not itself take. One lock rather than one per handle, because __add_data_to_file updates files – shared by both the .blk and the .rev sides – regardless of which handle it is writing through, and a second lock would only ever be taken together with the first, never instead of it.

add_block(block: Block) → None

Append block to the current .blk file; a no-op if held.

add_rev_block(rev_block: RevBlock) → None

Buffer rev_block for finalize to write out.

A no-op if rev_block’s own hash is already held, on disk or still pending – update_chain can generate the same patch more than once for a branch it has not yet committed to.

close() → None

Close the key-value store and any file still open for writing.

current_usage() → int

Bytes this store still accounts for, across every .blk/.rev file.

Core’s own BlockManager::CalculateCurrentUsage (node/blockstorage.cpp:811-818, at bitcoin/bitcoin@ca7162cde5) sums nSize + nUndoSize over every block file its own m_blockfile_info still tracks; self.files is this store’s own counterpart, one FileMetadata per .blk or .rev file not yet unlinked by _release, so the same sum over its size fields answers the same question. main._prune_chain’s own automatic- target walk is the one caller.

finalize() → None

Write every reverse patch buffered since the last finalize.

Each goes in the .rev file named for its own block’s .blk file (btclib-org/btclib-node#116), looked up now rather than at add_rev_block time since that is a pure buffer and this is where the write happens.

get_block(block_hash: bytes) → Block | None

Return the block stored under block_hash, or None if not held.

get_rev_block(block_hash: bytes) → RevBlock | None

Return the reverse patch for block_hash, or None if not held.

init_from_db() → None

Rebuild the in-memory index from what the store already holds.

prune_up_to(target_height: int, hash_at_height: Callable[[__annotationlib_name_1__], __annotationlib_name_2__]) → None

Delete every block and reverse patch from the last pruned height on.

hash_at_height is main.prune_up_to_height’s own active_chain.__getitem__ – the one caller this method has, shared by main._prune_chain’s own automatic-target walk and rpc.callbacks.prune_blockchain’s manual call: this store tracks locations by hash, never by height, so the height -> hash step lives with the caller that already holds BlockIndex.active_chain rather than being threaded through here. A no-op if target_height is at or behind what an earlier call already reached, the same idempotence add_block and add_rev_block already give the rest of this store – a retry after a crash, or a second automatic- target step that lands on a height an earlier one already passed, costs nothing extra.

rollback() → None

Discard every reverse patch buffered since the last finalize.

class btclib_node.block_db.BlockLocation(filename: str, index: int, size: int)

Bases: object

Where one block or reverse patch sits inside its own flat file.

filename names the .blk or .rev file it was appended to, index is the byte offset __add_data_to_file returned for it, and size is its length – together enough for __get_data_from_file to seek straight to it rather than scan the file.

classmethod deserialize(data: bytes) → BlockLocation

Parse a BlockLocation from the bytes serialize produced.

serialize() → bytes

Serialize this location to the bytes kept in the key-value store.

class btclib_node.block_db.Coin(tx_out: TxOut, height: int, is_coinbase: bool)

Bases: object

One UTXO row: an output paired with when it was made and how.

Core’s Coin (src/coins.h, at bitcoin/bitcoin@204256c73f) is the shape matched – a varint packing (height << 1) | coinbase, ahead of the output itself – and not matched in full: Core’s own version additionally runs the output through TxOutCompression, a space optimisation this class does not reproduce, so the two do not agree byte for byte and are not meant to. KeyValueStore’s own store is measured write-dominated rather than read-dominated – a modern block’s own reads costing on the order of 3us each against 17us for a delete or a put at eight million rows (btclib-org/btclib-node#586) – which argues for a varint kept as tight as var_int already makes it, not for folding in a second space optimisation on top of it.

height is the height of the block whose own transaction created this output, and is_coinbase is whether that transaction was the block’s own coinbase. UtxoIndex.add_block sets both when an output is first created; UtxoIndex.apply_rev_block restores a Coin exactly as add_block staged it for removal, so a coin a reorg brings back carries the height and the coinbase bit it was created with, never the height of the block being disconnected or of the one reconnecting it.

classmethod parse(data: BinaryData, *, check_validity: bool = True) → Coin

Build a Coin by parsing the bytes serialize produced.

serialize(*, check_validity: bool = True) → bytes

Serialize this Coin to the bytes kept under a utxo- key.

class btclib_node.block_db.FileMetadata(filename: str, size: int)

Bases: object

Bookkeeping for one flat file this store has written to.

filename is the .blk or .rev file, and size is how many bytes have been appended to it so far – both the next write’s own offset and, for a .blk file, what __find_block_file checks against the rotation threshold.

classmethod deserialize(data: bytes) → FileMetadata

Parse a FileMetadata from the bytes serialize produced.

serialize() → bytes

Serialize this metadata to the bytes kept in the key-value store.

class btclib_node.block_db.RevBlock(hash: bytes, to_add: list[tuple[OutPoint, Coin]], to_remove: list[OutPoint])

Bases: object

Undo data for one block, filed once its own branch connects.

to_add is the prevout each spent input consumed, restored on reversal; to_remove is every outpoint the block itself created, dropped on reversal. UtxoIndex.add_block builds one alongside the block it applies, and UtxoIndex.apply_rev_block is what walks it back. to_add carries a Coin, not a bare TxOut, for the same reason Coin exists at all: a coin a reorg restores is a coin the maturity rule has to be able to judge again, and only a Coin still carries what that needs. Core does the same for the same reason – its own CTxUndo holds a Coin rather than a CTxOut.

classmethod deserialize(data: bytes, *, check_validity: bool = False) → RevBlock

Parse a RevBlock from the bytes serialize produced.

serialize(*, check_validity: bool = False) → bytes

Serialize this reverse patch to the bytes stored in a .rev file.