Backfill historical bundles

Backfill applies a function to historical source rows and writes derived data for the corresponding bundle. Use the same transformation as per-block computation, with a historical input and output context.

Chain and historical context

A bundle identifies a partition of source data. It does not select the EVM state for every function read. If your function queries historical state, obtain the correct block identifier from each source row and pass it to the read.

Prerequisites

A bundle deployment with access to the source data files and BlockDB BatchWrite.
A compatible target block table and permission to overwrite its selected bundle.
The source schema required by the transformation.
Historical node data when the function also performs EVM reads.

Submit one bundle

Pass the source_code from per-block computation to this helper. Use your source table, target table, and selected bundle number.

from blockx import (
    SOURCE_ROW, BlockBundleCallConfig, BlockTableWriteHandler,
    Function, TaskBuilder,
)


def submit_bundle(number, source_table, target_table, source_code):
    task = TaskBuilder.build(
        BlockBundleCallConfig(number=number, triggerSources=[{
            "table": source_table,
            "func": Function(source_code=source_code),
            "params": ["eth", SOURCE_ROW],
        }]),
        BlockTableWriteHandler(targetTable=target_table, block_bundle=number),
    )
    result = task.submit()
    if not result.task_result.success:
        raise RuntimeError(result.task_result.failure_code)
    return task.task_id, result.handler_result

The input and output use the same number. For a range, submit only the intended bundle numbers and retain each task and writer result. Keep a record of failed or unknown submissions before deciding which work to repeat.

Backfill a bundle range

backfill_block_bundles submits an inclusive range of bundle numbers. BlockBundleCallConfigTemplate supplies the shared trigger configuration. The helper calls for_bundle(number) on the input and block-table writer templates so both use the same bundle number.

from blockx import (
    SOURCE_ROW, BlockBundleCallConfigTemplate, BlockTableWriteHandler,
    Function, backfill_block_bundles,
)


def backfill_range(start_bundle_id, end_bundle_id,
                   source_table, target_table, source_code):
    config = BlockBundleCallConfigTemplate(triggerSources=[{
        "table": source_table,
        "func": Function(source_code=source_code),
        "params": ["eth", SOURCE_ROW],
    }])
    handler = BlockTableWriteHandler(targetTable=target_table)
    summary = backfill_block_bundles(
        start_bundle_id, end_bundle_id, config, handler,
    )
    if not summary.ok:
        raise RuntimeError(summary.failed_bundles, summary.failure_codes)
    return summary

Use the same table and function prerequisites as the single-bundle example. A writer template has no block_bundle yet; instantiate it with for_bundle(number) before submitting an individual task. The range helper performs that step for you. The helper returns a BackfillSummary with succeeded, failed_bundles, and failure_codes. By default it aborts if the first bundle fails. These counters describe task outcomes; they do not establish storage-side job completion.

Define a reusable pipeline

Pipeline groups trigger tables, a function, and a target table. Trigger.params uses SOURCE_ROW to place the input record in the function arguments. Reuse the EVM trace transformation above:

from blockx import Function, Pipeline, SOURCE_ROW, Trigger


def make_pipeline(source_table, target_table, source_code):
    return Pipeline(
        triggers=[Trigger(
            table=source_table,
            func=Function(source_code=source_code),
            params=["eth", SOURCE_ROW],
        )],
        target_table=target_table,
    )

Create pipe with make_pipeline(source_table, target_table, source_code), then choose the operation:

Operation	Range and behavior
`pipe.backfill(block_start=18000001, block_end=18100000)`	Inclusive block heights; full bundles run as bundle tasks, and partial boundaries run as block tasks
`pipe.backfill(block_start=18000001)`	Ends at the minimum ready height of the trigger and dependency tables
`pipe.update()`	Long-running subscription; a block-table target first catches up once, then processes aligned block events

Choose heights available in your source tables. Bundle numbers passed to backfill_block_bundles and block heights passed to Pipeline.backfill are different units. Exact boundary processing and update() also require a block deployment. Add lookup-only upstream tables through Pipeline(depends=[...]). They constrain readiness but do not trigger calls or supply arguments. The pipeline chooses the writer from the target table kind; Pipeline(condition=...) forwards a condition only for normal-table output. During the subscription phase, update() does not backfill height gaps. Fill historical gaps explicitly with backfill(). Normal-table and time-table targets start subscribing without the initial block catch-up.

Confirm completion

For a real block-table bundle write, the writer result includes job_id. A successful task confirms BatchWrite submission, not storage-side job completion. Record the job identifier and confirm its BlockDB status and target read conditions before marking the range ready. A computation with no output is not by itself proof that a source partition was complete. Check source availability and the intended overwrite semantics, especially when all rows are filtered out.

Handle partial failures

Outcome	Action
Source preparation fails	Correct the source or configuration before repeating
Function fails	Check the input row, historical read context, and function error
Write commit outcome is unknown	Inspect the existing write job before submitting another overwrite
Some bundles succeed	Track completion per bundle and per write job

Get started

Core concepts

Guides

Reference

Architecture

Components

Development guide

Backfill historical bundles

Chain and historical context

Prerequisites

Submit one bundle

Backfill a bundle range

Define a reusable pipeline

Confirm completion

Handle partial failures

Continue reading

​Chain and historical context

​Prerequisites

​Submit one bundle

​Backfill a bundle range

​Define a reusable pipeline

​Confirm completion

​Handle partial failures

​Continue reading

Chain and historical context

Prerequisites

Submit one bundle

Backfill a bundle range

Define a reusable pipeline

Confirm completion

Handle partial failures

Continue reading