Call Builder and Writer Plugin

​

Call Builder

​

BlockTableCallConfig

{
  "block": { "id": "0x…", "height": 18000000, "timestamp": "2024-06-08T00:00:00Z" },
  "triggerSources": [
    {
      "table": "chain.trace.eth",
      "functionId": "",
      "code": { "sourceCode": "def _(chain_id, record):\n    ...\n" },
      "params": ["eth", "${chain.trace.eth}"],
      "operator": { "relations": [] }
    }
  ]
}

• A task can have multiple triggerSources; each is scanned independently and the results are merged into one call list.
• functionId and code.sourceCode are mutually exclusive: the former is a registered function, the latter is inline source code.
• When params is empty, the whole row is the only argument; when params is non-empty but has no "${<table>}" placeholder, all calls share the same static arguments, so the function runs only once.
• operator is optional; when omitted, every row of the trigger table at that block produces a call.

​

InputsCallConfig

{
  "function": "",
  "code": { "sourceCode": "def _(name):\n    ...\n" },
  "callList": [["ct2"], ["blockx"]]
}

• Performs no IO. callList[i] directly becomes the argument array of the i-th call.
• The field name is function, not functionId; blockx-py does this renaming automatically.
• Results are collected in completion order and are not guaranteed to match the row order of callList. When you need to map results back to inputs, have the return value carry the input.

​

BlockBundleCallConfig

{
  "blockBundle": { "number": 25001 },
  "triggerSources": [
    {
      "tableId": "chain.trace.op",
      "function": "",
      "code": { "sourceCode": "def _(chain_id, record):\n    ...\n" },
      "param": ["op", "${chain.trace.op}"],
      "operator": { "relations": [] }
    }
  ]
}

• A bundle is a data partition covering 1000 blocks, identified by blockBundle.number.
• tableId only identifies the source table. The Worker resolves this bundle’s data files from the table’s Iceberg snapshot by (tableId, number); state tables resolve through <tableId>._archive.
• The field names are function / param (singular), unlike functionId / params in BlockTableCallConfig.
• The bundle worker enables stream build by default: scanning and execution are pipelined, so even millions of rows do not need to be materialized into a full call list first.

​

Operator: filtering and deduplication

• filter supports and, or, not, equal, not_equal, gt, gte, lt, lte, is_null, and is_not_null. It evaluates with SQL three-valued logic and keeps only rows whose result is TRUE.
• deduplicate removes duplicates by the given columns; the key uses a deterministic encoding with type tags.
• dbScan executes the chain in Worker memory; bundleScan translates it to SQL and pushes it down to DuckDB.
• blockdb-py’s filter() / Operator produce this structure directly, so you can put it into operator as-is.

​

Writer Plugin

• Table-writing plugins read the target table’s column definitions first and project only the columns of the returned dict that belong to the target table; null values in outputs are skipped.
• When any handler’s config carries "debug": true, table-writing plugins only count rows and emit a table_write_debug log; they read no schema and send no write request. blockx-py turns this on with the environment variable BLOCKDB_DEBUG=1.
• Omitting resultHandler is also valid: after the Calls phase succeeds, the task goes straight to a terminal state and outputs are discarded.
• Writer Plugins are fail-fast: a write failure makes the task PLUGIN_FAILED, and retryable is determined by the plugin’s error classification.

​

BlockTableWriteHandler

• One name maps to two implementations, assembled at startup by the Worker process’s deployment profile (block / bundle); the caller cannot choose.
• The block cluster implementation requires config.block, writes once per task, and returns {written_rows, finish_time}.
• The bundle cluster implementation requires config.blockBundle and overwrites the same bundle of the target table through a BlockDB BatchWrite job (Init → presigned PUT → Commit).
• A shape mismatch fails in the Writer phase with PLUGIN_FAILED (block deployment requires config.block / bundle deployment requires config.blockBundle), by which time all calls have already run. blockx-py validates the shape against the target cluster before submitting, so you do not waste a full run.

​

NormalTableWriteHandler

• Upserts by the primary key id.
• condition is optional: {"column": "ts", "policy": "UpdateIfSmaller"} means a row is overwritten only when the new value is smaller in that column; policy is UpdateIfSmaller or UpdateIfLarger. This makes out-of-order or replayed writes naturally idempotent.
• Carries no block context, so the same config can be reused on both the real-time and backfill paths.

​

ReturnValueHandler

• Assembles outputs as-is into one JSON array and puts it in pluginResults[i].result. In blockx-py, read result.handler_result.
• The bundle worker does not register it; a task that carries it and is submitted to the bundle cluster is rejected with InvalidArgument at SubmitTask. blockx-py therefore always routes such tasks to the block cluster.

​

Combination cheat sheet

​

Summary

• Call Builders decide the input: BlockTableCallConfig reads a block, InputsCallConfig uses the task’s own arguments, BlockBundleCallConfig reads a bundle.
• Writer Plugins decide where results go: BlockTableWriteHandler writes L2, NormalTableWriteHandler writes L1, ReturnValueHandler returns them to the Client.
• The config shape of BlockTableWriteHandler must follow the cluster: block on the block cluster, blockBundle on the bundle cluster.
• operator filters and deduplicates trigger rows in the Builder phase to reduce the number of calls.

• Plugin system: interfaces, registry, assembly, and how to add a new Builder / Plugin.
• Bundle clusters: the bundleScan streaming scan and the BatchWrite write path.
• Submit a task: examples: the blockx-py code for every combination.