> ## Documentation Index
> Fetch the complete documentation index at: https://docs.blockx.chaintable.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Architecture overview

> What BlockX is, which components it is made of, how a task flows through the system, and the order in which contributors should read the docs.

BlockX is a function-execution framework for on-chain data. It receives tasks submitted by the Client, scans trigger data on a Worker, executes lightweight user-written Python functions in parallel, and writes the results back to BlockDB. The core computation model is single-row state computation from `onchain table -> onchain table`.

BlockX is only responsible for executing computations. Task generation, DAG orchestration, trigger management, and scheduled jobs are all the Client's responsibility.

This group of pages is for developers who want to contribute code to BlockX. If you only want to know where the code is and which directory to look at for a given change, go straight to [Repository layout and code layering](/en/architecture/repository-layout).

## The system at a glance

<Steps>
  <Step title="The Client requests execution resources">
    The Client calls `ReserveWorkerSlot(taskId)` on any Coordinator. The Coordinator picks a candidate Worker based on the Worker heartbeats in etcd, calls `RequestTaskSlot` on it, and hands `workerAddr + slotId` back to the Client. The Client can also bypass the Coordinator and connect to a Worker directly.
  </Step>

  <Step title="The Client submits the task">
    The Client calls `SubmitTask(task, slotId?)` on the target Worker. The Worker validates and activates the slot, creates the task context, and pins the function code snapshot this task will use.
  </Step>

  <Step title="The Worker executes in three phases">
    In the Builder phase, Call Builders run serially and scan trigger data to produce the `call list`; in the Calls phase, the dispatcher delivers calls in windows to the long-lived Python Executor process pool for parallel execution; in the Plugin phase, the Writer Plugin writes the aggregated `outputs` to BlockDB in one shot.
  </Step>

  <Step title="The Client fetches the result">
    `SubmitTask` only returns a submission acknowledgement. The Client obtains the terminal `TaskResult` from the same Worker through `GetTaskResult` or the `WatchTasks` stream.
  </Step>
</Steps>

See [Task lifecycle](/en/architecture/task-lifecycle) for the complete end-to-end sequence, the slot state machine, and the result model.

## Component diagram

```mermaid theme={null}
flowchart LR
    client[Client]

    subgraph control["Control plane"]
        coordinator["Coordinator<br/>(block / bundle variants)"]
        etcd[("etcd<br/>Worker registration / heartbeat")]
    end

    subgraph worker["Execution plane / Worker process"]
        wcore["WorkerCore<br/>slot / task state machine"]
        builder["Call Builder<br/>dbScan / callList / bundleScan"]
        dispatcher["DispatcherCore<br/>windowed dispatch / Executor selection"]
        fcode["Function Code View<br/>function version pinned by epoch"]
        io["IO access subsystem<br/>TaskIOScope / cache / admission"]
        plugin["Writer Plugin<br/>blockdbWrite / tableUpserts / bundleWrite"]
    end

    subgraph executors["Python Executor process pool"]
        exec["Executor + SDK hook"]
    end

    subgraph backends["External dependencies"]
        blockdb[("BlockDB")]
        meta[("Meta")]
        rpc[("Chain node RPC")]
    end

    syncinv["Sync Invoker<br/>(separate process, synchronous function calls, own Executor pool)"]

    client -->|"ReserveWorkerSlot (gRPC)"| coordinator
    coordinator <-->|watch| etcd
    coordinator -->|"RequestTaskSlot (gRPC)"| wcore
    client -->|"SubmitTask / GetTaskResult / WatchTasks (gRPC)"| wcore
    wcore -->|registration / heartbeat| etcd

    wcore --> builder --> dispatcher --> plugin
    dispatcher -->|"Get(functionId, epoch)"| fcode
    dispatcher <-->|"UDS: ExecuteCall / CallWaiting / CallFinished"| exec
    exec -.->|"IO and sub-function requests go back to the Worker over UDS"| io
    builder --> io
    plugin --> io
    io --> blockdb
    io --> meta
    io --> rpc
    fcode -->|subscribes to function code changes| blockdb

    client -->|"Invoke (gRPC)"| syncinv
    syncinv -.->|"reuses executor adapter / FCV / IO code"| exec
```

Every box in the diagram corresponds to a component page. `Sync Invoker` reuses the Worker's executor adapter, Pool Manager, Function Code View, and IO subsystem code, and starts an independent Executor pool in its own process, but does not go through the slot / task flow. The `bundle` cluster is deployed from the same code with a different process profile and etcd registration prefix, and is dedicated to large-scale bundle backfill tasks.

## Components and responsibilities

| Component                | Responsibility in one sentence                                                                                                                            | Main code                                                                                       | Docs                                                      |
| ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------- | --------------------------------------------------------- |
| Coordinator              | Picks a Worker from the etcd heartbeat view and reserves a slot on it on behalf of the Client; no central state, multiple instances allowed               | `internal/coordinator/`, `cmd/coordinator/`, `cmd/bundle_coordinator/`                          | [Coordinator](/en/components/coordinator)                 |
| Worker                   | Source of truth on the execution plane: slot allocation and reclamation, three-phase task execution, short-term result caching, etcd heartbeat            | `internal/worker/`, `cmd/worker/`, `cmd/bundle_worker/`                                         | [Worker](/en/components/worker)                           |
| Call execution subsystem | The dispatcher schedules calls in windows, selects Executors, and handles sub-function calls; the Executor Pool manages Python processes and UDS channels | `internal/worker/core/dispatcher*.go`, `internal/worker/adapters/executor/`, `api/uds/`         | [Call execution subsystem](/en/components/call-execution) |
| Function Code View       | Versioned view of function code inside the Worker; pins `taskCodeEpoch` when a task is activated; includes static auditing of function code               | `internal/functioncode/`, `python/blockx_audit/`                                                | [Function Code View](/en/components/function-code)        |
| IO access subsystem      | Single entry point for all BlockDB / RPC / Meta access: Worker-level cache, task-scoped singleflight, IO windows, backend-adaptive admission              | `internal/io/`                                                                                  | [IO access subsystem](/en/components/io-subsystem)        |
| Plugin system            | Call Builders turn trigger data into a `call list`; Writer Plugins persist `outputs`                                                                      | `internal/plugin/`                                                                              | [Plugin system](/en/components/plugins)                   |
| Python Executor          | Long-lived Python process that runs user functions with `greenlet` and suspends them on IO; the SDK hook intercepts IO requests back to the Worker        | `python/blockx_executor/`, `python/blockx_sdk/`                                                 | [Python Executor](/en/components/python-executor)         |
| Backend Adapter          | Go clients for external services such as BlockDB, NodeRPC, Meta, and Iceberg                                                                              | `internal/sdk/`                                                                                 | [Backend Adapter](/en/components/backend-adapter)         |
| Sync Invoker             | Function invocation entry point for synchronous request/response; reuses the Executor pool code but does not go through the task flow                     | `internal/syncinvoker/`, `cmd/syncinvoker/`                                                     | [Sync Invoker](/en/components/sync-invoker)               |
| Bundle cluster           | Runs bundle backfills with a separate process profile: the bundleScan builder and the bundleWrite / tableUpserts plugins                                  | `cmd/bundle_*`, `internal/plugin/callbuilder/bundlescan/`, `internal/plugin/event/bundlewrite/` | [Bundle clusters](/en/components/bundle)                  |
| Observability            | Structured logging, Prometheus metrics, tracing, usage collection, client id propagation                                                                  | `internal/obs/`, `internal/usage/`, `internal/common/clientid/`                                 | [Observability](/en/components/observability)             |

## Core constraints

These constraints run through the design of every component. If you find yourself breaking one of them while changing code, go back to the corresponding spec first.

* **Task-centric**: the task is the smallest scheduling unit and runs on exactly one Worker; all caching and state sharing is anchored to the task and released when the task ends.
* **All calls within a task run in parallel**: calls have no ordering dependencies between them and share the same read-only world state at a single point in time.
* **Function versions are fixed within a task**: `taskCodeEpoch` is pinned when the task is activated, and all subsequent top-level calls and sub-function calls resolve against the same code snapshot; hot code updates only affect new tasks.
* **Python-side IO must be hijackable**: every BlockDB / RPC request a user function makes through the SDK is hooked back to the Worker for unified handling; the Executor is never allowed to connect to external storage directly.
* **Functions are read-only; writes go through Plugins**: user functions cannot write to BlockDB; each task has at most one Writer Plugin, which writes everything in one shot during the Plugin phase.
* **No task-level automatic retry, no cancellation**: the framework only provides timeouts; whether to resubmit is up to the Client, based on `TaskResult.retryable`. Inside the Worker, bounded attempt retries of individual calls are allowed.
* **The Worker's local slot table is the single source of truth**: the Coordinator only maintains a discardable derived view, and etcd only carries registration and heartbeats. When multiple Coordinators pick the same Worker at the same time, the conflict converges at the Worker's atomic capacity check.
* **`TaskResult` only expresses task-level results**: it does not return the return value of each call; the only terminal states are `SUCCEEDED / FAILED`, and failures are expressed with an error code and `retryable`.

## Three design principles

* **Sans-IO**: state machines, scheduling decisions, and protocol mapping are concentrated in the pure in-memory `core/`; adapters take care of RPC, UDS, storage, clocks, and observability. `WorkerCore`, `CoordinatorCore`, and `DispatcherCore` all follow the one-way pattern "input event → core decision → adapter executes commands".
* **Scoped Resource Context**: subsystems such as IO access, which revolve around resource ownership and lifecycle reclamation, are organized into a Worker-level shared scope and task-level local scopes, binding quota, cache, singleflight, and deadline to the scope lifecycle.
* **Occam's Razor**: do not multiply entities beyond necessity. Prefer reusing existing concepts, states, interfaces, error codes, and protocols.

See [Design principles and code conventions](/en/architecture/design-principles) for how the principles turn into code constraints.

## When not to use BlockX

BlockX is not suited to the following computations:

* Computations that depend on a large number of rows from the same table, such as wide-range aggregation, complex window statistics, or candlestick (K-line) computation.
* Scenarios that need arbitrary time-window state or large-scale stateful computation.
* Complex computations that depend heavily on a streaming engine's exactly-once state recovery.
* Arbitrary incremental streaming computation over ordinary online tables.

## Suggested reading order

<Steps>
  <Step title="Start with the main path">
    [Task lifecycle](/en/architecture/task-lifecycle) and [Protocols and interfaces](/en/architecture/protocols). After reading them you should be able to say which RPCs, phases, and states a task goes through.
  </Step>

  <Step title="Then see how the repository is layered">
    [Repository layout and code layering](/en/architecture/repository-layout) and [Design principles and code conventions](/en/architecture/design-principles).
  </Step>

  <Step title="Dive into the component you need to change">
    Use the component table above to reach the corresponding page. Every component page lists code locations, core types, extension points, and tests.
  </Step>

  <Step title="Read the development guides before you start">
    [Local development environment](/en/development/getting-started), [Testing](/en/development/testing), and [Contributing](/en/development/contributing).
  </Step>
</Steps>

<Info>
  When you change behavioral semantics, update the design documents under `docs/specs/` in the BlockX repository first, then the code, and put both in the same PR.
</Info>