> ## Documentation Index > Fetch the complete documentation index at: https://docs.blockx.chaintable.com/llms.txt > Use this file to discover all available pages before exploring further. # Protocols and interfaces > Which protocols BlockX components use to talk to each other, what the messages look like, and what rules to follow when changing a protocol BlockX has only three kinds of inter-component protocols: gRPC (control plane and result plane), length-prefixed JSON frames over UDS (Worker and Python Executor), and JSON heartbeats in etcd (Worker registration and discovery). All protocol definitions shared across modules are consolidated in the `api/` directory; transport handlers, storage models, and state machines do not live there. For where this sits in the overall picture, see [Architecture overview](/en/architecture/overview). The constraints on `api/` come from `docs/specs/code-style.md` §3.1: it holds only protocol structures, message envelopes, and field conventions; structs keep the role of "data objects" and may include a small amount of side-effect-free validation or encoding helpers, but must not depend on external IO. | Caller | Callee | Transport | Definition file | | -------------------- | ------------------------ | ------------------------------------------------------------------------ | ----------------------------------------------------- | | Client | Coordinator | gRPC unary `ReserveWorkerSlot` | `api/grpc/coordinator/coordinator.proto` | | Client | Bundle Coordinator | gRPC unary `ReserveWorkerSlot` (separate service identity) | `api/grpc/bundlecoordinator/bundle_coordinator.proto` | | Coordinator | Worker | gRPC unary `RequestTaskSlot` | `api/grpc/worker/worker.proto` | | Client | Worker | gRPC unary `SubmitTask` / `GetTaskResult`, server-streaming `WatchTasks` | `api/grpc/worker/worker.proto` | | Client | Sync Invoker | gRPC unary `Invoke` / `DebugInvoke` / `Precheck` | `api/grpc/syncinvoker/sync_invoker.proto` | | Worker, Sync Invoker | Python Executor | UDS, `uint32` length prefix + JSON envelope (optional binary sidecar) | `api/uds/types.go`, `api/uds/codec.go` | | Worker | etcd | Lease-bound JSON heartbeat key | `api/etcd/registry.go`, `api/etcd/types.go` | | Worker, Sync Invoker | BlockDB / Meta / NodeRPC | gRPC / HTTP, clients in `internal/sdk/` | See [Backend Adapter](/en/components/backend-adapter) | ```mermaid theme={null} flowchart LR Client["Client / Indexer"] Coord["Coordinator"] BCoord["Bundle Coordinator"] Worker["Worker"] Exec["Python Executor"] SI["Sync Invoker"] Etcd["etcd"] Backend["BlockDB / Meta / NodeRPC"] Client -->|"gRPC ReserveWorkerSlot"| Coord Client -->|"gRPC ReserveWorkerSlot"| BCoord Coord -->|"gRPC RequestTaskSlot"| Worker BCoord -->|"gRPC RequestTaskSlot"| Worker Client -->|"gRPC SubmitTask / GetTaskResult / WatchTasks"| Worker Client -->|"gRPC Invoke / DebugInvoke / Precheck"| SI Worker -->|"UDS JSON frames"| Exec SI -->|"UDS JSON frames"| Exec Worker -->|"lease heartbeat"| Etcd Coord -->|"watch prefix"| Etcd BCoord -->|"watch prefix"| Etcd Worker -->|"gRPC / HTTP"| Backend SI -->|"gRPC / HTTP"| Backend ``` ## gRPC services All four proto files live in `api/grpc//`, and the generated Go code lives in sibling `*pb/` directories (`workerpb/`, `coordinatorpb/`, `bundlecoordinatorpb/`, `syncinvokerpb/`). Generation commands: ```bash theme={null} go install google.golang.org/protobuf/cmd/protoc-gen-go@latest go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest make proto ``` `make proto` is defined in the repo root `Makefile`; it generates all four services in one pass and also triggers `proto-blockdb`, `proto-meta`, and `proto-localtestservice` (these three belong to the backend adapters, with protos in `internal/sdk/*/proto/`). After changing a `.proto`, you must regenerate and commit the `*pb/` directories together with it. ### `WorkerService` (`api/grpc/worker/worker.proto`) ```protobuf theme={null} service WorkerService { rpc RequestTaskSlot(RequestTaskSlotRequest) returns (RequestTaskSlotResponse); rpc SubmitTask(SubmitTaskRequest) returns (SubmitTaskResponse); rpc GetTaskResult(GetTaskResultRequest) returns (GetTaskResultResponse); rpc WatchTasks(WatchTasksRequest) returns (stream TaskUpdate); } ``` * `RequestTaskSlot(task_id, ttl_ms) -> (slot_id, state)`: called by the Coordinator; `ttl_ms` only controls automatic reclamation of `ALLOCATED` slots and is not an execution timeout. * `SubmitTask(task: TaskInput, slot_id) -> state`: `slot_id` may be empty, in which case the Worker performs one `RequestTaskSlot` inline. `TaskInput` fields: `task_id`, `function_call_config{type, config bytes}`, `result_handler{type, config bytes}` (optional), `task_timeout_ms` (optional; 0 means the Worker default, and negative values or values exceeding the Worker's `maxTaskTimeoutMs` are rejected). Both `config`s are opaque JSON blobs; the Worker validates at the entry point that they are valid JSON (`submitTaskFromPB` in `internal/worker/adapters/wire.go`). * `GetTaskResult(task_id) -> (task_id, state, result?)`: `result` appears only in terminal states. * `WatchTasks(task_ids) -> stream TaskUpdate`: first pushes one current snapshot per `task_id`, then one update with `is_terminal=true` for each task as it reaches a terminal state. A `task_id` the Worker does not recognize is expressed with a non-empty `TaskUpdate.error` rather than failing the whole stream. It also serves as a run credential: once a task has had a Watch established, the disconnection of the last stream starts `watchDisconnectGraceMs`, and on timeout the task fails with `WATCH_DISCONNECTED`. * The `state` field is a string whose values come from `TaskState` in `internal/common/types/types.go`: `ALLOCATED`, `RUNNING`, `SUCCEEDED`, `FAILED`. * After the Client obtains `worker_addr` through the Coordinator, `SubmitTask` / `GetTaskResult` / `WatchTasks` all go directly to that Worker; the Coordinator does not relay results. Metadata conventions: request ownership uses `client-id` (with `x-instance-id` supported for compatibility), which the Worker normalizes at the `SubmitTask` boundary with `internal/common/clientid`; shadow forwarding is marked with `x-blockx-shadow-submit: true`, and the receiving side strips the `ResultHandler` (`internal/worker/adapters/shadow_forwarder.go`). ### `CoordinatorService` and `BundleCoordinatorService` ```protobuf theme={null} service CoordinatorService { rpc ReserveWorkerSlot(ReserveWorkerSlotRequest) returns (ReserveWorkerSlotResponse); } ``` `ReserveWorkerSlot(task_id) -> (task_id, worker_addr, slot_id)`. The slot TTL is determined by the Coordinator's configuration; there is no `chainId` or `reserveReqId` in the request. `bundle_coordinator.proto` defines `bundlecoordinator.v1.BundleCoordinatorService` with an identically named method and identical fields; the two sets of messages do not reference each other. The sole purpose of splitting them is to let an upstream proxy route by method path (`/coordinator.v1.CoordinatorService/ReserveWorkerSlot` vs `/bundlecoordinator.v1.BundleCoordinatorService/ReserveWorkerSlot`). `BundleCoordinatorServer` in `internal/coordinator/adapters/bundle_grpc_server.go` only converts messages and then delegates to the shared `CoordinatorServer`. See [Bundle clusters](/en/components/bundle) for details. ### `SyncInvokerService` (`api/grpc/syncinvoker/sync_invoker.proto`) ```protobuf theme={null} service SyncInvokerService { rpc Invoke(InvokeRequest) returns (InvokeResponse); rpc DebugInvoke(DebugInvokeRequest) returns (DebugInvokeResponse); rpc Precheck(PrecheckRequest) returns (PrecheckResponse); } ``` * One unary request executes one top-level call. The request uses `oneof function_source { function_id | inline_source }` (exactly one is required), `args_json` is the raw bytes of a JSON array, and `deadline_unix_ms` is an absolute deadline. * The response carries `call_id`, `success`, `result_json` or `Failure{code, message, retryable, origin}`, `execution_duration_ms`, and `queue_wait_ms`. `DebugInvoke` adds a `DebugOutput` (stdout/stderr tail, `error_stack`, `local_vars_json`). * `FailureCode` and `FailureOrigin` are proto enums, read directly from the proto; execution-phase failures are returned as response data (the gRPC status is still OK), and only admission-layer rejections produce a non-OK status, in which case `call_id` is returned via the trailer `x-blockx-sync-call-id` (`internal/syncinvoker/adapters/grpc_server.go`). * `Precheck` runs only the static audit without executing; a failed audit is `pass=false` + `findings` in a normal response, and non-OK statuses are reserved for cases where no conclusion can be reached (`FAILED_PRECONDITION` when auditing is disabled, `UNAVAILABLE` when the auditor is faulty, `INVALID_ARGUMENT` for empty source code). See [Sync Invoker](/en/components/sync-invoker) for details. ### Error model BlockX distinguishes two layers of errors, following the rules in `docs/specs/code-style.md` §7: * **Admission / protocol errors**: invalid arguments, invalid slot, no capacity, service not ready. These are returned as gRPC statuses and never enter `TaskResult`. * **Execution terminal states**: Builder / Call / Plugin / timeout failures after the task has been activated. These converge into `TaskResult`, and the gRPC status is OK. Stable business codes are defined in `internal/common/types/types.go`: ```go theme={null} type ErrorCode string const ( ErrInvalidArgument ErrorCode = "InvalidArgument" ErrInternal ErrorCode = "Internal" ErrNoSlot ErrorCode = "NoSlot" ErrUnavailable ErrorCode = "Unavailable" ErrInvalidSlot ErrorCode = "InvalidSlot" ErrNotFound ErrorCode = "NotFound" ErrNoExecutableSlot ErrorCode = "NoExecutableSlot" ErrSlotUncertain ErrorCode = "SlotUncertain" ) ``` On gRPC, business codes are **expressed only through the gRPC status code**; there are no status details and no trailers. The Worker-side mapping is `workerCodeToGRPC` in `internal/worker/adapters/wire.go`, the Coordinator-side mapping is `coordCodeToGRPC` in `internal/coordinator/adapters/grpc_server.go`, and the reverse mapping used when the Coordinator calls the Worker is `grpcErrToSlotError` in `internal/coordinator/adapters/worker_rpc.go`: | Business code | gRPC code | Description | | ---------------------------- | -------------------- | ----------------------------------------------------------------------------------------- | | `InvalidArgument` | `InvalidArgument` | Bad arguments; do not retry until corrected | | `NoSlot`, `NoExecutableSlot` | `ResourceExhausted` | No slot available; the Coordinator maps it back to `NoSlot` and treats it as backpressure | | `Unavailable` | `Unavailable` | Not ready or shutting down; no slot allocated | | `InvalidSlot` | `FailedPrecondition` | `slotId` does not exist, has expired, or does not match `taskId` | | `NotFound` | `NotFound` | Task does not exist or its result has been purged | | `SlotUncertain` | `Aborted` | A reservation was initiated but cannot be confirmed; wait one TTL and retry | | Other | `Internal` | | `worker.proto` declares an `enum ErrorCode` at the top, but nothing in the Go code uses `workerpb.ErrorCode`, and no status details are attached; callers can only branch on the status code. To restore fine-grained business codes, attach them via `google.rpc.ErrorInfo` details. ## UDS protocol (Worker ↔ Executor) The Worker listens on `/blockx-worker-.sock` (`NewAdapter` in `internal/worker/adapters/executor/adapter.go`); the Executor process connects on its own after starting and attaches via its first `Heartbeat`. The Sync Invoker reuses the same executor adapter and the same protocol. For field semantics see the comments in `api/uds/types.go` and `docs/specs/worker-executor-connection-and-python-sdk-hook.md` §4. ### Frame format `api/uds/codec.go` defines two frame types, with a maximum size of 16 MiB (`maxFrameSize`): ```text theme={null} legacy frame: uint32(BE) total length | JSON object sidecar frame: uint32(BE) total length | "BXB1" | uint32(BE) JSON length | JSON object | raw bytes ``` `BXB1` is a self-describing magic: messages without a sidecar are byte-for-byte identical to the old format; frames with a sidecar let protobuf bytes (for example BlockDB requests / responses) cross the Worker/Executor boundary without base64. The JSON object is a `MessageEnvelope`: ```go theme={null} type MessageEnvelope struct { ProtocolVersion string `json:"protocolVersion"` MessageType string `json:"messageType"` ExecutorID string `json:"executorId"` CallID string `json:"callId,omitempty"` RequestID string `json:"requestId,omitempty"` Payload json.RawMessage `json:"payload"` TraceContext map[string]string `json:"traceContext,omitempty"` BinaryPayload []byte `json:"-"` } ``` `ProtocolVersion` is currently always `"v1"` (`internal/worker/adapters/executor/send.go`). `TraceContext` carries W3C `traceparent` / `tracestate`, used to let span chains cross the UDS. ### Codec entry points | Function / type | Purpose | | ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ | | `WriteFrame(w, env)` | Encodes and writes one frame; `Payload` is concatenated as-is and is **no longer validated** as valid JSON, the caller guarantees it | | `ReadFrame(r)` / `ReadFrameInto(r, env)` | Reads one frame into an owned envelope | | `BorrowedFrameReader.Read / Release` | Zero-copy frame read; `Payload` is only valid until `Release`; use `DecodePayloadOwned` for data that needs to escape | | `EncodePayload / EncodePayloadInto` | Converts a payload struct to `json.RawMessage` | | `DecodePayload / DecodePayloadOwned` | Decodes with `UseNumber` semantics to avoid uint256 precision loss | | `IsJSONNull(raw)` | Checks whether a `RawMessage` is empty or `null` | The Go side uses `bytedance/sonic` for speed, and the wire bytes are identical to `encoding/json` (test cases such as `TestWriteFrameStdlibCanReadSonicOutput` in `api/uds/codec_test.go` pin this down). ### Message types and payloads Constants in `api/uds/types.go`: ```go theme={null} const ( // Worker -> Executor MsgExecuteCall = "ExecuteCall" MsgResumeCall = "ResumeCall" MsgCancelCall = "CancelCall" MsgHeartbeatAck = "HeartbeatAck" // Executor -> Worker MsgCallWaiting = "CallWaiting" MsgCallCompleted = "CallCompleted" MsgCallFailed = "CallFailed" MsgHeartbeat = "Heartbeat" ) ``` | Direction | Message | Payload type | Key points | | --------- | --------------- | ---------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | W→E | `ExecuteCall` | `ExecuteCallPayload` | `taskId`, `functionId`, `entrySelector`, `functionCodeDigest` or `functionCode`, `args`, `callBudgetMs`, `attemptSeq`, `debug`, optional `audit` (`AuditGatePayload`) | | W→E | `ResumeCall` | `ResumeCallPayload` | `resumeKind`, `result` or `error`, `budgetUsedMs`; `NewResumeCallPayload` expands errors that implement `StructuredError()` into a map | | W→E | `CancelCall` | `CancelCallPayload` | `reason`, `attemptSeq` | | W→E | `HeartbeatAck` | None | | | E→W | `CallWaiting` | `CallWaitingPayload` | `waitKind` is `io` / `subcall` / `function_code`; `taskId` is **required**; IO uses `mode`, `operation`, `backend`, `cacheKey`, `request` (`RawMessage`); subcall uses `functionId`, `args`, `ancestry`, `grantedBudgetMs` | | E→W | `CallCompleted` | `CallCompletedPayload` | `output` (`RawMessage`, never decoded along the way), execution duration, debug tail | | E→W | `CallFailed` | `CallFailedPayload` | `errorKind`, `detailCode`, `childOrigin`, `retryable`, `errorMessage`, `errorStack`, `localVars` | | E→W | `Heartbeat` | `HeartbeatPayload` | Queue counts, memory, CPU, `runningCallId`, `callStateCounts`, `lastSchedulerActiveAtMs` | The corresponding Python implementation: `python/blockx_executor/wire_keys.py` (field names and message type constants `MT_*` / `F_*` / `FP_*`), `python/blockx_executor/messages.py` (the `MessageEnvelope` dataclass), `python/blockx_executor/uds_codec.py` (`encode_frame`, `FrameDecoder`, `dumps_wire` / `loads_wire`, which also recognizes `BXB1`), and `python/blockx_executor/uds_session.py` (`UDSSession`, a single-connection reader/writer). Cross-language contract tests live in `python/tests/test_executor_protocol_contract.py` and `python/tests/test_uds_codec.py`. When changing payload fields, both the Go and Python sides must be updated; see [Python Executor](/en/components/python-executor). ## etcd registration and heartbeat `api/etcd/registry.go` defines the registry prefix constants and parser: ```go theme={null} const ( LegacyBlockWorkerRegistryPrefix = "/blockx/workers/" LegacyBundleWorkerRegistryPrefix = "/blockx/bundle-workers/" ProdDefaultBlockWorkerRegistryPrefix = "/blockx/prod/lanes/default/workers/" ProdDefaultBundleWorkerRegistryPrefix = "/blockx/prod/lanes/default/bundle-workers/" TestDefaultBundleWorkerRegistryPrefix = "/blockx/test/lanes/default/bundle-workers/" ) ``` * `ParseRegistryPrefix(prefix)` accepts only two formats: the two fixed legacy paths (mapped to an implicit `prod/default`), or the v2 form `/blockx//lanes//{workers|bundle-workers}/`. It must end with `/`, and segments may only contain lowercase letters, digits, and interior hyphens. * The Worker key is `prefix + workerAddr` (`key()` in `internal/worker/adapters/etcd_publisher.go`), and the value is the JSON of `WorkerHeartbeat`, bound to a lease (default TTL 10s) that is renewed by `KeepAlive`; `Deregister` revokes the lease, which deletes the key. Heartbeats are sent every 200ms by default (`HeartbeatIntervalMs`; 2000ms recommended in production). * Default prefixes per entry point: `cmd/worker` uses `/blockx/workers/`, `cmd/bundle_worker` uses `/blockx/bundle-workers/`; `cmd/coordinator` watches the legacy block prefix, and `cmd/bundle_coordinator` watches both the legacy and the prod v2 bundle prefixes, overridable with `COORDINATOR_REGISTRY_PREFIXES`. The Coordinator only watches an exact list of prefixes; watching the `/blockx/` root is not allowed. Heartbeat fields (`api/etcd/types.go`): ```go theme={null} type WorkerHeartbeat struct { WorkerAddr string `json:"workerAddr"` TaskSlots int `json:"taskSlots"` ReservedSlots int `json:"reservedSlots"` RunningTasks int `json:"runningTasks"` CPUPercent float64 `json:"cpuPercent"` MemoryPercent float64 `json:"memoryPercent"` MemoryAvailableBytes uint64 `json:"memoryAvailableBytes,omitempty"` MemoryTotalBytes uint64 `json:"memoryTotalBytes,omitempty"` ProcessRSSBytes uint64 `json:"processRssBytes,omitempty"` ErrorRate float64 `json:"errorRate"` LastHeartbeatMs int64 `json:"lastHeartbeatMs"` } ``` On the Coordinator side, `internal/coordinator/core/worker_view.go` reuses it directly with `type WorkerHeartbeat = etcd.WorkerHeartbeat`, and `decodeWorkerHeartbeat` in `internal/coordinator/adapters/etcd_watcher.go` falls back to the key with the prefix stripped when `workerAddr` is empty. Environment and lane do not enter the heartbeat fields, nor the protobuf. ## Task result model `TaskResult` only expresses the task-level result and does not include per-call return details. The proto definition (`worker.proto`): ```protobuf theme={null} message PluginResult { string plugin_name = 1; bool success = 2; string failure_code = 3; bytes result = 4; // JSON-encoded plugin result payload } message ExecuteResult { string failure_code = 1; bool retryable = 2; repeated PluginResult plugin_results = 3; } message TaskResult { bool success = 1; ExecuteResult execute_result = 2; // present only on terminal result } ``` * The terminal state is expressed by the outer `state`: `SUCCEEDED` corresponds to `success=true`, `FAILED` to `success=false`. Codes such as `TIMED_OUT` are not separate terminal states but values of `execute_result.failure_code`. * Task-level `failure_code`s that appear in the code: `BUILDER_FAILED`, `CALL_FAILED`, `PLUGIN_FAILED`, `TIMED_OUT`, `WATCH_DISCONNECTED`, `SLOT_LOST` (`internal/worker/core/worker.go`, `internal/worker/core/dispatcher.go`); the adapter side also uses `ACTIVATION_FAILED`, `IO_SCOPE_FAILED`, `BUILDER_NOT_FOUND`, `PLUGIN_NOT_FOUND`, `CANCELLED`, `UNKNOWN_FAILURE` (`internal/worker/adapters/orchestrator_phases.go`). These are string constants with no centralized enum. * `PluginResult.result` is JSON bytes; `ReturnValueResultHandler` puts the array of call outputs here to return to the Client. * The Go in-memory forms are in `internal/common/types/types.go` (`TaskResult` / `ExecuteResult` / `PluginResult` / `TaskUpdate`), and `taskResultToPB` in `internal/worker/adapters/wire.go` converts them to proto. The Go type `PluginResult` has `FailureMsg` and `Retryable` fields, but the `PluginResult` in `worker.proto` has no corresponding fields, and `taskResultToPB` does not transmit them. The Client cannot get plugin-level failure messages, only `failure_code`. ## Conventions for changing protocols From `docs/specs/code-style.md` §3.1 and `AGENTS.md`: * Prefer adding optional fields or new enum values. Do not silently change the meaning, type, or error-code semantics of existing fields; do not reuse deleted field numbers in proto. * Any `api/` change must update the corresponding spec (`worker.md`, `task-resource-coordinator.md`, `sync-invoker.md`, `worker-executor-connection-and-python-sdk-hook.md`, etc.) in the same change. Do not let structs in the code quietly become the new protocol truth. * Shared protocols are not scattered across adapters; they are consolidated in `api/`. Types in `api/` carry no business flow and do not depend on external IO. * When changing a UDS payload, Go (`api/uds/types.go`) and Python (`python/blockx_executor/wire_keys.py` and the corresponding payload construction) must be changed together, with test cases added to `python/tests/test_executor_protocol_contract.py`. Old peers tolerate new fields thanks to lenient JSON object semantics; old peers cannot consume `BXB1` sidecar frames, so the Worker and Executor must come from the same image. * When changing the gRPC error mapping, change both the forward mapping in `wire.go` and the reverse mapping in `worker_rpc.go`, and update `internal/worker/adapters/wire_test.go` and `internal/coordinator/adapters/worker_rpc_test.go`. * When changing registry prefix rules, update `api/etcd/registry_test.go` and `docs/specs/2026-07-28-registry-prefix-migration.md`. * When splitting out a new gRPC service identity (such as the bundle coordinator), use a separate proto and separate messages with no cross-references, so that the method path itself carries the routing identity. * Cross-component behavior is backstopped by the contract / system tests in `e2e/contract/` (`coordinator-worker`, `coordinator-etcd`) and `e2e/system/`; see [Testing](/en/development/testing). ## Related docs Specs in the blockx repo: * `docs/specs/code-style.md` §3.1 (`api/` constraints) and §7 (error and result expression). * `docs/specs/architecture.md` §4.3 (external results and protocols). * `docs/specs/worker.md` §2 (`RequestTaskSlot` / `SubmitTask` / result queries and subscriptions). * `docs/specs/task-resource-coordinator.md` §2 (Client → Coordinator and Coordinator → Worker protocols). * `docs/specs/sync-invoker.md` §3 (invocation protocol, including the full proto and Precheck). * `docs/specs/2026-07-23-bundle-coordinator-api.md` (separate service identity for the bundle coordinator). * `docs/specs/client-id-propagation.md` (`client-id` / `x-instance-id` propagation rules). * `docs/specs/worker-executor-connection-and-python-sdk-hook.md` §3–§5 (UDS connection model and message contract). * `docs/specs/2026-07-28-registry-prefix-migration.md` (etcd registry legacy / v2 prefixes). Site pages: [Architecture overview](/en/architecture/overview) · [Task lifecycle](/en/architecture/task-lifecycle) · [Worker](/en/components/worker) · [Task Resource Coordinator](/en/components/coordinator) · [Sync Invoker](/en/components/sync-invoker) · [Bundle clusters](/en/components/bundle) · [Python Executor](/en/components/python-executor) · [Backend Adapter](/en/components/backend-adapter)