> ## Documentation Index > Fetch the complete documentation index at: https://docs.blockx.chaintable.com/llms.txt > Use this file to discover all available pages before exploring further. # Task lifecycle > The end-to-end main path of a task: submitted by the Client, placed by the Coordinator, executed through three phases on the Worker, and queryable in its terminal state A task is the smallest scheduling and execution unit in BlockX: the Client generates a `taskId`, submits a `TaskInput` to a Worker, and the Worker runs the three phases Builder → Calls → Plugin locally and converges on a `TaskResult`. This page covers only that main path; for component details see [Worker](/en/components/worker), [Coordinator](/en/components/coordinator), [Call execution subsystem](/en/components/call-execution), and [Plugin system](/en/components/plugins); for the overall architecture see [Architecture overview](/en/architecture/overview). The main path relies on the following constraints (see also `docs/specs/architecture.md` §2.1 in the blockx repo): * Task-centric: shared state, read cache, and timeouts are all anchored to the task. * Single-Worker execution: a task runs on exactly one Worker. The Worker is unaware of the Coordinator and only exposes `RequestTaskSlot` / `SubmitTask`. * All calls within a task run in parallel: there are no ordering dependencies between calls; the dispatcher ramps them up gradually in windows. * Fixed function version per task: on activation, a `taskCodeEpoch` (`TaskContext.TaskCodeEpoch`) is pinned; code updates during that time only affect subsequent tasks. * Read-only functions: user functions never write to the database directly; all writes go through a Writer Plugin. * No task-level automatic retries: the Worker only does bounded attempt retries for individual calls (`CallRetryPolicy`); whether to resubmit a task is decided by the Client based on `TaskResult.executeResult.retryable`. * No cancellation: only task timeouts (`taskTimeoutMs`) and optional Watch-disconnect reclamation. ## Task model The structure submitted externally is the gRPC `TaskInput` (`api/grpc/worker/worker.proto`); the Go counterpart is `commontypes.TaskInput` (`internal/common/types/types.go`): | Field | Type | Description | | ---------------------- | ---------------- | ----------------------------------------------------------------------------------------------------------------------- | | `task_id` | `string` | A stable unique ID generated by the Client (uuid4 recommended). The idempotency key. | | `function_call_config` | `{type, config}` | `type` selects the Call Builder; `config` is opaque JSON parsed by the corresponding Builder. | | `result_handler` | `{type, config}` | Optional. `type` selects the Writer Plugin. At most one per task. | | `task_timeout_ms` | `int64` | Optional. `0` uses the Worker default; negative values are rejected; values above `maxTaskTimeoutMs` are also rejected. | Currently registered Builder `type`s (`internal/plugin/callbuilder/types/types.go`): `BlockTableCallConfig` (dbScan), `CallListCallConfig`, `BlockBundleCallConfig` (bundleScan). Currently registered Writer Plugin `type`s (`internal/plugin/event/types/types.go`): `BlockWriteResultHandler`, `ReturnValueResultHandler`, `BlockBundleWriteResultHandler`, `TableUpsertsResultHandler`. A minimal JSON example (assembled from the fields of `commontypes.TaskInput` and `call_list.CallListCallConfigBuilderDecl`; not a fixture from the repo): ```json theme={null} { "task_id": "3f2b1c9e-8a4d-4c1e-9b0a-6f7e5d4c3b2a", "functionCallConfig": { "type": "CallListCallConfig", "config": { "function": "hello_world", "callList": [["ct2"], ["blockx"]] } }, "resultHandler": { "type": "ReturnValueResultHandler", "config": {} }, "taskTimeoutMs": 30000 } ``` `TaskInput` carries no block context at the top level. Block information is carried by the specific Builder's config (for example `dbscan.DBScanBuilderDecl.Block`) and then written into each call by the Builder. When the Worker receives a request, `submitTaskFromPB` in `internal/worker/adapters/wire.go` converts the proto into `commontypes.SubmitTaskRequest` (also validating that `config` is valid JSON and `task_timeout_ms >= 0`); `parseTaskPayload` in `internal/worker/adapters/payload.go` then wraps it into `commontypes.TaskCtx`: ```go theme={null} // TaskCtx carries the immutable context injected when a task is activated. type TaskCtx struct { TaskInput Stage Stage `json:"stage"` DebugMode bool `json:"debugMode,omitempty"` Deadline time.Time `json:"deadline"` Ctx context.Context `json:"-"` Cancel context.CancelFunc `json:"-"` } ``` `TaskCtx` is the minimal context shared by the Builder, Writer Plugin, and IO scope; `Stage` is rewritten by the adapter to `builder` / `executor` / `plugin` as phases advance. `WorkerCore` only hangs it on `TaskContext.AdapterTaskCtx` as an opaque pointer. ## End-to-end sequence ```mermaid theme={null} sequenceDiagram participant C as Client participant Co as Coordinator participant E as etcd participant W as Worker participant B as Builder participant D as Dispatcher participant X as Executor participant IO as IO scope participant P as Writer Plugin W->>E: Register + heartbeat (WorkerHeartbeat) Co-->>E: watch Worker view C->>Co: ReserveWorkerSlot(task_id) Co->>W: RequestTaskSlot(task_id, ttl_ms) W-->>Co: slot_id, state=ALLOCATED Co-->>C: worker_addr, slot_id C->>W: SubmitTask(task, slot_id?) W-->>C: state=RUNNING W->>W: Resolve epoch, create TaskIOScope W->>B: Build(ctx, TaskCtx, io) B->>IO: Read B-->>W: CallList W->>D: StartCallPhase(CallList) D->>X: ExecuteCall (UDS) X->>IO: CallWaiting / IO request IO-->>X: ResumeCall X-->>D: CallCompleted / CallFailed D-->>W: ConvergeTaskCalls(outputs) W->>P: Execute(ctx, TaskCtx, outputs, io) P->>IO: Write P-->>W: PluginResult W->>W: ConvergeTask, release slot, cache result W-->>C: WatchTasks terminal TaskUpdate C->>W: GetTaskResult(task_id) ``` The 16 steps of the main path are laid out below (see also `docs/specs/architecture.md` §3). The Client generates a `task_id` and prepares a `TaskInput`. BlockX is not responsible for task generation, DAG orchestration, or scheduled triggering. Call `CoordinatorService.ReserveWorkerSlot(task_id)` (`api/grpc/coordinator/coordinator.proto`). The Coordinator rebuilds its Worker view from the heartbeats it watches in etcd (`CoordinatorCore.ApplyWorkerUpdate`) and picks candidates with `SelectCandidate`. Without going through the Coordinator, the Client can call `RequestTaskSlot` on a Worker directly, or call `SubmitTask` directly and let the Worker request the slot automatically. `CoordinatorServer.ReserveWorkerSlot` (`internal/coordinator/adapters/grpc_server.go`) tries candidate Workers sequentially, calling `WorkerService.RequestTaskSlot(task_id, ttl_ms)` on each, with `ttl_ms` taken from `CoordinatorCore.SlotTTLMs()`. On rejection it moves to the next candidate; an RPC timeout counts as "possibly allocated" and is retried a bounded number of times against the same Worker with the same `task_id`. `WorkerCore.HandleRequestTaskSlot` (`internal/worker/core/worker.go`) checks `freeCapacity()` under `o.mu`, switches a `FREE` slot to `ALLOCATED`, writes `LeaseDeadlineMs = now + ttlMs`, and returns `slot_id` and `state=ALLOCATED`. The Coordinator stores no task payload, stores no results, and writes no slot truth to etcd; `slot_id` is just an opaque string to it. The Client connects directly to `worker_addr` and calls `WorkerService.SubmitTask(task, slot_id)`. `Orchestrator.SubmitTask` (`internal/worker/adapters/orchestrator.go`) first computes the effective deadline (`WorkerCore.EffectiveTaskDeadlineMs`) and parses the payload, then hands off to `WorkerCore.HandleSubmitTask`. `HandleSubmitTask` does the following in order: ready/draining check; if the same `taskId` is already in `w.tasks`, return `RUNNING`, and if a retained result exists, return the terminal state; if no `slot_id` was provided, run `HandleRequestTaskSlot` inline once; verify that the slot exists, is `ALLOCATED`, and is bound to the same `taskId`; once that passes, create a `TaskContext` (`Phase = Preparing`) and return the `PrepareTaskActivation` command. The gRPC layer returns `state=RUNNING` at this point. `Orchestrator.activateAndRun` (`internal/worker/adapters/orchestrator_phases.go`) resolves the current `taskCodeEpoch` in a goroutine, pins a Function Code View snapshot, then feeds back `WorkerCore.HandleTaskActivationPrepared`: the slot switches to `RUNNING`, `Phase` advances to `Builder`, and `StartBuilderPhase` is emitted. It then creates the `TaskIOScope` (`WorkerIOScope.NewTaskScope`) and the adapter-side `taskRuntime`. If any step fails, it goes through `HandleTaskActivationFailed` with failureCode `ACTIVATION_FAILED` or `IO_SCOPE_FAILED`. `runBuilderPhase` first acquires `builderSem`, looks up the Builder in `BuilderRegistry` by `FunctionCallConfig.Type`, and calls `CallBuilder.Build(ctx, taskCtx, io)` to get a `cbtypes.CallList`. The result is handed back to the core via `HandleTaskPhaseFinished(PhaseBuilder, ...)`, and the core emits `StartCallPhase`. `runCallPhase` acquires `executorSem` and calls `DispatcherCore.PrepareCallPhaseWithDigests` to set up `TaskDispatchState`; the dispatch loop then sends calls to the Python Executor over UDS via `ExecuteCall` in windows. SDK IO requests inside the Executor come back to the Worker as `CallWaiting`, are handled by the IO scope, and then `ResumeCall`; subfunction calls likewise return to the dispatcher for rebinding. See [Call execution subsystem](/en/components/call-execution) and [IO access subsystem](/en/components/io-subsystem) for details. Once all root calls reach a terminal state, `DispatcherCore.checkConvergeTask` produces `ConvergeTaskCalls{Success, FailureCode, Outputs}`; `Outputs` contains only the return values of successful root calls, and subcall return values are not included. The adapter converts it into a `PhaseOutcome` and calls `HandleTaskPhaseFinished(PhaseCalls, ...)`. The core emits `StartWriterPhase{Outputs}`; `runWriterPhase` acquires `pluginSem`, and if `ResultHandler` is non-empty, looks up the plugin in `PluginRegistry` and calls `WriterPlugin.Execute(ctx, taskCtx, outputs, io)`, producing `[]PluginResult`. `HandleTaskPhaseFinished(PhasePlugin, ...)` calls `convergeTask`: `Phase = Terminal`, `taskIndex` switches from `ActiveSlotID` to `Result`, `RetainedUntil = now + ResultRetentionMs`, and within the same critical section `releaseSlot` returns the slot to `FREE`, removes the task from `w.tasks`, and emits `ConvergeTask`. The adapter's `executePhaseResult` cancels unfinished calls, closes the `TaskIOScope`, deletes the `taskRuntime`, and finally triggers `OnTaskTerminal`. `SubmitTask` does not return a `TaskResult`. The Client receives the terminal `TaskUpdate` via a `WatchTasks` stream, or polls `GetTaskResult`. `OnTaskTerminal` is wired to `SubscriptionManager.NotifyTerminal` (`internal/worker/adapters/stream_subscriber.go`), which pushes the terminal state once to every stream subscribed to that `taskId`. `WorkerCore.DeriveHeartbeat` derives `reservedSlots / runningTasks` from the slot table, and `EtcdPublisher.RunHeartbeatLoop` writes to etcd periodically; the Coordinator updates its view from this. ## Slots and admission `RequestTaskSlot` and `SubmitTask` are two steps of the same Worker protocol; whether the caller is the Coordinator or the Client makes no difference to the Worker. * `RequestTaskSlot(task_id, ttl_ms)` only reserves; it does not execute. `ttl_ms` must be positive (`<= 0` returns `InvalidArgument`) and only controls automatic reclamation of `ALLOCATED` slots; it is unrelated to the task timeout. `TickTimers` scans `ExpiredSlots` every 500ms; expired slots are reclaimed by `HandleSlotLeaseExpired`, which also clears task records still in `Preparing`. * `SubmitTask(task, slot_id?)`: when `slot_id` is provided it must match a record that is `ALLOCATED` and bound to the same `taskId`, otherwise `InvalidSlot`; when omitted, the Worker requests a slot inline once using `DefaultTTLMs`, returning `NoSlot` if there is no capacity. * Idempotency: while the same `taskId` is `ALLOCATED` / `RUNNING`, `RequestTaskSlot` returns the same `slot_id`; once it has entered `RUNNING` or its result is still within the retention window, `SubmitTask` returns the stable current state and never starts another execution. After the old execution finishes and its result expires, the same `taskId` can be allocated a slot again. * There is no release interface: if you obtain a slot and then abandon the submission, you can only wait for the TTL to reclaim it. The boundary between admission errors and execution terminal states (see also the end of `docs/specs/architecture.md` §4.3.1): every failure before the slot switches from `ALLOCATED` to `RUNNING` is returned as a gRPC status and never enters `TaskResult`; once `HandleSubmitTask` has created a `TaskContext`, subsequent activation failures, Builder / Calls / Plugin failures, and timeouts all converge into a `TaskResult` terminal state. | Error code (`commontypes.ErrorCode`) | gRPC code (`workerCodeToGRPC`) | When it occurs | | ------------------------------------ | ------------------------------ | ------------------------------------------------------------------------------------------ | | `InvalidArgument` | `InvalidArgument` | `taskId` is empty, `ttl_ms <= 0`, `task_timeout_ms` is invalid, `config` is not valid JSON | | `NoSlot` | `ResourceExhausted` | No free slot locally | | `InvalidSlot` | `FailedPrecondition` | `slot_id` does not exist, is not `ALLOCATED`, or is bound to a different `taskId` | | `Unavailable` | `Unavailable` | Worker is not ready (no Executor heartbeat yet) or is draining | | `NotFound` | `NotFound` | `GetTaskResult` cannot find that `taskId` | For details on the slot table, `taskIndex`, and heartbeat derivation see [Worker](/en/components/worker); for the Coordinator's candidate selection, circuit breaking, and uncertain-state retries see [Coordinator](/en/components/coordinator). ## The three phases `WorkerCore` only decides whether to enter the next phase and when to write the terminal state; actual execution happens in the adapter. Phases advance via command / event round trips: the core returns `StartBuilderPhase` → the adapter executes → it feeds back `HandleTaskPhaseFinished(PhaseBuilder, outcome)` → the core returns `StartCallPhase`, and so on (`internal/worker/core/types.go` defines all command types). | Phase | Concurrency | Phase-level admission (`core.AdmissionConfig`, defaults) | Effect of failure on the task | | ------- | ----------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Builder | Runs serially once; IO carries a Builder retry scope, and the IO core does bounded retries on retryable errors | `builderSem` (`BuilderSlots = 16`) | Immediately `FAILED`, `BUILDER_FAILED` / `BUILDER_NOT_FOUND`, `retryable` taken from the IO error | | Calls | Fully parallel; per-task concurrent in-flight cap `TaskMaxInflightCalls = 1024`; bounded attempt retries per call | `executorSem` (`ExecutorTaskSlots = 32`; an empty call list does not take one) | `CallFailurePolicy` currently only has `fastFail`: as soon as any root call fails terminally, dispatching stops, outputs are discarded, and the task converges to `CALL_FAILED` (`retryable=false`); output exceeding `MaxCollectedOutputBytes` is `OUTPUT_BYTES_EXCEEDED` | | Plugin | Runs serially once; IO carries a Result retry scope | `pluginSem` (`PluginSlots = 16`) | `FAILED`, `PLUGIN_FAILED` / `PLUGIN_NOT_FOUND`, `retryable` decided by `PluginError.Retryable` (unclassified errors default to retryable) | Other points: * If the task ctx is cancelled while a phase is waiting for admission (for example, it has already converged due to timeout), the phase fails with `CANCELLED`; before entering Calls, the deadline is also checked first, and if it has already expired the task converges directly as `TIMED_OUT` without taking `executorSem`. * After the Calls phase succeeds, the core always emits `StartWriterPhase`; when `ResultHandler` is empty, `runWriterPhase` still acquires `pluginSem` but does not execute a plugin, and `pluginResults` is empty. * Streaming Builders (`StreamingCallBuilder`, only for `BlockBundleCallConfig`, `StreamBuildConfig.Enabled` off by default) shrink the Builder phase to `PrepareStream` and move the scan into the Calls phase, constrained by `scanSem` (`ScanSlots = 8`); a failure midway through production means some calls have already executed. * Timeouts are handled by `TickTimers` scanning `TimedOutTasks` and calling `HandleTaskTimedOut`, which forces convergence to `TIMED_OUT` (`retryable=true`) without requiring immediate interruption of calls running in the Executor; in-flight calls receive `CancelCall`. A task has only four externally visible states (`commontypes.TaskState`); the internal phases (`core.TaskPhase`: `Preparing / Builder / Calls / Plugin / Terminal`) are all folded into `RUNNING`: ```mermaid theme={null} stateDiagram-v2 [*] --> ALLOCATED: RequestTaskSlot ALLOCATED --> [*]: Reclaimed on TTL expiry ALLOCATED --> RUNNING: SubmitTask validation passed RUNNING --> SUCCEEDED: Plugin phase succeeded RUNNING --> FAILED: Activation failed / BUILDER_FAILED / CALL_FAILED / PLUGIN_FAILED / TIMED_OUT / WATCH_DISCONNECTED SUCCEEDED --> [*]: Result retention window expired FAILED --> [*]: Result retention window expired ``` ## Results and queries `TaskResult` only expresses the task-level result and does not include each call's return value. The following example shows the in-memory Go type (`commontypes.TaskResult`) serialized as JSON. The gRPC proto carries the corresponding wire fields but omits some Go-only fields: ```json theme={null} { "success": false, "executeResult": { "failureCode": "PLUGIN_FAILED", "retryable": true, "pluginResults": [ { "pluginName": "BlockWriteResultHandler", "success": false, "failureCode": "PLUGIN_FAILED", "failureMsg": "..." } ] } } ``` * The terminal state is expressed by the `state` field (`SUCCEEDED` / `FAILED`); the Client does not need to infer it from `success`. `TIMED_OUT`, `WATCH_DISCONNECTED`, and so on are `failureCode`s under `FAILED`, not separate terminal states. * `failureCode`s that appear in the code: `ACTIVATION_FAILED`, `IO_SCOPE_FAILED`, `SLOT_LOST`, `BUILDER_NOT_FOUND`, `BUILDER_FAILED`, `CALL_FAILED`, `OUTPUT_BYTES_EXCEEDED`, `PLUGIN_NOT_FOUND`, `PLUGIN_FAILED`, `CANCELLED`, `TIMED_OUT`, `WATCH_DISCONNECTED` (defined in `internal/worker/core/worker.go`, `internal/worker/adapters/orchestrator_phases.go`, and `internal/worker/core/dispatcher_internal.go`). * `ReturnValueResultHandler` puts the array of call outputs into `pluginResults[i].result` (JSON bytes on gRPC). * The gRPC `PluginResult` has only four fields: `plugin_name / success / failure_code / result`; the `FailureMsg` and `Retryable` fields of the Go type are not sent on the wire (`taskResultToPB`). Both query entry points are provided by the Worker; the Coordinator is not involved: * `GetTaskResult(task_id)`: `WorkerCore.HandleGetTaskResult`. Returns `RUNNING` for an active task; `ALLOCATED` if only reserved and not yet submitted; `state + result` if terminal and not expired; otherwise `NotFound`. * `WatchTasks(task_ids) -> stream TaskUpdate`: `SubscriptionManager.WatchTasks`. It first sends a current snapshot for each `task_id` (an unknown `task_id` gets `is_terminal=true, error="not_found"` so the whole batch does not fail), then pushes one terminal update for each task still running; once all tasks are terminal, the stream closes itself. `TaskUpdate` contains `task_id / state / is_terminal / worker_addr / timestamp_ms / result / error`. There is no separate unsubscribe method; you unsubscribe by closing the stream. Watch also doubles as an optional "run credential": once a task has been watched for the first time, the disconnection of the last Watch triggers `HandleWatchDetached`, which sets `WatchDisconnectDeadlineMs = now + WatchDisconnectGraceMs`; if there is still no Watch when it expires, `HandleWatchDisconnected` converges the task to `FAILED / WATCH_DISCONNECTED / retryable=false`. With `WatchDisconnectGraceMs = 0` (the code default) this is fully disabled; tasks that have never been watched are unaffected. The Worker only starts the timer after gRPC confirms the stream has disconnected, so a silent network drop must first pass through keepalive (default 30s PING + 10s ACK). Result retention: `convergeTask` writes `RetainedUntil = now + ResultRetentionMs`, and `PurgeExpiredResults` in `TickTimers` clears it on expiry. The slot is reclaimed at the moment of convergence, independent of result retention. ## Timing semantics quick reference | Name | Default | Meaning | Defined in | | ------------------------------- | ---------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------- | | slot TTL (Coordinator side) | `SlotTTLMs = 5000` | The `ttl_ms` the Coordinator passes when calling `RequestTaskSlot`; env `COORDINATOR_SLOT_TTL_MS` | `internal/coordinator/core/coordinator.go` | | slot TTL (Worker inline) | `DefaultTTLMs = 5000` | TTL used for the automatic request when `SubmitTask` has no `slot_id` | `internal/worker/core/config.go` | | task timeout default | `TaskDeadlineMs = 300000` | Effective timeout when `task_timeout_ms = 0`, counted from when the Worker accepts the submission; env `TASK_DEADLINE_MS` | `internal/worker/core/config.go` | | task timeout cap | `MaxTaskTimeoutMs = 8h` (`DefaultMaxTaskTimeoutMs`) | Exceeding it is `InvalidArgument`; the Function Code View snapshot retention window must be no shorter than this; env `MAX_TASK_TIMEOUT_MS` | `internal/worker/core/config.go`, `internal/worker/app/config.go` | | Result retention | `ResultRetentionMs = 300000` | Window during which a terminal result is queryable / can be hit idempotently; env `RESULT_RETENTION_MS` | `internal/worker/core/config.go` | | Heartbeat interval | `HeartbeatIntervalMs = 200` | How often the Worker writes to etcd; 2000 recommended in production, env `WORKER_HEARTBEAT_INTERVAL` | `internal/worker/app/config.go` | | Heartbeat timeout (Coordinator) | `HeartbeatTimeoutMs = 10000` | Beyond this the Worker is considered lost and no longer routed to | `internal/coordinator/core/coordinator.go` | | Watch disconnect grace | `WatchDisconnectGraceMs = 0` (disabled) | Window from the last Watch disconnecting to ruling `WATCH_DISCONNECTED`; 120000 recommended in production, env `WATCH_DISCONNECT_GRACE_MS` | `internal/worker/core/config.go` | | gRPC keepalive | `GRPCKeepaliveTimeMs = 30000` / `GRPCKeepaliveTimeoutMs = 10000` | Determines how long before a silent disconnect is confirmed, which affects when the Watch grace starts | `internal/worker/app/config.go` | | Timer scan period | 500ms | `TickTimers`: slot expiry, task timeout, Watch grace, result cleanup | `internal/worker/app/app.go` | | Per-call execution budget | `CallDeadlineMs = 5000` | Counts only CPU + IO backend time, excluding queueing; independent of the task timeout | `internal/worker/core/config.go` | `WorkerCore.EffectiveTaskDeadlineMs` is where these values come together: a negative `task_timeout_ms` or one exceeding `MaxTaskTimeoutMs` returns `InvalidArgument` directly, and `0` takes `TaskDeadlineMs`. ## Unsuitable scenarios BlockX's task model targets single-row state, lightweight functions, and one-shot writes; the following scenarios are not a good fit (`docs/specs/architecture.md` §5): * Computations that depend on a large number of rows in the same table: wide-range aggregations, window statistics, candlestick (K-line) charts. * Anything requiring arbitrary time-window state or large-scale stateful computation. * Strong dependence on a streaming engine's exactly-once state recovery. * Arbitrary incremental streaming computation over ordinary online tables. For streaming needs on non-onchain tables, a compromise is to have the Writer Plugin write to both the online table and Kafka, and have the business side listen to the topic and submit new tasks as needed. ## Related docs Specs in the blockx repo: * `docs/specs/architecture.md`: §2 task model, §3 main path, §4.3 results and protocols, §5 unsuitable scenarios. * `docs/specs/worker.md`: §2 external protocol, §3 local state model, §4 execution flow, §8 failure semantics. * `docs/specs/task-resource-coordinator.md`: placement, uncertain-state retries, circuit breaking. * `docs/specs/call-execution-subsystem.md`: dispatcher, Executor, subfunction calls. * `docs/specs/plugin-system.md`: declaration structures for Call Builders and Writer Plugins. Site pages: * [Architecture overview](/en/architecture/overview): overall system architecture. * [Protocols and interfaces](/en/architecture/protocols): gRPC / UDS / etcd protocol details. * [Worker](/en/components/worker): slot table, `WorkerCore` events and commands, configuration. * [Coordinator](/en/components/coordinator): the `ReserveWorkerSlot` processing chain. * [Call execution subsystem](/en/components/call-execution): inside the Calls phase. * [Plugin system](/en/components/plugins): Builder and Writer Plugin extensions. * [IO access subsystem](/en/components/io-subsystem): `TaskIOScope`, retry scopes, backend admission.