Task lifecycle

• Task-centric: shared state, read cache, and timeouts are all anchored to the task.
• Single-Worker execution: a task runs on exactly one Worker. The Worker is unaware of the Coordinator and only exposes RequestTaskSlot / SubmitTask.
• All calls within a task run in parallel: there are no ordering dependencies between calls; the dispatcher ramps them up gradually in windows.
• Fixed function version per task: on activation, a taskCodeEpoch (TaskContext.TaskCodeEpoch) is pinned; code updates during that time only affect subsequent tasks.
• Read-only functions: user functions never write to the database directly; all writes go through a Writer Plugin.
• No task-level automatic retries: the Worker only does bounded attempt retries for individual calls (CallRetryPolicy); whether to resubmit a task is decided by the Client based on TaskResult.executeResult.retryable.
• No cancellation: only task timeouts (taskTimeoutMs) and optional Watch-disconnect reclamation.

​

Task model

{
  "task_id": "3f2b1c9e-8a4d-4c1e-9b0a-6f7e5d4c3b2a",
  "functionCallConfig": {
    "type": "CallListCallConfig",
    "config": {
      "function": "hello_world",
      "callList": [["ct2"], ["blockx"]]
    }
  },
  "resultHandler": {
    "type": "ReturnValueResultHandler",
    "config": {}
  },
  "taskTimeoutMs": 30000
}

// TaskCtx carries the immutable context injected when a task is activated.
type TaskCtx struct {
	TaskInput

	Stage Stage `json:"stage"`
	DebugMode bool `json:"debugMode,omitempty"`
	Deadline time.Time          `json:"deadline"`
	Ctx      context.Context    `json:"-"`
	Cancel   context.CancelFunc `json:"-"`
}

​

End-to-end sequence

​

Slots and admission

• RequestTaskSlot(task_id, ttl_ms) only reserves; it does not execute. ttl_ms must be positive (<= 0 returns InvalidArgument) and only controls automatic reclamation of ALLOCATED slots; it is unrelated to the task timeout. TickTimers scans ExpiredSlots every 500ms; expired slots are reclaimed by HandleSlotLeaseExpired, which also clears task records still in Preparing.
• SubmitTask(task, slot_id?): when slot_id is provided it must match a record that is ALLOCATED and bound to the same taskId, otherwise InvalidSlot; when omitted, the Worker requests a slot inline once using DefaultTTLMs, returning NoSlot if there is no capacity.
• Idempotency: while the same taskId is ALLOCATED / RUNNING, RequestTaskSlot returns the same slot_id; once it has entered RUNNING or its result is still within the retention window, SubmitTask returns the stable current state and never starts another execution. After the old execution finishes and its result expires, the same taskId can be allocated a slot again.
• There is no release interface: if you obtain a slot and then abandon the submission, you can only wait for the TTL to reclaim it.

​

The three phases

• If the task ctx is cancelled while a phase is waiting for admission (for example, it has already converged due to timeout), the phase fails with CANCELLED; before entering Calls, the deadline is also checked first, and if it has already expired the task converges directly as TIMED_OUT without taking executorSem.
• After the Calls phase succeeds, the core always emits StartWriterPhase; when ResultHandler is empty, runWriterPhase still acquires pluginSem but does not execute a plugin, and pluginResults is empty.
• Streaming Builders (StreamingCallBuilder, only for BlockBundleCallConfig, StreamBuildConfig.Enabled off by default) shrink the Builder phase to PrepareStream and move the scan into the Calls phase, constrained by scanSem (ScanSlots = 8); a failure midway through production means some calls have already executed.
• Timeouts are handled by TickTimers scanning TimedOutTasks and calling HandleTaskTimedOut, which forces convergence to TIMED_OUT (retryable=true) without requiring immediate interruption of calls running in the Executor; in-flight calls receive CancelCall.

​

Results and queries

{
  "success": false,
  "executeResult": {
    "failureCode": "PLUGIN_FAILED",
    "retryable": true,
    "pluginResults": [
      {
        "pluginName": "BlockWriteResultHandler",
        "success": false,
        "failureCode": "PLUGIN_FAILED",
        "failureMsg": "..."
      }
    ]
  }
}

• The terminal state is expressed by the state field (SUCCEEDED / FAILED); the Client does not need to infer it from success. TIMED_OUT, WATCH_DISCONNECTED, and so on are failureCodes under FAILED, not separate terminal states.
• failureCodes that appear in the code: ACTIVATION_FAILED, IO_SCOPE_FAILED, SLOT_LOST, BUILDER_NOT_FOUND, BUILDER_FAILED, CALL_FAILED, OUTPUT_BYTES_EXCEEDED, PLUGIN_NOT_FOUND, PLUGIN_FAILED, CANCELLED, TIMED_OUT, WATCH_DISCONNECTED (defined in internal/worker/core/worker.go, internal/worker/adapters/orchestrator_phases.go, and internal/worker/core/dispatcher_internal.go).
• ReturnValueResultHandler puts the array of call outputs into pluginResults[i].result (JSON bytes on gRPC).
• The gRPC PluginResult has only four fields: plugin_name / success / failure_code / result; the FailureMsg and Retryable fields of the Go type are not sent on the wire (taskResultToPB).

• GetTaskResult(task_id): WorkerCore.HandleGetTaskResult. Returns RUNNING for an active task; ALLOCATED if only reserved and not yet submitted; state + result if terminal and not expired; otherwise NotFound.
• WatchTasks(task_ids) -> stream TaskUpdate: SubscriptionManager.WatchTasks. It first sends a current snapshot for each task_id (an unknown task_id gets is_terminal=true, error="not_found" so the whole batch does not fail), then pushes one terminal update for each task still running; once all tasks are terminal, the stream closes itself. TaskUpdate contains task_id / state / is_terminal / worker_addr / timestamp_ms / result / error. There is no separate unsubscribe method; you unsubscribe by closing the stream.

​

Timing semantics quick reference

​

Unsuitable scenarios

• Computations that depend on a large number of rows in the same table: wide-range aggregations, window statistics, candlestick (K-line) charts.
• Anything requiring arbitrary time-window state or large-scale stateful computation.
• Strong dependence on a streaming engine’s exactly-once state recovery.
• Arbitrary incremental streaming computation over ordinary online tables.

​

Related docs

• docs/specs/architecture.md: §2 task model, §3 main path, §4.3 results and protocols, §5 unsuitable scenarios.
• docs/specs/worker.md: §2 external protocol, §3 local state model, §4 execution flow, §8 failure semantics.
• docs/specs/task-resource-coordinator.md: placement, uncertain-state retries, circuit breaking.
• docs/specs/call-execution-subsystem.md: dispatcher, Executor, subfunction calls.
• docs/specs/plugin-system.md: declaration structures for Call Builders and Writer Plugins.

• Architecture overview: overall system architecture.
• Protocols and interfaces: gRPC / UDS / etcd protocol details.
• Worker: slot table, WorkerCore events and commands, configuration.
• Coordinator: the ReserveWorkerSlot processing chain.
• Call execution subsystem: inside the Calls phase.
• Plugin system: Builder and Writer Plugin extensions.
• IO access subsystem: TaskIOScope, retry scopes, backend admission.