Skip to main content
A task starts at the Client, is placed by a Coordinator, and runs on one Worker; its calls land in that Worker’s Python Executor child processes. This page explains every term on that path; implementation details live on the component pages.

​
Processes

​
The path of one submission

1

Placement

The Client calls ReserveWorkerSlot(task_id) on any Coordinator. The Coordinator picks a candidate from the Worker view it rebuilds from etcd heartbeats, calls RequestTaskSlot(task_id, ttl_ms) on it, and hands worker_addr + slot_id back to the Client.
2

Submission

The Client connects directly to worker_addr and calls SubmitTask(task, slot_id). The Worker validates the slot, creates the task context, pins the code snapshot, and returns RUNNING. Without slot_id, the Worker requests a slot inline.
3

Fetching the result

The Client calls WatchTasks([task_id]) on the same Worker to subscribe to the terminal state, or polls with GetTaskResult(task_id). The Coordinator is no longer involved.
blockx-py never connects to an address directly: all RPCs are forwarded through the UDS proxy specified by PROXY_SOCKET_PATH. The proxy routes ReserveWorkerSlot to the block or bundle Coordinator by gRPC method path, and forwards WorkerService requests to the corresponding Worker by the x-blockx-worker-addr header. For local development, examples/local_test_service/local_proxy.py stands in for this hop.

​
Slot

A slot is a task concurrency quota on a Worker:
  • Each Worker has a fixed number of slots (TASK_SLOTS, default 50); one task occupies one slot.
  • RequestTaskSlot switches an idle slot to ALLOCATED and binds it to task_id; if no SubmitTask arrives within the TTL (SlotTTLMs = 5000 on the Coordinator side), the slot is reclaimed automatically.
  • Once SubmitTask passes validation, the slot enters RUNNING; the slot is released the moment the task converges to a terminal state, independent of the result retention window.
  • There is no centralized queue. When no slot is available, the Worker returns NoSlot (gRPC ResourceExhausted) and the caller backs off and retries; blockx-py keeps backing off until it gets a slot.
  • The Worker’s local slot table is the single source of truth. The Coordinator only holds a discardable derived view; when multiple Coordinators pick the same Worker at the same time, the conflict converges at the Worker’s atomic capacity check.

​
Inside the Worker

Between the Executor and the Worker is one long-lived UDS connection: the Worker sends ExecuteCall, the Executor replies CallWaiting when the user function issues IO or a sub-call, the Worker handles it and sends ResumeCall, and finally the Executor ends with CallCompleted / CallFailed. The Executor never connects to any external service directly.

​
Two clusters

The same code starts as two mutually invisible clusters with different app.Profile values: The task protocol, slot state machine, and heartbeat are identical in both clusters. blockx-py picks the cluster automatically from the config and handler: BlockTableCallConfig goes to block, BlockBundleCallConfig goes to bundle, anything with ReturnValueHandler always goes to block, and InputsCallConfig goes to bundle by default.

​
Sync Invoker

The Sync Invoker is the second entry point besides tasks: one gRPC unary Invoke executes a function once and returns the result, call_id, and attributed execution durations on the spot. It reuses the Worker’s Executor pool, UDS protocol, Function Code View, and read-only IO subsystem, but does not go through the Coordinator, has no slots, Builders, or Writer Plugins, and rejects all write requests. blockx-py’s function.call(...) goes through it when running outside an Executor.

​
Timing parameters quick reference

See “Timing semantics quick reference” in Task lifecycle for the full table.

​
Summary

  • The Coordinator only does placement and slot reservation; it is stateless and can run as multiple instances. The Worker is the single source of truth on the execution plane.
  • A slot is a task concurrency quota on a Worker; there is no centralized queue, so back off and retry when none is available.
  • Calls run in the Python Executor child processes on the Worker’s own host, and all IO goes back to the Worker’s IO subsystem over UDS.
  • block and bundle are two clusters built from the same code with different profiles; the Sync Invoker is a synchronous entry point that bypasses the task flow.
Continue reading: