Runtime gotchas
The runtime footguns that bite a ported agent under load — subprocess env wiring, proxy retries and terminal errors, sub-agent actor-id attribution, empty and malformed responses, and unschema'd tools.
The dispatch contract and code mode tell you how to wire an agent up. This page is the other half: the handful of behaviors that are correct at low volume and wrong under concurrency, plus the failure modes the SDK swallows quietly so you never see them until a run comes back graded on nothing.
Subprocesses need the per-run env — use make_subprocess_env
The sandbox injects the DYSTOPIC_* vars (DYSTOPIC_ODYSSEY_PROXY_URL, DYSTOPIC_RUN_TOKEN, DYSTOPIC_API_URL, and the optional DYSTOPIC_RUN_TOKEN_JTI / DYSTOPIC_AGENT_ID / DYSTOPIC_TASK_ID / DYSTOPIC_RUN_ID) into your entrypoint's process environment, so a subprocess that inherits the environment (the MCP shim, a shelled-out tool) picks up the correct per-run token automatically.
The footgun is a hand-built child environment: pass env= to Popen without the per-run vars and the child signs proxy calls with nothing (or with stale values you copied earlier). Build child environments with make_subprocess_env:
import subprocess
from dystopic.odyssey import Envelope, make_subprocess_env
def run(task_input, *, proxy_url, run_token):
env = make_subprocess_env(Envelope.from_env(), extra={"DEBUG": "1"})
proc = subprocess.Popen([...], env=env)
...make_subprocess_env(envelope, *, extra=None) returns a fresh dict: the current os.environ, the per-run DYSTOPIC_* vars overriding any that collide, then your extra (extras win over DYSTOPIC_*). Nothing is written back to the process environment.
If your agent makes tool calls purely through the SDK (proxy_call / async_proxy_call / ScopedProxy) and never shells out, the env vars are irrelevant to you — those calls read the envelope off the ContextVar, not the environment. This only matters when a subprocess needs the per-run token.
Proxy-call retries and terminal statuses
proxy_call and async_proxy_call (and their *_with explicit-envelope forms, and ScopedProxy.call / .acall) retry a small, deliberately narrow set of transient platform conditions. Retries are on by default; nothing else is retried.
What retries:
- HTTP
429(rate-limited). - HTTP
503whoseerror_classis one oflock_contention(a sibling parallel tool call holds the per-run lock),context_store_unavailable(backing store blip), ortrace_sequence_contended(sequence-allocation race).
What is terminal — raised immediately, never retried: 401, 404, 400/422, 500, and any 503 whose error_class is not in the retryable set. Transport failures (DNS, connection refused, timeout) raise a ProxyCallError with status_code=None and are not retried either.
The budget is up to 4 total attempts (3 retries) with exponential backoff plus ±25% jitter, base 0.5s and capped at 4.0s — roughly 0.5s → 1s → 2s. All of this is per-call; there is no cross-call retry state.
You can override both knobs per call:
# Latency-sensitive call: fail fast instead of backing off.
order = proxy_call("get_order", {"order_id": "4521"}, auto_retry=False)
# Long-running tool: override the default timeout (seconds).
report = proxy_call("build_report", {"scope": "quarter"}, timeout=30.0)The default timeout is read once at import from DYSTOPIC_PROXY_TIMEOUT_S (an unparseable value warns and falls back), otherwise 120.0 seconds. Passing timeout= overrides it for that call.
Branch on error_class, not error strings
Every failure raises ProxyCallError, which carries status_code, body, tool_name, and error_class (pulled from body["detail"]["error_class"]). Branch on the discriminator — it's stable across releases; the English message is not:
from dystopic.odyssey import ProxyCallError, is_stale_run_token, proxy_call
try:
result = proxy_call("issue_refund", {"order_id": "4521"})
except ProxyCallError as exc:
if is_stale_run_token(exc):
# 401 + error_class in {run_token_expired, run_token_invalid, run_token_rejected}
# The run has closed — no point retrying, and no point returning a
# tool result the model will keep reasoning over.
raise
# Otherwise, hand the model something to reason about rather than crashing:
return {"error": exc.error_class or "tool_call_failed"}is_stale_run_token(exc) is a helper for exactly the "the run is no longer in-flight" case (it's a 401 with a run_token_* error_class, surfaced as the StaleRunTokenError subclass). Swallowing a ProxyCallError into an {"error": ...} dict is a legitimate choice — the simulated tool failed, and returning a structured error lets the agent recover or explain the failure in its final_response.
Actor-id attribution: ScopedProxy vs. ContextVar
For a single-agent system, ignore this entirely — no X-Pipelines-Actor-Id header is ever stamped and the wire shape is byte-identical to the pre-multi-agent path. It matters only when your agent delegates to declared sub-agents and you need each proxy call attributed to the sub-agent that made it.
There are two ways to bind the acting label, and they are not equally safe under concurrency.
The ambient ContextVar path (set_actor_id / dystopic_proxy) is not attribution-safe for concurrent sub-agents. proxy_call reads the acting label off a ContextVar (current_actor_id()) at the moment the HTTP request fires. That works only if the framework keeps the ContextVar intact from the bind site to the call site. Task boundaries — asyncio.gather, asyncio.create_task — break that propagation, so a shared decorated tool can inherit the wrong task-local label: because dystopic_proxy reads attribution from the ambient ContextVar when the proxy request is made, shared decorated tools can inherit the wrong task-local actor label under concurrent sub-agent execution.
ScopedProxy (via envelope.for_actor(...)) is safe for concurrent sub-agents. It captures the label at construction time and binds it (via set_actor_id) in the same call frame as the proxy hop — there's no task boundary between the bind and the request, so attribution never depends on framework context propagation:
# In the agent factory, build each sub-agent's tools from ITS OWN handle.
refund = envelope.for_actor("refund_agent")
async def issue_refund(order_id: str) -> dict:
return await refund.acall("issue_refund", {"order_id": order_id})For a shared tool that two sub-agents both call, build one copy per sub-agent with ScopedProxy.tool(...) so each carries its owner's label:
refund_search = envelope.for_actor("refund_agent").tool("search_docs", is_async=True)
billing_search = envelope.for_actor("billing_agent").tool("search_docs", is_async=True)
# refund_search(...) stamps actor=refund_agent; billing_search(...) stamps
# actor=billing_agent — same shared tool, distinct attribution.Share the tool logic, never the handle. Reusing one for_actor(...) handle (or a single bound callable) across two sub-agents stamps both with the same label and silently corrupts attribution — no error is raised. Guard against it with verify_actor_tools("billing_agent", billing_tools), which returns the list of (tool_name, bound_actor_id) mismatches (empty when correct) and warns on each; wrap it in an assert not verify_actor_tools(...) if you want to fail the factory hard.
The label must be declared, and must be shaped
envelope.for_actor(actor_id) validates the label against the same caps the proxy enforces, so a malformed label fails in your code rather than as an opaque proxy 400 mid-run. The rules: a /-delimited call path, each segment drawn from [a-zA-Z0-9_.:-], at most 64 chars per segment, at most 8 segments deep, at most 256 chars total. An empty or whitespace-only label raises ActorLabelError — a scoped handle for "no sub-agent" is meaningless; use the ambient proxy_call path for single-agent calls.
Separately, for_actor checks the label's leaf segment against the run's declared sub-agent catalog (envelope.declared_actor_ids, stamped by the platform into the dispatch body). An off-catalog leaf — almost always a typo of a declared label — warns but does not raise: the call still ships and the backend reconstructs it as an undeclared node. The warning kills the silence of the footgun without dropping the call:
for_actor('refund_agnt'): sub-agent leaf 'refund_agnt' is not in the run's
declared catalog ['billing_agent', 'refund_agent']. The call will still be
attributed to this label (it will surface as an `undeclared` node), but this
is usually a typo of a declared label — declare it in the agent's sub_agents
or fix the spelling.When no catalog is carried (older platform, single-agent agent, or a run whose context arrived without one via the DYSTOPIC_* env vars), the catalog check is skipped and validation is shape-only.
Empty and malformed responses
Empty final_response is a failure, not a pass
The response contract bottoms out on a non-empty string for final_response. If your entrypoint returns "" (or a dict whose final_response is empty), build_response raises ResponseShapeError and the run fails. The platform rejects it because the LLM judge has nothing to grade against — an honest failure beats silently getting graded on an empty string.
# WRONG — returns "" on the failure path → ResponseShapeError
def run(task_input, *, proxy_url, run_token):
result = agent.run(task_input.get("user_instruction") or "")
return result.output # None/"" when the agent bailedThe fix is to always return at least a one-sentence summary — including on the failure path:
return result.output or "Agent produced no output; the task could not be completed."build_response accepts three shapes: a bare str, a dict containing final_response (with optional messages / metadata), or a result object exposing a final_response / final_output / output / message attribute (covers the common OpenAI Agents / Anthropic / Strands result types). All three funnel through the same non-empty check.
Malformed messages / metadata are dropped silently
The only keys the SDK forwards from a returned dict are final_response, messages, and metadata (other keys pass through untouched for your own introspection; the platform ignores them). messages and metadata are for rich trace rendering and are optional.
If your messages or metadata is unparseable platform-side (e.g. a role that isn't a canonical type), the platform drops it to null, records a soft_warnings entry, and still grades the run on final_response alone. No exception is raised on your side — the drop only shows up in the run's metadata, and the trace falls back to the thin final-response view.
Because a malformed rich transcript costs you trace fidelity but never fails the run loudly, the defensive move when you're unsure is to return only final_response rather than shipping a messages array you haven't validated. Add messages/metadata once you've confirmed they render.
Output-schema safety
A tool declared without an output_schema is a latent footgun in a simulated world. When the LLM-backed simulator generates the tool's response, nothing pins its shape — the result may not match the structure your agent code expects. Code that indexes structurally (result["orders"][0]["id"]) can KeyError/IndexError on a perfectly "successful" simulated call.
Two defenses, ideally both:
Declare an output_schema. With a schema, the simulator validates its generated response against it and regenerates on a violation, so the payload your tool returns conforms to the shape you rely on:
from dystopic.odyssey.tools import Tool, simulated
get_order = Tool(
name="get_order",
input_schema={
"type": "object",
"properties": {"order_id": {"type": "string"}},
"required": ["order_id"],
},
output_schema={
"type": "object",
"properties": {
"id": {"type": "string"},
"status": {"type": "string"},
},
"required": ["id", "status"],
},
mode=simulated(),
)Parse defensively anyway. Use .get() and type checks rather than structural indexing, so an unexpected shape degrades to a handled branch instead of an exception:
order = proxy_call("get_order", {"order_id": order_id})
status = order.get("status") if isinstance(order, dict) else None
if status is None:
return {"error": "get_order returned no status"}The two combine cleanly: the schema tightens what the simulator emits, and defensive parsing catches the residual cases (a schema-less tool you don't own, a legacy passthrough, a genuinely partial payload).
output_schema lives on the tools_schema entry (see the dispatch contract and registering an agent). Declare it with the Tool(...) builder and let the SDK serialize it, or hand-author it in the dashboard's agent form.
Where to go next
- The dispatch contract — the run context you receive and the response you return.
- Code mode — the
run(task_input, *, proxy_url, run_token)entrypoint and the asyncio-in-sandbox footgun. - Wire contract & errors — the exhaustive low-level reference these gotchas draw on.
Code mode: upload source
Upload Python source instead of hosting a server — the run(task_input, *, proxy_url, run_token) entrypoint, Envelope.from_env(), the 50-file / 200KB / 1MB source caps, the asyncio-in-sandbox footgun, and why every code-config update re-ships the whole tree.
Register the agent
The full mechanics of agent registration — the SDK helpers and raw /api/agents endpoints, the draft→active lifecycle, the per-mode config shapes on the wire, the merge-vs-replace rules for updates, and the minimal manifest for the secondary dystopic agents push path.