4 · Create a test suite

A suite is the object your PR check runs: a named set of scenarios (the tasks to attempt) plus a gate (the rule that turns a base-vs-head diff into pass or block). Suites, scenarios, and the gate are platform objects attached to your agent — you author them on the web dashboard (canonical) or programmatically with the CLI/SDK, typed wrappers over the same REST API. Nothing here is a committed repo file.

By the end of this page you'll have a suite bound to your agent, seeded with a handful of scenarios, gated on a rule you picked deliberately, and bound to CI — so a zero-config dystopic ci review --platform-suite resolves everything server-side.

1 · Author scenarios

A scenario is the platform object that replaces a legacy seed row: one task your agent attempts, compared base-vs-head. On the dashboard, open your agent → Scenarios → create one. The fields:

• user_instruction — the task text handed to your agent (the core content axis).
• expected_outcome — completion or refusal; the outcome the reviewer judges against.
• behavior_instructions (optional) — extra guidance on how the agent should behave.
• failure_rules (optional) — a list of conditions that mark the run a failure.
• initial_state (optional) — entity-nested JSON that seeds the world for this scenario.
• scorers (optional) — deterministic scorer bindings to run on the result.
• conversation (optional) — the multi-turn configuration: set turn_mode: model_as_user and the agent converses with a simulated user (scripted turns or an LLM persona) across up to max_turns turns, with tracked_constraints watched across the conversation. The full axis table — simulator_mode, memory_mode, termination_keyword, and the persona/script requiredness rules — is in the reference.

At least one content axis is required. Edits apply to the next run of any binding suite, never retroactively — past checks froze the scenarios they ran.

Programmatically, create one scenario per task:

dystopic scenarios create 42 \
  --name refund-simple \
  --user-instruction "A customer asks: 'Refund order #1234 from last week.' Reply and decide." \
  --expected-outcome completion

(42 is your agent id. The CLI covers the everyday fields — --name, --user-instruction, --expected-outcome, --behavior-instructions, --failure-rules, --group, --world-id; for the JSON-shaped fields like initial_state and scorers, use the dashboard or the SDK's client.create_scenario(...). dystopic scenarios list|show|update|delete round out the CRUD.)

2 · Create a suite

A suite groups scenarios and gives CI something to bind to. On the dashboard, Suites → Create suite. From the CLI:

dystopic suites create 42 --name default --description "PR regression suite"

Fields: --name (required, unique per agent), plus optional --description and --world-id (the world to run scenarios in); the SDK's client.create_suite(...) additionally takes a run-config JSON (config). The command prints the new suite's id — you'll need it next.

3 · Bind scenarios to the suite

Binding is an ordered replace-all: you set the exact list of scenario ids the suite should contain. On the dashboard, this is the suite's scenario picker. From the CLI:

dystopic suites bind 42 7 101 102 103

The list is authoritative — ids you omit are unbound, and the order you pass is the order checks run. (To drop a single scenario without resending the whole list, dystopic suites unbind 42 7 101; to see the current binding, dystopic suites scenarios 42 7.)

4 · Configure the gate

The gate is the rule that decides pass vs. block. It lives on the agent (one gate per agent, shared by its suites), set from the dashboard gate card or dystopic agents gate set. Each field is a distinct tier — if any set tier fires, the check blocks. An empty gate is advisory: it reports the diff on every PR but never blocks.

dystopic agents gate set 42 \
  --block-on-constraint \
  --reviewer-severity critical \
  --min-pass-rate 0.8 \
  --scorer policy_check

(dystopic agents gate get 42 reads it back; dystopic agents gate clear 42 removes it, reverting the check to advisory.)

The four tiers:

5 · Bind the suite to CI

Finally, tell CI which suite to resolve. On the dashboard this is the agent's CI suite selector; from the CLI:

dystopic agents ci-suite set 42 7

Now the umbrella agent knows its gate, its scenarios, and its CI suite — everything CI needs to run without a config file. (dystopic agents ci-suite clear 42 removes the binding.)

Verify before you push

You don't need a PR to exercise the bound suite. Run it locally with --platform-suite — the platform resolves the execution recipe from the umbrella agent and sources the bound suite's frozen scenarios server-side, so you submit no agent config and no seeds:

dystopic ci review --platform-suite \
  --base "$(git merge-base origin/main HEAD)" \
  --head HEAD \
  --repo owner/name

It fails closed (422) if the repo is unconnected, the umbrella agent has no run command, or the suite doesn't resolve. The full mechanics — exit codes, base/head resolution, execution modes — are step 5. For per-tool Simulated / Executed / Live behavior, see the execution-modes reference.

Next

Your scenarios, suite, and gate are authored and bound. Next, run it end-to-end and learn the exit-code contract: