# Replay runs (/api-v2/replay-runs)



Replay is a product, not a script. A run pins a baseline and a candidate execution profile and compares them over a captured **traffic manifest**. The report carries the provenance §20.5 requires and a verifiable digest (§20.7). Runs are prefixed `rpl_`. See the [replay guide](/guides/replay).

## Lifecycle [#lifecycle]

A run moves through `queued → running → completed | failed`, or `canceled` if you cancel it before it starts.

* **`routing_simulation`** runs **inline** at create time — it only compares routing policy (no inference), fully determined by the [alias resolver](/api-v2/model-aliases) over a fixed seed sweep, so it returns `completed` immediately.
* The other four classes are **scheduled onto the background runner**, which drains due runs and records the outcome. A run with a future `scheduled_for` stays `queued` until its time comes.
* If api-core restarts mid-run, the interrupted run is moved to `failed` (with a reason) rather than wedging the queue — resubmit to retry.

## Traffic manifest [#traffic-manifest]

Every class except `routing_simulation` replays an ordered list of [§20.2 trace envelopes](/guides/replay#the-trace-envelope). Supply it one of two ways:

* **Inline** — pass a `traces` array. Use this for `full_fidelity_evaluation`, where each trace carries the `messages` to re-execute (an `encrypted_full_fidelity` capture).
* **From your usage** — set `traffic_manifest_ref` to `"usage:N"` to build a manifest from your last **N** recorded usage events. These carry token shapes and timing but never prompt text, so a token-shape replay needs no new capture pipeline and leaks no PII.

A single run's manifest is capped at 5,000 traces.

All requests require a bearer API key. See [authentication](/api-reference/authentication).

## Create a replay run [#create-a-replay-run]

`POST /v2/replay-runs`

<ParamField body="baseline" type="string">
  The baseline target or profile, e.g. `openai/gpt-4o@2025-01-01` or `managed`. A name matching a live alias resolves through its weighted release; `provider/model[@revision]` is a fixed target.
</ParamField>

<ParamField body="candidate" type="string">
  The candidate to compare against, e.g. `byoc_us_east` or `anthropic/claude-3-7-sonnet`.
</ParamField>

<ParamField body="replay_class" type="string" default="routing_simulation">
  One of `routing_simulation`, `synthetic_performance`, `tokenized_performance`, `full_fidelity_evaluation`, `purge_verification`. See [classes](#replay-classes).
</ParamField>

<ParamField body="traffic_manifest_ref" type="string">
  A manifest reference. `"usage:N"` builds a manifest from your last N usage events.
</ParamField>

<ParamField body="traces" type="array">
  Inline manifest. Each entry is a trace envelope: `input_tokens`, `output_tokens`, optional `candidate_reuse_tokens`, `realized_reused_tokens`, `ttft_ms`, `latency_ms`, `namespace_generation`, and — for `full_fidelity_evaluation` — `messages` (`[{ role, content }]`).
</ParamField>

<ParamField body="repetitions" type="integer" default="1">
  Replay repetitions, clamped to 1–1000.
</ParamField>

<ParamField body="scheduled_for" type="string">
  RFC 3339 time to hold the run until. Absent means run as soon as possible.
</ParamField>

<ParamField body="concurrency" type="integer" default="1">
  The concurrency group the runner models; recorded in provenance.
</ParamField>

```bash
curl https://api.zumik.ai/v2/replay-runs \
  -H "Authorization: Bearer $ZUMIK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "baseline": "managed",
    "candidate": "byoc_us_east",
    "replay_class": "tokenized_performance",
    "traffic_manifest_ref": "usage:500"
  }'
```

```json
{
  "id": "rpl_01jy7nfg01j2k3l4m5n6o7p8qr",
  "object": "replay_run",
  "project_id": "prj_01jy7n0a4c8m2t6v9q3wrxk7bd",
  "created_at": "2026-06-15T16:18:09Z",
  "status": "queued",
  "baseline": "managed",
  "candidate": "byoc_us_east",
  "replay_class": "tokenized_performance",
  "traffic_manifest_ref": "usage:500",
  "repetitions": 1,
  "concurrency": 1,
  "scheduled_for": null,
  "started_at": null,
  "completed_at": null,
  "failure_reason": null,
  "attempt": 0,
  "manifest_size": 500,
  "metrics": null
}
```

<ResponseField name="status" type="string">
  `queued`, `running`, `completed`, `failed`, or `canceled`.
</ResponseField>

<ResponseField name="manifest_size" type="integer">
  How many trace envelopes the run will replay.
</ResponseField>

<ResponseField name="started_at / completed_at" type="string">
  RFC 3339 timestamps set as the run is claimed and finalized.
</ResponseField>

<ResponseField name="failure_reason" type="string">
  Present only on a `failed` run.
</ResponseField>

<ResponseField name="attempt" type="integer">
  Increments each time the runner claims the run.
</ResponseField>

<ResponseField name="metrics" type="object">
  Computed results once `completed`; `null` while queued/running. Shape depends on the class.
</ResponseField>

## Replay classes [#replay-classes]

| Class                      | What it does                                                                                                                                                                   | Manifest               |
| -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------- |
| `routing_simulation`       | Compares baseline vs candidate routing over a seed sweep — no inference. Runs inline.                                                                                          | none                   |
| `tokenized_performance`    | Replays exact recorded token shapes through the reuse-adjusted cost model; reports per-request cost + reuse-capture deltas with confidence intervals, plus routing divergence. | token shapes           |
| `synthetic_performance`    | Generates a structurally similar workload from the manifest's distribution and projects cost and TTFT (TTFT grounded in each trace's observed TTFT, reduced by extra reuse).   | token shapes           |
| `full_fidelity_evaluation` | Re-executes recorded turns through the broker for both targets and measures real latency, output-token rate, and output divergence. Budget-gated real spend.                   | `messages`             |
| `purge_verification`       | Confirms artifacts from a purged namespace generation can no longer be reused.                                                                                                 | `namespace_generation` |

## List replay runs [#list-replay-runs]

`GET /v2/replay-runs`

Returns `{ "object": "list", "data": [...] }`, newest first. Each entry is a run object (without the manifest body).

## Cancel a replay run [#cancel-a-replay-run]

`POST /v2/replay-runs/{replay_run_id}/cancel`

Cancels a still-`queued` run. A run that is already running or finished returns `400`.

## Retrieve a replay run [#retrieve-a-replay-run]

`GET /v2/replay-runs/{replay_run_id}`

Returns the run object. Poll it (or list) to watch a scheduled run progress to `completed`.

## Render a signed report [#render-a-signed-report]

`GET /v2/replay-runs/{replay_run_id}/report`

The self-describing report (§20.5/§20.7): the full provenance block, a traffic-manifest summary, the computed metrics (or a pending note before completion), stated assumptions and known limitations, the recommended profile, and a verifiable `evidence_digest`.

```json
{
  "object": "replay_report",
  "replay_run_id": "rpl_01jy7nfg01j2k3l4m5n6o7p8qr",
  "generated_at": "2026-06-15T16:19:42Z",
  "status": "completed",
  "baseline": "managed",
  "candidate": "byoc_us_east",
  "replay_class": "tokenized_performance",
  "provenance": {
    "trace_schema_version": "2026-06-01",
    "replay_runner_version": "rr_1.0.0",
    "runtime_engine_version": "api-core/0.1.0",
    "model_alias_release": { "baseline": null, "candidate": null },
    "resolved_model_revision": { "baseline": "unpinned", "candidate": "unpinned" },
    "prompt_compiler_revision": "pc_11",
    "tokenizer_revision": "tok_7",
    "cache_mode": "provider_default",
    "warmup_period_s": 0,
    "cold_start_period_s": 0,
    "request_arrival_schedule": "as_fast_as_possible",
    "concurrency": 1,
    "retry_policy": "none",
    "provider_rate_limits": "provider_default",
    "repetitions": 1,
    "confidence_intervals": "95% normal-approximation on per-request samples; p50/p95/p99 reported",
    "quality_evaluator_version": "qe_none",
    "failures_and_dropped": { "failures": 0, "dropped": 0 }
  },
  "traffic_manifest": {
    "ref": "usage:500",
    "traces": 500,
    "total_input_tokens": 8400000,
    "total_output_tokens": 210000,
    "total_realized_reuse_tokens": 3100000,
    "traces_with_full_fidelity_payload": 0
  },
  "metrics": {
    "metric_deltas": {
      "provider_cost_micros": { "baseline": 22100000, "candidate": 22100000, "delta": 0, "pct": 0 },
      "reuse_capture_pct": { "baseline": 36.9, "candidate": 36.9 }
    },
    "confidence_intervals": { "per_request_cost_savings_micros": { "n": 500, "mean": 0, "p50": 0, "p95": 0, "p99": 0, "ci95_low": 0, "ci95_high": 0 } },
    "recommended_profile": "baseline",
    "failures": 0,
    "dropped": 0
  },
  "assumptions": ["..."],
  "quality_guardrails": "token-shape replay does not execute inference; output quality is unchanged from the recorded run.",
  "known_limitations": ["..."],
  "recommended_profile": "baseline",
  "evidence_digest": "sig_b71e...4d"
}
```

<ResponseField name="provenance" type="object">
  Every §20.5 field: schema/runner/engine versions, alias releases, resolved model revisions, prompt-compiler and tokenizer revisions, cache mode, warmup and cold-start periods, arrival schedule, concurrency, retry policy, provider rate limits, repetitions, confidence-interval policy, quality-evaluator version, and the failure/dropped counts.
</ResponseField>

<ResponseField name="traffic_manifest" type="object">
  A summary of the replayed traffic: trace count, total input/output/reuse tokens, and how many traces carried a full-fidelity payload.
</ResponseField>

<ResponseField name="metrics" type="object">
  Class-specific comparison: metric deltas, confidence intervals, and `failures` / `dropped`.
</ResponseField>

<ResponseField name="quality_guardrails" type="string">
  What the class does and does not guarantee about output quality.
</ResponseField>

<ResponseField name="recommended_profile" type="string">
  The profile the report recommends (`baseline`, `candidate`, `either`, or `n/a`).
</ResponseField>

<ResponseField name="evidence_digest" type="string">
  Pinned at completion over provenance, the manifest summary, and metrics. With a signing key it is a keyed HMAC-SHA256 (`sig_…`) only Zumik can produce — tamper-evident; without one it is an unkeyed checksum (`sha256_…`).
</ResponseField>

## Errors [#errors]

| Status | Code                    | When                                                                                                                                                               |
| ------ | ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| 400    | `invalid_request_error` | `baseline`/`candidate` empty, a class that needs a manifest got none, an unsupported `traffic_manifest_ref`, a bad `scheduled_for`, or canceling a non-queued run. |
| 401    | `invalid_api_key`       | Missing or invalid API key.                                                                                                                                        |
| 404    | `invalid_request_error` | The run does not exist in this project.                                                                                                                            |

See the full table on [errors](/api-reference/errors).