# Docs


- **Get started**
- [Introduction](/): Zumik is the inference control plane for persistent AI agents. Measure repeated work, keep reusable state, and route to the cheapest reliable execution path.
- [Quickstart](/quickstart): Point an OpenAI client at Zumik, create a reusable artifact, and run a workload diagnostic in five minutes.
- [Authentication](/authentication): API keys, bearer auth, per-key budgets, console sessions, and how to rotate or revoke a key.
- [OpenAI compatibility](/openai-compatibility): The strict /v1 compatibility contract, exact request and response shapes, no proprietary JSON fields, and proprietary signal carried on Agent-* headers.
- [Glossary](/glossary): Short definitions for every term Zumik uses, each linked to the concept page that covers it in depth.
- [Changelog](/changelog): Notable changes to the Zumik platform, newest first.

- **Concepts & guides**
- Core concepts
  - [The inference control plane](/concepts/overview): Why persistent agents pay repeatedly for the same context, and how Zumik measures reuse, preserves state, routes cheaply, reproduces routing, and proves deletion.
  - [API surfaces](/concepts/api-surfaces): Strict OpenAI compatibility on /v1, native state on /v2, and the one rule that keeps them apart - proprietary behavior travels through headers or /v2, never by changing upstream JSON.
  - [Artifacts](/concepts/artifacts): The immutable, reusable content unit that unifies contexts, tools, and schemas under one object type, with strict immutability invariants.
  - [Bundles](/concepts/bundles): An ordered, immutable list of artifact references where the order carries meaning and is never sorted automatically.
  - [Sessions](/concepts/sessions): A causal state container for an agent workflow - not a cache entry. How a session anchors bundles, branches, and the events that accumulate over a task.
  - [Branches](/concepts/branches): Append-only event lines within a session, guarded by optimistic concurrency, with merges that are always explicit and never automatic.
  - [Snapshots](/concepts/snapshots): A compiled, ordered logical state pinned to a branch head - the unit a response binds to, sitting at Layer 1 of the identity model.
  - [The three identity layers](/concepts/identity-model): Logical identity, materialization identity, and KV-realization compatibility - why the same logical state can need different physical caches, and why cache details never reach the public API.
  - [Handles and fingerprints](/concepts/handles-and-fingerprints): Opaque public IDs that callers hold, versus internal tenant-scoped HMAC fingerprints that never leave the isolation boundary - and why raw content hashes are never exposed.
  - [Reuse metrics](/concepts/reuse-metrics): Opportunity versus realized reuse, the metrics that define each, and the five evidence levels that keep a prediction from being mistaken for a measured fact.
  - [Workload Reuse Score](/concepts/workload-reuse-score): The WRS formula, its six weighted components, the interpretation bands, and the deliberately separate deployment-readiness score.
  - [Model aliases](/concepts/model-aliases): Request a capability instead of a model string. Immutable alias releases, deterministic resolution, per-request resolution records, and no silent drift.
  - [Agent Hints](/concepts/agent-hints): A vendor-neutral, versioned contract for expressing intent - reuse, QoS, routing, retry safety - without exposing any provider- or engine-specific knob.
  - [Quality of service](/concepts/qos): QoS classes, the request you submit, and the formal outcome object that makes the platform accountable for whether latency and reliability targets were actually met.
  - [Execution profiles](/concepts/execution-profiles): Managed-provider default, BYOK, BYOC, hybrid, and OpenRouter emergency fallback - what each is for and who owns the control plane inside it.
  - [Capability manifests](/concepts/capability-manifests): Versioned, per-provider records of exactly what each provider and profile supports - the source of truth that routing decisions and purge claims both read.
  - [Retention and purge](/concepts/retention-and-purge): Delete revokes access; purge removes retained representations and returns a signed receipt. Guarantee classes, purge generations, and how resurrection is prevented.
- Guides
  - [OpenAI migration guide](/guides/openai-migration): Point an existing OpenAI client at Zumik with no body changes - the full /v1 compatibility surface, the base-url swap, what carries over unchanged, the optional Agent-* headers, and a testing checklist.
  - [Prompt caching](/guides/prompt-caching): Capture provider-native prompt caching through Zumik across OpenAI, Anthropic, Gemini, xAI, and Fireworks - and measure how much you actually reused.
  - [Prompt layout](/guides/prompt-layout): The recommended block ordering that preserves a reusable prefix, the reuse-killers that quietly defeat provider caching, and the zumik lint CLI and web prompt-linter that catch them.
  - [Sessions and branching](/guides/sessions-and-branching): Create a session, append events with optimistic concurrency, handle the 409 branch_version_conflict, fork branches, and merge explicitly with the four supported strategies.
  - [Snapshots and reproducibility](/guides/snapshots-and-reproducibility): Pin a snapshot to freeze a branch head, ordering, and prompt-compiler revision, understand exactly what a snapshot fixes, and replay recorded traffic against a pinned snapshot and alias release.
  - [Streaming](/guides/streaming): Server-sent events on /v1/chat/completions and /v1/responses, the chunk shape, stream_options.include_usage, and how the token-by-token relay bills exactly once from real usage.
  - [Idempotency and retries](/guides/idempotency-and-retries): Make mutating requests safely retryable with Agent-Idempotency-Key, understand the three retry types, and keep tool side effects safe when a request is replayed.
  - [Workload diagnostics](/guides/workload-diagnostics): Capture metadata traces, run the Agent Workload Efficiency Diagnostic, read the Workload Reuse Score and reuse waterfall, and get a recommended execution profile - before you change any infrastructure.
  - [Replay](/guides/replay): The trace envelope and its privacy modes, the five replay classes, the metrics a replay compares, and the signed report with provenance and an evidence digest - the system that justifies (or rejects) a BYOC migration.
  - [BYOK setup](/guides/byok-setup): Attach your own provider keys for all five providers, route eligible traffic through them, rotate without downtime, and bill against your own provider relationship.
  - [Subscription credentials](/guides/subscriptions): Attach a Claude Code or ChatGPT Codex subscription so eligible traffic runs against its bundled allowance at cache-discounted rates.
  - [QoS outcomes](/guides/qos-outcomes): Declare a QoS class and targets, read the formal qos_outcome with its admission, completion, and reason codes, and carry the same intent on OpenAI-compatible traffic through response headers and Agent Hints.
  - [Billing and budgets](/guides/billing-budgets): The pricing tiers, prepaid credits, hard monthly caps, 50/80/100% alerts, per-API-key budgets, and opt-in overage - plus how to read your account and what each error means.
  - [Purge semantics](/guides/purge-semantics): The difference between delete and purge, the five guarantee classes, the signed receipt, namespace-generation invalidation, and why a purged artifact can never be resurrected from a stale cache.
  - [Regional policy](/guides/regional-policy): Set data residency and allowed regions for a project, and understand how the Execution Broker enforces them by returning region_not_allowed before any provider call.
  - [Rate limits](/guides/rate-limits): The three layers of rate limiting - Cloudflare edge, Bifrost per-key, and API Core per-project - the 429 with Retry-After, and the stricter limits on auth endpoints.
- Security & compliance
  - [Security overview](/security/overview): Security-by-default at Zumik, tenant isolation, secret hygiene, encryption, and honest retention and purge, treated as baseline rather than a premium add-on.
  - [Tenant isolation](/security/tenant-isolation): How Zumik namespaces every object by tenant, uses opaque IDs and HMAC fingerprints, and verifies isolation with destructive cross-tenant tests against BOLA and IDOR.
  - [Data privacy and retention](/security/data-privacy-and-retention): Metadata-only tracing by default, the trace privacy modes, retention profiles, encryption in transit and at rest, and no raw prompt logs.
  - [GDPR and CCPA](/security/gdpr-ccpa): Data residency, the right to access, erasure, portability, and rectification, consent, the DPA and sub-processor list, and CCPA Do-Not-Sell and Global Privacy Control handling.
  - [Vulnerability checklist](/security/vulnerability-checklist): The vulnerability classes Zumik continuously verifies as absent across every service, deployment, and dependency, as a reference for security reviews, pen-test scope, and automated scanning.
- Pricing
  - [Plans](/pricing/plans): Pay-as-you-go prepaid credits, the control-plane fee that applies on every execution path, the managed-optimization pilot, and BYOC and enterprise contracts.
  - [Credits and budgets](/pricing/credits-and-budgets): Credits, project hard caps, 50/80/100% alert thresholds, per-API-key budgets, and opt-in overage.
  - [Reuse credit](/pricing/reuse-credit): How realized reuse becomes a visible credit on your bill, gross input charge minus reuse credit equals processed input charge.

- **API reference**
- Overview
  - [API reference](/api-reference/introduction): Base URLs, the /v1 and /v2 split, versioning, content types, and how Zumik's proprietary signal rides on Agent-* headers instead of the response body.
  - [Authentication](/api-reference/authentication): Bearer API keys (zk_live_...), per-key spending budgets, and the 401 error shape Zumik returns for a missing or invalid key.
  - [Errors](/api-reference/errors): The OpenAI-compatible error envelope and the full Zumik error-code table, with HTTP status, meaning, and how to fix each one.
  - [Idempotency](/api-reference/idempotency): Use Agent-Idempotency-Key to make retried POST requests safe, the agent-idempotent-replay response header, and the replay and in-flight-conflict semantics.
  - [Headers](/api-reference/headers): Every Agent-* request header Zumik reads and every response header it emits, including alias release, resolved provider, execution profile, trace id, region, and QoS signal.
  - [Pagination](/api-reference/pagination): How Zumik list endpoints return results. Lists are returned in full as a single page; the list envelope carries has_more for forward compatibility.
- v1 · OpenAI-compatible
  - [Responses](/api-v1/responses): The OpenAI-compatible Responses API on Zumik. Create responses with function tools and streaming, retrieve, delete, and cancel them, list input items, compact context, and count input tokens.
  - [Chat completions](/api-v1/chat-completions): POST /v1/chat/completions. The exact OpenAI chat-completions shape, including streaming over Server-Sent Events and stream_options.include_usage.
  - [Embeddings](/api-v1/embeddings): POST /v1/embeddings. The OpenAI-compatible embeddings shape, taking a single string or an array of strings and returning a data list of vectors.
  - [Models](/api-v1/models): GET /v1/models and GET /v1/models/{id}. The exact OpenAI model-list shape, covering Zumik aliases and the underlying provider models.
  - [Conversations](/api-v1/conversations): POST, GET, and DELETE /v1/conversations. OpenAI-compatible conversation containers that group related items and responses.
  - [Batches](/api-v1/batches): POST and GET /v1/batches and /cancel. The OpenAI-compatible Batch API for running many chat completions asynchronously from an uploaded input file.
  - [Files](/api-v1/files): POST /v1/files (multipart upload) and GET /v1/files/{id}/content. The OpenAI-compatible file store that Batch jobs read from and write to.
- v2 · Native state

  - **Native state**
  - [Artifacts](/api-v2/artifacts): Create, retrieve, and delete reusable immutable content units. Artifacts are the addressable building blocks of agent state on the native /v2 surface.
  - [Bundles](/api-v2/bundles): Create, retrieve, and delete immutable ordered sets of artifact references. Bundles assemble artifacts into a reusable agent prefix.
  - [Sessions](/api-v2/sessions): Create, retrieve, and delete causal state containers for agent workflows. A session holds a default branch and an append-only event history.
  - [Branches and events](/api-v2/branches): Fork session branches and append events with optimistic concurrency. Compare-and-swap on branch version and head event id prevents silent history rewrites.
  - [Snapshots](/api-v2/snapshots): Compile and retrieve the ordered logical state for a branch head. A snapshot pins what a model sees so a response can be replayed against a stable definition.
  - [Responses](/api-v2/responses): Create, retrieve, and cancel native responses. The /v2 surface pins session state and an explicit QoS request in the body, and reports the formal QoS outcome.
  - [Token counts](/api-v2/token-counts): Count the input tokens for a piece of text before you send it. A native /v2 helper for budgeting requests.

  - **Diagnostics & governance**
  - [Diagnostics](/api-v2/diagnostics): Run the Agent Workload Efficiency Diagnostic over metadata traces, retrieve a stored run, and render a signed report with an evidence digest.
  - [Replay runs](/api-v2/replay-runs): Pin a baseline and a candidate execution profile, replay a captured traffic manifest, and render a signed report with full provenance and an evidence digest.
  - [Purge jobs](/api-v2/purge-jobs): Run an auditable purge over artifacts and retrieve a signed receipt. Delete revokes access; purge removes retained representations and proves what was done.
  - [Model aliases](/api-v2/model-aliases): List and retrieve rich alias releases, and install a new immutable release as an admin. Aliases resolve to immutable releases so routing stays reproducible.
  - [Agent hints](/api-v2/agent-hints): Store a reusable Agent Hints object and reference it by id, or send hints inline on any request. Hints express intent without provider-specific knobs.
  - [Usage](/api-v2/usage): Read per-generation usage events, a rolled-up summary, and an optional grouped breakdown. Every event records tokens, realized reuse, and the full QoS outcome.
  - [Analytics](/api-v2/analytics): Advanced, server-side analytics over the full retained usage window — a time series, SLA attainment, latency percentiles, cost and savings, and reuse/cache distributions, with a time range, filters, and a rich grouped breakdown.

  - **Accounts & billing**
  - [API keys](/api-v2/api-keys): Create, list, and revoke API keys, and set a per-key spending limit. Keys are shown in full exactly once and stored only as a hash.
  - [Provider credentials (BYOK)](/api-v2/provider-credentials): Attach, list, rotate, and delete bring-your-own-key provider secrets. Secrets are sealed at rest and fingerprinted for display; the plaintext is never returned.
  - [Subscription credentials](/api-v2/subscription-credentials): Attach a Claude Code or ChatGPT Codex subscription so eligible traffic runs against your bundled allowance at the provider's cache-discounted rate.
  - [Regional policy](/api-v2/regional-policy): Set and read the project's data-residency policy. Requests that would route to a forbidden region are denied before any provider call.
  - [Retention profile](/api-v2/retention-profile): Set and read how much of a request Zumik retains. The default is metadata-only; opt into tokenized or encrypted full-fidelity traces when you need them.
  - [Billing](/api-v2/billing): Read the billing account, set a monthly budget, opt into overage, start Stripe Checkout, open the customer portal, and the Stripe webhook contract.
  - [SAML SSO](/api-v2/sso): Configure an enterprise SAML connection and let your team sign in to the Zumik console through your identity provider, with signed-assertion verification, replay protection, and JIT provisioning.
  - [Compliance](/api-v2/compliance): Read the machine-readable sub-processor list and read or update per-project privacy preferences. One source of truth behind the published legal pages.
  - [Data rights](/api-v2/data-rights): Export everything Zumik retains for a project, or request erasure. GDPR and CCPA data-subject rights, scoped strictly to the caller's project.
  - [BYOC clusters](/api-v2/byoc-clusters): Register, list, retrieve, heartbeat, and deregister bring-your-own-cloud clusters. The control plane records your data planes; it never holds GPUs.
  - [Support tickets](/api-v2/support): The enterprise support portal API - open a ticket, exchange a message thread with the Zumik team, and track status. First-response SLA is derived from the project's plan tier.

- **SDKs & tools**
- SDKs
  - [SDKs](/sdk/overview): The four first-party Zumik SDKs - Python, TypeScript, Go, and Rust - with install, ZUMIK_API_KEY auth, the /v1 OpenAI-compatible vs native /v2 split, and a capability matrix.
  - [Python SDK](/sdk/python): Install the zumik package, authenticate, and call /v1 and native /v2 endpoints with a synchronous httpx-backed client.
  - [TypeScript SDK](/sdk/typescript): Install @zumik/sdk, authenticate, and call the full /v1 OpenAI-compatible surface plus native /v2 artifacts, bundles, sessions, responses, diagnostics, and purge from Node or the browser.
  - [Go SDK](/sdk/go): A small standard-library HTTP client around the core endpoints. Covers /v1 responses and the most-used /v2 state and diagnostics calls.
  - [Rust SDK](/sdk/rust): An async reqwest-based client covering /v1 responses and the core /v2 state and diagnostics calls, returning serde_json::Value.
- CLI & tools
  - [zumik CLI](/tools/cli): The zumik command runs the workload-analysis funnel locally - score, diagnose, lint, and proxy - before any data leaves your environment.
  - [trace-analyzer](/tools/trace-analyzer): The @zumik/trace-analyzer npm tool turns a metadata-only trace bundle into a reuse waterfall and Workload Reuse Score, with no Rust toolchain and no raw prompts.
  - [Trace-capture proxy](/tools/trace-capture-proxy): zumik proxy sits in front of an OpenAI-compatible endpoint and records one metadata-only trace per request - token counts, timing, and a prefix fingerprint, never raw prompts.
  - [Prompt linter](/tools/prompt-linter): zumik lint and the web prompt-linter check a prompt's layout for the structure that defeats provider-native caching - volatile content in the stable prefix, bad ordering, and a sub-1024-token prefix.
- Framework integrations
  - [Coding agents](/integrations/coding-agents): Use Zumik as a drop-in OpenAI-compatible endpoint for Cline, Roo Code, Continue, Aider, and the OpenAI SDKs - one base URL, one key, full request fidelity, and caching on by default.
  - [LangChain](/integrations/langchain): ChatZumik and ZumikEmbeddings for LangChain (Python and JavaScript) - thin configurations of the LangChain OpenAI wrappers pinned to Zumik's /v1 surface.
  - [LlamaIndex](/integrations/llamaindex): The Zumik LlamaIndex LLM - a thin OpenAILike configuration pinned to Zumik's /v1 surface - for query engines, chat engines, and agents.
  - [Using the OpenAI SDK](/integrations/openai-sdk): Point a stock OpenAI SDK at Zumik's /v1 surface with a one-line base-URL swap, and when to prefer this over a native Zumik SDK.
  - [Marketplace connectors](/integrations/marketplace): Install Slack, Discord, or a custom-webhook connector from the Zumik marketplace to deliver account events - budget-threshold alerts today - to your own destinations. Catalog, install, test, and the SSRF-safe delivery contract.
- Examples
  - [Examples](/examples/overview): Runnable reference integrations against a live Zumik deployment - OpenAI migration, a coding agent, RAG over sessions, and a multi-turn agent runtime.
  - [Coding agent](/examples/coding-agent): A coding agent whose repo policy and tool bundle are a stable, reused prefix - built as artifacts, pinned to a session, then streamed over code.fast.
  - [RAG](/examples/rag): Retrieval-augmented generation where the retrieved documents form a stable prefix pinned to a session, so repeated queries reuse the compiled document context.
  - [Agent runtime](/examples/agent-runtime): A multi-turn agent loop on Zumik's native /v2 session and branch surface - an append-only transcript with optimistic concurrency and interactive QoS.

- **Self-hosting**
- Execution profiles
  - [Execution profiles](/execution/overview): The four ways Zumik runs a request - managed providers by default, BYOK, BYOC, and hybrid - plus the OpenRouter emergency fallback, and the rule that BYOC is only ever activated when replay proves it.
  - [Managed providers](/execution/managed-providers): The default execution profile - Zumik's contracted provider accounts reached through the Bifrost gateway, with provider-native prompt caching, Batch APIs, and service tiers turned on for you.
  - [BYOK profile](/execution/byok): Bring your own provider key for any of the five first-class providers. Zumik calls the resolved provider with your sealed credential, you keep the billing relationship, and you inherit every provider-native optimization.
  - [BYOC profile](/execution/byoc): Self-host the inference data plane in your own cloud while Zumik keeps owning policy, resolution, and purge evidence. When and why to do it, replay-gated activation, and the control-plane ownership split. Running it needs your own GPUs.
  - [Hybrid profile](/execution/hybrid): Managed providers for broad coverage and overflow, with BYOC hot lanes carrying the few model paths concentrated enough to justify dedicated infrastructure. The common shape for coding-agent platforms.
  - [OpenRouter emergency fallback](/execution/openrouter-fallback): The last-resort continuity layer - used only after a verified primary failure for a required model path, gated behind explicit policy, and audited on every use. Never routine arbitration.
- Providers
  - [OpenAI](/providers/openai): OpenAI through Zumik - automatic prompt caching at a 50% read discount, the Batch API at 50% off, flex/default/scale service tiers, response retrieval and background mode, and when the broker routes here.
  - [Anthropic](/providers/anthropic): Anthropic through Zumik - the deepest cache-read discount of any managed provider at 90%, explicit cache_control breakpoints, the Message Batches API at 50% off, zero-retention availability, and when the broker routes here.
  - [xAI (Grok)](/providers/xai): xAI Grok through Zumik - context caching at a 75% read discount, live web-search grounding, Grok-3 and the cost-optimized Grok-3 Mini, no Batch tier, and when the broker routes here.
  - [Google Gemini](/providers/gemini): Gemini through Zumik - both implicit and explicit caching at up to a 75% discount, the largest context window in the set at 1M+ tokens, first-class multimodal input, the Batch API at 50% off, manual cache clearing, and when the broker routes here.
  - [Fireworks AI](/providers/fireworks): Fireworks through Zumik - the cost- and speed-optimized lane for open-source models, speculative decoding for sub-100ms TTFT, serverless and dedicated tiers, async batch, no prompt caching, and when the broker routes here.
- Infrastructure
  - [Bifrost gateway](/infrastructure/bifrost): The optional Tier 1 gateway in front of the Product API Core - auth, quotas, layered rate limits, automatic failover across 23+ providers, the OpenRouter emergency path, and where each limit is enforced.
  - [BYOC stack](/infrastructure/byoc-stack): The NVIDIA Dynamo + SGLang + FlashInfer + LMCache + Mooncake + AIBrix data plane - what each component does, what runs where, the infra/byoc Helm chart, replay-gated activation, and the cluster registry. Running it needs your own GPU cluster.
  - [Portable Kubernetes](/infrastructure/portable-kubernetes): The EPP-owned BYOC alternative - llm-d / Gateway API Inference Extension + KServe + vLLM + AIBrix + LMCache, the infra/k8s-portable Helm chart, and how it differs from the Dynamo stack. Running it needs your own GPU cluster.
  - [Terraform](/infrastructure/terraform): The infra/terraform modules - Cloudflare DNS, WAF rate limiting, and zone TLS settings, plus a provider-secrets renderer - with init/plan/apply, remote state on R2, and how sensitive variables stay out of plan output.
  - [Multi-region & global infrastructure](/infrastructure/multi-region): Zumik's region model - the control-plane region registry, the GET /v2/regions map, data residency, and the Cloudflare load-balancer that geo-steers api.zumik.ai across regions with /healthz failover.
  - [Observability](/infrastructure/observability): The /metrics Prometheus surface api-core exposes, the two Grafana dashboards in infra/observability, and what a registered BYOC cluster's collector ships - one for the managed API today, one for GPU clusters when they exist.
  - [Uptime monitoring and self-healing](/infrastructure/uptime-monitoring): How Zumik's Cloudflare worker probes every property from the edge and auto-remediates the VPS during an incident, paging a human only when automation is exhausted.