docs · zero → ci

Developer docs

The journey from zero to a green pipeline, how to drive it through your agent — the primary way to use Peira (PEER-uh) — the full CLI, and what a compiled case actually looks like. Everything below is real — commands, flags, and output shapes come from the tool, not from marketing.

Getting started

Install

Node ≥ 18, one first-party dependency — no third-party code on the trust path. Running, validating, and rendering need nothing else, ever. Only the authoring commands (compile, adopt) and offline triage use a model, and they shell out to your own logged-in Claude Code CLI session: no API key to provision, nothing in CI.

Scaffold — peira init

One command sets the project up: bed.json, an example intent pair (one acceptance criterion, one invariant), an empty cases/, and AGENTS.md — the cross-tool agent instructions Claude, Cursor, and Copilot-style agents read, with a one-line CLAUDE.md import for Claude Code. Deterministic and zero-LLM like everything on the trust path; zero prompts, so your agent can run it too; and it never overwrites — every file reports created or kept.

peira init          # bed.json, intent/example.md, AGENTS.md (+ CLAUDE.md import), cases/
peira init --ci     # …plus a zero-LLM GitHub Actions workflow

The next two steps then reduce to: point bed.json at your service, and replace the example intent with your service's real promises — or just tell your agent what the service promises; AGENTS.md briefs it.

Describe your service — bed.json

The bed config is the only place Peira learns anything about your service. Everything except baseUrl is optional: users are named principals cases refer to as $users.alice (never raw credentials) — Basic, a login request that returns a token, or a static API key, and the case says $users.staff either way; reset is one HTTP call before each run pointed at your service's own wipe-state endpoint; drain tells the runner how to ask your service whether an async job has settled, so one case's leftovers can never poison the next case's timing; timeouts declares a slow environment's latency envelope (ceilings only — hitting one is an error verdict, never a fail); service tells peira run how to start the app under test — an already-answering baseUrl is reused, a server Peira started is killed (whole process group) when the run ends.

{
  "baseUrl": "http://localhost:8080",
  "users": {
    "alice": { "username": "alice", "password": "test-pw" },
    "staff": { "login": { "route": "/api/login", "body": { "email": "staff@example.com", "password": "…" },
                          "token": "body.token", "send": { "header": "Authorization", "format": "Bearer {{token}}" } } }
  },
  "reset": { "method": "post", "url": "/test/reset" },
  "drain": { "route": "/orders/status", "idParam": "id",
             "statusPath": "body.state", "terminal": ["SHIPPED", "CANCELLED"] },
  "service": { "command": "npm run dev", "cwd": "../orders-service" }
}

Keep one bed file per environment (bed.json, bed.ci.json): same cases, different target.

Write intent — intent/*.md

Intent is the human-owned source of truth: plain markdown, one ## section per acceptance criterion or invariant. Tags are optional but recommended — the id is a permanent lineage anchor, so a tagged section can be reworded freely without orphaning its cases. kind=invariant sections compile to templates that mint fresh seeded probes every run. Organize files by capability, not endpoint: the section is the unit of everything.

## Creating an order
<!-- peira: id=order-create kind=ac -->
POST /orders with a valid payment method returns 201 with the new order's id.

## Order isolation
<!-- peira: id=order-isolation kind=invariant -->
For all orders o, for all users u ≠ owner(o): GET /orders/{o} as u → 403.

Already have a messy test plan? peira adopt restructures it once — never rewrites — and prints a content-preservation report. You review and commit; from then on it is your document.

Compile

Runs on your Claude session. Every candidate case passes the same schema gate as a hand-written one; lineage is stamped mechanically; the compile manifest accounts for every section (compiled / skipped-with-reason / refused). Review the generated cases as a diff — that review is the human checkpoint the trust model is built on.

peira compile intent --out cases --bed bed.json
peira compile intent --dry-run    # report only — nothing written

--dry-run is the feedback loop on your document: how many sections compiled, and why any section was skipped ("states no verifiable behavior; names no route or status code") or a candidate refused. A skip is a note about your intent — usually prose, or a quality attribute no functional API case can state — not a failure of the tool.

Run locally — and close the loop

Verdicts are pass | fail | error — assertion failures and infrastructure failures are never conflated. The seed is always printed: any failure reproduces exactly with the same seed against the same service state. First runs usually surface both real bugs and stale intent — that's the point. One caveat: a fixed seed against a service you never reset collides with its own previous run's data, so cases that create state need a reset or a fresh seed — CI already uses the run id; locally, vary it too.

peira run cases --bed bed.json --seed 42 --evidence run.jsonl
peira run cases --bed bed.json --seed 42 --only CASE-order-cancel-001   # re-run the one failing case
peira run cases --bed bed.json --parallel 8   # worker pool; verdicts + evidence identical to serial
peira run cases --bed bed.json --intent intent --watch   # re-run on change, mapped by lineage
peira triage --evidence run.jsonl --intent intent   # proposes bug | drift | flake; applies nothing
# you adjudicate: fix the service, or edit the intent…
peira validate cases --bed bed.json --intent intent # stale flags name the affected cases
peira compile intent --out cases --bed bed.json --section <changed-section>

Watch mode maps changes by lineage, not an import graph: a case edit re-runs exactly that case; an intent edit re-checks staleness and names the affected cases — recompiling stays your call, never an LLM on a save hook. And share readable documentation any time: peira render cases --intent intent --evidence run.jsonl (Given/When/Then, or a full HTML run report; one-way output — regenerate, never edit).

CI — zero LLM

Commit intent/, cases/, and the bed configs. CI needs no key and no session; the exit code gates the merge, and --junit writes standard JUnit XML (pass/fail/error map to testcase/failure/error) so any CI test-report UI renders the run without wrapper scripts. When CI goes red, pull the evidence artifact and triage it locally — adjudication stays a human act, never a bot in the pipeline.

# .github/workflows/api-tests.yml
- run: npm ci
- run: docker compose up -d orders-service
- run: npx peira validate cases --bed bed.ci.json --intent intent
- run: npx peira run cases --bed bed.ci.json --seed ${{ github.run_id }} --evidence run.jsonl --junit junit.xml
- if: always()
  uses: actions/upload-artifact@v4
  with: { name: evidence, path: run.jsonl }

Then record the adjudicated run: peira evidence --evidence run.jsonl --triage run-triage.json --intent intent. Passing sections log applied; adjudicated drift logs contradicted with the verbatim note; peira trust shows the standings. Trust is earned by runs, not by hand.

Using Peira through your agent

This is the primary way to use Peira. It is agent-native by design: the authoring surfaces already run on your own Claude session, and the deterministic runner is exactly what makes agent-driven testing trustworthy. In practice you talk to your agent in intent-language — it edits the plan, compiles, runs, renders the report, and drafts triage for your adjudication.

The loop, as a conversation

you

Add coverage: cancelling an order that already shipped must be refused.

agent

Added a tagged section to intent/orders.md, compiled just that section, and ran the suite:

$ peira compile intent --out cases --bed bed.json --section order-cancel-shipped

$ peira run cases --bed bed.json --evidence run.jsonl

agent

1 failure — the service answers 200 where the intent requires 409. Triage proposes BUG with the request/response evidence attached. Here's the report; do we file it, or should the intent change?

$ peira triage --evidence run.jsonl --intent intent

$ peira render cases --intent intent --evidence run.jsonl --format html --out report.html

you

That's a bug. File it and record the run.

agent

Filed. Run recorded in the evidence ledger — the section logged applied (it did its job catching the violation).

$ peira evidence --evidence run.jsonl --triage run-triage.json --intent intent

Drop-in agent instructions

peira init scaffolds this as AGENTS.md — the cross-tool convention Claude, Cursor, and Copilot-style agents read, with a CLAUDE.md import for Claude Code — and the workflow above is what you get:

# API testing with Peira

Peira compiles a markdown test plan (intent/*.md) into JSON cases and runs them with
no model in the loop. Everything the tool can say is in one place — read it before you
write a case, and again after the tool is upgraded:

    peira reference

## The loop

- Intent is the source of truth. To change a test, edit its intent section, then
  recompile exactly that section:
    peira compile intent --out cases --bed bed.json --section <id>
- A case written by hand is fine; bind it to its section without a model:
    peira stamp cases --intent intent        (--check in CI: exit 1 if anything is unstamped or stale)
- Run and keep the evidence (the printed seed replays any failure exactly):
    peira run cases --bed bed.json --evidence run.jsonl
- On failures, triage and PRESENT the proposals — adjudication belongs to the
  human, never to you:
    peira triage --evidence run.jsonl --intent intent
- When the human wants to see results:
    peira render cases --intent intent --evidence run.jsonl --format html --out report.html
- After adjudication, record the run so intent sections earn trust:
    peira evidence --evidence run.jsonl --triage run-triage.json --intent intent

## Rules the gate enforces (validate says so, with the fix in the message)

- Never edit a compiled case to make a run green; fix the service or propose an intent change.
- from.intent is yours; from.hash never is — compile stamps it, `peira stamp` fills it.
- Inside a string use {{alias}}; a bare $alias is only the whole value.
- No wall-clock sleeps. Eventual consistency is pollUntil; cleanup is teardown {"drain": true}.
- Matchers stand alone: $any, $contains (string or all-of list), $notContains, $absent, $text, null.
  Negative claims are where the bugs are — assert what a user must NOT see or hold.
- Cases never contain credentials: auth is "$users.<alias>"; the bed defines the alias.
- A red run is pass | fail | error and the kinds are never conflated: error means the
  environment failed before the claim was judged — say so, do not report it as a bug.

Why this is safe to hand to an agent

The runner can't be sweet-talked

Verdicts are deterministic — a function of (cases, seed, service state). Zero LLM at runtime means an agent cannot wiggle a red run green; it can only fix the service or propose an intent change you approve.

The gate refuses, it never patches

Everything a model emits — compiled cases, triage proposals, adopted intent — passes a deterministic schema gate. Malformed output is refused with reasons, never silently corrected.

Nothing self-applies

Triage proposes bug | drift | flake; the human adjudicates. Intent is yours; cases are regenerable artifacts; the evidence ledger records what was decided, with the reason quoted verbatim.

It runs on your session

compile, triage, and adopt shell out to your own logged-in Claude Code CLI — the same session your agent lives in. No API key to provision, nothing extra to secure.

CLI reference

Twelve commands. Only compile, triage, and adopt ever touch a model — on your own session, never in CI.

init

peira init [dir] [--ci]

Scaffold a project: bed.json, example intent, AGENTS.md agent instructions (+ CLAUDE.md import), cases/. --ci adds a zero-LLM GitHub Actions workflow. Deterministic, zero prompts, never overwrites.

validate

peira validate [casesDir] [--bed <path>] [--intent <dir>]

Schema + static checks on every case; with --intent also flags stale cases and lints intent structure.

run

peira run [casesDir] --bed <path> [--seed <n>] [--evidence <path>] [--only <id>]… [--grep <substr>] [--parallel <n>] [--junit <path>] [--shard <i>/<n>] [--watch]

The deterministic runner. Zero LLM; seeded, reproducible; writes evidence JSONL with credentials redacted at write time. --only/--grep re-run just the cases you name; --parallel runs a worker pool with verdicts and evidence order identical to serial; --junit emits CI-standard XML; --shard fans out across machines in disjoint deterministic slices; --watch re-runs on change, mapped by lineage.

compile

peira compile [intentDir] --out <dir> [--bed <path>] [--section <id>]… [--dry-run]

Intent sections → schema-gated JSON cases via your own Claude session. --section recompiles exactly the named sections and merges the manifest. --dry-run reports without writing: how much of your intent compiles, and why any section was skipped or candidate refused.

stats

peira stats [casesDir] [--openapi <spec.json>]

DSL coverage, recurring escape-hatch shapes, and the refusal balance — per intent: cases, positive (expected status < 400), negative (≥ 400), negative-oracle ($absent / $notContains), with a line naming intents that test only the happy path. A suite drifts positive one happy path at a time; this is the per-run guard. With --openapi: endpoint coverage — which endpoints have no case.

triage

peira triage --evidence <run.jsonl> --intent <dir>

Offline failure classification: bug | drift | flake, judged against the intent text. Proposals only — nothing is ever applied.

evidence

peira evidence --evidence <run.jsonl> [--triage <file>] --intent <dir>

Records an adjudicated run into the evidence ledger (plus a portable JSONL export). Sections earn applied / contradicted per run.

trust

peira trust

The ledger standings — per intent section: applied, contradicted, runs, last applied.

render

peira render [casesDir] [--evidence <run.jsonl>] [--format md|html]

One-way readable documentation: Given/When/Then markdown, or a self-contained visual HTML run report with observed exchanges on failures.

adopt

peira adopt <messy.md> --out <intent/name.md>

One-time authoring assist: restructures an arbitrary document into tagged intent, with a content-preservation report. You review; you own the result.

stamp

peira stamp [casesDir] --intent <dir> [--check]

Bind hand-written cases to intent without a model: fills or refreshes from.hash from the live section text. from.intent is yours; from.hash never is. --check exits 1 if any case would change — the zero-LLM CI gate for lineage.

reference

peira reference

The installed version's complete vocabulary — case and bed schemas with every property described, the matchers, interpolation, principals, responses, verdicts, the CLI — as markdown, generated from the schemas themselves. What an agent reads instead of dist/; the AGENTS.md scaffold points here.

full flag glossary: peira help

Anatomy of a case

A case is JSON: optional setup steps, one test step, optional teardown. Five primitives cover what hand-written suites need — a real legacy suite re-expressed 27/27 specs with zero escape hatches.

{
  "id": "CASE-order-isolation-001",
  "title": "Another user cannot read my order",
  "from": { "intent": "order-isolation", "hash": "ae5ab7a63816" },
  "setup": [{
    "request": { "method": "post", "route": "/orders",
                 "auth": "$users.alice",
                 "body": { "note": "x {{unique.nonce}}" } },
    "capture": { "orderId": "body.id" }
  }],
  "test": {
    "request": { "method": "get", "route": "/orders/$orderId",
                 "auth": "$users.bob" },
    "expect": { "status": 403 }
  },
  "teardown": { "drain": true }
}

from: Lineage, stamped mechanically at compile time — never trusted from the model. When the intent section's text changes, this hash mismatch flags the case stale.
$users.alice: A bed principal by name. Cases never contain credentials; the bed maps names to auth per environment.
{{unique.nonce}}: Seed-derived discriminator: hash(seed, case id, key). Same seed → same value; no fixture files.
capture: Maps an alias to a response path (body.id). Later steps reference it as $orderId (whole value) or {{orderId}} inside strings.
expect: Subset matching, Jest toMatchObject parity, on status, headers (case-insensitive — {"content-type": {"$contains": "application/json"}}), and body. Matchers: {"$any": "string" | "number" | "boolean"}, {"$contains": "s" | ["a", "b"]}, {"$notContains": …}, {"$absent": true}, and literal null. Add pollUntil for eventual consistency — never wall-clock sleeps.
teardown.drain: Declares that this case must clean up; the bed's drain probe knows how. The runner polls every captured job to a terminal state before the next case runs.

Cases are regenerable artifacts — never hand-patch one into divergence. Change the intent, recompile the section, review the diff. That is the whole discipline.

Reference

The complete programmable surface, on one page — finite because the DSL is deliberately closed. Anything not listed here is refused by the schema gate; the vocabulary grows by amendment (evidenced by stats telemetry), never by extension hooks.

The case

One JSON file. id, from, and test are required.

id: CASE-<kebab-slug> — unique across the set; duplicates are refused.
from: Lineage {intent, hash}: which section, at which content hash, produced this case. Stamped mechanically, never trusted from the model; a mismatched hash flags the case stale. Minted instances add {template, seed, instance}.
setup: Optional array of steps — request steps or registry-step invocations — run in order.
test: Exactly one request step; the claim under test lives here.
teardown: {"drain": true} — after the verdict, every captured id is polled to a terminal state via the bed's drain probe, under the credentials that captured it.

A request step

request: method (get | post | put | delete | patch), route (starts with /), optional query and body. auth takes four forms: "$users.<alias>", a literal {username, password} for negative Basic tests, a literal {token} (optional send; defaults to Bearer) for negative token tests, or absent for anonymous.
headers: The case's own request headers — {"Accept": "text/html", "Accept-Language": "ko"} — interpolated. Applied lowest: the principal's attachment overrides them (a case cannot override its own Authorization) and the body kind owns content-type. content-length, transfer-encoding, host are refused, and so are Sec-* / Proxy-*, which the HTTP runtime rewrites silently.
form: {name: value} sent as application/x-www-form-urlencoded — a native form post, the body a browser sends before hydration. Mutually exclusive with body and multipart; password fields are redacted in evidence.
multipart: {fields?, files?: [{field, path | bytes, mimetype?, filename?}]} — send multipart/form-data instead of a JSON body (never both). files[].path is relative to the cases directory: an ordinary file in the repo, never inline bytes; validate fails if it is missing or over 256 KB. mimetype is explicit so a wrong-type refusal is a case. The evidence log records names and sizes, never content.
followRedirects: Default true. With false the step sees its own 3xx — expect.status 307 and expect.headers.location become assertable, and capture: {next: "headers.location"} means something.
capture: alias → dotted response path rooted at status, body, or headers (body.id, headers.location). A path missing from the response fails the case, naming the path.
pollUntil: Re-issues the request until an expect block matches — pinned 100ms interval, timeoutMs ceiling (default 10s or the bed's pollUntilMs). Non-convergence is a fail. The declarative replacement for sleeps, which are refused.

expect — the oracle

Subset matching, Jest toMatchObject parity: objects as subsets at every level, arrays index-wise with equal length, primitives strictly.

status: Exact status code.
headers: Response headers by name, case-insensitive (RFC 9110); values are a literal string or a matcher, nothing else. A missing header is a named diff.
body: Subset match against the JSON body.
oracle: {statusOnly: "<reason>"} — a note, not an assertion: this case asserts only the status on purpose (identical 404s so nothing can be inferred). The weak-oracle lint accepts it, render prints it, and it is refused beside a body or headers assertion.
bodySchema: A JSON-Schema subset the whole body must satisfy (type, required, properties, additionalProperties, enum, items, pattern, anyOf) — for "every element has shape X" claims.
{"$any": …}: Matcher: present, of type "string" | "number" | "boolean".
{"$contains": …}: Matcher: a string containing the substring — or every substring in a list (all of; each missing one is its own diff). The content-type matcher, and the oracle for text bodies: HTML and other non-JSON responses arrive as a string. One trap: server-rendered React separates adjacent text expressions with , so assert text from one expression or a stable attribute, not visible text that spans an interpolation.
{"$notContains": …}: Matcher: a string containing none of the listed substrings — "must not leak X". The open-redirect guard: location: {$notContains: "evil.example"}. The positive form alone is fooled by https://evil.example/?back=/hub.
{"$text": {contains, notContains}}: Matcher: the body as text — tags and comments stripped, whitespace collapsed — must contain all of and none of. Whole body only. Server-rendered React splits text with , so $contains on the raw body misses "총 2건"; $text finds it, and it is the one place contains and notContains apply to the same text body.
{"$absent": true}: Matcher: the key or header must not exist. Distinct from null; refused as the whole body. The motivating shape: an access map that omits denied permissions — GET /api/access as an editor → {"tenants": {"create": {"$absent": true}}}. Assert the omissions, not the grants: it asks what a user holds that they shouldn't, a question positive checks never pose.
null: Matcher: present and exactly null. Matchers stand alone and work in body, pollUntil.until, and header values. No custom matchers, by design — the vocabulary grows by amendment.

Interpolation

"$alias": A string that IS the reference resolves to the captured value, type-preserving.
{{alias}}: Inside any string, at any depth: String(value) spliced in. {{{{ escapes a literal {{.
unique.<key>: Seed-derived discriminator: hash(seed, caseId, key). Same seed → same value; no fixture files.
$users.<alias>: A bed principal — legal only in a request's auth position, never spliced into data.

The bed — bed.json

The only place Peira learns about your service. Everything except baseUrl is optional.

baseUrl: Where the service answers; --base-url overrides per invocation.
users: Named principals — cases say $users.alice, never credentials. Exactly one shape per alias: Basic {username, password}; login {login: {method?, route, body?, token, send}} — logs in once per run on first use (once even under --parallel), captures the token at a path like body.token, attaches it per send; or static {token, send} for API keys. send is {header, format containing {{token}}} or {cookie}. A refused login makes every case on that principal an error, never a fail. Tokens and passwords are scrubbed from the evidence log by value.
reset: {url, method?} — one wipe-state call before each run.
drain: {route, idParam, statusPath, terminal[]} — how to ask whether an async job settled; powers teardown.drain.
timeouts: Latency-envelope ceilings {requestMs?, pollUntilMs?, drainMs?, stepMs?}. Hitting one is an error, never a fail; the poll interval stays pinned for determinism.
service: {command, cwd?, readyMs?, reuse?} — how peira run starts the app under test. reuse (default) adopts an already-answering baseUrl and never kills it; a server Peira started is killed, whole process group, when the run ends.

Verdicts, exit codes, evidence

pass | fail | error: fail = an assertion did not hold; error = infrastructure failed before an assertion could be judged. Never conflated — and --junit maps them to testcase/failure/error losslessly.
exit codes: 0 all pass · 1 any fail/error (or the set was refused, or the service never answered) · 2 usage error.
run.jsonl: Append-only JSONL, one event per line: run-start, minted, case-start, http (every exchange, request + response + elapsedMs), step, case-verdict, drain-*, run-end (counts, wallMs, httpMs). This is the integration surface — triage, the ledger, and reports all read it.
redaction: Authorization, Cookie, and Set-Cookie values are stored as [REDACTED:<sha256-prefix>] at write time — equality across events survives, secrets never land in the log.

The escape hatch, and templates

steps: Generated procedure with a typed contract: {id, reads[], produces[], code}. Invoked in setup only as {"step": "STEP-…", "bind": {…}} — invocations structurally cannot carry expect or capture; the claim stays declarative. Every use is telemetry asking which primitive the DSL is missing.
holes: Invariant templates declare typed holes — principal (optionally distinctFrom another), expression ({{holes.x.code}} / {{holes.x.result}}), unique — and mint 5 fresh seeded cases per run. (template, seed, instance) reproduces any of them exactly.

authority: schema/case.schema.json · full document: docs/REFERENCE.md in the repo