Verify the Audit Chain
Anchor, verify and export the audit trail: what each verdict means, what the signed manifest covers, and which refusals a retry will never fix.
Take an anchor document you already hold, ask the backend whether it is genuine, then pull a signed slice of the chain and check the signature yourself, offline, without trusting the server that produced it.
Prerequisites
npm install @codespar/sdkThe REST client used here, cs.api, ships in @codespar/sdk 0.12.0 and later. Three things have to be true before any of it works.
| Requirement | How to check |
|---|---|
The key carries audit:read | GET /v1/whoami, field key.scopes. A * scope also passes. |
| You hold an anchor document | It exists outside the database only if the deployment runs AUDIT_ANCHOR_TARGET=webhook or file. See step 3. |
| You hold the export signing key, in hex | No route returns it. The keys live encrypted in audit_signing_keys under VAULT_MASTER_KEY. Without it, step 8 checks everything except the HMAC. |
AUDIT_ANCHOR_TARGET, AUDIT_ANCHOR_URL, AUDIT_ANCHOR_PATH, AUDIT_ANCHOR_SECRET and AUDIT_ANCHOR_INTERVAL_MS are deployment environment variables read at boot. On hosted CodeSpar you cannot change them, and there is no route that sets them. The default target is noop, which is what hosted production runs today.
Step 0: there is no call that anchors
The word "anchor" in this journey is not an API call. Anchoring is a periodic background job. It runs every DEFAULT_INTERVAL_MS, 60 minutes, signs up to 200 chain heads per pass, and is idempotent on the unique index over (org_id, head_sequence): running a pass again without the chain having moved writes nothing.
POST /v1/audit-events/anchors does not exist. There are 276 operations in the served document and none of them triggers a pass. The reason is in the route file header: operators do not drive anchoring pass by pass.
Step 1: get your org id and confirm the scope
The verify body requires org_id, and GET /v1/whoami is the only route that returns it alongside the scopes on your key.
import { CodeSpar } from "@codespar/sdk";
const cs = new CodeSpar({ apiKey: process.env.CODESPAR_API_KEY! });
const who = await cs.api.get("/v1/whoami");
const orgId = who.organization.id;
const scopes = who.key.scopes;
if (!scopes.includes("audit:read") && !scopes.includes("*")) {
throw new Error(`key ${who.key.id} has no audit:read scope`);
}Step 2: list the anchors your org has
const { anchors } = await cs.api.get("/v1/audit-events/anchors", {
query: { limit: 50, status: "local_only" },
});
for (const a of anchors) {
console.log(a.id, a.head_sequence, a.target, a.target_status);
}Newest first, ordered by anchored_at descending. limit is 1 to 200, default 50. status is one of pending, sent, local_only, failed, and omitting it returns every state. Each row carries id, head_sequence, head_hash, signature, signature_alg, target, target_status, target_response, delivery_attempts, anchored_at and delivered_at.
This response alone cannot build the verify body. It does not return org_id (take it from GET /v1/whoami), it does not return version (the literal 1), and the field it calls id is the field the verify body calls anchor_id. A client that forwards the list row straight into verify gets a 400.
target_status: "local_only" is a terminal state, not a sent that is running late. It means the anchor is durable in this database and witnessed nowhere outside it. Before the fix tracked as ent#795 these rows were written as sent with delivered_at filled in, and the 0149 migration backfilled them by the target_response = 'noop' discriminator. If your compliance story needs an external witness, local_only is the answer "there is none".
There is no pagination here. Only limit, capped at 200, and status. No cursor, no next_*. An org with more than 200 anchors cannot walk backwards through this route.
Step 3: fetch the anchor document you are checking
This step is not an API call. Take the document you stored when the anchor was delivered: the POST body your webhook target received, or one line of the JSONL file the file target appends to. It is the record the job signed, exactly:
{
"version": 1,
"anchor_id": "anc_8f3c1d2e4b5a6970",
"org_id": "org_01JQ8W3ZK4T6R2",
"head_sequence": 48211,
"head_hash": "3a1f9c77b0e4d2158c6a0b93e7f4d51c8a2e6b04f9d37c15a8b0e2f6c4d91738",
"anchored_at": "2026-09-14T11:00:00.000Z",
"signature_alg": "HMAC-SHA256",
"signature": "4b7e2f90c1a8d635e4f207b93c8a1d5e6072f4b8c9d130a6e2f5b7c8d9a04136"
}With no external target configured, this document exists nowhere outside the database. Verifying it against the same database that produced it is a check against accidental corruption, not against an adversary holding write access to it.
Step 4: verify the anchor
import { readFile } from "node:fs/promises";
const doc = JSON.parse(await readFile("anchor.json", "utf8"));
const res = await cs.api.response("post", "/v1/audit-events/anchors/verify", {
body: {
version: 1,
anchor_id: doc.anchor_id,
org_id: orgId,
head_sequence: doc.head_sequence,
head_hash: doc.head_hash,
anchored_at: doc.anchored_at,
signature_alg: "HMAC-SHA256",
signature: doc.signature,
},
});
if (!res.ok) {
console.error(res.status, res.data);
process.exit(1);
}
const { verdict, signature_valid, persisted_locally, persisted_matches } = res.data;
console.log(verdict, { signature_valid, persisted_locally, persisted_matches });Use cs.api.response() and not cs.api.request(). request() throws CodesparApiError on the 403 and the 503, and both of those are answers you want to read, not exceptions you want to catch.
The handler runs in a fixed order, and knowing it tells you what had already happened when an error came back: Zod on the body, then the cross-org check, then the presence of the secret, then the HMAC recomputation, then a single SELECT against the local row, then the verdict. Nothing is written. Repeat it as often as you like.
The four verdicts
| Verdict | What it says |
|---|---|
verified | Signature recomputes, and a local row with the same head_hash and signature exists. |
signature_ok_but_no_local_record | The signature is ours, but no row matches this (org_id, head_sequence). One side lost the row. |
local_record_diverges | The signature is ours, a row exists, and its head_hash or signature differs from the document. |
signature_invalid | The HMAC did not recompute. |
HTTP 200 does not mean verified. All four verdicts come back with 200. Only the verdict field separates them, and the generated SDK page lists the route as "(200, 403)", which reinforces the wrong reading. Branch on verdict, never on the status.
signature_invalid is the catch-all. The verdict is picked by an ordered chain of conditions, and every case where signature_valid is false falls into the last one, even when an identical local row exists. So signature_invalid tells you nothing about the state of the local record.
Two more things about that signature. It covers the exact string in anchored_at, inside the canonical join "v1|anchor_id|org_id|head_sequence|head_hash|anchored_at". Reformatting the timestamp, dropping the milliseconds, swapping Z for +00:00, or round-tripping it through a date parser, breaks the recomputation and returns signature_invalid with nothing at all having been tampered with. And signature_alg is validated as the literal "HMAC-SHA256" and echoed back, but it is not part of the signed join.
Also worth knowing: the body accepts head_sequence: 0 as a valid integer, but the table constrains the column to 1 or greater. A document with sequence 0 always comes back persisted_locally: false.
What the persistence flags actually compare
The served document describes persisted_locally as "a row of ours exists with this anchor_id". That is not what runs. The lookup is WHERE org_id = ? AND head_sequence = ?, and anchor_id never enters the query. persisted_matches compares two fields, head_hash and signature. Not anchored_at, not anchor_id, despite the description saying it matches field by field.
Refusals on verify, and which ones a retry fixes
| Status and code | What happened, and what to do |
|---|---|
400 invalid_body | The document did not match the eight required fields. Full envelope, details.issues from Zod. Retry does not fix it. |
403 cross_org_verify_denied | org_id in the body is not the org on your credential. Checked before the secret is read and before any signature is computed, because the signature derives from the owning org's secret. Use that org's credential. |
503 anchor_secret_unconfigured | AUDIT_ANCHOR_SECRET is not set on this deployment. This is server configuration, not a transient fault. Retry does not fix it, and a 503 here is the one case where the usual "back off and try again" reflex is wrong. |
403 forbidden | The scope gate, with message: "API key does not have the 'audit:read' scope.". This 403 is not in the served document for any anchor route, and it is a third shape of 403 alongside the two above. |
Step 5: export a slice of the chain
from and to are both required. agent_id is optional and narrows the slice.
curl -sS -N \
-H "Authorization: Bearer $CODESPAR_API_KEY" \
-D headers.txt \
-o export.json \
"https://api.codespar.dev/v1/audit-events/export?from=2026-09-01T00:00:00Z&to=2026-09-08T00:00:00Z"The curl example on the generated reference page for this route omits from and to. Copying it returns 400 invalid_query.
The handler order, again because it tells you what had already happened: the cross-org check, then Zod on the query, then parsing from and to as dates, then the from > to comparison, then a preflight COUNT(*) using the same WHERE the real query uses, and only then the content type, the x-export-row-count header, a 60 second timeout and reply.hijack(). After the hijack there is no status code left to send.
That ordering is what makes the 413 clean: the row cap is enforced before a single byte of body is written.
| Status and code | What happened, and what to do |
|---|---|
400 invalid_query | Query did not match the schema. Retry does not fix it. |
400 invalid_iso_8601 | from or to did not parse as a date. Retry does not fix it. |
400 from_after_to | The window is inverted. Retry does not fix it. |
403 forbidden | The path orgId on the deprecated alias is not the org on your credential, or the scope gate refused. Retry does not fix it. |
413 export_too_large | row_count exceeded EXPORT_ROW_LIMIT, 600000. An identical retry gets the same answer. Narrow the window, or pass agent_id. |
The export route does not use the standard error envelope. Its refusals come back as a raw body: {"error":"export_too_large","row_count":812004,"limit":600000}, or {"error":"forbidden"}. A client that reads body.error.code, which works everywhere else in this API, reads undefined here.
Step 6: read the response as a stream
The byte order is fixed. First {"export_id":…,"org_id":…,"rows":[, then one row at a time straight off a Postgres cursor in batches of 500 ordered by sequence_number ascending, then ],"manifest":{…},"export_signature":{"signed_payload":{…},"value":"<hex>"},"key_id":"…"}. The x-export-row-count header arrives before the first row.
import { createWriteStream } from "node:fs";
import { Readable } from "node:stream";
import { pipeline } from "node:stream/promises";
const url = new URL("https://api.codespar.dev/v1/audit-events/export");
url.searchParams.set("from", "2026-09-01T00:00:00Z");
url.searchParams.set("to", "2026-09-08T00:00:00Z");
const res = await fetch(url, {
headers: { Authorization: `Bearer ${process.env.CODESPAR_API_KEY!}` },
});
if (!res.ok) {
// Raw body here, not the usual envelope.
console.error(res.status, await res.text());
process.exit(1);
}
console.log("preflight count:", res.headers.get("x-export-row-count"));
await pipeline(
Readable.fromWeb(res.body as ReadableStream),
createWriteStream("export.json"),
);Do not pull the export through cs.api. cs.api.response("get", "/v1/audit-events/export") buffers: readBody calls res.text() and then JSON.parse. On an aborted export the parse fails and the catch returns the raw string, so you receive ok: true, status 200, and a string where you expected an object, with nothing thrown.
Failure after the first byte
Once the response is hijacked there is no status code left, so failure is signalled inside the body. The server appends {"error":"audit_export_aborted","phase":"stream"} or {"error":"audit_export_aborted","phase":"chain_event"} to the partially written body and destroys the socket. The result is deliberately malformed JSON, so an offline verifier refuses loudly instead of accepting a slice the chain never recorded. HTTP stays 200 throughout.
| Phase | What to do |
|---|---|
phase: "stream" | Retry. The export never completed and nothing signed left the building. |
phase: "chain_event" | Do not retry blind. The signed document is in your hands and the chain did not record it. That is manual reconciliation, and the server log carries export_id, row_count and final_hash. |
The timeout is 60 seconds on both request and reply, from AUDIT_EXPORT_TIMEOUT_MS. Blowing through it truncates the body mid-rows, also with 200. There is no Content-Length, because the body is chunked, so truncation is indistinguishable from a normal ending except by the JSON being malformed.
Step 7: the export writes to the chain it exported
After the cursor drains and before the response closes, the server appends an audit_export event carrying export_id, from, to, row_count, final_hash, requested_by, scope_version and filters. It takes the per-org advisory lock, so its sequence_number sits above every row it just shipped, by construction.
Exporting the same window twice returns a different row_count the second time. The first export became a chain row inside that window. Two identical exports are not expected to match. That is the design, not a defect.
A smaller version of the same effect: x-export-row-count comes from the preflight COUNT(*) taken before the cursor opened, while manifest.row_count counts what actually streamed. Rows landing in the window between the two reads make the numbers differ.
Step 8: check the file offline
The manifest carries nine fields: export_id, org_id, requested_by, requested_at, date_range, scope_version, filters, final_hash and row_count.
manifest.rows_digest does not exist in the response. The served document lists it as required on manifest, and the generated reference page repeats it in the example body. The real field lives at export_signature.signed_payload.rows_digest. The document says as much about itself: response schemas there are hand written and are not checked against handler output, so treat a response shape as documentation and not as a contract.
The published schema is also thinner than the payload in two other places. export_signature is documented as an opaque object, so the structure you need to verify anything, signed_payload plus a hex value, appears nowhere in the reference. Each entry of rows is documented as an empty object; the real shape is id, org_id, event_type, happened_at, payload, prev_hash, entry_hash, sequence_number, org_tombstone and created_at.
import { readFile } from "node:fs/promises";
import { createHash, createHmac } from "node:crypto";
// RFC 8785 (JCS). A plain JSON.stringify produces different bytes
// and every hash below will miss for no good reason.
import { canonicalize } from "./jcs";
const doc = JSON.parse(await readFile("export.json", "utf8"));
const rows = doc.rows as any[];
const chunks: string[] = [];
for (let i = 0; i < rows.length; i++) {
const row = rows[i];
// Strictly increasing. Gaps are legitimate: every export is a filtered slice.
if (i > 0 && row.sequence_number <= rows[i - 1].sequence_number) {
throw new Error(`chain_break at ${row.sequence_number}`);
}
// The prev_hash link is only checkable across consecutive sequence numbers.
if (i > 0 && row.sequence_number === rows[i - 1].sequence_number + 1) {
if (row.prev_hash !== rows[i - 1].entry_hash) {
throw new Error(`chain_break at ${row.sequence_number}`);
}
}
// Anonymized rows cannot be rehashed: the hash covers a pre-anonymization
// payload that no longer exists. They are validated by a pii_anonymized
// witness event, which has to be inside this same export.
if (!row.org_tombstone) {
const h = createHash("sha256")
.update(
row.prev_hash +
row.org_id +
row.event_type +
new Date(row.happened_at).toISOString() +
canonicalize(row.payload),
)
.digest("hex");
if (h !== row.entry_hash) throw new Error(`modified_row at ${row.sequence_number}`);
}
chunks.push(`${row.sequence_number}:${row.entry_hash}\n`);
}
// An empty export is valid, and requires both of these.
const expectedFinal = rows.length ? rows[rows.length - 1].entry_hash : "";
if (doc.manifest.row_count !== rows.length) throw new Error("manifest_mismatch");
if (doc.manifest.final_hash !== expectedFinal) throw new Error("manifest_mismatch");
const rowsDigest = createHash("sha256").update(chunks.join("")).digest("hex");
if (rowsDigest !== doc.export_signature.signed_payload.rows_digest) {
throw new Error("manifest_mismatch");
}
// The signing key arrives out of band, in hex. No route returns it.
const mac = createHmac("sha256", Buffer.from(process.env.AUDIT_SIGNING_KEY_HEX!, "hex"))
.update(canonicalize(doc.export_signature.signed_payload))
.digest("hex");
if (mac !== doc.export_signature.value) throw new Error("signature_mismatch");
console.log("PASS", rows.length, "rows");The genesis prev_hash, the value that precedes sequence 1, is e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855.
rows_digest is the check that catches renumbering. Gaps in sequence_number are accepted on purpose, and only adjacent rows can be linked through prev_hash. Without the digest, an attacker rewrites a non-final row and opens a one-row gap just after it, and final_hash, row_count and the HMAC all still verify. The digest binds the numbering to the hashes.
What the export signature never establishes: that nothing was withheld inside the declared scope. No offline check answers that. Re-exporting the same window and comparing is the only thing that does. The scope declaration answers the earlier question, whether the slice was narrowed, and says so in filters.
Step 9, optional: read the health verdict
const health = await cs.api.get("/v1/audit-events/health");
console.log(health.actionable_status);Read actionable_status, one of verifying, healthy, catching_up, link_unverifiable, degraded, broken. The plain status field only carries healthy or degraded and hides the distinction you came for.
This read has a declared side effect. A head-of-chain failure opens an incident and records the segment as unverifiable. It is the one call in this recipe that changes state.
What this does not do
- It does not anchor on demand. There is no
POST /v1/audit-events/anchorsand no pass trigger anywhere in the 276 operations. Anchoring is the hourly background job. - It does not configure the anchor target.
AUDIT_ANCHOR_TARGET,AUDIT_ANCHOR_URL,AUDIT_ANCHOR_PATH,AUDIT_ANCHOR_SECRETandAUDIT_ANCHOR_INTERVAL_MSare read from the deployment environment. No route sets them, and on hosted CodeSpar you cannot. - It does not hand you the export signing key. Searching the document for key routes finds agent keys and Pix keys. The audit signing keys stay encrypted under
VAULT_MASTER_KEYand no operation exposes them. Without the key, step 8 verifies the chain and the manifest but not the HMAC. - It does not page through anchors.
GET /v1/audit-events/anchorstakeslimitandstatusand nothing else. Past 200 anchors, the older ones are unreachable from this route. - It does not run from the CLI.
codesparexcludes theaudit-eventsandauditgroups from its surface, and theorgsgroup that carries the deprecated/v1/orgs/{orgId}/audit/*aliases as well. - It does not run from a meta-tool. None of the 15 meta-tools has an anchor, verify or export action.
- It does not ship an installable offline verifier. The package that implements the checks in step 8 is marked private, so it is not on npm and an external auditor cannot install it. The code above is the whole algorithm, which is why it is spelled out rather than referenced.
- It does not document every refusal it can return.
anchor_secret_unconfiguredappears in no error page. The scope-gate 403 appears on none of these routes in the served document. Thex-export-row-countheader exists only in prose, never as a declared response header.
Hold a Spend for Approval
Write an approval-required rule that stops a spend before dispatch, poll the hold from the caller side, decide it as an operator, and read which refusal a retry can fix.
Projects, Keys and Environments
Ask the server which project a key acts in, read the environment from the project row instead of the key prefix, and tell an empty read apart from a wrong key.