Evidence and signed exports
When the product says something happened, why should your auditor believe it?
The honest answer is not "because we say so". The record is append-only at the database level, chained so that a rewrite is detectable, its chain head anchored outside the database every hour, and exported under a signature a third party can check offline with a public key — no RapidValue account, no endpoint of ours, no secret of ours. This page walks that chain link by link, including where it stops.
The chain of custody
Four steps, each only as good as the one before it: the record is written so it cannot be altered, chained so alteration is detectable, exported with the claims about it bound into the same signed payload, and verified with a key obtained by a route the artefact does not control.
1. The record is append-only
Every governance action — a grant, a revocation, an approval, a configuration
change, a login — writes a row to audit_event. On PostgreSQL a trigger fires
BEFORE UPDATE OR DELETE on that table and raises an exception, so even an
application user with full table privileges cannot alter or remove a row. Rows
are inserted once, complete.
Each row also carries a chain_hash: SHA-256(previous_hash + canonical_row),
over the event's identifier, tenant, timestamp, type, actor email, subject and
payload, with payload keys sorted so JSON ordering cannot change the digest.
Each tenant has its own chain. GET /api/v1/audit/verify-chain walks it and
reports every link whose recorded hash does not match a freshly computed one;
altering one row breaks its hash and every hash after it. For each break it also
says what kind it is: fork_or_reorder when another chained row (or the start of
the chain) explains the stored hash, as with a pre-lock concurrent write, or
unexplained when nothing does. It reports whether any later row chains off the
broken one, and the point since which the chain has been intact. A
fork_or_reorder break that nothing is chained after is counted separately as
unanchored: no later row vouches for its content, so it is reported as unproven,
not as intact. intact itself still means "no break at all".
Two details of how the chain is written are worth stating, because they are where this kind of design usually goes wrong.
Writes for one tenant are serialised. Computing the next hash means reading the tenant's latest committed hash and inserting off it. Two concurrent requests would read the same predecessor and fork the chain — two rows claiming one ancestor, the exact ambiguity a chain exists to prevent. A per-tenant PostgreSQL advisory lock, scoped to the transaction, makes that pair atomic.
The chain is sealed immediately before commit, not when the event is recorded. Many actions write their audit row and then call a target system — provisioning a grant, disabling an account, running a sync — and those calls can block for tens of seconds when a target is unreachable. Taking the lock where the row is recorded would hold it across that call, and every other audited write for that tenant would queue behind it: one unreachable system stalls the tenant. So recording an event queues it, and the hashing and insertion happen in the last moment before commit. The rows still go in inside the caller's own transaction, so an action that rolls back writes no audit row — phantom evidence would be worse than a missing one.
2. The chain head is anchored outside the database
A hash chain on its own proves order and completeness only to someone who trusts the database: an administrator who rewrites a row can recompute every later hash, and the chain check would then report the forgery as intact. So the chain is also anchored.
Every hour (the interval is a platform setting), the scheduler takes each
tenant's chain head (the latest event's identifier and chain_hash, the number
of chained events and the head's timestamp), signs that document with the
tenant's anchor key and writes it to an external store. Because each
chain_hash covers every predecessor, rewriting any event at or before an
anchored head changes that head's hash (reported as head_rewritten), and a
deletion or back-dated insert changes the count (count_mismatch).
GET /api/v1/audit-packs/anchors/verify reads the anchors from the store
itself, not from the database's index of them, so deleting index rows hides
nothing, and checks each one against the live chain. Offline, the verifier does
the same with only the anchor key's public half and an unfiltered export (see
section 6). The store is meant to be write-once: an S3 Object Lock bucket in
compliance mode, where nobody, the storage account's owner included, can
overwrite or delete an anchor before its retention date. The window in which a
rewrite goes undetected is one anchoring interval.
Anchoring is a deployment setting: until a store is configured, no anchors are written and the anchor list stays empty, so the chain is then only as strong as section 1 describes.
3. Signed exports
GET /api/v1/audit/export (administrators and auditors only) streams events as
NDJSON, one object per line, each carrying its own chain_hash. The last line
carries the signatures. It records the event count, so a body with lines removed
fails on the count before it fails on a signature, and it holds an Ed25519
signature over a short statement: the SHA-256 of every preceding byte (newlines
included), the event count and the format. The signature line is not part of the
signed body. An HMAC over the body is still included, with its response header,
because existing integrations read it; only RapidValue can check that one.
4. Evidence packs: what is signed
An evidence pack is a ZIP produced against a framework template and a time window — SOX ITGC, ISO/IEC 27001:2022, HIPAA §164.312, GDPR Article 32, DORA, SOC 2, NIS2, an identity-lifecycle template, a just-in-time privileged access template and an AI-agent governance template ship today. It holds a summary page, one JSONL file per section, a manifest, a signature file, a verification recipe and the verifier script itself.
It also holds what a person reads without a script: a report.pdf with the pack's figures at a
glance, a contents list with the state of each section, the signing key and the signature check,
and per section its first 25 rows with names; and one CSV per section under data/ with names
beside the identifiers. Both are listed in the signed manifest like every other member. The
summary page links to them and loads nothing from the network.
The signature does not cover the ZIP bytes. It covers the manifest, serialised canonically — sorted keys, fixed separators, UTF-8. Archive member order, compression and per-entry timestamps all vary between runs, so a signature over the container could never be recomputed by the person holding the file, and a signature nobody can reproduce proves nothing.
The manifest is built to be worth signing:
- A SHA-256 and byte length for every other file in the archive, so one signature covers the contents transitively: altering a JSONL line contradicts the signed manifest.
- The claims made about the evidence, repeated inline — each section's title, kind, framework citation and row count. Swapping a cited control or under-reporting a row count breaks verification exactly as hard as editing the data. Evidence that could be relabelled undetectably is not evidence.
- The algorithm and key id that sign it, so stripping the signature or swapping in another key is detectable rather than looking like an unsigned pack.
- Its canonical form, so a future layout change makes an older verifier report "unknown canonical form" rather than a mismatch it cannot explain.
Regenerating the same window produces a different but equally verifiable pack, not an identical one: the generation timestamp moves, and snapshot sections such as the risk-score distribution describe the estate as it is now. We do not claim byte-identical regeneration, because it is not true.
The auditor's Home shows evidence readiness: one row per pack template, built from the packs actually generated, worst first. A pack is failed when its last run failed (the older completed pack stays downloadable), outdated when it is more than 100 days old, truncated when it holds fewer rows than its window had, not generated when it was never run, and otherwise ready. A pack generated for one agent never makes a framework ready. Each row also names the sections with no rows over the last 90 days; that is a fact about the estate, not a fault, so it does not change the status.
5. One signature, a public key, older packs still verifiable
Packs in canonical form v2 carry a single
Ed25519 signature over the manifest bytes, made with a per-tenant key whose
public half is published. Anyone holding that public key can check it, with the
verifier below or with OpenSSL 3 alone (openssl pkeyutl -verify -rawin; the
recipe in the pack shows the command). Ed25519 is deterministic, so there is no
per-signature random value to get wrong.
A v2 pack carries no HMAC. Older packs (form v1) carried an HMAC-SHA256 with a platform secret beside an ECDSA P-256 signature; both still verify, so a pack signed before the change does not need regenerating. The HMAC key is a dedicated evidence secret: it is no longer the secret that signs session tokens, and a production deployment refuses to start when the two are the same.
If no signing key can be resolved when evidence is produced — a vault outage, say — the pack falls back to the older HMAC shape and makes no claim to a signature it does not have. That raises a critical advisory, because the loss is unrecoverable: a pack cannot gain the third-party signature afterwards, only be regenerated.
6. Verifying without us
backend/scripts/verify_audit_pack.py is a single file with no RapidValue
imports, and every pack carries a copy of it as a signed member. It needs the
Python standard library, the cryptography package and a public key. It does
not need a database, an account, network access, any RapidValue code, or any
secret of ours. It verifies v2 and v1 packs, NDJSON exports and chain anchors
(--anchor, optionally --against-export with an unfiltered export), checks every
member digest, refuses extra or missing files, and exits non-zero on any failure.
--show prints the signing envelope without verifying, so an auditor can read the
key id first.
⚠ The public key embedded in the artefact is not a trust anchor.
Every pack and anchor carries a copy of the signing public key. It is there so you can see which key to ask for — nothing more. Anyone able to alter the pack can alter that copy and re-sign with a key of their own, so verifying against the key the file supplies proves only that the file is internally consistent. Obtain the key separately, from the tenant's published signing keys (an administrator exports them as
.pemfrom the Compliance page, or readsGET /api/v1/audit-packs/signing-keys), and match it by key id. The standalone verifier refuses to run without a key you supply, and treats disagreement with the embedded copy as a hard failure.
An upload endpoint also verifies a pack in memory and discards it, so returning one for checking never creates a second copy of the evidence. For an outsider the offline script is stronger, because it does not require trusting the party being audited to run the check.
7. Keys
Each tenant has one active signing key per purpose: one for evidence packs
and the audit export, one for chain anchors (both Ed25519), and the older
ECDSA P-256 key that still signs the BI export feed and the GDPR subject export.
A key id (apk- followed by part of the public key's SHA-256) is derived from the
public key itself, so an identifier cannot drift from the key it names. The
endpoint that verifies an uploaded pack does not accept an anchor key.
Rotation retires the active key of one purpose and mints a new one, and is never destructive: the retired key and its published public half are kept, with no delete path, so packs and anchors signed years earlier keep verifying under the key id that actually signed them. The published key list therefore includes retired keys. A rotation is itself an audit event.
Private keys are generated in the application and stored only in the vault (including a customer's own vault, where one is configured); no API response carries private key material — the read schema has no field that could. No key is derived from a platform secret. The vault is opaque storage rather than a signing service, so signing does bring the key into the application process for the duration of the operation. We state that rather than imply a hardware-backed guarantee we do not have.
8. Erasure and retention
A GDPR Article 17 erasure pseudonymises the identity — names, email, employee id, source identifiers and free attributes cleared or replaced with a stable pseudonym, linked accounts anonymised, and the login account with its credentials and MFA secret deleted outright.
Audit content is retained under Article 17(3)(b), processing necessary for
compliance with a legal obligation. The reasoning matters more than the
citation: an access-governance trail records who approved what, for whom and
when. Erasing it would not protect the data subject so much as destroy the only
means of showing that decisions about them were made properly — including the
record of the erasure itself. Grants and access requests are retained on the
same basis, as business records. The rule survives tenant deletion:
audit_event is the one table a hard delete does not touch, and the tenant
identifier is never reused afterwards.
What this does not prove
- There is no independent time source yet. An anchor proves that a chain head existed by the time it was written to the write-once store; there is no third-party timestamp (RFC 3161) on it.
- The scheme has not been externally attested, certified or audited. It is documented here so you can evaluate it yourself — the point of the offline verifier.
- A verifying pack does not make anyone compliant. It shows that a specific set of records, with specific claims attached, is what RapidValue produced for a stated window. Whether those records satisfy a control is your assessment.
- Some sections are still capped. Audit events, lifecycle events, certification decisions and privileged grants are streamed to disk in a fixed order, oldest first, with no row limit, so a quarter is always complete. The agent, execution and runtime-decision sections keep a cap. A section that hit its cap is declared, never silent: the signed manifest marks it as truncated with the number of rows available, the pack's index page says "Incomplete", and the report run and its delivery mail carry the same flag. The preview reports whether a window would exceed a cap.
- The risk snapshot covers people who have not left. It names its population and how many terminated identities it left out.
- Records written before the chain existed have no hash and are skipped by chain verification, which reports total and chained events separately so the gap is visible rather than implied.
- A section that fails to render is marked as failed in the signed manifest, per section and for the pack, and the summary page says "Not generated" for it, rather than a row count of zero that reads like an absence of findings.
- If sealing the chain itself fails, the rows are dropped with a critical log entry and the business transaction still commits. Platform availability is chosen over completeness of one audit row — loudly, never silently.
Further reading:
- Tier-3 Hybrid Architecture — where the platform runs, and what never crosses the wire
- API Access Governance — scoped machine credentials, and how their calls land in the same trail
- Reconciliation — Expected versus Actual, the engine most of these events describe