Downstream defensibility, side by side.
The question isn’t whether you have evidence.It’s what the evidence actually rests on.
Most governance programs produce evidence on infrastructure the buyer doesn’t own and can’t independently re-run. Plumbline writes every evaluation record onto a per-workspace HMAC chain at inference time, so what the auditor walks back is the same record the model produced — not a reconstruction joined together downstream. The page below is a structural comparison, not a personality contest.
§01 · Side by side
Six dimensions. One defensible answer each.
Every row below is a structural distinction in how evidence is captured, signed, walked back, packaged, and verified. The row favors Plumbline on the dimension of downstream defensibility — the question the auditor actually asks. The other column stays structural; the gap is named in operational terms.
| Dimension | PlumblineContinuous evaluation · HMAC-signed ledger · chain-replayable. | Compliance-theater governance toolkitsGeneric ML governance · post-hoc log aggregation · vendor-mediated replay. |
|---|---|---|
Evidence captureWhen is the record written, and from what? | Captured at inference time: the input is hashed, the | Captured post-hoc. Logs are aggregated from inference, observability, and feature-store traces after the fact, then joined into a record at audit time. The reconstructed record is only as complete as the upstream pipelines that fed it. |
Tamper-evidenceWhat makes an edit detectable, and at what point? |
| Provider-controlled signing infrastructure: opaque key custody, opaque rotation cadence, opaque audit of the signing key itself. Tamper-evidence is verifiable only with the vendor — a replay path the buyer cannot read end-to-end. |
Drift detectionWhat does "the model drifted" rest on? | Drift is re-derived against the | Drift lives in a vendor dashboard populated from per-bucket telemetry. Calibration history is reconstructed from those buckets at audit time — readable only through the vendor surface, with no guarantee the buckets themselves cover the same record set the auditor is asking about. |
ReplayabilityCan the score be re-derivable from the record alone? | Each record carries a | Replay is mediated by the vendor platform: the auditor (or buyer) must request a re-run through the control plane, with whatever inputs the vendor surfaces. The path from record → bundle is vendor-gated; the auditor cannot verify it independently. |
Audit packagingWhat does the buyer hand to the auditor? | A tamper-evident audit PDF: a cover with a | A vendor-rendered exporter producing a dashboard bundle or vendor-locked PDF. Hashing, signing, and chain provenance are abstracted behind the vendor surface; the buyer cannot reconstruct the bundle independently of the platform that produced it. |
Signing & verificationWho holds the keys, and who can verify? | Signing keys are bound to the | Keys are platform-managed and platform-verified. Verification effectively requires the vendor — the buyer validates by trusting the platform the records were produced on, with no independent re-derivation path the auditor can run themselves. |
§02 · The structural walk-back
A regulator opens a record from two quarters ago.Two paths through. Same question. Different answer.
- 1.Where did the record come from?On Plumbline, inference wrote the record onto the per-workspace chain at the moment the model surfaced it. On a governance toolkit with post-hoc log aggregation, the record was assembled later — the auditor reads an inferred reconstruction joined from telemetry streams.
- 2.Has anything changed since capture?On Plumbline, the HMAC-SHA-256 over canonical-JSON breaks visibly at the next record; detection runs at fetch. On a vendor-controlled signing infrastructure, key custody and rotation are opaque — the buyer cannot re-verify end to end.
- 3.Did the calibration drift before or after this decision? On Plumbline, the drift ledger re-runs against the same chain. On a vendor dashboard, drift is reconstructed from telemetry buckets that may not cover the same record set — the auditor has to trust the vendor’s reconciliation.
- 4.Can the score be re-derived from the record? On Plumbline, every record carries scorer id, scorer parameters, model id + version, and the input digest — the bundle is reproducible without a call back to engineering. On a vendor-mediated replay path, a re-run requires a platform round-trip the buyer cannot bypass.
- 5.What lands on the auditor’s desk? On Plumbline, a tamper-evident audit PDF with a SHA-256 footer of the canonical payload and an
X-PDF-SHA256response header on the download — verifiable with sha256sum. On a vendor-rendered bundle, the auditor reads a vendor-locked export whose provenance they cannot independently reconstruct. - 6.Can a third party verify the record tomorrow? On Plumbline, signing is anchored to the buyer workspace and verification runs through the public /verify page — pure computation, no oracle. On a platform-managed signing surface, verification effectively requires the vendor.
§03 · Cross-links
Read next, by structural question.
Start with a bounded evidence proof, or send the procurement team the packet.
The Evidence Pilot is $1,000 one-time for one workflow and one model over 30 days. If you are already in procurement, send the intake form and we'll route the right Team or Enterprise conversation.
§06 · Pricing
Ready to see this on your models?
Start with the $1,000 Evidence Pilot: one workflow, one model, and 30 days ending in a replayable evidence package. Plumbline captures forward from the moment you connect, and the full pilot fee is credited toward Regulated Team.