PLU-008 · Evidence FAQ

A reviewer’s guide to evidence and governance.

See what Plumbline captures, how records connect across evaluations, and where your team still needs its own clinical, security, and compliance judgment.

Questions from reviewers

Nine questions about the record

What does replayable evidence mean in Plumbline?
Plumbline records evaluation inputs and outputs with a model, run, version, and capture time. Each evaluation record has a hash and a link to its predecessor, so a reviewer can walk the captured sequence and check the links. A broken link is surfaced for review. This replay follows the stored evidence path; it does not rerun the model or establish that information supplied by a source system was true. The public evidence sample is fictional and illustrative: it shows a record format, but does not verify source information, establish compliance, or demonstrate clinical performance.

View the fictional, illustrative evidence sample

What is signed on an evaluation record?
The Node SDK returns a record ID, a 64-character hash, a creation time, and the preceding record hash. Audit exports also calculate a SHA-256 fingerprint and check that each record points to the prior hash in its run. These checks help a reviewer inspect record linkage; they do not independently prove the truth of captured inputs, identify a human signer, or guarantee that an upstream system was complete.
How can teams review evaluation history across model versions?
Captured records include model, run, and version context, allowing reviewers to compare recorded evaluations over time and connect a result to the version that produced it. An export can show the available records in their run sequence. The history only covers evaluations that were sent to Plumbline and retained in the selected scope; it cannot fill gaps in earlier or uncaptured activity.
What does the drift ledger show?
Drift events can be reviewed alongside evaluation records, with fields such as the metric, severity, run, and detection time included in audit exports. This gives teams context for investigating movement and recording follow-up. A drift event is a monitoring signal, not by itself a finding of harm or a decision about whether a model should be deployed; the team sets its criteria and determines the response.
How does the evaluation SDK fit an existing workflow?
The repository includes a zero-dependency Node SDK with a recordEval function. It sends the input, output, model, run, and version to the evaluation-record endpoint and returns the record ID, hash, timestamp, and predecessor link. It is designed to capture an evaluation from the workflow your team already runs; it does not host model inference. Keep the registered API key in your CI secrets manager. The Python snippet in the quickstart is illustrative; that package is not published.
How can healthcare teams use this evidence in governance reviews?
For workflows such as prior-authorization review, teams can bring model and version context, evaluation results, thresholds, decision rationale, reviewer notes, and record links into one review path. The public healthcare example uses fictional demo data; it is not real patient information, a customer deployment, or evidence of clinical performance or outcomes. Plumbline supports review of captured records, while clinical decisions remain with the responsible care and governance teams.
What can an evidence package contain?
A scoped audit export can include evaluation records and their hash links, model and version details, available drift events, alerts and resolution notes, reviewer sign-offs, and access or retention context. The contents depend on the records available for the selected model and scope. An empty or unavailable category stays visible as such; an export does not create evidence for an event Plumbline did not capture.
How does framework evidence mapping help a governance team?
Plumbline organizes evidence against selected framework topics, including NIST AI RMF, ISO/IEC 42001, and EU AI Act material in the product. The mapping helps reviewers find related evaluation, drift, replay, and follow-up records. It is an evidence-organizing aid, not a determination that a system or organization meets a framework, and it does not replace applicability analysis or legal review.
What can Plumbline establish, and what remains the team’s responsibility?
Plumbline can preserve and help verify relationships among records that were captured. It cannot prove that source data was accurate, that every relevant event was recorded, or that a healthcare workflow is clinically safe, HIPAA-compliant, legally compliant, certified, or regulator-approved. Your organization remains responsible for its data scope, access and retention controls, clinical judgment, and decisions about which obligations apply.

Explore the evidence

Walk through a sample record path.

The public sample uses fictional healthcare demo data to show decision context, evaluation history, record links, and reviewer notes. It illustrates the product surface; it is not a customer record or a verification of a live deployment.

Open the evidence sample