From registerto your first signed record.
The same registration endpoint the dashboard uses — wrapped here as a single public walkthrough so a new operator can see the wiring before signing in. Five minutes from a registration curl to a SHA-256 record chained into your ledger.
Step 01 · Register
Register a model.
The registration endpoint mints a one-time pl_… API key (only its SHA-256 is stored) and returns paste-runnable Node + Python snippets that the dashboard already wires for you after the call. It's gated by your signed-in session — paste your __Secure-better-auth.session_token cookie in place of $YOUR_SESSION_COOKIE below.
curl -X POST \
https://getplumblineai.com/api/models/register \
-H "Content-Type: application/json" \
-b "__Secure-better-auth.session_token=$YOUR_SESSION_COOKIE" \
-d '{"name":"eval-claude","version":"0.1.0","owner_team":"ai-eval-team"}'Response envelope (mirrors ModelRegisterResult):
{
"model_id": "<your_model_id>",
"api_key": "pl_<64-hex>", // shown ONCE — only sha256 is stored
"snippets": {
"node": "POLSIA_API_KEY=… POLSIA_MODEL_ID=… POLSIA_MODEL_VERSION=0.1.0 POLSIA_API_BASE=… node -e \"import('@plumbline/eval-sdk').then(m=>m.recordEval({input:'hello',output:'hi',version:'0.1.0'}))\"",
"python": "POLSIA_API_KEY=… \\\n POLSIA_MODEL_ID=… \\\n POLSIA_MODEL_VERSION=0.1.0 \\\n POLSIA_API_BASE=… \\\n python -c \"from polsia import record_eval; record_eval(input='hello', output='hi', version='0.1.0')\""
}
}Step 02 · Install
Install the SDK.
Two paths to the same wire. The Node one ships in this repo today (sdks/eval-sdk/); the Python one is illustrative — the package isn't published yet, but the env-var stack already matches the dashboard's install shape.
POLSIA_API_KEY=pl_<polsia-api-key> \ POLSIA_MODEL_ID=<polsia-model-id> \ POLSIA_MODEL_VERSION=<polsia-model-version> \ POLSIA_API_BASE=https://getplumblineai.com \ node -e "import('@plumbline/eval-sdk').then(m=>m.recordEval({input:'hello',output:'hi',version:'<polsia-model-version>'}))"# Illustrative Python snippet — the 'polsia' package is not published yet.
POLSIA_API_KEY=pl_<polsia-api-key> \
POLSIA_MODEL_ID=<polsia-model-id> \
POLSIA_MODEL_VERSION=<polsia-model-version> \
POLSIA_API_BASE=https://getplumblineai.com \
python -c "from polsia import record_eval; record_eval(input='hello', output='hi', version='<polsia-model-version>')"Step 03 · Capture
Trigger your first evaluation.
The SDK does this wire for you on the Node path — paste-run the Node snippet above and skip the curl. Need to see the raw shape? Hit /api/eval-records directly with the bearer key the registration call minted. Body matches EvalRecordCreate; the response is the right-rail sample on this page.
curl -X POST \
https://getplumblineai.com/api/eval-records \
-H "Content-Type: application/json" \
-H "Authorization: Bearer pl_<polsia-api-key>" \
-d '{
"model_id": "<polsia-model-id>",
"run_id": "smoke-test-001",
"version": "0.1.0",
"input": {"prompt":"hello"},
"output": {"text":"hi"}
}'After these integration steps, inspect a published evidence record in the public evidence sample.
Troubleshooting
Find the failing step.
Registration uses your signed-in session. SDK record capture uses the API key issued for one registered model.
- SDK:
missing env: POLSIA_API_KEYorPOLSIA_MODEL_ID Likely cause: the key or model ID is unset, or the values came from different registrations.
Next step: set both from the same registration response. Keep the one-time plaintext key in your CI secret store; do not commit it.
- SDK:
missing envfor run or version Likely cause:
POLSIA_RUN_IDorPOLSIA_MODEL_VERSIONis unset and was not passed to the call.Next step: set the missing variable, or pass
run_idand/orversiontorecordEval(...).- Connection error or unexpected response
Likely cause: without
POLSIA_API_BASE, the SDK targetshttp://localhost:3000; a wrong app origin can also return a connection error, 404, or 401.Next step:set it to this app's origin from the install snippet. The SDK adds
/api/eval-recordsitself.- Registration:
401 Unauthorized Likely cause: the
/api/models/registerrequest has no current better-auth session cookie, or its session expired.Next step: sign in again and send the current session cookie. A model
pl_…API key is for record capture, not registration.- SDK:
recordEval 401 Likely cause: the bearer API key is invalid, mistyped, or not registered on this app. A valid key for another model reaches the model-ID check below instead.
Next step:use that model's exact
pl_…key from registration. If its plaintext was lost, it cannot be recovered; have a workspace admin delete and re-register the model, then replace the CI secret.model_id does not match the authenticated modelLikely cause: the submitted model ID does not belong to the model authenticated by the bearer key (the API returns 400).
Next step: set
POLSIA_MODEL_IDto the ID returned alongside that same key; do not mix credentials from registrations.
Next · Pick a direction
Pasted it. Now where?
See a real chain-record against the public demo model, or look at what running this in production actually costs.