Grade your own simulated people

POST /v1/reports

Three sections on a simulator. Diversity: do the personas collapse onto the same answers. Fidelity: does the person given to the model change its answer. Validity: does the simulated study reach a real panel's answer.

Unlike an eval this needs no suite of ours, because Diversity is computable from your outputs alone. Upload rows of {persona_id, item_id, output} with POST /v1/uploads, then pass the ref here. Add condition of "own" or "swapped" to your rows to unlock Fidelity, and a panel of {item_id, answer} to unlock Validity.

Every section is always present in the result. One that could not be computed carries status: "unavailable" and a reason naming exactly what to upload for it, so a missing section never reads as a pass.

Credential. An API key (jean_live_…) or a dashboard session token.

curl --fail-with-body -X POST "https://api-833252724876.us-central1.run.app/v1/reports" \
  -H "Authorization: Bearer $JEAN_API_KEY" \
  -H "Idempotency-Key: YOUR-UNIQUE-REQUEST-KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "outputs_ref": "gs://jean-artifacts/orgs/org_x/uploads/upl_y/predictions",
    "label": "v3 personas, wave 2"
  }'

Body

Field	Type	Required	Notes
outputs_ref	string	yes	An upload of rows {persona_id, item_id, output}, optionally with condition of "own" or "swapped" to unlock Fidelity.
panel_ref	string or null	no	An upload of rows {item_id, answer} from real people, which unlocks Validity. Optional; without it that section is reported unavailable rather than omitted.
label	string or null	no

Response · 202

{
  "attempts": 1,
  "created_at": "2026-01-15T09:30:00Z",
  "finished_at": "2026-01-15T09:30:18Z",
  "id": "job_q1w2e3r4t5y6u7i8o9p0",
  "label": "nightly",
  "org_id": "org_k3j2h4g5f6d7s8a9q0w1",
  "request_id": "req_z9x8c7v6b5n4m3l2",
  "result": {
    "hfm_sha": "9f2c1ab4d7e0",
    "metric": "accuracy",
    "n": 200,
    "n_submitted": 200,
    "n_unmatched": 0,
    "suite_id": "suite:smoke-v1",
    "suite_version": "1",
    "value": 0.734
  },
  "started_at": "2026-01-15T09:30:02Z",
  "status": "succeeded",
  "trace_url": "https://storage.googleapis.com/jean-artifacts/.../trace.json",
  "type": "eval",
  "versions": {
    "hfm_sha": "9f2c1ab4d7e0",
    "suite_id": "suite:smoke-v1",
    "suite_version": "1"
  }
}

Errors

Status	Meaning
401	Unauthenticated, invalid token or revoked key.
403	Forbidden, session required or org required.
422	Validation failed.
429	Quota or rate limit exceeded; error.details says which, and when.
500	Internal error; quote error.request_id when reporting it.

Every one carries the same envelope.

API overview · Full contract