Skip to main content

Quickstart

Five steps. The only thing you need first is a browser.

1. Get a key​

Sign in at jeantechnologies.com/dashboard with Google or an email link. You get an organisation of your own on first sign-in, and every key, job and score below belongs to it.

Create a key in the dashboard and copy it. It is shown once and never again; if you lose it, revoke it and make another.

export JEAN_API_KEY="jean_live_..."

Keys are created from the dashboard, not from a key. An API key cannot mint another API key, so a leaked key cannot be used to grow itself a longer life.

2. See what you can be scored against​

curl --fail-with-body "https://api-833252724876.us-central1.run.app/v1/suites" \
  -H "Authorization: Bearer $JEAN_API_KEY"

Each suite names a metric, how many items it holds, and whether it returns per-item feedback. More about suites.

3. Submit predictions​

One prediction per item, with the item id the suite gave you. Send an Idempotency-Key and a retry after a timeout returns the original job instead of queueing a second one.

curl --fail-with-body -X POST "https://api-833252724876.us-central1.run.app/v1/evals" \
  -H "Authorization: Bearer $JEAN_API_KEY" \
  -H "Idempotency-Key: $(uuidgen)" \
  -H "Content-Type: application/json" \
  -d '{
    "suite_id": "suite:smoke-v1",
    "label": "first try",
    "predictions": [
      {"item_id": "item-00", "prediction": "whatever your model said"},
      {"item_id": "item-01", "prediction": "whatever your model said"}
    ]
  }'

You get back a job with "status": "queued". Scoring happens on a worker, not in your request. Those two predictions will score zero, which is the point: the number comes from comparing what you sent to data you do not have.

4. Poll the job​

curl --fail-with-body "https://api-833252724876.us-central1.run.app/v1/jobs/$JOB_ID" \
  -H "Authorization: Bearer $JEAN_API_KEY"

queued → running → succeeded, or failed with an error, or cancelled if you cancel it. Poll every couple of seconds. A first submission after a quiet period waits a few seconds longer while the worker starts.

A succeeded job carries a result: the metric, the value, how many items were scored, and how many you sent that the suite did not recognise. If the suite allows it, per_item says which items you got right — never what the right answer was.

5. Read the trace​

curl --fail-with-body "https://api-833252724876.us-central1.run.app/v1/jobs/$JOB_ID/trace" \
  -H "Authorization: Bearer $JEAN_API_KEY"

Every job records what ran, in what order, and how long each step took, so a number you disagree with can be argued about with evidence.

Then​