Skip to main content

Quickstart

Five steps. The only thing you need first is a browser.

1. Get a key​

Sign in at jeantechnologies.com/dashboard with Google or an email link. You get an organization of your own on first sign-in, and every key, job and score below belongs to it.

Create a key in the dashboard and copy it. It is shown once and never again; if you lose it, revoke it and make another.

export JEAN_API_KEY="jean_live_..."

Keys are created from the dashboard, not from a key. An API key cannot mint another API key, so a leaked key cannot be used to grow itself a longer life.

2. See what you can be scored against​

curl --fail-with-body "https://api-833252724876.us-central1.run.app/v1/suites" \
  -H "Authorization: Bearer $JEAN_API_KEY"

Each suite names a metric, how many items it holds, whether it returns per-item feedback, and whether it publishes its items. More about suites.

The items are what you predict. Fetch them by suite, paged with cursor; each carries the item_id you return verbatim and an input holding whatever you need in order to predict it, never the answer. On some suites the input is empty and the id is all you get: the suite describes what to predict for it, and the rest is your model's job. Pick a suite_id from the list above and use it below.

curl --fail-with-body "https://api-833252724876.us-central1.run.app/v1/suites/$SUITE_ID/items?limit=3" \
  -H "Authorization: Bearer $JEAN_API_KEY"

3. Submit predictions​

One prediction per item, with the item id the suite gave you, in the suite's own shape: a ranked list of ids, best first, for a next-item suite; the answer itself for a survey suite. Send an Idempotency-Key and a retry after a timeout returns the original job instead of queueing a second one.

PREDICTIONS=$(curl --fail-with-body -s "https://api-833252724876.us-central1.run.app/v1/suites/$SUITE_ID/items?limit=3" \
  -H "Authorization: Bearer $JEAN_API_KEY" | jq '[.items[] | {item_id, prediction: []}]')

curl --fail-with-body -X POST "https://api-833252724876.us-central1.run.app/v1/evals" \
  -H "Authorization: Bearer $JEAN_API_KEY" \
  -H "Idempotency-Key: $(uuidgen)" \
  -H "Content-Type: application/json" \
  -d "{\"suite_id\": \"$SUITE_ID\", \"label\": \"first try\", \"predictions\": $PREDICTIONS}"

You get back a job with "status": "queued". Scoring happens on a worker, not in your request. Those three empty rankings will score exactly zero, which is the point: the number comes from comparing what you sent to data you do not have. Put your model's rankings in prediction and the same call scores them. Ids the suite does not know are counted in n_unmatched and scored on nothing; a submission that matches nothing fails with invalid_predictions rather than scoring zero.

4. Poll the job​

curl --fail-with-body "https://api-833252724876.us-central1.run.app/v1/jobs/$JOB_ID" \
  -H "Authorization: Bearer $JEAN_API_KEY"

queued → running → succeeded, or failed with an error, or cancelled if you cancel it. Poll every couple of seconds. A first submission after a quiet period waits a few seconds longer while the worker starts.

A succeeded job carries a result: the metric, the value, how many items were scored, and how many you sent that the suite did not recognize. If the suite allows it, per_item says which items you got right, never what the right answer was.

5. Read the trace​

curl --fail-with-body "https://api-833252724876.us-central1.run.app/v1/jobs/$JOB_ID/trace" \
  -H "Authorization: Bearer $JEAN_API_KEY"

Every job records what ran, in what order, and how long each step took, so a number you disagree with can be argued about with evidence.

Then​