Say vs do: how much of human behavior lives in what people actually did?
Language models learn from what people write. Recommenders learn from what people do. The open question is whether a model trained on both understands people better than either alone.
Read the pieceChoosing a vendor
Who does what, and how to check their numbers.
Listen Labs, Outset, Simile, Fairgen and Jean: AI market research platforms compared
Two kinds of tool are merging: AI interviewers and digital twins. Here is what each platform leads with, who it sells to, and how to choose.
How accurate are digital twins? Every published number, side by side
Aaru says 3.53 points. Artificial Societies says 86%. Simile's founders say 85%. Semilattice says 0.775. These are four different measurements, and here is what each one means.
How to evaluate synthetic respondents and digital twins
Seven checks a buyer can run before trusting synthetic results, and the benchmark that matters most: how consistent real people are with themselves.
Synthetic respondents are the Wild West. It is time for a standard.
Every vendor reports accuracy its own way, the same words mean different things, and published studies disagree by a factor of four. Buyers need one bar to hold everyone to.
How digital twins work
What they are, how they are built, and where they fail.
How to build digital twins that behave like real people
The stages that decide quality: who you interview, how deeply, what model the twins run on, how you check them, and how you keep them fresh.
Digital twins, synthetic respondents, AI personas, user simulation: what the terms actually mean
Vendors use these words loosely. The useful question is what each unit is built from: one real person, or a description of a group.
What a digital respondent is, and is not
A model of a person that answers the way they would. Good for a thousand quick reads, bad as a substitute for ever talking to anyone.
Where synthetic respondents fail, and where they work
The documented failure modes, with numbers: too little variance, flattened groups, agreeable answers, sensitivity to wording. And the jobs they do well.
Training AI on people
Simulated users, and the data behind them.
Meta trained coding agents against a simulated user. The user was a prompt.
AutoInteract made small coding agents much better by having them practice on a simulated person. The simulator was a general model with a tuned prompt. We reran that step and found a trained simulator behaves more like real people.
Say vs do: how much of human behavior lives in what people actually did?
Language models learn from what people write. Recommenders learn from what people do. The open question is whether a model trained on both understands people better than either alone.
Running a study
A price test and an interview, start to finish.