How Might We Teach Computers to Understand Us?

Placeholder. This is a working summary of a paper we are drafting, Toward a Foundation Model of Human Behavior. It stands in for the paper while the paper is finished and will be replaced by it. It supersedes our earlier note on the same reading.

Psychology, cognitive science, sociology, anthropology, and economics have each tried to understand the human mind and the processes that produce our behavior. The goal has largely evaded them. Behavior is high-dimensional and context-dependent, it is observed only through sparse and fragmented traces, and the verbal theories built on top of those traces underdetermine prediction.

Deep learning has reached goals of this kind in other domains. Given a large corpus and a generic method, it matches or exceeds human ability at tasks that resisted every hand-built approach. The 2024 Nobel Prize in Chemistry is the instructive case. AlphaFold solved a fifty-year-old problem by supervised training on solved structures, and the models that followed it are foundation models in the sense below: a language model trained only to fill in masked residues of protein sequences learned to predict structure from a single sequence. Understanding of the protein fell out of predicting its sequence.

The paper asks whether the same holds for people, with deep learning models, and specifically large language models, as the candidate. This post is the argument in short form.

What it means to understand

Understanding cannot be cleanly attributed to machines, and the discussion slides quickly into epistemology. We adopt a working criterion instead. A computer understands a person insofar as it can predict or simulate what that person will do in any situation. This is Newell's criterion for a unified theory of cognition: one model that behaves as an almost arbitrary function of the environment, rather than a separate theory per task.

Let h_u be the recorded stream of a person u: the actions they took and the situations they took them in, as one sequence. The model is a single function, trained end to end, that predicts the next action from the stream:

Computing it forces the stream through an internal summary. The network compresses h_u into a state

and predicts from that state. Nothing about z_u is designed or trained separately. It is whatever summary of the person the model learns because prediction demands one. Prediction scores the next action against what the person did. Simulation samples it and feeds the sample back into the stream to roll forward, and when the simulated individual is a real person, simulation is graded as prediction. A good z_u carries predictive power across many tasks, including tasks absent from training, which we write as expected performance over a set of tasks T:

"More general" means nothing until that set is fixed, which is what a benchmark does. One more thing to set aside. Deployed systems hold a finite window over fragmented data, and memory systems and context engineering bridge the gap. They matter because the trained model fixes an upper bound on performance and supplying the right context is how a system reaches it. The paper is about the bound.

Four contributions

The paper does four things, in order.

1. One definition. Roll the definitions the fields have offered into one checkable definition of a foundation model of human behavior.
2. A survey. Survey three fields building toward it: human-computer interaction, simulation, and prediction and recommendation.
3. A common formula. Formalize their convergence. Each field's formula is reduced to one next-action model, with what each field keeps of the person's stream made explicit.
4. Directions that complete the circle. Outline the steps back to the goal, a computer that understands people, reached by unifying the data held in separate silos, in one language, into one model.

The first and the third are the ones a reader might call obvious once stated. We think they are worth stating anyway, in a form that can be checked, because the fields have not stated them and do not cite one another.

One definition

The term foundation model was coined in 2021: any model trained on broad data, generally with self-supervision at scale, that can be adapted to a wide range of downstream tasks. The definition has three checkable clauses, broad data, generic self-supervision, and adaptation to many tasks. The same report names the two properties that make such models significant: emergence, behavior induced rather than constructed, and homogenization, one model coming to underlie many applications.

The fields we survey have each offered their own name for the human version. A foundation model of human cognition. A general user model. A large behavioral model. A digital twin. Silicon sampling. Around them sit false friends that share the words and model something else: an agent's control policies, a digital human's appearance, the brain's response to stimuli. Our definition keeps what the near synonyms share and drops what is particular to one field. A foundation model of human behavior is a model that

• D1is trained on behavioral traces: timestamped records of actions people actually took, in context;
• D2under a generic self-supervised objective, not a task-specific supervised target;
• D3such that training learns a person-level representation z_u = f(h_u) from the person's history alone;
• D4and the one resulting artifact serves many tasks: prediction, ranking, simulation, profiling, and so on.

D2 and D4 carry over the foundation-model clauses. D1 and D3 say what the data and the product are. Next-action prediction is the leading candidate objective in the literature, and it is not part of the definition.

Does a language model already qualify? Yes, with limits. Text is human-generated, so a model of text is a model of human behavior, and the record bears this out, from theory-of-mind performance to reproduced behavioral experiments. But these models were not built for the role. Post-training collapses them into a single helpful assistant, narrowing the human distribution that pre-training absorbed. Pre-training sees documents with no person attached, never the per-person stream of D3. And text records what people say, not what they do. Whether these differences are fundamental, or a language model with the right context already suffices, is the question the rest of the paper turns on.

Three fields

With the definition in hand, we survey three fields in which models meeting it, in whole or in part, have been built. Each is taken in the same order: what it aims at, what has been built, how it is measured, and where it stops.

Human-computer interaction

The user-modeling tradition in HCI is as old as interactive computing. The first user-modeling paper, in 1979, built stereotype models of individual users in order to recommend novels. The aim has not changed: a system that serves a person should maintain a model of that person. Three forms are in use today. Commercial assistants keep curated user memories built from chat histories. General user models such as GUM take screenshots of any computer use and produce confidence-weighted natural-language propositions about the user's preferences, context, and intent, held on device; nothing is trained. Next-action predictors such as LongNAP train: from a month of screen recordings per user, a vision-language model learns by reinforcement to retrieve from a per-user memory and predict the user's next actions, rewarded by judged similarity to what the user actually did.

The memory m_u is an engineered stand-in for the learned z_u of Eq. 2, read by a pretrained model, with c_t the current screen. Neither system has a standing benchmark; each is graded against its own users. Shuffling the history's order costs accuracy, which is the sequential signal the equation assumes. The stream here satisfies D1 for one person at a time, with consent and depth the other fields lack, and a representation the user can read and correct. The published models train on tens of users. Population-scale training on such streams has not happened, and nothing about the method forbids it.

Simulation

Simulating people is an old ambition. What is new is the agentic instantiation: a language model given memory, reflection, and planning, placed in an environment with other such agents, produces believable individual and group behavior. Economics arrived independently, re-running classic behavioral experiments on language-model subjects and checking the results against the published human ones. Two lines are in use. The prompted line supplies the person in context: a demographic profile, or a two-hour interview transcript. The trained line moves the person into weights. HumanLM aligns a latent user state with the person's actual responses rather than imitating surface text; HumanLLM fine-tunes on the longitudinal public traces of 282,000 accounts; OdysSim builds a foundation model for behavior simulation outright. Training is displacing prompting.

A response y to a situation s given a supplied persona π_u, rather than a learned representation. The yardsticks are the most person-grounded of the three fields. Park and colleagues interview 1,052 people and grade the agent by normalized accuracy: its accuracy on held-out survey responses divided by the accuracy with which the participants replicate their own answers two weeks later. Interview-conditioned agents reach 0.83 of that ceiling. Three limitations. Grounding: a described persona is not the person, and prompted models reflect the opinions of some groups far more than others. Sensitivity: simulators diverge from humans under small rewordings of the same task. Confidence: no simulator reports calibrated confidence in what it predicts.

Prediction and recommendation

Recommendation predicts what a person will engage with next, from what they and others engaged with before. Its history is four pivots, each answering a limit of the last. Content-based filtering matched item features to a profile of what a person had liked. Collaborative filtering used other people's reactions instead, and worked because the signal is cheap and everywhere. Matrix factorization made the person a learned vector, a form psychometrics had written decades earlier. Deep learning and then sequence models made the person the hidden state of a model trained to predict the next item, and that state was pretrained and transferred: across tasks, across a retailer's surfaces, and as one user embedding serving a whole platform. The term foundation model was not yet in use; the practice was. The generative turn brought the whole language-model recipe. TIGER replaced item IDs with semantic IDs, HSTU recovered power-law scaling on raw event streams, and PLUM, OneRec, and rankers at Netflix and JD put generative models into production against mature baselines.

This is the one field that pretrains on behavior, and its pipeline mirrors the language model's: build a tokenizer, pretrain by next-token prediction over the event stream, roughly 18 billion samples a day at Kuaishou, then post-train with rewards. Here z_u is the sequence model's hidden state, exactly the f_θ(h_u) of Eq. 2, learned from behavior at population scale. It is also the one field that chooses its token, and the tokenizer is where much of the downstream performance is decided; we wrote about that choice separately in How Much Catalog Fits in a Semantic ID. Generalization has been demonstrated within one platform's surfaces and, through content tokens, across catalogs. It has not been demonstrated beyond that, which is not the same as being impossible. The field's own description of itself is the point to hold: constrained by isolated data, today's recommenders operate as domain specialists.

One model, three settings

Each field's models satisfy the definition. HCI's next-action predictors train on a person's screen stream (D1), by next-action prediction (D2), into a per-user memory and weights (D3), serving prediction and assistance (D4). Trained simulators train on elicited responses and public traces, by predicting the response, into weights, serving prediction and simulation. Recommenders train on interaction logs, by next-item prediction, into a hidden state, serving retrieval, ranking, and cold-start scoring.

One difference the training sections expose: only recommendation pretrains on behavior. HCI and simulation post-train a language model on it, so strictly, in those two fields the foundation model is the language model and the behavioral stage is adaptation. Recommendation has built a foundation model from behavior; the other two have adapted one to it. The question that remains is whether one model underlies the three, and the formulas say it does.

The stream h_u has four parts:

The persona π_u, what is known about the person before the record begins. The environment E, the setting and its rules: a catalog, an app, a survey instrument, a game world. The situations s_1:t+1 the person was in, the current one last. And the actions a_1:t they took. Eq. 1 predicts the action in the current situation from all four. Memory is not a fifth part but a summary of the four that a system keeps because it cannot hold the whole stream.

Each field's equation is Eq. 1 with the stream cut down to the part that field keeps:

Left of each equals sign is the field's formula as it wrote it. Right of it is Eq. 1 with the stream restricted.

Recommendation keeps the actions: the items are the actions, the catalog is a fixed environment, and persona and situation are dropped or partly tokenized. OneRec feeds identity, age, and gender through a static pathway; PLUM's prompt carries channel, watch time, and time since the last watch; HSTU interleaves contextual features with actions. HCI keeps the current situation, the screen, and summarizes everything else into the memory. Simulation keeps the situation and the environment and has the persona supplied by the experimenter rather than computed from a record. A language model is the same equation with words as actions.

The delta over a base language model

What does grounding a model in a person's own data add over the base model's prior? One measurement per field, one metric per row.

Each row has its own metric, so the table ranks nothing across rows. The deltas are real in every field, smallest where the target is a stated response the base model can already approximate, and largest where the target is a revealed action on a real log. None of these measurements says whether the gain generalizes beyond the task it was measured on. A generalist evaluation of the three fields' models on one task set does not exist, and the paper says what one would need.

Scaling and data

Language loss falls as a power law in parameters, data, and compute. Where behavioral papers publish fitted exponents, the numbers are close to the language ones. Compute-optimal allocation on 115 billion medical events gives N ∝ C^0.520 and D ∝ C^0.512, against 0.49 and 0.51 for text. A sequential recommender's loss falls faster in model size than a language model's, exponent 0.121 against 0.076. Several more report the power law without publishing an exponent: HSTU across three orders of compute, OneRec from 0.015B to 2.6B parameters, and SimBench's simulation fidelity log-linear in model size. Two qualifications. Scaling is task-dependent; Netflix fits an offset power law per task and finds some tasks near their ceiling within the observed range while others keep improving. And every fitted law carries an irreducible term, which for a person is the person's own inconsistency, the ceiling the interview-agent protocol already divides by. Scale is not what is missing.

Data is. For language the corpus existed before the models did. For behavior it does not, and the shape of what exists is specific. The corpora with consent see tens or thousands of people: GUM's eighteen participants, LongNAP's twenty, the 1,052 interviewees, the 2,058 people of Twin-2K-500. The corpora with millions see one platform or one record type: Yambda's one music service, MicroLens's one video platform, Kuaishou's lifelong but private log, life2vec's six million Danes in a restricted registry, CoMET's 118 million patients in health records alone. No corpus is longitudinal, cross-context, population-scale, and consented at once.

There is a second, structural difference. A pretraining corpus for language is organized by document. A person's traces are scattered across it, unlinked, and never presented as a sequence, which is why a language model meets D3 only in context. Every behavioral pipeline begins by undoing this, assembling person-major sequences. The unit of the language corpus is the document; the unit of the behavioral corpus must be the person.

Completing the circle

We began with a goal: a computer that understands people. The survey showed three fields fitting one model to three slices of a person's stream, and that is the case for unifying them. The slices belong to the same equation, so the data that produces them belongs in one corpus and one model. Architecture, objective, and scaling are settled to the standard of other fields. The corpus and the evaluation are not, and the directions are the steps from the three slices back to the whole.

A corpus with four properties

Breadth: the same person observed across contexts. Within a company the join is free and has been published; across companies no join exists, and the alternative is the consented panel, people who carry their own streams across platforms as GUM's participants did on their own devices. A panel of thousands is enough to measure cross-context transfer, not to pretrain on. Depth: long, rich per-person sequences. The evidence is consistent across fields that representation quality grows smoothly with sequence length, and the industrial frontier is lifelong sequences at lengths in the thousands. Consent: a behavior stream identifies its author, so de-identification is not a privacy mechanism for this data; consent at collection and the person's ability to read and revise the model are. Evaluation: G(z) is undefined until the task set is fixed. Each field has a candidate benchmark; none spans fields, and only the interview-agent protocol grades against the person's own consistency. A standing benchmark needs both.

A common language for the stream

Eq. 7 names four parts. A model can train across the holders of a person's data only if all four arrive in one token language, and that is the mission the directions reduce to: every holder tokenizes its slice of the stream, the tokens connect, and one model serves the fields' tasks. Today each field has solved one part its own way. Recommendation tokenized the action and is starting on the situation. HCI captions the screen into language. Simulation writes the persona and the situation as text. Language is the common denominator they are drifting toward, and OpenOneRec's interleaving of item codes with text is the first model that reads two of the parts in one context.

The precedents say a shared vocabulary is what lets a model cross walls. Medicine got cross-country transfer once every event was an ICD code. Pharma federated once every molecule was one fingerprint, with ten competing companies pretraining jointly without sharing data, and each improved. Behavior has no such standard for any of its four parts. The nearest existing piece is the semantic ID: a tokenizer built on a public content encoder gives every silo a common code for the action, each silo tokenizes locally, and what leaves is a sequence of codes rather than a log. The persona, environment, and situation have no equivalent yet. Once a language exists, two routes to training across holders are visible, federated pretraining and translation between the user embeddings each silo already holds, on the evidence that representations of scaled models converge. Both need the language first.

Learning from recommenders

The flow between recommendation and language models has run one way. PLUM states the forward problem: language models are not pretrained on user behavior, so adapting them to recommendation means teaching them a domain. The framing takes for granted that behavior is a domain to be taught, never a modality to learn from. Nothing in the recipe forbids the reverse. Behavioral tokens can share a context with text, a person can enter a language model as a learned embedding rather than a prompt, and a recommender's grounding is revealed rather than stated preference, the signal text lacks. Why has no language model been shown to gain knowledge of people from behavior? Either nobody measured it, the behavioral data is too narrow, the knowledge forms but has no path into the language circuits, or nothing person-shaped forms at all. The first is the cheapest to eliminate: OpenOneRec ships checkpoints with base models of the same sizes, and the simulation benchmarks are the person-task suites. If the transfer is zero, behavioral models remain a parallel line. If it is positive, behavioral data is the pretraining substrate that follows text.

None of this is for its own sake. The aim is a computer that understands a person, meaning one that can predict or simulate what they will do in any situation, and a universal representation matters only as the means to that. The common language and the corpus are how the stream that test requires gets assembled.

Limitations

The paper has a limitations section and it should. In short form:

• Three fields were chosen. Health, finance, advertising, and mobility enter only where they bear on scaling and transfer, and each could support a survey of its own.
• Two of the three fields' leading systems come from one research group, so some of the agreement reported is shared practice rather than independent arrival.
• Most industrial results are self-reported by the platforms that built the systems and have not been replicated outside them. One scaling study is a company blog post.
• The reduction to one formula is an identity of form. The deltas over a base model are measured on incommensurate metrics, so the cross-field comparison is qualitative.
• There is a standing argument that generalization in recommendation is limited by nature: that behavior on one platform, in one catalog, under one interface, is too bound to its context to carry over, and the working optimum is a specialist retrained often. The evidence against it is that a model pretrained on one platform's logs transfers to ten catalogs it never saw, at +26.8% average Recall@10, which is generalization across contexts of the item if not yet of the person. We read the objection as a claim about today's corpora rather than about behavior. The reverse-transfer test above is where it gets tested in the other direction, and it could come out against us.
• The competing null is text sufficiency: that a frontier language model with the right context is all that is required. It holds for population-level and stated-response tasks and fails for per-person revealed behavior, and the boundary between the two is itself the measurement the field should make.
• Much of the cited work is a 2025 or 2026 preprint. Every claim was checked against its source, and the field moves faster than a survey does.

And a risk that does not go away with better evidence. The representation that serves a person is the representation that profiles them. Private traits are predictable from records of Likes, targeting built on such predictions moves behavior, and language models infer personal attributes from text. Generality makes profiling a side effect rather than a design choice, so the governance question is who runs the model and under what consent.

Where we are

Deep learning reached goals of this kind in other domains once three things were in place: a generic objective, a model that scales under it, and a corpus large enough to scale on. For language the three arrived together, and understanding fell out. For people, two of the three are already in place. The objective is next-action prediction over a person's stream, and three fields have been fitting it without saying so, each with the stream cut down to the part it can see. Scaling holds wherever the data is dense, with exponents close to the language values. The third thing is missing, and it is missing in a specific way. The population-scale data sits inside single platforms and produces domain specialists. The cross-context data sits with tens or thousands of consenting people and is too small to pretrain on. No common token language joins them.

The claim is not that a model of people is assured. It is that, for the first time, the problem has the shape of the ones deep learning has solved, and that what stands between the present and a foundation model of human behavior is not an objective or an architecture but a corpus. It has to be built rather than found, with consent, across the silos, in one language. That is what we are building.

The reading list behind the paper is public and takes issues and pull requests. If you work in any of these fields and think we have mischaracterized your corner of it, we would like to hear it before the paper is finished rather than after.