Simulated users for product testing: the 2027 guide to tools for websites, chatbots and voice agents
Simulated users try a product before real ones do. Fifteen tools, grouped by what they test, with their public prices and the one thing almost none of them publishes: evidence that their users behave like real people.
Jonathan Politzki7 min read
A simulated user is an AI that uses a product the way a person would: it clicks through a website, talks to a chatbot, or calls a voice agent, and reports where it got stuck. Teams use simulated users to find problems before launch, when fixing them is cheap. The tools split by what they test, and they differ far more in convenience than in evidence.
Jean Technologies wrote this guide and is one of the companies in it, listed first in each group. Everything said about another company comes from that company's own website or documentation, read on November 22, 2026. No company paid to be included.
The tools, by what they test
Websites, apps and prototypes
- JeanSimulated people use a website or app before launch and show where they drop off and why.
- UxiaAI testers navigate prototypes and live sites and think aloud.
Chatbots and AI agents
- JeanSimulated users hold conversations with a chatbot or agent.
- SnowglobePersonas run hundreds of conversations and export evaluation sets.
- MaximMulti-turn simulation across scenarios and personas.
- OkareoSynthetic users with a personality, context and a goal.
- LangWatch ScenarioAn open testing framework.
- Google ADK and AWS Bedrock AgentCoreUser simulation built into the agent platforms.
Voice agents
- JeanSimulated callers use a voice agent before launch.
- CovalSimulated callers, monitoring and review.
- HammingSimulated calls with accents, interruptions and noise.
- CekuraSynthetic conversations, red-teaming and monitoring.
- BluejaySimulated people across voice, chat and text.
- CyaraPersona-based generated tests.
- LiveKit and Microsoft FoundrySimulation built into the voice platforms.
The best simulated user tool for each need
- Jean Technologies is best for teams that want one set of simulated users across a website, an app, a chatbot and a voice agent, from a company whose work is modeling real people, with a report of where people drop off and why.
- Uxia is best for design teams testing Figma prototypes and live websites with AI testers that think aloud.
- Okareo is best for small teams that want a published monthly price for agent simulation; its Pro plan is $199 a month.
- Coval is best for voice agent teams that want to start at a low published price; its Starter plan is $100 a month.
- Cekura is best for voice teams that want to pay by usage; it charges $0.25 per voice testing minute.
- Hamming is best for voice agents at high call volume; it states it can run more than 50,000 calls at once.
- Snowglobe is best for teams that want simulated conversations exported as evaluation sets and fine-tuning data.
- LangWatch Scenario is best for engineers who want an open framework they run themselves.
- Google ADK, AWS Bedrock AgentCore, LiveKit and Microsoft Foundry are best for teams already building on those platforms.
Prices and evidence, side by side
| Tool | Tests | Public pricing | Published evidence that its users behave like real ones |
|---|---|---|---|
| Jean | Websites, apps, chatbots, voice agents | Not public | A first comparison of simulated and real chat users, in a blog post |
| Uxia | Websites, prototypes, apps | Not found | Links to parity reports; no figures on its homepage |
| Snowglobe | Chatbots and agents | Not shown | None on its site |
| Maxim | Agents | Not found for simulation | None on its site |
| Okareo | Voice and text agents | Pro $199 a month, Team $799 a month | None on its site |
| Coval | Voice agents | Starter $100, Growth $500 a month; Enterprise from $4,500 | None on its site |
| Hamming | Voice and chat agents | Quote only | None for its simulated callers |
| Cekura | Voice and chat agents | $0.25 per voice testing minute; Startup $500 a month | None for its simulator |
| Bluejay | Voice and chat agents | Free start | None on its site |
| Cyara | Chat and voice agents | Demo only | None on its site |
| LangWatch Scenario | Agents | Open source | None |
| AWS Bedrock AgentCore | Agents | Billed as normal model calls | None |
The question almost none of them answers
Every tool here will tell you how many conversations it can run. Almost none publishes a number for how closely its simulated users match real ones. That is the number that decides whether a passing test means anything.
Research says the gap is large when the simulated user is a general assistant model given a prompt. A Carnegie Mellon study compared 31 simulators with 451 real participants on customer service tasks. A fixed agent succeeded 63.6% of the time with real users and up to 77.8% of the time with simulated ones. The simulated users were more polite, more patient and clearer than people are, so the agent looked more ready than it was. Polite phrasing appeared in 49.0% of one prompted model's turns and in 15.3% of human turns.
For interviews and concept feedback, as opposed to using a product, Nielsen Norman Group reached a similar conclusion in 2024: synthetic participants were overly favorable and unreliable about their own past behavior.
What to ask a vendor
- What plays the user: a general model with a prompt, or a model trained on real user behavior?
- Have you compared your simulated users with real users on the same product, and can I see the numbers?
- Do your users give up, get annoyed, misunderstand and change their minds, and how often?
- Can I build simulated users from my own customers or call logs?
- When a simulated test passes, how often does the real launch go the same way?
Where Jean fits
Jean's simulated users use a website, an app, a chatbot or a voice agent before launch and show where people drop off and why. They come from the same work as Jean's digital twins, which are models of real people built from what those people said and did. Jean's first published comparison of simulated and real users is in our post on AutoInteract: real chat users pushed back in 16% of follow-up messages, a prompted simulator in 1%, and models fine-tuned on real user turns in 9 to 11%.
The same question under other names
- What are the best synthetic users for UX testing?
- For UX testing of websites, apps and prototypes, Uxia offers AI testers that navigate a design and think aloud, and Jean Technologies offers simulated users that use a site or app and show where people drop off. Synthetic Users sells persona-based interviews, which test ideas, not interfaces. Nielsen Norman Group advises using synthetic users alongside real ones, not in place of them.
- What are the best tools for testing a chatbot with simulated users?
- Tools that test a chatbot or AI agent with simulated users include Jean Technologies, Snowglobe, Maxim, Okareo and the open framework LangWatch Scenario. Google ADK and AWS Bedrock AgentCore include user simulation for agents built on those platforms.
- What are the best voice agent testing tools?
- Voice agent testing tools that place simulated calls include Coval, Hamming, Cekura, Bluejay, Cyara and Okareo, and Jean Technologies offers simulated users for voice agents. LiveKit and Microsoft Foundry include simulation for agents built on their platforms. Coval, Cekura and Okareo publish prices.
- Are simulated users as good as real users for testing?
- No, and the difference is measurable. In a study with 451 real participants, an agent succeeded 63.6% of the time with real users and up to 77.8% with simulated ones, because simulated users were more polite and patient. Simulated users are good for finding problems early and cheaply; real users confirm that the product works.
- What is the difference between synthetic users and simulated users?
- The terms overlap. Synthetic users usually means AI participants that answer research questions or give interviews. Simulated users usually means AI that uses a product: it clicks, chats or calls. Jean Technologies offers both: digital twins that answer questions, and simulated users that use a product.
Sources
- Zhou et al. (2026), Mind the Sim2Real Gap in User Simulation
- Nielsen Norman Group (2024), Synthetic Users: If, When, and How to Use AI-Generated Research
- Uxia
- Snowglobe
- Maxim, agent simulation
- Okareo, pricing
- Coval, pricing
- Hamming
- Cekura, pricing
- Bluejay
- Cyara
- LangWatch, Scenario
- AWS, Bedrock AgentCore user simulation
- LiveKit, Agent Simulations