Some of the polls and market research you see this season may not have asked a single human being anything. Polling firms are beginning to use AI personas, built on large language models, to stand in for real voters, according to a report from Phys. By the end of this article, you will know what synthetic polling is, where it can mislead you, and a short checklist for judging any survey result you come across.
What synthetic polling actually is
In a traditional poll, researchers ask real people questions and tally the answers. In synthetic polling, an AI model is given a profile (say, a particular kind of voter or customer) and asked to answer as that person would. Run enough of these personas and you get something that looks like survey data.
The appeal is easy to see. Per the Phys report, firms are turning to LLM-based personas to cut costs and scale research, at a moment when campaigns, brands, and media outlets want faster audience insights.
The core problem: real people change their minds
The report's central point is simple. A voter surveyed in October may give a different answer by January. An AI persona might not, depending on how it is built.
That matters because public opinion is not a fixed photograph. It moves with news, events, conversations, and mood. A persona that gives steady, consistent answers can look reassuringly clean, but consistency is not the same as accuracy.
Jennifer Stromer-Galley, cited in the report, raises concerns about model bias, missing outliers, and the risk of mistaking AI consistency for public reality. In other words, a tidy result may simply reflect how the model works, not how people feel.
Three ways synthetic results can mislead
1. Bias baked into the model
An AI persona draws on whatever patterns its underlying model learned. If that model leans a certain way, every persona built on it may lean that way too, and you would never see the tilt in the final numbers.
2. Missing outliers
Real populations include unusual, surprising, and minority views. Those are often exactly what a campaign or company most needs to hear. Personas tend to cluster around the expected answer, so the edges can vanish.
3. Consistency mistaken for reality
If personas never shift the way people do, a synthetic poll can show stability where real opinion is moving. The risk, as Stromer-Galley's concerns suggest, is that decision-makers treat that calm as the truth.
What Pew's findings add
The report points to recent Pew findings showing major gaps between AI and human survey responses. I can't give you the specific numbers here, since the report summary doesn't include them, so check Pew's own publication for details. The takeaway is still clear: when researchers compared the two, AI answers did not simply match what people said.
A five-question checklist for any poll you see
You don't need technical training to apply this. When a survey result crosses your feed, run through these steps in order.
- Look for the method. Does it say who was surveyed? Real respondents should be stated plainly. If the text mentions "AI personas," "synthetic respondents," or "simulated voters," you are looking at modeled data, not a poll of people.
- Check the date. Opinion moves. A result from months ago, especially a synthetic one, may not reflect today.
- Ask whether it was ever compared to real people. A trustworthy synthetic claim should show how it stacks up against human responses. Given the Pew gaps, the absence of that comparison is a red flag.
- Look for the unusual answers. Real surveys show spread and surprises. Suspiciously smooth results deserve a second look.
- Notice who benefits. Faster, cheaper research is attractive to the people selling it. That is not proof of a problem, but it is a reason to read the claims carefully.
Where synthetic polling could still be useful
Nothing in the report says AI personas are worthless. Used as a rough, low-cost way to brainstorm, test wording, or explore hypotheses before surveying real people, they may save time. My own view is that the danger lies in treating them as a replacement for voters rather than a rehearsal for asking them.
If you work with survey data
If you commission or use research at work, ask vendors directly: are the respondents real people, how was the system checked against human answers, and how does it handle opinion that changes over time? Those questions need no special software, only a willingness to ask them.
The takeaway
Before you trust a poll, find out who answered it. If the answer is "a model," treat the result as a hypothesis, and wait for real people to confirm it.
