We asked AI one question 8 times. 1 in 3 names missed half the answers.
Original research across 15 US metros. 720 answers to six buyer questions, compared against each other to show how much one AI check can tell you.
AI search engines rarely give the same answer twice. In 720 Perplexity answers across 15 US metros in September 2026, 47% of named companies appeared in all 8 asks of a question. 32% appeared in 4 or fewer, and a single check misses those companies 75% of the time. Ask each question 8 times before trusting the result.
Ask an AI search engine the same question twice and you can get two different lists. We measured how different.
Six "who should I hire" questions about builders and remodelers. Fifteen US cities. Each question asked eight times in every city, in fresh sessions, with every answer stored word for word. That is 720 answers.
Then we compared the eight answers to each question against each other. Which companies showed up every time, which showed up sometimes, and what one check on its own would have told you.
The numbers
Perplexity, September 10 to 11, 2026. 15 metros, 6 questions, 8 asks each.
| Measure | Result | Read it as |
|---|---|---|
| Companies named in one answer | About 6 | The list a buyer actually sees |
| Different companies named across 8 answers | About 8 | The real shortlist is bigger than any one answer |
| Share of the 8-answer shortlist one answer shows | About 75% | One check sees three quarters of the picture |
| Named companies that appeared in all 8 answers | 47% | The regulars. These are stable. |
| Named companies that appeared in 4 of 8 or fewer | 32% | About 1 in 3 names is on the edge |
| Chance one check misses a company on the edge | 75% | For those firms, one check is usually wrong |
| Overlap between two answers to the same question | About 80% | Similar, never identical |
What we found
The top of the list holds still
About half the companies named for a question appeared in all eight answers. Those regulars made up more than 80% of any single answer. If a company is one of them, one check will find it almost every time.
This matches what we saw in our builders study. Once an engine knows a company well, it keeps naming it.
What this means: if you are already a regular, one check tells you the truth. The problem is that you cannot know you are a regular from one check.
The edge of the list moves
About 1 in 3 named companies showed up in half the answers or fewer. About 1 in 6 showed up exactly once in eight tries.
For a company in that group, a single check misses it 3 times out of 4. Run it once and you would conclude AI has never heard of you. Run it eight times and you would see you are on the list some of the time.
We ran the same count on the small builders we had confirmed by hand for the first study. The pattern held. 54% of the times one was named for a question, it was named in all eight answers. 24% of the time it was named in four or fewer.
Why this matters: the edge is where a smaller company trying to break in sits. It is the exact group a one-time check reads worst.
Broad questions rotate more
How stable an answer was depended on how specific the question was. Overlap is the share of names two answers to the same question had in common.
| Question | Overlap |
|---|---|
| Who are the top luxury home builders in [city]? | 85% |
| Which builders in [city] specialize in ADUs and home additions? | 81% |
| Who is the best custom home builder in [city]? | 79% |
| Who is the best design-build remodeling firm in [city]? | 77% |
| Who are the most trusted kitchen and bathroom remodelers in [city]? | 74% |
| Who is the best general contractor in [city]? | 58% |
"General contractor" covers everything from a bathroom to an office tower. The engine picked a different slice of that each time, so the names changed with it. Narrow questions had a short, settled list.
What this means: the vaguer the question your buyers ask, the more checks you need before you trust the result.
Some cities are steadier than others
Average overlap between two answers ranged from 67% in the least stable city to 86% in the most stable. No city gave the same list every time.
What this means: a stability number from one market does not transfer to another. Measure where your buyers are.
How we ran it, and what it does not show
Read this before quoting any number above.
Every answer came from Perplexity's API with web search on, which is how it runs by default. That is a close proxy for the consumer app, not the same thing. We did not measure ChatGPT, Gemini or Google's AI Overviews here. Variation on those engines may be higher or lower.
These are the same answers behind Which Local Builders Does AI Name?, collected September 10 to 11, 2026. We used the six questions that ask for companies and left out the four cost and planning questions, which rarely name anyone. The eight asks for a question ran close together, so this measures variation within a day or two, not drift over months.
Company names were pulled from each answer automatically, then normalized so "Smith Homes" and "Smith Custom Homes" count once. We read random samples of extracted names by hand, and more than 9 in 10 were real companies. Two spellings of one firm can still slip through as two names, which would make answers look slightly less stable than they are. Treat every figure as approximate. We ran the count three ways while tightening the extraction, and the headline numbers moved by single digits.
Overlap between two answers is the number of companies both named, divided by the number of companies either named. We averaged it over every pair of the eight answers to a question.
No individual company is named on this page. Many are small businesses that did not ask to be part of a study.
How many times should you check?
Eight is what we use. It is enough to separate a company that is always named from one that is named sometimes, and it costs a few minutes by hand.
Pick the three questions a buyer in your market asks before they call anyone. Ask each one eight times in the AI assistant your customers actually use, in fresh chats. Write down every company named and count how many of the eight answers each one appears in.
- Named in 6 or more: you are a regular for that question.
- Named in 1 to 5: you are on the edge. This is the group with the most to gain.
- Never named: you are not on the list yet. Look at who the regulars are and what their sites say that yours does not.
The free AI Visibility Check is a fast first look. It asks once, so read its result as one sample of the eight.
Cite this study as MarqueeOS, "Does AI Give the Same Answer Twice?", October 2026, and link to this page.
Keep reading
Which Austin Trades Does AI Name? 160 Answers, 33 of 40 Never Came Up
33 of 40 Austin trades companies were never named in 160 AI answers. No plumbing answer named any of the five we checked, and cost questions named nobody.
Read the guide → Research · 5 min readWhy AI Uses Your Website But Recommends Someone Else
One Austin plumber's site was listed as a source in all 8 asks and the company was never named. Why it happens and what to try.
Read the guide →Want the eight-answer read?
The Diagnostic runs this measurement on your business, across the questions your buyers ask, and ends in a written plan with a fixed fee.
See the Diagnostic →