The conversations you cannot observe

The most influential assessment happens in private chats. Two partial measures remain — the impression without the question, the simulated prompt without the user — and the choice is which one you run decisions on.

This phenomenon is the precondition of the other three, and it is why this function cannot be run from a dashboard. The corpus puts it in the starkest available form, in the Social Listening entry of Trust, AI literacy and calibration: “the most influential conversation about the brand happens in private chats, unobservable” — the user compares alternatives, complains, chooses, and nobody is listening. It is “a structural loss of visibility, not one better tooling closes”.

The order of magnitude helps make clear what is being discussed. A study conducted on OpenAI’s data measures 2.6 billion messages a day on consumer plans in June 2025, against 451 million in June 2024 (How People Use ChatGPT, NBER Working Paper 34255, September 2025). That is not an estimate: it is the count of total volume. Those conversations are analysable at industrial scale — that paper demonstrates it by classifying around 1.1 million of them through a procedure in which “no human ever reads the contents of a message” — but the only party that can do it is the vendor, inside its own perimeter, and what gets published is aggregated by task category. Never by brand. No version of that work exists from which a company could learn what is said about it.

The non-click has a number

The corpus calls informed non-opener the user who gets the value from the summary and never opens — “it is not an absence of interest, it is an absence of trace” (The customer journey shortens). Pew Research Center measured it on real behaviour, not on simulated prompts: 900 US adults who agreed to share their browsing, 68,879 distinct Google searches in March 2025.

When an AI summary appears, a click on a traditional result happens in 8% of visits, against 15% when there is no summary.
A click on a link inside the summary happens in 1% of visits.
And the figure that matters most here: 26% of pages with an AI summary end the browsing session entirely, against 16% of pages with traditional results only.

A quarter of the time, the user is done. They did not give up: they got what they were looking for, and nowhere do they appear as a satisfied person — where they appear at all, they appear as traffic that did not arrive. (Pew Research Center, July 2025; March 2025, US only, Google only. The ratio between the two click rates is more robust than the absolute level.)

The two measures that remain, and what they do not contain

The first is official and has no questions. Since 3 June 2026 Search Console offers the Search Generative AI performance reports: impressions inside AI Overviews, AI Mode and the generative features of Discover, with pages, countries, devices and dates. Impressions only — no clicks, CTR, average position or queries, and with the three surfaces aggregated. You know you appeared. You do not know which question you were answering, or what happened next.

The second is simulated and has no users. AI visibility tools run prompt sets built at a desk: repeatable by construction and, for the same reason, unlike the way people actually write. The corpus is already clear on the limits — “any tool selling a ranking position inside AI is selling an artefact” — and supplies the four questions to ask a vendor before signing: how many times do you run each prompt, how do you build and update the set, do you publish the methodology in verifiable form, do you sell ranking positions. One limit remains that none of the four resolves: what you measure are the answers to your own questions.

Neither measures what the phenomenon describes. Both are legitimate and both are partial, and the partiality is of two different kinds: the first is blind to intent, the second is blind to the population.

The decision

The question. Which of the two partial measures do we accept running this function’s decisions on — and do we say so, or do we pretend to have data?

The default by omission. You do not choose, and the intermediary chooses: it is Google that decides which metrics exist in its reports, and the model vendor that decides the aggregate is published by task category and not by brand. The difference from the other defaults on this site is that here no setting gets chosen: what gets chosen is the perimeter of what is knowable at all, and it gets chosen for everyone.

The cost of either branch. The impression without the question costs you that you optimise for appearing without knowing which intent you are answering, and that the 26% of sessions that end there will never show up as satisfied demand: in your numbers it will be an arrival that did not happen, and the natural reading of an arrival that did not happen is that the channel does not work. The simulated prompt without the user costs you a stable measure of a world that does not exist — a clean time series, comparable with itself, built on questions no customer has written — with the specific risk that stability gets mistaken for accuracy. And there is a cost common to both branches: both measures are bought or built, and neither answers the question it is bought for.

How it is measured. This is the phenomenon where the question folds back on itself, and it is better said than dodged: there is no measure of the unobserved share of the conversation, because measuring it would require observing it. Only one honest thing can be done, which is to declare coverage: which surfaces are covered, by what method, and which part is avowedly outside. The rest moves to the proxies that do contain the conversation — support requests, questions asked during sales, the internal search engine, interviews — which are not representative but are real, that is, they have the opposite defect to the simulated prompt.

Who decides. Social Listening / Brand Intelligence for the perimeter of listening, AI Visibility Analyst for the measurement, Digital / Web Analyst for the downstream part. But the real decision sits upstream of all three: it is the choice of how much blindness the organisation is willing to put in writing, and that gets taken where objectives are approved — because an objective on an unobservable channel is an objective nobody will be able to declare met.

→ a particular case of What AI traffic is worth, and the precondition of the other three phenomena in this chapter: it explains why none of the three can be tended from a dashboard.

Effect on the professions

Social Listening / Brand Intelligence — This is the role hit hardest, in a way that changing tools does not fix. The craft’s historical object — public conversation — narrows twice: it loses share to private chat, and it loses quality because the social signal is diluted by synthetic content, so mention volume is worth less as an indicator. What remains is a different job: tending the sources the model learns from instead of listening to what people say, and putting in writing which part is not covered. Declaring coverage becomes part of the deliverable, not a methodological footnote.
AI Visibility Analyst — Keep the distinction between volatility and blindness, because they resemble each other and are treated in opposite ways. Synchronic volatility is handled with repetitions, diachronic volatility with time series — 40–60% of cited domains change within a month. Blindness is not handled at all, because it does not concern the stability of the measure but the population it is taken on. A prompt set run a hundred times is still a prompt set you wrote.
Digital / Web Analyst — The unobservable part has a direct effect on downstream containers, and it needs explaining before somebody reads the numbers backwards: the user who ends the session after the summary is not a lost user, but in the data they look like one. This sits alongside the fact already recorded in The customer journey shortens, that AI-influenced traffic arrives far less as direct and far more as branded search. You need a dedicated channel group, and the explicit caveat that a share of satisfied demand does not enter the attribution model — and will not.
UX Researcher — This is the figure holding the only remaining access to real conversation, and it changes in importance for a new reason: not because qualitative research is better, but because the quantitative half of this channel does not exist. Interviews, observed sessions, questions to support and to the internal search engine become the only source on how people actually phrase requests — with the advantage over the simulated prompt of containing real phrasings, and the known limit of not being representative. It is a reversal of the usual relationship between qualitative and quantitative: here the qualitative does not illustrate the data, it replaces it.