# The conversations you cannot observe

> The most influential assessment of a brand happens in private chats, unobservable. Anyone who has to quantify this limit is left with only two measures, both partial.

Private conversations that influence AI-mediated search remain largely outside observation and measurement.

This limit is the precondition of the consequences covered in [Emerging effects](https://www.shaped-by-ai.com/en/impacts/emerging-effects/) — representation, comparison, reputational risk — and it is why this function cannot be run from a dashboard. The corpus puts it in the starkest available form, in the Social Listening entry of [Trust, AI literacy and calibration](https://www.shaped-by-ai.com/en/impacts/what-is-happening/cross-cutting-themes/trust-ai-literacy-calibration/): "the most influential conversation about the brand happens in private chats, unobservable" — the user compares alternatives, complains, chooses, and nobody is listening. It is "a structural loss of visibility, not one better tooling closes".

The order of magnitude helps make clear what is being discussed. A study conducted on OpenAI's data measures **2.6 billion messages a day** on consumer plans in June 2025, against 451 million in June 2024 (*How People Use ChatGPT*, NBER Working Paper 34255, September 2025). That is not an estimate: it is the count of total volume. Those conversations are analysable at industrial scale — that paper demonstrates it by classifying around 1.1 million of them through a procedure in which "no human ever reads the contents of a message" — but the only party that can do it is the vendor, inside its own perimeter, and what gets published is aggregated by task category. **Never by brand.** No version of that work exists from which a company could learn what is said about it.

## The non-click has a number

The corpus calls *informed non-opener* the user who gets the value from the summary and never opens — "it is not an absence of interest, it is an absence of trace" ([The customer journey shortens](https://www.shaped-by-ai.com/en/impacts/what-is-happening/search-becomes-conversation/the-customer-journey-shortens/)). Pew Research Center measured it on real behaviour, not on simulated prompts: 900 US adults who agreed to share their browsing, 68,879 distinct Google searches in March 2025.

* When an AI summary appears, a click on a traditional result happens in **8%** of visits, against **15%** when there is no summary.
* A click on a link *inside* the summary happens in **1%** of visits.
* And the figure that matters most here: **26% of pages with an AI summary end the browsing session entirely**, against 16% of pages with traditional results only.

A quarter of the time, the user is done. They did not give up: they got what they were looking for, and nowhere do they appear as a satisfied person — where they appear at all, they appear as traffic that did not arrive. *(Pew Research Center, July 2025; March 2025, US only, Google only. The ratio between the two click rates is more robust than the absolute level.)*

## The two measures that remain, and what they do not contain

Anyone who has to quantify this limit has two measures available today, both legitimate and both partial — and the partiality is of two different kinds.

**The first is official and has no questions.** Since 3 June 2026 Search Console offers the *Search Generative AI performance* reports: impressions inside AI Overviews, AI Mode and the generative features of Discover, with pages, countries, devices and dates. Impressions only — no clicks, CTR, average position or queries, and with the three surfaces aggregated. You know you appeared. You do not know which question you were answering, or what happened next.

**The second is simulated and has no users.** AI visibility tools run prompt sets built at a desk: repeatable by construction and, for the same reason, unlike the way people actually write. The corpus is already clear on the limits — "any tool selling a ranking position inside AI is selling an artefact" — and supplies the four questions to ask a vendor before signing: how many times do you run each prompt, how do you build and update the set, do you publish the methodology in verifiable form, do you sell ranking positions. One limit remains that none of the four resolves: **what you measure are the answers to your own questions.**

Neither measures what people actually do: the first is blind to intent, the second is blind to the population. Search Console shows that content appeared, but not which question it answered or whether the session ended with a satisfied demand. The prompt set shows how the system answers the organisation's questions, but not whether those questions match users' questions. Using either measure without stating its limit produces a tidy time series, but not necessarily more precise knowledge of the channel.

Coverage therefore needs to be declared: which surfaces are observed, by what method and which part remains outside. Proxies closer to the real conversation — support requests, questions asked during sales, internal search and interviews — are not representative, but they contain actual phrasings. They have the opposite weakness to the simulated prompt: less stability, but greater proximity to the questions people actually ask.

## The decision this observation informs

The decision is [What AI traffic is worth](https://www.shaped-by-ai.com/en/impacts/the-decisions-ai-forces/what-ai-traffic-is-worth/): choosing metrics consistent with what can actually be observed, and declaring the limits of coverage. Value cannot be estimated by conflating impressions, clicks and sessions that ended without a click.

[Staying in AI answers](https://www.shaped-by-ai.com/en/impacts/the-decisions-ai-forces/staying-in-ai-answers/) instead concerns presence and visibility in a channel where many conversations remain unobservable. The choice must therefore separate holding presence from knowing what users actually asked.

## Effects and role responsibilities

* [Research & Information Architecture](https://www.shaped-by-ai.com/en/impacts/roles/research-ia/) integrates support, sales, internal research and interviews as qualitative sources, making their coverage and biases explicit.
* [SEO, GEO and AI Visibility](https://www.shaped-by-ai.com/en/impacts/roles/seo-ai-visibility/) versions prompt sets and states their population, surfaces and frequency, separating simulated testing from real questions.
* [Operations & Data](https://www.shaped-by-ai.com/en/impacts/roles/operations-data/) separates AI, branded and direct traffic and documents which events are observable and which fall outside the measurement system.
* [Brand & PR](https://www.shaped-by-ai.com/en/impacts/roles/brand-pr/) uses qualitative sources and indirect signals without presenting them as a complete measure of the conversation.

Related reading: [The customer journey shortens](https://www.shaped-by-ai.com/en/impacts/what-is-happening/search-becomes-conversation/the-customer-journey-shortens/).