# Trust, AI Literacy and Calibration

> Users misunderstand GenAI in two opposite directions: overestimation and unjustified distrust. Calibration is built in degrees.

Users misunderstand GenAI in two opposite directions.

**Overestimation.** Many treat ChatGPT as an omniscient source, forgetting that an LLM generates plausible text, not certified truth, and that it can assert the false with the same confidence as the true. The mechanism is documented: the formal quality of the text is mistaken for the quality of the facts, to the point that people follow an AI's advice more than a human expert's, even against available evidence (*Computers in Human Behavior*, 2024). A paradoxical relationship holds: the less one understands how AI works, the more inclined one is to use and trust it — a "magical" perception of technology. Maximum risk in the health, legal, and financial domains, and for young people.

The mechanism behind overestimation has a name in HCI literature: the **CASA — Computers Are Social Actors** paradigm ([Nass, Steuer and Tauber, CHI 1994](https://dl.acm.org/doi/10.1145/191666.191703); Reeves and Nass, *The Media Equation*, 1996). The experiments show that people apply the same social rules to systems that they use with other people, automatically and even when they explicitly deny it makes sense: they reciprocate politeness, attribute competence from surface-level linguistic cues, and trust a fluent interlocutor more. A conversational system triggers these reflexes more than any earlier interface, because fluent natural language is its default mode.

On the opposite side — what makes content perceived as credible, regardless of who or what produces it — the [Stanford web credibility guidelines](https://credibility.stanford.edu/guidelines/index.html) (B.J. Fogg, Stanford Persuasive Technology Lab, since 1998) remain a reference: ten empirical criteria — make accuracy verifiable, show who is behind the content, make expertise and sources explicit, avoid errors, which damage credibility more than an accurate piece builds it — that predate AI by two decades and remain the most cited starting point for designing the trust signals a conversational interface has to build from scratch.

There is a specific case worth isolating, because it touches our work closely: **trust built on facts is transferred to recommendations**, which however work in a completely different way. On factual accuracy the models show high performance; but when they return a list of products, services, or providers, the selection and order are the outcome of a probabilistic extraction, and change at every execution — to the point that the same question asked a hundred times does not produce the same ranking twice (SparkToro/Gumshoe, January 2026; see [Search becomes conversation](https://www.shaped-by-ai.com/en/impacts/what-is-happening/search-becomes-conversation/)). Those who read that list instead interpret it as a reasoned judgment. It is a mental model asymmetry that no interface declares today.

**Baseline skill.** Using chat effectively requires skills that are not acquired spontaneously: writing good prompts, verifying outputs, managing context. It is the [**articulation barrier**](https://www.linkedin.com/pulse/prompt-driven-ai-ux-hurts-usability-jakob-nielsen/) described by Jakob Nielsen: the difficulty of translating a need into an effective request. It is not a niche barrier: according to the OECD PIAAC 2023 survey (published December 2024) **about 26% of adults in OECD countries are at the lowest literacy level (Level 1 or below), and 18% are below the basic level in all measured domains** — with strong differences between countries. The feeling of not having to learn anything does not correspond to a simpler interaction: the 19.1% figure on users who use prompting strategies confirms it (see [Intent-based interaction](https://www.shaped-by-ai.com/en/impacts/what-is-happening/search-becomes-conversation/intent-based-interaction/)).

**Unjustified distrust.** At the opposite extreme, the black box nature generates a distrust that blocks even reasonable delegations. The 2026 case is agentic checkout: **58% of shoppers use AI to inform themselves, 37% start a purchase path with it, but only 17% are comfortable completing it** (ChannelEngine, January 2026, 4,500 shoppers on marketplaces in five countries). OpenAI took note by withdrawing Instant Checkout on 24 March 2026, five months after launch; Walmart measured in-chat purchases at a third of the click-out conversion rate, while generating double the new customers. The pattern is clear: **people discover products in AI and buy them on the site.**

The result is polarization; few have calibrated trust, aware of the limits. Calibration is built in degrees — reversible actions, then low-risk, then high-impact — and the more autonomous the agent, the more the user must understand what it does and why. AI literacy is a design variable, not a detail of the user context — and, as the Istat gradient of 9 to 1 by education shows, it is not neutrally distributed.

## The decision this observation informs

The decision is how much conversion the organisation is willing to lose for calibrated trust. Without an explicit choice, silence wins, and this phenomenon explains precisely why: how much to tell the user about the system's limits is a product decision, and today almost nobody makes it consciously — product metrics therefore reward silence. On delegation, in the absence of a scale, the level is in effect decided by development, case by case.

Disclosing costs friction at every single step: every source shown, every uncertainty signal, every confirmation point is one more point of exit. Staying silent costs calibration, and it is a cost that shows up behaviourally and late — cancellations after an automatic action, external verification of the output, drop-off at confirmation points, switching automation off after a trial measure the damage once it is done. Neither extreme works, and the limit of the analysis has to be stated too: the indicators of the benefit exist, the measure of the cost per individual lever does not.

The choice belongs to the Product Manager together with Product & Design: they are the two roles that can realistically own the five artefacts described below — the first for the threshold of what gets disclosed, the second for the service artefacts that make it operational. Until someone owns them, silence decides. The consequences fall on [The delegation scale](https://www.shaped-by-ai.com/en/impacts/the-decisions-ai-forces/how-far-can-the-system-go/), for the perimeter of actions, and on [Does engagement stay a KPI?](https://www.shaped-by-ai.com/en/impacts/the-decisions-ai-forces/does-engagement-stay-a-kpi/), for the metric every uncertainty signal has to be defended against.

## Effects and role responsibilities

* **[Service Designer](https://www.shaped-by-ai.com/en/impacts/roles/product-design/)** — Calibrating trust is not a job of its own: it is a set of responsibilities scattered across UX design, content design, product and legal, and precisely for that reason, if nobody holds them together, they do not get exercised. The realistic owner is service design, because the artefacts involved are all service choices; the threshold of what gets disclosed is agreed with the Product Manager and with marketing, who pay its cost in conversion. The concrete perimeter is made of five artefacts.
  **(1) An explicit delegation scale.** For every action the system can perform, define the level: *suggested* (the system proposes, the user executes), *executable with confirmation*, *automatic*. And the criteria to move up a level: reversibility of the action, economic impact, presence of personal or third-party data. Without this scale, the level is in fact decided by development, case by case.
  **(2) Confirmation points where the action is irreversible.** Practical rule: explicit confirmation if the action cannot be undone within a time useful to the user. Payments, sends to third parties, cancellations, data sharing, actions with legal effects. The confirmation must say *what* is about to happen in concrete terms, not ask a generic "confirm?".
  **(3) Situated uncertainty signals.** The generic disclaimer ("AI can make mistakes") calibrates nothing: it is background noise the user learns to ignore. What is needed is uncertainty where it exists: the source is dated, the data was not found, the answer only holds for a certain case. Better a few precise signals than a permanent warning.
  **(4) Traceability of actions and sources.** What the system did, with which tools, on which data, with what degree of confidence. The IBM study (see [Agentic AI](https://www.shaped-by-ai.com/en/impacts/what-is-happening/agentic-ai-from-showing-to-doing/)) lists exactly what users ask to know: it is a good starting point for a requirement, not an advanced feature.
  **(5) Recovery paths.** Undo, correction, escalation to a human. A system without *undo* is not delegable, and designing delegation without designing the way back is the most common shortcut.
  **How to measure.** Not with trust questions in a questionnaire, because declared and acted trust diverge systematically (see [Empathy and social presence](https://www.shaped-by-ai.com/en/impacts/what-is-happening/cross-cutting-themes/empathy-and-social-presence/)). Behavioral indicators: cancellation rate after an automatic action, frequency of external verification of output, abandonment at confirmation points, share of users who disable automation after trying it. The vocabulary for this calibration — *trust calibration*, *misuse*, *disuse* — comes from the human-factors literature on high-automation contexts ([Lee and See, 2004](https://doi.org/10.1518/hfes.46.1.50.30392)) and applies here to conversational and agentic systems (see [The delegation scale](https://www.shaped-by-ai.com/en/impacts/the-decisions-ai-forces/how-far-can-the-system-go/)).
  Since 2 August 2026 part of this work is no longer discretionary (see [The regulatory perimeter](https://www.shaped-by-ai.com/en/impacts/emerging-effects/the-regulatory-perimeter/)). → Useful resources: [Trust and Reliance on AI — Overreliance (Computers in Human Behavior)](https://www.sciencedirect.com/science/article/pii/S0747563224002206); [People Devalue Generative AI's Competence but Not Its Advice](https://pmc.ncbi.nlm.nih.gov/articles/PMC11332189/); [How AI and Human Behaviors Shape Psychosocial Effects of Chatbot Use (MIT Media Lab + OpenAI)](https://arxiv.org/html/2503.17473v1)
* **[Content Strategist](https://www.shaped-by-ai.com/en/impacts/roles/content-editorial/)** — Design the communication of reliability: where the system is reliable, where not, how to interpret and verify outputs. It is content design and editorial positioning, not just UX writing.
* **[UX Researcher](https://www.shaped-by-ai.com/en/impacts/roles/research-ia/)** — AI literacy in personas as an explicit variable: how the user uses AI, how they understand it, how they calibrate trust, how they react to errors. Add trust calibration indicators, and measure them **also at the behavioral level**: declared trust and real reliance diverge systematically (see [Empathy and social presence](https://www.shaped-by-ai.com/en/impacts/what-is-happening/cross-cutting-themes/empathy-and-social-presence/)). Standard tools do not exist yet: it is a discipline still evolving.
* **[Social Listening / Brand Intelligence](https://www.shaped-by-ai.com/en/impacts/roles/brand-pr/)** — The role moves upstream, and this is where it distinguishes itself from the [AI Visibility Analyst](https://www.shaped-by-ai.com/en/impacts/what-is-happening/search-becomes-conversation/the-customer-journey-shortens/): **that one measures what the model says about the brand, this one guards the sources from which it learns and the conversation that surrounds it.** Three concrete changes. First, the object of listening changes: the sources that feed generative answers — reviews, sector forums, UGC, comparisons — become more relevant than traditional social channels, and must be guarded as such (it is the same earned lever of [Search becomes conversation](https://www.shaped-by-ai.com/en/impacts/what-is-happening/search-becomes-conversation/)). Second, the quality of the social signal drops, diluted by synthetic content: the volume of mentions loses value as an indicator. Third, and more delicate: **the most influential conversation about the brand happens in private chats, not observable** — the user compares alternatives, complains, chooses, and no one listens. It is a structural loss of visibility, not fillable with better tools. New risk to manage: hallucination about the brand is reputational damage without a rectification channel — there is no "right of reply" inside a generative answer, and the only lever is correcting the sources upstream. → Useful resource: [AI social listening — YouScan](https://youscan.io/blog/ai-social-listening/)
* **[Product Manager](https://www.shaped-by-ai.com/en/impacts/roles/leadership-strategy/)** — **How much to tell the user about the system's limits is a product decision, and today almost no one makes it consciously.** The problem, concretely: declaring that an answer is generated by an AI, that a quote is a non-binding estimate, that advice does not replace a professional, that the system did not find confirmation of a data point — these are all choices that reduce the use of the function and, downstream, short-term conversion. Product metrics therefore reward silence, and in the absence of an explicit decision the default prevails: say nothing. The question to put on the table is who has the authority to set the threshold — whether the PM, legal, design, or the client — and on what criterion, given that neither extreme works: the permanent warning is ignored, the absence of warning produces uncalibrated reliance and, in case of error, a liability problem. Since 2026 the margin of discretion narrows anyway: some of these declarations become mandatory.