Users misunderstand GenAI in two opposite directions.
Overestimation. Many treat ChatGPT as an omniscient source, forgetting that an LLM generates plausible text, not certified truth, and that it can assert the false with the same confidence as the true. The mechanism is documented: the formal quality of the text is mistaken for the quality of the facts, to the point that people follow an AI’s advice more than a human expert’s, even against available evidence (Computers in Human Behavior, 2024). A paradoxical relationship holds: the less one understands how AI works, the more inclined one is to use and trust it — a “magical” perception of technology. Maximum risk in the health, legal, and financial domains, and for young people.
There is a specific case worth isolating, because it touches our work closely: trust built on facts is transferred to recommendations, which however work in a completely different way. On factual accuracy the models show high performance; but when they return a list of products, services, or providers, the selection and order are the outcome of a probabilistic extraction, and change at every execution — to the point that the same question asked a hundred times does not produce the same ranking twice (SparkToro/Gumshoe, January 2026; see Search becomes conversation). Those who read that list instead interpret it as a reasoned judgment. It is a mental model asymmetry that no interface declares today.
Unjustified distrust. At the opposite extreme, the black box nature generates a distrust that blocks even reasonable delegations. The 2026 case is agentic checkout: 58% of shoppers use AI to inform themselves, 37% start a purchase path with it, but only 17% are comfortable completing it (ChannelEngine, January 2026, 4,500 shoppers on marketplaces in five countries). OpenAI took note by withdrawing Instant Checkout on 24 March 2026, five months after launch; Walmart measured in-chat purchases at a third of the click-out conversion rate, while generating double the new customers. The pattern is clear: you discover in AI, you buy on the site.
The result is polarization; few have calibrated trust, aware of the limits. Calibration is built in degrees — reversible actions, then low-risk, then high-impact — and the more autonomous the agent, the more the user must understand what it does and why. AI literacy is a design variable, not a detail of the user context — and, as the Istat gradient of 9 to 1 by education shows, it is not neutrally distributed.
Effect on the professions
UX Researcher — AI literacy in personas as an explicit variable: how the user uses AI, how they understand it, how they calibrate trust, how they react to errors. Add trust calibration indicators, and measure them also at the behavioral level: declared trust and real reliance diverge systematically (see Empathy and social presence). Standard tools do not exist yet: it is a discipline still evolving.
Social Listening / Brand Intelligence — The role moves upstream, and this is where it distinguishes itself from the AI Visibility Analyst: that one measures what the model says about the brand, this one guards the sources from which it learns and the conversation that surrounds it. Three concrete changes. First, the object of listening changes: the sources that feed generative answers — reviews, sector forums, UGC, comparisons — become more relevant than traditional social channels, and must be guarded as such (it is the same earned lever of Search becomes conversation). Second, the quality of the social signal drops, diluted by synthetic content: the volume of mentions loses value as an indicator. Third, and more delicate: the most influential conversation about the brand happens in private chats, not observable — the user compares alternatives, complains, chooses, and no one listens. It is a structural loss of visibility, not fillable with better tools. New risk to manage: hallucination about the brand is reputational damage without a rectification channel — there is no “right of reply” inside a generative answer, and the only lever is correcting the sources upstream. → Useful resource: AI social listening — YouScan
Trust Designer — It is the least defined role on the list, and it is worth being explicit: in most teams it does not exist as an autonomous figure. It is a set of responsibilities today dispersed among UX designer, content designer, PM, and legal — and precisely for this reason, if no one holds them together, they are not exercised. The concrete perimeter is made of five artifacts. (1) An explicit delegation scale. For every action the system can perform, define the level: suggested (the system proposes, the user executes), executable with confirmation, automatic. And the criteria to move up a level: reversibility of the action, economic impact, presence of personal or third-party data. Without this scale, the level is in fact decided by development, case by case. (2) Confirmation points where the action is irreversible. Practical rule: explicit confirmation if the action cannot be undone within a time useful to the user. Payments, sends to third parties, cancellations, data sharing, actions with legal effects. The confirmation must say what is about to happen in concrete terms, not ask a generic “confirm?”. (3) Situated uncertainty signals. The generic disclaimer (“AI can make mistakes”) calibrates nothing: it is background noise the user learns to ignore. What is needed is uncertainty where it exists: the source is dated, the data was not found, the answer only holds for a certain case. Better a few precise signals than a permanent warning. (4) Traceability of actions and sources. What the system did, with which tools, on which data, with what degree of confidence. The IBM study (see Agentic AI) lists exactly what users ask to know: it is a good starting point for a requirement, not an advanced feature. (5) Recovery paths. Undo, correction, escalation to a human. A system without undo is not delegable, and designing delegation without designing the way back is the most common shortcut. How to measure. Not with trust questions in a questionnaire, because declared and acted trust diverge systematically (see Empathy and social presence). Behavioral indicators: cancellation rate after an automatic action, frequency of external verification of output, abandonment at confirmation points, share of users who disable automation after trying it. Since 2 August 2026 part of this work is no longer discretionary (see The regulatory perimeter). → Useful resources: Trust and Reliance on AI — Overreliance (Computers in Human Behavior); People Devalue Generative AI’s Competence but Not Its Advice; How AI and Human Behaviors Shape Psychosocial Effects of Chatbot Use (MIT Media Lab + OpenAI)
Product Manager — How much to tell the user about the system’s limits is a product decision, and today almost no one makes it consciously. The problem, concretely: declaring that an answer is generated by an AI, that a quote is a non-binding estimate, that advice does not replace a professional, that the system did not find confirmation of a data point — these are all choices that reduce the use of the function and, downstream, short-term conversion. Product metrics therefore reward silence, and in the absence of an explicit decision the default prevails: say nothing. The question to put on the table is who has the authority to set the threshold — whether the PM, legal, design, or the client — and on what criterion, given that neither extreme works: the permanent warning is ignored, the absence of warning produces uncalibrated reliance and, in case of error, a liability problem. Since 2026 the margin of discretion narrows anyway: some of these declarations become mandatory.
Content Strategist — Design the communication of reliability: where the system is reliable, where not, how to interpret and verify outputs. It is content design and editorial positioning, not just UX writing.