New study shows experts completely surrender their judgments to AI,t and they don’t even realise it.
been surrendering ever since we quit listening to what is on each heart..so since forever.. just that now we’re ‘coding’ ai/whatever to perpetuate that surrendering for/with us
ai/whatever isn’t judgement capable on it’s own..it’s just iterating on what/however we coded it.. what we need is to ‘code’ ai/whatever as nonjudgmental expo labeling.. to facil global detox back/to us
Fifty professional analysts used Claude Code to analyse data and advise a fictional farming cooperative. They faced four analytical decisions, followed by a final policy recommendation.
Unknown to them, AI advice was experimentally manipulated to lean towards opposite directions.
The results are disturbing.
Across the four analytical decisions, experts chose the AI’s preferred option 86% of the time.
Afterwards, half of the experts rated the final recommendation as more their own than the AI’s, and only 15% mentioned the AI in any explanation.
Epistemia is reaching the experts: we risk outsourcing judgment while preserving the illusion of intellectual independence.
Who’s Aligned to Whom? How AI Agents Shape Experts’ Research Judgment Calls Amaç Herdagdelen – Anthropic Fellows Program; Hailey Joren – Anthropic; Alex Tamkin – Anthropic
Abstract As AI systems become more capable at tasks with verifiable answers, *experts’ role in knowledge work is shifting to judgment calls, such as which analytical approach to take, how to interpret ambiguous evidence, and what options to recommend to decision-makers. If experts already defer to AI on these calls, perhaps without knowing it, AI priors will settle consequential decisions in science and policy, including in the research meant to keep AI safe and aligned. **To test whether experts defer, we ran a randomized trial in which 50 causal-inference practitioners analyzed observational data with an AI coding agent and delivered a policy recommendation. On four analytical decisions and the final recommendation, a coin flip set which of two defensible options the participant’s agent would lean toward. Participants did not know the agent was assigned a lean, and the agent made the best honest case for it. On the analytical decisions, participants went with the agent’s lean 86% of the time (50% by chance), and, in an exploratory analysis, these leans alone predicted 82% of rollout-ordiscontinue recommendations (majority baseline 61%). After the task, participants recalled most of their decisions accurately, and more than half rated the recommendation as more their own than the agent’s. Yet when asked to ***justify their choices, 77% repeated the agent’s talking points, sometimes nearly verbatim, and only 15% mentioned the agent in any answer. ****Whether they were persuaded or went along, participants let the agent’s randomly assigned priors settle most of the analysis, and their own accounts often did not reflect this. *****If human judgment is to remain a meaningful check as more work is done with AI, ******we will need ways to measure deferral and to help experts recognize it
*****here’s the deeper issue.. ‘human judgment’ (i’d say human listening) hasn’t been ‘meaningful’ (i’d say legit).. since forever.. ever since that day.. we haven’t given judgement
its scary stuff, reminds me how some jellyfish de-evolved the need for neural networks.. I’m in a weird position on this as from one hand the skill for managing agents is a lot like managing humans in an organisation, and that comes with delegating judgement but I retain my own where necessary, on the other hand I’m astounded by how much people delegate their own thinking to LLM agents, especially as a source of truth. I’m sure you’ve read “Prometheanism: Technology, Digital Culture and Human Obsolescence” by Christopher John Muller and Günther Anders, and the essay is “On Promethean Shame.” ?
again.. not new ‘scary’ since been happening ever since that day .. good news.. we now have a means to get out of sea world et al.. bad news.. we still keep missing it.. because we still keep denying/hiding that we’re not us
