TL;DR
The researchers found that AI advice suppressed suspension of judgment from 44% to 3%, accuracy from 27% to 9%, while confidence increased from 30% to 76%. People relied on incorrect answers from the AI.
Researchers from three French and Italian universities discovered that Access to AI advice collapsed people’s willingness to say “I don’t know” from 44% to 3%. Accuracy fell from 27% to 9%. Meanwhile, confidence increased from 30% to 76%. “People got a lot worse, the precision was only a third, but they were twice as confident,” said Valerio Capraro, associate professor at the University of Milano-Bicocca.
The study, written by Capraro with Chiara Marcoccia of École Normale Supérieure and Walter Quattrociocchi of Sapienza University of Rome, it deliberately used questions where AI models often fail: visual details from movies, like the color of a team’s uniform in Bend It Like Beckham. The researchers used Step 3.5 Flash, a model that was generally incorrect on these questions, precisely so any reduction in judgment could not be explained as sensible delegation to a trusted tool. Some participants who would have answered correctly on their own asked the AI and got it wrong.
Monetary incentives helped, but not much. Willingness to admit ignorance increased from 3% to 8% and accuracy from 9% to 16%, both still well below the non-AI baselines of 44% and 27%. Wharton researchers coined the term “cognitive performance” earlier this year. to describe the same phenomenon: people who accept incorrect answers from AI 80% of the time report higher confidence than those who work without AI. The new study adds clearer data. It’s not just that people trust incorrect answers from AI. It is that the mere availability of AI suppresses the cognitive habit of recognizing what is not known.
The 💜 of EU technology
The latest rumors from the EU tech scene, a story from our wise founder Boris and some questionable AI art. It’s free, every week, in your inbox. Register now!
“For humans, the ability to say ‘I don’t know’ is very important because it represents the recognition of the limits of our own knowledge,Capraro said. He is particularly concerned about children, who grow up with these systems before they have developed critical thinking skills. Google’s AI search overhaul replaced links with trusted AI-generated summariesand Common Sense Media this week called that design a “unacceptable risk” for students. The pattern is consistent: AI products are designed to respond, never to say “I don’t know.“The humans who use them are learning to do the same.
