Research published this year has given a name to something employers have been mulling over for some time. Skilling is what happens when an expert stops practicing and gets worse. Lack of training is what happens when a newbie never becomes good in the first place, and it’s the most uncomfortable problem, because the people it affects are the ones companies are already dealing with. hire less than.
The clearest evidence comes from a randomized controlled trial conducted by Anthropic researchers Judy Hanwen Shen and Alex Tamkin. published in january.
They recruited 52 mostly young software engineers, gave half of them an AI assistant, asked them all to learn Trio, a Python library that none of them knew, and then quizzed everyone about the concepts they had used minutes before.
The AI group averaged 50%. The group that coded manually had an average of 67%. Anthropic describes the gap as the equivalent of nearly two letter grades and was statistically significant, with a p-value of 0.01.
The 💜 of EU technology
The latest rumors from the EU tech scene, a story from our wise founder Boris and some questionable AI art. It’s free, every week, in your inbox. Register now!
The speed benefit, which is the only reason anyone turns to the wizard, didn’t really materialize. The AI group finished about two minutes faster, a difference that did not reach significance, in part because several participants spent up to 11 minutes writing queries, about a third of their allotted time.
They learned less, didn’t finish faster, and performed worse on what matters most when the machine fails. That thing is the purge, where the gap between the groups was wider. The control group, denied an assistant, found bugs and had to solve them, which is a fair description of how you learn to debug. The AI group made no mistakes.
Medicine has come at the same concern from a different direction. TO Perspective of natural medicine published in May, led by Duke-NUS Medical School with co-authors from Harvard, UCL and King’s College London, he coined it for students who rely on AI during their formative clinical years and never construct the reasoning that safe, independent practice requires.
Add a third category that gets even less attention: the lack of skills, the learner who accepts an AI mistake uncritically and files it away as fact.
Those authors are careful in a way their coverage hasn’t always been. They write that there is no direct evidence of medical training. The argument is based on learning theory and early signs from non-clinical settings, that is, from studies like Anthropic.
His prescription is a three-phase framework: develop non-AI competencies, then teach people to calibrate their skepticism, and then introduce the tools under supervision.
How the tool is used matters more than whether it is used. On the Anthropic test, high scorers asked conceptual questions or requested explanations along with the code. Those who scored low delegated wholesale or relied on the assistant to debug for them.
Employers are already evaluating this. Gartner predict that the atrophy of critical thinking will push half of global organizations to require “AI-free” skills assessments by 2026, which is a polite way of saying that hiring managers no longer trust a portfolio.
Meanwhile, Ford has been rehiring of engineers to fix what their artificial intelligence systems did wrong, an expensive demonstration of what happens when the people who could have caught the error are no longer on the payroll.
The essay has real limits, and so say its authors. The sample was small, the questionnaire measured comprehension immediately rather than months later, and it used a sidebar assistant rather than an agent coder. Researchers expect the impact of these to be more pronounced, not less.
It is worth noting who directed it. Anthropic sells the assistant and has published an article arguing that using the assistant carelessly makes you worse at your job. This is either unusual frankness or the openness of a proposal for modes of learning, and both readings may be true.
What the research doesn’t say is that young people should code by hand. What it says is that shortcut and skill are not the same path, and that the industry has been assuming that they were for two years.
