The arrival of artificial intelligence has been greeted with the usual mixture of hysteria and utopianism that accompanies all technological progress. One group imagines the extinction of human intelligence. Another imagines a paradise in which machines do all the thinking while mankind enjoys effortless prosperity. Both are wrong. Artificial intelligence is neither a curse nor a miracle. It is a tool. Like every powerful tool, it offers golden opportunities to those who use it intelligently, though dangers to those who surrender their judgment to it.

There is – if I may use a word for some reason much overused of late – an irony here that deserves attention. The people who will gain most from artificial intelligence are unlikely to be those who trust it most. They will be those who approach it with disciplined scepticism. If we wish to obtain the maximum benefit from AI, we must cultivate the habit of distrusting it.

You may think this to go against common sense. We normally distrust incompetent assistants, not brilliant ones. Yet artificial intelligence differs from previous technologies in one important respect. Most machines reveal their limitations immediately. AI often disguises them. A calculator either gives the correct answer or it does not. If it displays that two plus two equals seventeen, almost everyone notices the mistake. A steam engine either runs or it does not. A broken television displays only static. Artificial intelligence, on the other hand, is capable of producing answers that appear convincing but are wrong. It speaks with confidence. It reasons fluently. It cites imaginary authorities. It invents facts with the fluency of a race across the water. It is the first widely available technology that can imitate competence so successfully that users cease to examine whether competence is really there. And this introduces a danger unlike anything produced by earlier technological revolutions.

I return to the pocket calculator, which is the parallel most commonly cited. People once feared these would produce a generation incapable of arithmetic. The prediction was exaggerated. Calculators undoubtedly reduced the need for routine calculation, but they also allowed students to tackle mathematical problems that would previously have consumed most of their time. Used intelligently, calculators became instruments of higher mathematical thought rather than substitutes for it.

Even so, they did impose costs. Researchers led by Mark LaCour deliberately programmed calculators to provide incorrect answers for certain problems. What they discovered was alarming. Many students accepted the incorrect answers without question. Even when the results were obviously implausible, some participants were unwilling to challenge what the machine had produced. The machine had acquired an authority out of proportion to its demonstrated reliability.

Something similar has happened with satellite navigation. GPS systems have transformed driving. They reduce stress. They avoid congestion. They make unfamiliar journeys almost effortless. Few sensible people would voluntarily return to unfolding paper maps in the rain beside a busy motorway. Yet convenience has its price. Research by Louisa Dahmani of Harvard Medical School and Véronique Bohbot of McGill University found that greater lifetime dependence on GPS navigation was associated with weaker spatial memory. Other studies have shown that pedestrians navigating by smartphone frequently take longer routes and make more unnecessary stops than those relying upon traditional maps. The machine has removed the necessity of thinking about geography.

Online search has produced another version of the same phenomenon. The internet has placed almost the whole accumulated knowledge of mankind within immediate reach. This is an astonishing achievement. Yet psychologists now speak of the “Google effect“: people remember information less well when they expect it always to remain available online. Instead of remembering facts, we remember how to retrieve them.

None of these developments is undesirable. Human beings have always delegated routine tasks to technology whenever possible. We no longer grow our own corn. grind our own flour. Delegation, or the division of labour, itself is not the problem. Artificial intelligence, however, changes the scale of delegation. Previous technologies replaced particular mental tasks. AI threatens to replace judgment itself.

This is why Steven Shaw and Gideon Nave of the Wharton School have introduced the striking expression “cognitive surrender.” Their phrase deserves to enter the language, because it identifies a danger that extends beyond the workplace. Their experiments resembled the earlier calculator studies. Volunteers answered difficult questions with AI assistance. Unknown to the participants, the researchers deliberately inserted errors into some of the machine’s responses. The results showed that, when the artificial intelligence produced accurate answers, those using it outperformed people relying entirely on their own reasoning. This should surprise nobody. A competent assistant usually improves performance. When the AI produced incorrect answers, however, the situation reversed. Participants using the machine performed worse than those working without assistance. They ceased evaluating the evidence independently. They accepted the machine’s conclusions automatically. They had not only delegated part of the task, but had surrendered responsibility for thinking. This distinction is fundamental.

Delegation has always been part of what civilisation means. We employ accountants instead of calculating taxes ourselves. We consult doctors rather than studying every branch of medicine. We rely on engineers to build bridges instead of learning structural mechanics. Civilisation depends on intelligent delegation. Cognitive surrender is different. It occurs when we cease asking whether delegated judgments deserve our confidence. The machine ceases to become our servant and assumes the role of master.

The danger is intensified because artificial intelligence possesses what can only be called a persuasive personality. Large language models do not just provide information. They converse. They explain. They reassure. They apologise. They display what appears to be confidence without ever displaying uncertainty in proportion to their ignorance. Human beings are naturally inclined to trust confident speakers. AI exploits this instinct whether intentionally or not.

There is therefore a lesson for anyone wishing to benefit from this remarkable technology. Artificial intelligence should be treated less as an oracle than as a clever junior assistant. A competent junior assistant saves enormous amounts of time. He drafts letters. He gathers information. He prepares summaries. He suggests approaches that his employer may not previously have considered. But no sensible barrister signs pleadings prepared by an inexperienced clerk without reading them carefully. No experienced editor publishes an article after glancing at the first paragraph. Supervision remains indispensable.

The same principle should govern our relationship with AI. The machine should produce the first draft, not the final judgment. Indeed, here lies one of the greatest opportunities presented by artificial intelligence. Because it performs routine intellectual labour so rapidly, it allows educated people to spend more time exercising the uniquely human faculties of criticism, comparison and evaluation. This is the opposite of intellectual laziness. Used properly, AI should raise the level of human thought rather than lower it.

Unfortunately, this outcome is not guaranteed. Like every labour-saving device, artificial intelligence can encourage indolence. A student who simply copies machine-generated essays learns little. A lawyer who files unverified citations risks professional disgrace. A journalist who republishes fabricated quotations destroys what little credibility he naturally possesses. Every profession already contains examples of embarrassing failures produced by uncritical dependence on AI. The technology is not at fault. The fault lies in abandoning intellectual responsibility.

There is an instructive lesson from another recent experiment. Stefanos Poulidis and his colleagues studied chess players learning with AI assistance. Some students could summon advice whenever they wished. Others received guidance only at predetermined moments. The results were unexpected. Those with unrestricted access to AI improved less than those whose assistance was rationed. Constant availability discouraged independent analysis. The students began consulting the machine before attempting to solve problems for themselves. They became consumers of solutions rather than producers of understanding.

This suggests an important principle extending beyond chess.

Artificial intelligence is most valuable after we have attempted to solve a problem ourselves. Our first effort creates the mental framework necessary to evaluate the machine’s suggestions. Without that framework we possess no reliable method for distinguishing brilliance from nonsense. The AI may improve our thinking only after we have already begun thinking.

There is, I think, an older philosophical lesson here. For centuries the educated classes have regarded scepticism as one of the central intellectual virtues. Socrates questioned accepted wisdom. Francis Bacon urged systematic doubt before accepting received opinion. The scientific revolution itself rested on the refusal to accept authority without examination. Artificial intelligence makes this ancient habit more important than ever.

The irony is real. The greatest intellectual invention since the printing press demands those habits of independent thought that modern education has done so much to weaken. The future belongs neither to those who reject AI nor to those who worship it. It belongs to those capable of collaborating with it while remaining permanently prepared to contradict it.

This may also explain why artificial intelligence is likely to increase rather than reduce the value of genuine education. Facts become cheaper every year. Judgment becomes more valuable. Memorisation alone is increasingly insufficient. The ability to recognise contradictions, detect implausibilities, compare competing explanations and ask the awkward question that everyone else has forgotten to ask becomes the scarce commodity. Machines increasingly generate information. Human beings must increasingly evaluate it.

Artificial intelligence therefore presents us with an unusual challenge. Every previous intellectual tool demanded that we learn how to use it. This one demands something more difficult. It requires us to learn when not to believe it. That is not a weakness of the technology, but the price of its extraordinary power.

Those who cultivate cognitive resistance will enjoy immense advantages. They will write faster, research more effectively, analyse more data and solve more complex problems than previous generations could have imagined. Those who surrender their judgment will become efficient repeaters of whatever errors the machine happens to produce. The difference between these two groups will not be determined by the quality of the artificial intelligence they possess. It will be determined by the quality of the natural intelligence they preserve.