In an unnervingly placid voice, a computer endowed with its own intelligence speaks to a human being and offers him some advice. “I can see you are really upset about this,” the machine says. “I honestly think you ought to sit down calmly. Take a stress pill and think things over.”
The scene, of course, is from one of the greatest films of the 20th century, the 1968 sci-fi thriller 2001: A Space Odyssey, and the name of what remains perhaps the most famous computer ever is HAL, short for heuristically programmed algorithmic computer. The astronaut who looms large on the screen as HAL addresses him with a chilling purr is named Dave, but he is really a stand in for all of us.
Nothing attests for that more than the fact that we may now be approaching the moment when the prescient and terrifying vision of the movie’s creators, the director Stanley Kubrick and author Arthur C. Clarke, may finally be coming to life—or perhaps better, turning into an objective reality right before our eyes.
The film 2001 can be interpreted in many ways, but at its heart lies a tale of technological dystopia in which a sentient and all-powerful computer with no history of ever having committed an error goes horribly rogue, taking over a manned spaceship and coolly killing its crew members. Two of the astronauts go into a space pod to ensure they won’t be overheard while they make plans to shut HAL down. One of them is soon lost in the void, murdered by the computer. After finding himself trapped outside the spaceship, the lone survivor, Dave, then orders HAL to open its doors and let him back in.
“I’m sorry Dave, I’m afraid I can’t do that,” HAL answers in his crushingly smooth voice.
“What’s the problem?” Dave asks.
“I think you know what the problem is just as well as I do,” HAL replies. “This mission is too important for me to allow you to jeopardize it.”
Even though Dave and his last surviving fellow crew mate had been careful enough to prevent HAL from overhearing their plans, they had not counted on the computer’s ability to read their lips from afar, through the small window of their pod. Dave ultimately succeeds in getting back into the mothership and disabling HAL. But the scariest part of this movie may be the thought that if Kubrick and Clarke were making their masterpiece today, it is highly unlikely that they would have thought that humans could have come out of this predicament so well.
For months now, I have been following developments in the world of artificial intelligence (AI) and its most popular manifestations, such as ChatGPT. And the impression that these developments have left me with is that however fast we imagine the landscape of this technology to be capable of changing, our visions just keep falling behind. I was put in the mind of HAL and 2001, though, because of an item that I came across scarcely a week ago that makes the movie’s lip reading seem already hopelessly antiquated. There may soon be no need to bother with such an indirect method, because in the brave new world of AI, scientists and engineers are already perfecting ways of reading people’s brain activity—i.e., their thoughts—simply by monitoring the electrical signals that we all give off.
In recent weeks, there has been a cascade of warnings from the people who know the field of AI best, as well as from what you might call curious bystanders—people much like me—hinting that humanity has unleashed a monster that it may not be able to control. Beyond the stream of warnings about how AI could soon be used to do things like rig elections, commit financial fraud on a huge scale, help unleash new weapons systems, and facilitate the worst kinds of political propaganda, we are now inundated with reports of AI’s negative effects in the present through its applications that have already entered the workplace and consumer marketplace.
Driven by its capacities, computer giant IBM, for example, is eliminating thousands of jobs, and fear of the looming threat to AI could pose to jobs in myriad other industries, such as writers in Hollywood, is creating anxiety and disruption. Meanwhile, off-the-shelf technology that you or I can already buy and use is quickly creating the capacity to essentially make convincing virtual clones of ourselves.
Perhaps the most famous of the many warnings about all of this so far came from Geoffrey Hinton, an AI pioneer who recently resigned from Google. In an interview with BBC News that brought to mind a future of Terminator-style automatons, potentially rogue doomsday machines, and an unlimited array of disaster scenarios that could be brought about by AI technology in the hands of what are often called bad actors, he warned:
Right now, what we are seeing is things like GPT4. It eclipses a person in the amount of general knowledge it has, and it eclipses them by a long way. In terms of reasoning, it is not as good, but it already does simple reasoning. And given the rate of progress, we expect things to get better quite fast, so we need to worry about that. Right now, they are not more intelligent than us as far as I can tell, but I think they soon may be.
The very next speaker in that BBC News segment was Junaid Mubeen, a research mathematician and expert on the human-AI interface. Toward that conversation’s end, he concluded darkly that “[Hinton’s] own warning does seem to be coming quite late in the day. And there is a sense that maybe the genie is out of the bottle now, and there is no real accounting for what happens next.”
These days, op-eds in leading Western newspapers have been full of fine-sounding but vague advice about how to get ahead of this potential threat. This week, for instance, the Financial Times editorial board urged the tech industry to “agree and implement some common principles on transparency, accountability and fairness,” adding that where computer algorithms are used in what it called “critical, life-and-death areas, such as healthcare, the judicial system and the military,” government preapproval should be required.
But I have to say that this feels very namby-pamby, and let me tell you why. Google is a good place to start any explanation of why a proposal like this seems so impossible. The company that rose to huge profitability on its search engine business had long been working quietly on figuring out how to integrate AI into its products but had held off, fearing that the risks had not yet been fully understood. Then along came Microsoft with a powerful new product of its own called Edge, which incorporated ChatGPT (which Microsoft has invested in) into its browser and search products, betting that their power and novelty would allow Microsoft to make dramatic gains in a market that Google has long owned virtually by itself.
The point that flows from this might be called principle No. 1: Profit will usually trump prudence where big and sudden gains seem attainable. Google, observing this principle, has eagerly followed Microsoft, as one might expect, throwing much of its erstwhile caution to the wind.
Principle No. 2 goes beyond corporate greed, which many people have taught us is a positive force in capitalism, and into the realm of the equally powerful logic of nation-states. Is it possible to imagine a world where China does not seek to match the recent advances that big U.S. corporations have made in AI applications? For me, the answer is an obvious “no,” and in fact, China has rushed in recent weeks to roll out products it hopes can rival those of Google and Microsoft. And this is not just about the world’s two superpowers, either. Others who wish to catch up, or at least keep pace, will inevitably throw their hats into the ring in a contest that fairly mirrors nuclear proliferation.
Not to be entirely hopeless here, but it is pretty much impossible to predict where this will all go once another technology that looms just over the horizon eventually matures: quantum computing. Marrying that to AI would exponentially magnify the challenge that humanity faces in keeping machines in a secure place.
Yet my most serious fears date to a much earlier time, when some visionaries were already beginning to see all this coming. The movie 2001: A Space Odyssey came out in 1968. Two years later, it was followed by another cultural blockbuster, but one that is mostly forgotten now. In 1970, in a book called Future Shock that became an enormous bestseller in its day, the author Alvin Toffler—along with his uncredited collaborator and wife, Heidi—foresaw the entire human dilemma that I have described here, even if some of the specific technological details eluded their imaginations more than 50 years ago, as seems reasonable.
In a book that frequently lapses into clunky language, Toffler wrote the following warning with utter clarity—for my money, better than the editorialists of the Financial Times, and on the same point:
[T]echnological questions can no longer be answered in technological terms alone. They are political questions. Indeed, they affect us more deeply than most of the superficial political issues that occupy us today. This is why we cannot continue to make technological decisions in the old way. We cannot permit them to be made haphazardly, independently of one another. We cannot permit them to be dictated by short-run economic considerations alone. We cannot permit them to be made in a policy vacuum. And we cannot casually delegate responsibility for such decisions to businessmen, scientists, engineers or administrators who are unaware of the profound consequences of their own actions.
And yet that is exactly what we seem to be doing.
Another thinker helps us understand why, and his conclusions are hardly encouraging. In his book Seeing Like a State: How Certain Schemes to Improve the Human Condition Have Failed (and others), the Yale University political scientist James C. Scott wrote about the propensity of humans to make “heroic assumptions” about the power of our own cleverness to come up with solutions to whatever problem rears its head. The book explores what Scott calls “high modernism”—essentially, the effort to use science and theory to order and regularize the social world, and to use theories of the future to remake the present. Scott defines high modernism in these terms:
It is best conceived as a strong, one might even say muscle-bound, version of the beliefs in scientific and technical progress that were associated with industrialization in Western Europe and in North America from roughly 1830 until World War I. At its core was a supreme self-confidence about continued linear progress, the development of scientific and technical knowledge, the expansion of production, the rational design of social order, the growing satisfaction of human needs, and, not least, an increasing control over nature.
The lesson of AI and of formidable breakthroughs to come, such as quantum computing, though, is that we may now be reaching the point where something most unnatural to humans is the only thing that can save us: humility.