The idea of artificial general intelligence as we know it today starts with a dot-com blowout on Broadway. 

Twenty years ago—before Shane Legg clicked with neuroscience postgrad Demis Hassabis over a shared fascination with intelligence; before the pair hooked up with Hassabis’s childhood friend Mustafa Suleyman, a progressive activist, to spin that fascination into a company called DeepMind; before Google bought that company for more than half a billion dollars four years later—Legg worked at a startup in New York called Webmind, set up by AI researcher Ben Goertzel. Today the two men represent two very different branches of the future of artificial intelligence, but their roots reach back to common ground.

Even for the heady days of the dot-com bubble, Webmind’s goals were ambitious. Goertzel wanted to create a digital baby brain and release it onto the internet, where he believed it would grow up to become fully self-aware and far smarter than humans. “We are on the verge of a transition equal in magnitude to the advent of intelligence, or the emergence of language,” he told the Christian Science Monitor in 1998.

Webmind tried to bankroll itself by building a tool for predicting the behavior of financial markets on the side, but the bigger dream never came off. After burning through $20 million, Webmind was evicted from its offices at the southern tip of Manhattan and stopped paying its staff. It filed for bankruptcy in 2001.

But Legg and Goertzel stayed in touch. When Goertzel was putting together a book of essays about AI a few years later, it was Legg who came up with the title. “I was talking to Ben and I was like, ‘Well, if it’s about the generality that AI systems don’t yet have, we should just call it Artificial General Intelligence,’” says Legg, who is now DeepMind’s chief scientist. “And AGI kind of has a ring to it as an acronym.”

The term stuck. Goertzel’s book and the annual AGI Conference that he launched in 2008 have made AGI a common buzzword for human-like or superhuman AI. But it has also become a major bugbear. “I don’t like the term AGI,” says Jerome Pesenti, head of AI at Facebook. “I don’t know what it means.”

He’s not alone. Here's Julian Togelius, an AI researcher at New York University: “Belief in AGI is like belief in magic. It is a way of abandoning rational thought and expressing hope/fear for something that cannot be understood.”

Part of the problem is that AGI is a catchall for the hopes and fears surrounding an entire technology. Many of the challenges we face today, from climate change to failing democracies to public health crises, are vastly complex. If we had machines that could think like us or better—more quickly and without tiring—then maybe we’d stand a better chance of solving these problems (so the idea goes). As the computer scientist I. J. Good put it in 1965: “the first ultraintelligent machine is the last invention that man need ever make.”

Some would have you believe that such an invention is just around the corner. Elon Musk, who invested early in DeepMind and teamed up with a small group of mega-investors, including Peter Thiel and Sam Altman, to sink $1 billion into OpenAI, has made a personal brand out of wild-eyed predictions. But when he speaks, millions listen. A few months ago he told the New York Times that superhuman AI is less than five years away. “It’s going to be upon us very quickly,” he said on the Lex Fridman podcast. “Then we’ll need to figure out what we should do, if we even have that choice.” 

In May, Pesenti shot back. “Elon Musk has no idea what he is talking about,” he tweeted. “There is no such thing as AGI and we are nowhere near matching human intelligence.” Musk replied: “Facebook sucks.”

Such flare-ups aren’t uncommon. Browse the #noAGI hashtag on Twitter and you’ll catch many of AI’s heavy hitters weighing in, including Yann LeCun, Facebook’s chief AI scientist, who won the Turing Award in 2018, and Andrew Ng, former head of AI at Baidu and cofounder of Google Brain. “Let’s cut out the AGI nonsense and spend more time on the urgent problems,” Ng tweeted.

But with AI’s recent run of successes, from the board-game champion AlphaZero to the convincing fake-text generator GPT-3, chatter about AGI has spiked. Even though those tools are very far from being all-rounders—AlphaZero cannot write stories and GPT-3 cannot play chess, let alone understand why stories and chess matter to people—the goal of building an AGI, once thought crazy, is becoming acceptable again. 

Some of the biggest, most respected AI labs in the world take this goal very seriously. OpenAI has said that it wants to be the first to build a machine with human-like reasoning abilities. DeepMind’s unofficial but widely repeated mission statement is to “solve intelligence.” Top people in both companies are happy to discuss these goals in terms of AGI.

Half a century on, we’re still nowhere near making an AI with the multi-tasking abilities of a human—or even an insect.

“Talking about AGI in the early 2000s put you on the lunatic fringe,” says Legg. “Even when we started DeepMind in 2010, we got an astonishing amount of eye-rolling at conferences.”  But things are changing. “Some people are uncomfortable with it, but it’s coming in from the cold," he says.

So why is AGI controversial? Why does it matter? And is it a reckless, misleading dream—or the ultimate goal? 

What is AGI?

The term has been in popular use for little more than a decade, but the ideas it encapsulates have been around for a lifetime. 

In the summer of 1956, a dozen or so scientists got together at Dartmouth College in New Hampshire to work on what they believed would be a modest research project. Pitching the workshop beforehand, AI pioneers John McCarthy, Marvin Minsky, Nat Rochester, and Claude Shannon wrote: “The study is to proceed on the basis of the conjecture that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it. An attempt will be made to find how to make machines use language, form abstractions and concepts, solve kinds of problems now reserved for humans, and improve themselves.” They figured this would take 10 people two months.

Fast-forward to 1970 and here’s Minsky again, undaunted: “In from three to eight years, we will have a machine with the general intelligence of an average human being. I mean a machine that will be able to read Shakespeare, grease a car, play office politics, tell a joke, have a fight. At that point the machine will begin to educate itself with fantastic speed. In a few months it will be at genius level, and a few months after that, its powers will be incalculable.”

Three things stand out in these visions for AI: a human-like ability to generalize, a superhuman ability to self-improve at an exponential rate, and a super-size portion of wishful thinking. Half a century on, we’re still nowhere near making an AI with the multitasking abilities of a human—or even an insect.

That’s not to say there haven’t been enormous successes. Many of the items on that early bucket list have been ticked off: we have machines that can use language, see, and solve many of our problems. But the AIs we have today are not human-like in the way that the pioneers imagined. Deep learning, the technology driving the AI boom, trains machines to become masters at a vast number of things—like writing fake stories and playing chess—but only one at a time. 

When Legg suggested the term AGI to Goertzel for his 2007 book, he was setting artificial general intelligence against this narrow, mainstream idea of AI. People had been using several related terms, such as “strong AI” and “real AI,” to distinguish Minsky’s vision from the AI that had arrived instead. 

Talking about AGI was often meant to imply that AI had failed, says Joanna Bryson, an AI researcher at the Hertie School in Berlin: “It was the idea that there were people just doing this boring stuff, like machine vision, but we over here—and I was one of them at the time—are still trying to understand human intelligence,” she says. “Strong AI, cognitive science, AGI—these were our different ways of saying, ‘You guys have screwed up; we’re moving forward.’”

This idea that AGI is the true goal of AI research is still current. A working AI system soon becomes just a piece of software—Bryson’s “boring stuff.” Meanwhile, AGI becomes a stand-in for any AI we just haven’t figured out how to build yet, always out of reach.

Sometimes Legg talks about AGI as a kind of multi-tool—one machine that solves many different problems, without a new one having to be designed for each additional challenge. On that view, it wouldn’t be any more intelligent than AlphaGo or GPT-3; it would just have more capabilities. It would be a general-purpose AI, not a full-fledged intelligence. But he also talks about a machine you could interact with as if it were another person. He describes a kind of ultimate playmate: “It would be wonderful to interact with a machine and show it a new card game and have it understand and ask you questions and play the game with you,” he says. “It would be a dream come true.”

When people talk about AGI, it is typically these human-like abilities that they have in mind.  Thore Graepel, a colleague of Legg’s at DeepMind, likes to use a quote from science fiction author Robert Heinlein, which seems to mirror Minsky’s words: “A human being should be able to change a diaper, plan an invasion, butcher a hog, conn a ship, design a building, write a sonnet, balance accounts, build a wall, set a bone, comfort the dying, take orders, give orders, cooperate, act alone, solve equations, analyze a new problem, pitch manure, program a computer, cook a tasty meal, fight efficiently, die gallantly. Specialization is for insects.”

And yet, fun fact: Graepel’s go-to description is spoken by a character called Lazarus Long in Heinlein’s 1973 novel Time Enough for Love. Long is a superman of sorts, the result of a genetic experiment that lets him live for hundreds of years. During that extended time, Long lives many lives and masters many skills. In other words, Minsky describes the abilities of a typical human; Graepel does not. 

The goalposts of the search for AGI are constantly shifting in this way. What do people mean when they talk of human-like artificial intelligence—human like you and me, or human like Lazarus Long? For Pesenti, this ambiguity is a problem. “I don’t think anybody knows what it is,” he says. “Humans can’t do everything. They can’t solve every problem—and they can’t make themselves better.”

So what might an AGI be like in practice? Calling it “human-like” is at once vague and too specific. Humans are the best example of general intelligence we have, but humans are also highly specialized. A quick glance across the varied universe of animal smarts—from the collective cognition seen in ants to the problem-solving skills of crows or octopuses to the more recognizable but still alien intelligence of chimpanzees—shows that there are many ways to build a general intelligence. 

Even if we do build an AGI, we may not fully understand it. Today’s machine-learning models are typically “black boxes,” meaning they arrive at accurate results through paths of calculation no human can make sense of. Add self-improving superintelligence to the mix and it’s clear why science fiction often provides the easiest analogies. 

Some would also lasso consciousness or sentience into the requirements for an AGI. But if intelligence is hard to pin down, consciousness is even worse. Philosophers and scientists aren’t clear on what it is in ourselves, let alone what it would be in a computer. Intelligence probably requires some degree of self-awareness, an ability to reflect on your view of the world, but that is not necessarily the same thing as consciousness—what it feels like to experience the world or reflect on your view of it. Even AGI’s most faithful are agnostic about machine consciousness. 

How do we make an AGI?

Legg has been chasing intelligence his whole career. After Webmind he worked with Marcus Hutter at the University of Lugano in Switzerland on a PhD thesis called “Machine Super Intelligence.” Hutter (who is now also at DeepMind) was working on a mathematical definition of intelligence that was limited only by the laws of physics—an ultimate general intelligence.

The pair published an equation for what they called universal intelligence, which Legg describes as a measure of the ability to achieve goals in a wide range of environments. They showed that their mathematical definition was similar to many theories of intelligence found in psychology, which also defines intelligence in terms of generality.

At DeepMind, Legg is turning his theoretical work into practical demonstrations, starting with AIs that achieve particular goals in particular environments, from games to protein folding. 

The tricky part comes next: yoking multiple abilities together. Deep learning is the most general approach we have, in that one deep-learning algorithm can be used to learn more than one task. AlphaZero used the same algorithm to learn Go, shogi (a chess-like game from Japan), and chess. DeepMind’s Atari57 system used the same algorithm to master every Atari video game. But the AIs can still learn only one thing at a time. Having mastered chess, AlphaZero has to wipe its memory and learn shogi from scratch.

Legg refers to this type of generality as “one-algorithm,” versus the “one-brain” generality humans have. One-algorithm generality is very useful but not as interesting as the one-brain kind, he says: “You and I don’t need to switch brains; we don’t put our chess brains in to play a game of chess.”

Moving from one-algorithm to one-brain is one of the biggest open challenges in AI. A one-brain AI would still not be a true intelligence, only a better general-purpose AI—Legg’s multi-tool. But whether they’re shooting for AGI or not, researchers agree that today’s systems need to be made more general-purpose, and for those who do have AGI as the goal, a general-purpose AI is a necessary first step. There is a long list of approaches that might help. They range from emerging tech that’s already here to more radical experiments (see box). Roughly in order of maturity, they are:

• Unsupervised or self-supervised learning. Labeling data sets (e.g., tagging all pictures of cats with “cat”) to tell AIs what they’re looking at during training is the key to what’s known as supervised learning. It’s still largely done by hand and is a major bottleneck. AI needs to be able to teach itself without human guidance—e.g., looking at pictures of cats and dogs and learning to tell them apart without help, or spotting anomalies in financial transactions without having previous examples flagged by a human. This, known as unsupervised learning, is now becoming more common.
• Transfer learning, including few-shot learning. Most deep-learning models today can be trained to do only one thing at a time. Transfer learning aims to let AIs transfer some parts of their training for one task, such as playing chess, to another, such as playing Go. This is how humans learn.
• Common sense and causal inference. It would be easier to transfer training between tasks if an AI had a bedrock of common sense to start from. And a key part of common sense is understanding cause and effect. Giving common sense to AIs is a hot research topic at the moment, with approaches ranging from encoding simple rules into a neural network to constraining the possible predictions that an AI can make. But work is still in its early stages. 
• Learning optimizers. These are tools that can be used to shape the way AIs learn, guiding them to train more efficiently. Recent work shows that these tools can be trained themselves—in effect, meaning one AI is used to train others. This could be a tiny step toward self-improving AI, an AGI goal. 

All these research areas are built on top of deep learning, which remains the most promising way to build AI at the moment. Deep learning relies on neural networks, which are often described as being brain-like in that their digital neurons are inspired by biological ones. Human intelligence is the best example of general intelligence we have, so it makes sense to look at ourselves for inspiration. 

But brains are more than one massive tangle of neurons. They have separate components that collaborate. 

Hassabis, for example, was studying the hippocampus, which processes memory, when he and Legg met. Hassabis thinks general intelligence in human brains comes in part from interaction between the hippocampus and the cortex. This idea led to DeepMind’s Atari-game playing AI, which uses a hippocampus-inspired algorithm, called the DNC (differential neural computer), that combines a neural network with a dedicated memory component. 

Artificial brain-like components such as the DNC are sometimes known as cognitive architectures. They play a role in other DeepMind AIs such as AlphaGo and AlphaZero, which combine two separate specialized neural networks with search trees, an older form of algorithm that works a bit like a flowchart for decisions. Language models like GPT-3 combine a neural network with a more specialized one called a transformer, which handles sequences of data like text.

Ultimately, all the approaches to reaching AGI boil down to two broad schools of thought. One is that if you get the algorithms right, you can arrange them in whatever cognitive architecture you like. Labs like OpenAI seem to stand by this approach, building bigger and bigger machine-learning models that might achieve AGI by brute force. 

The other school says that a fixation on deep learning is holding us back. If the key to AGI is figuring out how the components of an artificial brain should work together, then focusing too much on the components themselves—the deep-learning algorithms—is to miss the wood for the trees. Get the cognitive architecture right, and you can plug in the algorithms almost as an afterthought. This is the approach favored by Goertzel, whose OpenCog project is an attempt to build an open-source platform that will fit different pieces of the puzzle into an AGI whole. It is also a path that DeepMind explored when it combined neural networks and search trees for AlphaGo. 

“My personal sense is that it’s something between the two,” says Legg. “I suspect there are a relatively small number of carefully crafted algorithms that we'll be able to combine together to be really powerful.”

Goertzel doesn’t disagree. “The depth of thinking about AGI at Google and DeepMind impresses me,” he says (both firms are now owned by Alphabet). “If there’s any big company that’s going to get it, it’s going to be them.”

Don’t hold your breath, however. Stung by having underestimated the challenge for decades, few other than Musk like to hazard a guess for when (if ever) AGI will arrive. Even Goertzel won’t risk pinning his goals to a specific timeline, though he’d say sooner rather than later. There is no doubt that rapid advances in deep learning—and GPT-3, in particular—have raised expectations by mimicking certain human abilities. But mimicry is not intelligence. There are still very big holes in the road ahead, and researchers still haven’t fathomed their depth, let alone worked out how to fill them. 

“But if we keep moving quickly, who knows?” says Legg. “In a few decades’ time, we might have some very, very capable systems.”

Why is AGI controversial?

Part of the reason nobody knows how to build an AGI is that few agree on what it is. The different approaches reflect different ideas about what we’re aiming for, from multi-tool to superhuman AI. Tiny steps are being made toward making AI more general-purpose, but there is an enormous gulf between a general-purpose tool that can solve several different problems and one that can solve problems that humans cannot—Good’s “last invention.” “There’s tons of progress in AI, but that does not imply there’s any progress in AGI,” says Andrew Ng.

Without evidence on either side about whether AGI is achievable or not, the issue becomes a matter of faith. “It feels like those arguments in medieval philosophy about whether you can fit an infinite number of angels on the head of a pin,” says Togelius. “It makes no sense; these are just words.”

Goertzel downplays talk of controversy. “There are people at extremes on either side,” he says, “but there are a lot of people in the middle as well, and the people in the middle don’t tend to babble so much.”

Goertzel places an AGI skeptic like Ng at one end and himself at the other. Since his days at Webmind, Goertzel has courted the media as a figurehead for the AGI fringe. He runs the AGI Conference and heads up an organization called SingularityNet, which he describes as a sort of “Webmind on blockchain.” From 2014 to 2018 he was also chief scientist at Hanson Robotics, the Hong Kong–based firm that unveiled a talking humanoid robot called Sophia in 2016. More theme-park mannequin than cutting-edge research, Sophia earned Goertzel headlines around the world. But even he admits that it is merely a “theatrical robot,” not an AI. Goertzel’s particular brand of showmanship has caused many serious AI researchers to distance themselves from his end of the spectrum.

In the middle he’d put people like Yoshua Bengio, an AI researcher at the University of Montreal who was a co-winner of the Turing Award with Yann LeCun and Geoffrey Hinton in 2018. In a 2014 keynote talk at the AGI Conference, Bengio suggested that building an AI with human-level intelligence is possible because the human brain is a machine—one that just needs figuring out. But he is not convinced about superintelligence—a machine that outpaces the human mind. Either way, he thinks that AGI will not be achieved unless we find a way to give computers common sense and causal inference. 

Ng, however, insists he’s not against AGI either. “I think AGI is super exciting, I would love to get there,” he says. “If I had tons of spare time, I would work on it myself.” When he was at Google Brain and deep learning was going from strength to strength, Ng—like OpenAI—wondered if simply scaling up neural networks could be a path to AGI. “But these are questions, not statements,” he says. “Where AGI became controversial is when people started to make specific claims about it.”

An even more divisive issue than the hubris about how soon AGI can be achieved is the scaremongering about what it could do if it’s let loose. Here, speculation and science fiction soon blur. Musk says AGI will be more dangerous than nukes. Hugo de Garis, an AI researcher now at Wuhan University in China, predicted in the 2000s that AGI would lead to a world war and “billions of deaths” by the end of the century. Godlike machines, which he called “artilects,” would ally with human supporters, the Cosmists, against a human resistance, the Terrans. 

“Belief in AGI is like belief in magic. It is a way of abandoning rational thought and expressing hope/fear for something that cannot be understood.”

It certainly doesn’t help the pro-AGI camp when someone like de Garis, who is also an outspoken supporter of “masculist” and anti-Semitic views, has an article in Goertzel’s AGI book alongside ones by serious researchers like Hutter and Jürgen Schmidhuber—sometimes called “the father of modern AI.” If many in the AGI camp see themselves as AI’s torch-bearers, many outside it see them as card-carrying crazies, throwing thoughts about AI into a blender with wild claims about the Singularity (the point of no return when self-improving machines outstrip human intelligence), brain uploads, transhumanism, and the apocalypse.

“I’m not bothered by the very interesting discussion of intelligences, which we should have more of,” says Togelius. “I’m bothered by the ridiculous idea that our software will suddenly one day wake up and take over the world.”

Why does it matter?

A few decades ago, when AI failed to live up to the hype of Minsky and others, the field crashed more than once. Funding disappeared; researchers moved on. It took many years for the technology to emerge from what were known as “AI winters” and reassert itself. That hype, though, is still there.

“All of the AI winters were created by unrealistic expectations, so we need to fight those at every turn,” says Ng. Pesenti agrees: “We need to manage the buzz,” he says.

A more immediate concern is that these unrealistic expectations infect the decision-making of policymakers. Bryson says she has witnessed plenty of muddle-headed thinking in boardrooms and governments because people there have a sci-fi view of AI. This can lead them to ignore very real unsolved problems—such as the way racial bias can get encoded into AI by skewed training data, the lack of transparency about how algorithms work, or questions of who is liable when an AI makes a bad decision—in favor of more fantastical concerns about things like a robot takeover.

The hype also gets investors excited. Musk’s money has helped fund real innovation, but when he says he wants to fund work on existential risk, it encourages researchers to talk up far-future threats. “Some of them really believe it; some of them are just after the money and the attention and whatever else,” says Bryson. “And I don’t know if all of them are entirely honest with themselves about which one they are.”

The allure of AGI isn’t surprising. Self-reflection and creation are two of the most human of activities. The drive to build a machine in our image is irresistible. Many people who are now critical of AGI flirted with it in their earlier careers. Like Goertzel, Bryson spent several years trying to make an artificial toddler. In 2005, Ng organized a workshop at NeurIPS (then called NIPS), the world’s main AI conference, titled “Towards human-level AI?” “It was loony,” says Ng. LeCun, now a frequent critic of AGI chatter, gave a keynote. 

These researchers moved on to more practical problems. But thanks to the progress they and others have made, expectations are once again rising. “A lot of people in the field didn't expect as much progress as we’ve had in the last few years,” says Legg. “It’s been a driving force in making AGI a lot more credible. “

Even the AGI skeptics admit that the debate at least forces researchers to think about the direction of the field overall rather than focusing on the next neural network hack or benchmark. “Seriously considering the idea of AGI takes us to really fascinating places,” says Togelius. “Maybe the biggest advance will be refining the dream, trying to figure out what the dream was all about.”