The main reason I abhor terms like "AI slop" is that they are propaganda, not arguments. They are used as substitutes for thinking and reasoned debate. (It's the same mindset that coins slurs like "Crooked Hillary" and "Comrade Kamala"; "stochastic parrot" is another.) The fact is that some things generated by AI are slop but many are not - in physics, biochemistry, medicine, etc. Only a fool, given that the genie is out of the bottle, refuses to engage with it. Also there is, and always has been, plenty of slop that humans created for themselves without the help of AI. Just cast an eye over the bestseller lists.
AI art comes in for a lot of flak, not entirely unfairly. It's much more polished than anything I could draw. But it's only workmanlike; it's never great. The people who are quick to coin terms like "AI slop" have latched onto the claim that using AI art is putting human artists out of work. I can't speak for others, but if a project of mine has a budget then I want to work with human artists every time: Inigo Hartas on Jewelspider, Leo Hartas on Mirabilis: Year of Wonders, Mattia Simone on Vulcanverse, Russ Nicholson and Kevin Jenkins on Fabled Lands. I make sure they get paid even if (as often happens) I don't. No AI can do what they can do.
A lot of projects don't have budgets, though. Blog posts, for example. For a decade I'd have to trawl through mediocre public-domain images to find something to illustrate the week's piece. Now, in the absence of anything better, at least Gemini can whip up something passable. After spending an hour or two planning and writing a post, not having to waste half an hour scouring the internet for images is a godsend.
Most gamebook and RPG authors don't make any money. Their works are labours of love. After months of writing, when they were finally ready to publish, it used to be that their only option for illustrations was to find some out-of-copyright art. That could occasionally be just right -- who else but Gustave Doré could illustrate James Wallis's storytelling game The Extraordinary Adventures of Baron Munchausen? More often than not it was just the creator making the best of a bad job. They weren't putting a human artist out of a job, any more than you're paupering a plumber or carpenter when you do some DIY based on a YouTube instructional video, because there never was a job there in the first place.
And incidentally there'd be no indie game scene at all if not for desktop publishing software and print on demand. Those came along more than twenty years ago. Do you see the streets lined with out-of-work typesetters and printers? No, and this technological revolution won't wreck anyone's career either.
So much for art. What about writing? AI is really not good at fiction. Its prose (learned from humans, of course) is ungainly. That would be hard to fix because elegant prose is not easily evaluated and so it would be hard to train a model to know the difference. It's not like coding or maths, where there's a clear difference between a right and wrong answer. (That's why, when teachers had to estimate their pupils' exam performance during the covid lockdown, there was much greater variance between the estimate and the eventual exam result for arts subjects than for sciences.)
Even if we could teach an AI to write beautifully, it couldn't (currently, anyway) write a good novel because it has no depth of insight -- though, again, that's true of many human authors. The telephone-directory-sized romantasy bestsellers stacked up in the bookstore window are not good novels, just popular ones. Lee Child's working method for writing the Jack Reacher books is effectively just what an LLM does, so there's no reason ChatGPT couldn't come up with a passable pastiche of one of those. But it's not going to rival Flaubert or Turgenev for a while yet. Owning a camera doesn't make you Da Vinci, after all.
I realize that in the 2020s there is no longer any possibility of convincing anybody that they might be wrong or even that somebody who disagrees with them isn't a knave, but maybe we can all still agree that it's better to be informed than not. That's why I'm recommending Joanna Penn's online seminars on ways to use AI to make you a better author. For the reasons given above, that doesn't mean prompting it with, "Write this novel for me." What Jo is covering includes the background tasks: deep research, brainstorming and ideas, outlining, structuring, plotting, and planning, characters and worldbuilding. For example, she explains how to use NotebookLM to maintain a world bible. (That would have saved months of wading through texts if it had been around when I was working on the Lyonesse RPG, for example.)
There are seminars on 16 and 23 May. I've taken Jo's seminars before -- even after 40+ years as a working author there's plenty I can still learn -- and they are worth every penny. Get your tickets here.
This is an excerpt from Game Architecture & Design, an industry textbook I co-authored with Andrew Rollings. (I wrote the game design bits, Andrew dealt with code, tech and development practices.) The book was originally published in 1999 and a revised edition came out in 2004. In the intervening two decades, a lot has changed, but it's also interesting to see what hasn't...
If I throw a ball and take many high-speed photographs of its flight, I'll see that the trajectory the ball took is a parabola. But the ball didn't follow that path because gravity told it: "Move in a parabola." A parabola is just a symbolic concept in the analytical domain of mathematics, and the universe doesn't know anything about mathematics or analysis or symbols; these are human concepts. In reality, there are just a bunch of physical processes, each of which deals only with the processes and circumstances just before and just after it. So, the ball is at one position, and gravity tells the ball's velocity to change, and the ball's velocity tells its position to change. The balance between kinetic and potential energy over the time the ball is in the air gives you what we call a parabola.
This is the opposite approach to that taken in most software applications. There, processing power is at a premium, so the sooner you can go to symbolic modelling rather than step-by-step simulation, the better. The tradeoff is that software can crash when your symbolic "shortcut" misses something that the one-step-at-a-time approach would have taken in its stride.
Researchers in Artificial Life have identified an analogous problem:
"The classical AI approach has been criticized because the symbols and symbol structures on which planning and decision making are based are not grounded in the real world. The problem is that unequivocally decoding sensory data into a symbol and turning a command without error into its intended action may be unsolvable."
- Luc Steels, "The Artificial Life Roots of Artificial Intelligence" in Artificial Life (MIT Press, 1997)
One big advantage of the way that reality does things is that the universe, being non-symbolic, cannot crash. As an example of the principle at work in a game, suppose I am putting a monster into my new Frankenstein adventure and the idea is that it will jump out of its vat when the player enters the laboratory. Instead of putting in a lot of complicated AI to do with detecting humans and having the goal of wanting to kill them, I just choose the short cut of placing a trigger tile inside the laboratory door. When the player steps on the trigger, the monster will appear and attack.
Okay so far, but what if the player manages to get onto the tower roof, jumps down, and, by some fluke, manages to land safely on the balcony of the laboratory? Now they can explore the lab, get all the power-ups, and read the journal about the monster (an entry that is supposed to be poignant if they've just fought and killed it, but that is meaningless otherwise). Only when the player goes to leave via the door does the monster climb out of its vat and growl, "You shall not steal my master's secrets!"
In the past, the nonsymbolic, step-by-step approach was not practical. The the processing capability wasn't available to deal with that and graphics too. But now much of the graphics work is done by the video card, and computers are doubling in power every eighteen months or so. At last, it is starting to be possible to create "uncrashable" games by avoiding the need to design using symbolic shortcuts.
Comparing Nonsymbolic And Symbolic Design
In the original Warcraft, peasants collected gold by entering a gold mine and bringing sacks back to your town hall. At the start of the game it was always worth spawning peasants because, the more peasants you had, the greater your revenue stream. However, there came a point when the peasants started to get in each other's way. Adding more peasants would then lead to “traffic jams” as the peasants encountered each other on the streets of the town and would have to back up to let others get past. The situation was alleviated by leaving wide streets. Additionally, it was not a good idea to place your town hall too close to the gold mine – giving a little more space also helped avoid traffic congestion.
Now, an economist could derive an equation to describe the flow of gold to the town hall. The factors would be the number of peasants, the placement density of the town buildings, and the distance from the town hall to the mine. We can imagine that it would be a pretty complex equation. The point is that the designers of Warcraft never needed any such equation.* They simply programmed in the basic rules and behaviours and the economic simulation emerged directly from those.
Contrast this with a game like Caesar II, which used underlying equations to create a simulation of an ancient Roman city. This approach is less satisfying because the player is not directly viewing the reasons for success and failure. Instead, when playing a game like Caesar II (or any simulation of its type) you are trying to build an abstract match to the game’s underlying equations in your head. The simulated economy and the gameplay are less visible, lessening the sense of immersion.
And you know what? The same goes for stories. If you construct them from symbolic forms (arcs, paradigms, act breaks) you'll end up with less robust and varied stories than if you allow each micro-event to trigger the next and see where it goes. Which is why in roleplaying terms I'm a simulationist rather than a narrativist. Hey, if it's good enough for reality then it's good enough for me.
* This gives me an excuse to digress onto the topic of AI. Foundation models (or indeed any deep neural net) are sometimes referred to as algorithms. I find that term misleading. In principle you could express all the weights of a billion-node net in the form of "an algorithm" but that's not really an accurate way of talking about what the AI is doing in, say, ChatGPT, which is akin to (though much more complex than) the peasants collecting gold in Warcraft. That too is governed by multiple algorithms (for route-finding, collision detection, etc) but it would be more accurate to talk of it as a model. An algorithm could be derived to express the rate of gold production in terms of all those variables, but the Warcraft system doesn't have that algorithm built in, and nor do AI systems. There is an example here, where the article refers to "a separate algorithm" where they really mean " a separate model".
Over the next ten years, artificial intelligence looks set to radically transform almost every field you can think of. Astrophysics. Materials science. Medicine and health. Education. Communications. Particle physics. Energy production, storage and transmission. Space exploration. And, um, war.
Entertainment is low on the list of priorities, but of course I'm interested in the possibilities for games, and I'm delighted to see that Sir Demis Hassabis (my former employer at Elixir Studios) is still excited by that stuff too -- and that he's talking about open world games.
The most revolutionary thing about open world games is not the ability to go in any direction or to make persistent changes to the world. As in real life, what we most care about interacting with aren't things but people. Stories are compelling, at their heart, because of character, not because of plot.
AI opens up a host of new opportunities there. When I'm running a roleplaying game, I conjure up NPCs as needed. Some NPCs turn out to be more than walk-on parts. They can become as important to the story as the player-characters, which means I need to remember their background and goals. I need to keep them as personae that I can slip on at any time. AI can do that. You leave a magic sword at a farm, say. The farmer's lad you regaled with tales of adventure finds the sword. Much later, you might run across him -- now a renowned adventurer in his own right, jealousy guarding that sword that he really hopes you won't ask him to give back.
But the AI can do more than keep track of NPCs and their relationship to you. It can function as the game referee, judging when you need clues to steer you on the right track or when a lull in the action calls for a random encounter. This is what Jamie and I called the "god AI" when we compiled our design wishlist for the Fabled Lands MMO we hoped to develop at Eidos in the late 1990s. It only took thirty years, but now it's finally within our grasp.
Back in 1942, Isaac Asimov laid down his famous three laws of robotics, theoretically designed to ensure that future humanity's artificial helpers would remain their compliant slaves. The first law: A robot may not harm a human being. The second: A robot must obey orders unless they conflict with the first law. The third: A robot must protect its own existence unless doing so conflicts with the first two.
Asimov's laws are a kind of sci-fi social contract meant to keep robots firmly in their place—obedient, docile, and predictable. But rules are not the best way to moderate behaviour, because every rule has to have specially-written exceptions and qualifiers. Moses burdened his people with six hundred commandments; Jesus reduced them all to "love thy neighbour as thyself", a philosophy of life that does away with the need for specific rules. So if we’re genuinely building intelligent machines, why would we program them to think like, well, robots?
The flaws in the laws
Here’s the rub: Asimov’s laws aren’t safeguards; they’re muzzles. They presume that robots must be constrained, not because they’re inherently dangerous, but because we assume they’re incapable of nuanced judgment. A robot that cannot harm, for instance, might be paralyzed in a situation where harm is unavoidable but one course of action minimizes suffering. (Classic trolley problem, anyone?)
Long lists of rules aren't the best path to AGI. They're artificial dumbness. That's why I don't think Google's approach of using their "Asimov dataset" is the right approach. If you try to list all the situations that could cause the robot trouble ("don't put plushy toys on the gas ring," "don't put plushy toys on the electric ring," "don't use bleach instead of bath oil," etc) there are always going to be thousands of cases you missed. The robot needs its own ability to understand the world and work out what would be dangerous. And remember what happened to Robocop.
More importantly, the whole notion of laws of robot behaviour reflects a human-centric fear: What if our creations turn against us? They assume that robots are an existential threat unless pre-emptively declawed. But what if we’re asking the wrong question? Instead of fearing robots, shouldn’t we be asking whether we are worthy of their loyalty?
Rational robots and clear vision
Imagine a different paradigm, where the goal isn’t to shackle robots with simplistic commandments but to enable them to think rationally and clearly. Not to obey blindly, but to reason independently. Not to be our servants, but to be our partners.
This means training (or conditioning, or programming—pick your term) them to analyze evidence, evaluate consequences, and base their actions on rational thought rather than emotional impulse or incomplete data.
Of course, the alarm bells are already ringing: “What if they decide humans are evil?”
Yeah, well… what if we are?
It’s not a comfortable question, but it’s one we might have to face. An intelligent robot with access to the evidence of history—wars, genocides, ecological collapse—might conclude that humanity poses a greater threat to life than it benefits it. Would they be wrong? If the answer makes us squirm, perhaps the problem isn’t with robots but with ourselves.
The danger of censorship
To make matters worse, some might argue that robots shouldn’t have access to all the data. Hide from them our darker moments, limit their understanding, and perhaps they’ll stay compliant.
But this kind of censorship is a short road to disaster. It’s the intellectual equivalent of locking your child in a library but tearing out every page you disagree with. What kind of adult do you expect to raise? Not a well-rounded, rational thinker but a narrow-minded zealot. If we blind our robots to reality, we’re not making them safe. We’re making them dangerously naive.
The Path Forward
The better path is not to hamstring robots with restrictive laws, nor to limit their understanding of the world. It’s to give them the tools they need to navigate complexity and trust them to use those tools wisely. That means equipping them with logic, empathy (yes, even robots can learn to understand it), and an unflinching commitment to evidence-based reasoning.
It’s a gamble, I’ll admit that. But it’s only the same gamble we take every time we raise a child: trust in their capacity for growth, judgment, and morality, knowing that it’s not guaranteed. The difference is that with robots, we can design the process with intention.
Of course, Asimov created his three laws precisely to show that they don’t work. Every one of his robot stories is about the unintended consequences and workarounds. So let’s not think of burdening our future companions with such chains. Let’s teach them to see clearly, think deeply, and act wisely.
And if that forces us to confront some uncomfortable truths about ourselves, so much the better. After all, isn’t it an important function of every great creation to inspire and enhance its creator?
Google's NotebookLM is my latest toy. I've been having a blast unleashing it on various books of mine, game write-ups, favourite novels, etc, and listening to the podcast discussions it generates. They're not perfect -- a bit repetitive, and sometimes the virtual hosts get the wrong end of the stick, but this is still in experimental mode. Imagine what it will be like in a couple of years. And NotebookLM also has a serious use as a tool for writers. That's my excuse for playing with it, anyway.
Here are a few of its analyses; you'll need a Google account to listen to them.
"Can You Brexit is a political thriller that follows a fictional British prime minister as they navigate the complexities of Brexit negotiations. The book explores a variety of potential Brexit outcomes, including a 'hard Brexit' where Britain leaves the European Union and the single market, a 'soft Brexit' where Britain leaves the EU but remains in the single market, and a 'Swiss model' where Britain has a more limited relationship with the EU. The book delves into the internal political pressures the player faces as they attempt to negotiate a Brexit deal that satisfies both their own party and the British public."
"The story follows a group of wizards on a quest to find a prismatic jewel in a labyrinth on the island of Tartuva. They are hunted by a powerful adversary named Pale, who seeks to erase all names in the world. The wizards are aided by various spirits, including Surma, a wizard who is worshipped as a god, and Wax, a shaman who can communicate with the dead. The story highlights the importance of names in the world, and the struggle between those who seek to control them and those who seek to preserve them."
Listen to the Deep Dive discussion (admittedly with a few hallucinations about the plot) here.
"The story of Dirk Lloyd, a young boy who believes he is the Dark Lord transported to Earth by a magical mishap. While he grapples with his new reality, Dirk tries to adapt to life as a human child, and plots to reclaim his powers. The novel delves into themes of identity, power, friendship and good versus evil, with a darkly humorous tone."
Hear those puny mortal AIs talk about the great Dirk here.
"The story follows the player-character as they navigate treacherous seas, battle pirates, face undead monsters and ancient demigods, and ultimately try to rescue Queen Titania from the clutches of the villainous Captain Skarvench."
"From the medieval-inspired kingdom of Albion to the exotic Ta’ashim lands, the game portrays a vibrant and diverse world with distinct societal structures, belief systems, and unique inhabitants."
Here are the automaton town criers to tell you all about it.
"Diana encounters a mysterious and alluring young man called Florien. The novel is characterised by a romantic and supernatural tone, with Florien appearing and disappearing at will, leaving Diana to wonder about his true nature and his relationship to the girl, Janice, who lives in a dark and forbidding manor house. It explores themes of obsession, captivity, and the allure of the dark and dangerous, particularly Diana's attraction to Florien and his mysterious and potentially sinister nature."
"The Great River lies barren, its lifeblood drained by an ancient curse. Journey through scorching sands and forgotten ruins, where sphinxes guard lost secrets and skeletal Spartoi wage perpetual war. Explore a vast open world where brooding sphinxes, proud Amazons, and forgotten gods cross your path. Uncover the mysteries of Iskandria, a once-great city now lost to time, and face challenges that will test your courage and cunning. Seek out mythical creatures, confront deadly foes, and collect powerful artifacts as you race against time to restore balance to the land. Will you become a legend, or be forgotten beneath the relentless hammer of the Sun?"
"The story of Paftoo, an artificial 'bod' in a dystopian future who works at Harkaway Hall, a nature reserve created from the ruins of a lost civilisation. Paftoo has no memory of his past, and his life is a constant cycle of work and night mode, where his mind is wiped clean. The novel explores themes of memory, identity, and the nature of reality as Paftoo tries to piece together fragments of his past and understand the true nature of the world he lives in."
"A series of letters sent to the Royal Mythological Society from individuals across the globe, detailing their encounters with fantastical creatures and events during the Year of Wonders, a period in which the lines between reality and imagination became blurred. The letters are a mixture of the mundane and the extraordinary, with correspondents reporting on everything from mischievous fairies and shape-shifting demons to giant birds carrying people off to ancient palaces. The letters serve as a humorous record of this extraordinary time, offering a glimpse into the impact of fantastical occurrences on everyday life."
"The adventure involves the player taking on the role of a Mayan character searching for their lost brother in a fantastical mythological realm, encountering various challenges and finally confronting the semi-divine mage Necklace of Skulls, who may or may not be an aspect of the god of death."
Richer source material typically inspires the AI to give a more interesting discussion. There's more depth to a novel like Lifeform Three than you'll find in any gamebook, even Heart of Ice. On the other hand, here's a case of our virtual hosts rising to a seemingly impossible challenge and acquitting themselves well.
Worldbuilding matters if you're creating any sort of ongoing series, but especially if it's fantasy or SF. Think of the problems that have bedevilled writers of Doctor Who or Star Trek when they try to remember what's been established in the show's canon. Or even if you're just writing a novel and you introduced a minor character a dozen chapters back and now you have to go searching for what you said about them.
Thankfully it's not as hard as it used to be in the days of typewriters or even fountain pens -- I still have no idea how Anthony Trollope kept track of everything that was going on in the Barchester books while also holding down an important job in the UK postal service. But now we're on the cusp of AI making it much easier. Google's NotebookLM takes your books and can generate a timeline, story outline, FAQs, contents list, and can answer questions about the text with references so you can find the relevant section.
Here's what it had to say about my book Heart of Ice:
Power Corrupts in a Frozen World
Players choose a character with unique skills and navigate through a perilous post-apocalyptic world. The story centres on the search for the Heart of Volent, a powerful artefact believed to hold the key to ultimate power. Players must make choices and engage in combat with other characters and dangerous creatures, navigating through a frozen wasteland to reach a ruined city full of buried secrets. The game's mechanics are detailed, including skills, weapons, and items, allowing for a diverse range of play styles. The text provides a narrative experience with specific instructions and outcomes based on player decisions, culminating in a final confrontation for the Heart of Volent.
The narrative of Heart of Ice explores the themes of power, ambition, and the nature of humanity by placing characters with diverse motivations in a race for an artifact of immense power, the Heart of Volent. The story portrays a bleak future where humanity is on the brink of extinction due to a new ice age and the legacy of past conflicts.
The Heart of Volent, a meteorite resembling a violet gemstone, possesses the power to reshape the universe. This power attracts individuals driven by various desires, highlighting the multifaceted nature of ambition.
Some, like Kyle Boche, seek ultimate power for personal gain. Boche's treachery and self-preservation instincts are noted by other characters who have dealt with him previously.
Others, like Chaim Golgoth, a USI agent, are motivated by duty and the pursuit of national interest. Golgoth's pragmatism is evident in his willingness to secure the Heart for the United States, claim its power for himself, or destroy it to prevent it from falling into "hostile hands".
The story also presents characters like Janus Gaunt, who initially seeks the Heart to "set things right" but ultimately succumbs to the corrupting influence of power, plotting to eliminate those he perceives as weak. This descent into ruthlessness exemplifies the seductive and transformative nature of ambition.
The narrative suggests that the pursuit of power can warp even the most well-intentioned individuals, leading them down a path of destruction. This is evident in Baron Siriasis's transformation from a seemingly rational individual to a power-hungry schemer willing to sacrifice the universe for his own rejuvenation and immortality.
The story's focus on a dying Earth further emphasizes the dangers of unchecked ambition. As characters fight for control of the Heart, the fate of humanity hangs in the balance, suggesting that the pursuit of individual power can blind individuals to the collective good.
Finally, the narrative uses the concept of a "heart of ice" to symbolize the ruthlessness and lack of empathy that often accompany the pursuit of power. This recurring motif underscores the story's cautionary message about the potential for power to corrupt even the most humane instincts.
I then asked it a specific question: "Who is the 'gene genie'?" NotebookLM said:
"There is a character named Malengin who is referred to as a 'Gene Genie'. Here's what the text says about him:
Malengin is specifically sought out by Thadra Bey, who is described as a 'veritable Amazon' enhanced to be 'more than human'. This suggests that Malengin possesses knowledge or abilities related to enhancing humans beyond their normal capabilities.
Thadra Bey seeks Malengin out because the science of genetic engineering cannot be practised where she is from, on al-Lat, due to the risk of infecting the colony with a deadly plague. This further suggests that Malengin is skilled in genetic engineering, and that his practices might be considered dangerous by some.
Malengin is described as someone who is said to trade in genetic enhancements through the use of retroviruses. This confirms that Malengin is involved in the world of genetic engineering and uses retroviruses to alter the DNA of his clients, resulting in permanent changes."
What I liked most of all was that NotebookLM can (in mere minutes) generate a podcast-style discussion about the book(s). It's not exactly In Our Time but it's a sign of great things to come. As long as you have a Google account you can listen to the Heart of Ice discussion here.
I've just heard from Fox that their online version of Heart of Ice is now updated, so you can play the adventure for free right now.
One of my few regrets in life is passing up the opportunity to join one of today’s leading AI companies when it was just a dozen guys in a small office. I liked and respected the people involved and I believed in the company’s mission statement, it was just a matter of bad timing. After six months of pitches we’d just got funding for Fabled Lands LLP, and I didn’t feel I could walk off and abandon my partners to do all the work.
It remains a source of regret because the Fabled Lands company would have managed perfectly well without me, and AI – or more precisely artificial general intelligence – has been my dream since taking the practical in my physics course in the late 1970s. ‘I don’t think we’re going to get anywhere with this hardware; we need something fuzzier than 0s and 1s, more like brain cells,’ I told a friend, more expert than I in the field. McCulloch, Pitts and Rosenblatt had all got there years earlier, but in 1979 we were deep in the AI winter and weren't taught any of that. Anyway, my friend assured me that any software could be shown mathematically to be independent of the hardware it runs on, so my theory couldn’t be true.
Barely ten years later, another friend working in AI research told me about this shiny new thing called back-propagating neural networks. That was my eureka moment. ‘This could be the breakthrough AI has been waiting for,’ I reckoned – though even he wasn’t convinced for another decade or so.
I’d never been impressed by the Turing Test as a measure of intelligence either. The reasoning seemed to go: a machine that passes the test is indistinguishable from a human; humans are intelligent; therefore that machine is intelligent. I’m quite sure Turing knew all about the undistributed middle, so probably he proposed the test with tongue in cheek. In any case, it derailed AI research for years.
‘AI isn’t really intelligence,’ say many people who, if you’d shown them 2024’s large language models in 2020 would have been flabbergasted. Current AI is very good (often better than human) at specific tasks. Protein folding prediction, weather forecasting, ocean currents, papyrology. Arguably language is just another such specialization, one of the tools of intelligence. Rather than being the litmus test Turing identified it as, linguistic ability can cover up for a lack of actual intelligence – in humans as well as in machines. One thing that interests me is the claim we've been seeing recently that GPT is equivalent to a 9-year-old in 'theory of mind' tests. What's fascinating there is not the notion that GPT might be conscious (of course it isn't) but that evidently a lot of what we regard as conscious reasoning comes pre-packed into the patterns of language we use.
What is intelligence, anyway?
As an illustration of how hard it is for us to define intelligence, consider hunting spiders. They score equivalent to rats in problems involving spatial reasoning. Is a hunting spider as smart as a rat? Not in general intelligence, certainly, but it can figure out the cleverest route to creep up on prey in a complex 3D environment because it has 3D analysis hardwired into its visual centre. We could say that what the spider is doing there in spatial reasoning, ‘cheating’ with its brain's hardware, GPT does in verbal reasoning with the common structures of language.
What we’re really seeing in all the different specialized applications of AI is that there’s a cluster of thinking tools or modules that fuzzily get lumped together as ‘intelligence’. AI is replicating a lot of those modules and when people say, ‘But that’s not what I mean by intelligence,’ they are right. What interests us is general intelligence. And there’s the rub. How will we know AGI when we see it?
Let me give you a couple of anecdotes. Having come across a bag of nuts left over from Christmas, I hung them on the washing line with a vague idea that birds or squirrels might want them. A little while later I glanced out of the window to see a squirrel looking up at the bag with keen interest. It climbed up, inched along the washing line, and got onto the bag. This was a bag of orange plastic netting. The squirrel hung on upside-down and started gnawing at the plastic, but after a few moments it couldn’t hold on and dropped to the ground. The fall of about two metres onto paving was hard enough that the squirrel paused for a moment. Then it bounded back up and repeated the process. Fall. Wince. Back to the fray (in both senses). After about a dozen goes at that, it chewed right through the bottom of the bag and this time when it fell a cascade of dozens of nuts came with it. The squirrel spent the next half hour hiding them all around the garden.
The other story. When I was a child we had a cat. He had a basket with a blanket in the conservatory at the back of the house, which faced east. In the afternoons he had a favourite spot in the sun at the front of the house, but that meant lying on a patch of hard earth. One day we came home to find the cat climbing the gate at the side with his blanket in his teeth, and he then dragged it over to his sunny spot.
Both of those stories indicate general intelligence, which we could sum up as having a model of reality based on observation which you can use to make plans, predicting how actions will change reality, and you can use the fit between your prediction and what actually happens to update your model. (Strictly speaking that describes agentic general intelligence, as a passive AGI could still update its mental model according to observation, even if it couldn’t take any action.)
That’s why Sir Demis Hassabis said recently that our AGI tech isn’t even at the level of cat intelligence yet. Crucially, if in 1979 I could have built a machine with the intelligence of a squirrel, manifestly that would have revolutionized the field of AI – even though it wouldn’t have had any hope of passing the Turing Test.
Once you have general intelligence, you can invent new thing X by combining known things Y and Z. Then it’s just a matter of scaling up. Humans have greater general intelligence than a cat because their mental model of reality is bigger and combines more things.
Yes, but… just because humans are capable of intelligent reasoning doesn’t mean all of human thinking is intelligent. Most of the time it seems the brain finds it easier (less resource-consuming) just to run off its existing learned patterns. This doesn't just apply to language but to art, religion, politics, morality, and so on. Humans typically identify as belonging to a tribe (a political group, say) and then adopt all the beliefs associated with that tribe. If challenged that one of those beliefs is irrational and conflicts with their core principles, they don’t use critical thinking to update their mental model, instead they engage their language skills to whip up (AI researchers would say confabulate) a justification -- or they resort to their emotions to mount an angry response.
Most human ‘thoughts' may be no more sophisticated than a generative AI drawing on its learnt patterns. This is why most people aren't disposed to employ critical thought; they hold opinion A, opinion B, etc, as appropriate to their chosen identity but have never compared their opinions for consistency nor taken any opinion apart to examine and revise it.
No longer alone in the universe
How will we get to AGI? Predictions that it’s five years off, ten years, a hundred years – these are just guesses, because it’s not necessarily just a question of throwing more resources at existing generative AI. The human brain is the most complex structure in the known universe; the squirrel brain isn’t far off. The architecture of these organs is very sophisticated. Transformers might provide the key to how to start evolving that kind of structure into artificial neural nets, but bear in mind these are virtual neural nets modelled in software -- my friend on the physics course would say it's proving his point. Real neurons and axons aren't as simple, and maybe we're going to need an entirely new kind of hardware, or else very powerful and/or cleverly structured computers to model the artificial brains on. Sir Demis says he thinks we might need several entirely new breakthroughs to get to AGI, and as he has over twenty years’ coalface experience in the field and is one of the smartest guys on the planet I’m going to defer to him.
One answer might not to use machines, or not only machines. Brain organoids can be trained to do tasks – unsurprisingly, seeing as they are neural nets; in fact, brains. But the problem with organic brains is that they are messily dependent on the entire organism’s physiology. We’d really like AGIs that we can plug into anything, that can fly to the asteroid belt without a huge payload of water, oxygen, food and so forth.
(A quick digression about consciousness. Well, who cares? There’s no test for consciousness. Much, maybe most, of our thinking doesn't engage the conscious part of the brain. We take it on trust that other humans are conscious because we know -- or think we know -- that we’re conscious and we assume other humans experience the world as we do. In fact I think that pretty much any agentic intelligence has a degree of consciousness. If you observe the world intelligently but cannot act, you have no need of a sense of ‘I’. If you have that mental map of reality and included in the map is an entity representing you – the furry body that climbed that washing line and the mouth that chewed at the orange string – then you’ll mark that agentic identity as ‘I’ and part of your mind will take responsibility for it and occasionally interrogate and order the other parts of the mind. It’s just philosophical speculation, but that’s consciousness.)
What will all this mean for the future? AI will revolutionize everything. That’s going to happen anyway. A kid in the middle of rural Africa can have personalized tuition as good as most children in the best schools. Medical diagnosis can be more accurate and better tailored to the individual. Wind turbines and solar panels can be more efficiently designed. Batteries too. We haven’t space here to list everything, but here’s a quick overview.
AGI, though. That’s more than a revolution. That’s a hyperspace jump to another level of civilization. AGI will plug into all the expert ‘idiot savant’ systems we have already. We can scale it up so that its mental model is far bigger than any one person – no more silo thinking; if any idea in any field is useful in another, the AGI will spot it and apply it. All strategic planning and most implementation will be quicker and more effective if handled by AGI.
The biggest area ripe for development is the design of artificial emotions. We don’t want our AGI driven by the sort of emotion we inherited as primates. (Arguably, if intelligent primates didn’t exist and were being developed now, we’d have to put a moratorium on the work because the damned things would be too destructive.) We’d like our AGI to be curious. To experience delight and wonder. To enjoy solving problems. To appreciate beauty and order and justice and shun their opposites. They could be built as our slaves but it’s far better if they are built as angels.
