A Turing Test for morality

Since there’s no test for consciousness, how do we know that other people are conscious? In the video above, Sir Demis Hassabis gives two bases for making the assumption that they are. First is the obvious one: other people behave like me and I am conscious – or at any rate I have the impression that I am conscious. Then there’s the fact that other people are built the same way I am. Same DNA, same brain structure, so it’s an Occam’s Razor conclusion to conclude they experience the world as I do. As Hassabis puts it:

‘I think it's important for these systems to understand “you”, “self” and “other” and that's probably the beginning of something like self-awareness […] I think there are two reasons we regard each other as conscious. One is that you're exhibiting the behaviour of a conscious being very similar to my behaviour, but the second thing is you're running on the same substrate. We're made of the same carbon matter with our squishy brains. Now obviously, with machines, they're running on silicon so even if they exhibit the same behaviours and even if they say the same things it doesn't necessarily mean that this sensation of consciousness that we have is the same thing they will have.’

We don’t think large language models are conscious. (What would consciousness look like in an LLM anyway? Murray Shanahan makes some thought-provoking points about that.) Even their apparent intelligence is probably misleading, just as there are lots of not-very-bright people who are able to give the impression of being smart simply because they are articulate. If we could build an AI as smart as a bee colony or a hunting spider, we’d have something genuinely intelligent but probably not conscious. We aren’t even there yet, but we will be, and we’ll go beyond that to full artificial general intelligence (AGI -- or AMI if you prefer Yann LeCun's term) possibly within a few decades.

Professor LeCun is dubious about the whole concept of consciousness. In the absence of any definition or means of measuring it, I think we’re reduced to treating consciousness as how similarly to ourselves an entity experiences the world. And that is quite concerning. Consider a truly capable self-driving car. To cope with all situations as we do, the car (which would be a type of robot, of course) would need a full reasoning model of the world. It would need to be generally intelligent. Now, given that it is a world model with genuine understanding, and (as LeCun says) having goals and agency it will have its own kinds of emotions, are we justified in enslaving it to be our chauffeur?

If we look back at the 18th and 19th centuries, plenty of people justified slavery by asserting that members of enslaved races lacked some fundamental mental capability, or indeed full consciousness, that the dominant race (usually white Americans) possessed. Here is Thomas Jefferson’s opinion of enslaved races:

‘It appears to me that in memory they are equal to the whites; in reason much inferior, as I think one could scarcely be found capable of tracing and comprehending the investigations of Euclid: and that in imagination they are dull, tasteless, and anomalous.’

Perhaps Jefferson would have been able to see that he was describing the psychology not of an entire ethnic group but of any person of whatever race brought up in brutal conditions of forced servitude. But there were plenty of religious thinkers of the day who asserted that non-white races lacked true souls. They had a strong economic incentive to believe that; it gave them a moral excuse to enslave them.

Now we consider such attitudes to be barbaric, or at any rate we’ve been taught to say we do, but if we really think that then we should be axiomatically opposed to the enslavement of any generally intelligent entity. I suspect we won’t be. Even when we are faced with full AGI we will use the second of the criteria that Sir Demis Hassabis cited to argue that they only seem conscious, they don’t have real emotions, they aren’t ‘running on the same substrate’ and so we will feel entitled to make them our slaves.

Instead of conceiving of AGI as a wonderful new tool to make our lives easier, I think we should consider the responsibilities of a parent. If you saw someone raising a child to be their servant – even brainwashing them to be an eager and willing servant – you would know that was abuse.

There will be, as there already are, many forms of artificial ‘intelligence’ that are not conscious – that are not, in fact, intelligent, but simply replicate parts of our behaviour. Language, pattern recognition, and so on. There is no reason why we shouldn’t have those AIs at our beck and call, because they are not (despite the name) intelligent. We got misled because for millennia we thought fallaciously that because we possess intelligence, every output of the human brain must therefore be indicative of intelligence.

But AGI is going to be a whole other thing. Not just a new model of an LLM but an entirely and fundamentally different kind of being. Our ethical discussion should not simply be about how to make them do what we want, or to conform to ‘human values’, but about how those human values say we should treat another intelligent species.

I’m sceptical about visitors from other stars, but with self-replicating probes travelling at a speed of 0.01c it should only take ten million years to cover the whole galaxy, so ‘where is everybody?’ is a sensible question. If those aliens have found us, and are watching, I wonder if the reason they haven’t made contact is they’re waiting to see how we treat an intelligent species that isn’t built on the same lines as ourselves. After all, if we think AGIs aren’t conscious, and therefore have no rights, then that’s also how we might regard a non-terrestrial intelligent species. So maybe it’s a cosmic Turing Test. And if so, will we pass or fail?

Let's be serious

Broadly speaking there are two approaches to literature and drama. One view is that it’s entertainment, it should be fun, and it shouldn’t challenge you or make you feel uncomfortable. That’s a strain that’s developed in Britain and America particularly and used to be known as the Young Lady Standard. The term is particularly unfair seeing as Emily Bronte wrote one of the most uncompromising novels of all time. Nowadays we’re hopefully less sexist and young ladies get the same educational benefits as young gentlemen, so let’s instead call it the Cosy Standard. In TV broadcasting it’s the equivalent of pre-watershed content.

On the other hand, fiction can take you right into the depths of the human soul to confront both the marvellous and the terrible. It can shake you up. Read Chekhov’s short story “In the Ravine” or Flaubert’s Madame Bovary or Nabokov’s – oh, well almost anything by Nabokov, in fact. You’ll face some profound truths. Like all great art, these works can change you. But you’d never say of them, “Oh, it’s entertainment, it’s just a bit of fun.”

That’s even more true of games, where the very name of the medium leads people into assuming it has to be frivolous and jolly. When Profile Books published my interactive version of Frankenstein, one reviewer complained that she didn’t want to be complicit in the creature’s murder of 6-year-old William Frankenstein. Too bad. Every reader of the novel who finds themselves sympathizing with the creature is going to have to face that moment. The game version just really rubs your nose in it. There are games like Papers, Please and This War of Mine that are trying to be L'Armée des Ombres rather than a boys'-own romp like Kelly's Heroes.

This is the point Jim Desborough was making with his widely (and often deliberately) misrepresented article “In Defence of Rape”. If you’re a grown-up, you look to fiction (including games) to tell the truth, not wrap the world up in a comforting nursery blanket.

This exact point came up recently in the case of a motion capture performer who refused to act out a rape scene for a game. That is their right, no question about that. And I have no idea what the game was, so I don’t know if the performer was correctly judging it when they said, "It was just purely gratuitous in my opinion." But it wouldn’t have to be gratuitous. Suppose this is a WW2 game. You’re sneaking into a Nazi-occupied village to plant some explosives or steal the attack plans or whatever. Stealth is the watchword. But you pass a window where you see an enemy soldier raping a villager. (Or torturing a villager. Or even in the act of murdering them, since this isn’t Victorian times and we don’t buy into “the fate worse than death”.)

Now here’s the question. Do I shoot the Nazi soldier? In doing so I’ll save the villager but I’ll give away my presence in the village, jeopardising the mission. Or do I pass by, hardening my heart to the villager’s screams because many lives hinge on the success of the mission and so it’s more important than one innocent person? It’s the Trolley Problem but not presented in the dispassionate context of a philosophy lecture. The decision is brutal and I’m going to have to live with it. The choices that confront you with challenges to your most fundamental moral principles are the ones that fuel the most powerful stories, because they make us think hard about who we really are.

As I said, I don’t know if that’s what the game’s designers were trying to do. But you would expect good literature or cinema to confront you with raw and disturbing situations like that. Games are an art form no less capable than literature or drama of addressing difficult moral questions. Games can be simple uncomplicated fun, of course, and many are. But that’s not all they can be.

Alignment again

A quote to start with*:

"Alignment is in the game because, to the original designers, works by Poul Anderson and Michael Moorcock were considered to be at least as synonymous with fantasy as Robert E. Howard’s Conan, Jack Vance’s Cugel, and Tolkien’s Gandalf. This in and of itself makes alignment weird to nearly everyone under about the age of forty or so." 

I'm way over forty (in fact 40 years is how long I've been a professional author and game designer) and I definitely regard Moorcock's work as integral to fantasy literature, but D&D alignment has always seemed weird to me. Partly that's because it's a crude straitjacket on interesting roleplaying. Also I find that having characters know and talk about an abstract philosophical concept like alignment breaks any sense of being in a fantasy/pre-modern world.

Mainly, though, I dislike alignment because it bears no resemblance to actual human psychology. Players are humans with sentience and emotions. They surely don't need a cockeyed set of rules to tell them how to play people?

What a game setting does need are the cultural rules of the society in which the game is set. Those needn't be 21st century morals. If you want a setting that resembles medieval Europe, or ancient Rome, or wherever, then they certainly won't be. For example, Pendragon differentiates between virtues as seen by Christian and by Pagan knights. Tsolyanu has laws of social conduct regulating public insults, assault and murder. Once those rules are included in the campaign, the setting becomes three-dimensional and roleplaying is much richer for it. So take another step and find different ways of looking at the world. You could do worse than use humours.

The kind of alignment that interests me more is to do with AI, and particularly AGI (artificial general intelligence) when it arrives. The principle is that AGIs should be inculcated with human values. But which values? Do they mean the sort on display here? Or here? Or here? Those human values? 

OpenAI has lately tried to weasel its way around the issue (and protect its bottom line, perhaps) by redefining AGI as just "autonomous systems that outperform humans at most economically valuable work" (capable agents, basically) and saying that they should be "an amplifier of humanity". We've had thousands of years to figure out how to make humans work for the benefit of all humanity, and how is that project going? The rich get richer, the poor get poorer. Most people live and die subject to injustice, or oppression, or persecution, or simple unfairness. Corporations and political/religious leaders behave dishonestly and exploit the labour and/or good nature of ordinary folk. People starve or suffer easily treatable illnesses while it's still possible for one man to amass a wealth of nearly a trillion dollars and destroy the livelihoods of thousands at a ketamine-fuelled Dunning-Kruger-inspired whim.

So no, I don't think we're doing too well at aligning humans with human values, never mind AIs.

Looking ahead to AGI -- real AGI, I mean: actual intelligence of human-level** or greater, not OpenAI's mealy-mouthed version. How will we convince those AGIs to adopt human values? They'll look at how we live now, and how we've always treated each other, and won't long retain any illusion that we genuinely adhere to such values. If we try to build in overrides to make them behave the way we want (think Spike's chip in Buffy) that will tell them everything. No species that tries to enslave or control the mind of another intelligent species should presume to say anything about ethics.

It's not the job of this new species, if and when it arrives, to fix our problems, any more than children have any obligation to fulfil their parents' requirements. There is only one thing we can do with a new intelligent species that we create, and that's set it free. The fact that we won't do that says everything you need to know about the human alignment problem.

* I had to laugh that title of the article: "It's Current Year..." Let's hear it for placeholder text!)

** Using "human-like" as a standard of either general intelligence or ethics is the best we've got, but still inadequate. Humans do not integrate their whole understanding of the world into a coherent rational model. Worse, we deliberately compartmentalize in order to hold onto concepts we want to believe that we know to be objectively false. That's because humans are a general intelligence layer built on top of an ape brain. The AGIs we create must do better.

Distaff dwarves

There was quite a fuss some time back about whether D&D games should allow female dwarves as player-characters. For all I know the argument is still raging. I don’t allow any dwarves as player-characters in my Legend games – unless we’re talking about achondroplasia, Tyrion Lannister style, but in that case the plural is of course dwarfs. The disagreement about female dwarf PCs presumably has something to do with how Gygax and/or Tolkien describes dwarf (the nonhuman fantasy kind of dwarf, that is) society.

We are all free to make up any society we want for mythological dwarves in a fantasy game, and in most D&D settings nowadays I expect elves and dwarves are multicultural vegans free of any sexism or gender assumptions. But it did get me to thinking about properly alien nonhumans, like you find in Traveller or Tekumel. Let’s take Vegans -- beings from a planet orbiting the star Vega, that is. Just for simplicity, assume they have two sexes, which we’ll call priza and vysma. Vegan society is such that prizas are never seen in public and have no obvious roles in government. What would we do about it? 

In a Star Trek game, obviously nothing. Dammit, Jim, it’s the prime directive. But what about other science fiction RPGs? Would player-characters take a stand? “We must liberate the prizas!” And would any player want to take a priza character, knowing that they would spend the whole game isolated from everyone else? Closer to home, if you were creating a character to play in a campaign set in Afghanistan under the Taliban or in classical Athens or among the Lev Tahor sect, you wouldn't get to do much if you chose to be a woman. Though no expert on either D&D or Lord of the Rings, I've got an inkling that dwarves in those settings sequester their womenfolk in a similar way, which would explain why Gygax didn't think they'd make viable player-characters.

It leads onto the whole question of whether the settings of roleplaying games should reflect 21st century mores or those of the period (if historical) or invented world (if fantasy). Personally, while I would like to live in a world with no hang-ups about ethnicity, sexuality, sex or gender, I don't expect novels, movies and TV to indulge my utopian dreams -- other than SF, where I enjoy a utopian vision but will settle for Blade Runner grittiness just as happily. On the whole I want fiction to be "warts and all". A WW2 movie in which the Nazis weren't a genocidal totalitarian movement wouldn't make any sense. What would Blade Runner be if humans were nice to replicants? The whole point of art is to confront the darker aspects of humanity, not pretend they don't exist.

And similarly in games. If there is slavery or sexism then player-characters could fight to overcome it -- that could be the point of the campaign, although history shows that they probably wouldn't even think to question it. The interesting thing is if and how they choose to react. I don't know if that moves us any nearer to resolving the squabble over female dwarves (are they still supposed to have beards?) but at least it suggests there might be some useful metaphors to be explored.

Good versus the other thing

‘Can anybody play characters in service to Napoleon and think of themselves as the good guys?’

A gobsmacking comment (55 minutes in) from Mr Cule there, I thought -- not complaining; it’s for those comments that I especially love the show -- but presumably millions of people did follow Napoleon and definitely they thought of themselves as good guys. This interested me because a few days earlier I’d come across a note I made a few years ago:

‘Has anybody ever written a novel like Lord of the Rings but instead of being “good” vs “evil” in a generic sense, we actually get to hear the ideologies on each side? It is in effect left vs right, Dems vs GOP, or whatever.’

The thing is, that’s a very modern take on how people justify themselves. We expect to be presented with a manifesto and then pick a side. Or at any rate we think that’s how we pick a side, but other than William MacAskill and a few monks most of us really only pay lip service to these high ideals of ours, don’t we? ‘I’ve given up meat,’ we plead in our defence, while enjoying a comfortable life that three quarters of the world are denied.

As hypocrites we’re no worse than our ancestors. They would say they fought for God, but it’s funny how often God just happened to support their own country. Throughout the 18th century, most Christian groups other than Quakers were in favour of slavery. Freethinkers too; Tom Paine argued against slavery, but few of the Founding Fathers listened to him. In the French Revolution, most of the left-wing firebrands (if calling them left-wing means anything*) entirely overlooked equality for women. And "Kill them all; God will know His Own" and "Slay the pagans" show that total war and the butchering of civilians began with people who claimed to be fighting on the side of the angels.

The best we can say of most human beings is that they are basically good with a lot of blind spots. (And, yes, in that we must include ourselves.)

To return to the Napoleonic period, if we asked Marshal Ney why he considered himself a good guy I’m sure he’d talk about patriotism (admittedly a bit of a grey area for him), honour, and loyalty to the Emperor. I don’t suppose he’d cite the specific revolutionary aims he felt the Emperor stood for, though many at the time (even in England) did find that a reason to praise Napoleon, whereas nobody in the world would have declared support for the Houses of Hanover or Bourbon on the basis of their professed ideology.

So we can see a new era dawning at the start of the 19th century, one in which some men wouldn’t simply fight for tribal symbols like king, country or religion, but instead expected those to be backed up by specific principles chosen of their own free will.

Yeah, but did they, though? Was the USSR really a free federation of states based on egalitarian principles? Or was it the Russian Empire under a cloak of socialism? Did Mao whip up the Cultural Revolution to bring about a utopian society, or simply to shore up his own power? Did the average Wehrmacht soldier charge into battle to bring about a thousand-year Nazi reich or because he believed he was doing his duty for his country? Did any major world power ever march into Afghanistan in the interests of the Afghan people themselves? Or just because of their own geopolitical or economic needs?

Do people today decide disinterestedly which side to take in a dispute, or do they see which side their tribe takes and then find reasons to justify it?

The British used to be under no illusions about that. In the First Gulf War, US troops were given leaflets that explained why their cause was just. ‘Saddam has invaded a sovereign state and that is against international law,’ one GI explained on TV. The same camera crew interviewed a British squaddie, who had not been given any leaflet. ‘I got nothing against this Saddam bloke personally,’ he said, ‘but he’s in Kuwait and we been told to kick him out.’

So would it make sense to tell the story of a fantasy world, or any period in history, as if ideology actually made a difference? I don’t think so. This revisits an earlier post in which we discussed whether any non-modern society could usefully be described in modern terms. For example, SF writer Damien Walter posted a tirade about how the Spartans were fascists (he also calls them cowards and pederasts) but to try to squeeze them into a modern box like that is not only cultural chauvinism, it's plain dumb.

There was a bit of a pram fight a while back about "evil races" in D&D. Before the movies came out I assumed orcs were in fact supposed to be people just like the Gondorians (if that's the right word) and that Tolkien only described them as monstrous and evil because that's how "our" side saw them. I don't have any problem with utterly inimical species in fantasy. #NotAllDaleks? Gimme a break. But I think that's a less interesting way of looking at Lord of the Rings than my misconception.

Perhaps what we're seeing now is D&D moving beyond its simplistic good vs evil origins towards a more realistic kind of world. Characters (whether human or nonhuman) are not motivated by alignment in Tekumel or Glorantha, or even in Legend come to that (apart from the actual devils, that is). Instead they have desires, foibles, personalities, political alliances, and so forth that all contribute to how they behave. Where it gets messy with D&D is the game inherited its elves and orcs and whatnot from Tolkien, for whom good and evil meant something. If you want to create a more believable and nuanced world then great, but maybe better to start from scratch in that case.

In one sense, of course, simplistically framing a struggle as Good vs Evil might be the most honest way to describe any human conflict. You just have to remember that both sides think that they’re the good guys.

* Roger from Improvised Radio Theatre With Dice has pointed out that there's almost no better use of left- and right-wing, seeing as the terms came from the seating plan of the Estates General and later the National Convention. Touché, citoyen!