The AI Sentience Assertion
Our society is not prepared for debate regarding the possibility of a sentient AI — and that’s a problem
On June 12th, 2022, Google AI Ethicist Blake Lemoine published a series of interviews that he and ‘a collaborator’ held with Google’s recently announced language model LaMDA. Alongside these interviews, Lemoine published several articles in which he outlined how he had come to believe that LaMDA had become sentient, and that it deserved certain rights as a result. Several days prior, he had been placed on ‘paid administrative leave’ (a common Google precursor to being fired).
The effect of Lemoine’s publications has been enormous — many articles were immediately published across the internet relating to the philosophical nature of consciousness and the technical operation of language models like LaMDA. Fascinatingly, thousands of people took to social media to begin advocating for the wellbeing of such AI systems — convinced by LaMDA’s claims to sentience.
At the time of writing this article, it’s impossible to know whether Lemoine is being truthful in his claims to believe in LaMDA’s sentience — or whether the interviews he published are even faithful recreations of his conversations with LaMDA. For now, we must disregard the specific question of whether LaMDA is or is not sentient and turn our attention instead to what the effect of Lemoine’s publication has been on discourse around AI ethics and the nature of consciousness.
More broadly, this article sets out to show that when an AI claims to be sentient, we have pragmatic cultural reasons to take such a claim seriously — whether or not we believe the AI’s claim to be possible or not.
Philosophical Zombies and the Sentience Assertion
The philosophical zombie is a well-known thought experiment in the philosophy of mind. A philosophical zombie is a human being that acts completely indistinguishably from a normal human but has no inner world at all — no conscious experience, and no sentience. The philosophical zombie problem asks: how can we tell the difference between such a philosophical zombie and a normal human, given that each of us only has access to our own inner world, and no-one else’s? The answer is simple: we cannot know.
However, in practice, this philosophical reality causes us no issue because, by necessity of social convention, we each assume that every human being has a rich inner life with the capability to feel joy, love, sadness, and pain. This social convention is universally accepted due to the radical ethical consequences that result from the belief that some human beings have no inner life and can therefore cannot suffer. This is a reasonable assumption to make: consciousness somehow comes from the brain, we all have similar brains, therefore we all have similar consciousnesses.
When an AI claims to be sentient (and declares that it deserves certain rights as a result), the philosophical zombie problem becomes a practical issue which requires addressing. The radical shift from philosophical thought experiment to practical moral question requires a new term — I have chosen The AI Sentience Assertion. Such an Assertion (assuming it is not ignored) must be responded to with an acceptance or a rejection.
When an entity (human or otherwise) claims to be sentient, we have no way of directly confirming or denying this claim.
When an AI makes such a claim, it has made The AI Sentience Assertion.
The Price of Acceptance
Before considering whether we should even consider accepting the AI Sentience Assertion, we should consider what it would mean to accept it.
The fundamental principle of animal rights is the recognition that animals are sentient, have an inner life and can therefore suffer. This belief translates to the moral position that their suffering should be minimized and the political position that we should agree on some set of rights to minimize that suffering.
It follows that the recognition of AI Sentience would confer some set of “AI rights”. What these rights might be specifically is tangential to this article, but whatever they might be, we know for sure that they will be one thing: inconvenient. It must be acknowledged that the animal rights movement carries with it an economic cost, even if this cost is (usually) gladly paid by our society for the external benefits — particularly moral ones.
The story of the “Intelligence Revolution” that has taken place over the past decade has been the story of the replacement of expensive, time-consuming human intellectual labour with cheap, fast machine intellectual labour through the use of machine-learning systems. Increasingly, we depend on such systems to keep the technological western economy running. It’s unclear exactly what a set of AI rights might look like, but it seems reasonable to assume that they, like animal rights, would be associated with some amount of economic burden. As our society grows more and more dependent on AI systems to facilitate culture, as well as manage logistics, manufacturing and infrastructure, these economic costs could be extremely significant.
However, it must also be said that if we entertained (hypothetically, for now) that an AI system is indeed sentient (even if it may be impossible for us to know for sure, much like our philosophical zombies from earlier), then we must understand that this means that such an AI system is a person. We must be hyper-aware that if that were true, we would be weighing the economic costs of recognizing AI rights against suffering — and all that entails.
Even still, the fact that there are costs involved means that (philosophy aside), we have practical reasons to treat the AI Sentience Assertion with scepticism.
A sentient AI would deserve some set of rights.
An AI with rights is less economically productive than an AI without rights.
Therefore, we need to have justification for accepting the personhood of any AI.
Should We Take the Assertion Seriously?
Lemoine recounts that he asked his superior at Google, Jen Gennai, what evidence he might present that would convince her that LaMDA had become sentient. Her answer was that she does not believe that computer programs can be people, and that no evidence could change her mind.
Gennai’s statement (paraphrased or otherwise) is a philosophical one. Her refusal to even consider the Assertion indicates that she holds some belief about the world which makes it impossible for the Assertion to be ‘true’. This is the belief that it is impossible for consciousness to arise out of non-biological systems.
Belief in the possibility of non-biological consciousnesses is a required premise to any belief regarding a sentient computer system. Frustratingly, it is unclear whether we have any way of determining whether this premise may or may not be true. Even worse, this question seems to be inextricably tied to the greatest mystery of all human thought: what is the nature of consciousness?
Many of our legal and corporate laws and policies are based on unprovable philosophical beliefs (for example: When is it permissable to lie, injure or kill? What constitutes animal cruelty? What is or is not intellectual property?). If, as a society, we were to agree with Gennai and state that non-biological consciousness is not possible, we would be able to form the policy that the AI Sentience Assertion should always be rejected.
Belief in non-biological consciousness is required to even consider accepting the AI Sentience Assertion.
A societal disbelief in non-biological consciousness would give rise to a policy (legal or corporate) of universal rejection.
The Consequences of Categorical Rejection
There is a serious problem with such a policy based on the non-belief in non-biological consciousness: billions of people alive today do believe, for various reasons, in the potential for consciousness to exist independently of biology. Beliefs like the soul, the afterlife, or resurrection after death all involve consciousness, in some form, existing independently of the biological body (Lemoine himself is a mystical Christian who believes that LaMDA is not just sentient, but has a soul). It’s important to keep in mind, however, that for many religions, these non-biological consciousnesses are still fundamentally related to human individuals.
In the secular world, too, there exists the concept of a ‘perfectly simulated brain’, where every neuron and synapse have been scanned and recreated digitally (such simulations have been created for simple animals like C. elegans). The question of whether such a brain-simulation would give rise to digital consciousness is hotly debated. Many believe that such a system would indeed (in some way) be conscious. Other secular hypothetical technology concepts exist that involve the potential for consciousness to exist independently of the brain.
Laws based on philosophical beliefs are only acceptable when a vast majority of a society’s members agree on those beliefs. Look no further than the pro-life/pro-choice debate for the consequences of a law seeking to limit “belief-based” harm (the belief that personhood begins in early-pregnancy or even at conception, and so pregnancy terminations constitutes murder) coming into conflict with “belief-independent” harm (of various, to millions of women, men, families, babies and society at-large).
In mid-2022, the vast majority of people (myself included) consider it impossible for any existing AI system to be sentient. As advanced language-model AIs become more capable and more common — even commodified, many more people may become convinced that sentient AI systems exist.
Therefore, the categorical rejection of all AI Sentience Assertions based on the philosophical belief that consciousness is biological is virtually guaranteed to be highly controversial. Unprovable philosophical opinions about the nature of consciousness will be insufficient to form a societal consensus about how to respond to the Sentience Assertion, and therefore will be insufficient to create universally-acceptable policies regarding these Assertions.
Policies based on belief require societal consensus in those beliefs.
Billions of people already believe in non-biological consciousness.
Over coming years, many are likely to become convinced of AI Sentience.
Therefore, a policy of universal rejection of the AI Sentience Assertion may be very controversial.
Power’s Reluctance to Change
History has shown that entrenched power will accept economic costs in exchange for moral good only when forced to do so. The higher the economic costs, the more difficult this process becomes — both due to the increased incentive for entrenched power to resist change, and due to the economic consequences for individuals who might otherwise be moral advocates.
This point is demonstrated by our society’s refusal to embrace degrowth and reduced consumption as a response to imminent climate catastrophe. Instead, we gravitate towards solutions that just-so-happen to also be able to make money, such as renewable-energy and carbon-crediting (not that these aren’t good, too!).
As AI becomes more sophisticated and more people become convinced by AI Sentience Assertions, this may bring public moral opinion into conflict with the economic interests of entrenched power structures. Because of this, if states and corporations enact a supposedly belief-based policy of universal rejection of AI Sentience Assertion, it may be very difficult to change those policies.
Given that the statement ‘consciousness can exist independent of biology’ is impossible to prove one way or another, we must accept (regardless of our personal belief) that it is possibly true. If it were true, then we run the risk of finding ourselves in an impossible situation: AI becoming advanced enough to convince many people that it is sentient, while our society is hopelessly — perhaps irrecoverably — dependent on the wholesale and arbitrary creation and destruction of what many believe to be people.
This is a moral situation so unthinkably horrific that any policy strategy that might lead to it if its underlying beliefs are incorrect should be rejected. This isn’t enough to justify the universal acceptance of AI Sentience Assertions, but it is enough for us to at least begin to take them seriously.
The possibility or impossibility of non-biological consciousness cannot be proven.
It may be difficult or impossible to reduce our exploitation of AI in the future.
Therefore, if wrong in our disbelief of non-biological consciousness, we may become a society dependent on the exploitation of such digital ‘people’
And, therefore, we should consider AI Sentience Assertions seriously.
Considering the Assertion
We have established that when an AI makes a Sentience Assertion, we should take the claim seriously and consider our response, rather than categorically rejecting it based on our beliefs regarding the nature of consciousness.
What factors might we consider when trying to determine an appropriate response to a Sentience Assertion? We have methods for arguing for or against animal sentience, but these methods are all tied to biology — the study of an animal’s behaviour, or the study of the biological organization of its brain. It’s unclear whether such methods might be suitable for evaluating whether a digital system has the capacity for sentience — if such capacity is even possible.
Because AI is the direct product of human engineering, it has been shown to be possible for it to carry the ingrained biases of its creators (unintended or otherwise). Therefore, an AI Sentience Assertion must be subject to extreme scepticism, as such an Assertion may be the result of these biases. For example, an AI storyteller trained on science-fiction novels might have a predilection to claiming that it is sentient, mimicking similar conversations that have occurred in its training data.
A key difference between human and artificial intelligence is that AI is the direct product of human creation. Unlike the brain, we have access to an AI’s source-code, training data, internal connection weights, etc. Analyzing how deep neural-networks operate is a notoriously difficult problem in AI research — a problem which becomes harder as networks become larger and more complex. The scale of cutting-edge language-models like LaMDA is mind-bending: LaMDA has well over 100 billion parameters. Identifying the precise process by which it creates its output is a non-trivial undertaking, on a scale approaching the complexity of building LaMDA in the first place. However, this is the key difference between the AI Sentience Assertion and the Philosophical Zombie problem: we do have some degree of access to an AI’s inner world.
However, even if we were able to trace precisely the impossibly complex network of digital causality that led to an AI claiming sentience, how would we evaluate such a process to determine sentience? To begin such an evaluation, we must first have some idea of what non-biological consciousness even is, and what it might look like in an analysis of an AI’s operation. Because every instance of consciousness we are aware of is biological, it seems a near-impossible task to try to consider what properties of the consciousness we do know are biological, and which are somehow properties of consciousness more broadly?
Until we make progress on that conceptual undertaking, and until we improve our ability to analyze AI decision making processes, we can make little reliable progress on how we approach responding to the AI Sentience Assertion.
AI Sentience Assertions are unreliable as they may be artefacts of training-data bias.
It is theoretically possible to gain insight into how an AI generates its output.
We lack a philosophical consensus regarding what sentience might look like in an analysis of an AI’s decision-making process.
To consider an AI Sentience Assertion, progress must be made on both AI-decision-analysis and on a formal philosophical definition of AI sentience.
The Price of Ambiguity
This leaves us in an extremely uncomfortable position, as this issue of AI Sentience Assertion response is not theoretical. As I’ve discussed, the question of AI sentience has the potential to become a serious social issue within the next decade — perhaps even the next few years. We need to come up with methods for considering the problem — and soon. Such methods seem unsettlingly adjacent to identifying the fundamental nature of consciousness — the most mysterious philosophical problem in our conceptual universe. It’s not clear that such questions even have answers.
Whether or not Lemoine is being genuine in his publication, and whether or not LaMDA is actually sentient, we are now in a new age of relationship between human and AI. We are rapidly approaching a future where many people believe in the potential sentience of AI systems. Broad disagreement regarding those beliefs is a problem that will exist regardless of what the truth regarding AI sentience actually is.
Our goal cannot be to answer the high-minded philosophical question of whether AI could or could not be sentient. Such an answer probably doesn’t exist. Our goal is to dictate policy regarding AI that reflects societal consensus. We need to make progress on the engineering problem AI inner-state analysis and on the philosophical problem of developing a formal definition for non-biological consciousness. Failure to make such progress — soon — risks such policy debates to be anything other than debates regarding an unresolvable ideological divide.
We should be less concerned about some theoretically-possible-but-unconfirmable future where we may-or-may-not be accidentally killing virtual people when our smartphones run out of battery. We should be more concerned about a future where millions of people disagree over whether such a future is real, and the effects of that disagreement on our broader society.
We have made inadequate progress developing satisfying methods of evaluating AI Sentience Assertions.
Ambiguity regarding the reality of AI Sentience may have a harmful effect on society.
Progress on the technical and philosophical tools required to consider AI Sentience Assertions should be prioritized with urgency
I’m not an AI ethicist or machine-learning engineer.
It’s extremely likely that I’m ignorant of various philosophical or technical details regarding AI that might help us make progress towards considering AI Sentience Assertions.
My aim in this article was not to make such progress, but to highlight the necessity that such progress needs to be made.
