In 2022, Google engineer Blake Lemoine announced that LaMDA, a language model he had been testing, was sentient: conscious, self-aware, able to think and feel. He released transcripts of their conversations to make his case. The public was skeptical, and Google let him go. COSM audiences heard him tell his side of the story at COSM 2022.
The chatbots are far more fluent now, and Lemoine’s question has not gone away. In The Emergent Mind, Gaurav Suri and his co-author, Jay McClelland, give it a careful hearing, and their answer is more interesting than either “it’s alive” or “it’s just autocomplete.” At COSM 2026, Suri and philosopher Jay Richards will take up the question together.
Same Recipe, Different Kitchen
Suri’s central claim is that the human mind emerges from the interaction of neurons, none of which can think on its own. Large language models (LLMs) such as ChatGPT, he points out, rest on the same principles: simple processing units, connections whose strengths change with learning, and knowledge spread across many units at once. Human thought and an LLM’s abilities are each, in his phrase, “an emergent consequence of a distributed, interactive activation process.” One network is biological, the other artificial.
So the thinking in these models, Suri writes, is “similar to ours in some ways but not in others.” A model trained only to predict the next word turns out to do arithmetic, translate, reason, and even make humanlike mistakes. Yet it learns from roughly a hundred thousand times more text than a person reads in a lifetime. It learns from words alone, not from sight, touch, or the results of its own actions. And it learns passively, while we choose what to learn from.
One might add that a system trained on the written record of human thinking will naturally produce text that sounds like thinking. The open question is whether resemblance in output means resemblance in kind.
A Memory That Forgets?
Suri points to a revealing limitation. Tell a chatbot, mid-conversation, that “Jane’s sister Mary has a new baby named Allen,” and it can tell you who Allen’s aunt is. But that knowledge lives only in its context window, the running text of the current conversation. Close the window, Suri notes, and “no record of the sentence” remains. A person told the same thing once will usually remember it.
Newer systems get around this by saving past conversations to an external store and pulling them back in when needed. Suri admits that whether such a system then answers “by reasoning or by retrieving the answer” is unclear.
That raises a puzzle philosophers have argued over since Andy Clark and David Chalmers’s 1998 paper “The Extended Mind.” They imagined Otto, a man with memory loss who writes everything down in a notebook he carries everywhere. When Otto checks the notebook for a museum’s address, is the notebook part of his mind? If so, perhaps an AI’s database is part of its mind. If not, where does a mind end and its tools begin? And for a system spread across thousands of processors, what exactly is the “it” that thinks?
Whose Goals?
The book’s most striking concession concerns goals. Today’s AI systems, Suri and McClelland write, “generally lack goal systems of their own.” The goals that organize their behavior belong “largely” to their human designers, who choose the training data and the rewards used to fine-tune them.
Humans, by contrast, have brain systems that let goals shape everything we do, from planning to reasoning, sometimes across a decade or more. Suri thinks such systems could someday be built into AI, cautiously and with guardrails, as a path to more autonomous machines.
Here Richards’s work comes in. In The Human Advantage, he argues that machines, however capable, remain tools that serve human purposes. Suri’s book offers a detail worth pondering on that score. Asked whether it could pass the Turing test, the famous challenge to tell a machine from a person in conversation, ChatGPT demurred. Suri calls that answer “almost surely the result of careful tuning.” If designers can shape what a system says about itself, what can its self-reports tell us? Turing argued that we accept other people’s reports of their own feelings. Lemoine accepted LaMDA’s. Should we?
What to Listen For
- Is resemblance enough? If an LLM arrives at humanlike abilities by humanlike means, does it have a mind, or only the outputs of one?
- Where is the mind? In the trained connections, the context window, the external database, the data center?
- Can borrowed goals become one’s own? And if a machine’s purposes come from its designers, what does that suggest about where ours come from?
- Is anyone home? Suri’s book leaves consciousness as an open question. How would he answer Lemoine today, and how would Richards?
Join us at COSM in Scottsdale, AZ, November 18–20; our panel is on Friday, November 20, at 10 am. Next in this series: why are there minds at all?