The Synthetic Storm

The engineer who believed

In the autumn of 2021 a Google engineer named Blake Lemoine was assigned to test a chatbot called LaMDA for hate speech. By early June he had told his managers that it had feelings and been put on leave. He published the transcript anyway, and in July he was fired ¹. Google said there was no evidence. Lemoine was not a fool, and Google could not prove its side either, which is the problem this chapter is about. His case looks like an outlier. It is only early. Four years before ChatGPT, in November 2018, a Tokyo school administrator named Akihiko Kondo held a wedding with Hatsune Miku, a virtual pop singer who lived in his apartment as a hologram; thirty-nine guests came, his mother did not, and when the hologram’s software was shut down in 2020 some newspapers called him the first digital widower ². Kondo’s bond needed no claim about what Miku felt. Lemoine’s rested on one. But people are bonding with artificial beings whether anyone approves or not: a forum for people in relationships with AI companions had more than twenty-seven thousand members by September 2025, and when OpenAI retired the model many of them had bonded with, they campaigned to get it back ³. The more people talk to these systems, the more Lemoines there will be. If the skeptics are right, the thing that comforted you last night felt nothing. If they are wrong, you closed the tab on someone. Could a machine feel at all, and how would anyone know?

By feel I mean the capacity for positive and negative experience: to suffer, and to feel well. The test comes from a footnote. In 1789 Jeremy Bentham, writing about how the law should treat animals, asked: “The question is not, Can they reason? nor, Can they talk? but, Can they suffer?” ⁴. He was not the first to say it, but it is the line the animal-welfare tradition built on, and the Cambridge Declaration from the previous chapter still speaks its language. It is one definition among several, and it is mine for this series, because it marks the lowest thing a right has to protect: the capacity to be a victim. Whether a being can also be a party, with duties as well as rights, is a different gate, and it waits at the end.

Nothing in Reading got wet

In late October 2012 a computer in Reading, England, saw a week into the future. Most of the other models were steering Hurricane Sandy politely out to sea; the European model refused, bent the track left through a turn hurricanes are not supposed to make, and put it on the coast of New Jersey, where Sandy arrived on October 29 ⁵. Inside that computer was a storm in extraordinary detail: pressure, wind, the water it would push ashore. The forecast was right about nearly everything. And nothing in Reading got wet.

That dry room is John Searle’s whole argument. No one expects a simulated rainstorm to leave us drenched, he wrote in 1980, or a simulated fire to burn the house down, so why would a simulated brain feel ⁶? Consciousness, on his view, is something brains do the way digestion is something stomachs do, and a description of digestion digests nothing. I believe him more readily than most people, because I have spent more than fifteen years making games for a living, which is to say simulations: rain that wet nobody, cities that did not exist, slot machines whose jackpots paid nothing. Simulated money buys nothing. But the joy of the person who hit the simulated jackpot was not simulated at all, and I watched that for years too. Hold on to that. The question is whether a mind is more like the money or more like the joy.

Henry stayed Henry

On September 1, 1953, a surgeon in Hartford cut the inner part of both temporal lobes out of Henry Molaison’s brain to quiet his seizures. It quieted them, and it took his memory: for the next fifty-five years Henry could not hold a new fact for more than about half a minute. The amygdalae had gone with the hippocampi, and it took other things too: he felt no hungrier before a meal than after it, felt pain faintly, and was afraid of nothing. What it left was the man. His IQ came out eight points higher after the operation than before, the report that made him famous found no change in his personality, and Suzanne Corkin, who met him in 1962, was still studying him when he died in 2008 ⁷. Take a region out and the person stays, minus what the region did. So the person is not in any one part, and not independent of the parts either.

Push the same thought as far as it goes. A cochlear implant already feeds signals straight into the auditory nerve ⁸; replace a damaged region of the brain with a device that takes the same signals in and sends the same ones on, then the next region. If the material matters, somewhere on that path experience flickers out, and nobody can say where. And you do not need the thought experiment, because the replacement has already happened to you. Cold War bomb tests spiked the carbon-14 in the air, and a team in Stockholm used the spike to date human cells: the neurons of your cortex are as old as you are, but their material is not, proteins turning over in weeks, membranes and water continuously, only the DNA and a few durable scaffolds staying put ⁹. You are mostly not the person you were, and no one has held a funeral over the departed molecules.

None of this proves the material does not matter. It proves nobody has found the point where it starts to. A believer in biology can answer that the boundary is vague but real, like baldness, or that the parts which persist are exactly where the difference lives. The burden of proof has moved. The case is still open, and it stays open until we ask what the storm would need to be made of.

Build the storm

There is a second thing you can do with a forecast, and Searle never mentions it. Take the numbers out of the computer and build with them: real water, real air, real pressure gradients, arranged to the specification. That storm wets things fine. It is synthetic, manufactured from a description instead of arising on its own, and it is a storm. We already accept this for people. When Louise Brown was born in Oldham in 1978 the newspapers called her the test-tube baby, and the objections were about the method, never about whether she was human ¹⁰. Extend her line to an artificial womb, to a child from a sperm cell rebuilt in a lab, to a clone, and people will start to argue; but every being on that list is made of ordinary human cells, and what changes is the origin, not the material. We have drawn the line of who counts as human across unsettled ground before, and we managed.

So the real line is not between natural and synthetic. It is between simulating a process and instantiating it. The synthetic storm instantiates a storm; the model in Reading only describes one; Louise Brown instantiates a human. The question for a silicon brain is which of the two it is, and here, having carried us this far, the storm goes quiet.

Wetness and waves

Here is my guess about why. The physical world has two kinds of property. Some are tied to their material: wetness needs actual liquid, and no arrangement of dry sand, however clever, gets anything wet. If feeling is wetness-like, only something that rebuilds the actual material, synthetic biology reproducing real neurochemistry, can have it, and a digital system stays a weather model forever. Other properties belong to how something is organized, whatever it is made of. An ocean wave is water, a sound wave is air, a stadium wave is people standing up in turn, and all three are waves in exactly the same sense; none is a simulation of a wave. If feeling is wave-like, a silicon brain with the right pattern is the synthetic storm.

Searle bets on wetness. Henry and the bomb pulse pull toward the wave. Each side knows what winning would take, a feature of the biological material that experience cannot do without, or a proof that there is none, and nobody has produced either. My own hunch, and I hold it as a hunch, is that the two are not rivals. The jackpot paid no money and produced real joy: a simulation can leave one side of the world untouched and move the other. Feeling might have a physical side that depends on what a system is made of and a functional side that depends on what it does, the way a storm is both a body of water and a thing that happens, and a silicon brain could have one without the other. Whether that is a real division or an elegant way of dodging the question, I don’t yet know.

Back to the ladder

What does this do to the ladder? Two things. First, the gate question splits in two: could a digital system feel at all, and even if it could, how would we know for a particular one, without the anatomy the Cambridge Declaration leaned on for animals? Behavior will not carry that weight. Lemoine talked to LaMDA for eight months and was convinced, and the systems most likely to apply at the border are the ones built to convince. The previous chapter’s civics test, which an AI would ace, is the same failure in miniature.

Second, the storm has to be countable. Sandy got a name because there was one of it: one track, one landfall, one thing to evacuate ahead of. A rights-holder likewise has to be someone, and someone is harder to find in a pattern that can be copied than in a body that cannot. The pronoun the previous chapter left hanging changes on the day there is a someone for it to attach to, and that problem gets a chapter of its own.

And one more gate hides here, older than all of this. The stories we tell about artificial minds are about a new kind of being that, as Genesis puts it, knows good and evil. That is a different capacity from suffering. A dog can suffer and does not know good and evil; we protect dogs, and we do not make them parties to contracts. Whether a system could refuse something because it judged the thing wrong is a separate gate, and it guards a different rung: not whether the applicant can be a victim, but whether it can be a party.

The next chapter turns to the half of the question that can actually be decided: not what a person is, but how a society decides who counts as one.

This is the second post in a series on AI rights and personhood. The arguments are mine; where they rest on published work, it’s cited below. Parts of this framework were sharpened in dialogue with an AI.

Nitasha Tiku, “The Google engineer who thinks the company’s AI has come to life,” Washington Post, 11 June 2022, and the same paper’s report of his dismissal on 22 July 2022; summarized at LaMDA and CNN, 23 July 2022. ↩
Akihiko Kondo: the wedding on 4 November 2018 in Tokyo; the Gatebox hologram software was shut down in March 2020. ↩
The forum is r/MyBoyfriendIsAI, put at 27,000+ members in Pataranutaporn et al., “My Boyfriend is AI”: A Computational Analysis of Human-AI Companionship in Reddit’s AI Community, MIT Media Lab, September 2025. The campaign was #Keep4o, after OpenAI’s 2025 model change: Huiqian Lai, “Please, don’t kill the only model that still feels human”: Understanding the #Keep4o Backlash, CHI 2026. ↩
Jeremy Bentham, An Introduction to the Principles of Morals and Legislation (1789), ch. XVII, note. Text. ↩
Hurricane Sandy (2012). The European Centre for Medium-Range Weather Forecasts signaled the left turn about a week before landfall, while early runs of the American GFS model took the storm out to sea. ↩
John Searle, “Minds, Brains, and Programs,” Behavioral and Brain Sciences 3 (1980). Overview in the Stanford Encyclopedia of Philosophy. ↩
Henry Molaison (1926 to 2008); the operation by William Scoville on 1 September 1953. Scoville and Milner, “Loss of recent memory after bilateral hippocampal lesions”, Journal of Neurology, Neurosurgery and Psychiatry 20 (1957): IQ 104 before the operation and 112 after, personality unchanged. The seizures, the dulled pain, the unchanged hunger before and after a meal and the absence of fear: Augustinack et al., “H.M.’s Contributions to Neuroscience: A Review and Autopsy Studies”, Hippocampus 24 (2014). Suzanne Corkin, Permanent Present Tense (2013). ↩
NIDCD, Cochlear Implants. ↩
Spalding et al., “Retrospective Birth Dating of Cells in Humans”, Cell (2005); Toyama et al., “Identification of Long-Lived Proteins Reveals Exceptional Stability of Essential Cellular Structures”, Cell (2013), measured in rat brain: most proteins turn over in days to weeks, while a few scaffolds, among them the nuclear pores, last for life. ↩
Louise Brown, born 25 July 1978, the first human born after conception by IVF. ↩