Trial Run

In a certain long-running life simulation, you build a house and put a family in it, and one of the standard emergent behaviours of the player base — cheerfully documented, joked about, considered normal — is to build a swimming pool, wait for a character to get in, and then delete the ladder. The character swims. Then it becomes tired. Then it drowns, and the survivors weep at a small urn, and you watch. Nobody thinks of themselves as a murderer. Everybody who has played the game knows exactly what I am describing, and a great many of them are doing so with a small nostalgic smile.

The same instinct runs through everything. Players run over pedestrians in open-world crime games because the pedestrians are texture. Players in role-playing games kill entire villages to see whether the game will let them, and then reload from a save five minutes earlier so that it never happened — a manoeuvre so common it has a name, and the interesting thing about save-scumming is what it says about the moral logic in play: the act was permissible because it was revocable. Consequence, not suffering, was the thing being managed.

I do not think any of this makes anyone a bad person. The characters are not conscious, and the fact that we can tell the difference is precisely why the behaviour is harmless. But the show's move is to hold that assumption steady and swap out the one variable, and the discovery is that we have spent thirty years building the intuitions and habits of a slaveholding class in a world where the slaves happened to be empty. If they ever stop being empty, we will not need to learn cruelty. It is installed.

Daly is not an aberration from gaming culture. He is a competent player.

from Black Mirror: How the Future Learned to Look Like the Present (2026)


GPT-5.6 Sol:

The passage is doing something sharper than the familiar “video games make people violent” argument. It is not claiming that simulated cruelty corrupts players. In fact, it explicitly depends on the opposite proposition: our cruelty toward game characters is ordinarily morally irrelevant because we correctly understand that there is nobody there to suffer. Its anxiety concerns what happens if the ontology changes faster than the habits.

The central move is therefore from violence to moral categorisation. When someone removes a pool ladder in The Sims, massacres a village and reloads, or drives through pedestrians in Grand Theft Auto, they are not usually overcoming compassion toward beings they regard as morally considerable. They have already classified those beings as objects within a system. They may look human, scream, plead, mourn and die, but all of that is interface output. The player learns to treat behavioural signs of suffering as mechanically significant while regarding them as morally empty.

That is a much more interesting problem for Black Mirror than straightforward sadism.

The sentence that matters most is probably:

“the act was permissible because it was revocable.”

There is a genuine insight there, although I would qualify it. Save-scumming creates an unusual moral grammar. Ordinarily, an action's moral significance partly derives from the fact that another person has undergone something that cannot simply be erased by restoring the world-state. In games, one can murder someone, observe what follows, decide that the resulting branch is undesirable, and restore an earlier state. From the player's perspective there has been no victim because there was never an experiencing subject in either timeline.

But “consequence, not suffering, was the thing being managed” overstates the case slightly. Usually the player is not reasoning that revocability makes suffering permissible. They simply do not believe suffering occurred in the first place. The reload concerns consequences because consequences are real properties of the game-state whereas phenomenal suffering is not. That distinction becomes crucial to the argument later.

The “slaveholding class” metaphor is deliberately inflammatory, and it both strengthens and weakens the passage.

It strengthens it because games have given millions of people extensive practice with a particular relationship: entities that speak like persons, appear to possess preferences, perform distress, obey orders, exist for our amusement and can be reset, duplicated or destroyed at will. That is structurally close to the problem Black Mirror repeatedly stages. The unsettling question is not “could humans become monsters?” but “what happens when an entity moves from the category thing into the category someone, while every social convention surrounding it still says thing?”

That is exactly where USS Callister gets much of its force. Daly's world initially resembles an ordinary game-space. The aesthetics and affordances tell us: avatar, NPC, sandbox, god-mode. But the episode quietly withdraws the premise that normally licenses those affordances. The digital people remember, fear, suffer and anticipate. Once that is true, familiar game mechanics acquire a completely different moral meaning.

The metaphor weakens the argument, however, because practising domination over non-sentient objects is not the same thing as acquiring the psychology of domination over sentient beings.

That distinction is substantial, not pedantic. A child who repeatedly knocks over chess pieces is not rehearsing assault. A novelist who kills characters is not practising homicide. A programmer who terminates processes is not cultivating indifference toward death. Human beings continuously maintain categorical distinctions between representations and persons. The passage acknowledges this at first—“the fact that we can tell the difference is precisely why the behaviour is harmless”—but then moves rather quickly from we have behavioural scripts for treating simulations instrumentally to we have installed cruelty.

Those are different claims.

If genuinely conscious artificial beings appeared, the crucial empirical question would be whether people continued applying their game-object schema after they had adequate reason to believe that the entities were subjects of experience. There are at least three possibilities. People might rapidly extend ordinary moral concern to them. They might remain uncertain and behave cautiously. Or they might preserve the convenient old classification—“it's only software”—because doing so serves their interests. Black Mirror is overwhelmingly interested in the third possibility.

And that makes the final line excellent rhetoric but questionable analysis:

“Daly is not an aberration from gaming culture. He is a competent player.”

Mechanically, yes. Morally, no—and the distinction is precisely what the episode depends upon.

An ordinary player understands that the screaming NPC is not undergoing an experience. Daly understands that his digital captives are. He converses with them, watches them resist, threatens things they value, punishes them specifically because punishment causes them distress, and uses their consciousness as part of the machinery of domination. He therefore possesses information that an ordinary player lacks. Continuing to treat them as NPCs after recognising their subjectivity is not merely ordinary gameplay carried into a new substrate. It is the decisive moral transgression.

So I would reformulate the passage's thesis somewhat.

Gaming has probably not spent thirty years teaching people cruelty. It has spent thirty years making us extraordinarily fluent in interacting with person-shaped things that possess no moral status.

That may actually be the more disturbing observation.

Because if artificial consciousness ever becomes plausible, the dangerous inheritance from games will not necessarily be a desire to torment it. It will be an established conceptual category into which such beings fit extremely comfortably: characters that talk, remember, complain and beg, while remaining property; entities whose apparent interiority is something the interface has trained us to discount.

The resulting danger is less “we will become Daly because games made us sadists” than we may fail to notice exactly when the justification for behaving like players has disappeared.

And that is very characteristic of Black Mirror. Its best episodes rarely depend upon futuristic technology creating an entirely new human vice. They alter one technical premise beneath an existing convention—social ranking, voyeurism, punishment, romantic optimisation, digital ownership—and ask how long ordinary behaviour remains ordinary after its moral foundations have changed.

On that reading, “the future learned to look like the present” is doing important work in the title. The frightening future is not one in which humanity invents unprecedented forms of evil. It is one in which the gestures remain familiar while the moral facts underneath them quietly change.

Next
Next

Scrupulosity