The AI Effect

the pattern repeats, in each generation, with the same shape. Something is identified as the mark of real intelligence, or real understanding, or real mind. A machine is built which does it. The criterion is quietly revised, and the achievement is reclassified as mere mechanism.

Chess was going to be the test; Deep Blue won in 1997, and chess was thereafter revealed to have been a matter of brute search all along. Go was going to be the test, because it required intuition; that fell too. Natural, flexible conversation was so obviously the test that Turing built his whole proposal around it, and the interesting thing about the present moment is that conversation has now substantially fallen, and rather than conceding the point we have done what we always do, which is to decide that conversation was never the criterion after all and that the real criterion is something else — grounding, or understanding, or embodiment, or consciousness.

A computer scientist named Larry Tesler is supposed to have summarised this as: intelligence is whatever machines haven't done yet. The line gets attributed in slightly different forms, and it has become known as the AI effect.

I want to make two opposite points about this, because the pattern is usually deployed as a cudgel by one side only.

The first is that the goalpost-moving is real, it is embarrassing, and we are doing it right now. Anyone who confidently asserts that the current systems obviously lack real understanding should be aware that they are the latest in a long and consistently mistaken line, and that their predecessors were just as sure.

The second is that the revisions have often been correct. Deep Blue really did not understand chess in the way a grandmaster does; the discovery that the game could be won by search really was a discovery about chess rather than about intelligence. The criteria keep failing not only because we are bad losers but because they were never good criteria — because we had been using performance as a proxy for an underlying property, and each machine demonstrated that the proxy could be achieved without the property. That is genuine information, even if it arrives in the form of a moved goalpost.

Both things are true. We are unreliable judges with a history of moving the line, and the line has often deserved to move.

from Are LLMs Conscious?: Language, Experience and the Problem of Artificial Minds (2026)