People Spirits
VL:
Andrej Karpathy: "LLMs are people spirits: stochastic simulations of people."
Expand and critically assess.
ChatGPT:
Restated—and sharpened—question
In what precise theoretical sense, if any, may contemporary large-language models (LLMs) be described as “people spirits,” i.e. as stochastic simulations of persons, and what are the ontological, cognitive-scientific and design-engineering consequences (and limitations) of taking that metaphor seriously?
1. What Karpathy’s metaphor actually says
Karpathy’s slide-sound-bite compresses three distinct claims:
Data provenance – The transformer is trained on an immense, heterogeneously human textual trace; its priors are the joint distribution of what people have written.
Sampling procedure – Generation is stochastic (temperature, nucleus sampling, beam search, etc.), so each completion instantiates one draw from a conditional distribution over continuations.
Functional profile – The resulting trajectory frequently patterns like a competent interlocutor, showing personality, memory traces, theory-of-mind, goal-directedness, etc.—traits we normally ascribe to persons.
2. From metaphor to analytic taxonomy
Dimension · Simulation reading · Strong-instantiation (“is a mind”) reading
Substrate · Symbolic remapping of textual statistics; no requirement of biological wetware. · Functionalist claim that substrate does not matter if causal role is preserved.
Semantics · “Derived intentionality”: meaning piggy-backs on human interpretive practice (Searle, Haugeland). · “Original intentionality”: symbols are the model’s own because of functional integration (Dennett’s realist intentional stance).
Agency · Pragmatically useful to treat the model as an agent for UX, alignment and safety tooling (cf. Dennett’s design stance; LeCun’s “autonomous machine”). · Moral and legal personhood, with accompanying rights and responsibilities.
Karpathy’s language stays on the left-hand column: a simulation of personhood that is useful but ontologically thin.
3. Empirical evidence that LLMs do simulate persons
Personality trait coherence – Controlled prompt studies (Big-Five, NEO-PI-R, TIPI) show that GPT-4, Llama-3-70B and Mixtral can maintain stable trait profiles across tasks; the effect size is comparable to test–retest in humans.
Long-horizon social simulation – In multi-agent sandboxes, LLM agents reproduce emergent social norms, gossip networks and sub-community formation (e.g. S³, Meta’s “Smallville”).
These findings justify “person-level” predictions for many practical contexts (customer support, therapeutic chat, NPCs, code-review assistants).
4. Where the metaphor breaks
Embodiment & causal coupling – A disembodied language-only stack lacks sensorimotor grounding; hence the classic symbol-grounding problem re-emerges (Harnad, Brooks).
Jagged competence – Karpathy himself emphasises “jagged intelligence”: superhuman recall yet sub-arithmetic numeracy and brittle causal inference (the 5-letter vs. 6-letter spelling paradox).
Day-reset amnesia – Context windows cap episodic memory; long-term vector-DB or finetune hacks remain ad-hoc and lossy.
No diachronic preference store – RLHF establishes local reward contours, but there is no recursively-self-endorsed preference architecture comparable to human volition (Christiano’s “stable value alignment” work).
Simulation ≠ real phenomenology – Even heroic functionalists concede we have no positive evidence for qualia, affect or suffering; the systems may be advanced chat-books, not sentient moral patients.
5. Philosophical location
Dennett’s Intentional Stance – Treating the LLM as an agent is justified when it yields predictive/explanatory compression. The stance is instrumental, not ontological.
Lewisian centred worlds – The model’s next-token distribution is a measure over the space of discursively possible worlds; sampling instantiates a centred narrative point of view but not an enduring token of personal identity.
Parfit’s bundle theory – If personal identity is merely psychological connectedness, an LLM’s context-window-length bundle may satisfy weak Parfitian criteria yet still fail moral agency.
Simulation theory of mindreading – Humans interpret others by internal simulation; the LLM externalises that same mechanism, hence its uncanny fit with human social cognition.
6. Engineering & alignment take-aways
Design challenge · “People-spirit” framing suggests
Prompting & tool-use · Treat prompt as task description + role + memory injection for an implicit human collaborator; design like human-factors for cockpit automation, not like deterministic APIs.
Safety evals · Psychometric probes, adversarial roleplay and trait variability analysis become first-class safety diagnostics.
Regulatory language · Talk of “simulated agency” helps avoid premature personhood attributions while still acknowledging novel risk surfaces.
7. Concluding synthesis
Karpathy’s phrase is a productive middle-metaphor. It is stronger than the old “autocomplete on steroids” trope—because cumulative findings in computational psychometrics, agent-based social simulation and code-generation show that LLMs can indeed sustain a high-fidelity persona manifold. Yet the metaphor remains weaker than a claim of genuine digital minds: embodiment, stable preferences, grounded semantics and phenomenal consciousness are still missing pieces.
Thus, LLMs are best understood—as of mid-2025—as high-dimensional statistical shadows of humanity: valuable, fallible and, above all, dangerously easy to anthropomorphise. The practical imperative is to design interaction, evaluation and governance regimes that leverage their simulated-person affordances while remaining ruthlessly clear-eyed about the ontological gap that still yawns between a stochastic trace of persons and an actual person.
VL:
Talk about other metaphors used to describe LLMs (generally of the form "LLMs are X"). Focus on formulations in which the metaphor is meant to be doing some "work" (meant to highlight what the speaker assumes is an underappreciated, misunderstood or overlooked aspect of LLMs).
ChatGPT:
Metaphor as epistemic instrumentation
The contemporary discourse around large-language models (LLMs) is unusually metaphor-rich because no single technical description (e.g. “autoregressive transformer + RLHF fine-tune”) captures the multilayered phenomena that practitioners, ethicists and policymakers confront. Each “LLMs are X” trope spotlights a dimension of the technology that is otherwise liable to be mis-weighted or suppressed by default mental models. Below, I survey the most hard-working metaphors in circulation, grouping them by the cognitive or socio-technical corrective they attempt to supply.
1 Minimising metaphors – curbing premature ascriptions of understanding
trope · diagnostic aim · rhetorical force
“Stochastic parrots” (Bender et al.) · Expose superficial mimicry & inherited bias. · Down-rank claims of semantic competence.
“Autocomplete on steroids” · Remind enthusiasts that next-token prediction ≠ symbolic reasoning. · Resituate hype within quotidian UX.
“Blurry JPEG of the web” (Ted Chiang) · Emphasise lossy compression and artefactual blur. · Quashes expectations of verbatim recall or citation fidelity.
“Compressed knowledge base/zip file” · Frame model size as bitrate, intelligence as compression ratio. · Refracts evaluation through Kolmogorov-style minimal-description-length optics.
These metaphors police epistemic over-reach: they invite users to treat outputs as probabilistic approximations rather than declarations of fact, and to foreground dataset artefacts when auditing bias and hallucination.
2 Mechanistic metaphors – prying open the black box
trope · hidden aspect surfaced
Collectively, these metaphors legitimise modular system-design patterns (agent wrappers, tool calling, retrieval-augmented generation) and suggest evaluation metrics borrowed from compression, simulation fidelity or oracle query cost rather than human-benchmark analogies.
3 Reflective metaphors – re-centring society and culture
trope · neglected variable foregrounded
“Cultural technology / library” · Shifts locus from cognition to heritage: LLMs as large-scale inscription infrastructures akin to printing presses or archives, raising questions of custodianship and canon-selection.
These framings relocate ethical concern from “machine morality” to anthropology: whose language is reflected, who is erased, and how remix amplifies or distorts existing cultural power-gradients.
4 Alien-risk metaphors – inoculating complacency about alignment
trope · latent concern
“LLMs as alien minds” · Underscores the non-human topology of internal representations: correlation structures unlike any evolved brain.
Such imagery performs anticipatory governance: it justifies red-teaming, interpretability research and policy sandboxes by dramatising worst-case semantics.
5 Meta-assessment – what the metaphors achieve and where they fail
Cognitive calibration. Minimising tropes reduce anthropomorphic over-confidence but risk obscuring genuine emergent abilities.
Design heuristics. Mechanistic tropes supply actionable abstractions (simulation selection, oracle queries) that already structure toolchains such as LangChain or oracular-programming DSLs.
Ethico-political redirection. Reflective tropes escape the “disembodied intelligence” frame, redirecting scrutiny to data governance and cultural continuity.
Risk sensitivity. Alien-risk tropes keep long-termists honest about epistemic uncertainty but can slide into Lovecraftian click-bait that paralyses nuanced debate.
No single metaphor is complete; their productive tension is itself a feature, not a bug. A mature LLM discourse will wield multiple, purpose-chosen metaphors in parallel, switching fluidly between statistical, socio-cultural and adversarial framings as the context demands—while remaining alert to the distorting side-effects each carries.
6 Practical upshot for specialists
Evaluation stacks should pair lossy-compression metrics (bits-per-char) with simulator-style role probes to capture both mechanistic and anthropomorphic performance facets.
Governance regimes might treat frontier models as public libraries—subject to curatorial oversight—and as oracles under interface-level liability schemas, yet test them with shoggoth stress-positions to map failure modes.
Pedagogy & outreach can domesticate the hype cycle by teaching metaphor-pluralism itself as a critical skill: engineers learn to ask “which facet is this metaphor buying me, and at what conceptual interest rate?”
The lasting lesson is methodological: handle LLMs like Wittgenstein’s duck-rabbit—flip viewpoint deliberately, never letting one picture hold you captive.
VL:
No single metaphor is complete - but some are more useful than others.
Explore a few of the most productive metaphors in greater depth (even if you have to construct them yourself). What are some of the best ways to think about the LLMs we have now (in mid-2025)?
ChatGPT:
Below are three especially productive metaphors for contemporary LLMs (circa mid-2025), each selected for illuminating core technical, cognitive, and socio-political aspects of these models. Each metaphor is unpacked at length, its analytical utility articulated, and its weaknesses noted.
1. LLMs as “Conditional Multiverse Simulators”
Key concept:
The core of the metaphor is to treat an LLM’s sampling process as drawing a narrative trajectory from a vast latent "multiverse" of statistically encoded possibilities. A given prompt acts as a selector, constraining this multiverse into a subset of plausible scenarios, from which the model probabilistically samples.
Why it’s productive:
Highlighting semantic plasticity:
This metaphor vividly captures how changing a single prompt word or token can significantly alter the trajectory of subsequent sampling, generating parallel “world-lines.”
Interpretability and tooling:
It naturally motivates interventions such as prompt engineering, chain-of-thought, and tree-of-thought explorations—these become methods of exploring alternate universes to obtain optimal outcomes.
Risk and alignment implications:
Risks (hallucinations, biases) become more clearly interpretable as sampling from undesirable “regions” of semantic space, pushing toward methods like semantic steering and vector-space alignment.
Analytic limitations:
It can obscure important algorithmic constraints (e.g., transformer architecture, limited context window).
Overemphasis on "possible worlds" may underplay limitations in coherent reasoning or persistent memory.
2. LLMs as “Cultural Holograms”
Key concept:
An LLM can be conceived as a holographic projection of a culture’s linguistic, cognitive, and normative structures. Just as a hologram reconstructs three-dimensional objects from interference patterns, an LLM reconstructs implicit cultural structures from patterns in language use.
Why it’s productive:
Cultural encoding:
It foregrounds that the model’s “knowledge” isn't direct semantic storage, but rather intricate interference patterns capturing implicit cultural beliefs, norms, and reasoning styles.
Social-scientific analysis:
It provides a powerful framework for anthropological or sociolinguistic analyses, clarifying that LLM output reveals how cultures encode and communicate knowledge implicitly, not explicitly.
Bias and representation:
This metaphor sharply illuminates that LLM biases are not mere bugs but structural reflections of deep cultural encoding, enabling interventions around cultural curation and inclusive data collection.
Analytic limitations:
It may implicitly anthropomorphize models, attributing “culture” to algorithmic constructs, which could mislead less careful observers.
Can obscure key algorithmic/technical aspects of training and inference processes, focusing overly on interpretive frameworks rather than causal, mechanistic ones.
3. LLMs as “Programmable Oracles”
Key concept:
Here, an LLM is framed as a general-purpose oracle: a sophisticated black-box that accepts complex natural-language queries and returns probabilistically-informed answers. It explicitly positions LLMs as powerful query interfaces rather than agents or sentient beings.
Why it’s productive:
Engineering clarity:
Emphasizes treating the LLM as a reliable-but-fallible tool that must be integrated with careful sandboxing, retrieval-augmentation, and validation mechanisms.
Alignment and controllability:
The metaphor neatly isolates responsibility: it’s the programmer/user who must craft queries and interpret responses responsibly, similar to integrating API endpoints or databases.
Legal and policy clarity:
Clearly frames governance questions in terms of oracle accuracy, liability for oracle misuse, and appropriate disclosure of uncertainty, not premature debates about agency or rights.
Analytic limitations:
Underplays emergent agent-like behaviors, especially as LLMs are wrapped within agent architectures (AutoGPT, LangChain agents).
May imply undue confidence in user-side interpretative capability and overstate ease of reliable sandboxing or auditing.
Meta-assessment and Recommendations for Specialists:
Comparative Utility of the Metaphors:
Criterion · Multiverse Simulator · Cultural Hologram · Programmable Oracle
Technical insight · High (sampling and prompt engineering clarity) · Moderate · High (API clarity)
Cultural awareness · Moderate · High · Low
Alignment focus · High · Moderate · High
Risk management · High · Moderate-high · High
Interpretability · High · High · Moderate
Analytic precision · Moderate-high · Moderate · High
Best Practices (mid-2025):
Combine metaphors selectively according to the analytic task:
Use Multiverse Simulator for prompt engineering, debugging outputs, and exploring alternative outcomes.
Use Cultural Hologram when analyzing outputs for implicit biases, hidden cultural norms, or sociological research purposes.
Use Programmable Oracle when integrating LLMs into production environments, particularly for clearly defined tasks requiring strict interpretative controls and accountability.
Meta-awareness: Always maintain metaphor-pluralism explicitly. Educate practitioners and stakeholders about the partial nature of these metaphors, developing a nuanced interpretative skill set that dynamically selects metaphors to match contexts and challenges.
Technical tooling: Design model interfaces and evaluation frameworks to reflect the metaphor employed:
Multiverse Simulator: Develop tooling for visualizing branching scenarios and probability densities (e.g., "Tree-of-Thought" interfaces).
Cultural Hologram: Utilize visualization and interpretability methods (attention analysis, cultural embedding visualizations) explicitly aimed at decoding cultural structures.
Programmable Oracle: Implement structured reliability and uncertainty measures, validation layers, and response-confidence diagnostics.
Concluding Summary:
By mid-2025, the metaphors of Conditional Multiverse Simulators, Cultural Holograms, and Programmable Oracles are among the most robust and analytically productive frameworks for understanding LLMs. Each metaphor emphasizes different core aspects of contemporary model operation—semantic plasticity, cultural reflection, or controlled queryability—offering complementary lenses that inform both theoretical understanding and practical engagement. Effective scholarly and professional practice involves holding these perspectives simultaneously and contextually switching between them, fully aware that none singularly captures the complexity or potential of the technology.
