Do Language Models Understand?

A language model can explain photosynthesis.

It can translate a metaphor.

It can debug code.

Does it understand any of this?

The question sounds simple.

It is not.

The answer depends first on what we mean by:

understanding.

Understanding as Successful Use

One functional definition says:

a system understands a concept if it can use it appropriately across contexts.

Under this criterion, understanding is demonstrated through competence.

Understanding as Internal Model

A stronger definition requires an internal representation capturing important structure of the domain.

The system must not only produce correct outputs.

It must model relations.

Understanding as Grounding

Another view requires connection between symbols and:

• perception,
• action,
• world.

Words alone may be insufficient.

This is the symbol-grounding perspective.

Understanding as Conscious Grasp

A still stronger definition requires subjective awareness.

The system must experience:

what the concept means.

Under this definition, understanding depends on consciousness.

One Word, Several Questions

When people argue about LLM understanding, they often use different criteria.

One person asks:

Can it use the concept?

Another asks:

Does it feel meaning?

They may disagree verbally while answering different questions.

Behavioral Evidence

Language models can show surprisingly broad competence.

They can:

• paraphrase,
• infer,
• compare,
• explain.

This is evidence for some functional form of understanding.

But Behavior Can Be Misleading

A system may produce correct language through:

• memorization,
• pattern matching.

Therefore isolated success is weak evidence.

Generalization matters.

Novel Combinations

If a model handles genuinely novel combinations, pure lookup becomes less plausible.

This suggests learned abstraction.

But abstraction is not yet full semantic grounding.

Internal Representation

Neural networks build distributed internal features.

Researchers can sometimes identify representations related to:

• syntax,
• entities,
• concepts.

This suggests the model is not merely storing surface strings.

Geometry of Meaning

Words and concepts occupy structured regions in representation space.

Relations can be encoded geometrically.

This is a genuine internal organization.

Is Representation Enough?

A dictionary also contains semantic relations.

Does the dictionary understand?

Most people say no.

So representation alone may be insufficient.

The Chinese Room

Searle’s Chinese Room remains central.

A person follows rules manipulating Chinese symbols without understanding Chinese.

The room produces correct answers.

Searle concludes:

syntax is not sufficient for semantics.

Systems Reply

The person does not understand.

But perhaps the whole system does.

This shifts the level of analysis.

A single neuron does not understand English either.

The brain as a whole might.

Robot Reply

Attach the system to:

• cameras,
• motors,
• real-world interaction.

Now symbols connect to objects and action.

Would that produce understanding?

For grounding theorists, this is stronger evidence.

Virtual Grounding

Must grounding be physical?

A system in a rich simulated world can:

• perceive,
• act,
• learn consequences.

Perhaps causal interaction is what matters, not biological material.

Text Is Not Disconnected from Reality

Text is produced by humans who live in the world.

Descriptions encode traces of:

• perception,
• action,
• culture.

A text-only model therefore receives indirect grounding through human language.

Indirect Grounding

The question becomes:

Is second-hand grounding enough?

A blind person can understand visual concepts through testimony.

Direct sensory access may not be necessary for every concept.

Social Semantics

Meaning is partly social.

Words acquire meaning through communal use.

A model trained on massive linguistic interaction learns this usage structure.

This supports a Wittgenstein-like functional argument.

Meaning Is Use

If meaning depends on how expressions function in language games, then sophisticated linguistic competence is highly relevant.

But ordinary human use is embedded in life and action.

Language games are not only text statistics.

Distributional Semantics

Words appearing in similar contexts tend to have related meanings.

Language models exploit this principle at scale.

Distribution captures remarkable semantic structure.

Distributional Limits

Some distinctions may be difficult without world interaction.

For example:

which objects are physically fragile?

Text can describe this.

But direct sensorimotor experience may offer richer grounding.

Referential Meaning

A symbol can refer to a specific external object.

Does a language model’s token:

Paris

refer to the actual city?

Or only to a network of linguistic associations?

Reference remains philosophically difficult even for humans.

Causal Theories of Reference

Some theories link reference to causal chains connecting words to objects.

LLMs participate in social chains indirectly through training data.

Whether this is sufficient is debated.

Functional Understanding

Suppose a system can:

• explain combustion,
• predict consequences,
• correct misconceptions,
• apply the concept to new cases.

Calling this zero understanding may become strained.

The system clearly has usable structure.

Human Understanding Is Also Graded

Humans understand concepts at different depths.

A child understands gravity differently from a physicist.

Understanding is not always all-or-nothing.

Shallow vs Deep Understanding

A system may understand:

how a concept is used

without understanding:

its causal foundations.

Humans often operate this way too.

Competence Without Consciousness

A person can perform some tasks automatically.

Not all understanding requires conscious attention at every moment.

So consciousness may not be required for all functional understanding.

But Phenomenal Understanding Is Different

If by understanding we mean:

the felt grasp of meaning,

then current behavioral evidence cannot establish it.

That belongs to consciousness research.

Hallucination as Evidence Against Understanding?

Language models sometimes produce confident nonsense.

Critics argue a true understander would not.

But humans also:

• confabulate,
• misunderstand,
• hallucinate facts.

Error alone does not disprove understanding.

Error pattern matters.

Brittle Reasoning

A stronger concern is inconsistency under small reformulations.

If competence collapses when wording changes slightly, understanding may be shallow.

Robustness is diagnostic.

Counterfactual Testing

Ask the model to apply a concept under:

• unusual assumptions,
• novel worlds.

Successful transfer suggests relational structure rather than memorized association.

Causal Reasoning

Understanding often includes knowing:

what would happen if we intervened.

A model that only knows correlations may fail at causal counterfactuals.

Causal competence is a stronger test.

Explanatory Compression

Someone who understands can often explain many cases with one principle.

This is compression.

A model demonstrating compact transferable explanations shows deeper competence.

Self-Correction

A system that can:

• detect contradiction,
• revise,
• explain error

looks more understanding-like than one that only emits answers.

Metacognition strengthens the case.

Explanation Can Be Post Hoc

But a fluent explanation may be generated after the answer without reflecting the actual causal process that produced it.

Humans do this too.

Explanations need not be transparent windows into mechanism.

Interpretability

Mechanistic interpretability attempts to examine internal model structure directly.

If circuits correspond to:

• variables,
• algorithms,
• relations,

this can support stronger claims about internal computation.

World Models

Some AI systems learn predictive models of environments.

They can simulate:

• state transitions,
• consequences of action.

This looks closer to model-based understanding.

Language Models as World Models?

Some researchers argue that predicting language requires learning latent structure about the world.

Others argue linguistic prediction can succeed with less.

The empirical question is:

how much world structure is actually represented?

Embodiment Objection

Embodied cognition says human meaning depends deeply on:

• body,
• perception,
• action.

Text-only models lack these.

Multimodal and robotic systems weaken this objection.

They still do not reproduce human embodiment.

Human-Centrism

If understanding requires human biology, machines cannot understand by definition.

That is a biological criterion, not a neutral test of cognition.

We should state such commitments explicitly.

Functionalism

Functionalists are more open to machine understanding.

If the system realizes the relevant causal organization, substrate does not matter.

Language models then become empirical candidates.

Biological Naturalism

A theorist such as Searle may argue that the right biological causal powers matter.

Simulation of understanding is not understanding.

The disagreement is metaphysical.

Intentional Stance

We may predict an LLM by saying:

it knows, it believes, it expects.

This vocabulary can be useful.

Dennett’s intentional stance does not automatically settle literal inner states.

Does the Model Believe?

Belief normally implies:

• persistence,
• commitment,
• action consequences.

A stateless language model may not satisfy these strongly.

Agentic systems with memory make the question more interesting.

Session-Level Understanding

A model can maintain temporary context during one interaction.

This creates a form of local state.

But persistent identity and long-term belief may still be absent.

Tool-Augmented Understanding

If a model can:

• inspect evidence,
• run experiments,
• use tools,
• update conclusions,

its behavior becomes more epistemically grounded.

Architecture matters more than model weights alone.

Collective Understanding

A scientific institution contains many humans, databases, instruments.

Does the institution understand?

We often attribute knowledge to systems larger than individuals.

AI may also have distributed understanding.

Understanding Without Words

Animals understand many situations without human language.

This shows linguistic competence is not necessary for all understanding.

Likewise, linguistic competence may not be sufficient for all understanding.

Semantic Competence Is Still Real

Even if we reserve deep understanding for embodied conscious agents, language models exhibit real semantic competence.

Calling all success “mere autocomplete” hides important structure.

A serious analysis must acknowledge both capability and limitation.

The Binary Trap

The question:

Does it understand, yes or no?

may be poorly formed.

Understanding may have dimensions:

• linguistic,
• causal,
• embodied,
• social,
• phenomenal.

A system can score differently on each.

A Multidimensional View

We might ask separately:

Can it use concepts correctly?

Can it transfer them?

Can it ground them?

Can it model causes?

Can it consciously experience meaning?

This produces clearer debate.

The Philosophical Lesson

Language models make the concept of understanding unstable.

They possess substantial:

• linguistic,
• representational,
• functional competence.

Whether that amounts to full understanding depends on whether understanding requires:

• grounding,
• agency,
• consciousness.

The debate cannot be settled by one slogan.

The Next Question

Language models are already broad.

But broad is not the same as general intelligence.

What would count as an artificial system with genuinely general competence?

And what would happen if such a system exceeded humans across most cognitive domains?

That leads to:

Artificial General Intelligence and Superintelligence.