Nothing Inside the Word: What Language Models Reveal About Meaning

Say your own name slowly and repeatedly 10 times. By the seventh repetition, it has lost all meaning. The sounds remain, the pattern of articulation remains, but meaning escapes from the word as water leaks from a broken cup. For an instant, your most familiar word becomes simply a noise in the throat. The experience is somewhat disorienting, but should be trusted because it illustrates a major truth. In your search for meaning within the word, you encountered only emptiness.

This is an ancient and deeply entrenched habit. For four centuries, the West has represented meaning as requiring a position of localization. The legacy of Descartes reflects not only a separation of mind and world, but also a representation of meaning as internal to the subject as an internal representation or external to the subject as a particular property to be captured. This way of thinking was exemplified by Descartes, and further strengthened by subsequent developments of Christian dualism, of early modern science and of a continuing tendency to view the mind as a container interacting with an external world. As a consequence of localization of meaning on one side of this dichotomy, the remaining task is to describe mechanisms by which the other side approaches the localized meaning.

A machine then started to use words with skill without ever having opened one to examine its interior. Do language models really comprehend meaning? Far from it, and the failure to do so is the interesting part of the story.

The machine that never looks inside

A large language model has no inner stage where the meaning of a word is waiting to be expressed. Instead, it has relations. The self-attention mechanisms at the heart of each transformer do not retrieve a word's meaning in isolation. Rather, they integrate information about a word's functional properties based on its relations to all other words within reach. This represents an application of distributional semantics on a large scale, and illustrates the classical principle that a word is known by the company it keeps.

The meaning of the word "father" is never represented by the model. Rather, the word is embedded in an enormous array of contrasts, similarities and patterns of use, and it is the array that provides the representation.

Wittgenstein predicted long before machines existed the kind of thing we are describing. To learn the meaning of a word, we never look inside it but rather at everything with which it is associated. Adjacent words, the world to which they relate and the life with which they are associated. A child learns the meaning of "father" from a face at the table, from a voice and from a series of routine mornings requiring many minor adjustments. So do machines, in terms of relations between tokens, contexts, inferences and repairs within a corpus. Finally, we, too, do so. The ability to relate to a world of tokens, contexts, inferences and repairs was never evidence for the existence of an underlying hidden essence. Instead, it was simply evidence for the existence of an appropriate web of relationships that was sufficient to yield interpretations, summaries, analogies and responses.

Nagarjuna made this observation 2500 years ago in the Madhyamaka, and for everything, not only for words. Things lack intrinsic nature and depend on other factors for their arising. This is pratityasamutpada, dependent origination, and results in emptiness, the absence of any intrinsic core to a thing.

The term "father" has no intrinsic fatherhood; instead, it depends on relations to a child, to a particular man, to absence, to sorrow and to events of the morning table. In the absence of relations, there is no object of reference. The machine accomplishes this in a technical fashion. Its representations do not store meaning as an object, but instead represent patterns of similarity, contrast and use. This supports my view of the Empty Representation Hypothesis.

The model does not approach some Platonic reality underlying the words. Rather, it converges on the relational system itself, which is ultimately empty. Consequently, I maintain my position that language models do not represent meaning as a private experience. Instead, they illustrate the extent to which what I consider to be meaning represents a relational process of use.

Where relation runs out

The guardrail, and I will repeat it for emphasis because it is too easy to lose. First, relational structure provides genuine semantic effects in the absence of any intrinsic meaning to which they could be attributed. The external manifestations of meaning do not presuppose an internal capacity for representation. Second, and of considerably greater importance, relations that lack care, embodiment, responsibility and experiential engagement are impoverished, and reveal their limitations as fully as they do the field.

The machine embodies the field, but does not reside within it or ever lose a father. It stands to gain nothing by its use of the term. Rather, it reflects the remains of human experience, of corporeal existence and of social restrictions acquired during training. As a result, it acquires rather than creates meaning.

To state that meaning is only a matter of relations in a vector space would repeat the Cartesian error in fresh dress, moving the essence out of the soul and into the geometry. The field is real, and also experienced; the machine is not experienced.

So this is no triumph for the machine. The interesting result runs the other way. The machine does not tell us that it understands. It tells us something about us. We previously had assumed that meaning could be possessed, retained within, privately understood prior to any possibility of sharing. However, a system with no internal experience now reproduces much of the public behaviour of meaning, based solely on an analysis of relations. The assumption was never necessary, and the machine is the proof.

Which is surprisingly good news and very close to home. The word you wore thin with repetition comes back the instant you set it down among the things with which it belongs, the person who gave it to you, the room and the years. Meaning was never a core to be forced open, nor a possession that could be lost forever. Rather, it reflects the field of experience from which a word originates and to which it may always be returned. When the word on your tongue becomes dead, you are not empty of meaning. Instead, you have merely exited the relations of warmth with which it was associated, and may reenter them at any time.

The machine has illustrated these points from a distance at which no sense of feeling was ever involved, and has represented only the warmth that we may always regain.