When you speak, what comes first, the idea or the word? Do you first feel a thought inside you, and only after that go searching for the right word to wrap around it? I think we all do. The word is never the start. The word is just the skin. The idea, the consciousness, is the thing sitting under it.

Sometimes the word comes out first. We call this “speaking before you think”. In certain situations, where the brain is deprived of context, this very much resembles next-word prediction in that we say plausible-sounding nonsense.

One clear example of this is split-brain patients – people who have had the connection between their brain hemispheres severed in a rather crude (but effective) treatment of difficult epilepsy resistant to other treatments (this way the seizures are usually limited to one hemisphere at a time). Speech is produced in the left hemisphere, so these patients frequently cannot convert thoughts occurring in their right hemisphere into speech.

The experiment setup shows them an image or instruction in only one visual field, such as “pick up a pen”. When this instruction is shown in the right visual field (connected to the left, speech-producing hemisphere), the patient has no trouble following it using their right hand and explaining why they performed the action they did. However, when it’s shown only in the left visual field (connected to the right, mute hemisphere), the patient’s left hand will pick up the item, but the when asked, the patient has no idea why they did so, and their speech-producing left hemisphere will in many cases just make shit up – or, to borrow LLM parlance, hallucinate.

Long back, Homo sapiens learned to speak and think. The real magic was not the sound itself. It was that we could hold abstract ideas in our head, things like law, justice, and philosophy. That is what made us stand out from every other animal. Different groups of humans made different sounds for the same ideas, and we call these sounds languages.

Thinking almost definitely evolved before speaking. In fact, there is some fossil evidence that human brains got smaller after we evolved speech. One explanation for this is that speech made thinking more efficient, as abstract concepts could now be represented as words rather than having to “brute-force” them with other parts of the brain.

It’s in fact quite outrageous to suggest that thinking is a consequence of evolving speech, because this would imply that most animals on the planet have no thoughts of any kind at all. If so, why would they bother carrying all that gray matter around? Perhaps the writer of this article thinks exclusively in internal monologue and has no (cognizant) abstract or pictorial thoughts.

Now come back to my first question. An LLM predicts the next word based on all the words before it. That is the whole story. There is no idea sitting underneath. The words are everything. For an LLM, words are the source, and any meaning is just a byproduct that falls out by accident.

We don’t know this. In some respects there is an idea sitting underneath, because the model has a metric fuckton of latent space state that changes as it processes text. We don’t know how thoughts or consciousness arise, so for all we know it is entirely possible that there’s flickers of consciousness happening in the matrix that makes up an LLM every time the model iterates.