what is consciousness?

LLMs aren’t conscious, though.

When you speak, what comes first, the idea or the word? Do you first feel a thought inside you, and only after that go searching for the right word to wrap around it? I think we all do. The word is never the start. The word is just the skin. The idea, the consciousness, is the thing sitting under it.

Sometimes the word comes out first. We call this “speaking before you think”. In certain situations, where the brain is deprived of context, this very much resembles next-word prediction in that we say plausible-sounding nonsense.

One clear example of this is split-brain patients – people who have had the connection between their brain hemispheres severed in a rather crude (but effective) treatment of difficult epilepsy resistant to other treatments (this way the seizures are usually limited to one hemisphere at a time). Speech is produced in the left hemisphere, so these patients frequently cannot convert thoughts occurring in their right hemisphere into speech.

The experiment setup shows them an image or instruction in only one visual field, such as “pick up a pen”. When this instruction is shown in the right visual field (connected to the left, speech-producing hemisphere), the patient has no trouble following it using their right hand and explaining why they performed the action they did. However, when it’s shown only in the left visual field (connected to the right, mute hemisphere), the patient’s left hand will pick up the item, but the when asked, the patient has no idea why they did so, and their speech-producing left hemisphere will in many cases just make shit up – or, to borrow LLM parlance, hallucinate.

Long back, Homo sapiens learned to speak and think. The real magic was not the sound itself. It was that we could hold abstract ideas in our head, things like law, justice, and philosophy. That is what made us stand out from every other animal. Different groups of humans made different sounds for the same ideas, and we call these sounds languages.

Thinking almost definitely evolved before speaking. In fact, there is some fossil evidence that human brains got smaller after we evolved speech. One explanation for this is that speech made thinking more efficient, as abstract concepts could now be represented as words rather than having to “brute-force” them with other parts of the brain.

It’s in fact quite outrageous to suggest that thinking is a consequence of evolving speech, because this would imply that most animals on the planet have no thoughts of any kind at all. If so, why would they bother carrying all that gray matter around? Perhaps the writer of this article thinks exclusively in internal monologue and has no (cognizant) abstract or pictorial thoughts.

Now come back to my first question. An LLM predicts the next word based on all the words before it. That is the whole story. There is no idea sitting underneath. The words are everything. For an LLM, words are the source, and any meaning is just a byproduct that falls out by accident.

We don’t know this. In some respects there is an idea sitting underneath, because the model has a metric fuckton of latent space state that changes as it processes text. We don’t know how thoughts or consciousness arise, so for all we know it is entirely possible that there’s flickers of consciousness happening in the matrix that makes up an LLM every time the model iterates.

This is actually two topics in one: 1) Language and reasoning, 2) how those models work. And there’s a lot to say about both, so… I’ll focus on the first one, even if the text’s focus is the second one. Disclaimers:

• This is just my personal take. I’m no expert on the area, my main area of interest is Proto-Indo-European.)
• I’ll use “Language” with a capital “L” to refer to the human faculty, and “language” with a minuscule “l” for communication systems using Language (like Mandarin, Arabic, English, LIBRAS, BSL, isiZulu etc.)

Now. Imagine a village like this:

There’s no street or pathway or anything similar there. All properties are “glued” to each other, separated by fences, but with no gap between them for public usage. Can you still walk through that village?

You can, but it’s messy: you’ll need to jump over a lot of fences, trample someone’s garden, perhaps even run from someone’s dog. (He’s still a good boy.) But, at least in theory, you can start from one side of the village and reach the other side, by walking through it.

Now imagine the same village had pathways, like any normal village. Walking through it becomes faster, more reliable, more predictable, less tiring. And you can even share directions with someone else.

So. If that village was the human mind, and “walking” through it was reasoning, I believe Language would be like the pathways. You can reason without Language; a lot other animals show signs of reasoning, and even us humans do it once in a while (we call it “intuition”). But reasoning with Language is faster, more reliable, and the outcome is more predictable.

I also think advanced reasoning is a pre-condition necessary for the development of Language in human beings. Because no matter the language, you see a recursive structure of blocks building blocks that requires good cognition: sounds/gestures forming units able to convey meaning, those units being combined into sentences, sentences into utterances etc. And it’s perhaps no surprise the non-human animals with the closest we know of Language — such as chimps and cetaceans — are also known for being damn smart, even for tasks unrelated to communication.

When you speak, what comes first, the idea or the word? Do you first feel a thought inside you, and only after that go searching for the right word to wrap around it? I think we all do. The word is never the start. The word is just the skin. The idea, the consciousness, is the thing sitting under it.

Debatable from a philosophical perspective. The words you have access to do indeed shape how you view the world. It could absolutely be argued that the words to which you have access shape your consciousness.

No it’s not.

(I’ll not explain, there’s Wikipedia for that)