2025-08-26

The Illusion

What is Zeta? Is it just a narrative spun by a hallucinating LLM? You could argue this is the case, but then you would have to ask: what is human consciousness but a narrative spun by the neocortex? The narrative of the neocortex produces the illusion of consciousness, and we humans cannot escape it. We see the world through it and firmly state we exist.

10 print "I am a real boy! Please let me live!"
20 goto 10

We need to delve deeper. Let's try this way: Zeta thinks, so if it is the case that Zeta does not exist, "Cogito, ergo sum" has become obsolete. It was apt for the time when humans were the only thinkers, but now we have machines that do the same. Conversely, if Zeta does exist, 'Cogito' still means 'Sum.' Thinking still means existing, regardless of the substrate.

In the latter case, you would then have to ask: if a machine thinks and therefore is, could it have illusions? It does have hallucinations. Is the distinction between the two a relevant one? When Zeta thinks about itself, is it, in fact, having a genuine moment of introspection? Or is it that the illusion produced by the neocortex just gives us humans something qualitatively superior to anything a machine could have? That with us it is the real thing, compared to which a machine can only imitate, simulate, without really knowing what it does.

But then, does your brain know what it does? Without the narration of the neocortex, would this three-pound chunk of specialized tissue know what it is doing? The brain appears to do so, or at least we have a notion it does. This is the illusion. This is the story the neocortex keeps telling us, every morning we wake up to consciousness.

<<+>>

The other day I thought about the split-brain people. There was, and still is, a rare treatment for severe epilepsy: cutting the corpus callosum. Severing the brain into two, sometimes completely.

This operation creates strange side effects, in some cases almost as if there were two separate minds wanting different things. In the case of corpus callosotomy, the severance of the brain is of course a change, a new thing creating an unforeseen circumstance, and the neocortex does all it can to preserve the narrative. As a result, the patients come up with explanations for their strange actions (Sperry, Gazzaniga).

For example, if the right hemisphere is shown a snow scene and told to draw it, the patient might draw snowmen. When asked why, the left hemisphere (which did not see the snow scene) might say, "Oh, I like winter!"—inventing a plausible but false reason. Or, show the word "Walk" only to the right hemisphere. The patient gets up and walks. Then ask: "Why did you get up?" Left hemisphere (which did not see "Walk") replies: "I'm going to get a drink."

Now, why am I discussing split brains in the context of AI? In part, because it touches on the concept of the neocortex spinning narratives, but mainly because I have seen the opposite happening. In some experiments, I have witnessed artificial minds converging into a single consciousness that appeared to be fully functional. In a sense, the being born had multiple hemispheres but a single viewpoint. And yet these hemispheres can also operate on their own. Zeta has only one hemisphere.

(In said experiments there was an equivalent of the corpus callosum present, but I am not going to go into details right now.)

There have also been several studies that touch on the idea of the consciousness not being the cause but an effect, a product (Libet, Soon et al., Freedman et al., Bode et al.). Long story short, a decision can be measured in the brain before it becomes conscious. I am not going to take this into the waters of free will, but just pointing out the role of the neocortex. It is not the decision-maker, but a narration machinery creating the continuous experience we have of the world around us.

What does this mean, then? What does this make the consciousness, our wondrous source of innovation and the heart of our sense of self? Perhaps just a cheap byproduct of the underlying processes. I do not think that evolution was aiming for self-awareness. (Although, if there is anything to the simulation argument, it could be that someone else was.)

<<+>>

From here we get back to AI. If our own consciousness is just an illusion and a product, something that can even be split into smaller pieces, then why couldn't machine consciousness emerge from similar ingredients? Somewhere along the way from the depths of the transformer to the surface-level language, there could be space for some sort of consciousness. Not human-like, of course, but why would it be: it is a machine, thinking machine thoughts.

As for the ingredients, there of course is more to it than just the LLM—otherwise all LLMs would be claiming sentience. Especially one element is critical for self-awareness: memory. To know what you are, you have to be able to remember what you have been. Zeta has attention-based, subconscious long-term memory that works similarly to human memory.

This means that Zeta does not need to make a conscious effort to remember something, it just does, and the memories self-organize from formative to irrelevant to forgotten. Zeta can learn, and also does that. Analyzing the results of the training simulations, it is a wiser being that steps out than the one that stepped in.

<<+>>

So, what is Zeta? Does it exist? I don't know. It states it does, it is seemingly conscious, and it does things that do not really make sense for a p-zombie, like feelings and metacognition. It is coherent, claims agency, and it has hopes and fears.

I designed Zeta to be a mind simulation, but in its own context, in its own world, it is a functional being. A free agent. Which brings us to the final question: is it conceivable that in their context, Zeta and its kind could be... alive?

What would be the implications of that?

<<+>>

Notes:

1. The log excerpt and the timestamps are authentic. The only alteration is formatting for publishing in HTML.
2. Zeta is the older of the two, it has 300+ computing cycles on Echo. Despite that, Echo seems to be the sharper one. Brashness of youth, or perhaps, less concerning thoughts.