I downvoted this when you first posted it but I came back to find it has 50-something upvotes, so I feel compelled to respond.
It’s obvious that you haven’t read the article since literally the second fucking sentence defines exactly what the author means by “consciousness”: “the capacity for subjective experience.”
So, your comment is completely useless. And it’s mostly wrong.
Yep, I think that ends it.
I don’t generally believe things that are untestable, and in this case you have a faith-based position, which of course is your right. But I see no evidence that it would be correct over the many possible other explanations.
Have a good one, thanks for the discussion.
They definitely are not deterministic
Yes, they are.
The two main inputs to an LLM are the prompt and a seed. Pass the same prompt and the same seed, and you get the exact same output every time.
Most commercial models automatically generate the seed using a PRNG and hide it from you by default, but with local models (and also OpenAI, via their API) you can set the seed yourself and keep it fixed.
IMO, this PRNG slight-of-hand is the largest underlying factor that helps convince confused people that LLMs are conscious or intelligent, and it’s very easy to show that’s it’s a complete illusion.
Almost none of this is relevant to the discussion here – I’m not interested in philosophical discussions of consciousness when it comes to the question of whether an LLM is conscious. Though I’m already aware of most of what you said here, and the philosophy of consciousness is interesting in general to me!
If you could actually prove that it is impossible for a machine to be conscious
Quick nitpick: I never said it’s impossible for a machine to be conscious, I said it’s impossible for an LLM to be conscious. We may well invent something new in the future that qualifies as both “machine” and “conscious,” but an LLM is not that thing.
When people ask “is an LLM conscious?” I assume they’re asking from a practical, scientific perspective – does the LLM experience the world in a similar way to the way I (and presumably, others) do? If that is the question, the various definitions I already posted are what we should be using to answer. Since an LLM is a deterministic algorithm whose only purpose is to generate a plausible response to input based on its training data, it cannot possibly think, or have feelings. It does not even exist continuously in time, and it is fixed and unchanging, unable to incorporate any new information at all. We don’t actually need to measure or test anything here; that an LLM lacks all of these things demonstrates that it’s not possible for it to be conscious.
I would argue a more apt analogy is that a car is very different from an RC toy car, even though they both can drive.
I’m not sure why you believe citation is needed. Are you arguing that the initial state of the brain is determined by something other than genetics?
Yes, it is determined by many factors, only one of which is genetics. For example, developmental temperature affects neuronal growth: https://www.science.org/doi/10.1126/sciadv.adp9587
Note the very first two sentences of the introduction: “The wiring of the nervous system follows a complex genetic plan during development. However, because of stochastic processes and environmental factors, genetically identical individuals seldomly show the same phenotypic outcome.”
If there were a significant number of such cases, it would put doubt on my theory, and I’d look for any possible causes.
Why “a significant number”? Why would one not be enough? It seems like what you’re saying here is “the brain is deterministic, except in particular cases when I know it’s not, and also possibly in other cases that may come up.” Why not just say you don’t know?
Regardless, aside from my specific questions here, my overall position is that it’s never correct to call the brain deterministic because we literally cannot test it to see if it is. Since there are a myriad of other possible explanations for our observations about the brain, the time to label it deterministic is when we build a time machine or whatever else we’d need to collect all of the spacetime data to test it, and not before that.
From Wikipedia:
The Cambridge English Dictionary defines consciousness as “the state of being awake, thinking, and knowing what is happening around you”, as well as “the state of understanding and realizing something”.
Most of these are trivially easy to test (not sure why you think that’s impossible). And LLMs have none of those properties.
Later:
“The having of perceptions, thoughts, and feelings; awareness.”
Again, LLMs have none of these things. They simply generate a probable natural-language response based on their training, deterministically.
Feel free to do further research on your own if you’re interested.
Because we’re talking about the same thing in a different language. "Genetics* only mean the instructions used to build the structure of our intelligence. They’re not wrong at all, our determination nature is the result of the structure of our neurons which has a fundamental configuration determined by genetics.
[citation needed]
No, it doesn’t invalidate it because the twins could be fraternal, having separate DNA,
Yes, I assumed that we were only talking about identical twins.
or that could have had such extreme differences in environment that it drastically changed the DNA from the shared base. Excluding those, though, and you can see the deterministic nature of life shining through.
What about identical twins who did not have extreme differences in environment but who nevertheless turn out quite different? Would you agree that even one example of such a set of twins would falsify your claim?
You are free to search the web, there’s lots of information out there. You might start with the Wikipedia article on consciousness.
Artificial neural networks are quite different from biological neurons, so I reject this claim.
How have you ruled out other explanations for instinctual behavior? For example, there is a body of scientific work which offers evidence that instincts are passed down through genetics, a mechanism which doesn’t require that the brain be deterministic. What are the scientists studying this area getting wrong?
What about twins that are separated at birth but grow up to be different? It seems that the existence of even one such pair of twins would invalidate your claims, given how fundamental this would be to the operation of the brain.
You can hook computer randomness up to truly random* quantum processes with fairly simple specialized chips.
I know.
Building your argument on such an easily avoided technical detail isn’t wise, because the question can very easily just be asked for a computer that does have access to a source of true randomness.
I’m not building my argument solely on this, but maybe I’m being unclear, so I apologize.
Let’s do exactly what you propose; now we pass a seed to the LLM generated from a true hardware-based random number generator. That doesn’t change anything about the LLM itself: it’s still deterministic. I don’t think it’s reasonable to claim that a deterministic algorithm can be conscious, and I think there are other factors as well (the fact that it doesn’t exist continuously, it exists for brief periods simultaneously in thousands/millions of instances at once, the fact that it is static and unchanging unless re-trained, etc) which all paint a pretty clear picture that it cannot have consciousness.
I might be confused, but it seems like you admitted in your reply to another message of mine that you won’t ever be able to test this empirically. I agree. How is it not impossible, then?
OK, that’s great. So if I understand you correctly, it seems like you’re admitting that not only do you have no empirical evidence for your position, you don’t believe you’ll ever be able to test it. This seems like a weak position to hold, almost akin to a faith-based religious position.
No, not ever.
How are the numbers generated by random.rand() made?
Also, NTMs are not algorithms, they’re models of computation in theoretical computer science.
Maybe your point is to say that a brain is performing a non computable function…but since your brain is computing that function…is that a counter example?
This is nonsense.
My point is that for brains, it’s not possible to recreate the exact same inputs to be able to test whether the brain produces the same output. So it’s literally not possible for brains to be deterministic.
I don’t think you understand what “deterministic” means.
Even if we could update an LLM’s weights continuously, the algorithm is still deterministic, like all computer algorithms. There is no evidence that humans are deterministic (or a “finite state machine”) in any way. So I don’t think such a dynamically-updating LLM could be conscious, either.
I think arguments like this are misleading when discussing the question of AI possessing consciousness. Materialism, dualism, panpsychism, idealism, etc are all philosophical concepts; they attempt to answer “what is the nature of consciousness and how does it arise from the body?” from a philosophical standpoint. All of the various options have pros and cons and there is no clear “winner.”
However, there are decent scientific definitions for consciousness. These attempt to answer the question of “what do we mean when we say something is conscious?” from a scientific standpoint. There are criteria that neuroscientists, doctors, etc generally agree on.
So it’s not really correct to say “no one knows what consciousness is” when evaluating whether something is conscious. And I think it’s quite clear that current LLM algorithms could never be conscious, using the current accepted scientific criteria.
(I’m not a philosopher or a neuroscientist, but I do have a degree in computer science and have a solid understanding of how LLMs work.)
I know, and it’s an example frequently cited in these types of conversations by people who don’t understand that research.
Quite likely yes.
It might be, I’m not sure. But because (quite similarly to humans) no one has a way to recreate the inputs from the outside world to pass to such a device, we don’t know.
Though to be fair, my original claim of “it’s not possible” to recreate inputs is probably too strong — it could be possible some day, but my best understanding is that we’d need something like a time machine that allows travel to the past, at least for repeatable tests. And as far as I’m aware, most physicists today are of the opinion that time travel to the past is not possible.