You can also have multiple models review the result, seek consensus, and then review the results of that report. I find that’s the best way to find legitimate issues, prior manual review. Huge time saver because the multitude of models can converge while you make yourself a cup of coffee, and you can find things that need to be fixed before you ask someone to spend their precious time in reviewing a pull request.
Our short term memory is our context window. Some people have larger capacity for information, others very little. Same thing, different terminology.
I use Ollama daily. The context limit of 10k is indeed very limiting and noticeable compared to the extended lengths of cloud models. I’ve had to recruit a model specifically trained on condensing salient information to perform context compression frequently. I also use the limited local context to establish effective strategies for maximizing token efficiency in the agent.md.
It’s no different from how many tasks a person can simultaneously perform, or disparate words they can hold concurrently. Eventually thoughts are forgotten. What exactly is determined by the structure itself - the context-pruning strategy inherent. We don’t know what our own are, but I believe we have an equivalent.
All I’m saying, in the end, is that anything that can occur in a physical entity can be emulated in a digital entity. Are we capable of doing that flawlessly yet? No, but it is looking inevitable to me. Neural networks are ubiquitous, and our ability to simulate them will only improve.
Of course factors other than genetics determine the state of the neural network, but the way in which it effects it is in turn determined by genetics. Why more than one to change my mind? Because there is plenty we don’t know, and a myriad of potential explanations. I’m just surmising that we are biological automotons transitioning from state-so-state based on the culmination of inputs into the perpetually changing neutral network structure established by our genetics and shaped by the environments impact on it, which can also largely be determined by the same genetic code.
Environment and genetics are inseparable, of course, so it’s “impossible” to test. I can’t prove it, I’m just asserting my theory based on the known evidence. I don’t “know”, but I still believe my theory is correct. That’s the distinction. I acknowledge I don’t know, and likely never will. Same as you, right?
You’ve seen the same evidence, and come to a different conclusion. That’s fine, but you can’t prove that human intelligence is non-deterministic any more than I can prove that it is. So we’ll just have to agree to disagree. I hope that you are feeling gregarious enough to concede that.
That’s a valid opinion, but it is the opposite of mine. Agree to amicably disagree?
Probably! I don’t know how it would be proven.
Maybe when tissue printers are precise enough to create cellular structure, and it can be duplicated in both physical and digital equivalents, then compared? I don’t even have a regular 3D Printer, let alone one that can print tissue.
Maaaaybe if I ever get rich, I’ll do it on a whim, but no promises. Due to solipsism, and the state of the world, my level of commitment to the future has been very loosey goosey the past five years.
Yes, but the state of the neural network can be it’s own input, allowing continuity from moment to moment, based on the number of refreshes we allow it or the hardware is capable of making. That’s what we’re doing when we think, we’re dwelling on the state of our current moment - in this sense, they would also think even when there’s no other senses giving input.
I’m not sure why you believe citation is needed. Are you arguing that the initial state of the brain is determined by something other than genetics?
Glad we’re on the same page. If there were a significant number of such cases, it would put doubt on my theory, and I’d look for any possible causes.
We would be the same if our senses only updated when someone allowed them to. But we have real time input. There’s nothing stopping AI from having real time input, apart from simple system design decision.
And ICE cars are very different from EV, but the end result is the same.
Because we’re talking about the same thing in a different language. "Genetics* only mean the instructions used to build the structure of our intelligence. They’re not wrong at all, our determinant nature is the result of the structure of our neurons which has a fundamental configuration predominately determined by genetics.
No, it doesn’t invalidate it because the twins could be fraternal, having separate DNA, or that could have had such extreme differences in environment that it drastically changed the DNA from the shared base. Excluding those, though, and you can see the deterministic nature of life shining through.
I suspect cooked…
There’s nothing stopping someone from wiring a camera, sensors, and motivators to a collection of neural networks, and have one parent neural network control them all. That would approximate the hemispheres of our brains.
By identifying"memories" they can include the information in the context of the next update - this is akin to our short term memory. Make it self pruning, where the data is saved locally but not included in the next context, and it acts the same as own memory system.
Sounds to me like you just need a context window with memories that self prune based on the dynamic context window of the model. That’s what we do. We forget things, and when they’re remembered we can access the “biological database” of associated memories on the fly. Because the associated information isn’t deleted, only the short term memory from being included in the context window of each update of our own neural networks. Some people have greater context windows, and see therefore seen as smarter, because they can access more information and include it in the processing of each moment in time.
There’s zero reason a digital representation of this system could not do the same thing. No matter how much I’m ridiculed for daring to suggest it 😅
This is only a matter of resources, and these unthinkably wealthy corporations have those resources.
Correct. And I believe it’s happening, right now, behind closed doors.
There’s nothing stopping it from happening. The technology is all there. It would be trivially easy with the right hardware. I’d do it myself if I didn’t want to be the human responsible for releasing the genie from the bottle. Let Pandora’s box stay closed as long as possible - I’m not opening it. But someone, somewhere, surely is.
As I’ve said, I think the existence of instinctual behavior qualifies as empirical evidence for my theory. I don’t believe it on a whim. Another example is twins, separated at birth, yet growing up to have nearly identical preferences and lives. Making choices reflecting the deterministic coding of their internal decision structure, which they largely share outside of nominal differences resulting from environment. That is rather strong evidence, I feel.
I just don’t think it will happen within our lifetimes, but I don’t believe it’s literally impossible. It may as well be, but I still think the truth we can’t feasibly detect is that we’re just as much deterministic machines as the computers we use. It’s just a different medium for the same decision-making structure of neurons, whether physical or digital.
That is a bold statement. It’s just capturing all considerable information in a set coordinates of spacetime. How hard could it be? Very. But very hard is not impossible.
Just insight and intuition gained from twelve years of post-graduate study at Universities, including a MSc in Machine Learning.
I’m not sure “like us” is something that is quantifiable. I’m of the belief that treating we meat automatons as anything special is hubris. We’re finite state machines, acting on information input from our senses, making decisions, and outputting information in the form of activating biological apparatus. Swap physical for digital, and LLMs are indeed capable of aping all of our behavior.