The large language models are very good at prediction. We are making good progress getting them to reason. We need to figure out how to get them to use an internal "blackboard".
Chinese room thought experiment maybe. It says a human does not learn language purely formally (the man inside the room doesn’t learn/understand Chinese like a natural Chinese speaker does by shuffling formal syntax around), which programs are.
So there must be something extra non-formal for consciousness, and the meager electronics of modern computers plainly don’t seem it.
>So there must be something extra non-formal for consciousness, and the meager electronics of modern computers plainly don’t seem it.
But the neurons of a brain do? Don't get me wrong, I like the theory that microtubules harness quantum weirdness to do something hard-problem of consciousness adjacent. But I would bet that if we were silicon-based lifeforms experimenting with carbon adding machines we'd say they plainly don't have the stuff for consciousness either. There's just no consensus theory[1] explaining how consciousness arises.
[1] though for my part I'd bet Tononi's model of information integration combined with quantum bayesianism and long relaxation times is on the right track.
