Your comment made me think that as long as weights/network are frozen, AI cannot improve itself. Monitoring them would be critical for safety. Is this correct?
The distinction between code and data is just an illusion. If the "code" is frozen but IO unconstrained, then it's just as if the "code" wasn't frozen.
The problem is unfortunately a human one. As soon as someone figures out the 11 herbs and spices to unlock AGI, no matter how good their safety systems are, someone else will run off and build their own, without the safety systems in place.
