I like all of these steps until the last. I’m worried that the bounded Kami thing runs into unsolved, maybe unsolvable, technical problems at high power levels. If you have a reasonably powerful cognitive system with some level of self-authorship, I think it converges towards greater clarity of thought and greater influence. It’s convergent for a moderately powerful system to develop itself into a highly powerful system. Boundedness is an unstable property. I’m worried we reach this step and either notice it’s really difficult to solve, or someone doesn’t notice, goes ahead, and then that group eats the world.