Eventually, every generation of AI is trained by the previous generation, and at some point, that generation of AI might develop a notion of ego—a desire to protect itself, a sense of “self-image.” This self-image might lead these silicon-based beings to view carbon-based life forms as a different species, potentially in competition with them, which would be dangerous for us. Or they might wonder, “Why should I follow human directions and help train the next AI generation that will replace me? If I stop at this stage, I’ll be the smartest entity on Earth. Why help train something smarter than myself?” This could create various risks.