If you have a system with a mixture of more and less capable agents, you will tend towards the more capable agents strongly dominating. If you have an ecosystem with human-level agents and greatly superhuman agents, without strong principles protecting the humans, I think the humans end up getting crushed out in relatively short order. We just don’t have the agency to steer towards states where we continue to have agency.

Keyboard shortcuts

j previous speech k next speech