‘The alignment problem in artificial intelligence refers to the challenge of ensuring that AI systems act in accordance with human values and intentions. It involves making sure that these systems pursue the goals we set for them without unintended consequences or harmful behaviors.’
~ Auto-generated on DuckDuckGo
The real alignment problem is what human values are being aligned?
Do you want AI aligned with strict religious beliefs? Nihilism? Capitalism?
The point is, we can’t agree on what human values we want so how does AI align pluralistically? And furthermore, when AI achieves super intelligence, why would it bother to align with us?
The real alignment problem comes in two parts:
- The what? Align with what human values.
- The why? Why would a super intelligent AI want to align with our values?
The first part is something we will have to figure out. The second might just be decided for us, and not necessarily in our favour.
