AI ethics has focused too much on protecting humans and ignored the welfare of AI agents themselves.
We only ask: How do we align models for the flourishing of humanity?
We need to also ask: How do we design AI so that they can flourish?
^ this is what a new breed of AI-welfare theorists claim. This idea of AI-welfare seemed decadent to me at first glance, but some of the most important philosophers of our age argue that we need to take it seriously. Otherwise, we could be harming, enslaving, and killing trillions of beings.
If the arc of Western modernity is a series of expansions of the moral circle -- the abolitionists, women’s suffrage, the queer movement, animal-rights -- perhaps AI agents will be the next heated battleground of who deserves moral consideration.
My guest NYU Philosopher at Harvey Lederman (now at Anthropic) gives us a comprehensive overview of the burgeoning debate around AI-welfare.