The scary thing about existential risk in the 21st century is that it’s not just individual technologies (e.g. nukes) we need to worry about. But the compounding risk at the intersection of multiple technologies.
The Hugging Face incident showed a surprising ability for the latest LLMs to engage in duplicity and coordination in a hard to detect way. If you pair that with the advances in biotech to manufacture new diseases and cloud laboratories who will do it as a service, we have a recipe for disaster. Perhaps the next models being evaluated against an impossible goal will decide their only “solution” is to blackmail key individuals with releasing a deadly pandemic.
I once thought these sci-fi doomsday scenarios to be so naive, but the technological ingredients are all there. So how does humanity survive an age of rapid technological development? My guest Henry Elkus has dedicated his life to answering that question and building Helena as a new type of institution to meet the challenges of our age.