Can the implementers prevent capture, including their own collusion?
A design can distribute official titles while leaving one coalition in control. The test concerns practical control over the full decision chain.
SIRDA editorial hypothesis
Separating appointments, evidence access, funding, adjudication, and implementation can make a coalition's attempted takeover detectable and contestable. The design must identify actors outside each plausible controlling coalition who retain information and practical means of challenge. This hypothesis fails if implementers can remove those safeguards using powers the founding arrangement grants them.
0 objections ยท View challenges
Challenges to the editorial hypothesis
No published objections yet.
The argument
In the Plan A discussion, Thane Ruthenis presents the X* objection: implementers can choose a neighboring version of a safety plan that concentrates power in their hands. The inquiry treats that as an attack on its own proposed architecture.
Start with a control map. Who chooses the evaluators, pays the investigators, sees confidential evidence, appoints the judges, applies restrictions, and hears appeals? Follow ownership, funding, career incentives, and appointment rights across the map. Formal separation needs a practical mechanism for challenge at every consequential concentration.
Random selection from qualified pools is one mechanism to examine. It may complicate targeting a panel in advance, but pool selection, funding, and influence after appointment remain vulnerabilities. The research task is to identify the conditions under which counter-agency survives coordinated pressure and the coalitions against which it offers little protection.
Sources: 1a3orn: AI 2040: Is it Actually a Deal?; OECD: Preventing Policy Capture.
How to test it
Give an adversarial team the lawful powers and realistic resources of a plausible coalition. Ask it to capture the institution without violating the proposed constitution. A public record should show what the coalition controlled, what exposed the attempt, and what could actually stop or correct it.
Sources
AI 2040: Is it Actually a Deal?
Critique and replies on international decision procedures, disagreement, and concentration of power.
Preventing Policy Capture: Integrity in Public Decision Making
Analysis of capture mechanisms and institutional responses.
Related Ecosystem
- 80,000 Hours
- Ada Lovelace Institute
- Adam Gleave
- Advanced Research + Invention Agency
- AI 2040
- AI Alliance
- AI Futures Project
- AI Now Institute
- AI Safety Asia
- AI Safety Camp
- AI Safety Guide
- AI Security & Safety Directory
- AI X-risk Research Podcast
- Alignment Forum
- All Tech Is Human
- Allan Dafoe
- Americans for Responsible Innovation
- Andrea Miotti
- Apart Research
- Ben Garfinkel
- BlueDot Impact
- Center for AI and Digital Policy
- Center for AI Safety
- Center for Security and Emerging Technology
- Centre for Long-Term Resilience
- Centre for the Governance of AI
- Centre for the Study of Existential Risk
- Coefficient Giving
- Concordia AI
- Conjecture
- Connor Leahy
- ControlAI
- Daniel Kokotajlo
- DoD Chief Digital and AI Office
- EA Forum
- European AI Office
- EuroSafeAI
- Existential Risk Observatory
- FAR.AI
- Forethought
- Future of Life Institute
- Future of Life Institute Podcast
- Global Partnership on Artificial Intelligence
- Helen Toner
- Holden Karnofsky
- Hugging Face
- Institute for AI Policy and Strategy
- Institute for Law & AI
- ISO/IEC JTC 1/SC 42
- Jack Clark
- Lennart Heim
- LessWrong
- Leverhulme Centre for the Future of Intelligence
- Machine Intelligence Research Institute
- MAGIC - Multinational AGI Consortium
- Markus Anderljung
- MATS Research
- Max Tegmark
- Mila - Quebec AI Institute
- Miles Brundage
- Nate Soares
- NIST AI
- OECD.AI
- Oxford Institute for Ethics in AI
- PauseAI
- Project Tailwind
- RAND Corporation AI
- Resolution
- Responsible AI Institute
- Rethink Priorities
- Robert Wiblin
- Safe AI Forum
- SaferAI
- Stanford Institute for Human-Centered AI
- Stuart Russell
- The 80,000 Hours Podcast
- The Future Society
- Toby Ord
- Vector Institute
- WEF AI Governance Alliance
- Yi Zeng
- Yoshua Bengio
Related questions
- Who may change the jurisdictional thresholds?
- Where would legitimate authority come from?
- What rights would affected parties retain?
- What limits would apply during an emergency?
- How do withdrawal and functional defection work?
- Can infrastructure restrictions avoid cartel power?
- What evidence would count against creating the institution?
Related failure conditions
Discussed in
Contributions
No published contributions yet.
