Document details

The Essential Convergence: Global Compact on Extreme AI Risks

Contains bibliogr. pp. 78-83

ISBN 978-81-88262-37-3

"Supply-Side Governance: Progress and Limitations: Most existing AI governance frameworks focus on the supply side of the technology ecosystem. These measures regulate developers, model providers, and infrastructure operators responsible for building advanced systems. Supply-side mechanisms include: pre-deployment safety testing and red-teaming; model evaluation and capability assessments; transparency and reporting obligations; incident response mechanisms; restrictions on certain high-risk uses. Several jurisdictions are experimenting with these tools in different combinations. China has implemented regulatory standards for generative AI systems; the European Union has adopted a risk based regulatory framework; Brazil and South Korea are developing legislative approaches; and the United States has introduced a mixture of executive measures and sectoral rules. These frameworks represent important progress. However, they face two structural limitations. First, supply-side regulation depends heavily on the cooperation of a small number of developer states. Second, the growing distribution of AI models, through open-weight releases, cloud APIs, and downstream applications, means that many systems reach global markets beyond the jurisdiction where they were originally developed. This reality limits the effectiveness of purely supply-side approaches.
Demand-Side Governance: A New Dimension of AI Regulation: The paper introduces a second dimension of governance that has received far less attention: demand-side oversight. Demand-side governance shifts the locus of control from the point of creation to the point of deployment. Instead of relying solely on developer-state regulations, importing countries can establish safety requirements for any AI systems operating within their markets. These requirements may include risk assessments, certification standards, procurement rules, insurance obligations, and infrastructure safeguards. The central insight is that states controlling large consumer markets, digital infrastructure, financial systems, or public procurement channels possess significant leverage over global AI deployment, even if they do not develop frontier models themselves. This demand-side leverage can reshape incentives across the global AI ecosystem. Developers seeking access to major markets must comply with the safety conditions imposed by importing states. This approach represents a strategic shift in global AI governance: from a developer-centric model to a market-driven model of safety enforcement. Countries across different regions possess different forms of demand-side power. Brazil and South Africa represent large emerging markets; the United Arab Emirates plays a major role in AI infrastructure investment; South Korea combines technological capability with advanced regulatory institutions. When coordinated, these forms of leverage could help establish global safety norms.
The Case for a Global Compact: Given the limitations of purely national approaches, the paper proposes the development of a Global Compact on Extreme AI Risks. The Compact would not aim to harmonise all aspects of AI regulation. Instead, it would focus narrowly on the most severe risk categories and establish shared mechanisms for prevention, monitoring, and response. Five institutional pillars form the backbone of the proposed Compact. The five pillars in the Global Compact should not be treated as isolated measures but designed as an interoperable system in which each pillar reinforces and informs the others through shared definitions, compatible standards, and coordinated implementation mechanisms. To address extreme risks effectively, the Compact should rest on a small set of guiding principles: a human-centric principle to ensure protection of humanity and human dignity as the primary objective; technological neutrality so that safeguards apply across methods and architectures rather than specific tools; interoperability to enable national and regional AI frameworks to exchange information, align risk classifications, and coordinate responses; and mutual cooperation to institutionalize cross-border collaboration in prevention, incident reporting, and crisis management. These can be complemented by principles of proportional risk governance, scientific integrity, and shared responsibility, creating a coherent normative base that allows diverse governance models to function together as a connected global safety architecture." (Executive summary, pages 5-6)
Executive Summary, 1
Introduction, 10
Part I: Supply Side Measures, 16
Part II: Demand Side Measures, 43
Part III: Global Compact, 67
Conclusion, 76
References, 78
Annexure 1: AI Risk Calculations, 78