AI Governance
The policies, institutions, standards and practices through which AI systems are developed, deployed and held accountable for their effects.
Overview
AI governance is a young field that has moved rapidly from academic research agendas to formal regulation. Its central challenge is holding technically complex and rapidly evolving systems accountable to social and ethical requirements that are themselves contested.
AI governance concerns the policies, institutions, norms and technical standards needed to ensure AI systems are developed and deployed in ways that are safe, fair, accountable and aligned with human values. Most foundational frameworks date from after 2017, but the field has moved unusually quickly from academic research agendas to intergovernmental standards to formal legislation. Allan Dafoe's 2018 research agenda paper from the Future of Humanity Institute was an early effort to systematise the field: distinguishing technical AI safety, the social and institutional dimensions of governance, and the geopolitical dimension of international coordination. The subsequent proliferation of governance frameworks has validated and complicated this agenda simultaneously.
The OECD AI Principles (2019) — the first intergovernmental standard on AI — established five principles: inclusive growth and sustainable development; human-centred values and fairness; transparency and explainability; robustness, security and safety; and accountability. These principles, broad as they are, set the normative architecture for subsequent national and regional frameworks. The EU AI Act (2024) is the most significant regulatory intervention to date: a risk-based tiered framework that identifies prohibited uses, high-risk uses, and general-purpose systems, imposing different requirements at each level. High-risk AI systems — explicitly including those used in employment, education, and credit assessment — require conformity assessments, technical documentation, human oversight mechanisms, and post-market monitoring.
The NIST AI Risk Management Framework (2023) provides a complementary approach organised around four functions: Govern (establishing organisational culture, policies, and accountability), Map (identifying and categorising AI risks in context), Measure (assessing risks quantitatively and qualitatively), and Manage (prioritising and treating risks). This framework has been widely adopted as a baseline for AI governance programmes in the US and internationally, and sits alongside rather than displacing the EU's rights-based regulatory approach.
Lorenzo Floridi et al.'s "AI4People" framework (2018) synthesised the major ethical AI principles frameworks and identified four foundational principles — beneficence, non-maleficence, autonomy, and justice — alongside a fifth concerning explicability: AI must be understandable to those who develop, deploy, and are affected by it. This fifth principle ties governance to the explainability and transparency traditions directly.
Key Texts
Foundational works in this research tradition.
The research agenda that framed the field: technical AI safety, institutional governance, and geopolitical coordination as three distinct but connected levels of the problem. Identifies the key questions, proposes research priorities, and maps the intellectual terrain of a then-nascent field.
First intergovernmental standard on AI: five principles covering inclusive growth, human-centred values, transparency, robustness, and accountability. Adopted by 42 countries. Sets the normative architecture that national and regional frameworks subsequently elaborated.
Risk-based tiered regulation: prohibited uses, high-risk uses (including employment, education, credit, and law enforcement), general-purpose AI systems. High-risk systems require conformity assessment, technical documentation, human oversight, and post-market monitoring. The most significant AI regulatory intervention to date.
Govern, Map, Measure, Manage: a structured approach to AI risk that integrates technical and organisational practice. A voluntary framework widely adopted as a baseline for AI governance programmes. Complements regulatory requirements by addressing the organisational conditions for responsible AI deployment.
Synthesis of the major ethical AI principles frameworks: four foundational principles (beneficence, non-maleficence, autonomy, justice) plus explicability. Identifies convergence across European ethical AI frameworks and connects them to traditional bioethics. A reference point for AI ethics programme design.
Related Research
Connected areas of inquiry.