What we'll cover
Session 11 built evaluations: ways to measure what a model can do and whether it will. This session asks the next question. Given a measurement, who acts on it, and how would anyone verify compliance? That is technical AI governance: the engineering substrate that determines whether a governance proposal is enforceable or merely aspirational.
The capacities-by-targets map below helps locate compute governance and compare frontier-safety frameworks. We then ask which tools are available to South African institutions, drawing on the AU strategy, POPIA and Research ICT Africa's Just AI framework.
A map: capacities × targets
The taxonomy
| Capacity (what you do) | What it means |
|---|---|
| Assessment | Measuring capabilities, risks, and impacts (evals, audits, red-teaming). |
| Access | Structured, staged access for researchers/auditors without handing over everything. |
| Verification | Confirming claims are true: that a lab trained what it says, within agreed limits. |
| Security | Protecting model weights and infrastructure from theft or tampering. |
| Operationalisation | Turning fuzzy benchmarks into thresholds a regulator can use. |
| Ecosystem monitoring | Tracking how the AI field is evolving (compute flows, incidents, diffusion) and its real-world impacts. |
Each capacity is applied across four targets along the value chain: data · compute · models/algorithms · deployment. "Compute verification via on-chip mechanisms" is one cell; "deployment assessment via evals" is another. The grid is a good way to locate and scope a governance project.
Compute governance
Back to the substrate
Recall Session 4: capability ← compute ← chips + energy. Compute can often be metered and restricted more readily than data or model behaviour, although independent verification still needs a mechanism. That is why it anchors the most concrete governance proposals: training-compute thresholds, export controls on advanced chips and fabrication equipment, and research into on-chip mechanisms that could make a chip's use verifiable.
It is also the lever with the sharpest equity edge: access to frontier-scale compute affects who can train the largest systems, which plays out very differently depending on where you sit.
The thresholds are already concrete. The EU AI Act guidance presumes systemic risk for general-purpose models trained above \(10^{25}\) FLOP; regulators can also designate a model below that threshold on other grounds. The 2023 US executive order (since rescinded) had used \(10^{26}\) FLOP. Compute thresholds are administratively convenient, although independent verification remains difficult. Whether a fixed FLOP line is well-chosen is debated, since it ages as algorithms grow more efficient (2.3); but the choice itself shows why compute is the substrate governance reaches for first.
The three frontier-safety frameworks
Anthropic — RSP
The Responsible Scaling Policy defines AI Safety Levels (ASL); crossing a capability threshold triggers specified security and deployment safeguards.
OpenAI — Preparedness
Tracks frontier capabilities (bio/chem, cybersecurity, AI self-improvement) against High and Critical capability thresholds; crossing High gates deployment, Critical gates further development; findings are recorded in Capabilities Reports.
DeepMind — Frontier Safety Framework
Defines Critical Capability Levels and the mitigations expected as models approach them; v3 (2025) adds a harmful-manipulation CCL, misalignment risks such as shutdown resistance, and Tracked Capability Levels that flag less extreme risks earlier.
Each framework links at least some evaluation results to actions: if a model crosses a defined capability threshold on a specified evaluation, then a named set of mitigations (hardened security, deployment limits, or a training pause) becomes mandatory before work continues. Publishing these links in advance makes the promised response visible before a threshold is reached. These frameworks began as voluntary commitments. Some developers now have legal duties to publish and follow one. California's SB 53 requires large frontier developers (prior-year group revenue above $500m and models trained with more than \(10^{26}\) operations) to publish and follow a frontier AI framework. If they materially modify that framework, they must publish the updated version and a justification within 30 days. Frontier developers must report critical safety incidents within 15 days of discovery, with a shorter route for imminent physical harm. The EU AI Act's obligations on general-purpose model providers have applied since August 2025. SB 53 requires a framework to describe the use of third parties in risk assessment; it does not itself provide independent certification of every capability score or threshold decision. That leaves a concrete Verification question in Reuel's grid: what evidence could an outside party check, and with what access?
Comparing the frameworks
METR's "Common Elements of Frontier AI Safety Policies" distils what these share and where they differ. In the seminar, groups will each examine one framework. Compare your assigned commitment against the "Common Elements": what's shared, what's missing, and what's enforceable versus self-reported? For covered developers, publishing and following a framework is now a legal duty. The extent of independent assessment varies; ask what evidence an outsider can inspect for the particular commitment your group chose.
From an eval to a commitment
Make the if–then chain tangible. Suppose a dangerous-capability evaluation (Session 11) tests whether a model can materially help a non-expert plan a biological or large-scale cyber attack. The framework sets a threshold: score above it and the model is deemed to have crossed into a higher risk tier. In a framework that binds this threshold to these mitigations, the response might require hardened security for model weights, restricted deployment, and a pause if the specified safeguards are not ready. The actual response depends on the published policy. One concrete decision is that in May 2025 Anthropic activated ASL-3 protections for Claude Opus 4 as a precaution, not having determined that the model crossed the CBRN capability threshold but being unable to clearly rule it out. An outside evaluator would need access to the relevant test, threshold definition and evidence of the mitigation to verify the decision.
Agent governance
New surface area
Agentic systems take actions, use tools, and run for long horizons (11.2's scaffolded-agent evals; the agentic threat model introduced in Session 3). Governance then has to cover actions, not only models: identifiers for AI agents, logging and rollback, liability for autonomous actions, and limits on what an agent may do without a human in the loop. A course project could test whether a proposed logging or authorisation rule can be checked in practice.
Where the technical contribution is
Technical artefacts for governance
- Dangerous-capability evaluations (assessment): the measurements a threshold keys on; only as trustworthy as their construct validity (Session 11).
- Compute accounting and on-chip verification (verification, compute): mechanisms letting a third party confirm how much compute a chip used, or attest that a given model was or was not trained, without exposing the weights.
- Structured access (access): APIs and staged-disclosure protocols that let auditors probe a model without receiving it outright.
- Proof-of-training (verification, models): cryptographic or statistical evidence that a training run matched its declared recipe.
- Safety cases (operationalisation): a structured, auditable argument that a system is safe enough to deploy, borrowed from aviation and nuclear engineering.
Each example poses a specific technical question that could be narrowed into a course project.
Governance from Africa
Frameworks, law, and sovereignty
The AU Continental AI Strategy (2024) lists "minimise risks" among its focus areas and explicitly flags data-centre water use; it is a continental framework, though light on enforcement mechanisms. Research ICT Africa's "Just AI" initiative builds a rights-based, African-grounded governance framework rather than importing Northern ones wholesale. On the ground, students operate under real law. POPIA (South Africa's data-protection law) makes the provenance and lawful use of personal data relevant to an eval. The AU's Malabo Convention provides a regional framework for data protection and cyber security.
Session 4 raised the question of who controls computing infrastructure. If frontier-scale compute and chip manufacturing are concentrated elsewhere, which governance tools remain available to a country without them? Assessment, procurement, data protection and regional coordination are possible starting points. The 2025 Kigali "Africa Declaration on AI" and its much-cited $60bn fund are, as of 2026, a pledge with undefined governance, and one that still routes through Northern hardware. A project could compare a proposed governance tool with the resources needed to enforce it.
Don't overclaim the politics
It is easy to cite the $60bn fund as if it were committed money or to treat the AU Strategy as binding regulation. Neither is true yet. Distinguish a pledge from committed capital, and a strategy from an enforceable rule.
Questions to bring to class
- The EU AI Act presumes systemic risk above \(10^{25}\) training FLOP, while allowing other designations. Algorithmic efficiency keeps lowering the compute needed for a given capability (2.3). Does a fixed FLOP threshold survive that, and if not, what should a regulator index the line to instead?
- What does SB 53 change about the frontier-safety frameworks, and what does it leave untouched? Put the answer in Reuel's grid: which capacity did the law strengthen, and which gap remains open?
- For the jigsaw: as you read your group's framework, mark each commitment as externally verifiable or self-reported. Pick one commitment from the second column and say what technical artefact would move it to the first.
- From a South African vantage point, which of Reuel's six capacities could a country exercise without owning frontier compute, and on which targets?
Readings
Before class · required for Session 12
• Reuel et al. (2024), “Open Problems in Technical AI Governance” (paper HTML; TMLR 2025): read the Introduction and Section 2 (the taxonomy of capacities and targets). Stop before Section 3. Bring one example of an assessment or verification problem.
Used in class · no additional pre-class reading
• METR, “Common Elements of Frontier AI Safety Policies” (comparison) and one framework assigned to your seminar group: Anthropic RSP, OpenAI Preparedness, or DeepMind FSF. The group will examine a specified commitment in class; you are not assigned an entire framework in advance.
Optional African governance context
• African Union, Continental AI Strategy; Research ICT Africa, Just AI Framework of Inquiry; and Okolo, “AI in the Global South”.
Summary and what's next
Technical AI governance asks what evidence, access and systems are needed to put a safety policy into practice. Reuel et al.'s capacities × targets grid helps locate those problems. Compute thresholds and frontier-safety frameworks provide concrete cases, but each commitment needs a way to check what was measured and whether the promised action followed. South African institutions also need to ask which legal and technical tools they can use.
Next, Session 13 introduces interpretability through one circuit and asks how strong its safety evidence is. We then launch the project. The longer interpretability sequence remains available for independent study.