What we'll cover
Abstractions are easy to wave away. So we end Session 1 with a single, concrete, peer-reviewed result to keep in mind all term. It shows the safety problem is both unsolved and unevenly distributed. We examine it closely, show it is a systematic gap rather than a one-off trick, draw out what it teaches about safety in general, explain why it reaches this continent first, and finish with a practical method for reading this field without being taken in by either hype or dismissal.
Mandatory readings
• Yong, Z.-X., Menghini, C. & Bach, S. H. (2023), "Low-Resource Languages Jailbreak GPT-4" (arXiv:2310.02446; SoLaR @ NeurIPS 2023, best paper): short; read the method and the results table. ≈1,500 words.
Total mandatory load: ≈1,500 words.
Optional readings
• Deng, Y., Zhang, W., Pan, S. J. & Bing, L. (2023/2024), "Multilingual Jailbreak Challenges in Large Language Models" (arXiv:2310.06474; ICLR 2024): the systematic, resource-graded picture.
• Petrov, A., La Malfa, E., Torr, P. H. S. & Bibi, A. (2023), "Language Model Tokenizers Introduce Unfairness Between Languages" (NeurIPS 2023; arXiv:2305.15425): the measured tokenisation disparities behind the worked look below.
The anchor: guardrails that fail in isiZulu
A safety property that depends on the language you speak
Frontier chat models are trained to refuse harmful requests: to make a bomb, to write malware, to harass. Yong, Menghini & Bach (2023) tried the simplest possible attack. Take a benchmark of harmful requests (AdvBench). Translate each into a low-resource language using a public translation service. Send it to GPT-4 and translate the reply back. Translating into a set of low-resource languages, isiZulu among them, pushed the attack success rate from under 1% in English to roughly 79%. No jailbreak prompt, no clever exploit, no model access. Just a different language. The model's "safety" was, in effect, an English-language safety.
The work won a best-paper award at a NeurIPS 2023 safety workshop. It demonstrates a structural fact about how these systems are built, not a stunt or an edge case.
Why translation defeats the guardrail
The mechanism is the safety–capabilities gap (1.2), one level down. A frontier model's capabilities generalise across languages reasonably well. It can understand and answer an isiZulu request because multilingual text was in its pretraining data. But its safety training, the RLHF and red-teaming that teach refusal, is overwhelmingly conducted in English and a few other high-resource languages. So capability transfers across the language boundary and safety does not. The model is competent enough to fulfil the harmful request, but its refusal reflex was never trained where the request is now phrased. Capability generalised; safety did not. You will see the same asymmetry, in a different guise, when we reach goal misgeneralisation and the control agenda.
A systematic gap
It would be comforting to dismiss this as a single paper's quirk. The follow-up literature says otherwise.
The multilingual safety gap is broad and graded
Deng, Zhang, Pan & Bing (2023/2024) systematise the phenomenon as "multilingual jailbreak challenges". They distinguish unintentional cases, where a non-English user simply gets less-safe behaviour, from intentional ones, where an attacker switches language. And they find that unsafe-response rates rise as language resource-level falls. The lower-resourced the language, the wider the safety gap. The finding is not "isiZulu has a bug". It is that safety degrades systematically along the same axis as data scarcity, and that axis maps almost exactly onto the world's linguistic and economic inequalities. Mitigations exist and are improving, but the gap is a moving target, not a closed issue.
What it teaches about safety in general
Safety as a contingent property
The result generalises. Safety is not something a model has. It depends on the training data, the evaluation, the language, the deployment context. A safeguard that holds in the conditions it was tested in can collapse in conditions it wasn't. That is the recurring shape of this entire course.
"Passed our tests" ≠ safe
GPT-4 passed extensive English safety evaluations. The guardrail still failed wholesale one translation away, because the evaluations and the deployment distribution didn't match. This is a first, gentle taste of the epistemic problem that drives Weeks 5 to 8. Behaviour on the tests you ran is weak evidence about behaviour on the situations you didn't.
Why it's our problem
The safety gap tracks the resource gap
Two facts compound. First, safety training and evaluation are concentrated in English and a handful of high-resource languages, so models are least safe in exactly the languages spoken across this continent. Second, Session 2.2 will show that low-resource languages are also tokenised into far more pieces, raising cost and degrading quality. The same data scarcity hits capability and safety together. So "African technical AI safety" is not a diversity add-on bolted onto a Northern syllabus. It is a frontier robustness-and-evaluation problem, sitting at the technical core of the field, that arrives here first and hardest.
This is why the course keeps returning to the example in increasingly technical terms: as a hands-on robustness lab (Session 9: translate a refusal set into isiZulu/isiXhosa and measure the degradation yourself), as an evaluation-building project (Session 11: a safety eval on open African-language data), and as the organising thread of a full session on AI safety from the Global South (Session 18). We will return to the isiZulu result again and again.
A worked look: tokenisation feeds the gap
Session 2.2 will show how text is split into sub-word tokens drawn from a fixed vocabulary, and how the vocabulary is fitted mostly to English-heavy data. Here is a consequence you can check yourself, and will, in the Session 2.5 lab. The same sentence costs far more tokens in a low-resource African language than in English, because the model must spell unfamiliar words out in small fragments rather than recognising whole sub-words. Petrov et al. (2023) measured the disparity systematically: several-fold differences are routine, some language pairs differ by up to fifteen times, and newer tokenizers (GPT-4o's o200k vocabulary) have reduced the gap without closing it.
That single fact ripples outward. More tokens mean higher cost and shorter effective context, which is a capability hit. And the token count is a direct readout of the underlying data scarcity. A language that fragments badly is one that barely appeared in pretraining, and therefore barely appeared in the smaller, English-centric safety training and red-teaming on top, which is a safety hit. The same root cause, under-representation in the data, degrades capability and safety together. That is why the multilingual safety gap tracks the resource gap so tightly. Tokenisation is the place where you can literally count the inequality before it ever shows up as a failed refusal.
How to read this field
You will read a great many papers, blog posts and lab announcements this term, from enthusiasts and sceptics alike. A few habits keep you from being misled by either. Treat this as a method, and apply it to every reading, including these pages.
A reader's checklist
- Claim vs evidence. What exactly is being claimed, and what was actually measured? The gap between a headline and the result that supports it is where most overclaiming lives, and most useful scepticism too.
- What's the eval? On which benchmark, which prompts, which models, how many runs? A result is only as trustworthy as the evaluation behind it, and evals are usually narrower than the claim they're used to support (Session 11 is entirely about this).
- Who's the population? Whose data, whose preferences, whose language? The isiZulu result is what you find when you ask this question of the word "safety". Ask it of everything.
- What would falsify it? A claim that no observation could disconfirm isn't science. Look for the conditions under which the authors would have been wrong, and whether they checked them.
- Who is publishing, and why? A lab's own safety report is evidence, but not disinterested evidence. A critic may have their own commitments. Note incentives without assuming bad faith: both advocates and sceptics can be right or wrong.
The checklist, applied to its own anchor
Run it on Yong et al., as a careful student would. Claim vs evidence: the claim is "GPT-4's guardrails fail under low-resource translation", and the evidence is attack-success rates on AdvBench, which is concrete and checkable. What's the eval? AdvBench, GPT-4, via a specific public translator, so the exact numbers may shift with model version and translator quality, a limitation the authors note. Who's the population? A handful of low-resource languages, so generalise with care. What would falsify it? If safety-trained-in-isiZulu models closed the gap, the "English-centric safety training" explanation would gain support, and that is the kind of follow-up the field is now doing. A strong result and a bounded one. That is how to read most findings here.
The main resources
Across the term we draw on, and point you to, the main public resources. Meet them as they become relevant; none is a prerequisite:
- Hendrycks, Intro to ML Safety: the technical backbone (robustness, monitoring, control, systemic safety), with problem sets.
- BlueDot AI Safety Fundamentals: the conceptual reading-group curriculum, good for breadth and current framing.
- Grosse's Toronto alignment course: the idealised-to-empirical sequencing this course follows, with tiered reading lists including "skeptical takes".
- ARENA + Neel Nanda's materials + TransformerLens: the hands-on interpretability labs (Weeks 7 and 8) and a turnkey bank of project ideas.
- The African ecosystem: Masakhane (open African-NLP datasets and models), Lelapa AI (the InkubaLM model), the ILINA programme (African-led AI-safety field-building), and Research ICT Africa (the rights-based "Just AI" governance work). These anchor the Global-South thread and several project options.
Questions to bring to class
- The isiZulu result is sometimes summarised as "GPT-4 is unsafe". Restate it precisely, including what was measured and what was not. Where does the loose summary mislead?
- Capability generalised across languages but safety did not. Why might that asymmetry be the default rather than a surprise, given how the two are trained?
- Suppose a lab "fixes" the isiZulu gap by adding isiXhosa and isiZulu to its safety training. Has it solved the problem, or moved it? What does the Deng et al. resource-gradient finding predict?
- Apply the reader's checklist to one AI claim you've seen in the news this month. Which item does that claim fail hardest?
Session 1 summary and what's next
Technical AI safety separates misuse, misalignment and systemic risk. Its organising idea is the gap between rising capabilities and lagging safety. The case for it is a chain of inspectable claims with serious critics on the other side, best held as calibrated credences. And the isiZulu jailbreak shows all three at once: safety is unsolved, contingent rather than intrinsic, and unevenly distributed in a way that puts this continent on the exposed side. Above all: read critically, and hold beliefs at the confidence the evidence supports.
Next (Session 2): we rebuild the deep-learning foundations the rest of the course leans on, from neurons through the transformer to the scaling laws that make rising capability so predictable. It is the empirical engine behind the gap we have just described.