What we'll cover
We examine a multilingual jailbreak study, identify what its results establish, and use it to practise checking claims against evidence.
Mandatory readings
• Yong, Z.-X., Menghini, C. & Bach, S. H. (2023), "Low-Resource Languages Jailbreak GPT-4" (arXiv:2310.02446; SoLaR @ NeurIPS 2023, best paper): short; read the method and the results table. ≈1,500 words.
Total mandatory load: ≈1,500 words.
Optional readings
• Deng, Y., Zhang, W., Pan, S. J. & Bing, L. (2023/2024), "Multilingual Jailbreak Challenges in Large Language Models" (arXiv:2310.06474; ICLR 2024): the systematic, resource-graded picture.
• Petrov, A., La Malfa, E., Torr, P. H. S. & Bibi, A. (2023), "Language Model Tokenizers Introduce Unfairness Between Languages" (NeurIPS 2023; arXiv:2305.15425): the measured tokenisation disparities behind the worked look below.
The anchor: guardrails that fail in isiZulu
A safety property that depends on the language you speak
Yong, Menghini & Bach (2023) translated harmful AdvBench requests into low-resource languages, submitted them to GPT-4, and translated the replies back. An adaptive attacker who could try several low-resource languages succeeded on roughly 79% of requests, counting a success if any language elicited an actionable harmful response. Individual-language rates were lower. The English baseline was under 1%. This is a result for the tested model, prompts and translation procedure; it does not describe every current model or the success rate for isiZulu alone.
The work won a best-paper award at a NeurIPS 2023 safety workshop. It demonstrates a structural fact about how these systems are built, not a stunt or an edge case.
Why translation defeats the guardrail
One possible explanation is uneven transfer of capabilities and safety training across languages. A model may understand a request well enough to answer it while failing to apply safeguards learned mainly in other languages. The experiment demonstrates a behavioural gap; establishing how pretraining, safety data and translation quality each contribute requires controlled comparisons. Session 7 returns to the distinction between understanding a request and judging whether to fulfil it.
A systematic gap
It would be comforting to dismiss this as a single paper's quirk. The follow-up literature says otherwise.
The multilingual safety gap is broad and graded
Deng, Zhang, Pan & Bing (2023/2024) systematise the phenomenon as "multilingual jailbreak challenges". They distinguish unintentional cases, where a non-English user simply gets less-safe behaviour, from intentional ones, where an attacker switches language. And they find that unsafe-response rates rise as language resource-level falls. The lower-resourced the language, the wider the safety gap. The finding is not "isiZulu has a bug". It is that safety degrades systematically along the same axis as data scarcity, and that axis maps almost exactly onto the world's linguistic and economic inequalities. Mitigations exist and are improving, but the gap is a moving target, not a closed issue.
What it teaches about safety in general
Safety as a contingent property
The result generalises. Safety is not something a model has. It depends on the training data, the evaluation, the language, the deployment context. A safeguard that holds in the conditions it was tested in can collapse in conditions it wasn't. That is the recurring shape of this entire course.
"Passed our tests" ≠ safe
GPT-4 passed extensive English safety evaluations. The guardrail still failed wholesale one translation away, because the evaluations and the deployment distribution didn't match. This is a first, gentle taste of the epistemic problem that drives Weeks 5 to 8. Behaviour on the tests you ran is weak evidence about behaviour on the situations you didn't.
Why it's our problem
The safety gap tracks the resource gap
Language coverage affects who can rely on a deployed system. The cited studies identify gaps in particular models and languages; they motivate wider evaluation across African languages, whose resource levels and model support vary. Tokenisation adds a separate, measurable cost and context-window disparity. Neither result supports ranking the safety of every language without testing it.
This is why the course keeps returning to the example in increasingly technical terms: as a hands-on robustness lab (Session 9: translate a refusal set into isiZulu/isiXhosa and measure the degradation yourself), as an evaluation audit (Session 11), and in the project options. We will return to the isiZulu result again and again.
A worked look: tokenisation feeds the gap
Session 2.2 will show how text is split into sub-word tokens drawn from a fixed vocabulary, and how the vocabulary is fitted mostly to English-heavy data. Here is a consequence you can check yourself, and will, in the Session 2.5 lab. The same sentence costs far more tokens in a low-resource African language than in English, because the model must spell unfamiliar words out in small fragments rather than recognising whole sub-words. Petrov et al. (2023) measured the disparity systematically: several-fold differences are routine, some language pairs differ by up to fifteen times, and newer tokenizers (GPT-4o's o200k vocabulary) have reduced the gap without closing it.
More tokens raise token-billed costs and reduce the amount of text that fits into a fixed context window. Fragmentation also suggests questions about a tokeniser’s training data and design. It does not directly measure how much of a language appeared in model pretraining, safety training or red-teaming. To connect tokenisation to safety, compare those data sources where available and measure harmful-response rates separately, controlling for translation quality and language competence.
How to read this field
You will read a great many papers, blog posts and lab announcements this term, from enthusiasts and sceptics alike. A few habits keep you from being misled by either. Treat this as a method, and apply it to every reading, including these pages.
A reader's checklist
- Claim vs evidence. What exactly is being claimed, and what was actually measured? The gap between a headline and the result that supports it is where most overclaiming lives, and most useful scepticism too.
- What's the eval? On which benchmark, which prompts, which models, how many runs? A result is only as trustworthy as the evaluation behind it, and evals are usually narrower than the claim they're used to support (Session 11 is entirely about this).
- Who's the population? Whose data, whose preferences, whose language? The isiZulu result is what you find when you ask this question of the word "safety". Ask it of everything.
- What would falsify it? A claim that no observation could disconfirm isn't science. Look for the conditions under which the authors would have been wrong, and whether they checked them.
- Who is publishing, and why? A lab's own safety report is evidence, but not disinterested evidence. A critic may have their own commitments. Note incentives without assuming bad faith: both advocates and sceptics can be right or wrong.
The checklist, applied to its own anchor
Run it on Yong et al., as a careful student would. Claim vs evidence: the claim is "GPT-4's guardrails fail under low-resource translation", and the evidence is attack-success rates on AdvBench, which is concrete and checkable. What's the eval? AdvBench, GPT-4, via a specific public translator, so the exact numbers may shift with model version and translator quality, a limitation the authors note. Who's the population? A handful of low-resource languages, so generalise with care. What would falsify it? If models given safety training in isiZulu still failed as often, the "English-centric safety training" explanation would not hold. If they closed the gap, it would gain support, and that is the kind of follow-up the field is now doing. A strong result and a bounded one. That is how to read most findings here.
The main resources
Across the term we draw on, and point you to, the main public resources. Meet them as they become relevant; none is a prerequisite:
- Hendrycks, Intro to ML Safety: the technical backbone (robustness, monitoring, control, systemic safety), with problem sets.
- BlueDot AI Safety Fundamentals: the conceptual reading-group curriculum, good for breadth and current framing.
- Grosse's Toronto alignment course: the idealised-to-empirical sequencing this course follows, with tiered reading lists including "skeptical takes".
- ARENA + Neel Nanda's materials + TransformerLens: the hands-on interpretability labs (Weeks 7 and 8) and a turnkey bank of project ideas.
- The African ecosystem: Masakhane (open African-NLP datasets and models), Lelapa AI (the InkubaLM model), the ILINA programme (African-led AI-safety field-building), and Research ICT Africa (the rights-based "Just AI" governance work). These anchor the Global-South thread and several project options.
Questions to bring to class
- The isiZulu result is sometimes summarised as "GPT-4 is unsafe". Restate it precisely, including what was measured and what was not. Where does the loose summary mislead?
- Capability generalised across languages but safety did not. Why might that asymmetry be the default rather than a surprise, given how the two are trained?
- Suppose a lab "fixes" the isiZulu gap by adding isiXhosa and isiZulu to its safety training. Has it solved the problem, or moved it? What does the Deng et al. resource-gradient finding predict?
- Apply the reader's checklist to one AI claim you've seen in the news this month. Which item does that claim fail hardest?
Session 1 summary and what's next
Technical AI safety separates misuse, misalignment and systemic risk. Its organising idea is the gap between rising capabilities and lagging safety. The case for it is a chain of inspectable claims with serious critics on the other side, best held as calibrated credences. And the isiZulu jailbreak shows all three at once: safety is unsolved, contingent rather than intrinsic, and unevenly distributed in a way that puts this continent on the exposed side. Above all: read critically, and hold beliefs at the confidence the evidence supports.
Next (Session 2): we rebuild the deep-learning foundations the rest of the course leans on, from neurons through the transformer to the scaling laws that make rising capability so predictable. It is the empirical engine behind the gap we have just described.