The question
If a model sees [A][B] … [A] and predicts [B], what has it learned? An output test tells us that the completion works on the tested sequences. A circuit account tries to show which parts of the model carry out the matching and copying. Induction heads give us a small, concrete case on which to practise judging that evidence.
The two-step algorithm
- Mark the previous token. An early attention head writes information about the token before each position into the residual stream, the shared vector that the model’s layers read and update.
- Match and copy. A later head can use that information to find an earlier occurrence of the current token. Its attention points to the token that followed the earlier occurrence, and its output raises the probability of copying that token.
For [cat][sat] … [cat], the second head should attend to the earlier [sat] position and favour sat as the next token. This is the induction pattern. The source paper develops the QK matching and OV copying matrices in detail; you do not need the full derivation to evaluate the central claim.
What would count as evidence?
First, show the behaviour on repeated sequences that were not used to select the head. Second, inspect the attention pattern: does it point to the token after the matching prefix? Third, intervene. Remove or replace the candidate head and measure whether the specific repeat-completion advantage falls. Repeat with a comparison head and with sequences that should not need the pattern. A colourful attention plot alone meets only the second of these checks.
The inference ladder
| Observation | Claim it supports | What it does not establish |
|---|---|---|
| The model completes repeated sequences | The tested behaviour exists | Which mechanism produced it |
| A head attends at the induction position | A plausible candidate mechanism | That the head is necessary for the behaviour |
| A targeted ablation reduces repeat completion | A causal contribution in this model and test distribution | A complete account of in-context learning, or a frontier-scale safety guarantee |
The safety limit
A verified circuit in a small model does not show that a large model is transparent, that its other circuits have been found, or that a deceptive model cannot evade an audit. The useful habit is to specify the model, input distribution, intervention and remaining alternative explanations before using a mechanistic result in a safety argument.
In-class exercise
In pairs, design an ablation test for the induction claim. State the outcome measure and one control that would distinguish a specific loss of repeat copying from general damage to the model. Then make two sentences about the result: the strongest claim a positive test would license, and a stronger claim it would leave open.
From the example to the project
Interpretability is an optional project route for students ready to do additional practical preparation with the instructor. A project on a circuit needs a baseline, an intervention and a held-out prompt distribution. The course project also welcomes narrower work based on the robustness experiment, evaluation audit or governance exercise. Bring one question and one possible disconfirming result to the project launch.
Readings and optional depth
Already assigned for Session 13
• Return to the bounded Olsson et al. excerpt assigned on the preceding page. The link there opens the full article; read only the named sections. This worked example adds no new pre-class reading.
For independent study
• The full Olsson et al. paper develops the induction-head account. It is optional and is not a prerequisite for the project.