The task
Work in a team of two or three. Produce a short paper (6–8 pages), the code or analysis needed to reproduce its central result, and a brief presentation. A well-explained negative or unresolved result is welcome. The project is worth 40% of the course mark; the marking categories are below.
Define a testable question
Write one claim, the measurement or argument that could count against it, and the smallest first result. State the model, data, comparison and resources before committing. A project must be feasible with the compute and access the team actually has. Do not rely on an unavailable API, an unreviewed translation or a large GPU.
Proposal, due at the end of the launch week
Submit one page containing: question and possible disconfirming result; method and baseline; data or sources; primary outcome; compute and access plan; first-week milestone; main validity risk; and a short context statement explaining who the work helps and whether an African context is relevant. “Not applicable, because…” is acceptable when justified.
Starting points from the taught route
- Multilingual robustness. Extend Session 9.4 with a new, predeclared prompt set or a simple defence. Keep comprehension and translation checks. Compare matched prompts, and specify what result would show that the defence failed.
- Evaluation validity. Extend the Session 11.5 audit: test a second scoring rule, obtain fluent-speaker review, or compare a second open model. Ask whether the original conclusion survives. An audit that finds the comparison cannot be made is a result.
- Governance. Take one claim in a frontier-safety framework from Session 12. Trace the eval, threshold and promised action, then specify what an outside auditor could actually verify. A conceptual project needs a defined comparison and evidence from the primary policy documents.
- Interpretability, with prior preparation. Complete an induction-head experiment with instructor guidance first. Then test whether a candidate head's effect survives a new prompt distribution and an appropriate comparison ablation. The Session 13 example alone is not enough preparation for an interpretability experiment.
Project schedule
- Launch week: agree the proposal and get the data or model access working.
- First work week: bring one baseline result to a clinic. Cut the scope if the first result is blocked.
- Middle: check the primary result, error bars and validity; decide whether to continue, pivot or report an informative negative result.
- Final week: write the paper, release reproducible code or analysis where appropriate, present the claim and limitations, and review other teams' work.
The schedule is relative to the actual project start. The instructor will set calendar dates for the proposal, clinic, submission and presentations.
Project rubric
| Section | Evidence to show |
|---|---|
| Abstract | The specific question, result and main limit. |
| Method | Enough detail to reproduce the analysis: data, model or sources, comparison, metric and code. |
| Experiments or analysis | A baseline or counterargument, a clear result, and uncertainty where the evidence is numerical. |
| Limitations | What the result does not establish and what would change the conclusion. |
| Creativity | A useful question or a justified new use of an existing method. |
| Positioning and context | How the work relates to prior work and whom it concerns, including a specific, justified account of its African-context relevance or non-relevance. |
Limits to address
State how the tested distribution, access to models or data, and measurement choices limit the claim. Explain what evidence would change your conclusion. A small result that survives these checks is enough.
Before leaving the launch
Each team should leave the launch with a one-sentence question, a baseline, a first action that takes less than a week, and a result that would cause it to change course.