MAM5020F — Generative AI for Research
Critical Evaluation & Limitations of AI
Benchmarks, failure categories, and where AI is now genuinely strong
Contents
15 papers spanning benchmark contamination, structural failure modes (the reversal curse, sycophancy, hallucination), and recent cases of AI contributing to genuine mathematical and physical discovery.
Each entry links to the canonical version of the paper, on arXiv, the journal, or the publisher. Where a paper is paywalled, the DOI is given for UCT-library access.
9.1 · The Trajectory of LLM Capabilities
Investigating Data Contamination in Modern Benchmarks for Large Language Models
LatestEval — dynamic test construction to avoid data contamination
Bridging the Gap: Enhancing LLM Performance for Low-Resource African Languages with New Benchmarks, Fine-Tuning, and Cultural Adjustments
IrokoBench: a benchmark for African languages
ProgramBench: Can Language Models Rebuild Programs From Scratch?
9.2 · Three Categories of Failure
The Reversal Curse: LLMs Trained on “A is B” Fail to Learn “B is A”
Mathematical Capabilities of ChatGPT
Towards Understanding Sycophancy in Language Models
Why Language Models Hallucinate
9.3 · Where AI Is Now Genuinely Strong
AI-Newton: A Concept-Driven Physical Law Discovery System without Prior Physical Knowledge
Mathematical exploration and discovery at scale
Single-minus gluon tree amplitudes
Solving an Open Problem in Theoretical Physics using AI
Primitive sets and von Mangoldt chains: Erdős Problem #1196 and beyond
AI Co-Mathematician: Accelerating Mathematicians with Agentic AI
9.4 · Illusions of Understanding
Linked but not redistributed
Messeri, L. & Crockett, M. J. (2024). Artificial intelligence and illusions of understanding in scientific research. Nature 627, 49–58. DOI:10.1038/s41586-024-07146-0 9.4
Subscription article: the abstract is free to read; the full text needs institutional access (UCT library, through the DOI).
9.5 · Verification Protocols for a Moving Target
Original protocol/workflow content.
9.6 · Hands-On Activities and Assessment
Assessment design.
Drafted with Claude (Anthropic) and reviewed by Jonathan Shock before publication. AI-generated errors are possible — if you spot one, please email jonathan.shock@uct.ac.za. Full detail: AI Content Disclaimer.
© 2026 Jonathan Shock · MAM5020F: Generative AI for Research · CC BY 4.0