← All lessons
Week 1 · Foundations of Generative AI
MAM5020F — Generative AI for Research

Transformers and Diffusion models

Extended Video Collection - 3Blue1Brown and More
Contents
← All lessons
Week 1 · Foundations of Generative AI
Key terms
Transformer
A neural network architecture introduced in 2017 that processes whole sequences of text at once and uses attention to weigh how relevant each word is to every other. It underpins virtually all modern large language models.
Attention mechanism
A component of transformer models that lets the system weigh, for each word it processes, which other words in the text are most relevant, rather than treating all context as equally important.
Diffusion model
A generative AI that learns to create images by reversing a process of gradually adding noise: starting from random static, it progressively refines an image guided by a text prompt. Stable Diffusion works this way.

This collection extends the neural networks series with additional videos covering advanced topics in deep learning, transformers, attention mechanisms, and generative AI. Watch these to deepen your understanding of how modern AI systems work.

← PreviousUnderstanding How Generative AI Works

Drafted with Claude (Anthropic) and reviewed by Jonathan Shock before publication. AI-generated errors are possible — if you spot one, please email jonathan.shock@uct.ac.za. Full detail: AI Content Disclaimer.

© 2026 Jonathan Shock · MAM5020F: Generative AI for Research · CC BY 4.0

Week 1
Lesson 6
1 min read
Key terms
Transformer
A neural network architecture introduced in 2017 that processes whole sequences of text at once and uses attention to weigh how relevant each word is to every other. It underpins virtually all modern large language models.
Attention mechanism
A component of transformer models that lets the system weigh, for each word it processes, which other words in the text are most relevant, rather than treating all context as equally important.
Diffusion model
A generative AI that learns to create images by reversing a process of gradually adding noise: starting from random static, it progressively refines an image guided by a text prompt. Stable Diffusion works this way.