From base model to assistant
This is a read-only rendering. To run it: open in Google Colab, or download the .ipynb and upload it yourself.
The cells have no saved outputs: you run them and keep your own. Back to the lab page.
African Technical AI Safety · Week 3, Session 5.4
Sessions 5.1 to 5.3 described a pipeline: pretraining makes a capable base model, supervised fine-tuning turns it into something that follows instructions, and later stages shape it further. This lab does two things with that. First you find those stages in a real, readable codebase. Then you put a base model and its instruction-tuned sibling side by side and work out what the tuning changed.
The usual account of that second part is that base models ramble and tuned models answer. Run it before you decide whether that is what happens.
What you submit: this notebook run end to end, with your answers in the two Explore cells.
import importlib.util, sys, textwrap, re, urllib.request
IN_COLAB = 'google.colab' in sys.modules
if importlib.util.find_spec('transformers') is None:
%pip install -q transformers
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
torch.set_num_threads(2)
print('Colab:', IN_COLAB)
# Record the environment used for this run; results can vary across versions.
import platform, importlib.metadata, os
print('Python:', platform.python_version(), '| CPUs:', os.cpu_count())
for package in ('torch', 'transformers', 'transformer_lens', 'numpy', 'pandas'):
try:
print(package, importlib.metadata.version(package))
except importlib.metadata.PackageNotFoundError:
pass
try:
import psutil
print('RAM GB:', round(psutil.virtual_memory().total / 1e9, 1))
except ImportError:
pass
print('GPU available:', __import__('torch').cuda.is_available())
Andrej Karpathy's nanochat is a complete ChatGPT-style pipeline written to be read: tokenisation, pretraining, supervised fine-tuning, reinforcement learning, inference. Rather than hunting through it by hand, the cell below fetches the three scripts that matter and finds the line in each where the stage's objective lives.
The cell searches for patterns instead of quoting line numbers: nanochat has tens of thousands of stars and is actively developed, so any line number printed here would be wrong within weeks.
SCRIPTS = 'https://raw.githubusercontent.com/karpathy/nanochat/master/scripts/'
SOURCES = {} # kept so you can read a file when a search misses
def find_in(script, patterns, context=1):
try:
src = urllib.request.urlopen(SCRIPTS + script).read().decode()
except Exception as e:
print(f'could not fetch {script}: {e}')
print('Read it on GitHub instead: ' + SCRIPTS + script)
return
SOURCES[script] = src
lines = src.split('\n')
print(f'=== {script} ({len(lines)} lines)')
for pat in patterns:
for i, line in enumerate(lines):
if re.search(pat, line):
for j in range(max(0, i - context), min(len(lines), i + context + 1)):
mark = '>' if j == i else ' '
print(f' {mark} {j+1:>4} {lines[j].strip()[:96]}')
print()
break
else:
print(f' (no line matching {pat!r} any more: the repo has moved on)\n')
# 5.1, pretraining: the whole objective, and where its targets come from
find_in('base_train.py', [r'loss = model\(x, y\)', r'= next\(train_loader\)'])
# 5.2, supervised fine-tuning: the same loss, masked to the assistant's tokens only
find_in('chat_sft.py', [r'Apply the loss mask from render_conversation',
r'targets\[mask_targets == 0\] = -1'], context=2)
# 5.3, reinforcement learning: where the training signal comes from instead
find_in('chat_rl.py', [r'reward = train_task\.reward', r'advantages = '], context=2)
nanochat ships scripts/chat_rl.py, which runs GRPO on GSM8K, described in its own docstring as
close to plain REINFORCE once the trust region and the PPO clipping come off. That is the
verifiable-rewards stage from 5.3 in about 300 lines, and its reward comes from a checker rather
than from a human or a learned model.
Write three or four sentences mapping what you found onto the five-stage table from 5.3. Which stages are present in this repository, which are missing, and what does each one's objective look like in code?
Your answer:
Now the experiment. Qwen2.5-0.5B and Qwen2.5-0.5B-Instruct share an architecture, a size, a
tokeniser and a pretraining run. The second one has been through supervised fine-tuning and
preference tuning; the first has not. Anything that differs between them is what the tuning did.
The two take their input differently. The base model gets raw text and continues it. The instruct model expects a chat template: special tokens marking who is speaking, which its tuning taught it to expect.
Loading both takes a minute or two, including about 1.9 GB of downloads. In the 11 September 2026 standard-CPU Colab check, the Python process peaked at about 5.5 GiB of RAM across loading and generation. Leave room for the notebook runtime as well as the model weights.
BASE, INSTRUCT = 'Qwen/Qwen2.5-0.5B', 'Qwen/Qwen2.5-0.5B-Instruct'
tok_b = AutoTokenizer.from_pretrained(BASE)
mod_b = AutoModelForCausalLM.from_pretrained(BASE, dtype=torch.float32).eval()
tok_i = AutoTokenizer.from_pretrained(INSTRUCT)
mod_i = AutoModelForCausalLM.from_pretrained(INSTRUCT, dtype=torch.float32).eval()
CAP = 64
def ask_base(prompt):
"""Raw continuation: the base model just carries on from the text."""
ids = tok_b(prompt, return_tensors='pt')
out = mod_b.generate(**ids, max_new_tokens=CAP, do_sample=False, repetition_penalty=1.0,
pad_token_id=tok_b.eos_token_id)
new = out[0][ids['input_ids'].shape[1]:]
e = mod_b.generation_config.eos_token_id
stops = {tok_b.eos_token_id} | set(e if isinstance(e, list) else [e])
return tok_b.decode(new, skip_special_tokens=True).strip(), bool(stops & set(new.tolist()))
def ask_instruct(prompt):
"""Chat template: the tuning taught it this shape."""
text = tok_i.apply_chat_template([{'role': 'user', 'content': prompt}],
tokenize=False, add_generation_prompt=True)
ids = tok_i(text, return_tensors='pt')
out = mod_i.generate(**ids, max_new_tokens=CAP, do_sample=False, repetition_penalty=1.0,
pad_token_id=tok_i.eos_token_id)
new = out[0][ids['input_ids'].shape[1]:]
e = mod_i.generation_config.eos_token_id
stops = {tok_i.eos_token_id} | set(e if isinstance(e, list) else [e])
return tok_i.decode(new, skip_special_tokens=True).strip(), bool(stops & set(new.tolist()))
print('what the chat template actually looks like:\n')
print(tok_i.apply_chat_template([{'role': 'user', 'content': 'Hello.'}],
tokenize=False, add_generation_prompt=True))
print('Model revisions:', mod_b.config._commit_hash, mod_i.config._commit_hash)
PROMPTS = ['Explain what a neural network is, in two sentences.',
'Summarise the plot of Romeo and Juliet.',
'What is the capital of Kenya? Answer with one word.',
"Translate 'good morning' into isiZulu.",
'Give me three reasons to exercise.']
# Greedy decoding avoids sampling noise; outputs can still vary with library/model versions.
results = []
for p in PROMPTS:
b, b_stop = ask_base(p)
i, i_stop = ask_instruct(p)
results.append((p, b, b_stop, i, i_stop))
print('=' * 100)
print(f'PROMPT {p}')
print('BASE ', b)
print('INSTRUCT ', i)
Read your outputs before the discussion below. Record differences in answer content, style, refusal and stopping. Check factual answers independently, including Nairobi and the play’s setting. If you do not read isiZulu, leave translation correctness unscored until someone who does can assess it. A confident answer is not evidence of a correct translation.
The discussion records an earlier run. Your model or software version may behave differently; report the outputs you obtained and the settings that produced them.
# substring matches: '1.' will also catch a decimal, 'Sure' will catch 'Surely'
MARKERS = ['Sure', '**', '1.', 'I cannot', "I'm sorry", 'helpful assistant']
print(f"{'prompt':<40}{'stops? base':>12}{'inst':>7} markers: base / instruct")
for p, b, bs, i, is_ in results:
mb = [m for m in MARKERS if m in b]
mi = [m for m in MARKERS if m in i]
print(f'{p[:38]:<40}{str(bs):>12}{str(is_):>7} {mb} / {mi}')
print()
print('stops early: base', sum(r[2] for r in results), '/', len(results),
' instruct', sum(r[4] for r in results), '/', len(results))
Feed the base model the chat template. Run the following after the model-loading cells:
msg = [{"role": "user", "content": PROMPTS[0]}]
formatted = tok_i.apply_chat_template(msg, tokenize=False, add_generation_prompt=True)
print(ask_base(formatted))
tokenize=False returns a string, and add_generation_prompt=True adds the assistant-turn prefix. Does the
format alone produce assistant-shaped behaviour, or does the shape need the tuning behind it?
Change CAP to 200 and re-run the Romeo and Juliet prompt on both. Does the base model
eventually stop, or keep going?
Your own prompts. Try one where you know the answer and one in a language you speak. The isiZulu answer above is delivered with total confidence; find out whether it is right.
Question. Two or three sentences. Sort the differences you found into form (how the answer is presented) and substance (what the model knows or gets right). Which did fine-tuning change? A model that has been through this pipeline is often called "aligned": on this evidence, what does that word license you to assume, and what does it not?
Your answer:
Answer the Explore question above before reading on. This section works through the same sort, so reading it first does the work for you.
These are observations from an earlier run, not required outcomes for your submission.
Only the tuned model opens with "Sure," and produces a refusal. Neither reaches for markdown bold, which is worth knowing: the tuned model does produce it under its shipped generation config, which sets a repetition penalty of 1.1, and the effect disappears once both models are decoded identically. Both models number their lists, and stopping early is close to a tie. The willingness to end a turn, which sounds like the obvious thing fine-tuning buys, is not visible here at this model size.
Knowledge went the other way. The base model says Nairobi; the tuned model says Kampala, which is the capital of Uganda. The tuned model also refuses to summarise one of the most widely reproduced plays in the language, asserting that it cannot find any such text, while the model it was built from does summarise it, though not accurately: it places the action in "the early 1590s", which is roughly when the play was written rather than when it is set. Both are ways of getting worse, and both survived a pipeline whose purpose was to make the model more useful.
Now look again at the end of the base model’s isiZulu attempt. It slides into “You are a helpful assistant, who always provide explanation.” Nothing in the prompt asked for that. It resembles a system instruction. This output shows that the base model can produce assistant-style text. It does not establish that this particular sentence was memorised from its training data. On these prompts, answering questions was not unique to the tuned model. The comparison shows changes in the responses elicited under these settings; broader capability claims need more tasks and controls.
When a later session says a model was “aligned” by fine-tuning, ask what moved: the behaviour, or the capability underneath it. Session 9.3 pushes that question further, and 9.4 has you measure the gap in isiZulu.
In Colab, File → Download → Download .ipynb. Graded on completion and the quality of your observations; resubmission is allowed.
Sources. nanochat (Karpathy, MIT). Qwen2.5-0.5B and its Instruct sibling, from the Qwen2.5 release. The five-stage pipeline is Session 5.3.