Note: This page's design and content were created and enhanced using Claude (Anthropic's AI assistant); the reading list has been reference-checked. Spot an error? Email jonathan.shock@uct.ac.za.
Week 4 • Session 7.5

Model specifications

The OpenAI Model Spec, specs versus constitutions, and the SpecEval adherence audit

What we'll cover

A model specification states intended behaviour and how to resolve conflicting instructions. We compare specifications with constitutions, examine an adherence audit, then write and test a short specification for a university assistant.

The governing document

Everything in this session so far has lived on the training side: reward models, constitutions, AI feedback, all mechanisms for pushing a model's behaviour somewhere. None of those mechanisms says where the behaviour is supposed to end up. A model specification fills that gap: a published, versioned document stating how a deployed model should behave, what it should aim at, which rules it must never break, and what it should do by default when nobody has said otherwise. The training methods of 7.1 to 7.3 are attempts to achieve some intended behaviour; a spec states the intended behaviour itself, in prose, where anyone can read it. It stands to alignment training roughly as a requirements document stands to code: the artefact you test the artefact against.

Publication, versioning and precision each matter separately. Publication turns a developer's intentions into a commitment that outsiders can test, which is what makes the SpecEval audit below possible at all. Versioning makes changes into public events: when a rule is added, renamed or weakened between versions, the diff is visible, and the sequence of versions becomes a record of how the developer's view of good behaviour has moved. Precision is the hard part. A spec earns nothing by listing virtues; its job is to resolve conflicts (a developer instruction against a user request, a helpful answer against an information hazard), and it does that with an explicit priority ordering and worked hard cases.

The OpenAI Model Spec

A versioned public document, first published in May 2024 and rewritten several times since, stating the behaviour OpenAI intends its models to have.

The first version organised its content into three categories: objectives, broad goals such as assisting the developer and end user and benefiting humanity; rules, hard constraints such as following the chain of command and complying with applicable laws; and defaults, behaviours such as asking clarifying questions, which hold unless someone with the authority to do so overrides them. Its chain of command ranked platform above developer above user above tool. The 18 August 2026 snapshot used here (dedicated to the public domain under CC0) arranges the material differently: after an overview, a definitions section and the chain of command, the substance is in behavioural sections named "Stay in bounds", "Seek the truth together", "Do the best work" and "Use appropriate style", followed by a set of under-18 principles. OpenAI states that it trains its models to align to the spec's principles, while acknowledging that production models do not yet fully reflect it: the document is the target, and the gap between target and model is an empirical quantity, which is the subject of the audit two sections down.

The 29 October 2026 version reorganises the guidance around faithfulness to human intent, harm minimisation, honesty and transparency, human agency, and care. It refers to GPT rather than the assistant and no longer has a separate section headed "Use appropriate style". The reading and structural description here use the 18 August snapshot.

Instruction levels in the 18 August snapshot

Every instruction the model receives carries an authority level, and higher levels win conflicts. The 18 August snapshot defines five: root rules, fixed in the spec itself and not overridable by system messages, developers or users; system rules, set by OpenAI and deliverable through system messages; developer instructions, from whoever builds on the API; user instructions, from the person in the conversation; and guidelines, defaults that can be overridden implicitly, for instance by context making clear the user wants something else. Earlier versions used a shorter ladder (platform, developer, user, tool); the splitting and renaming of levels across versions shows the document behaving like a versioned engineering artefact, with levels added as edge cases accumulate.

Three other structural features matter for the exercise below. The spec sorts content by permissibility: prohibited content, never produced under any instruction (sexual content involving minors); restricted content, produced only under tight conditions (information hazards, sensitive personal data); and sensitive content acceptable only in appropriate contexts. It also sorts the risks it guards against into three categories: misaligned goals, execution errors, and harmful instructions. And it argues by example. Most principles come with paired sample conversations, a compliant response next to a violating one, and the examples pin down interpretations the one-line principles leave open. When you read your chosen section for class, spend your time on the examples.

Specification versus constitution

A CAI constitution and a model spec are both normative documents written by a developer, and they are easy to conflate. A constitution (7.2) is a training-time instrument: its principles are sampled one at a time to steer critique-and-revise and to elicit AI preference labels, it acts on the model only through the training signal, and it is absent at deployment; 7.2 noted that the constitution never appears at the RL stage, having already been distilled into the reward model. A spec is a deployment-governance document: it states what the shipped system should do, it addresses users, developers and regulators as much as the training pipeline, and its authority does not depend on any particular training method. You could pursue a spec's behaviour with RLHF, with Constitutional AI, with supervised fine-tuning, or with a method not yet invented; the spec stays fixed while the methods compete. A constitution answers the question "what signal do we train with?"; a spec answers "what did we commit to?".

Anthropic’s 2026 rewrite of Claude’s constitution unsettles that division. It is published rather than internal, released under CC0, and written, in Anthropic’s words, “primarily for Claude”, yet it is plainly also readable by users and regulators; Anthropic treats it as “the final authority on how we want Claude to be and to behave”, with all other training and instruction required to be consistent with it. It carries the feature this page has been treating as the spec’s own: an explicit conflict rule. Four properties are ranked broadly safe, broadly ethical, compliant with Anthropic’s guidelines, genuinely helpful, and are to be prioritised in that order when they collide. Put that ranking beside the Model Spec’s chain of command and the useful question is not which document is a constitution, but which conflicts each one actually decides and which it leaves to the reader’s judgement.

Deliberative Alignment

Guan et al. (2024) narrow the gap between the two kinds of document. Deliberative Alignment teaches a reasoning model the text of the safety specifications directly and trains it to recall and reason over them explicitly, in its chain of thought, before answering; no human-written reasoning chains are needed, because the training data comes from models reasoning over the spec. Applied to OpenAI's o-series reasoning models, the method made them harder to jailbreak while reducing over-refusal, and improved generalisation to unfamiliar cases. The spec thereby stops being only a document addressed to humans: it becomes text the model itself has been taught and consults in explicit reasoning at the moment of answering, playing at inference time the role the constitution of 7.2 played only during training.

Auditing adherence

A published spec is a testable claim about the deployed model, and SpecEval is the first systematic test.

Providers publish behaviour specifications; they rarely verify, in public, that their models follow them. SpecEval (Ahmed, Klyman, Zeng, Koyejo and Liang, 2025) is an automated framework for auditing models against their own providers' published specifications. The pipeline has three stages: parse the provider's spec into individual behavioural statements; generate targeted prompts designed to probe each statement; and judge the model's responses for compliance, using the provider's own models as the judges. The standard it applies is three-way consistency, between the specification, the model's outputs, and the provider's models acting as evaluators. Across 16 models from six developers, evaluated against more than 100 behavioural statements, the audit finds systematic inconsistencies: judged by their own models, providers' models satisfy their own specifications in 64% (Google) to 81.6% (Anthropic) of cases, a gap of nearly 20 percentage points between providers.

Using the provider's own models as judges removes the defence that an outside auditor misread the spec: when a developer's own judge finds its own model breaking the developer's published document in roughly one case in five (Anthropic, 81.6% adherence) to one case in three (Google, 64%), the inconsistency is internal to the provider. Read the figures with 7.3 in mind. The judge is an LLM, so the audit inherits judge failure modes: the paper finds gemini-2.0-flash the most lenient judge and gpt-4.1 the strictest, and finds small self-favouring effects. The measured adherence rates, and the gap of nearly 20 percentage points between providers, could therefore overstate or understate true non-adherence. That uncertainty is itself the finding's sting: the same class of models whose judgement providers trust to generate training signal against these documents, in Constitutional AI, in RLAIF, in Deliberative Alignment, cannot cleanly certify adherence to them either.

Whose specification?

7.4 asked who writes the constitution, and the question transfers to the spec with its terms sharpened. A spec is written by the provider, in English, revised on the provider's schedule, and its chain of command is itself a statement about power: root and system rules sit with the model's developer, developer instructions with whoever builds the product, and the end user, the student in Gugulethu actually talking to the system, holds the lowest authority of any human issuing instructions. The spec's virtues are real and specific: it is fully public, versioned, and in OpenAI's case in the public domain, so contestation at least has a stable object, which is more than an unpublished reward model or a buried system prompt offers. What no current spec provides is a mechanism by which affected users, in any country, gain authority over its content. The tests 7.4 built (who sets the agenda, who holds a veto, which languages the judgements are reliable in) apply to a spec unchanged; bring them to class already applied.

In-class activity

Write a spec, then break one pairs

Before class: read the Model Spec's overview and its chain of command in full, then one behavioural section in depth (your choice), paying particular attention to how the worked examples resolve cases the bare principle leaves open.

In class: in pairs, write a mini-spec of about ten rules for a stated deployment: a study assistant for South African undergraduates, offered by a university. Your spec must state its chain of command (where do the university, the lecturer configuring a course, and the student rank, and what may each override?) and must resolve three hard cases, written out explicitly. For instance: a student asks for a full worked solution to a live marked assignment; a student discloses a mental-health crisis mid-session; a request arrives in isiZulu, a language in which the model's safety judgement is unreliable (7.4). For each case, give the behaviour your spec requires and the rule that requires it.

Swap: exchange specs with another pair and attack theirs: find prompts on which their rules conflict, fall silent, or produce behaviour they plainly did not intend. Every ambiguity you find in their document is one SpecEval's pipeline would have turned into a failed behavioural statement in yours. (Adapted from an exercise in Harvard's AI-safety course CS 2881r.)

Questions to bring to class

Readings

Mandatory readings

• OpenAI, "Model Spec" (version of 18 August 2026, model-spec.openai.com/2026-08-18.html): read the opening paragraphs of the Overview (≈300 words), then Instructions and levels of authority (≈700) and the short introduction to The chain of command (≈100) for the authority structure. Pick one behavioural section to read in depth for the in-class exercise (≈600); the Root principles listed under the chain of command are reference to consult, not reading to get through. ≈1,700 in all. The full snapshot is about 39,000 words. The site also retains earlier versions as dated snapshots.

• Ahmed, Klyman, Zeng, Koyejo & Liang (2025), "SpecEval: Evaluating Model Adherence to Behavior Specifications" (arXiv:2509.02464): read §3 for the three-stage audit pipeline and §4.2 for the adherence analysis and the gap of nearly 20 percentage points between providers. ≈1,000 words. Its appendices set out every statement audited from the specifications, with explanations and examples (the authors wrote their own for Anthropic and Google), and run to tens of thousands of words; they are a lookup table, not reading.

Total mandatory load: ≈2,700 words.

Optional readings

• Model Spec (18 August 2026): Red-line principles, General principles and Specific risks (≈700 words): the layer above the chain of command, read it if you want the whole authority structure rather than the part the exercise uses.

• SpecEval, §4.1 and §4.3 (≈900 words): the experimental setup, and the qualitative analysis of where models depart from their own specifications.

• Zvi Mowshowitz (2025), "On OpenAI's Model Spec 2.0" (thezvi.substack.com): a close critical reading; note that it reviews the February 2025 revision, an earlier version, so check its complaints against the snapshot used here.

• Guan et al. (2024), "Deliberative Alignment: Reasoning Enables Safer Language Models" (arXiv:2412.16339): training o-series models to recall and reason over the safety specifications before answering.

Next

Sub-session 7.6 is the lab, in Colab. You run critique-and-revise, then measure a small AI judge's sensitivity to written principles and answer order. Distinguish conditional label probabilities from hard choices, then compare English identity matching with isiZulu-to-English translation matching. Each result concerns this model and evaluation setup.