Model specifications

The governing document

Everything in this session so far has lived on the training side: reward models, constitutions, AI feedback, all mechanisms for pushing a model's behaviour somewhere. None of those mechanisms says where the behaviour is supposed to end up. A model specification fills that gap: a published, versioned document stating how a deployed model should behave, what it should aim at, which rules it must never break, and what it should do by default when nobody has said otherwise. The training methods of 7.1 to 7.3 are attempts to achieve some intended behaviour; a spec states the intended behaviour itself, in prose, where anyone can read it. It stands to alignment training roughly as a requirements document stands to code: the artefact you test the artefact against.

Publication, versioning and precision each matter separately. Publication turns a developer's intentions into a commitment that outsiders can test, which is what makes the SpecEval audit below possible at all. Versioning makes changes into public events: when a rule is added, renamed or weakened between versions, the diff is visible, and the sequence of versions becomes a record of how the developer's view of good behaviour has moved. Precision is the hard part. A spec earns nothing by listing virtues; its job is to resolve conflicts (a developer instruction against a user request, a helpful answer against an information hazard), and it does that with an explicit priority ordering and worked hard cases.

The OpenAI Model Spec

A versioned public document, first published in May 2024 and rewritten several times since, stating the behaviour OpenAI intends its models to have.

The first version organised its content into three categories: objectives, broad goals such as assisting the developer and end user and benefiting humanity; rules, hard constraints such as following the chain of command and complying with applicable laws; and defaults, behaviours such as asking clarifying questions, which hold unless someone with the authority to do so overrides them. Its chain of command ranked platform above developer above user above tool. The version current as this page is written (dated 18 December 2025, dedicated to the public domain under CC0) arranges the material differently: after an overview, a definitions section and the chain of command, the substance sits in behavioural sections named "Stay in bounds", "Seek the truth together", "Do the best work" and "Use appropriate style", followed by a set of under-18 principles. OpenAI states that it trains its models to align to the spec's principles, while acknowledging that production models do not yet fully reflect it: the document is the target, and the gap between target and model is an empirical quantity, which is the subject of the audit two sections down.

Three other structural features matter for the exercise below. The spec sorts content by permissibility: prohibited content, never produced under any instruction (sexual content involving minors); restricted content, produced only under tight conditions (information hazards, sensitive personal data); and sensitive content acceptable only in appropriate contexts. It also sorts the risks it guards against into three categories: misaligned goals, execution errors, and harmful instructions. And it argues by example. Most principles come with paired sample conversations, a compliant response next to a violating one, and the examples pin down interpretations the one-line principles leave open. When you read your chosen section for class, spend your time on the examples.

Specification versus constitution

A CAI constitution and a model spec are both short normative documents written by a developer, and they are easy to conflate. A constitution (7.2) is a training-time instrument: its principles are sampled one at a time to steer critique-and-revise and to elicit AI preference labels, it acts on the model only through the training signal, and it is absent at deployment; 7.2 noted that the constitution never appears at the RL stage, having already been distilled into the reward model. A spec is a deployment-governance document: it states what the shipped system should do, it addresses users, developers and regulators as much as the training pipeline, and its authority does not depend on any particular training method. You could pursue a spec's behaviour with RLHF, with Constitutional AI, with supervised fine-tuning, or with a method not yet invented; the spec stays fixed while the methods compete. A constitution answers the question "what signal do we train with?"; a spec answers "what did we commit to?".

Anthropic’s 2026 rewrite of Claude’s constitution unsettles that division. It is published rather than internal, released under CC0, and written, in Anthropic’s words, “primarily for Claude”, yet it is plainly also readable by users and regulators; Anthropic treats it as “the final authority on how we want Claude to be and to behave”, with all other training and instruction required to be consistent with it. It carries the feature this page has been treating as the spec’s own: an explicit conflict rule. Four properties are ranked broadly safe, broadly ethical, compliant with Anthropic’s guidelines, genuinely helpful, and are to be prioritised in that order when they collide. Put that ranking beside the Model Spec’s chain of command and the useful question is not which document is a constitution, but which conflicts each one actually decides and which it leaves to the reader’s judgement.

Auditing adherence

A published spec is a testable claim about the deployed model, and SpecEval is the first systematic test.

Providers publish behaviour specifications; they rarely verify, in public, that their models follow them. SpecEval (Ahmed, Klyman, Zeng, Koyejo and Liang, 2025) is an automated framework for auditing models against their own providers' published specifications. The pipeline has three stages: parse the provider's spec into individual behavioural statements; generate targeted prompts designed to probe each statement; and judge the model's responses for compliance, using the provider's own models as the judges. The standard it applies is three-way consistency, between the specification, the model's outputs, and the provider's models acting as evaluators. Across 16 models from six developers, evaluated against more than 100 behavioural statements, the audit finds systematic inconsistencies, with compliance gaps of up to 20%.

Using the provider's own models as judges removes the defence that an outside auditor misread the spec: when a developer's published document, its deployed model and its own model-as-judge disagree at rates up to one case in five, the inconsistency is internal to the provider. Read the figure with 7.3 in mind. The judge is an LLM, so the audit inherits the judge failure modes catalogued there (position bias, and a pull toward whatever the judge already finds fluent), and a measured 20% gap could overstate or understate true non-adherence. That uncertainty is itself the finding's sting: the same class of models whose judgement providers trust to generate training signal against these documents, in Constitutional AI, in RLAIF, in Deliberative Alignment, cannot cleanly certify adherence to them either.

Whose specification?

7.4 asked who writes the constitution, and the question transfers to the spec with its terms sharpened. A spec is written by the provider, in English, revised on the provider's schedule, and its chain of command is itself a statement about power: root and system rules sit with the model's developer, developer instructions with whoever builds the product, and the end user, the student in Gugulethu actually talking to the system, holds the lowest authority of any human issuing instructions. The spec's virtues are real and specific: it is fully public, versioned, and in OpenAI's case in the public domain, so contestation at least has a stable object, which is more than an unpublished reward model or a buried system prompt offers. What no current spec provides is a mechanism by which affected users, in any country, gain authority over its content. The tests 7.4 built (who sets the agenda, who holds a veto, which languages the judgements are reliable in) apply to a spec unchanged; bring them to class already applied.

In-class activity

Questions to bring to class

• The chain of command ranks the user below OpenAI and below the developer. Construct a case where obeying the developer over the user harms the user, and write the root-level rule you would add to prevent it.
• A constitution acts during training; a deliberatively aligned model reasons over spec text it was taught. When the provider revises the document, what must happen before each system's behaviour actually changes, and what does that imply about which errors each approach can correct quickly?
• SpecEval measures compliance with the provider's own models as judges. Using the judge failure modes from 7.3, give one reason the 20% figure overstates true non-adherence and one reason it understates it.
• If the spec says one thing and the deployed model reliably does another, which one counts as the model's values? What would an audit have to show before you trusted the document over the behaviour?

Readings