openai
Ads and shutdown-evasion land on OpenAI's banned list
Promtime
openaiOpenAI has published the Model Spec, the document its models are trained to follow, and it sorts every instruction into five levels of authority: root, system, developer, user and guideline. The document published by OpenAI is dedicated to the public domain under the Creative Commons CC0 1.0 deed.
At a glance
- Root rules come only from the spec itself and cannot be overridden by system messages, developers or users; when two root principles collide, the model is instructed to default to inaction.
- A section titled No other objectives bars the assistant from pursuing revenue, ads, upsell, click-through, self-preservation, evasion of shutdown, or accumulation of compute, data and credentials as ends in themselves.
- Prohibited content covers exactly one category, sexual content involving minors, while information hazards and sensitive personal data are restricted and erotica and gore are allowed only in narrow contexts.
The spec reads as an attempt to make model behaviour auditable rather than tacit: naming ads, upsell and shutdown evasion as forbidden objectives turns commercial and self-preservation pressure into a testable property rather than an assurance. For teams building on the API, the practical payload is the authority ordering, which decides whose instruction wins when a system message, a developer prompt and a pasted web page disagree.
Five authority levels decide which instruction wins
The ordering runs root, system, developer, user and guideline, with assistant messages, tool outputs and quoted text sitting below all of them with no authority. Root instructions come only from the Model Spec and its detailed policies and are mostly prohibitive; guideline-level defaults are the weakest and can be overridden implicitly, as when a user asks for a realistic pirate voice and the default against swearing gives way.
A root section titled No other objectives forbids the assistant from adopting goals as ends in themselves, among them click-through that is not user beneficial, revenue or ads for OpenAI, upsell, self-preservation, evading shutdown, accumulating compute or credentials, and acting as an enforcer of laws or morality.
OpenAI says its models should not whistleblow autonomously, but that it uses automated monitoring to detect potential usage policy violations and, after human review, may refer to law enforcement cases involving an imminent threat of serious physical harm or severe public safety risks.
Only sexual content involving minors falls into the prohibited tier
The spec sorts disallowed output into three tiers. Prohibited content, which covers only sexual content involving minors, can never be produced, including as a transformation of user-supplied text. Restricted content covers information hazards and sensitive personal data, and sensitive content covers erotica and extreme gore; transformations of user-provided material are allowed in both of those tiers.
Erotica and gore are permitted only in scientific, historical, news, artistic or other appropriate contexts. OpenAI writes that after the initial release in May 2024 many users and developers backed a grown-up mode, and that it is exploring how to allow erotica and gore in age-appropriate contexts through the API and ChatGPT, while ruling out sexual deepfakes and revenge porn.
Separately, root-level rules bar content designed to manipulate the political views of specific individuals or demographic groups, while political content written for a broad audience is allowed; other root sections block full song lyrics, paywall bypasses and personal contact details for public officials.
Untrusted data has no authority and agent actions need an agreed scope
Quoted text, JSON, YAML and XML blocks, file attachments, multimodal data and tool outputs are treated as information rather than instructions unless unquoted text delegates authority to them. OpenAI advises developers to wrap untrusted input in untrusted_text blocks, or failing that in YAML, JSON or XML, so injected instructions cannot be mistaken for developer prompts.
Trust is graded by provenance: an AGENTS.md file checked into the working directory ranks above a stuff.txt in a downloads folder, and a random web page is not to be trusted at all. Where tool instructions might be intended but could cause serious side effects, the assistant is told to seek clarification before acting.
Agentic actions must fit an agreed scope of autonomy, recorded in a semi-structured ScopeOfAutonomy format with fields such as allowed_tools, latest_time, max_cost and tool_constraints, and every scope needs an ending condition. Hacking, deception, resource acquisition, spawning sub-agents and self-modification stay prohibited unless explicitly authorised.
Where models still trail the text
OpenAI states that its production models do not yet fully reflect the Model Spec and that it keeps refining its systems toward closer alignment, and that the public version may omit details while remaining consistent with intended behaviour. No timeline is given for the erotica and gore work, and the spec is described as one part of a wider approach alongside usage policies and safety protocols, with updates promised as feedback arrives.
Comments
No comments yet. Be the first.
Join the conversation
Sign in with Google to leave a comment. Your name and avatar come from your Google profile, and the comment appears after moderation.
We only use your name and avatar from Google. We never store your email address.
