OpenAI Shifts Risk Assessment Toward Deployment Simulation

OpenAI has introduced a method called Deployment Simulation designed to predict how candidate models will behave in real-world settings before they are released to the public. According to a June 16, 2026, announcement from OpenAI, the process involves taking recent conversations from deployment, removing the original assistant responses, and regenerating them using a new candidate model.

This approach marks a shift from traditional pre-deployment evaluations, which OpenAI reports typically rely on synthetic, manually written, or production prompts specifically selected to be adversarial or high-severity. While OpenAI notes that traditional evaluations remain essential for stress-testing low-probability, high-severity risks, Deployment Simulation is intended to improve the estimation of undesired behavior frequencies and surface novel forms of misalignment.

OpenAI reports that the simulation method addresses three primary limitations of traditional testing:

* **Coverage:** Rather than relying on the manual effort to create new evaluations, the quality of risk assessment now scales with compute. By simulating more traffic, the lab can achieve greater coverage of undesirable behaviors. * **Selection Bias:** Because the method uses a distribution of prompts representative of recent usage, it reduces the bias found in evaluations built around previously known harms. * **Evaluation Awareness:** OpenAI states that models are increasingly able to recognize when they are being tested, which can distort safety measurements. Deployment Simulation mitigates this, as models do not appear able to distinguish simulated conversations from real deployment traffic.

OpenAI has already applied this method to several GPT-5-series Thinking deployments and complex agentic rollouts involving tool use. The company noted that the simulation can also be used for risk assessment prior to internal model deployments.

Despite these advancements, OpenAI acknowledged limitations regarding the frequency of detected behaviors. In its experiments, the company stated that the approach is not expected to measure behaviors that occur with a frequency of less than 1 in 200,000 messages.