OpenAI found that enabling retained reasoning and compaction in its Responses API lifted GPT-5.6 Sol's ARC-AGI-3 benchmark score from 13.3% to 38.3% while cutting output tokens by 6x, with implications worth testing for teams configuring AI agents in long-running workflows.
Back to NewsOpenAI published findings on July 29, 2026 showing that GPT-5.6 Sol scored just 13.3% on the ARC-AGI-3 public benchmark under the official evaluation harness. After enabling two settings in its Responses API, the same model scored 38.3%, nearly three times higher, while using six times fewer output tokens.
The two settings are retained reasoning and compaction. Both are API configuration choices, not model changes.
ARC-AGI-3 measures how well AI agents learn and reason through unfamiliar 2D games without explicit instructions. The official harness is intentionally generic, without tools or special features. ARC's stated rationale is that a simple harness makes model shortcomings more visible and comparisons more fair.
That design choice created a specific problem for GPT-5.6 Sol. The harness discarded all private reasoning after each game action. The model could see a record of past moves and brief accompanying notes, but not the plans, insights, or thoughts that led to them. With every turn, GPT-5.6 Sol was asked to figure out the game anew, unable to remember its past thinking.
The harness also used a rolling truncation window. When context exceeded 175,000 characters, the oldest messages were dropped. OpenAI's own implementation used a limit of 175,000 tokens instead, which proved similar in practice because the vast majority of text is action grids, which tokenize at a 1:1 ratio. So the model was simultaneously losing memory of its reasoning and, eventually, its action history.
OpenAI's analysis concluded that much of the model's confusion came from the harness settings, not from inherent model limits.
OpenAI's models are trained to think using private reasoning messages before producing replies or tool calls. Those messages are retained in conversation history by design. That is also how the models run in ChatGPT and Codex.
In the Responses API, for GPT-5.6, passing the previous response ID automatically retains reasoning across tool calls and turns. In the benchmark context, GPT-5.6 Sol spent less time deliberating before each action with that setting active. It could build on earlier conclusions instead of reconstructing them.
Compaction replaces rolling truncation. OpenAI describes summarization as how its models are trained and deployed -- if a conversation grows too long, the model summarizes it and continues. That trained behavior is what the compaction setting in the Responses API activates. With compaction enabled, GPT-5.6 Sol was better able to preserve what it had learned about each game across longer runs, achieving a higher score with fewer output tokens.
The score difference is stark. On one specific puzzle game, no frontier model solves any level beyond the first on the official leaderboard for this game. With retained reasoning and compaction, GPT-5.6 Sol solves all six. ARC-AGI-3 scores measure Relative Human Action Efficiency, a metric comparing model performance to a human baseline. OpenAI estimates the average human tester scored 48% on the same tasks.
This finding is not limited to benchmarks. Metrotechs analysis: any integration that invokes a reasoning model across multiple steps faces the same structural choice. Does the agent carry its thinking forward, or does it start fresh each time?
Metrotechs analysis: consider a reasoning model working through a multi-step process such as order triage, document review, support escalation, or code generation. Discarding reasoning between steps may impose the same penalty OpenAI measured on ARC-AGI-3. S1 establishes the effect in the benchmark context. Whether it extends to enterprise workflows is consistent with the mechanism but not independently confirmed in the available source.
Metrotechs analysis: the token-efficiency finding matters separately. A 6x reduction in output tokens may translate to meaningful cost reduction at scale, if token pricing applies to these workloads. The source does not discuss pricing, so teams should model their own call volumes. The directional signal is clear: better configuration produced both better output and lower compute use on the ARC-AGI-3 public task set.
OpenAI attributes the improvement specifically to the Responses API with retained reasoning and compaction enabled. The source does not describe how these settings interact with all model versions or third-party orchestration layers. Metrotechs analysis: teams using middleware, custom harnesses, or non-Responses API integrations should verify whether those paths expose equivalent controls.
If your AI agent is underperforming on a multi-step workflow, check three things before assuming the model is the constraint.
First, confirm whether your integration retains reasoning across turns. If your harness or orchestration layer drops the reasoning trace after each step, the model is re-starting its analysis every time.
Second, check how your integration handles context limits. The ARC-AGI-3 harness used rolling truncation as its design choice: when context exceeded 175,000 characters, the oldest messages were discarded. OpenAI's own implementation used a tokens-based limit instead, which proved similar in practice because the vast majority of text is action grids, which tokenize at a 1:1 ratio.
If your setup uses rolling truncation, the agent is operating with an incomplete record of its own work.
Third, if you are using OpenAI's Responses API directly with GPT-5.6, verify that you are passing the previous response ID on each call. OpenAI identifies that as the mechanism that retains reasoning across tool calls and turns for GPT-5.6 specifically.
The ARC-AGI-3 result is OpenAI's own benchmark on OpenAI's own model. Independent confirmation of the same effect in enterprise workflows is not in the available source. The underlying mechanism is consistent with the benchmark evidence: an agent that cannot remember its reasoning will re-do that reasoning repeatedly. That is worth testing before the next production deployment review.
