OPENAI DISPUTES GPT-5.6 SOL BENCHMARK RESULTS

OpenAI achieved a 38.3% score on the ARC-AGI-3 benchmark using GPT-5.6 Sol with its proprietary API and two additional settings, compared to Anthropic's Opus 5 performance. However, the official test environment registered GPT-5.6 Sol at just 7.8%. OpenAI attributes the discrepancy to outdated API features in the ARC Prize's provider-neutral test setup. The company argues that the official benchmark environment may not reflect the model's current capabilities when accessed through its latest API infrastructure. ARC Prize maintains that its test environment is designed to remain provider-neutral and standardized across all submissions. The competing claims highlight ongoing questions about how AI models should be fairly evaluated when companies have access to their own optimized deployment methods versus standardized testing frameworks. The dispute underscores broader tensions in AI benchmarking: whether neutral test conditions or real-world API performance better represents model capabilities.