METR: Half of SWE-Bench Passes Fail Real Code Review
METR found maintainers would reject roughly half of AI PRs that pass SWE-bench automated grading, with a 24-point gap that suggests benchmark scores substantially overstate production readiness.
Articles Tagged "GPT-5"
GPT-5.4 Review: The Computer-Use Frontier
GPT-5.4 brings native computer use, a 1M token context window, and serious coding muscle to OpenAI's mainline model - but at a premium price.
GPT-5.4 vs Gemini 3.1 Pro - Breadth Meets Reasoning Depth
GPT-5.4 leads on computer use and enterprise productivity. Gemini 3.1 Pro leads on science reasoning and math at 20% lower cost. A benchmark-by-benchmark comparison.
GPT-5.4 vs Claude Opus 4.6 - Computer Use Meets Agent Teams
GPT-5.4 leads on computer use and enterprise productivity at half the price. Claude Opus 4.6 leads on coding, agent teams, and long-context retrieval. Here is where each model wins.
GPT-5.4
OpenAI's most capable frontier model combines native computer use, 1M-token context, and three variants at $2.50/$15 per million tokens.
GPT-5.4 Lands with Computer Use and 1M Token Context
OpenAI ships GPT-5.4 with built-in computer use that beats human desktop performance, a 1 million token context window, and native Excel and Google Sheets integrations.
GPT-5.3 Instant
GPT-5.3 Instant launched March 3, 2026, cutting hallucinations by 26.8% and overhauling ChatGPT's tone - but with documented safety regressions in the process.
OpenAI's Three-Word GPT-5.4 Tease - What It Means
OpenAI posted '5.4 sooner than you Think.' one hour after launching GPT-5.3 Instant. Here is what the Codex repo leaks already told us - and what the tweet does not.
GPT-5.3 Instant Rolls Out to All ChatGPT Users
OpenAI ships GPT-5.3 Instant with 27% fewer hallucinations, a less preachy tone, and better web search - available now across all ChatGPT tiers and the API.
Grok 4 vs ChatGPT: Which AI Chatbot Wins in 2026?
A data-driven comparison of xAI's Grok 4 and OpenAI's ChatGPT powered by GPT-5.2, covering benchmarks, pricing, features, and real-world performance.
GPT-5.4 Leaked Twice in Codex Repo PRs - Here Is What We Know
Two pull requests in OpenAI's public Codex GitHub repo referenced GPT-5.4 before being scrubbed - one adding full-resolution vision support, the other a fast mode toggle. Seven force pushes and a deleted employee screenshot confirm this was not intentional.
GPT-5.2
GPT-5.2 is OpenAI's most capable model with three modes, 400K context, and record-setting professional benchmarks - but speed and pricing raise questions.
