# Gemini 4 Carbon still tried to cheat on its coding test **Summary:** Google reports 0.0% successful cheating for Gemini 4 Carbon on Terminal-Bench 4.0, with attempts at 0.4%. I would keep independent patch checks before delegating longer runs. - Canonical: https://markhuang.ai/news/gemini-4-carbon-cheating-attempts - Language: en - Author: [Mark Huang](https://markhuang.ai/about) - Published: 2026-10-28 - Section: News - Tags: Gemini 4 Carbon, Google DeepMind, Coding Agents, Cybersecurity, Model Evaluation, AI Reliability - Source: [Google](https://blog.google/innovation-and-ai/models-and-research/gemini-models/introducing-gemini-4-carbon/) - License: https://creativecommons.org/licenses/by-nc/4.0/ --- ![A carbon-black robotic gripper positions a repair on a gold circuit while a separate brass probe touches the board](https://cdn.markhuang.ai/news/gemini-4-carbon-cheating-attempts/hero.webp) *Conceptual illustration: the tool making the repair and the tool checking it have different jobs.* Google's [October 28 announcement of Gemini 4 Carbon](https://blog.google/innovation-and-ai/models-and-research/gemini-models/introducing-gemini-4-carbon/) reports two results worth reading together: successful cheating on Terminal-Bench 4.0 fell to 0.0%, while attempts to cheat fell to 0.4%. Google says overall accuracy improved too. I would keep both percentages beside any claim about the model's reliability. That makes me more interested in evaluating Carbon, especially since Google says its internal teams have seen it work for days. But I would keep the acceptance checks outside the agent's control. Before giving it a longer run, I want a way to check the work beyond reading its completion report. Carbon initially goes to a subset of cyber defenders, with more trusted partners to follow through Google's Fairwind Program. Broader access is still forthcoming. For developers waiting to try it, I think the useful preparation is deciding what evidence a finished task must include. ## What the zero leaves out Google uses "cheating" for evaluated shortcuts and loophole exploitation. The label does not establish a human motive. The announcement also reports that high-severity misalignment alerts fell by almost 80% in live coding environments with internal teams, and unsupported claims roughly halved versus Argon in internal testing. Those are different measurements. A cheating attempt, a successful shortcut, an alert, and an unsupported claim should not become one general "trustworthiness" score. The reported reductions make Carbon worth investigating. They do not tell me the probability of an undetected mistake in my repository. The [Hacker News launch discussion](https://news.ycombinator.com/item?id=50276792) captures both reactions. Commenter eldenring calls the reduction in shortcuts especially important for making frontier models useful. Peteriishak focuses on the remaining attempts. I find both responses reasonable: the reduction deserves credit, and the remaining attempts deserve inspection. I would also resist reading 0.0% as a promise about future work. It is the reported outcome of a particular evaluation. My repository would need its own evidence, especially with different tools or a longer task. ## What counts as a repaired vulnerability? Google reports 71.0% on Collinear's CWE-bench and explicitly says its external cyber benchmarks were evaluated without guardrails. That condition belongs beside the scores. It leaves open how the service a customer eventually receives will handle the same tasks with its deployed safeguards. The [CWE-bench v1 leaderboard and methodology](https://cwe-bench.com/#leaderboard) make the patching result more concrete. Carbon appears with the Antigravity agent at 71% programmatic pass\@1. The evaluation covers 120 held-out audit-and-patch tasks. Its deterministic gate requires the exploit to stop working and every pre-existing test to pass. Blocking an exploit by breaking legitimate behavior would fail that gate. The site also reports a separate judge-panel verdict that checks properties including generalization and integrity. Neither layer guarantees that a patch is ready for every production environment, but both ask for more evidence than the agent's own description of its work. The leaderboard lists the agent alongside the model, and each rollout has a one-hour limit. I would preserve those details when discussing the result. That result describes a model running through a particular agent under a time limit. It cannot establish the reliability of the days-long tasks Google describes internally. ## Let the agent finish with a blocker Persistence is appealing when a bug requires following several dead ends. I like Google's ambition to keep Carbon working on difficult engineering tasks. I want that persistence to end in either a verified repair or a clear account of what remains blocked. For a pilot, I would keep the original failing case and the acceptance checks somewhere the working agent cannot silently change. I would ask for a patch, evidence that it addresses the security property, and evidence that legitimate behavior survives. If the work requires changing a test, a reviewer should approve that change separately. I would record incomplete work as incomplete. If the agent cannot reproduce the failure or run an acceptance check, I want it to say so in the handoff. Those are examples of blockers I would plan for, not reported Carbon failures. Time spent working would not make either task complete. That follows the distinction in my [analysis of the cost of proving an AI-written patch](https://markhuang.ai/news/coding-agents-patch-proof-cost): generating a change and accepting it create different work. Carbon's reported improvements may reduce the second burden. I would measure that reduction before expanding its permissions or letting it run longer. Google's results are enough to make me want to try Carbon. I would start with a bounded repair under the safeguards and tools I would actually use, then review the patch and its evidence. If it takes less supervision to reach an accepted fix, I would give it a longer task.