Tag: AI

  • Gemini 4 Carbon (2026): Benchmarks, Cyber Claims, and Fairwind Access

    Gemini 4 Carbon (2026): Benchmarks, Cyber Claims, and Fairwind Access

    Bottom line: On October 28, 2026 Google DeepMind announced Gemini 4 Carbon — its strongest frontier model yet, #1 on Artificial Analysis Intelligence Index v5.0 at score 59, with headline cybersecurity numbers and a Fairwind rollout that starts with cyber defenders rather than a public API. Developers debating “benchmarks vs real coding” should separate checkable evidence from Google-reported scores: only one of three cyber SOTA claims sits on a public leaderboard today. This sits beside our notes on Alphabet Q3 / AI Mode monetization and the Connected Apps Platform preview — product surfaces and model capability are different layers of the same Google AI stack.

    Primary sources: Google’s Carbon announcement, Artificial Analysis’s Oct 28 Index write-up, and Gautam Parab’s independent check of the three cyber claims.

    What Gemini 4 Carbon is — and who can use it now

    Carbon follows Gemini 4 Argon by about two weeks. Google positions it for hard, long-horizon engineering and agentic workflows — multi-day internal runs that push through dead ends on root-causing and patching subtle bugs across large codebases. Multimodality covers text, image, video, and speech input with text output; context is 1M tokens (Artificial Analysis).

    Access today is limited. Google says it is in the U.S. government’s voluntary pre-release testing process. Carbon goes first to a subset of cyber defenders, then to additional trusted partners through the Fairwind Program. Broader developer, business, and consumer availability is promised “as soon as possible,” without a dated GA. Artificial Analysis likewise notes selected-user rollout only.

    Treat Carbon as a capability signal and Fairwind waitlist item — not a default production swap unless you are already in that cohort.

    Benchmarks at a glance

    MetricGemini 4 CarbonNotes
    AA Intelligence Index v5.059 (#1)Leads 4 of 9 Index evals
    AA Coding Agent Index59 (2nd)Claude Opus 5.5 max 62 (Claude Code); Carbon + Antigravity CLI
    WebDev Arena#4 (1768 Elo)+89 Elo; Google-reported
    SEC-bench Pro88.2%Google-reported; not on public board
    ExploitGym54.8%Google-reported; no public scores
    CWE-bench (Collinear)71.0%Public v1 leaderboard #1
    AA-Omniscience v1.154Accuracy 58%; hallucination 11% (2nd-lowest)
    Terminal-Bench cheat / Gray Swan IPI0.0% / 0.0%Google safety claims (Argon IPI was 0.7%)

    On Intelligence Index tasks at standard pricing, Artificial Analysis puts Carbon at about $7.76 per task (~23% less than Claude Opus 5.5 max at $10.04), with fewer output tokens (123k vs 173k). Useful if you compare agents on $/successful task, not Elo alone.

    Coding strength vs Claude Opus 5.5

    Google stresses long-horizon coding uplift and the WebDev Arena jump. Artificial Analysis’s sharper frame: with Antigravity CLI, Carbon (high) scores 59 on the Coding Agent Index — three points behind Claude Code with Opus 5.5 (max, 62). Carbon is also second on Terminal-Bench 4.0 (64%) and FrontierCode v1.1 (51%).

    That is a real recovery for Gemini on AA’s coding-agent view — still second, not first. Claude-centric teams should not assume a day-to-day win until Fairwind expands and third-party harnesses publish comparable runs. Gemini-centric shops now have an Index signal that Carbon is in the same band. On AA-Briefcase, Carbon improves 126 Elo over Argon to 1609 but still trails several peers — gains without universal leadership.

    Cyber claims: evidence grades

    Google leads with three cybersecurity percentages (evaluated without guardrails). Security teams should grade the evidence before trusting the slides.

    • CWE-bench 71.0% — checkable. Collinear’s public v1 board lists Carbon first. Next cluster ties at 68% (Grok 4.7, GPT-6 Astra, Argon); Opus 5.5 at 67%. Parab notes the lead is ~3.5 tasks on a 120-task set, with binomial SE around a 70% score >4 points — larger than the lead. Still the one claim you can verify on the owner’s table.
    • SEC-bench Pro 88.2% — Google-reported. Public overall-board top Parab cites: GPT-5.5 at 58.4% (201/344). No Gemini on those boards; Carbon’s figure is Google’s run. The ~30-point gap is not like-for-like vs October 2026 frontier peers on the same harness.
    • ExploitGym 54.8% — Google-reported. Repo describes the task set and submission format; no public scores posted. “SOTA” rests on Google’s number alone.

    Google also cites >2× secure-code generation success vs Argon (internal), Chrome feedback of >10× better high-quality patches vs another lab’s frontier model, and Wiz observations on web pen-test benchmarks. Directional practitioner signals — not substitutes for same-harness public boards. Briefing rule: lead with CWE-bench; flag the other two as vendor-reported.

    Safety metrics Google highlights

    • Terminal-Bench 4.0 cheating: 0.0% success; attempts 0.4%, with higher accuracy.
    • Gray Swan IPI: 0.0% attack success (Argon 0.7%).
    • Unsupported claims: ~2× fewer vs Argon; caveats in the deliverable or clear withheld-action lists.
    • Overreach / under-service: −26% / −29% on Google’s multi-step plan metrics; high-severity live misalignment alerts down ~80% internally.

    AA-Omniscience: Carbon 54 (vs Opus 5.5 max 48, Argon 46); accuracy 58% (+10 vs Argon); hallucination 11% (still 2nd-lowest; Opus/GPT-6 Astra ~50% on that metric). More coverage with a controlled hallucination rise — worth watching in RAG-heavy workflows. Separately, publishers tracking crawl/training policy should keep Google’s shared robots.txt AI-training token on the radar; capability launches and training controls are related governance surfaces.

    Fairwind rollout and pricing

    Rollout: Fairwind → cyber defenders first → trusted partners → broader product surfaces later. U.S. voluntary pre-release testing called out. No GA date.

    Intro price (same as Argon intro): $2 / 1M input, $10 / 1M output, cached input 95% off. After intro: $4 / $20. Artificial Analysis charts standard pricing and notes a 50% launch discount with no confirmed end date. Model evals at both rates; assume $4/$20 for procurement unless Finance locks a discount window.

    Practical takeaways — what to do now

    1. Cyber defenders / MSSPs / cloud-sec: Ask about Fairwind. Evaluate on your vuln→exploit→patch pipelines; use CWE-bench as the public reference; demand harness details for SEC-bench Pro / ExploitGym-style claims.
    2. Platform / software teams: Keep your current coding agent as baseline until GA. Side-by-side Terminal-Bench / FrontierCode-class tasks plus one real multi-hour repo task when you get access.
    3. Gemini / Vertex shops: Track Fairwind → GA; design prompt-injection and scope-control tests that mirror the IPI / overreach claims rather than trusting the blog alone.
    4. Publishers / SEO: Carbon does not change crawl or citation rules by itself. Pair model-watch with Connected Apps, AI Mode economics, referral reality, and training-policy tokens.
    5. Executive one-liner: “#1 on AA Index v5 at 59; #2 on Coding Agent Index; one of three cyber SOTA claims is on a public board; Fairwind-limited; intro $2/$10 then $4/$20.”

    FAQ

    Is Gemini 4 Carbon publicly available?

    Not generally. Google is rolling it first to a subset of cyber defenders and then to trusted partners via Fairwind, while participating in U.S. voluntary pre-release testing. Broader availability is promised without a dated GA.

    Did Carbon take #1 on Artificial Analysis?

    Yes on Intelligence Index v5.0 at 59. It is second on the Coding Agent Index (59 vs Claude Opus 5.5 max at 62). It leads several Index components but not every coding or knowledge-work board.

    Are the cybersecurity SOTA claims independently verified?

    Partially. CWE-bench 71% is on Collinear’s public leaderboard. SEC-bench Pro 88.2% and ExploitGym 54.8% are Google-reported without public like-for-like peer tables as of Parab’s Oct 29 check. Do not treat the public SEC-bench top (e.g., GPT-5.5 58.4%) as a head-to-head against Carbon.

    How is Carbon priced?

    Intro: $2 per million input and $10 per million output, cached input 95% off. After intro: $4 / $20. Artificial Analysis notes a 50% launch discount versus standard rates.

    How does this relate to AI Mode and Connected Apps?

    Carbon is a model-capability and access story. AI Mode monetization and the Connected Apps Platform preview are product-distribution and economics stories on the same Google stack — useful context for publishers, separate from whether you evaluate Carbon under Fairwind.


    Sources: Google DeepMind — Introducing Gemini 4 Carbon (Oct 28, 2026); Artificial Analysis — Gemini 4 Carbon #1 on Intelligence Index v5; Gautam Parab — Three SOTA claims, one leaderboard. Analysis original to Web Observer.