Gemini 4 Hydrogen leads on SWE-sweep and trails Sonnet on terminal tasks
Google introduced Gemini 4 Hydrogen on November 12, adding a model for everyday coding and agent tasks to the Gemini API and Google AI Studio. It is also available through other Google products, including Search and Antigravity.
Hydrogen is adapted from Gemini 4 Carbon, according to Google's model card. Google positions it for tasks such as editing code and running tests. In Google's selected results, Hydrogen scores 13.5% on SWE-sweep against Sonnet's 3.1%, but trails Sonnet by about seven percentage points on Terminal-Bench 4.1. At its highest thinking level, Hydrogen cost $3.16 per task in Artificial Analysis's Intelligence Index evaluation.
What the benchmarks show
On the Artificial Analysis Intelligence Index, a composite of nine evaluations, Google reports that Hydrogen scores 51 at its default high thinking level, compared with 50 for Gemini 4 Argon. Hydrogen reaches 54 at its highest xhigh setting. Artificial Analysis reports a median of 41 for models in a similar price tier. The index combines scores across coding, reasoning and other tasks.
Google's model card gives a more detailed view of coding and computer-use performance:
| Benchmark | Gemini 4 Hydrogen | Claude Sonnet 5.5 | GPT-6.1 Terra |
|---|---|---|---|
| SWE-sweep, subtask-weighted reward | 13.5% | 3.1% | 5.8% |
| Terminal-Bench 4.1, cheating-adjusted | 57.0% | 63.9% | Not reported |
| OSWorld 2.1, offline subset | 77.6% | 83.9% | 74.2% |
Compare results within each benchmark; Google used different setups and sourced some scores from other providers.
SWE-sweep tests repository coding work; its figure is a subtask-weighted reward. Terminal-Bench tests agents on terminal tasks. OSWorld tests computer use, and Google's comparison covers its 82-task offline subset. Hydrogen leads this selection on SWE-sweep, while Sonnet leads on terminal and computer-use tests. Google also reports 18.2% for Gemini 3.8 Flash on Terminal-Bench 4.1, compared with Hydrogen's 57%.
The evaluation methodology describes different setups and sources. For SWE-sweep, Google tested Hydrogen at xhigh with Antigravity CLI 1.4.8 and used competitor results from the public leaderboard. For Terminal-Bench 4.1, it used Hydrogen at high effort with a mini-swe-agent vision harness and averaged five attempts per task; the Sonnet result came from the leaderboard. For OSWorld, Google tested Hydrogen at xhigh and Sonnet, then reported the best of three runs for each. Terra's score came from its system card.
Google says Hydrogen at its default high setting completes tasks 2.5 times faster than Gemini 3.8 Flash. In its own Intelligence Index evaluation, Artificial Analysis reports an average of 13.7 minutes and $3.16 per task for Hydrogen at xhigh. The median time for models in a similar price tier was 13.0 minutes. Hydrogen generated 304 million output tokens across the index, compared with a median of 72.8 million for models in a similar price tier. That is about four times the median.
Pricing, access and model switching
The Gemini API price is $1.50 per million input tokens and $7.50 per million output tokens, with the same rates across its context window. The one-million-token window covers input and output together. Synchronous responses default to a maximum of 64,000 output tokens. Background long decode supports up to one million output tokens within that shared window. The model's knowledge cutoff is March 2026, though some domains may be less current. The model has four thinking levels: low, medium, high and xhigh.
Google lists the following access at launch:
| Product | Access |
|---|---|
| Gemini API, AI Studio and Android Studio | Available to developers; Google lists all AI Studio tiers. |
| Google Search | Google says Hydrogen is available to users globally. |
| Gemini app | Personal accounts with Google AI Plus or higher can select Hydrogen. For accounts without a plan, Google says automatic selection uses Flash-Lite for most prompts and may use Hydrogen for those needing more reasoning. |
| Google Antigravity | Available to paid Google AI Pro and Ultra subscribers and to enterprise users. |
| Gemini for business | Available through the API or model drop-down. |
| Gemini in Google Sheets | Available to eligible business subscriptions, starting with Google AI Pro and Ultra subscribers for consumers. |
Alongside the model, Google introduced Orchestrator in Antigravity. It switches between models such as Carbon and Hydrogen within a session: a stronger model can plan or revisit a difficult problem, while Hydrogen handles execution. Google reports that on Terminal-Bench 4.0, Orchestrator achieved task success close to Carbon while using 24% less quota and finishing 19% faster on average. The reported saving measures Antigravity quota; API bills depend on token use and pricing.
Google's model card also reports a 5.57 percentage-point increase in unjustified refusals compared with Gemini 3.8 Flash in its automated evaluation, especially for multimodal prompts. Google says the updated evaluation methods make these results unsuitable for direct comparison with earlier model cards, and its full safety and security report is forthcoming. Performance on individual repositories, including review time and token use at higher thinking levels, remains to be tested.
Member discussion