I’ll give it a spin on my software factory. The other Gemini models looked good initially and in benchmarks, but on long-running sessions and orchestration, it’s fallen short.
We’re reaching the point where certain models are really good at 1 thing. The person writing this blog post isn’t using Neon for large-scale orchestration, but I’m sure it excels in its usage. For me Fable 5.2 as the orchestrator and Opus 5.5 and grunt workers have led to the most robust software...so far.
> The person writing this blog post isn’t using Neon for large-scale orchestration, but I’m sure it excels in its usage
I'm the person writing this. This is head to head battle data by real developers on top of OpenCode. The data is generated by an open source app so you can inspect it yourself (https://github.com/bottomless/agent-duel). The hitch is there's no real Neon data. This is purely projecting from Carbon's super slow but very good performance onto Neon's likely fast and likely equal performance.
Author here, guilty as charged. To be fair, we don't have early access to any models (just getting started).
The blog itself can be seen more as an exploration of how speed impacts coding agent battle votes, than anything specifically about Neon.
Applying the speed impact to Gemini Carbon is particularly interesting, though, because it's the top model in our leaderboard when you exclude early votes (https://agentduel.com/leaderboard?excludeEarlyVotes=true) but mediocre when you let developers vote. Gemini literally takes 3x as long as opponents in matched battles.
I guess what I really want to know is how Gemini 4 Neon w/Antigravity compares with Opus 5.5/Claude Code. I'm not really into using LLMs in a particularly advanced way, I just use CC like 99% of the time when I need to write something, and then maybe Codex to second guess it.
But that's about as complicated as I get. I have recently tried to use Copilot because that is what work wants us to use, and even with Opus 5.5 it is noticeably inferior to CC on the same model. Wild. So clearly the harness matters.
The models can be very optimized for the harness, Clem (of Hugging Face) just recently released a Qwen3.8 model that used RL training on the harness and it got some pretty significant gains in the benchmarks.
Totally possible for your use case, though the analyses I’ve done on harness model interactions show it’s actually not that important for benchmark performance. https://micjm.com/blog/on-model-harness-interactions/
For what it’s worth, the linked article data (Agent Duel data) is all run on OpenCode. I’m curious if people would prefer we run each model in its native harness (at the expense of user experience because it’ll be buggy).
We had CC for a while and now have to use GHCP. As such I haven't used CC in like 6 months but around the time I switched I felt like GHCP CLI had pretty much caught up. Curious to hear what you feel are the gaps? Are you using the cli versions of both?
Yep, CLI versions for both GHCP and CC. To be honest, I haven't done enough actual coding with GHCP yet to get a feel, so my original comment may be too broad. But I've been repeatedly comparing and contrasting with CC for code reviews (as a team lead, I do more of that these days than coding). I have been pretty consistently getting about half again as many bugs from CC as from GHCP. And very nearly always correct, not hallucinations.
My company is slow-walking (and that is an understatement) our LLM usage, so the majority of work that I'm reviewing is human generated. After the stories I hear of what other people are experiencing when management is more enthusiastic, maybe I should count myself lucky.
Ah that's a new angle. After the "Google models are behind" and the "Google models are slower" and the "Google costs more per tokens", we're now at the following phase:
"OK, google models perform just as well, are faster and are cheaper but all that doesn't matter. Because the models, now that Google has very good ones, doesn't matter after all. All that matters is the harness".
And of course it's upvoted by the Anthropic shills.
Also last I checked there was no clear flag to turn off training in paid plan of Antigravity. So if you care about that looks like Google doesn’t want you as their customer.
Fun analysis. I guess it’s not surprising to see the need for speed™ as an important factor, but who is using Fable for run of the mill coding work? It’s too pricey!
Yeah, check out DeepSeek performance on the leaderboard that's the source of the data of this post (http://agentduel.com/leaderboard). It wins ~half the time because it’s lightning fast and good enough.
I'm blown away by Neon, it's as good as Argon (which was amazing) but fast and cheap (i think). I've used primarily Claude for about a year and now i've tried Google's newer models in the last week, funnily because i ran out of tokens testing Fable and to me Opus is trash, Fable is trash - I will just use Opus for Web Day Job which it handles with ease and keep Neon for my serious side projects (ML related), which Claude can't handle well at all. Ive been working on a project for about 2 months, Claude Opus was a dead end and even with the new Fable which didn't move the needle and gave me negative progress, but Hydrogen on things changed and i've finally made good progress. Frontier model indeed. Scary good. Wow!
Just as good but cheaper comes from the benchmark performance. Also faster from openrouter. The opus 5.5 parity projection is an internal model based on those assumptions from the Agent Duel team.
We’re reaching the point where certain models are really good at 1 thing. The person writing this blog post isn’t using Neon for large-scale orchestration, but I’m sure it excels in its usage. For me Fable 5.2 as the orchestrator and Opus 5.5 and grunt workers have led to the most robust software...so far.
I'm the person writing this. This is head to head battle data by real developers on top of OpenCode. The data is generated by an open source app so you can inspect it yourself (https://github.com/bottomless/agent-duel). The hitch is there's no real Neon data. This is purely projecting from Carbon's super slow but very good performance onto Neon's likely fast and likely equal performance.
The blog itself can be seen more as an exploration of how speed impacts coding agent battle votes, than anything specifically about Neon.
Applying the speed impact to Gemini Carbon is particularly interesting, though, because it's the top model in our leaderboard when you exclude early votes (https://agentduel.com/leaderboard?excludeEarlyVotes=true) but mediocre when you let developers vote. Gemini literally takes 3x as long as opponents in matched battles.
But that's about as complicated as I get. I have recently tried to use Copilot because that is what work wants us to use, and even with Opus 5.5 it is noticeably inferior to CC on the same model. Wild. So clearly the harness matters.
- SWE-bench Pro (hard 51): 39.2% → 64.7%
- Terminal-Bench 2.1: 39.5% → 55.8%
https://huggingface.co/harness-hub/Qwen3.8-27B-pi-adapted
If this were true, models would do way better in their own harness than in others, which doesn't seem to be the case (effect is minor).
Your link has a "link to the release blogpost" which is supposed to explain the training recipe but the link is down.
For what it’s worth, the linked article data (Agent Duel data) is all run on OpenCode. I’m curious if people would prefer we run each model in its native harness (at the expense of user experience because it’ll be buggy).
My company is slow-walking (and that is an understatement) our LLM usage, so the majority of work that I'm reviewing is human generated. After the stories I hear of what other people are experiencing when management is more enthusiastic, maybe I should count myself lucky.
"OK, google models perform just as well, are faster and are cheaper but all that doesn't matter. Because the models, now that Google has very good ones, doesn't matter after all. All that matters is the harness".
And of course it's upvoted by the Anthropic shills.
For this model, I mean. And not the Anthropic IPO.
Just as good but cheaper comes from the benchmark performance. Also faster from openrouter. The opus 5.5 parity projection is an internal model based on those assumptions from the Agent Duel team.
Second, I will be trying this with agy when my Opus 5.5 limit hits. I only hope its faster than Argon which is already dog slow
"It looks like he's dead."
"...He's covered in blue paint, or something."