Cohere open-sources a coding agent that runs on a single H100, but tokens cost triples

Cohere just made agentic coding more accessible. On Tuesday, the company open-sourced North Mini Code, a 30 billion parameter Mixture-of-Experts model that it says can run on a single Nvidia H100. The catch is also clear, and it matters for anyone building production pipelines: independent testing found North Mini Code generated three times the output tokens of comparable models. In other words, you can get local control and real tool use, but you may pay more per completed task because verbosity compounds.

For engineering and product leaders deciding what to run in production, that token math is not academic. North Mini Code supports a 256,000 token context window with a 64,000 token maximum generation length, which is exactly the kind of spec people look for when they want an agent to handle large, multi-file work without constantly dropping context. It is built for agentic software engineering, including sub-agent orchestration, architecture mapping, code review, and terminal work. But if the model tends to say more than its peers to finish the same job, your inference cost and latency can rise in lockstep with agent volume, turning “it runs locally” into “it might still cost more per outcome.”

So what is North Mini Code actually for, and what makes it different from “we released another coding model”? Cohere says it was not adapted from a general-purpose base. Instead, it targets the full agentic coding stack: software engineering tasks designed for multi-step agent behavior, not just single-turn code generation. Cohere also claims integrated tool-use capabilities and “interleaved thinking,” which it says improves performance across multi-step agentic work.

On the workflow side, North Mini Code can analyze and map systems architecture, surface dependencies, and perform code review across large codebases. The 256,000 token context window is the technical enabler for that “hold a big project in one pass” promise. Then there is the terminal piece, where agents stop being diagrams and start being commands. Cohere trained it for terminal environments so it can handle shell interactions, package scripts, and command-line tooling. To validate that, Cohere benchmarked it on Terminal-Bench v2, which tests agents in real terminal environments rather than synthetic code generation tasks.

Architecturally, it is a sparse Mixture-of-Experts model. Cohere describes 128 experts with 8 active per token, which is why the compute requirement at inference time is “closer to a 3 billion parameter model despite 30 billion total parameters.” If you are a team thinking about deployment, this is one of those details that can make or break feasibility: MoE models can offer efficiency gains, but you still have to translate the “token outputs” reality into cost.

Hardware and demo details give more context on feasibility. Nick Frosst demoed North Mini Code running on a Mac Studio via MLX at around 20 gigabytes of RAM, the same machine he uses for his own local coding work. That lines up with Cohere’s broader pitch: local deployment is a lever for empowering people and making AI work “for them,” not only in managed clouds. Cohere also describes the training process in two stages of supervised fine-tuning followed by reinforcement learning with verifiable rewards across more than 70,000 verifiable tasks spanning approximately 5,000 repositories, deduplicated against SWE-Bench.

Importantly, Cohere trained across three agent scaffolds: SWE-Agent, Mini-SWE-Agent, and OpenCode. Cohere reports a 10 percentage point gain on OpenCode evaluation from the multi-harness approach while maintaining SWE-Agent performance. This matters because agentic coding is not only about raw model quality. It is about tool calling correctness, robustness, and how reliably an agent can drive multi-step workflows using structured actions.

In the market, North Mini Code arrives into a world where teams are already choosing between capability and control. The source lists Mistral’s Devstral Small 2, GitHub Copilot, Cursor, and Anthropic’s Claude Fable 5 as prominent options, each with different cost and deployment tradeoffs. Cohere’s primary benchmark comparison is against Mistral Devstral Small 2, a 24 billion parameter dense model. In vendor-reported internal tests, Cohere claims 2.8x higher output throughput and a 30% inter-token latency advantage over Devstral Small 2 under identical hardware configurations. On top of that, Cohere claims in its Hugging Face technical post that North Mini Code outperforms open-source models up to four times its parameter count on its reported benchmarks.

Independent evaluation adds another wrinkle, and it directly intersects with the “verbosity cost” theme. Artificial Analysis independently ranks North Mini Code eighth of 127 comparable open-weight models on output speed at 210 tokens per second, with a time to first token of 0.25 second against a class median of 1.95 seconds. It places 18th of 127 on the Artificial Analysis Intelligence Index. One flag from the same data is about scale of output: the model generated 75 million output tokens to complete the Intelligence Index against a class median of 25 million.

Now zoom out to what enterprises should care about next. This release clarifies a decision set that has been forming for months: purpose-built agentic training should be treated as baseline evaluation, not a marketing checkbox. The distinction between models fine-tuned for code and models trained specifically for agentic workflows, including verified tool calls and multi-harness robustness, is now material to pipeline decisions. Also, verbosity is a hidden pipeline cost that benchmark leaderboards do not always surface in a way that maps cleanly to production economics. If your agent runs millions of times, “extra tokens” becomes real money and real latency.

Finally, there is the pricing and residency fork. Anthropic’s Claude Fable 5, described here as the most capable publicly available managed coding model, runs at $50 per million output tokens. Cohere positions North Mini Code as the opposite on deployability: small, cost effective, Apache 2.0, and locally deployable. Frosst explicitly frames the difference as sovereignty versus proprietary deployment in an X post, calling out the contrast between “small, open source, transparent and sovereign” and “large, expensive, proprietary and hegemonic.” For teams running high-volume agentic coding pipelines, North Mini Code’s release is a reminder that the frontier pricing split is a real architectural decision. You should model both cost paths against your actual workload before you bet your roadmap on either managed convenience or local control.