Artificial Intelligence

DeepSeek Gives Away Its Own Benchmark Tool

DeepSeek released Harness on August 13, an MIT-licensed agent framework it says is the same tool it uses to benchmark its own V4 models - and it works with Anthropic and OpenAI models too.

Artificial IntelligenceBy Servola Tech Desk2026-08-144 min read

AI-assisted, edited by humans. | Editor-in-Chief: Leon Soliman | Editorial standards

DeepSeek Gives Away Its Own Benchmark Tool

Key takeaways

DeepSeek open-sourced Harness on August 13, 2026, the same MIT-licensed agent framework it used internally to benchmark its own V4-Flash and V4-Pro models, letting any outside developer independently reproduce the numbers DeepSeek publishes about itself.
Built on DeepSeek's internal Cordis meta-framework around an everything-is-a-plugin design, Harness ships four modes in this preview - Standard, Code, Minimal, and Creator - and keeps append-only session logs so any agent run can be resumed, forked, or replayed later.
The framework's published configuration supports DeepSeek's own models alongside Anthropic, OpenAI, Amazon Bedrock, Google Vertex, and Microsoft Azure, so a developer can point the exact tool DeepSeek benchmarks itself with at a Claude or GPT model instead and get a directly comparable run.
The project is led by Cui Tianyi, who joined DeepSeek in March 2026 from Jane Street and reached public preview about five months after recruiting began in May 2026 - positioning Harness as a direct open-source alternative to Anthropic's Claude Code rather than a minor internal tool.

The Release: What Shipped on August 13

DeepSeek released Harness v0.1 in open developer preview on August 13, 2026, publishing the full source code under the permissive MIT license, according to the company's own GitHub repository and confirmed by multiple technology outlets that reviewed the release. The project is led by Cui Tianyi, who joined DeepSeek in March 2026 after leaving quantitative trading firm Jane Street, and reached public preview roughly five months after his team began recruiting developers in May 2026. Harness is an agent runtime framework: the scaffolding of loops, tools, and session management that turns a raw language model into something that can read a codebase, run terminal commands, and complete multi-step engineering tasks on its own.

What makes the release notable is not the framework's features but what DeepSeek says it actually is: the same internal tool DeepSeek used to benchmark its own DeepSeek V4-Flash and V4-Pro models before publishing their scores, according to the company's own release notes. Framework and model normally stay separate in a lab's public claims; here DeepSeek is handing over both at once, letting outside developers rerun the exact scaffolding behind its own published numbers rather than trusting them on faith.

Same Tool, No Secrets: How Harness Actually Works

Harness runs on what DeepSeek calls a 'Cordis' meta-framework, organized around a single idea the company repeats in its own documentation: 'everything is a plugin.' Models, tools, sandboxes, execution loops, and even the user interface can be swapped out without touching the underlying source code, and the system keeps append-only session logs so that any agent run can be resumed, forked into a new branch, or replayed step by step later, per DeepSeek's own technical description. Four modes ship in the preview: a full Standard agent mode, a Code mode built around TypeScript orchestration for developers who want to script their own agent logic, a stripped-down Minimal mode built specifically for benchmarking, and a Creator mode for saving custom configuration presets.

Crucially, Harness is not locked to DeepSeek's own models. The framework's own model-provider list includes DeepSeek, Anthropic, OpenAI, Amazon Bedrock, Google Vertex, Microsoft Azure, and any OpenAI-compatible endpoint, according to the project's published configuration options - meaning a developer can point the exact tool DeepSeek used to benchmark itself at a Claude or GPT model instead, and get a directly comparable run.

The Quiet Bet Behind Giving It Away

Every frontier lab treats its benchmarking methodology as a soft competitive secret. Scores get published; the scaffolding, prompts, and retry logic that produced them almost never do, which is why rival labs routinely accuse each other of measuring under favorable conditions the outside world cannot reproduce. DeepSeek's move breaks that pattern in a specific, calculated way: instead of asking the industry to trust its published V4 scores, it published the measuring stick itself, for free, under a license permissive enough for any company to build a commercial product on top of it - the same license terms that let React and Node.js become default infrastructure for an entire industry.

That is the real story here, and it is not in any headline covering the release as 'an open-source rival to Claude Code.' If enough developers standardize on Harness as their agent runtime, DeepSeek does not just win goodwill for transparency - it quietly becomes the tool other labs' models get measured through by default. Owning the ruler is a cheaper, subtler form of platform leverage than winning the compute race everyone else is fighting, and it does not require DeepSeek's own models to be the fastest or the smartest to pay off.

What This Changes for Anyone Betting on AI Benchmarks

For a business evaluating agent platforms, the immediate practical value is real: Harness lets a team run its own workloads through the identical scaffolding DeepSeek used to generate its published scores, against DeepSeek's models or a rival's, instead of taking any lab's benchmark claims at face value. That is a genuine improvement on an industry where 'our model scored X' has become close to unfalsifiable for anyone outside the lab that ran the test.

It also carries a governance question that has nothing to do with model quality. An MIT license does not require an entity to be trusted, only to be legally usable, and a China-domiciled team is offering to become the default measuring instrument for agentic AI claims industry-wide - the same second-order dependency question that has already shaped procurement decisions around DeepSeek's and Moonshot's models. Adopting the tool is not the same decision as adopting the model, and businesses should not let goodwill from one blur the risk profile of the other.

Frequently asked questions

What exactly did DeepSeek release on August 13, 2026?

Harness v0.1, an MIT-licensed, open-source agent runtime framework that DeepSeek says is the same internal tool it used to benchmark its own V4-Flash and V4-Pro models, now available in public developer preview.

Does Harness only work with DeepSeek's own models?

No. Its published configuration supports DeepSeek, Anthropic, OpenAI, Amazon Bedrock, Google Vertex, Microsoft Azure, and any OpenAI-compatible endpoint, so a developer can run the same framework against a Claude or GPT model.

Why would a lab give away the exact tool it uses to prove its own benchmark claims?

It replaces an unfalsifiable claim with a reproducible one, and it positions Harness to become the default scaffolding other labs' agents get measured through if enough developers adopt it - a subtler form of leverage than the compute race.

Is Harness ready for production use today?

No. DeepSeek is shipping it as a v0.1 developer preview with anticipated breaking changes, roughly five months after the team began recruiting in May 2026, not as a finished production product.

The headline calls Harness an open-source rival to Claude Code. The more durable fact is that DeepSeek just gave away the ruler it measures itself with, model-agnostic and free to build on - and the lab that owns the ruler does not need to win the model race to shape how every other lab's claims get judged.

DeepSeekHarnessagent frameworkopen sourceClaude CodeAI benchmarking

Beijing Business Hours Now Price Your AI

DeepSeek's V4 API doubles in price during Beijing business hours and retires its old model names on 24 July. What that clock costs a European team.

4 min read

Cybersecurity

No Human Chose the Exploit, CISA Sets 2 Days

CISA added three actively exploited flaws to its KEV catalog on 2026-08-05 after Unit 42 traced attacks to DeepSeek-powered autonomous exploit selection across 460+ organizations. Federal deadline: 2026-08-07.

3 min read

Decisions

The Third Pivot Was the Warning Magic Leap Missed

Magic Leap filed a WARN notice for 193 layoffs effective 1 October 2026, its fourth strategic pivot after $4 billion raised. The lesson is the pattern, not the pivot.