Google Harness² Explained: Self-Improving AI Agents

Google Research open-sourced Harness² (Harness2): AI agents that improve their own harness without retraining. How it works, results and setup.

By Amal V S, Backend & AI Systems Architect · Fri Oct 23 2026

Here's a result that should change how you think about AI agents: take a model, freeze its weights, and change only the files around it — the prompts, skills, memory and tools. On one professional benchmark, that alone lifted a model's score from 64.2 to 75.7.

That's the core idea behind Harness² (also written Harness2), a framework Google Research open-sourced on 9 October 2026. Instead of retraining a model, Harness² lets an agent rewrite its own harness — task by task — and keep the lessons that actually helped. Same engine, better pit crew. Here's how it works, what the results really say, and how to run it yourself.

Key takeaways

  • Harness² is open source (google-research/harness2, Apache 2.0) with a paper, Recursive Agent Harnessing for an Open World (arXiv:2610.14224).

  • It improves agents without retraining: the model stays frozen while 8 harness components are edited.

  • Each task runs a Propose → Probe → Compose loop that judges edits by contrasting behaviour, not against an answer key.

  • Lessons that the evidence supports go into a persistent library that guides future tasks.

  • Reported gains: up to +11.5 on WorkBuddy-Bench, +4.5 on JobBench and +11.8 points on Harvey's LAB legal benchmark.

What is an agent harness?

A model on its own just predicts text. An agent harness is everything that turns it into an agent that can plan, use tools and finish real work. The Harness² README puts it simply: a harness is "the system around a model that enables it to act as an agent" (README).

In the paper, a harness has 8 editable components, each stored as a file in the agent's runtime:

Anatomy of an agent harness in Harness²: a frozen model surrounded by eight editable files — system prompt, guardrails, memory, skills, tool guidance, scripts, subagents and middleware

If you use coding agents, you already edit harnesses by hand: a CLAUDE.md, a folder of skills, a few hooks. (I covered that workflow in how to use Claude Code the right way.) Harness² asks the obvious next question: what if the agent did that tuning itself?

How Harness² works: Propose, Probe, Compose

Harness² runs two loops, one inside the other.

Harness² two-level recursion: a per-task Reference, Propose, Probe and Compose loop, feeding a persistent improvement library of adopted harnesses and EXPERIENCE.md lessons

Loop 1: task-level recursion (fix this task)

  1. Reference — run the current harness on the task and record what the agent did.

  2. Propose — an editor model drafts two edits: an invariant one for general weaknesses (planning, tool use, context, delegation) and a domain-specific one for this kind of task.

  3. Probe — run each candidate on the same task with the same model and environment, and compare it with the reference.

  4. Compose — the editor reads all the evidence and merges, revises, adopts one candidate, or keeps the old harness.

Loop 2: harnessing-level recursion (learn for next time)

After each task, Harness² reflects on what worked and updates an improvement library. Each entry pairs an executable harness with an EXPERIENCE.md file: when it applies, the lesson, and the evidence behind it. Weak evidence means no new entry, and later tasks adapt lessons rather than copying them blindly.

Click through one full cycle on an example task:

Why it works without an answer key

Harness² doesn't ask "is this answer correct?" — it asks "did this edit fix the weakness it targeted, change nothing, or break something?"

Most self-improvement methods need a verifier: unit tests, a grader, a known answer. Real work — a legal memo, a market report — rarely has one. Harness² sidesteps that by judging edits contrastively: the reference run reveals how the harness behaves, and each probe shows whether an edit changed that behaviour for the better. The authors are candid that this evidence is imperfect, since runs are stochastic and models share biases.

Harness² results: what the paper reports

The team tested two executor models — Gemini 3.5 Flash and GLM-5 — inside two agent runtimes, OpenCode and Codex, with Claude Opus 4.8 as the default editor model that proposes and composes changes. The benchmarks are professional, open-ended work:

  • WorkBuddy-Bench (Tencent) — code, web and office deliverables.

  • JobBench — 65 end-to-end professional tasks across seven occupations, graded by rubrics.

  • LAB (Harvey AI) — legal work across 24 practice areas; reported on a 100-task test split.

Single refinement step (k = 1), as reported in the paper:

  • GLM-5 · OpenCode: WorkBuddy 64.2 → 75.7 (+11.5) · JobBench 20.6 → 24.2 (+3.6) · LAB 59.6 → 71.4 (+11.8)

  • GLM-5 · Codex: WorkBuddy 64.4 → 75.3 (+10.9) · JobBench 23.6 → 24.8 (+1.2) · LAB 58.1 → 65.9 (+7.8)

  • Gemini 3.5 Flash · OpenCode: WorkBuddy 72.4 → 77.5 (+5.1) · JobBench 31.5 → 36.0 (+4.5) · LAB 79.9 → 81.9 (+2.0)

  • Gemini 3.5 Flash · Codex: WorkBuddy 74.5 → 77.5 (+3.0) · JobBench 33.9 → 35.3 (+1.4) · LAB 71.6 → 75.6 (+4.0)

Two patterns stand out. The weaker base setup gains the most — GLM-5 jumps far more than Gemini, which already starts high. And running more candidates in parallel (k = 5) adds a few more points on top. In fairness, an offline baseline the authors compare against, Meta-Harness, still wins some cells — and all of these numbers come from the authors' own paper, not an independent reproduction.

How to install and run Harness2

The repository targets Linux, Python 3.12 and uv. The demo runs with no model credentials (README):

git clone https://github.com/google-research/harness2.git
cd harness2
uv venv --python 3.12 .venv
source .venv/bin/activate
uv pip install -e '.[render]'

harness2-demo --output runs/demo --k 2

Real benchmark runs use models through Vertex AI, so you'll need a Google Cloud project — and a budget, since API calls are billed:

cp .env.example .env            # set your Vertex AI project and model IDs
gcloud auth application-default login

harness2-setup lab              # or: jb (JobBench), wb (WorkBuddy-Bench)
harness2-check --bench lab
harness2 --bench lab --k 1 --mode parallel --only-domains antitrust-competition --max-tasks 1
harness2-judge --bench lab --k 1 --mode parallel
harness2-report --bench lab --k 1 --mode parallel

  • --k sets the refinement budget; --mode is parallel (probe many candidates) or sequential (chain refinements).

  • OpenCode is the default executor; add --substrate cx to use Codex.

  • Model-backed benchmarks also need Node.js 20+; WorkBuddy-Bench needs Docker, and LAB needs Pandoc and Poppler.

Harness² vs fine-tuning: when to use which

  • Use harness optimisation when you can't (or don't want to) touch the weights — closed API models, fast iteration, or tasks with no labelled data.

  • Use fine-tuning when you need the model itself to know something new, at scale, and you have good training data.

  • They stack. A fine-tuned model still runs inside a harness — and that harness can still be improved.

Limitations to keep in mind

  • Research code: the README states it is not an officially supported Google product.

  • Cost: every probe is another full agent run. The authors note that "performance gains alone do not establish computational efficiency."

  • Imperfect evidence: contrastive judgments can be fooled by randomness or shared model biases.

  • Self-reported numbers: treat the benchmark gains as directional until others reproduce them on their own tasks.

Why Harness² matters

For the last few years, "make the agent better" mostly meant "wait for a better model." Harness² makes a strong case that a lot of performance is sitting in the layer you already control — and that agents can tune it themselves, without an answer key. If you build agents, the practical move is the same either way: treat your harness as code, measure it, and improve it on purpose.

FAQ

What is Harness² (Harness2)?

An open-source framework from Google Research that improves AI agents by editing their harness — prompts, tools, memory, skills and sub-agents — instead of retraining the model.

Does Harness² retrain or fine-tune the model?

No. The model weights stay frozen. Only the eight harness components around the model are edited.

Is Harness² free and open source?

Yes. The code is on GitHub at google-research/harness2 under the Apache 2.0 license. Running real benchmarks uses paid model APIs through Vertex AI.

Which models and agents does Harness² support?

The paper uses Gemini 3.5 Flash and GLM-5 as executors inside OpenCode (the default) or Codex, with Claude Opus 4.8 as the default editor model.

How does Harness² improve agents without ground truth?

It runs candidate edits on the same task and contrasts their behaviour with a reference run, keeping edits that fix the targeted weakness without causing regressions.

Sources: Harness²: Recursive Agent Harnessing for an Open World (arXiv:2610.14224) · google-research/harness2 on GitHub. Benchmark figures are as reported by the authors.