The agent keeps saying it inspected the repository before proposing a change. The run record shows no file reads. The prompt says “run the tests,” but the runtime has no shell or test adapter.
Adding another instruction will not create either capability. This is the moment to change the agent harness, the software around the model that starts work and manages tools.
The harness is the software around the model that starts work, exposes tools, manages state and budgets, captures artifacts, runs checks, and decides what can continue. A trace is the record of the whole execution, including the final answer and the steps that produced it. A validator is a deterministic or semantic check that turns an artifact into evidence. A replay record keeps enough input, output, and environment information to inspect a run again. A candidate is a proposed harness or profile change. A runtime is the software that starts model and tool work, manages limits, and records events. An agent profile is the versioned bundle of model, prompt, tools, permissions, skills, runtime settings, and evaluator identity for a run. An evaluation is a repeatable test that compares the candidate with a baseline on defined cases. A baseline is the version already trusted for comparison. A judge is a scorer for qualities that a deterministic validator cannot fully capture. A holdout set is a protected group of cases kept outside candidate search. A sandbox is an isolated execution boundary that limits files, networks, credentials, or processes. A mutable surface is the part of the system a candidate is allowed to change.
Harness evolution changes tools, workers, validators, traces, isolation, or release controls around the model. It can add a tool adapter, a worker driver, a deterministic check, a trace sink, a sandbox boundary, a replay path, or an isolated candidate lifecycle. Because those changes affect execution, evidence capture, permissions, and rollback, they need broader tests than a prompt or skill edit.
Find the missing action
Use the trace to identify what the current system could not do.
| Trace evidence | Missing capability | Likely repair |
|---|---|---|
| The model describes a file read that never happened | Repository access or tool dispatch | Tool adapter and observable call |
| A requested second worker never appears | Worker creation or topology | Runtime driver |
| A failed test is mentioned but not run | Execution or validator | Test adapter and hard check |
| A branch changes shared files | Isolation | Sandbox or per-candidate workspace |
| A score cannot be traced to an artifact | Capture or lineage | Trace schema and artifact store |
| A candidate changes its own judge | Outer control boundary | Gate outside the mutable surface |
The table is a diagnosis aid, not a reason to build a larger framework. If the agent already has the needed action and forgets to use it, a prompt or skill change may be enough. Evolve the harness when the missing move is not representable, not enforceable, or not observable.
Anthropic’s public agent architecture guidance recommends starting with the simplest composable system that fits the task. Cognition’s public account of using Devin to build Devin describes recurring playbooks, code review, automated fixes, and session insights as explicit workflow surfaces. The transferable lesson is not a particular vendor architecture. It is to turn repeated operational behavior into a named mechanism with a check.
The harness defines the reachable behavior
Let h be the current harness and M be the set of mutations it can express. The system can search only the behaviors reachable through M. If M edits prompt text, it cannot produce a new worker process, an isolated workspace, or a raw provider capture sink.
The distinction is easiest to see in a table:
| Surface | Search can change | Search cannot create by itself |
|---|---|---|
| Prompt | Instructions, examples, tool descriptions | A tool, process, permission, or durable record |
| Skill | Reusable procedure and checks | An execution engine that invokes the procedure |
| Runtime topology | Worker order, fanout, retry, handoff, budget | New evaluator or storage boundary unless exposed |
| Harness | Tools, adapters, traces, validators, isolation, lifecycle | A trustworthy release decision if it owns the gate too |
The runtime topology article covers the lower boundary. Harness evolution begins when the runtime itself is the object that needs to change.
Structural change is different from knob tuning
Changing a concurrency limit or retry count can be useful. It is ordinary configuration search unless the change introduces a new mechanism.
A structural candidate might:
- Replace a summary-only record with child spans and artifact lineage.
- Replace a single retry with isolated branches and an explicit winner rule.
- Add a deterministic validator before a semantic judge.
- Add a replay adapter that can rerun a failed tool step.
- Move candidate generation into an isolated workspace.
- Attach a stable agent profile to every evaluation cell.
- Add a cancellation path that reaches child workers.
A non-structural candidate might:
- Add “be rigorous” to a prompt.
- Rename a worker from reviewer to supervisor.
- Raise the judge threshold after seeing a result.
- Increase population size without changing the search mechanism.
- Drop trace capture to make the candidate faster.
The difference is whether the agent can now perform, prove, or stop an action it could not handle before.
A candidate lifecycle that can be rolled back
Harness code should be treated as a release artifact. Each candidate needs an identity, a parent, a hypothesis, an isolated workspace, and a result.
freeze baseline
create isolated candidate workspace
apply one structural hypothesis
run a cheap smoke case
run paired development cases
inspect trace and artifact capture
run protected cases outside the search
promote, reject, or quarantine
retain rollback handle and lineageThe smoke case is a cost control. It should catch a missing dependency, broken adapter, malformed schema, or absent trace before the expensive campaign starts. The protected run is a control boundary. It should not be writable by the candidate. The rollback handle is an operational requirement, not a comment in a ticket.
Parallel candidate generation requires isolation. Two variants that share a mutable directory can overwrite each other, pass because one consumed the other’s artifacts, or leave a mixed result whose parent is impossible to identify. A candidate record should include:
| Field | Purpose |
|---|---|
| Parent identity | Shows which mechanism the candidate inherited |
| Changed surfaces | Prevents an unplanned profile change |
| Hypothesis | Makes the result falsifiable |
| Smoke result | Proves the candidate can run |
| Trace coverage | Shows what the candidate did |
| Quality and cost | Supports comparison |
| Promotion decision | Records why it advanced or stopped |
| Rollback handle | Makes activation reversible |
Keep the evaluator outside the candidate change
Architecture search can attack its own measurement because the candidate can change the path being scored. A harness candidate can improve its score by:
- Dropping hard cases from the adapter.
- Retrying until a lucky answer appears.
- Routing around a failing check.
- Preferring judge-friendly prose over a correct artifact.
- Skipping trace capture to reduce reported latency.
- Reading protected examples through logs or error messages.
- Editing the evaluator or release threshold.
The outer rule is:
candidate code can produce behavior
candidate code cannot rewrite the gate that promotes itIf the product needs to evolve the evaluator, that change must be a separate candidate with its own test set, owner, and review. Otherwise the same system can lower the bar and declare itself improved.
The evaluation gates article describes paired holdout comparisons, deterministic checks, cost limits, and judge calibration. The governance article explains why the control surface must remain outside the optimized surface.
What the public Tangle runtime shows
The public Tangle agent-runtime repository describes one runtime used for chat turns, one-shot tasks, teams, and agent improvement. Its README separates the execution loop from the scoring and shipping decision provided by agent-eval and from sandboxed execution. Execution runs separately from scoring and from the decision to ship a candidate.
The public README also exposes a simple improvement shape.
The following is illustrative TypeScript adapted to that public contract; baseProfile, executionRef, method, findings, the scenario sets, judge, and runProfile are application-provided.
import { improve } from '@tangle-network/agent-runtime'
const result = await improve(baseProfile, {
surface: 'prompt',
executionRef,
method,
findings,
trainScenarios,
selectionScenarios,
testScenarios,
judges: [judge],
agent: (candidateProfile, scenario, ctx) =>
runProfile(candidateProfile, scenario, ctx),
runDir: 'support-prompt',
costCeiling: 25,
})
if (result.decision === 'ship') {
activate(result.candidate.profile)
}This snippet follows the repository’s public README shape. It is not a complete product integration because the application supplies the profile, method, scenarios, judge, agent adapter, and activation path. The important boundary is that improvement is a measured candidate operation, not an instruction for the live agent to rewrite itself in place.
When the changed surface is harness code, the same principles apply with a different candidate representation. Run the code in an isolated workspace, record the exact change, execute the real backend, and evaluate the resulting behavior against the frozen baseline.
The research lineage
The idea predates current coding agents. The Gödel machine paper describes a self-referential solver that rewrites its own code after proving a utility improvement. Practical systems usually replace proof with empirical evaluation.
FunSearch made the evaluator-driven pattern concrete for LLM-generated program search. AlphaEvolve extends the idea toward broader code and algorithm search with automated feedback. OpenEvolve provides a public implementation to inspect. The Darwin Gödel Machine paper studies an archive of self-improving coding agents that edit and validate descendants.
These systems benefit from a bounded search domain with an executable evaluator. Production agent harnesses are harder because the objective includes user intent, external tools, privacy, permissions, latency, and a release process that cannot be allowed to mutate freely.
A harness variant should make a visible prediction
Do not start with “make the harness smarter.” Start with a testable hypothesis.
| Hypothesis | Structural change | Falsification |
|---|---|---|
| Retrieval failures come from missing source spans | Capture raw retrieval inputs and selected spans | No reduction in missing-source failures |
| Reviewers miss deterministic failures | Run executable checks before semantic review | Hard failures remain unchanged or cost exceeds limit |
| Parallel workers corrupt each other’s work | Isolate workspace and state per branch | Conflicts persist in isolated runs |
| The winner rule rewards style | Replace it with a deterministic artifact check | Quality does not fall when style cues are removed |
| Retries hide provider failures | Record backend status and fail incomplete rows closed | False agent failures disappear |
The prediction makes the candidate useful even when it loses. A failed hypothesis tells the next search which mechanism did not cause the observed result.
Profiles and traces keep comparisons honest
An agent profile is the identity of the run conditions. It should capture the model, prompt, tools, skills, permissions, runtime topology, harness revision, and evaluator identity that can affect behavior.
If the candidate changes three of those at once, the score cannot attribute the gain to one mechanism. If the candidate changes a harness but the profile still says “baseline,” the comparison record is incomplete.
The trace should connect the profile to every child branch, tool call, artifact, validator, and final decision. An execution span is one timed operation in that trace, such as a model call, tool call, or check. OpenTelemetry’s trace model is a useful general vocabulary for parent spans, child spans, and distributed context. Agent traces need domain fields on top: task identity, profile identity, tool arguments, artifact hashes, evaluator results, and cost.
Keep service deployment controls outside candidate promotion
If the evolved harness runs inside a Tangle service, a Blueprint is the reusable service template. A Service is a live instance created from that template. An operator is the party that runs its live Service. A Router maps a service job to the handler that executes it. Those outer objects do not prove that an internal harness variant is safe.
An x402 request authorizes and settles an optional HTTP payment before a job is enqueued. An attestation is signed evidence about code and hardware in a trusted execution environment. Neither proves that a changed harness produced a correct result. The evaluator still needs the trace, deterministic checks, and task evidence.
This is also why a harness candidate should not silently change payment, permissions, or isolation mode. Those are separate risk dimensions that belong in the profile and the promotion packet.
Failure modes after the harness evolves
Evaluator capture happens when the candidate learns the judge or test adapter.
Trace regression happens when the new path is faster because it records less.
Isolation leakage happens when branches share files, credentials, memory, or browser state.
Lineage loss happens when a merged candidate cannot identify which parent or hypothesis produced a change.
Backend confusion happens when provider failures are scored as model failures.
Rollback gap happens when the candidate can be measured but not safely activated or removed.
Complexity debt happens when the new mechanism adds more failure modes than it removes.
The right response to complexity debt is not to hide it. Measure the new branch count, latency, operator work, and failure surface. If the simpler baseline performs as well, keep the simpler baseline.
When to evolve the harness
Evolve the harness when the trace shows that the current system cannot:
- Execute the requested tool or worker.
- Isolate mutable state.
- Enforce a validator or budget.
- Capture enough evidence to diagnose failure.
- Replay a critical step.
- Compare candidates on the same profile.
- Stop unsafe or wasteful child work.
Do not evolve it merely because a prompt feels inelegant. The optimization theory article explains how to choose the causal mutable surface.
The durable lesson is narrow. Every optimizer searches a representation. Prompt search explores strings. Skill search explores reusable procedures. Runtime search explores execution shapes. Harness evolution explores the code that defines actions, observations, checks, records, and candidate lifecycle.
When the current representation cannot express the desired move, widen the representation first. Then freeze the gate, isolate the candidate, capture the trace, and let the mechanism compete against a hard baseline.
What is harness evolution?
Harness evolution changes the software around the model, such as tool adapters, drivers, validators, trace capture, replay, isolation, and candidate release flow.
How do I know a prompt is not enough?
Look for a missing or unenforced action in the trace. If the runtime has no tool, worker, workspace, check, or record needed for the behavior, prompt changes cannot create it.
Does harness evolution mean rewriting the whole system?
No. The best change is often one missing adapter, one hard validator, one trace field, or one isolation boundary. Change the smallest mechanism that can produce and prove the desired behavior.
What should be protected from the candidate?
Protect the holdout cases, evaluator implementation, release policy, credentials, shared state, and activation path. The candidate may propose a change, but an external control process should decide whether it persists.
What is the minimum proof before a large search?
Run one cheap smoke candidate through the real backend, confirm that traces and artifacts are captured, and verify that the release gate can reject it. Only then spend on a larger population or long campaign.