Convergence

Motivation

Twenty coding agents produce twenty divergent threads and nothing that brings them back together. What you get is a grid of terminal panes, each sensible on its own, and a person carrying the only copy of the plan.

A terminal multiplexer with no visualisation really bothers me. It gives you panes and withholds everything else:

• convergence logic
• stopping criteria
• branch visibility
• merge review
• summarising where a goal has got to
• controlled resource allocation
• better branch management
• better convergence surfaces
• explicit stopping conditions

Every one of those is something a person currently does by remembering.

Git already models half of it. A branch is one attempt, a pull request is a candidate for admission, a merge is the decision, and a conflict is two attempts claiming the same lines. What git has no field for is what the attempts were for, or when there have been enough of them. Both stay in your head, and reading twenty diffs against a target you are holding from memory is where the ceiling actually sits.

So the harness stores the parts git leaves out:

• a written target, with acceptance criteria and a termination condition
• a queue that admits one candidate at a time, so review stays sequential
• a rule for the good parts of a branch that does not get merged

A model can do the reviewing. It cannot do it against a target nobody wrote down.

The coordination cost

Twenty agents is harder than five because the work you are doing is not the agents' work, it is the coordination between them, and coordination grows with the square of how many there are. Five things that must agree have ten pairs. Twenty have a hundred and ninety. Working memory does not grow to match, and mine capped out at about three agents, because working memory was the thing doing the coordinating.

The square only appears if you coordinate at the level of what each agent is individually doing. Coordinate the state they are collectively trying to reach and another agent adds nothing to what you hold. The destination does not get bigger. The agents sit below the level you are managing at.

Same move statistical mechanics makes on a gas, which is why the vocabulary stuck: track temperature and pressure, not particles, and the particles are free to be as numerous as they like.

The object model

The harness has five objects, and the interesting property is which of them get names and which do not.

The lifetime column carries the design. The agent doing the work is the row marked ephemeral; the destination and the record never get erased. The minds are anonymous and memoryless, with no mechanism for one to know about another. In the spec I called them water, which sounds decorative and is doing real work: the point of water is that you never care which particular water.

The spec states the inversion against Gas Town: Gas Town manages labor, convergence manages a watershed. The two designs differ on one axis, which is which side of the arrangement is expensive to preserve:

• Gas Town gives workers names, identities, memories and roles, and lets tasks flow past them. The worker's accumulated understanding is the asset. This is coherent and it is where most of these systems land, because it maps onto how a team of people works.
• Convergence persists destinations and treats workers as disposable and interchangeable. The target's accumulated progress is the asset.

Neither is more correct in general. Having watched twenty agents produce nineteen branches I did not want, I wanted the target preserved, which is also what makes it affordable to run twenty and throw away nineteen.

Mergeable results

Parallel work does not require parallelisable problems. It requires mergeable results.

The usual objection is that the problem does not decompose. Most interesting problems do not. But you do not need the problem to split into independent halves. You need to run several whole attempts and combine what each got right, which is a much weaker requirement.

Everything can run in parallel except the merge. The merge is sequential, gated by a human, and the entire ergonomics of the thing point at that one valve.

The first live drive, three minds on one task:

• Approaches, deliberately different: minimal, library-first, state machine.
• Isolation: one checkout each, a separate folder on disk holding the same project at a different version, so all three edit the same files without ever seeing each other.
• Tests: each mind its own.
• Result: all three produced candidates, the library-first branch was grafted, and the malformed-input tests from a pruned branch, better than the winner's, were grafted on top of it.

Selection is a person at the lock, and the event vocabulary forces the reason to be written down: a prune requires a reason, a harvest requires the residue. From the ledger of a later run, where the harness was building itself, the reason recorded on one prune:

lost to .2: same core helper, but .2 records branch.reconciled on the ledger with reducer legality — reconciliation is a transition

and the residue harvested beside it:

reconcile.1's surgical version was correct but recorded nothing — transitions belong on the ledger

Both branches solved the stated problem, and both passed their own tests. The axis that separated them was whether the solution recorded a transition on the ledger, which is a property of this system rather than of the task, and it became visible because two attempts differed on it.

Selection did not happen at the level of whole candidates. The loser contributed. Once you can merge below the level of the whole attempt, running many attempts stops being wasteful, because a branch you throw away can still leave its best part behind.

Design principles

Six rules held the implementation down, and five of them are about refusing to build things.

The ledger is the only truth. One file per project, append-only, so every line is a thing that happened in the order it happened. There is no stored answer to "what is the state right now". You get the state by replaying every line from the top, so the state cannot disagree with the history: it is made out of the history each time you ask.

That buys the property the whole system rests on, no second store and no cache that might drift out of step, and it makes the failure case boring. An append-only file can be caught mid-write when a process dies, so the last line is half there and the reader skips it rather than falling over. Nothing earlier is affected, because nothing earlier was ever going to be rewritten.

The whole event vocabulary fits on a screen, which is the test that the design stayed small:

basin.declared    {basin, title, macrostate, criteria[], termination, budget}
basin.opened      {basin, wave}
branch.spawned    {basin, branch, approach, gitBranch, worktree, model}
branch.searching  {basin, branch}
branch.candidate  {basin, branch, summary, evidence[], commits, usage}
branch.failed     {basin, branch, reason, usage?}
branch.grafted    {basin, branch, mergeCommit}
branch.pruned     {basin, branch, reason}
basin.converged   {basin, note}
basin.killed      {basin, reason}
basin.harvested   {basin, frontier[], residue[]}
frontier.added    {question, from}

The commits field on branch.candidate does more work than it looks like. A mind claiming it is finished with nothing committed gets rejected by the runner rather than believed, which is the shape of every guarantee here: the claim is checked against the git history rather than taken from the agent's own account of itself.

The interface is a pure projection. It renders derived state, never writes, and can print one frame and exit, which makes it scriptable and makes it safe to run beside a live engine.

Git is the artifact store. A branch's work product is its git branch, and merging is git merge --squash. Diffing, history and conflict detection already exist and are better than anything I would write.

Minds are subprocesses rather than a framework. One non-interactive agent run per worktree, with the command injectable, so the engine underneath is swappable and the tests are hermetic.

Some mistakes are impossible to write down, wherever making them so was cheap. A destination cannot be declared without a stop condition, because the function that creates one refuses to return without that field. A branch that has not produced a candidate cannot be merged, because the code that advances the state refuses that transition. Neither rule lives in a document anyone has to remember.

No daemons. The runner is a foreground process that fans minds out and appends events as they finish, and the interface polls the ledger file rather than being pushed to. Nothing runs when nothing is running, so there is no background state that can get out of step with the file.

Six minds built a working demo command-line tool, three competing on the core and three more running as concurrent basins grafted sequentially through real conflicts. The ledger survived concurrent appends from three processes, which is the property append-only was bought for. The harness was then used to build itself.

Build or adopt

The question stops being how to build this and becomes which parts of it are mine to build at all.

So assume the commodity infrastructure is already in hand and costs nothing, then ask what still does not work. The branch-and-pull-request flow gives you branch visibility, merge review, a sequential valve, and a permanent record of what changed and why. An issue tracker plus a prompt holds a destination and its history. A worktree gives an agent an isolated working environment, which is a pattern I already run.

Three things still break:

• No live view. Git tells you what happened, never what is happening right now.
• No timeline across parallel efforts. History is per branch, so there is no day-by-day read of several pursuits at once.
• Memory belongs to a session, not to a pursuit. Close the session and the accumulated understanding of the goal goes with it.

Those three are the object. The general form of the move is the transferable part: when you are building a system, the first useful question is which of its components are commodity infrastructure wearing an unfamiliar name. The branch isolation, the merging, the history and the conflict detection were a worse version of git, and those were most of the code. The ledger, the basins, the frontier and the lock semantics were not.

The unit of work

The last piece came from watching rather than designing: one long-lived session per goal, alive across weeks, carrying its own context, ending when the goal is met or abandoned.

That is the unit, and the two things it is not fail on the same axis, which is that neither has a lifetime tied to the goal.

• Not an agent. Partitioning by persona produces a roster of named workers, and a persona has no relationship to what is being reached for. Partition by goal instead.
• Not a run. A run is ephemeral by construction, and the accrued context is the asset. One tab had six hundred and twenty-four thousand tokens of a single pursuit in it, which is well past the point where starting a clean session is a saving rather than a loss.

Session duration is goal lifetime, which makes the whole thing a version control graph with the nouns substituted:

The substitution is the object I had been building toward: the nodes are not notes about goals, they are the sessions pursuing them.

The sentence I keep trying to make precise is that intelligence is divergence multiplied by convergence, and it is philosophy rather than mechanism, which pisses me off. The half I can defend mechanically is the second. Divergence is easy. Convergence is where the work is, and the systems that will matter are the ones that make many attempts cheap to combine rather than cheap to launch.