Banning the word 'context'

A multi-step AI process reads a document, asks a model something about it, takes the answer, does something with it, and feeds the result into the next step. The open question is what travels between those steps.

• A block of text. The obvious answer and a bad one. The next step has to parse it back into something usable, and parsing prose is where these systems break.
• JSON. Worse, for a different reason. The thing being passed around is often a document that a person also needs to read and edit.

So the type stays a placeholder called IdyllicValue, standing for whatever the right answer turns out to be. It will not specify, and the reason is one word in the specification doing four different jobs.

Motivation

The specification will not close. Every path forward opens into more paths, nothing forces a choice between them, and the problem grows each time you look at it directly. That is a superposition rather than confusion, and the difference decides what to do about it: confusion has a wrong answer waiting to be corrected, and here there is nothing to correct:

the idea is that we want to keep it simple (keep it primarily agentic) but have clear semantics about how context is implicitly transferred. i think this is what i'm missing clarity on, because if we want the agentic process to potentially work for 20-50 steps the context between should be light? so i was thinking just references, but fuck when I think about this i can sense it getting extremely cloudy again and exploding into superposition of possibilties

The phrase carrying the damage is "clear semantics about how context is implicitly transferred". Every load-bearing word in it is vague, and the vaguest is the one the design leans on hardest.

The overloaded term

Context, in that sentence, stands for four separate things with four different lifetimes.

A word that means four things cannot be given a type, so the type system stalls until the vocabulary does. The fix is blunter than the diagnosis: rather than define the word more carefully, ban it.

let's decide to not use the term "context" in any form in order to be more clear, please offer precise alternatives

Removing it from the vocabulary entirely, rather than agreeing to use it carefully, is the part that does the work. A word you are allowed to keep will keep absorbing whatever is nearby, because that is what a comfortable word does. Taking it away forces every use to declare which of the four things it meant, and the ones that cannot declare were never saying anything.

The replacement vocabulary, and the verdict on each name:

1. environment / scope
2. resolveddocument
3. llmrequest (cuz we can put things like provider / temperature / tools)
4. ExecutionState (it feels like environment -- but we could always merge later)

• environment / scope and resolveddocument, taken as offered.
• llmrequest, renamed, and the parenthesis is the argument. The thing carries provider, temperature and tools alongside the text, and a name that says payload keeps hiding them.
• ExecutionState, accepted with a doubt attached: it might turn out to be the environment under another name. Recording the doubt costs less than resolving it early.

Disambiguating a term this way is a hundredfold amplifier, "the same way we don't use the term agents or memory."

Reducing to one type

Banning a word tells you how many things were being confused. It does not reduce how many things there are, and the type system still carries a container for every use.

The design it inherits is semantic objects: typed containers the model holds a reference to rather than reading, one per kind of thing. A journal collection, a blog post, a health record, each with its own fields and methods. It works, and it costs a new type for every new use.

The collapse is to a single container, and the clause that makes it a simplification rather than a retreat is the parenthetical one:

the document is going to be the first and only semantic object we focus on, perhaps later we'll bridge the concept but this is the only container we need to focus on even if generic because I think we can just bring our wishes into the documents (imagine defining custom functions on the document etc)

The argument for many types is that different things need different operations, and a journal collection genuinely does need different methods than a blog post. Move the functions into the container and the variety moves with them, out of the type system. One generic thing that holds custom behaviour covers what a dozen specific things were covering, and nobody has to design a type before they can start.

Specifying the container first

With one container to design, the next move looks obvious: specify what the container guarantees. The specification that comes out is a list of eight properties an IdyllicValue should have.

• structured rather than plain text
• connected, carrying citations back to its sources
• intelligible to a model
• readable by a person
• operational, meaning it can carry things that execute rather than only things that describe
• composable
• editable
• metadata-rich

Building it property by property, each with the smallest demonstration that it is real, is the plan that follows from such a list. It fails immediately, and the mechanism is general:

I don't know what the fucking IdyllicValue structure is, we have to figure this out by necessity therefore it can't be the fucking first thing we do, it's a superposition to collapse

A data structure is determined by the operations performed on it. If those operations do not exist yet, specifying the structure means guessing which operations will matter, and the guess fails in the way that is hardest to detect: the structure comes out exactly good enough to build the things you imagined, and awkward for everything you did not.

The test that catches the same error early asks what a design proves:

what are we trying to prove here, structured output without json? nothing here really screams "single LLM call couldn't solve that" why do we even need this parseability

If a design cannot show something a single model call could not have done, it has not yet earned its complexity, whatever its properties list says.

Reframing to operations

Replacing the noun question with a verb question is what unblocks it:

instead of thinking of the 8 properties of IdyllicValue, let's think about 8 properties of documents as computational substrate. what is necessary? and then let's think about 8 tools that we want to give AI to manipulate. for example adding annotations is KILLER feature because it opens up automating annotations as a UX primitive. like, what if I want to create a system that auto-grades my korean lesson quizzes?

"What data structure do we pass" is a question you can answer wrongly for a long time without noticing. "What can an AI do to a document" produces a list you can start building on Monday, and each item on it fails visibly.

Annotation is the operation worth building first, because an annotation binds to a specific block of a document that a person is looking at. That binding is available only because the editor underneath, BlockNote, models a document as a list of addressable blocks rather than as a stream of rich text, so a block already carries an identity a comment can attach to. The primitive exists because someone else had already made the document addressable.

A model that can attach a comment to a paragraph has a way to say something that lands in the place it is about, and a person can see it, disagree with it, and delete it. That makes automated commentary an interface rather than a feature. The Korean quiz grader is the smallest complete version: the quiz is a document, the grading is annotations on it, and the thing worth using exists as soon as those two work.

Making documents executable has a cost. If a document can contain instructions and an agent reads documents, then anyone who can write to a document can instruct the agent. Homoiconicity, the property of a system where the program and the data have the same form, is a lovely thing to have and it hands you a prompt injection surface for free. Any primitive that generates from a document has to be scoped with that in mind.

What it cost

What travels between steps is the document: one generic container, addressable at the block level, carrying its own functions, with the operations on it settled before the structure of it. The boundary that keeps that honest is that the interchange value is not the editor's document model. They are different objects, and conflating them puts the editor's shape into the protocol.

Two working habits come with it, and both are about when a decision gets made rather than about what to decide.

• Ban the word that is doing too much work and see what has to be invented to replace it. It is cheaper than an architecture and it fixes the same class of problem, because a specification you cannot write in precise words is not a specification you have.
• Build the concrete case, then let the shape fall out of it. The structure of the thing passing between steps cannot be designed in advance, because it is determined by operations that have not been built.

The second is the same error as designing a compiler's intermediate representation before writing any of the code it is supposed to produce. The representation gets shaped by imagined uses, the real ones arrive later, and they do not fit.