Working backwards from the mobile API

An abstraction carrying three jobs cannot be argued out of one of them. Building the product for a phone deleted the argument along with most of the codebase, by my own count over 80k, and everything the product did survived.

Motivation

The project had spent a long stretch on one idea: a document could be a program. You wrote something that looked like a page of notes, the parts of it were blocks, and the blocks carried behavior, so the same object was your memory, your logic, and your schedule at once.

That unification was the interesting claim and it was also the thing quietly making everything hard. A block that has to be all three has to satisfy three sets of requirements, and you cannot see the conflict from inside the abstraction.

Pulling the execution part out into its own package tests that directly, because the size of the seam is a measurement rather than an opinion about architecture:

fully refactored the idyll engine. it's its own package now. the SaaS part of the project is now decoupled and the bridge code is centralized in less than 400 lines of code, well structured. that bridge code parameterizes the engine and connects it to the components and routes and db etc. so all the hard lifting and semantics and processing is truly in the engine.

Four hundred lines of glue is few enough to read in one sitting, so the two really are separable.

Two dependency choices then reversed, both on the same axis:

• Pusher out, a plain WebSocket server in. Pusher solves fan-out at a scale I did not have and charges a hosted dependency for it.
• Supabase back onto Convex. The requirement was the subscription model, not the storage. Convex pushes query results to clients when the underlying data changes, so the event-sourced function streaming that had taken three long days to assemble against Pusher took five minutes against Convex. A dependency chosen for its storage and kept for its push semantics was chosen against the wrong axis.

The mobile constraint

A phone screen shows one thing at a time. Three jobs cannot share it, so an abstraction carrying three has to declare which one it is on any given screen, and that declaration is a ruling on what a document is allowed to be:

then i thought: "wait why dont we just make documents just documents.. no blocks"

The document keeps the memory job and gives up the other two, which move somewhere they can be seen.

Splitting documents, prompts and functions

ur goign to laugh at me again. I did an insane refactor, kind of ditched the entire codebase and just worked backwards from the mobile API. now it's on hono and we're not using vercel anymore, just everything on railway. reduction in code: over 80k. core functionality is all still there.

The stack moved with it: Remix out, Hono in as the server framework, Vercel to Railway. Once the product is a phone talking to an API, a framework organised around server-rendered routes is carrying weight for a shape that no longer exists, and the hosting question stops being about page delivery and becomes one about persistent connections and consistent latency. Those are the axes I compared the two on, and edge distribution is what the move gave up.

The method is in one clause. I worked backwards from the mobile API, meaning the surface the phone actually needed came first and everything upstream was rebuilt to serve it, rather than the server being designed and the phone being given whatever it exposed. Most of what went was not wrong. It was answers to questions the phone never asked.

The conversation block

Splitting the block leaves the question of what replaces it. The granular model composes an automation out of small pieces ahead of time, which fixes every step before the user has said anything. What people needed was the opposite: work that runs after each reply and depends on what the reply was, and a pre-composed automation has no slot to put it in.

rather than the granular [approach] we created this new block structure that is easy to define in an iOS shortcuts like interface. It's called the conversation block, a start conversation and end conversation. the things that go inside the conversation block are basically on every message after the user response. you can insert operations to manipulate the context so that has access to the history. the insight is that you're not supposed to try to make a granular automation, it's mostly about how to script conversational [flow]

The structure is a start, an end, and a slot between them, drawn here as shape rather than notation, since the notation is not in the record:

start conversation
    ... blocks here run on every turn,
        after the user replies and before the assistant answers
end conversation      <- decides, from what those blocks produced,
                         whether the conversation is over

That middle position is the useful invention. It is exactly where you want to update a note, fetch something, or change what the assistant can see, and no ordinary chat interface gives you a place to stand there.

A refinement made the closing block conditional rather than fixed:

and the endconversation block can use intermediate results of blocks inside that scope to determine to end or not

A loop that waits to be stopped becomes one that ends when the work inside it says so. The finished conversation is then context for whatever block runs next, which is what makes them composable.

What the split cost

feels bad to kinda nuke 2 months of forcing the unified prompt model

Forcing is the load-bearing word. A model being forced has stopped producing answers and started being held in place, and a form factor is what makes the holding visible.

The passive document did not last either. Making documents just documents simplified a great deal and lost the thing we still wanted, which was functions, and another project shipping the same simplification is what made that loss legible. The mobile app became its own thing, with a modular approach to declarative agent building underneath it.

What the constraint settled

A form factor is a cheap way to run an experiment that theory runs badly. Six months of argument about whether one abstraction was carrying too many jobs produced no verdict. One narrow screen produced a verdict in under a week, because a screen cannot be argued with.

A design question that will not resolve on argument can usually be handed to a physical constraint instead.