A mobile builder for personal agentsEditing an agent's routines and data from a phone.
Building Idyllic - Prior Iterations
2024 and 2025
A personal agent needs to be available while someone is using it throughout their day. A plan changes away from the desk. A note occurs to someone while they are out. If every adjustment to the agent requires a laptop, its builder is absent during much of the experience it is supposed to support.
I began exploring a mobile version of Idyllic in 2024, and by 2025 I was working on a builder where people could assemble agents on the phone itself. An agent had a system prompt, reusable functions and persistent data. The challenge was to make those parts editable on a narrow screen while keeping the resulting behavior understandable.
Adding the resources an agent needs
The agent screen separated its prompt from the components attached below it. The Add menu exposed a small set of component types, then each selected component could have its own configuration. This kept the overview readable without requiring every setting to fit on one screen.
The components supplied different kinds of resources:
- Functions held repeatable procedures made from blocks.
- Notes gave the agent documents to consult or update, with a description of what belonged in each one.
- Tables gave it structured records with named columns.
- Integrations and MCP servers appeared as ways to connect external capabilities.
Coachbear illustrated how the parts could work together. Its configuration contained a Morning Routine function, notes for ongoing observations and a Food Log table. The table had columns for Name, Timestamp, Calories and Notes, plus a description telling the agent when to add an entry. Its card displayed a Read/Write setting beside the schema.
Asking an agent to keep a food log leaves open where the information should go. Attaching a table supplies a particular destination and tells the agent what a record contains. A later question can refer to the accumulated records. The notes provide a different destination for observations that belong in prose rather than columns.
This made configuration part of the agent's usable environment. The person could inspect the objects the agent had been given, open their settings and change what was attached. The overview showed more than a prompt because the agent's behavior depended on more than its instructions.
Keeping the program readable on a phone
I liked the iOS Shortcuts programming model because it was linear enough to read on a phone while still supporting loops and conditions. A procedure could proceed from top to bottom, with extra structure appearing where the work needed it.
The Morning Routine editor followed that approach. Every operation occupied a row containing its name and inputs. Earlier results could appear as blue references inside later inputs, so the procedure could pass generated text between steps without copying that text into several prompts.
The visible sequence starts with generation and extraction. A For each item block takes the extraction result as its input. Inside the loop, another generation step precedes an If/Otherwise branch. One branch contains a notification. The alternative contains generation and web search before its notification.
The layout explains several relationships at once. Vertical order shows the sequence of operations. Indentation places actions inside the loop or branch that controls them. The End If block marks where the conditional finishes, and the blue references show which previous result a later action uses. The prompts are partly clipped in the screenshot, but the control flow remains readable.
This was the property I wanted from the editor: a person could look at the program and follow how an intermediate result would be used. The procedure could become more complex without immediately requiring a wide canvas of connected nodes.
Letting AI edit the visible procedure
Small changes could be made directly, but a larger revision might require many taps. I proposed putting an AI toolbar above the keyboard, where someone could describe a change and have the assistant modify the blocks in place. Undo would remain available afterward.
Both editing methods would act on the same procedure. A person could ask the assistant to alter it, inspect the changed blocks and then continue editing directly. The visible program would preserve the result of the conversation instead of requiring someone to infer the current behavior from a history of requests.
Julius and I were still working through the split between building an agent and using it. A chat with the finished agent should perform work; the editing assistant would change how that work happened. We also deferred bringing the builder into one mobile iteration while prioritizing the user-facing experience. The toolbar was a design proposal, and the supplied screens capture several parts of the mobile exploration.
The phone itself offered useful capabilities for personal agents. Location could supply context, push notifications could bring someone back when input was needed, and health data suggested applications using information already recorded on the device. I wanted the authoring experience close to those uses, so a person could adjust their tool where they were using it.
Generating a calculation from a new question
Tables opened a different route to useful behavior. A saved function could describe work someone already knew they wanted to repeat. A question about accumulated data might call for a calculation the person had never specified before.
When I was choosing how to support those tables, one observation mattered: AI generated MongoDB's query language well in my experiments. I expected modest personal tables with frequently changing columns, and I wanted the agent to work with their contents. The query language was therefore part of the interface I was choosing for the agent.
The Food Log experiment showed an aggregation generated during a conversation. An aggregation is a sequence of database operations that transforms records into a result. The tool card exposed stages including sorting and grouping, then returned two groups named quick_followup and spaced_out, each with a count and an average calorie-change value.
The agent could formulate a calculation against the attached table and use the returned values in its response. I did not have to build a dedicated reporting function for that exact question. The table supplied the stored records, its schema described the available fields, and the generated query selected the operations to perform on them.
The response described how one food choice affected the next, which went beyond what the calculation established. Group averages can show a difference in the recorded data without proving that one choice caused another. The tool card made the calculation's output visible enough to separate the returned numbers from the agent's interpretation.
These two forms of programming could coexist in the same agent. A saved function preserved a procedure the person wanted to specify and inspect. A generated query could answer a new question over records already present in a table. On the phone, the builder exposed the procedures and resources, while the conversation gave the agent a place to put them to use.