From Telegram workflows to AI programsJournaling experiments with n8n and Telegram.
Building Idyllic - Prior Iterations
March to July 2024
I was trying to make AI useful across a process that lasted longer than one conversation. My first working example was a journaling workflow: I sent a message to a Telegram bot, and n8n added it to that day's Notion page. Redis carried messages between the services, and a separate workflow could send the same material to mem.ai.
That solved capture. I could get a thought out of my head without opening a document and finding the right place for it. Using the accumulating material was a different problem. An AI response would need context from earlier entries, and improving that response would require remembering what had produced it. The next experiments followed those requirements into feedback, process state and eventually a small program format.
Connecting feedback to the answer that produced it
Suppose I told the Telegram bot that an answer had missed something important. By the time I sent that feedback, I might already have changed the prompt. The journal might contain new entries. Sending the feedback alongside the current prompt would give a refinement process the wrong evidence: it would be judging instructions and context that had never produced the answer I was criticizing.
My feedback design therefore saved the intermediate records. A template and a set of context produced a rendered prompt. The model invocation saved that prompt and its output. When the workflow delivered the answer through Telegram, it attached the Telegram message ID to the invocation.
A reply to that message could then point back to the exact invocation. The proposed feedback record included a good or bad judgment and an explanation. That explanation mattered because “bad” alone gave the system little direction. An answer could be factually wrong, omit relevant context or use the wrong format, and each failure would suggest a different change.
I sketched a refinement step that could revise the instructions or add examples to the prompt. I was also considering changing one variable at a time so that a later result could be compared with the earlier one. The loop was still a design, but its storage requirement was clear: useful feedback had to remain attached to the prompt, context and output it concerned.
Giving an assistant a place in a process
A sequence of model calls becomes more complicated when a person participates between steps. Julius and I needed a concrete process to explore, so we used his design proposal workflow. He was focused on the user experience, while I was working on the developer tools that could support it.
A request first needed Julius's decision about whether to pursue it. Research and a meeting established what the client wanted. A rough proposal collected deliverables, milestones and estimates. Negotiation revised that proposal until there was an agreement, after which the work produced tasks, files and progress updates for review.
This gave the assistant specific jobs. It could research an incoming request, extract information from meeting notes or turn agreed details into a proposal using a template. Julius still owned the decision to proceed. The process stage could determine which materials the assistant needed and which actions were appropriate. Preparing a proposal before a scope had been agreed was a different operation from updating tasks during the work.
I modeled those stages in XState and brought the model into TypeScript. The state machine described the process; a running actor represented one instance moving through it. Messages could tell that actor that something had happened, such as a proposal being accepted. I had the stage model working before I had useful agent behavior attached to it.
The proposal also exposed something that a stage diagram left out. A negotiation might change the deadline and remove a deliverable while remaining in the same stage. “In negotiation” would still be true, but an assistant preparing the next proposal needed the revised terms. The current context had to change even when the stage did not.
That led me to distinguish the history of what had happened from the current account of the work. A stored conversation could preserve how the deadline changed. The next action needed the deadline now in force. I considered an event log and processes that would use new events to update state, while leaving open how strict the overall process model should be.
Describing a program independently of its executor
The workflow experiments also made me interested in generating interfaces. In a small text-to-form experiment, I used a nested Zod schema to describe the form elements a model could return. The model produced a structured specification before the result became HTML. The schema gave the form a constrained representation that application code could interpret.
I wanted a similar separation for the operations behind the interface. A saved program could describe its steps as data, while an executor supplied the code that ran them. I built a pure-data representation and a Python executor, then tested an essay-ranking script that processed essays in parallel.
Writing that small program exposed practical gaps. The format could assign a value to one variable name, but could not yet express all the assignments I needed. It also lacked string interpolation. A literal instruction and an instruction containing the result of a previous step needed different treatment. These were problems I could find by trying to express a program, before asking a model to generate one.
A proposed essay workflow made the larger requirements easier to see. It would generate three essays and display them for a person to choose from. The program would retain the selection, run critique and revision steps on that essay, then turn the result into a tweet. I had tested the ranking component; this longer interaction was the design I wanted the program format to support.
The selection step changes the runtime. Generation can finish while the program waits for a person. Their choice becomes an input to the next operation, so it needs to be stored somewhere the executor can read. The interface also needs to know when to display the essays, when to accept a selection and which result to show during revision. A chain of model calls alone does not describe those interactions.
The architecture drawing separates a composer, a program description, an executor and a database that supplies state to the interface. I hoped an AI composer could eventually help create the programs. My immediate job was smaller and more concrete: write a useful program myself, run it, and discover what the format could not yet express. Waiting for a selection and carrying that selection into a later step were requirements the essay example could actually test.