Will ChenFrom Telegram workflows to AI programsTelegram feedback loops, Julius’s proposal process and the first experiments with AI programs represented as data.
Building Idyllic - Prior Iterations
March to July 2024
In March 2024, I made a small workflow that let me journal through Telegram. A message to the bot became an entry on that day's Notion page. I used n8n to connect the services and Redis to pass messages between them. The capture step worked, although I had not yet figured out how to use the accumulating material.
This became one of the starting points for Idyllic. I wanted to build AI tools around information that persisted between conversations. A journal entry could become context for a later question. A response could receive feedback. The next invocation should be able to use what happened during the previous one.
A feedback loop for Telegram
On April 11, I drew a feedback system for the n8n workflow. A template and some context produced a prompt. The model generated an answer, which the workflow sent to Telegram. Alongside that answer, the system would store the prompt, the output and the Telegram message ID.
Keeping the message ID mattered because feedback arrives after generation. If I replied that an answer was useful, or explained what it had missed, the system needed to connect that feedback to the invocation that produced it. The second page sketches a refinement loop that would use those records to change the prompt or add examples. The drawing still has questions in it. I was specifying how the loop might work, with the data model chosen to make refinement possible.
An assistant in the proposal process
At the same time, Julius and I were considering how an AI assistant could participate in a longer process. Julius was working on the design and user experience; I was particularly interested in the developer tools that would make these systems easier to construct. We needed a process concrete enough to describe, so we used his design proposal workflow.
A client request starts with a decision about whether to take it on. Research and a meeting establish what the client needs. A rough proposal identifies deliverables, milestones and estimates. Further discussion revises it until there is an agreement. Work then produces files, progress updates and material for review.
An assistant could help at several points without owning the whole process. It could research a request, extract useful information from meeting notes or prepare a proposal from an agreed template. Julius would still decide whether to proceed. Those boundaries gave us a way to distinguish an AI action from a human decision and to specify which information each stage required.
I modeled the proposal process as a state machine and imported it into TypeScript on April 14. That represented the stages and transitions; attaching useful agent behavior was the next task. I also started a visual editor with React Flow and tried a chat interface. By April 15, a terminal chat with message history worked, while the dynamic context and task interactions were still being worked out.
The proposal exercise also exposed a limit in a strict process diagram. Proposals accumulate information through negotiation. Their meaning changes while their stage may stay the same. My notes considered a looser description of the process that could evolve with the work, while keeping the state machine as a concrete place to begin.
Programs represented as data
By July, I was exploring a different part of the same construction problem: how to describe an AI program independently of the application running it. One experiment generated a form from a request. It passed through a planning step and a structured specification of the elements before reaching HTML. I used a nested schema to constrain what the model could produce.
The form experiment put an interface at the end of a model pipeline. That raised a practical question: could the pipeline itself become something a person could save, edit and run again? I began representing programs as data, separate from the Python code that executed them. An essay-ranking script gave me a small working example, including parallel processing of the essays.
The architecture drawing from this period places the program description between a composer and an execution engine. The engine runs the steps, while a database carries state back to the interface. The wider drawing includes a proposed AI composer; the working experiments at this point were narrower, with programs I had written and tested myself.
A description of a program could eventually support several ways of making it: writing it directly, editing a diagram, or asking AI to help construct it. Before that, the runtime needed to execute it and the interface needed to make its results usable. A form was one possible result. An essay ranking was another. The next designs asked what happens when the result also needs buttons, selection, progress and a place to keep its own state.