How I rebuilt a blockchain's documentation

Motivation

I joined Terra in January 2020 as the person responsible for developer experience, meaning that if an outside programmer wanted to build something on the chain, everything they read on the way was mine. The chain already existed and had users. It had no way for a stranger to find out how any of it worked.

Three kinds of reader want different things:

• Validators, running the machines that keep the network alive. They need operational instructions and will lose money if the instructions are wrong.
• Integrators, writing programs that talk to the chain. They need a reference for every message the chain accepts.
• Newcomers, who have heard of the project and want to know what it is before committing an afternoon.

One site has to serve all three without any of them feeling like they are reading somebody else's mail.

What a developer needs

Four properties of a developer's experience, taken from an API design book and ranked. The order is the content, because it inverts what a documentation site looks like it should optimise for.

• Clarity, meaning intuitive visuals, which I put last.
• Ease of use, meaning quick access and speed.
• Stability, meaning reliability and consistency.
• Function, meaning expected behaviour, which I put first.

The line I wrote under it: focus on function first and form second. Putting visual clarity last in a list about developer experience looks wrong until you have watched somebody try to use a beautiful reference that is missing three endpoints. A programmer reading documentation is not browsing. They arrived with a specific thing they are trying to make the machine do, and the site either contains it or it does not. Everything else is a tiebreaker.

I also wrote down a funnel, four stages a developer passes through: hearing that the platform exists, learning to use it, building with it, and getting far enough to be glad they did. Documentation is the intervention for the first two, which is narrower than "documentation is important" and it told me what to work on.

Choosing a model to follow

Knowing which two stages to work on does not tell you what the site should look like, and the quickest way to get a shape is to copy one that works. The obvious source is our own Python SDK docs: the files are right there and already match the house style.

That clone lasted a day. Copying your own adjacent docs feels like reuse and is inheritance: you take on the previous author's idea of who the reader is and what can be assumed of them, silently, because none of those assumptions are written down in the files you copied.

The replacement was Stellar's documentation, and the property that mattered was being outside the organisation. It costs a few days of reading and buys a second opinion about what a stranger needs.

Reading before reorganizing

Copying a model settles what the site should look like and says nothing about what belongs in it. The day's plan was a mind map and an outline. Two hours of reorganizing later, the correction:

Feels like I should read the validator documentation first before trying to reorganize it.

It reads as obvious written out, and a morning had already gone into proving it is not: reading the thing you are about to replace feels like wasted motion when the plan is to throw it away.

A reorganisation is a claim about what the material is made of, and you cannot make that claim about material you have not read. Without reading it you move the shapes around until the table of contents looks tidy, which produces a document with a good table of contents and the same problems.

The order that came out of it is specification, mind map, outline, content: three structural passes before a sentence gets written for the page.

Outlines instead of cranking

Staging the passes is only worth anything if the unit inside them is right, and the unit was what I had been getting wrong. One of my own daily entries carried the header "Crank out a rough Client SDK Spec", and the same verb came back four months later as the diagnosis rather than the plan:

you're not supposed to crank them out .. you're supposed to write an outline and ask what do they want to know..

• Cranking treats documentation as a queue of articles with a length target, so the measure of a good day is how many you finished. That measure is available immediately and is uncorrelated with whether anybody can now do the thing.
• Asking what they want to know makes the unit the question rather than the article, and questions can be ordered by how early a reader hits them.

Around the same time I decided the tutorial should come before the page explaining what Terra is, which only makes sense once the reader's question is the organising unit rather than the subject matter.

The upstream documentation gap

Having the method did not make the site good on schedule. Criticism six months in separated into four problems that had been arriving as one:

1. The surface area was larger than one person's week.
2. Everything had to be technically correct in a domain I was still learning.
3. One person could check the hard sections, and they were not available.
4. Several sections could only be written by first working out how Tendermint and CosmWasm behaved, and at that point neither had written the documentation I needed.

The fourth describes a moment in 2020 rather than either team. Both were young and moving fast, and documentation is the thing that lags when a project is still deciding what it is.

Three of the four get better if I work harder or longer. The fourth does not, and that asymmetry is what made separating them worth doing. You cannot write a good third-layer document when the two layers underneath are undocumented. What you are doing in that situation is reverse-engineering somebody else's system and explaining your own on top of the reconstruction, at your own risk, and no amount of better writing at my layer changes it.

The recovery is the same separation applied to the work: focus on what is important, then fix the docs, then polish the English. Triage before repair, repair before finish. Running them together is what turns a criticism into a rewrite.

Generating what could be generated

Doing this a second time, for a different protocol, settled the method into a sequence:

1. Understand the whole protocol as process flows, before touching the site.
2. Structure.
3. Write.
4. Stop, on an explicit condition: the point at which revision is producing diminishing returns.

The largest and least stable part of either site was the reference, one page per message a contract accepts, listing every field and its type. Written by hand, those pages are the first to go wrong, because a field gets renamed in the code and nothing forces the page to notice.

The way out was already in the toolchain. Contracts on this chain publish machine readable descriptions of the messages they accept, in JSON Schema, so a tool can read those descriptions and rebuild the original Rust types from them. The core is one recursive function asking what kind of schema node it is looking at:

def parse_schema(schema: dict, env: dict = {}):
    if "anyOf" in schema:
        variants = [parse_schema(branch, env) for branch in schema["anyOf"]]
        return Enum(name=schema.get("title"), variants=variants)

    if schema.get("type") == "object":
        members = {n: parse_schema(m, env) for (n, m) in schema["properties"].items()}
        return Struct(name=schema.get("title"), members=members)

    if schema.get("type") == "array":
        return Vec(parse_schema(schema["items"], env))

    if schema.get("type") == "string":
        return String()

The mapping is one line per node kind, and the recursion handles the nesting:

Print the recovered type as a reference page and it cannot drift from the contract, because it was derived from what the contract publishes about itself.

What it cannot recover is meaning. A schema carries names, types and nesting, and says nothing about what a message is for or when you would send it. So the generator produces the half that goes stale and leaves the half that needed a person, which is the division worth keeping rather than the script.

It points at a larger idea I had written down in May: a documentation generator generator, a kit for writing documentation generators rather than one more generator. Anything a machine can read about your system is documentation you do not have to keep in sync by hand, and the parts you keep by hand are then the parts that actually needed a person.