Semantic objectsgiving models handles instead of data

I started working on semantic objects after Claude failed a simple task in a todo application. I gave it every task and asked for everything due that day. The answer looked reasonable and omitted several items.

The prompt already contained the data. The model understood the request. It still could not reliably perform an exhaustive scan because generation is sampled one token at a time. Asking again could produce a different incomplete subset.

I wanted the model to express the selection and let deterministic code execute it. "All todos due today" is a small request regardless of whether the collection contains ten rows or ten thousand. The rows should remain outside the prompt until the model asks for specific contents.

I called the resulting reference a semantic object: a typed handle to data that stays where it already lives.

The model chooses the operation; code processes the collection

Pasting a collection into context gives the model two jobs. It has to decide what the user means and apply that decision to every item. Models are good at the first job and unreliable at the second when completeness matters.

I split them:

1. The model selects an operation such as "filter by today's date."
2. Application code runs that operation over the full collection.
3. The runtime returns a scalar, a small result, or another handle.

This also reduces schema errors. My tool calls had become deeply nested because one request needed to describe the collection, filter, operation, and output shape at once. Models filled those schemas incorrectly once they reached more than a couple of levels. A handle moves the collection and its structure out of the call. The model only supplies the object ID, method, and arguments.

A conventional query language solved part of this problem. I began with GraphQL because it already expressed deterministic selection and composition. An augmented SQL could have done the same. I did not need another query syntax yet.

Queries still return rows. Returning thousands of filtered rows to the model recreates the context problem one step later. I needed operations to return references as well as values.

A semantic object describes data the model has not read

A semantic object contains enough metadata for the model to decide whether and how to use it:

SemanticObject
  id
  kind
  description
  fields[]
    name
    type
    description
  methods[]
    name
    signature
    description
  backing data reference

The object does not contain a pasted copy of the collection. Its backing reference points to the application data. The model sees the object's name, type, description, fields, and available methods.

Descriptions matter because the model cannot inspect an implementation body. For a method such as getDateRangeView, the signature explains the argument shape and the description explains its behavior. Together they are the interface the model programs against.

I kept the runtime API small:

Most calls only move metadata. readObject is the explicit escape hatch that loads content. invokeMethod can return a small value or another semantic object.

I used kind as the type identifier

I considered id, ref, URI, and kind for the field that identifies an object's type.

id already names a particular object. URI implies a location, while the value identifies a type. ref says little about what it references. I chose kind because JSON APIs commonly use it for a resource category and it remains distinct from the instance ID.

This choice matters more for a model-facing API than for an ordinary library. The model learned each word from many unrelated APIs. Reusing an overloaded term such as type adds ambiguity before the model sees any documentation. A familiar, narrower word makes the schema easier to complete correctly.

Kinds are namespaced so two applications can define JournalEntry without colliding. An object can report a kind such as so.idyllic.JournalCollection, and lookupTypeInfo returns the shared definition.

The reference has to survive the prompt boundary

The easiest way to fake this design is to resolve an @ mention into text before sending the prompt. The interface would look correct while the entire collection still entered context.

I wrote the prototype around one non-negotiable test: the mention must remain a handle when the request reaches the model.

The rest of the acceptance criteria made that observable:

• I could add a semantic object through the existing @-mention interface.
• The prompt contained the object ID and relevant metadata without its full contents.
• The model could inspect the object's fields and methods.
• The model could invoke one method and chain another call from its result.
• Several objects could appear in the same request.
• Content only entered context through an explicit read or a method returning content.

I tested those capabilities incrementally: an object with metadata only, a filtered view, one method call, a chain of calls, and several objects. Each stage had a visible failure condition. I could inspect the actual prompt and verify that no hidden expansion had occurred.

Methods can return new handles

The working prototype used about 700 journal entries from a corpus of roughly 2,500. Claude began by inspecting the collection:

Tool call: inspectObject { "objId": "journal-collection" }

Tool result:
  Journal Collection (so.idyllic.JournalCollection)
  A collection of journal entries

  Fields:
  - title
  - content
  - date

  Methods:
  - getDayCount() -> number
  - getEntriesForDay(date: YYYY-MM-DD) -> JournalEntry[]
  - getTotalEntryCount() -> number
  - getDateRangeView(startDate, endDate) -> JournalCollection

The important method was getDateRangeView. It filtered the backing collection and returned another handle:

Tool call: invokeMethod {
  "objId": "journal-collection",
  "methodName": "getDateRangeView",
  "args": {
    "startDate": "2024-04-01",
    "endDate": "2024-04-30"
  }
}

Tool result:
  New semantic object:
  4f6b305b-3f7c-472a-9ac6-bc63b3045c8c
  Journal entries from 2024-04-01 to 2024-04-30

Tool call: invokeMethod {
  "objId": "4f6b305b-3f7c-472a-9ac6-bc63b3045c8c",
  "methodName": "getDayCount",
  "args": {}
}

Tool result: 28

Claude narrowed the collection and counted the matching days without reading the entries. The filtered view behaved like the original collection because both implemented the same kind and methods.

Returning handles is what makes operations composable. A method that returns contents ends the lazy chain and spends context. A method that returns another semantic object lets the model narrow, join, or transform again before deciding what to read.

The prototype exposed errors in the metadata

The first successful run also found inconsistencies in the design. The implementation called the inspection operation analyzeObject while the draft API called it inspectObject. The object returned so.idyllic.JournalCollection while the proposed naming scheme included an extra prototype# segment. A filtered April view inherited a parent description that said April through June.

The method calls still worked, but the metadata was part of the model-facing API and had to be correct. I standardized the operation name, simplified the kind format, and regenerated descriptions for derived objects from their actual filters.

That last bug was especially useful. A stale description can mislead the model even when the underlying object contains the right data. Semantic objects defer the data, so their metadata carries more responsibility than ordinary labels.

Semantic objects make retrieval programmable

Conventional retrieval chooses passages with a similarity search configured before the model begins reasoning. It returns the selected text to the prompt.

A semantic object lets the model choose operations during the task. It can inspect a collection, take a date range, narrow it again, count the result, and only then read specific entries. Deterministic code performs each collection operation, while the model decides which operation answers the user's question.

I kept the object machinery minimal after the prototype. A semantic object does not need a class hierarchy, decorators, or an internal event system. It needs an ID, useful metadata, a backing data reference, and a set of operations. Specialized classes only help when an application has behavior that requires them.

The result is a simple division of work. The model reasons about names, types, and operations. The runtime performs exhaustive scans and retains large intermediate collections. Data enters context only when the model requests it. That is the problem semantic objects solved for me: Claude could work with a collection much larger than its prompt without pretending it had read every row.