Engineering
Every note-taking tool with an automatic organizer eventually grows a bucket called Other, Misc, or Uncategorized. Ours did too. One morning the Nebula view had 36 memories sitting in a theme literally named "Uncategorized", next to a second catch-all named "Other". Two buckets for the same failure.
We treated that as a bug, not as a property of the data. This post is about what the bug turned out to be, and the design we ended up with.
What "Uncategorized" was actually made of
When we traced those 36 records, none of them were genuinely unclassifiable. They came from two mechanical gaps. The fallback theme that was supposed to catch low-confidence records was unreachable because of a threshold nobody could hit. And a small cluster that had been split off from a larger one lost its parent label on the way, so it arrived at the visualization with no theme at all.
That is the general pattern. "Uncategorized" is rarely about ambiguous content. It is a place where a pipeline stopped making decisions, and then presented the absence of a decision as if it were a category.
Ten doors, chosen to be stable
The fix started with a decision that sounds obvious and is not: the top level of the taxonomy is fixed. Ten doors, written by hand, meant to be mutually exclusive and collectively exhaustive for what one person actually says into a microphone over years:
- Building
- Work & People
- Business & Growth
- Money & Investing
- Learning & Ideas
- Creating & Expression
- People & Relationships
- Body & Mind
- Life Admin
- Direction & Meaning
Emergent topics are exciting in a demo and exhausting to live with. If the top level is allowed to drift, "career" and "work" become separate notebooks one month and merge the next, and every screen that depends on those names has to cope. Stability at the top is what lets everything underneath move.
Each door has a small set of slots, around forty in total, also written by hand. Slots are where the real assignment happens.
A classifier that must choose
Assignment is done by a small hosted language model, in batches, against the closed list. It sees a door, its slots, a one-line description of each, and a dozen notes. It must return one slot id per note, as JSON, and nothing else. Ids that are not in the list are rejected and the row is retried on its own.
Two details matter more than the model:
- There is no confidence threshold that sends a record to a fallback. Every record gets the best available slot. If the model is unsure, the assignment is marked low-confidence, which the interface can show, but the record still has a home.
- There is a separate sink for things that are not memories at all: test utterances, a bare "okay", a fragment the speech engine hallucinated from silence. Those are hidden, not shelved. Hiding is an explicit decision with an explicit rule. A bucket named Other is neither.
Labels are stored on the record itself, next to a hash of the taxonomy text that produced them. If we edit a slot description, only records that were labeled by an older version are re-queued. Records a person moved by hand are never touched by the machine again.
Let the third layer emerge
Underneath a slot, the structure is allowed to grow on its own. Records are embedded, and clusters form inside each slot from the embedding geometry. A cluster is only split when it gets large, roughly 150 records, and only if the two halves are actually different, which we check by running the split three times and refusing when the candidates look like duplicates. The half that keeps the original anchor keeps the name; the other half gets a new one.
This is the layer where "music writing" and "music performance" can appear as two clusters without anyone predefining them, while both stay inside the same slot and the same door. Emergence where it is cheap to be wrong, stability where it is expensive.
Why the pictures are not generated
Every door and every slot has artwork, and none of it is produced on demand by an image model. It is a curated library, one plate per door and per slot, drawn in one style. Emergent clusters inherit the plate of the slot they live in.
We tried the other route. Generated covers are impressive for a week and then the shelf looks like ten different illustrators had a disagreement. A memory you will look at for years should not change its face every time a model is retrained.
What still goes wrong
The embedding space is not isotropic. On one real corpus, a single door absorbed most of the volume simply because the model represents work-related text more densely. Classification by a language model reading the words does far better than distance in that space, which is why the door decision is not made by nearest-centroid.
And the classifier is a judgment, not a measurement. It will put a note about pricing a side project under Business & Growth when the person thinks of it as Money. That is fine. The person can move it, the machine remembers not to move it back, and the taxonomy titles are localized into eight languages so the correction feels like editing a shelf, not fighting a model.
What the system will not do anymore is shrug. There is no drawer for things it did not bother to decide.
Not a pitch — these are the holes we fell into while building it.