We dumped ten years of SharePoint into one index.

Note · KBMill · problem · cosine has no chronological conscience

The migration looked like progress: every site, every year, one vector index, one on-prem model on top. Then a 2014 procedure answers a 2024 question. A footer outranks the section. Nobody can mute one bad manual without boiling the soup. That is not scale. That is a melt.

Cosine has no chronological conscience. Vector similarity is blind to recency — that is why the old procedure wins when it scores higher.

How we see the problem

One index mixes domains, trust levels, and time. Retrieval finds the nearest fluent neighbor — not the neighbor you would sign. That failure mode sits next to wrong neighbor and opposite bounded bricks (composition closes the cross-brick question on this card).

Teams build the melt because it feels like “one KB.” Platforms sell it. It can work for narrow, clean, single-domain piles. On a real SharePoint decade it often becomes the reason the local LLM looks competent in demos and dangerous in production.

What you already tried

Metadata filters

Help when tags are honest and complete — across a decade-old share, half the pile is often Untitled_v2_final_backup.pdf. Metadata is rarely that clean.

Re-embed the soup / “prefer recent” prompts

Re-embedding does not split the soup. Soft prompt rules do not stop an old chunk from winning cosine if it scores higher. You still lack a package you can hand a team with a mute list for one locus.

Fair comparison

One melted index

Optimizes central search over everything. Fast to stand up; hard to trust and update.

Brick manufacture + composition

Optimizes human-sized units: ops in one brick, params in another, related when needed — not one infinite timeline. Other plants or internal platforms can compose too; when they work, they stop behaving like a single unmutable melt. Composition, not “more chunks in one bucket.”

How we mill it

We mill knowledge bricks — portable ZIPs with muted junk listed (LIBRARY_CARD and muted chunk flags — the residual board) — and keep seams when they are obvious (separate files, domains, volumes). On the public mill, a job defaults to what the pile warrants; we do not promise a full federation product in the hopper UI yet. The teaching point stands: do not melt ten years into one index and call it done. See separate gallery bricks, not one ZIP of everything: Public brick shelf. Open COMPOSITION_ArduPilot_Plane — it shows one vehicle domain as a shelf (ops wiki, params, MAVLink as separate bricks) so a query can load the facets it needs without recreating a mini-melt.

You drop files in the bin for a pile you want milled. Complexity stays in the plant. You pay only if we produce.

What this costs

Multi-locus piles trend Hard — manufacture difficulty, not prestige. Drop on the mill when you are ready to stop feeding the soup.