Successfully Wrong

What Bearing Says When You Ask

Ask Bearing what it knows about the health of its own corpus, and this is what comes back, verbatim, from a real run against a schema table, a wiki-exported glossary, and one hand-written decision record:

Last indexed: 2026-07-26 19:59:29Z

## business — STALE
- documents: 1
- asOf range: 2026-06-02 to 2026-06-02 (oldest 54d)
- producer: wiki-export, last run 2026-07-26 02:00:00Z, ok, 22 docs
  Catalog and Finance spaces

## decisions
- documents: 1
- asOf range: 2026-03-04 to 2026-03-04 (oldest 144d)
- producer: no state recorded

## schema
- documents: 1
- asOf range: 2026-07-24 to 2026-07-24 (oldest 2d)
- producer: sqlgen, last run 2026-07-24 06:12:00Z, ok, 48 docs
  Catalog schema, post-deploy hook, release 2026.7.3

Three origins, three different verdicts, and not one of them came from an exception. The business origin — pages exported from a wiki two months ago — is flagged STALE against its own thirty-day threshold. The schema origin, regenerated two days ago by a deploy hook, is fine. The decisions origin is a hundred and forty-four days old and not flagged at all — and says something else again: no producer state recorded, offered as a fact rather than buried as a gap.

The Failure Mode With No Symptom

None of Bearing's other tools would have told you any of this. A search returns a ranked passage whether that passage is two days old or two years old — a stale answer and a correct one look identical on the wire, both a plausible string of text a model will read and act on. A document fetch returns exactly the file it was asked for, correctly, regardless of whether the file still describes a system that works the way it says. Nothing throws. Nothing times out. The corpus is, by every measure a conventional health check would apply, up.

That's the specific shape of the problem, and it's why the check exists at all:

Freshness Isn't One Number

The instinct, once freshness matters at all, is to pick one number and apply it everywhere — thirty days, say, past which anything gets a warning. Bearing's default is per-origin instead, and the defaults aren't arbitrary:

public Dictionary<string, int> StaleDaysByOrigin { get; init; } = new()
{
    ["schema"] = 7,
    ["impl"] = 7,
    ["business"] = 30,
    ["decisions"] = 3650   // decisions do not go stale; they get superseded
};

Schema and implementation notes get seven days, because both are regenerated by something running on a deploy cadence measured in days — a schema doc older than a week may already be describing a database that has moved on. Business documentation gets thirty, because wiki pages move at the pace people write them, not the pace code ships. Decisions get thirty-six hundred and fifty — ten years, which is really just “never” spelled as a number a health check can still compare against — because a decision record isn't a description of a moving system. It's a historical fact. The reasoning behind a choice doesn't go stale when the world changes around it; it gets superseded by a new decision record, which is a different event, not the same one aging.

Recording Your Own Failure

None of this works if a producer can fail and leave no trace. Bearing's producer contract puts the obligation on the writer, not the reader: every run, success or failure, drops a state file, and a failure is recorded the same way a success is — just with the outcome flipped:

Write this even when the run fails, with success: false and the error in
message. A producer that fails silently is exactly the failure
context_health exists to catch, and it can only catch it if failure gets
recorded.

The decisions origin above has no producer state at all, and context_health says so plainly rather than presenting the absence as either a pass or a silent gap. That's deliberate: decisions in this corpus are hand-written and reviewed by a person, not generated by a script, so there is no automated run to record. A report with only two states — ok or stale — would have nowhere honest to put that fact. Having three is what lets it stay honest about something that isn't broken and also isn't being watched by anything.

Proving the Alarm Actually Rings

A health check that has never seen a failure hasn't been tested — it's been hoped about. So the test suite includes a producer state file built to fail on purpose:

File.WriteAllText(Path.Combine(Root, "_state", "wiki-export.json"), """
    {"producer":"wiki-export","lastRun":"2026-07-26T02:00:00Z","success":false,
     "documentCount":0,"message":"export API timed out"}
    """);

and a passing assertion that checks the report surfaces exactly that outcome, not a sanitised version of it:

Assert.NotNull(health["schema"].LastProducerRun);
Assert.True(health["schema"].LastProducerRun!.Success);

Assert.NotNull(health["business"].LastProducerRun);
Assert.False(health["business"].LastProducerRun!.Success);

That's a small thing to verify and an easy one to skip — nobody schedules time to write a test for the code path that only runs once something else has already gone wrong. It's also the one test in the suite that stands in for the entire reason the tool exists: an alarm that has never been checked against a real failure is a decoration, not an alarm.

The General Shape

Most software is monitored by whether it's running — process alive, port open, a health endpoint answering. That check is sufficient for almost everything, because almost everything's failure mode is loud: it crashes, it times out, it returns an error a caller has to handle. A context layer can fail while satisfying every one of those checks, because its job was never to respond correctly — only to respond. And “responds” is not the same property as “responds with something true,” any more than “compiles” is the same property as “correct.”

So the health check can't be something added once retrieval starts feeling off, because the entire premise is that it won't feel off. It has to exist before the first real query does, as a tool the model itself can reach for — not a thing a human eventually gets suspicious enough to go looking for.

Related