Grounded Docs
v0.2.0 · alpha Download Changelog GitHub
>_ docs / concepts / Recall & ranking
§ 02 Concepts

Recall & ranking

Recall is deterministic and inspectable. It embeds your query once, searches two lanes in parallel, fuses them with reciprocal rank fusion, applies a handful of explicit boosts, and returns one flat list ranked by score. limit is a TOTAL across facts, sessions and docs — not a quota per source. The best match is rank 1 whatever lane it came from.

// The two lanes

Two lanes, fused #

For each source type, Grounded runs a lexical lane (full-text search — SQLite FTS5 or Postgres tsvector) and a vector lane (cosine similarity over sqlite-vec or pgvector). A card's matchedBy records where it came from: lexical, vector, or both.

// The fusion

Reciprocal rank fusion #

The lanes are merged by rank, not by comparing incomparable scores. Each item accrues a fused score from its rank in each lane:

RRF

score += 1 / (k + rank) across the vector lane then the lexical lane, with k = 60 (configurable) and 0-based ranks. Appearing high in either lane — or modestly in both — lifts an item.

// Ranking pressure

Boosts #

The fused score is multiplied by a small set of per-lane boosts. They tilt ties; they are not meant to override retrieval:

  • Facts — ×(1 + 0.5 × importance), a ceiling of ×1.5 at importance = 1, then the scope-affinity multiplier. Pinning is ×1.0 — deliberately no retrieval boost at all: a pinned fact is already guaranteed into the startup brief, and paying for the same fact twice made the fact lane unbeatable regardless of what the query asked for.
  • Sessions — × 0.5^(ageDays / 30): a recency half-life of 30 days, so a month-old session weighs half a fresh one.
  • Docs — ×1.25 when active; archived and missing chunks get no boost.

Every one of these numbers lives in config.toml under [recall.boosts] — nothing is hidden in the model. Set any of them to 1 to turn that pressure off.

Docs are the guidance that pilots project work; facts are hard guardrails. The multipliers are tuned so a fact never crowds out the document that actually answers the query.

// Whose rule is this

Fact scope #

Every fact carries a scope: global, or a narrowed one like project:alpha, machine:laptop, agent:reviewer. Scope is read twice — once as a filter, once as a boost.

As a hard filter. A recall that declares project: "alpha" returns global and project:alpha facts only; another project's rules are removed outright, not merely demoted. global always passes. Declare nothing and no filter applies — a plain recall (and the startup brief) still sees your machine: and agent: facts. Pass factScopes to name the set yourself. meta.scopeFilter reports what was applied, so a missing row is always explainable.

As a boost. Whatever survives the filter is then multiplied by one of four tiers:

TierWhen×
matchThe query names the scope as a whole word, or the fact's project: scope is the declared project.1.5
mismatchA project was declared and this fact belongs to a different one.0.8
specificA non-global scope under a context that neither names nor contradicts it.1.0
neutralA global or unscoped fact.1.0

specific is 1.0 — off by default. It used to be 1.3, on the theory that a narrowed fact is more specific than an unrelated global rule. The hard filter now removes the foreign-project rows it was aimed at, leaving it inflating machine: and agent: rows with no evidence from the query. The two surviving tiers are the ones that read the caller's context: the query names the scope, or the caller declares the project.

The hard filter usually reaches a foreign-project fact before mismatch can demote it — that tier now only bites when the caller passes an explicit factScopes set that admits other projects.

// The final shape

Lane share & ordering #

The answer is one flat list, score-descending. There is no per-source section and no fixed tier order — a doc can be rank 1 and a fact rank 9. What stops any one lane from taking the whole answer is recall.laneShare, a ceiling on each lane's share of limit:

>_ recall.laneShare limit 20
0.25
Facts — at most 5 of 20
guardrails
0.5
Sessions — at most 10 of 20
history
1
Docs — up to the full 20
guidance

Two rules keep this honest. The share applies only to multi-source recall — ask for sources: ["fact"] and you get a full limit of facts, so a facts-only caller is never quietly truncated. And a share can never round to zero: every lane keeps at least one slot. Setting laneShare.fact = 1 restores unbounded behaviour.

Ties. Fused scores collide exactly — a single-lane hit at rank 0 is always 1/60. recall.tieBreakOrder settles those, default ["doc", "fact", "session"], then by id. Archived docs are forced last regardless. The order is total and config-fixed, so the same corpus ranks identically on SQLite and Postgres.

Documents, not chunks. Docs are indexed in chunks, but recall answers with documents: chunks of the same file collapse into one result carrying its best-scoring chunk, plus a chunks count. One long file can no longer occupy five slots while the file that actually answers the query sits below the fold.

The response isn't a bare array of cards — it's a {data, meta} envelope. meta.bySource reports how many of each source type came back versus how many matched, so a lane trimmed by its share is visible rather than silent, and truncated flips true whenever the share or the limit cut the set. meta.available is a floor, not an exact total — the vector lane has no match predicate to count, only a k. See the API's return shapes for the full envelope.

The console and ground_recall over MCP both read this one list; grouping by source, if you want it, is a client-side choice.

// No model, no problem

Lexical-only fallback #

If embeddings are turned off (provider = "none") or the vector index can't load, the vector lane simply drops out: recall runs lexical-only, every card reports matchedBy: "lexical", and no error is raised. Grounded degrades to full-text search rather than failing.