Recall pipeline

One recall call inside chittad, from query to returned hits. The design premise, measured on the golden set, is that the top-20 cut — not ranking precision — was losing gold memories the retriever had already found. So the pool gets wider and the filter that narrows it is deliberately generous.

Over MCP, recall is a gateway: the strategy argument picks which daemon retrieval tool actually runs, and the gateway applies the rerank stage around it. The default is hybrid, not pure semantic.

Multi-lane fusion uses reciprocal-rank fusion with k = 60. Passing disable_hdc: true skips the hyperdimensional-computing¹⁰ lane; it exists for ablation and benchmarking, not for normal use.

recall_analogy performs explicit proportional relation transfer (a:b :: c:?) only. Indexed triplets identify the predicates from a to b and return actual outgoing neighbors of c, with supporting edges. Missing relations produce an abstention with a reason. No model call or vector ranking is used. The frozen benchmark returned hit@1 and hit@3 of 14/14, 14/14 negative abstentions and zero unsupported answers; see the evidence.

On every prompt, hooks/prompt-core.sh runs several recall calls in parallel — each in a backgrounded subshell writing to its own file — then merges them with a lane marker per line. The whole hook works to a wall-clock budget (CHITTA_HOOK_BUDGET_MS, 6000 ms by default) because the harness gives UserPromptSubmit far less time than the individual timeouts sum to.

A deterministic correction lookup runs alongside these and is not one of the fuzzy lanes: it is a keyed check, not a ranked search, and it survives the silencing rules below.

Any lane can be turned off for a single agent process with CHITTA_ABLATE_LANES, a comma list of the short names above. That is the mechanism the benchmark’s ablation matrix drives; see Benchmarks.

Retrieving a memory is not the same as showing it to the model. Every merged candidate carries an admission reason — the lane it came from — and every rejection is counted by cause. The hook renders that as per-item lane tags plus one summary line, so the policy that decides what enters the context window is inspectable rather than implicit.

Terse negations and turns with no distinctive tokens skip topic recall lanes. Under UNKNOWN, surviving hybrid, keyword and cross-realm rows must share a distinctive turn token; semantic/context rows require the small-realm exception as well. Keyword rows from a turn with fewer than two distinctive tokens need BM25 at least 60 (CHITTA_KW_SINGLE_TOKEN_MIN). Deterministic correction checks remain separate. The broader semantic entity check still records shadow evidence.

The hybrid lane’s response header carries the daemon’s Platt-calibrated maximum relevance, a single scalar standing in for “does this store know anything about this question”. The hook bins it into two bands and states the answer to the model in words. This is hook-side admission logic, not a daemon mechanism — there is no confidence-band parameter inside chittad.

Two bands, not three, on purpose. Version 5.70.0 shipped a three-band tag; store drift then inverted the boundary between the top two bands, with the supposedly stronger band scoring gold-hit@3 0.62 against the weaker band’s 0.64, and no threshold restored a monotonic separation. A confident-sounding tag that can be wrong at rank one is worse than a coarser honest one, so the two top bands were merged into a single confident-but-verify band. The KNOWN-versus-UNKNOWN split is monotonic.

The displayed percentage is recalibrated separately. Raw Platt maxrel read a mean of 0.844 on a frozen golden snapshot where the true hit@3 was 0.533, an expected calibration error of 0.31. A logistic refit over the logit of maxrel cuts leave-one-out calibration error to 0.159 and removes the over-confidence. Only the number shown is recalibrated; the band itself stays on raw maxrel, so behaviour is unchanged and the displayed percentage uses the fitted calibration.

UNKNOWN silence. The per-item percentage from multi-lane fusion is max-normalized, so rank one always reads 100% for any query at all, off-topic included — which defeats a per-item floor. When the absolute signal says UNKNOWN, the semantic, context and fuzzy-correction lanes are treated as noise and dropped. Keyword, hybrid and cross-realm hits survive only if they share a distinctive token with the current turn, which preserves the exact-lookup case without letting stop-word matches leak arbitrary memories. Set CHITTA_UNKNOWN_SILENCE=0 to disable.

Small-realm relaxation. A cross-project calibration under-scores a genuinely small realm: a handful of on-topic memories can pin the band below the threshold even when the top hit is strong. When the hybrid header shows a short result page in a non-default realm with a confident top hit, the token-share rule is extended to the semantic and context lanes instead of dropping them outright. The fuzzy correction lane stays silenced regardless, because a wrong correction costs more than a missed one.

When a cross-encoder is available, the MCP recall gateway fetches four times the requested limit and scores query/document pairs with cross-encoder/ms-marco-MiniLM-L-6-v2. Ranking uses sigmoid(logit) multiplied by the kind envelope 0.7 + 0.3 × kind_prior, preserving the distinction between curated corrections and operational fragments. Missing, insufficient or malformed candidate/score output falls back to the daemon's ordering. This gateway stage is separate from native candidate pooling.

An ONNX int8 export is preferred when CHITTA_RERANK_ONNX_DIR points at one. Thread count and maximum sequence length are set by CHITTA_RERANK_THREADS (8) and CHITTA_RERANK_MAX_LEN (128). If no reranker can be loaded the pipeline runs without it — this stage is optional.

The project was renamed from cc-soul on 2026-09-02; legacy CC_SOUL_* aliases are documented in the rename note.

Set CHITTA_ADMIT_DEBUG=1 to get one line on stderr per candidate, showing its lane, confidence, the confidence band, and whether the small-realm relaxation applied.

Store scoring reads scoring.json. Kind priors are correction/preference 1.3, wisdom 1.1, insight 1.05, operational 0.8 and episode 0.7. The operational prior ranks verbatim distiller fragments below curated conclusions at equal similarity. These are inputs to the bounded scoring pipeline, not calibrated probabilities.

replication_max defaults to 0 (disabled); CHITTA_REPLICATION_MAX overrides it. The opt-in 1.15 cap weights distinct identifiable source sessions, not repeated reads. The measured store had no memories with two independent replications, and the golden delta was zero; no benefit has been established. CHITTA_RECALL_PROFILE=1 writes phase timers to stderr.

Retrieval changes are graded on a fixed golden set of 30 queries with hand-labelled gold memory ids, scored as mean nDCG at a limit of 20 with the reranker on. The set is small and drawn from one store, so it is directional evidence, not a general claim.

The immediately preceding baseline on the same set read 0.442. Separately, making hybrid the default strategy measured as a strict superset of pure semantic: nDCG@20 up 0.08 on active queries, two more passing queries, and no regressions.

The end-to-end question — does any of this make an agent finish a real task more often or more cheaply — is a different measurement, and it lives on the Benchmarks page.