Commit Graph

7 Commits

Author SHA1 Message Date
5aeeb1cad1 feat: query-relevance ordering for memory selection
get_memories_for_context now accepts an optional query string.
When provided, candidate memories are reranked by lexical overlap
with the query (stemmed token intersection, ties broken by
confidence) before the budget walk. Without a query the order is
unchanged — effectively "by confidence desc" as before — so
non-builder callers see no behaviour change.

The fetch limit is raised from 10 to 30 so there's a real pool to
rerank. Token overlap reuses _normalize/_tokenize from
reinforcement.py so ranking and reinforcement matching share the
same notion of distinctive terms.

build_context passes the user_prompt through to both the identity/
preference and project-memory calls. The retrieval harness
regression the fix is targeting:

- p05-vendor-signal FAIL @ 1161645: "Zygo" missing from the pack
  even though an active vendor memory contained it. Root cause:
  higher-confidence p05 memories filled the 25% budget slice
  before the vendor memory ever got a chance. Query-aware ordering
  puts the vendor memory first when the query is about vendors.

New regression test test_project_memories_query_relevance_ordering
locks the behaviour in with two p05 memories and a tight budget.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 12:47:05 -04:00
8ea53f4003 feat: fold project-scoped memories into context pack
The retrieval-quality review on 2026-04-11 found that active
project/knowledge/episodic memories never reached the pack: only
Trusted Project State and identity/preference memories were being
assembled. Reinforcement bumped confidence on memories that had
no retrieval outlet, so the reflection loop was half-open.

This change adds a third memory tier between identity/preference
and retrieved chunks:

- PROJECT_MEMORY_BUDGET_RATIO = 0.15
- Memory types: project, knowledge, episodic
- Only populated when a canonical project is in scope — without
  a project hint, project memories stay out (cross-project bleed
  would rot the signal)
- Rendered under a dedicated "--- Project Memories ---" header
  so the LLM can distinguish it from the identity/preference band
- Trim order in _trim_context_to_budget: retrieval → project
  memories → identity/preference → project state (most recently
  added tier drops first when budget is tight)

get_memories_for_context gains header/footer kwargs so the two
memory blocks can be distinguished in a single pack without a
second helper.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 11:35:40 -04:00
f2372eff9e fix(P1+P2): alias-aware project state lookup + slash command corpus fallback
Two regression fixes from codex's review of the slash command
refactor commit (78d4e97). Both findings are real and now have
covered tests.

P1 — server-side alias resolution for project_state lookup
----------------------------------------------------------
The bug:
- /context/build forwarded the caller's project hint verbatim to
  get_state(project_hint), which does an exact-name lookup against
  the projects table (case-insensitive but no alias resolution)
- the project registry's alias matching was only used by the
  client's auto-context path and the retriever's project-match
  boost, never by the server's project_state lookup
- consequence: /atocore-context "... p05" would silently miss
  trusted project state stored under the canonical id
  "p05-interferometer", weakening project-hinted retrieval to
  the point that an explicit alias hint was *worse* than no hint

The fix in src/atocore/context/builder.py:
- import get_registered_project from the projects registry
- before calling get_state(project_hint), resolve the hint
  through get_registered_project; if a registry record exists,
  use the canonical project_id for the state lookup
- if no registry record exists, fall back to the raw hint so a
  hand-curated project_state entry that predates the registry
  still works (backwards compat with pre-registry deployments)

The retriever already does its own alias expansion via
get_registered_project for the project-match boost, so the
retriever side was never broken — only the project_state lookup
in the builder. The fix is scoped to that one call site.

Tests added in tests/test_context_builder.py:
- test_alias_hint_resolves_through_registry: stands up a fresh
  registry, sets state under "p05-interferometer", then verifies
  build_context with project_hint="p05" finds the state, AND
  with project_hint="interferometer" (the second alias) finds it
  too, AND with the canonical id finds it. Covers all three
  resolution paths.
- test_unknown_hint_falls_back_to_raw_lookup: empty registry,
  set state under an unregistered project name, verify the
  build_context call with that name as the hint still finds the
  state. Locks in the backwards-compat behavior.

P2 — slash command no-hint fallback to corpus-wide context build
----------------------------------------------------------------
The bug:
- the slash command's no-hint path called auto-context, which
  returns {"status": "no_project_match"} when project detection
  fails and does NOT fall back to a plain context-build
- the slash command's own help text told the user "call without
  a hint to use the corpus-wide context build" — which was a lie
  because the wrapper no longer did that
- consequence: generic prompts like "what changed in AtoCore
  backup policy?" or any cross-project question got a useless
  no_project_match envelope instead of a context pack

The fix in .claude/commands/atocore-context.md:
- the no-hint path now does the 2-step fallback dance:
    1. try `auto-context "<prompt>"` for project detection
    2. if the response contains "no_project_match", fall back to
       `context-build "<prompt>"` (no project arg)
- both branches return a real context pack, fail-open envelope
  is preserved for genuine network errors
- the underlying client surface is unchanged (no new flags, no
  new subcommands) — the fallback is per-frontend logic in the
  slash command, leaving auto-context's existing semantics
  intact for OpenClaw and any other caller that depends on the
  no_project_match envelope as a "do nothing" signal

While I was here, also tightened the slash command's argument
parsing to delegate alias-knowledge to the registry instead of
embedding a hardcoded list:
- old version had a literal list of "atocore", "p04", "p05",
  "p06" and their aliases that needed manual maintenance every
  time a project was added
- new version takes the last token of $ARGUMENTS and asks the
  client's `detect-project` subcommand whether it's a known
  alias; if matched, it's the explicit hint, if not it's part
  of the prompt
- this delegates registry knowledge to the registry, where it
  belongs

Unrelated improvement noted but NOT fixed in this commit:
- _rank_chunks in builder.py also has a naive substring boost
  that uses the original hint without alias expansion. The
  retriever already does the right thing, so this secondary
  boost is redundant. Tracked as a future cleanup but not in
  scope for the P1/P2 fix; codex's findings are about
  project_state lookup, not about the secondary chunk boost.

Full suite: 162 passing (was 160), 1 warning. The +2 is the two
new P1 regression tests.
2026-04-07 07:47:03 -04:00
14ab7c8e9f fix: pass project_hint into retrieve and add path-signal ranking
Two changes that belong together:

1. builder.build_context() now passes project_hint into retrieve(),
   so the project-aware boost actually fires for the retrieval pipeline
   driven by /context/build. Before this, only direct /query callers
   benefited from the registered-project boost.

2. retriever now applies two more ranking signals on every chunk:
   - _query_match_boost: boosts chunks whose source/title/heading
     echo high-signal query tokens (stop list filters out generic
     words like "the", "project", "system")
   - _path_signal_boost: down-weights archival noise (_archive,
     _history, pre-cleanup, reviews) by 0.72 and up-weights current
     high-signal docs (status, decision, requirements, charter,
     system-map, error-budget, ...) by 1.18

Tests:
- test_context_builder_passes_project_hint_to_retrieval verifies
  the wiring fix
- test_retrieve_downranks_archive_noise_and_prefers_high_signal_paths
  verifies the new ranking helpers prefer current docs over archive

This addresses the cross-project competition and archive bleed
called out in current-state.md after the Wave 1 ingestion.
2026-04-06 18:37:07 -04:00
b0889b3925 Stabilize core correctness and sync project plan state 2026-04-05 17:53:23 -04:00
531c560db7 feat: Phase 1 ingestion hardening + Phase 5 Trusted Project State
Phase 1 - Ingestion hardening:
- Encoding fallback (UTF-8/UTF-8-sig/Latin-1/CP1252)
- Delete detection: purge DB/vector entries for removed files
- Ingestion stats endpoint (GET /stats)

Phase 5 - Trusted Project State:
- project_state table with categories (status, decision, requirement, contact, milestone, fact, config)
- CRUD API: POST/GET/DELETE /project/state
- Upsert semantics, invalidation (supersede) support
- Context builder integrates project state at highest trust precedence
- Project state gets 20% budget allocation, appears first in context
- Trust precedence: Project State > Retrieved Chunks (per Master Plan)

33/33 tests passing. Validated end-to-end with GigaBIT M1 project data.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-05 09:41:59 -04:00
b4afbbb53a feat: implement AtoCore Phase 0 + Phase 0.5 (foundation + PoC)
Complete implementation of the personal context engine foundation:
- FastAPI server with 5 endpoints (ingest, query, context/build, health, debug)
- SQLite database with 5 tables (documents, chunks, memories, projects, interactions)
- Heading-aware markdown chunker (800 char max, recursive splitting)
- Multilingual embeddings via sentence-transformers (EN/FR)
- ChromaDB vector store with cosine similarity retrieval
- Context builder with project boosting, dedup, and budget enforcement
- CLI scripts for batch ingestion and test prompt evaluation
- 19 unit tests passing, 79% coverage
- Validated on 482 real project files (8383 chunks, 0 errors)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-05 09:21:27 -04:00