feat: Phase 6 — Living Taxonomy + Universal Capture

Closes two real-use gaps:
1. "APM tool" gap: work done outside Claude Code (desktop, web, phone,
   other machine) was invisible to AtoCore.
2. Project discovery gap: manual JSON-file edits required to promote
   an emerging theme to a first-class project.

B — atocore_remember MCP tool (scripts/atocore_mcp.py):
- New MCP tool for universal capture from any MCP-aware client
  (Claude Desktop, Code, Cursor, Zed, Windsurf, etc.)
- Accepts content (required) + memory_type/project/confidence/
  valid_until/domain_tags (all optional with sensible defaults)
- Creates a candidate memory, goes through the existing 3-tier triage
  (no bypass — the quality gate catches noise)
- Detailed tool description guides Claude on when to invoke: "remember
  this", "save that for later", "don't lose this fact"
- Total tools exposed by MCP server: 14 → 15

C.1 Emerging-concepts detector (scripts/detect_emerging.py):
- Nightly scan of active + candidate memories for:
  * Unregistered project names with ≥3 memory occurrences
  * Top 20 domain_tags by frequency (emerging categories)
  * Active memories with reference_count ≥ 5 + valid_until set
    (reinforced transients — candidates for extension)
- Writes findings to atocore/proposals/* project state entries
- Emits "warning" alert via Phase 4 framework the FIRST time a new
  project crosses the 5-memory alert threshold (avoids spam)
- Configurable via env vars: ATOCORE_EMERGING_PROJECT_MIN (default 3),
  ATOCORE_EMERGING_ALERT_THRESHOLD (default 5), TOP_TAGS_LIMIT (20)

C.2 Registration surface (src/atocore/api/routes.py + wiki.py):
- POST /admin/projects/register-emerging — one-click register with
  sensible defaults (ingest_roots auto-filled with
  vault:incoming/projects/<id>/ convention). Clears the proposal
  from the dashboard list on success.
- Dashboard /admin/dashboard: new "proposals" section with
  unregistered_projects + emerging_categories + reinforced_transients.
- Wiki homepage: "📋 Emerging" section rendering each unregistered
  project as a card with count + 2 sample memory previews + inline
  "📌 Register as project" button that calls the endpoint via fetch,
  reloads the page on success.

C.3 Transient-to-durable extension
(src/atocore/memory/service.py + API + cron):
- New extend_reinforced_valid_until() function — scans active memories
  with valid_until in the next 30 days and reference_count ≥ 5.
  Extends expiry by 90 days. If reference_count ≥ 10, clears expiry
  entirely (makes permanent). Writes audit rows via the Phase 4
  memory_audit framework with actor="transient-to-durable".
- POST /admin/memory/extend-reinforced — API wrapper for cron.
- Matches the user's intuition: "something transient becomes important
  if you keep coming back to it".

Nightly cron (deploy/dalidou/batch-extract.sh):
- Step F2: detect_emerging.py (after F pipeline summary)
- Step F3: /admin/memory/extend-reinforced (before integrity check)
- Both fail-open; errors don't break the pipeline.

Tests: 366 → 374 (+8 for Phase 6):
- 6 tests for extend_reinforced_valid_until covering:
  extension path, permanent path, skip far-future, skip low-refs,
  skip permanent memories, audit row write
- 2 smoke tests for the detector (imports cleanly, handles empty DB)
- MCP tool changes don't need new tests — the wrapper is pure passthrough

Design decisions documented in plan file:
- atocore_remember deliberately doesn't bypass triage (quality gate)
- Detector is passive (surfaces proposals) not active (auto-registers)
- Sensible ingest-root defaults ("vault:incoming/projects/<id>/")
  so registration is one-click with no file-path thinking
- Extension adds 90 days rather than clearing expiry (gradual
  permanence earned through sustained reinforcement)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-04-18 08:08:55 -04:00
parent cc68839306
commit 02055e8db3
7 changed files with 736 additions and 0 deletions

View File

@@ -604,6 +604,93 @@ def auto_promote_reinforced(
return promoted
def extend_reinforced_valid_until(
min_reference_count: int = 5,
permanent_reference_count: int = 10,
extension_days: int = 90,
imminent_expiry_days: int = 30,
) -> list[dict]:
"""Phase 6 C.3 — transient-to-durable auto-extension.
For active memories with valid_until within the next N days AND
reference_count >= min_reference_count: extend valid_until by
extension_days. If reference_count >= permanent_reference_count,
clear valid_until entirely (becomes permanent).
Matches the user's intuition: "something transient becomes important
if you keep coming back to it". The system watches reinforcement
signals and extends expiry so context packs keep seeing durable
facts instead of letting them decay out.
Returns a list of {memory_id, action, old, new} dicts for each
memory touched.
"""
from datetime import timedelta
now = datetime.now(timezone.utc)
horizon = (now + timedelta(days=imminent_expiry_days)).strftime("%Y-%m-%d")
new_expiry = (now + timedelta(days=extension_days)).strftime("%Y-%m-%d")
now_str = now.strftime("%Y-%m-%d %H:%M:%S")
extended: list[dict] = []
with get_connection() as conn:
rows = conn.execute(
"SELECT id, valid_until, reference_count FROM memories "
"WHERE status = 'active' "
"AND valid_until IS NOT NULL AND valid_until != '' "
"AND substr(valid_until, 1, 10) <= ? "
"AND COALESCE(reference_count, 0) >= ?",
(horizon, min_reference_count),
).fetchall()
for r in rows:
mid = r["id"]
old_vu = r["valid_until"]
ref_count = int(r["reference_count"] or 0)
if ref_count >= permanent_reference_count:
# Permanent promotion
conn.execute(
"UPDATE memories SET valid_until = NULL, updated_at = ? WHERE id = ?",
(now_str, mid),
)
extended.append({
"memory_id": mid, "action": "made_permanent",
"old_valid_until": old_vu, "new_valid_until": None,
"reference_count": ref_count,
})
else:
# 90-day extension
conn.execute(
"UPDATE memories SET valid_until = ?, updated_at = ? WHERE id = ?",
(new_expiry, now_str, mid),
)
extended.append({
"memory_id": mid, "action": "extended",
"old_valid_until": old_vu, "new_valid_until": new_expiry,
"reference_count": ref_count,
})
# Audit rows via the shared framework (fail-open)
for ex in extended:
try:
_audit_memory(
memory_id=ex["memory_id"],
action="valid_until_extended",
actor="transient-to-durable",
before={"valid_until": ex["old_valid_until"]},
after={"valid_until": ex["new_valid_until"]},
note=f"reinforced {ex['reference_count']}x; {ex['action']}",
)
except Exception:
pass
if extended:
log.info("reinforced_valid_until_extended", count=len(extended))
return extended
def expire_stale_candidates(
max_age_days: int = 14,
) -> list[str]: