feat: Phase 3 V1 — Auto-Organization (domain_tags + valid_until)
Adds structural metadata that the LLM triage was already implicitly
reasoning about ("stale snapshot" → reject). Phase 3 captures that
reasoning as fields so it can DRIVE retrieval, not just rejection.
Schema (src/atocore/models/database.py):
- domain_tags TEXT DEFAULT '[]' JSON array of lowercase topic keywords
- valid_until DATETIME ISO date; null = permanent
- idx_memories_valid_until index for efficient expiry queries
Memory service (src/atocore/memory/service.py):
- Memory dataclass gains domain_tags + valid_until
- create_memory, update_memory accept/persist both
- _row_to_memory safely reads both (JSON-decode + null handling)
- _normalize_tags helper: lowercase, dedup, strip, cap at 10
- get_memories_for_context filters expired (valid_until < today UTC)
- _rank_memories_for_query adds tag-boost: memories whose domain_tags
appear as substrings in query text rank higher (tertiary key after
content-overlap density + absolute overlap, before confidence)
LLM extractor (_llm_prompt.py → llm-0.5.0):
- SYSTEM_PROMPT documents domain_tags (2-5 keywords) + valid_until
(time-bounded facts get expiry dates; durable facts stay null)
- normalize_candidate_item parses both fields from model output with
graceful fallback for string/null/missing
LLM triage (scripts/auto_triage.py):
- TRIAGE_SYSTEM_PROMPT documents same two fields
- parse_verdict extracts them from verdict JSON
- On promote: PUT /memory/{id} with tags + valid_until BEFORE
POST /memory/{id}/promote, so active memories carry them
API (src/atocore/api/routes.py):
- MemoryCreateRequest: adds domain_tags, valid_until
- MemoryUpdateRequest: adds domain_tags, valid_until, memory_type
- GET /memory response exposes domain_tags + valid_until + created_at
Triage UI (src/atocore/engineering/triage_ui.py):
- Renders existing tags as colored badges
- Adds inline text field for tags (comma-separated) + date picker for
valid_until on every candidate card
- Save&Promote button persists edits via PUT then promotes
- Plain Promote (and Y shortcut) also saves tags/expiry if edited
Wiki (src/atocore/engineering/wiki.py):
- Search now matches memory content OR domain_tags
- Search results render tags as clickable badges linking to
/wiki/search?q=<tag> for cross-project navigation
- valid_until shown as amber "valid until YYYY-MM-DD" hint
Tests: 303 → 308 (5 new for Phase 3 behavior):
- test_create_memory_with_tags_and_valid_until
- test_create_memory_normalizes_tags
- test_update_memory_sets_tags_and_valid_until
- test_get_memories_for_context_excludes_expired
- test_context_builder_tag_boost_orders_results
Deferred (explicitly): temporal_scope enum, source_refs memory graph,
HDBSCAN clustering, memory detail wiki page, backfill of existing
actives. See docs/MASTER-BRAIN-PLAN.md.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -264,6 +264,94 @@ def test_expire_stale_candidates(isolated_db):
|
||||
assert mem["status"] == "invalid"
|
||||
|
||||
|
||||
# --- Phase 3: domain_tags + valid_until ---
|
||||
|
||||
|
||||
def test_create_memory_with_tags_and_valid_until(isolated_db):
|
||||
from atocore.memory.service import create_memory
|
||||
|
||||
mem = create_memory(
|
||||
"knowledge",
|
||||
"CTE gradient dominates WFE at F/1.2",
|
||||
domain_tags=["optics", "thermal", "materials"],
|
||||
valid_until="2027-01-01",
|
||||
)
|
||||
assert mem.domain_tags == ["optics", "thermal", "materials"]
|
||||
assert mem.valid_until == "2027-01-01"
|
||||
|
||||
|
||||
def test_create_memory_normalizes_tags(isolated_db):
|
||||
from atocore.memory.service import create_memory
|
||||
|
||||
mem = create_memory(
|
||||
"knowledge",
|
||||
"some content here",
|
||||
domain_tags=[" Optics ", "OPTICS", "Thermal", ""],
|
||||
)
|
||||
# Duplicates and empty removed; lowercased; stripped
|
||||
assert mem.domain_tags == ["optics", "thermal"]
|
||||
|
||||
|
||||
def test_update_memory_sets_tags_and_valid_until(isolated_db):
|
||||
from atocore.memory.service import create_memory, update_memory
|
||||
from atocore.models.database import get_connection
|
||||
|
||||
mem = create_memory("knowledge", "some content for update test")
|
||||
assert update_memory(
|
||||
mem.id,
|
||||
domain_tags=["controls", "firmware"],
|
||||
valid_until="2026-12-31",
|
||||
)
|
||||
with get_connection() as conn:
|
||||
row = conn.execute("SELECT domain_tags, valid_until FROM memories WHERE id = ?", (mem.id,)).fetchone()
|
||||
import json as _json
|
||||
assert _json.loads(row["domain_tags"]) == ["controls", "firmware"]
|
||||
assert row["valid_until"] == "2026-12-31"
|
||||
|
||||
|
||||
def test_get_memories_for_context_excludes_expired(isolated_db):
|
||||
"""Expired active memories must not land in context packs."""
|
||||
from atocore.memory.service import create_memory, get_memories_for_context
|
||||
|
||||
# Active but expired
|
||||
create_memory(
|
||||
"knowledge",
|
||||
"stale snapshot from long ago period",
|
||||
valid_until="2020-01-01",
|
||||
confidence=1.0,
|
||||
)
|
||||
# Active and valid
|
||||
create_memory(
|
||||
"knowledge",
|
||||
"durable engineering insight stays valid forever",
|
||||
confidence=1.0,
|
||||
)
|
||||
|
||||
text, _ = get_memories_for_context(memory_types=["knowledge"], budget=600)
|
||||
assert "durable engineering" in text
|
||||
assert "stale snapshot" not in text
|
||||
|
||||
|
||||
def test_context_builder_tag_boost_orders_results(isolated_db):
|
||||
"""Memories with tags matching query should rank higher."""
|
||||
from atocore.memory.service import create_memory, get_memories_for_context
|
||||
|
||||
create_memory("knowledge", "generic content has no obvious overlap with topic", confidence=0.8, domain_tags=[])
|
||||
create_memory("knowledge", "generic content has no obvious overlap topic here", confidence=0.8, domain_tags=["optics"])
|
||||
|
||||
text, _ = get_memories_for_context(
|
||||
memory_types=["knowledge"],
|
||||
budget=2000,
|
||||
query="tell me about optics",
|
||||
)
|
||||
# Tagged memory should appear before the untagged one
|
||||
idx_tagged = text.find("overlap topic here")
|
||||
idx_untagged = text.find("overlap with topic")
|
||||
assert idx_tagged != -1
|
||||
assert idx_untagged != -1
|
||||
assert idx_tagged < idx_untagged
|
||||
|
||||
|
||||
def test_expire_stale_candidates_keeps_reinforced(isolated_db):
|
||||
from atocore.memory.service import create_memory, expire_stale_candidates
|
||||
from atocore.models.database import get_connection
|
||||
|
||||
Reference in New Issue
Block a user