feat: Phase 3 V1 — Auto-Organization (domain_tags + valid_until)
Adds structural metadata that the LLM triage was already implicitly
reasoning about ("stale snapshot" → reject). Phase 3 captures that
reasoning as fields so it can DRIVE retrieval, not just rejection.
Schema (src/atocore/models/database.py):
- domain_tags TEXT DEFAULT '[]' JSON array of lowercase topic keywords
- valid_until DATETIME ISO date; null = permanent
- idx_memories_valid_until index for efficient expiry queries
Memory service (src/atocore/memory/service.py):
- Memory dataclass gains domain_tags + valid_until
- create_memory, update_memory accept/persist both
- _row_to_memory safely reads both (JSON-decode + null handling)
- _normalize_tags helper: lowercase, dedup, strip, cap at 10
- get_memories_for_context filters expired (valid_until < today UTC)
- _rank_memories_for_query adds tag-boost: memories whose domain_tags
appear as substrings in query text rank higher (tertiary key after
content-overlap density + absolute overlap, before confidence)
LLM extractor (_llm_prompt.py → llm-0.5.0):
- SYSTEM_PROMPT documents domain_tags (2-5 keywords) + valid_until
(time-bounded facts get expiry dates; durable facts stay null)
- normalize_candidate_item parses both fields from model output with
graceful fallback for string/null/missing
LLM triage (scripts/auto_triage.py):
- TRIAGE_SYSTEM_PROMPT documents same two fields
- parse_verdict extracts them from verdict JSON
- On promote: PUT /memory/{id} with tags + valid_until BEFORE
POST /memory/{id}/promote, so active memories carry them
API (src/atocore/api/routes.py):
- MemoryCreateRequest: adds domain_tags, valid_until
- MemoryUpdateRequest: adds domain_tags, valid_until, memory_type
- GET /memory response exposes domain_tags + valid_until + created_at
Triage UI (src/atocore/engineering/triage_ui.py):
- Renders existing tags as colored badges
- Adds inline text field for tags (comma-separated) + date picker for
valid_until on every candidate card
- Save&Promote button persists edits via PUT then promotes
- Plain Promote (and Y shortcut) also saves tags/expiry if edited
Wiki (src/atocore/engineering/wiki.py):
- Search now matches memory content OR domain_tags
- Search results render tags as clickable badges linking to
/wiki/search?q=<tag> for cross-project navigation
- valid_until shown as amber "valid until YYYY-MM-DD" hint
Tests: 303 → 308 (5 new for Phase 3 behavior):
- test_create_memory_with_tags_and_valid_until
- test_create_memory_normalizes_tags
- test_update_memory_sets_tags_and_valid_until
- test_get_memories_for_context_excludes_expired
- test_context_builder_tag_boost_orders_results
Deferred (explicitly): temporal_scope enum, source_refs memory graph,
HDBSCAN clustering, memory detail wiki page, backfill of existing
actives. See docs/MASTER-BRAIN-PLAN.md.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -48,7 +48,25 @@ You will receive:
|
||||
|
||||
For each candidate, output exactly one JSON object:
|
||||
|
||||
{"verdict": "promote|reject|needs_human|contradicts", "confidence": 0.0-1.0, "reason": "one sentence", "conflicts_with": "id of existing memory if contradicts"}
|
||||
{"verdict": "promote|reject|needs_human|contradicts", "confidence": 0.0-1.0, "reason": "one sentence", "conflicts_with": "id of existing memory if contradicts", "domain_tags": ["tag1","tag2"], "valid_until": null}
|
||||
|
||||
DOMAIN TAGS (Phase 3): A lowercase list of 2-5 topical keywords describing
|
||||
the SUBJECT matter (not the project). This enables cross-project retrieval:
|
||||
a query about "optics" can pull matches from p04 + p05 + p06.
|
||||
|
||||
Good tags are single lowercase words or hyphenated terms. Mix:
|
||||
- domain keywords (optics, thermal, firmware, materials, controls)
|
||||
- project tokens when clearly scoped (p04, p05, p06, abb)
|
||||
- lifecycle/activity words (procurement, design, validation, vendor)
|
||||
|
||||
Always emit domain_tags on a promote. For reject, empty list is fine.
|
||||
|
||||
VALID_UNTIL (Phase 3): ISO date "YYYY-MM-DD" OR null (permanent).
|
||||
Set to a near-future date when the candidate is time-bounded:
|
||||
- Status snapshots ("current blocker is X") → ~2 weeks out
|
||||
- Scheduled events ("meeting Friday") → event date
|
||||
- Quotes with expiry → quote expiry date
|
||||
Leave null for durable decisions, engineering insights, ratified requirements.
|
||||
|
||||
Rules:
|
||||
|
||||
@@ -196,11 +214,36 @@ def parse_verdict(raw):
|
||||
|
||||
reason = str(parsed.get("reason", "")).strip()[:200]
|
||||
conflicts_with = str(parsed.get("conflicts_with", "")).strip()
|
||||
|
||||
# Phase 3: domain tags + expiry
|
||||
raw_tags = parsed.get("domain_tags") or []
|
||||
if isinstance(raw_tags, str):
|
||||
raw_tags = [t.strip() for t in raw_tags.split(",") if t.strip()]
|
||||
if not isinstance(raw_tags, list):
|
||||
raw_tags = []
|
||||
domain_tags = []
|
||||
for t in raw_tags[:10]:
|
||||
if not isinstance(t, str):
|
||||
continue
|
||||
tag = t.strip().lower()
|
||||
if tag and tag not in domain_tags:
|
||||
domain_tags.append(tag)
|
||||
|
||||
valid_until = parsed.get("valid_until")
|
||||
if valid_until is None:
|
||||
valid_until = ""
|
||||
else:
|
||||
valid_until = str(valid_until).strip()
|
||||
if valid_until.lower() in ("", "null", "none", "permanent"):
|
||||
valid_until = ""
|
||||
|
||||
return {
|
||||
"verdict": verdict,
|
||||
"confidence": confidence,
|
||||
"reason": reason,
|
||||
"conflicts_with": conflicts_with,
|
||||
"domain_tags": domain_tags,
|
||||
"valid_until": valid_until,
|
||||
}
|
||||
|
||||
|
||||
@@ -259,6 +302,24 @@ def main():
|
||||
if args.dry_run:
|
||||
print(f" WOULD PROMOTE {label} conf={conf:.2f} {reason}")
|
||||
else:
|
||||
# Phase 3: update tags + valid_until on the candidate
|
||||
# before promoting, so the active memory carries them.
|
||||
tags = verdict_obj.get("domain_tags") or []
|
||||
valid_until = verdict_obj.get("valid_until") or ""
|
||||
if tags or valid_until:
|
||||
try:
|
||||
import urllib.request as _ur
|
||||
body = json.dumps({
|
||||
"domain_tags": tags,
|
||||
"valid_until": valid_until,
|
||||
}).encode("utf-8")
|
||||
req = _ur.Request(
|
||||
f"{args.base_url}/memory/{mid}", method="PUT",
|
||||
headers={"Content-Type": "application/json"}, data=body,
|
||||
)
|
||||
_ur.urlopen(req, timeout=10).read()
|
||||
except Exception:
|
||||
pass # non-fatal; promote anyway
|
||||
try:
|
||||
api_post(args.base_url, f"/memory/{mid}/promote")
|
||||
print(f" PROMOTED {label} conf={conf:.2f} {reason}")
|
||||
|
||||
@@ -176,6 +176,8 @@ def parse_candidates(raw, interaction_project):
|
||||
"content": normalized["content"],
|
||||
"project": project,
|
||||
"confidence": normalized["confidence"],
|
||||
"domain_tags": normalized.get("domain_tags") or [],
|
||||
"valid_until": normalized.get("valid_until") or "",
|
||||
})
|
||||
return results
|
||||
|
||||
@@ -250,6 +252,8 @@ def main():
|
||||
"project": c["project"],
|
||||
"confidence": c["confidence"],
|
||||
"status": "candidate",
|
||||
"domain_tags": c.get("domain_tags") or [],
|
||||
"valid_until": c.get("valid_until") or "",
|
||||
})
|
||||
total_persisted += 1
|
||||
except urllib.error.HTTPError as exc:
|
||||
|
||||
Reference in New Issue
Block a user