Antoine/ATOCore
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(R7/R9): overlap-density ranking + project trust-preservation

Browse Source 
R7: ranking scorer now uses overlap-density (overlap_count /
memory_token_count) as primary key instead of raw overlap count.
A 5-token memory with 3 overlapping tokens (density 0.6) now beats
a 40-token overview memory with 3 overlapping tokens (density 0.075)
at the same absolute count. Secondary: absolute overlap. Tertiary:
confidence. Targeting p06-firmware-interface harness fixture.

R9: when the LLM extractor returns a project that differs from the
interaction's known project, it now checks the project registry.
If the model's project is a registered canonical ID, trust it. If
not (hallucinated name), fall back to the interaction's project.
Uses load_project_registry() for the check. The host-side script
mirrors this via an API call to GET /projects at startup.

Two new tests: test_parser_keeps_registered_model_project and
test_parser_rejects_hallucinated_project.

Test count: 280 -> 281.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Anto01
2026-04-12 14:34:33 -04:00
parent 1a2ee5e07f
commit 8951c624fe
4 changed files with 78 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files

20
tests/test_extractor_llm.py
View File

@@ -107,8 +107,11 @@ def test_parser_falls_back_to_interaction_project():
assert result[0].project == "p06-polisher"
def test_parser_keeps_model_project_when_provided():
"""Model-supplied project takes precedence over interaction."""
def test_parser_keeps_registered_model_project(tmp_data_dir, project_registry):
"""R9: model-supplied project is kept when it's a registered project."""
from atocore.models.database import init_db
init_db()
project_registry(("p04-gigabit", ["p04", "gigabit"]), ("p06-polisher", ["p06"]))
raw = '[{"type": "project", "content": "x", "project": "p04-gigabit"}]'
interaction = _make_interaction()
interaction.project = "p06-polisher"
@@ -116,6 +119,19 @@ def test_parser_keeps_model_project_when_provided():
assert result[0].project == "p04-gigabit"
def test_parser_rejects_hallucinated_project(tmp_data_dir, project_registry):
"""R9: model-supplied project that is NOT registered falls back
to the interaction's known project."""
from atocore.models.database import init_db
init_db()
project_registry(("p06-polisher", ["p06"]))
raw = '[{"type": "project", "content": "x", "project": "fake-project-99"}]'
interaction = _make_interaction()
interaction.project = "p06-polisher"
result = _parse_candidates(raw, interaction)
assert result[0].project == "p06-polisher"
def test_missing_cli_returns_empty(monkeypatch):
"""If ``claude`` is not on PATH the extractor returns empty, never raises."""
monkeypatch.setattr(extractor_llm, "_cli_available", lambda: False)