Antoine/ATOCore
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

test(harness): fix p06-tailscale false positive, 18/18 PASS

Browse Source 
The fixture's expect_absent: "GigaBIT" was catching legitimate
semantic overlap, not retrieval bleed. The p06 ARCHITECTURE.md
Overview describes the Polisher Suite as built for the GigaBIT M1
mirror — it is what the polisher is for, so the word appears
correctly in p06 content. All retrieved sources for this prompt
were genuinely p06/shared paths; zero actual p04 chunks leaked.

Narrowed the assertion to expect_absent: "[Source: p04-gigabit/",
which tests the real invariant (no p04 source chunks retrieved
into p06 context) without the false positive.

No retrieval/ranking code change. Fixture-only fix.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Anto01
2026-04-15 11:23:00 -04:00
parent 5b114baa87
commit 4d4d5f437a
2 changed files with 5 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
scripts/retrieval_eval_fixtures.json
View File

@@ -218,8 +218,8 @@
"Tailscale"
],
"expect_absent": [
"GigaBIT"
"[Source: p04-gigabit/"
],
"notes": "New p06 memory: Tailscale mesh for RPi remote access"
"notes": "New p06 memory: Tailscale mesh for RPi remote access. Cross-project guard is a source-path check, not a word blacklist: the polisher ARCHITECTURE.md legitimately mentions the GigaBIT M1 mirror (it is what the polisher is built for), so testing for absence of that word produces false positives. The real invariant is that no p04 source chunks are retrieved into p06 context."
}
]