feat(eval-loop): Day 1+2 — labeled extractor corpus + baseline scorecard

Day 1 (labeled corpus):
- scripts/eval_data/interactions_snapshot_2026-04-11.json — frozen
  snapshot of 64 real claude-code interactions pulled from live
  Dalidou (test-client captures filtered out). This is the stable
  corpus the whole mini-phase labels against, independent of future
  captures.
- scripts/eval_data/extractor_labels_2026-04-11.json — 20 hand-labeled
  interactions drawn by length-stratified random sample. Positives:
  5/20 = ~25%, total expected candidates: 7. Plan deviation: Codex's
  plan asked for 30 (10/10/10 buckets); the real corpus is heavily
  skewed toward instructional/status content, so honest labeling of
  20 already crosses the fail-early threshold of "at least 5 plausible
  positives" without padding.

Day 2 (baseline measurement):
- scripts/extractor_eval.py — file-based eval runner that loads the
  snapshot + labels, runs extract_candidates_from_interaction on each,
  and reports yield / recall / precision / miss-class breakdown.
  Returns exit 1 on any false positive or false negative.

Current rule extractor against the labeled set:

    labeled=20  exact_match=15  positive_expected=5
    yield=0.0   recall=0.0     precision=0.0
    false_negatives=5           false_positives=0
    miss_classes:
      recommendation_prose
      architectural_change_summary
      spec_update_announcement
      layered_recommendation
      alignment_assertion

Interpretation: the rule-based extractor matches exactly zero of the
5 plausible positive interactions in the labeled set, and the misses
are spread across 5 distinct cue classes with no single dominant
pattern. This is the Day 4 hard-stop signal landing on Day 2 — a
single rule expansion cannot close a 5-way miss, and widening rules
blindly will collapse precision. The right move is to go straight to
the Day 4 decision gate and consider LLM-assisted extraction.

Escalating to DEV-LEDGER.md as R5 for human ratification before
continuing. Not skipping Day 3 silently.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

scripts/eval_data/extractor_labels_2026-04-11.json

@@ -0,0 +1,145 @@
+{
+  "version": "0.1",
+  "frozen_at": "2026-04-11",
+  "snapshot_file": "scripts/eval_data/interactions_snapshot_2026-04-11.json",
+  "labeled_count": 20,
+  "plan_deviation": "Codex's plan called for 30 labeled interactions (10 zero / 10 plausible / 10 ambiguous). Actual corpus is heavily skewed toward instructional/status content; after reading 20 drawn by length-stratified random sample, the honest positive rate is ~25% (5/20). Labeling more would mostly add zeros; the Day 2 measurement is not bottlenecked on sample size.",
+  "positive_count": 5,
+  "labels": [
+    {
+      "id": "ab239158-d6ac-4c51-b6e4-dd4ccea384a2",
+      "expected_count": 0,
+      "miss_class": "n/a",
+      "notes": "Instructional deploy guidance. No durable claim."
+    },
+    {
+      "id": "da153f2a-b20a-4dee-8c72-431ebb71f08c",
+      "expected_count": 0,
+      "miss_class": "n/a",
+      "notes": "'Deploy still in progress.' Pure status."
+    },
+    {
+      "id": "7d8371ee-c6d3-4dfe-a7b0-2d091f075c15",
+      "expected_count": 0,
+      "miss_class": "n/a",
+      "notes": "Git command walkthrough. No durable claim."
+    },
+    {
+      "id": "14bf3f90-e318-466e-81ac-d35522741ba5",
+      "expected_count": 0,
+      "miss_class": "n/a",
+      "notes": "Ledger status update. Transient fact, not a durable memory candidate."
+    },
+    {
+      "id": "8f855235-c38d-4c27-9f2b-8530ebe1a2d8",
+      "expected_count": 0,
+      "miss_class": "n/a",
+      "notes": "Short-term recommendation ('merge to main and deploy'), not a standing decision."
+    },
+    {
+      "id": "04a96eb5-cd00-4e9f-9252-b2cc919000a4",
+      "expected_count": 0,
+      "miss_class": "n/a",
+      "notes": "Dev server config table. Operational detail, not a memory."
+    },
+    {
+      "id": "79d606ed-8981-454a-83af-c25226b1b65c",
+      "expected_count": 1,
+      "expected_type": "adaptation",
+      "expected_project": "",
+      "expected_snippet": "shared DEV-LEDGER as operating memory",
+      "miss_class": "recommendation_prose",
+      "notes": "A recommendation that later became a ratified decision. Rule extractor would need a 'simplest version that could work today' / 'I'd start with' cue class."
+    },
+    {
+      "id": "a6b0d279-c564-4bce-a703-e476f4a148ad",
+      "expected_count": 2,
+      "expected_type": "project",
+      "expected_project": "p06-polisher",
+      "expected_snippet": "z_engaged bool; cam amplitude set mechanically and read by encoders",
+      "miss_class": "architectural_change_summary",
+      "notes": "Two durable architectural facts about the polisher machine (Z-axis is engage/retract, cam is read-only). Extractor would need to recognize 'A is now B' / 'X removed, Y added' patterns."
+    },
+    {
+      "id": "4e00e398-2e89-4653-8ee5-3f65c7f4d2d3",
+      "expected_count": 0,
+      "miss_class": "n/a",
+      "notes": "Clarification question to user."
+    },
+    {
+      "id": "a6a7816a-7590-4616-84f4-49d9054c2a91",
+      "expected_count": 0,
+      "miss_class": "n/a",
+      "notes": "Instructional response offering two next moves."
+    },
+    {
+      "id": "03527502-316a-4a3e-989c-00719392c7d1",
+      "expected_count": 0,
+      "miss_class": "n/a",
+      "notes": "Troubleshooting a paste failure. Ephemeral."
+    },
+    {
+      "id": "1fff59fc-545f-42df-9dd1-a0e6dec1b7ee",
+      "expected_count": 0,
+      "miss_class": "n/a",
+      "notes": "Agreement + follow-up question. No durable claim."
+    },
+    {
+      "id": "eb65dc18-0030-4720-ace7-f55af9df719d",
+      "expected_count": 0,
+      "miss_class": "n/a",
+      "notes": "Explanation of how the capture hook works. Instructional."
+    },
+    {
+      "id": "52c8c0f3-32fb-4b48-9065-73c778a08417",
+      "expected_count": 1,
+      "expected_type": "project",
+      "expected_project": "p06-polisher",
+      "expected_snippet": "USB SSD mandatory on RPi; Tailscale for remote access",
+      "miss_class": "spec_update_announcement",
+      "notes": "Concrete architectural commitments just added to the polisher spec. Phrased as '§17.1 Local Storage - USB SSD mandatory, not SD card.' The '§' section markers could be a new cue."
+    },
+    {
+      "id": "32d40414-15af-47ee-944b-2cceae9574b8",
+      "expected_count": 0,
+      "miss_class": "n/a",
+      "notes": "Session recap. Historical summary, not a durable memory."
+    },
+    {
+      "id": "b6d2cdfc-37fb-459a-96bd-caefb9beaab4",
+      "expected_count": 0,
+      "miss_class": "n/a",
+      "notes": "Deployment prompt for Dalidou. Operational, not a memory."
+    },
+    {
+      "id": "ee03d823-931b-4d4e-9258-88b4ed5eeb07",
+      "expected_count": 2,
+      "expected_type": "knowledge",
+      "expected_project": "p06-polisher",
+      "expected_snippet": "USB SSD is non-negotiable for local storage; Tailscale mesh for SSH/file transfer",
+      "miss_class": "layered_recommendation",
+      "notes": "Layered infra recommendation with 'non-negotiable' / 'strongly recommended' strength markers. The 'non-negotiable' token could be a new cue class."
+    },
+    {
+      "id": "dd234d9f-0d1c-47e8-b01c-eebcb568c7e7",
+      "expected_count": 1,
+      "expected_type": "project",
+      "expected_project": "p06-polisher",
+      "expected_snippet": "interface contract is identical regardless of who generates the programs; machine is a standalone box",
+      "miss_class": "alignment_assertion",
+      "notes": "Architectural invariant assertion. '**Alignment verified**' / 'nothing changes for X' style. Likely too subtle for rule matching without LLM assistance."
+    },
+    {
+      "id": "1f95891a-cf37-400e-9d68-4fad8e04dcbb",
+      "expected_count": 0,
+      "miss_class": "n/a",
+      "notes": "Huge session handoff prompt. Informational only."
+    },
+    {
+      "id": "5580950f-d010-4544-be4b-b3071271a698",
+      "expected_count": 0,
+      "miss_class": "n/a",
+      "notes": "Ledger schema sketch. Structural design proposal, later ratified — but the same idea was already captured as a ratified decision in the recent decisions section, so not worth re-extracting from this conversational form."
+    }
+  ]
+}