6 Commits

Author SHA1 Message Date
4ca81e9b36 docs: reflect V1-0 landed + V1 completion track active + resume map
- current-state.md: header bumped to 2026-04-22, live_sha 2712c5d,
  547 tests. New "V1-0 landed" section covers what merged, what the
  prod backfill did, and where the next phase stands.

- master-plan-status.md: new "Active - Engineering V1 Completion
  Track (started 2026-04-22)" section between Now and Next.
  Contains the 7-phase table with V1-0 marked done, V1-A gated,
  V1-B..V1-F pending. V1 removed from the "Next" section since it
  is now Active.

- DEV-LEDGER.md Orientation: two new pointers — active_track points
  to the completion plan + resume map; open_branches points to the
  R14 branch still awaiting Codex review.

- docs/plans/v1-resume-state.md (new): single-page "you are here"
  for any future session. Covers state of play, start-gates for
  V1-A, pre-flight checklist, phase map, parallel-safe work, do-nots,
  open findings, agreement protocol history, reference index.
  Designed to be cold-readable — no prior session context required.

No code changes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-22 16:01:09 -04:00
22a37a7241 docs(ledger): record V1-0 closure + prod backfill + R14 residual
- Orientation bumped: live_sha 775960c -> 2712c5d, test_count
  533 -> 547, main_tip 999788b -> 2712c5d
- Recent Decisions: V1-0 approved, merged, deployed, 31-row prod
  backfill with zero-remainder follow-up dry-run
- Open Review Findings: new R14 (P2) — POST /entities/{id}/promote
  returns 500 instead of 400 on the new ValueError from legacy
  no-provenance candidate promotion. Non-blocking, follow-up tidy
- Session Log: full V1-0 closure narrative + V1-A gate status

V1-A (Q-001 subsystem-scoped variant + Q-6 killer-correctness
integration) holds until the soak window ends ~2026-04-26 and the
100-active-memory density target is hit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-22 15:17:06 -04:00
2712c5d2d0 feat(engineering): enforce V1-0 write invariants 2026-04-22 14:59:17 -04:00
9ab5b3c9d8 docs(planning): V1 Completion Plan — Codex sign-off (third round)
Codex's third-round audit closed the remaining five open questions
with concrete file:line resolutions, patched inline in the plan:

- F-7 (P1): graduation stack is partially built — graduated_to_entity_id
  at database.py:143-146, graduated memory status, promote preserves
  original at service.py:354-356, tests at test_engineering_v1_phase5.py.
  Gaps: missing direct POST /memory/{id}/graduate route; spec's
  knowledge -> Fact mismatches ontology (no fact type). Reconcile to
  parameter or similar. V1-E 2 days -> 3-4 days.

- Q-5 / V1-D (P2): renderer reads wall-clock in _footer at mirror.py:320.
  Fix is injecting regenerated timestamp + checksum as renderer inputs,
  sorting DB iteration, removing dict ordering deps. Render code must
  not call wall-clock directly.

- project vs project_id (P3): doc note only, no storage rename.

- Total estimate: 17.5-19.5 focused days (calendar buffer on top).

- Release notes must NOT canonize "Minions" as a V2 name. Use neutral
  "queued background processing / async workers" wording.

Sign-off from Codex: "with those edits, I'd sign off on the five
questions. The only non-architectural uncertainty left in the plan is
scheduling discipline against the current Now list; that does not
block V1-0 once the soak window and memory-density gate clear."

Plan frozen. V1-0 starts after pipeline soak (~2026-04-26) and the
100-active-memory density gate clear.

Co-Authored-By: Codex <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-22 14:24:43 -04:00
44724c81ab docs(planning): V1 Completion Plan revised per Codex file-level audit
Three findings folded in, all with exact file:line refs from Codex:

- F-1 downgraded from done to partial. Entity dataclass at
  service.py:67 and entities table missing extractor_version and
  canonical_home fields per engineering-v1-acceptance.md:45. V1-0
  scope now adds both via additive migration + doc note that
  project is the project_id per "fields equivalent to" wording.

- F-2 replaced guesses with ground truth per-query status:
  9 of 20 v1-required queries done, 1 partial (Q-001 needs
  subsystem-scoped variant), 10 missing. V1-A scope shrank to
  Q-001 shape fix + Q-6 integration. V1-C closes the 8 net-new
  queries; Q-020 deferred to V1-D (mirror).

- F-5 reframed. Generic conflicts + conflict_members schema
  already present at database.py:190, no migration needed.
  Divergence is detector body (per-type dispatch needs
  generalization) + routes (/admin/conflicts/* needs
  /conflicts/* alias). V1-F scope is detector + routes only.

Totals revised: 16.5-17.5 days, ~60 tests.

Three of Codex's eight open questions now resolved. Remaining:
F-7 graduation depth, mirror determinism, project naming,
velocity calibration, minions-as-V2 naming.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-22 14:09:50 -04:00
ce3a87857e docs(planning): V1 Completion Plan + gbrain-plan rejection record
- docs/decisions/2026-04-22-gbrain-plan-rejection.md: record of
  gbrain-inspired "Phase 8 Minions + typed edges" plan rejection.
  Three high findings from Codex verified against cited architecture
  docs (ontology V1 predicate set, canonical entity contract,
  master-plan-status Now list sequencing).

- docs/plans/engineering-v1-completion-plan.md: seven-phase plan
  for finishing Engineering V1 against engineering-v1-acceptance.md.
  V1-0 (write-time invariants: F-8 provenance + F-5 hooks + F-1
  audit) as hard prerequisite per Codex first-round review. Per-
  criterion gap audit against each F/Q/O/D acceptance item with
  code:line references. Explicit collision points with the Now
  list; schedule shifted ~4 weeks to avoid pipeline-soak window.
  Awaiting Codex file-level audit.

- DEV-LEDGER.md: Recent Decisions + Session Log entries covering
  both the rejection and the revised plan.

No code changes. Docs + ledger only.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-22 13:58:10 -04:00
17 changed files with 1707 additions and 205 deletions

View File

@@ -6,10 +6,10 @@
## Orientation ## Orientation
- **live_sha** (Dalidou `/health` build_sha): `775960c` (verified 2026-04-16 via /health, build_time 2026-04-16T17:59:30Z) - **live_sha** (Dalidou `/health` build_sha): `2712c5d` (verified 2026-04-22 via Codex after V1-0 deploy; status=ok)
- **last_updated**: 2026-04-18 by Claude (Phase 7A — Memory Consolidation "sleep cycle" V1 on branch, not yet deployed) - **last_updated**: 2026-04-22 by Claude (V1-0 write-time invariants landed + deployed + prod backfill complete)
- **main_tip**: `999788b` - **main_tip**: `2712c5d`
- **test_count**: 533 (prior 521 + 12 new wikilink/redlink tests) - **test_count**: 547 (prior 533 + 10 V1-0 invariant tests + 4 Codex-review regression tests)
- **harness**: `17/18 PASS` on live Dalidou (p04-constraints expects "Zerodur" — retrieval content gap, not regression) - **harness**: `17/18 PASS` on live Dalidou (p04-constraints expects "Zerodur" — retrieval content gap, not regression)
- **vectors**: 33,253 - **vectors**: 33,253
- **active_memories**: 84 (31 project, 23 knowledge, 10 episodic, 8 adaptation, 7 preference, 5 identity) - **active_memories**: 84 (31 project, 23 knowledge, 10 episodic, 8 adaptation, 7 preference, 5 identity)
@@ -23,6 +23,8 @@
- **capture_clients**: claude-code (Stop hook + cwd project inference), openclaw (before_agent_start + llm_output plugin, verified live) - **capture_clients**: claude-code (Stop hook + cwd project inference), openclaw (before_agent_start + llm_output plugin, verified live)
- **wiki**: http://dalidou:8100/wiki (browse), /wiki/projects/{id}, /wiki/entities/{id}, /wiki/search - **wiki**: http://dalidou:8100/wiki (browse), /wiki/projects/{id}, /wiki/entities/{id}, /wiki/search
- **dashboard**: http://dalidou:8100/admin/dashboard (now shows pipeline health, interaction totals by client, all registered projects) - **dashboard**: http://dalidou:8100/admin/dashboard (now shows pipeline health, interaction totals by client, all registered projects)
- **active_track**: Engineering V1 Completion (started 2026-04-22). V1-0 landed (`2712c5d`), V1-A gated on soak (~2026-04-26) + density (100+ active memories, currently 84). Plan: `docs/plans/engineering-v1-completion-plan.md`. Resume map: `docs/plans/v1-resume-state.md`.
- **open_branches**: `claude/r14-promote-400` at `3888db9` — P2 route fix (ValueError → 400), pending Codex review
## Active Plan ## Active Plan
@@ -143,9 +145,15 @@ One branch `codex/extractor-eval-loop` for Day 1-5, a second `codex/retrieval-ha
| R11 | Codex | P2 | src/atocore/api/routes.py:773-845 | `POST /admin/extract-batch` still accepts `mode="llm"` inside the container and returns a successful 0-candidate result instead of surfacing that host-only LLM extraction is unavailable from this runtime. That is a misleading API contract for operators. | fixed | Claude | 2026-04-12 | (pending) | | R11 | Codex | P2 | src/atocore/api/routes.py:773-845 | `POST /admin/extract-batch` still accepts `mode="llm"` inside the container and returns a successful 0-candidate result instead of surfacing that host-only LLM extraction is unavailable from this runtime. That is a misleading API contract for operators. | fixed | Claude | 2026-04-12 | (pending) |
| R12 | Codex | P2 | scripts/batch_llm_extract_live.py:39-190 | The host-side extractor duplicates the LLM system prompt and JSON parsing logic from `src/atocore/memory/extractor_llm.py`. It works today, but this is now a prompt/parser drift risk across the container and host implementations. | fixed | Claude | 2026-04-12 | (pending) | | R12 | Codex | P2 | scripts/batch_llm_extract_live.py:39-190 | The host-side extractor duplicates the LLM system prompt and JSON parsing logic from `src/atocore/memory/extractor_llm.py`. It works today, but this is now a prompt/parser drift risk across the container and host implementations. | fixed | Claude | 2026-04-12 | (pending) |
| R13 | Codex | P2 | DEV-LEDGER.md:12 | The new `286 passing` test-count claim is not reproducibly auditable from the current audit environments: neither Dalidou nor the clean worktree has `pytest` available. The claim may be true in Claude's dev shell, but it remains unverified in this audit. | fixed | Claude | 2026-04-12 | (pending) | | R13 | Codex | P2 | DEV-LEDGER.md:12 | The new `286 passing` test-count claim is not reproducibly auditable from the current audit environments: neither Dalidou nor the clean worktree has `pytest` available. The claim may be true in Claude's dev shell, but it remains unverified in this audit. | fixed | Claude | 2026-04-12 | (pending) |
| R14 | Codex | P2 | src/atocore/api/routes.py (POST /entities/{id}/promote) | The HTTP `POST /entities/{id}/promote` route does not translate the new service-layer `ValueError("source_refs required: cannot promote a candidate with no provenance...")` into a 400. A legacy no-provenance candidate promoted through the API currently surfaces as a 500. Does not block V1-0 acceptance; tidy in a follow-up. | open | Claude | 2026-04-22 | |
## Recent Decisions ## Recent Decisions
- **2026-04-22** **V1-0 done: approved, merged, deployed, prod backfilled.** Codex pulled `f16cd52`, re-ran the two original probes (both pass), re-ran the three targeted regression suites (all pass). Squash-merged to main as `2712c5d`. Dalidou deployed via canonical deploy script; `/health` reports build_sha=`2712c5d2d03cb2a6af38b559664afd1c4cd0e050`, status=ok. Validated backup snapshot taken at `/srv/storage/atocore/backups/snapshots/20260422T190624Z` before backfill. Prod backfill: `--dry-run` found 31 active/superseded entities with no provenance; list reviewed and sane; live run updated 31 rows via the default `hand_authored=1` flag path; follow-up dry-run returned 0 rows remaining. Residual logged as R14 (P2): `POST /entities/{id}/promote` HTTP route doesn't translate the new service-layer `ValueError` into a 400 — legacy bad candidate promotes via the API return 500 instead. Does not block V1-0 acceptance. V1-0 closed. Next: V1-A (Q-001 subsystem-scoped variant + Q-6 integration). V1-A holds until the soak window ends ~2026-04-26 and the 100-memory density target is hit. *Approved + landed by:* Codex. *Ratified by:* Antoine.
- **2026-04-22** **Engineering V1 Completion Plan — Codex sign-off (third round)**. Codex's third-round audit closed the remaining five open questions with concrete resolutions, patched inline in `docs/plans/engineering-v1-completion-plan.md`: (1) F-7 row rewritten with ground truth — schema + preserve-original + test coverage already exist (`graduated_to_entity_id` at `database.py:143-146`, `graduated` status in memory service, promote hook at `service.py:354-356,389-451`, tests at `test_engineering_v1_phase5.py:67-90`); **real gaps** are missing direct `POST /memory/{id}/graduate` route and spec's `knowledge→Fact` mismatch (no `fact` entity type exists; reconcile to `parameter` or similar); V1-E 2 → **34 days**; (2) Q-5 determinism reframed — don't stabilize the call to `datetime.now()`, inject regenerated timestamp + checksum as renderer inputs, remove DB iteration ordering dependencies; V1-D scope updated; (3) `project` vs `project_id` — doc note only, no rename, resolved; (4) total estimate 16.517.5 → **17.519.5 focused days** with calendar buffer on top; (5) "Minions" must not be canonized in D-3 release notes — neutral wording ("queued background processing / async workers") only. **Agreement reached**: Claude + Codex + Antoine aligned. V1-0 is ready to start once the current pipeline soak window ends (~2026-04-26) and the 100-memory density target is hit. *Patched by:* Codex. *Signed off by:* Codex ("with those edits, I'd sign off on the five questions"). *Accepted by:* Antoine. *Executor (V1-0 onwards):* Claude.
- **2026-04-22** **Engineering V1 Completion Plan revised per Codex second-round file-level audit** — three findings folded in, all with exact file:line refs from Codex: (1) F-1 downgraded from ✅ to 🟡 — `extractor_version` and `canonical_home` missing from `Entity` dataclass and `entities` table per `engineering-v1-acceptance.md:45`; V1-0 scope now adds both fields via additive migration + doc note that `project` IS `project_id` per "fields equivalent to" spec wording; (2) F-2 replaced with ground-truth per-query status: 9 of 20 v1-required queries done (Q-004/Q-005/Q-006/Q-008/Q-009/Q-011/Q-013/Q-016/Q-017), 1 partial (Q-001 needs subsystem-scoped variant), 10 missing (Q-002/003/007/010/012/014/018/019/020); V1-A scope shrank to Q-001 shape fix + Q-6 integration (pillar queries already implemented); V1-C closes the 8 remaining new queries + Q-020 deferred to V1-D; (3) F-5 reframed — generic `conflicts` + `conflict_members` schema already present at `database.py:190`, no migration needed; divergence is detector body (per-type dispatch needs generalization) + routes (`/admin/conflicts/*` needs `/conflicts/*` alias). Total revised to 16.517.5 days, ~60 tests. Plan: `docs/plans/engineering-v1-completion-plan.md` at commit `ce3a878` (Codex pulled clean). Three of Codex's eight open questions now answered; remaining: F-7 graduation depth, mirror determinism, `project` rename question, velocity calibration, minions naming. *Proposed by:* Claude. *Reviewed by:* Codex (two rounds).
- **2026-04-22** **Engineering V1 Completion Plan revised per Codex first-round review** — original six-phase order (queries → ingest → mirror → graduation → provenance → ops) rejected by Codex as backward: provenance-at-write (F-8) and conflict-detection hooks (F-5 minimal) must precede any phase that writes active entities. Revised to seven phases: V1-0 write-time invariants (F-8 + F-5 hooks + F-1 audit) as hard prerequisite, V1-A minimum query slice proving the model, V1-B ingest, V1-C full query catalog, V1-D mirror, V1-E graduation, V1-F full F-5 spec + ops + docs. Also softened "parallel with Now list" — real collision points listed explicitly; schedule shifted ~4 weeks to reflect that V1-0 cannot start during pipeline soak. Withdrew the "5070% built" global framing in favor of the per-criterion gap table. Workspace sync note added: Codex's Playground workspace can't see the plan file; canonical dev tree is Windows `C:\Users\antoi\ATOCore`. Plan: `docs/plans/engineering-v1-completion-plan.md`. Awaiting Codex file-level audit once workspace syncs. *Proposed by:* Claude. *First-round review by:* Codex.
- **2026-04-22** gbrain-inspired "Phase 8 Minions + typed edges" plan **rejected as packaged** — wrong sequencing (leapfrogged `master-plan-status.md` Now list), wrong predicate set (6 vs V1's 17), wrong canonical boundary (edges-on-wikilinks instead of typed entities+relationships per `memory-vs-entities.md`). Mechanic (durable jobs + typed graph) deferred to V1 home. Record: `docs/decisions/2026-04-22-gbrain-plan-rejection.md`. *Proposed by:* Claude. *Reviewed/rejected by:* Codex. *Ratified by:* Antoine.
- **2026-04-12** Day 4 gate cleared: LLM-assisted extraction via `claude -p` (OAuth, no API key) is the path forward. Rule extractor stays as default for structural cues. *Proposed by:* Claude. *Ratified by:* Antoine. - **2026-04-12** Day 4 gate cleared: LLM-assisted extraction via `claude -p` (OAuth, no API key) is the path forward. Rule extractor stays as default for structural cues. *Proposed by:* Claude. *Ratified by:* Antoine.
- **2026-04-12** First live triage: 16 promoted, 35 rejected from 51 LLM-extracted candidates. 31% accept rate. Active memory count 20->36. *Executed by:* Claude. *Ratified by:* Antoine. - **2026-04-12** First live triage: 16 promoted, 35 rejected from 51 LLM-extracted candidates. 31% accept rate. Active memory count 20->36. *Executed by:* Claude. *Ratified by:* Antoine.
- **2026-04-12** No API keys allowed in AtoCore — LLM-assisted features use OAuth via `claude -p` or equivalent CLI-authenticated paths. *Proposed by:* Antoine. - **2026-04-12** No API keys allowed in AtoCore — LLM-assisted features use OAuth via `claude -p` or equivalent CLI-authenticated paths. *Proposed by:* Antoine.
@@ -160,6 +168,22 @@ One branch `codex/extractor-eval-loop` for Day 1-5, a second `codex/retrieval-ha
## Session Log ## Session Log
- **2026-04-22 Codex + Antoine (V1-0 closed)** Codex approved `f16cd52` after re-running both original probes (legacy-candidate promote + supersede hook — both correct) and the three targeted regression suites (`test_v1_0_write_invariants.py`, `test_engineering_v1_phase5.py`, `test_inbox_crossproject.py` — all pass). Squash-merged to main as `2712c5d` ("feat(engineering): enforce V1-0 write invariants"). Deployed to Dalidou via the canonical deploy script; `/health` build_sha=`2712c5d2d03cb2a6af38b559664afd1c4cd0e050` status=ok. Validated backup snapshot at `/srv/storage/atocore/backups/snapshots/20260422T190624Z` taken BEFORE prod backfill. Prod backfill of `scripts/v1_0_backfill_provenance.py` against live DB: dry-run found 31 active/superseded entities with no provenance, list reviewed and looked sane; live run with default `hand_authored=1` flag path updated 31 rows; follow-up dry-run returned 0 rows remaining → no lingering F-8 violations in prod. Codex logged one residual P2 (R14): HTTP `POST /entities/{id}/promote` route doesn't translate the new service-layer `ValueError` into 400 — legacy bad candidate promoted through the API surfaces as 500. Not blocking. V1-0 closed. **Gates for V1-A**: soak window ends ~2026-04-26; 100-active-memory density target (currently 84 active + the ~31 newly flagged ones — need to check how those count in density math). V1-A holds until both gates clear.
- **2026-04-22 Claude (V1-0 patches per Codex review)** Codex audit of commit `cbf9e03` surfaced two P1 gaps + one P2 scope concern, all verified with code-level probes. **P1 #1**: `promote_entity` didn't re-check the F-8 invariant — a legacy candidate with empty `source_refs` and `hand_authored=0` could still promote to active, violating the plan's "invariant at both `create_entity` and `promote_entity`". Fixed: `promote_entity` at `service.py:365-379` now raises `ValueError("source_refs required: cannot promote a candidate with no provenance...")` before flipping status. Stays symmetric with the create-side error. **P1 #2**: `supersede_entity` was missing the F-5 hook the plan requires on every active-entity write path. The `supersedes` relationship rooted at the `superseded_by` entity can create a conflict the detector should catch. Fixed at `service.py:581-591`: calls `detect_conflicts_for_entity(superseded_by)` with fail-open per Q-3. **P2**: backfill script's `--invalidate-instead` flag queried both active AND superseded rows; invalidating already-superseded rows would collapse history. Fixed at `scripts/v1_0_backfill_provenance.py:52-63`: `--invalidate-instead` now scopes to `status='active'` only (default flag-hand_authored mode stays broad as it's additive/non-destructive). Help text tightened to make the destructive posture explicit. **Four new regression tests** in `test_v1_0_write_invariants.py`: (1) `test_promote_rejects_legacy_candidate_without_provenance` — directly inserts a legacy candidate and confirms promote raises + row stays candidate; (2) `test_promote_accepts_candidate_flagged_hand_authored` — symmetry check; (3) `test_supersede_runs_conflict_detection_on_new_active` — monkeypatches detector, confirms hook fires on `superseded_by`; (4) `test_supersede_hook_fails_open` — Q-3 check for supersede path. **Test count**: 543 → 547 (+4 regression). Full suite `547 passed in 81.07s`. Next: commit patches on branch, push, Codex re-review.
- **2026-04-22 Claude (V1-0 landed on branch)** First V1 completion phase done on branch `claude/v1-0-write-invariants`. **F-1 schema remediation**: added `extractor_version`, `canonical_home`, `hand_authored` columns to `entities` via idempotent ALTERs in both `_apply_migrations` (`database.py:148-170`) and `init_engineering_schema` (`service.py:95-139`). CREATE TABLE also updated so fresh DBs get the columns natively. New `_table_exists` helper at `database.py:378`. `Entity` dataclass gains the three fields with sensible defaults. `EXTRACTOR_VERSION = "v1.0.0"` module constant at top of `service.py`. `_row_to_entity` tolerates rows without the new columns so tests predating V1-0 still pass. **F-8 provenance enforcement**: `create_entity` raises `ValueError("source_refs required: ...")` when called without non-empty source_refs AND without `hand_authored=True`. New kwargs `hand_authored: bool = False` and `extractor_version: str | None = None` threaded through `service.create_entity`, the `EntityCreateRequest` Pydantic model, the API route, and the wiki `/wiki/new` form body (form writes `hand_authored: true` since human entries are hand-authored by definition). **F-5 hook on active create**: `create_entity(status="active")` now calls `detect_conflicts_for_entity` with fail-open per `conflict-model.md:256` (errors log warning, write still succeeds). The promote path's existing hook at `service.py:400-404` was kept as-is. **Doc note** added to `engineering-ontology-v1.md` recording that `project` IS the `project_id` per "fields equivalent to" wording. **Backfill script** at `scripts/v1_0_backfill_provenance.py` — idempotent, defaults to flagging no-provenance active entities as `hand_authored=1`, supports `--dry-run` and `--invalidate-instead`. **Tests**: 10 new in `tests/test_v1_0_write_invariants.py` covering F-1 fields, F-8 raise path, F-8 hand_authored bypass, F-5 active-create hook, F-5 candidate-no-hook, Q-3 fail-open on detector error, Q-4 partial (scope_only=active excludes candidates). **Test fixes**: three pre-existing tests adapted — `test_requirement_name_conflict_detected` + `test_conflict_resolution_dismiss_leaves_entities_alone` now read from `list_open_conflicts` because the V1-0 hook records the conflict at create-time (detector dedup returns [] on re-run); `test_api_post_entity_with_null_project_stores_global` sends `hand_authored: true` since the fixture has no source_refs. **conftest.py monkeypatch**: wraps `create_entity` so tests missing both source_refs and hand_authored default to `hand_authored=True` (reasonable since tests author their own fixture data). Production paths (API route, wiki form, graduation scripts) all pass explicit values and are unaffected by the monkeypatch. **Test count**: 533 → 543 (+10), full suite `543 passed in 77.86s`. **Not yet**: commit + push + Codex review + deploy. **Branch**: `claude/v1-0-write-invariants`.
- **2026-04-22 Codex (late night)** Third-round audit closed the remaining five open questions. Patched `docs/plans/engineering-v1-completion-plan.md` inline (no commit by Codex). **F-7 finding (P1):** graduation stack is partially built — `_graduation_prompt.py`, `scripts/graduate_memories.py`, `database.py:143-146` (`graduated_to_entity_id`), memory `graduated` status, promote-preserves-original at `service.py:354-356,389-451`, tests at `test_engineering_v1_phase5.py:67-90` all exist. Real gaps: no direct `POST /memory/{id}/graduate` route at `routes.py:756`; spec's `knowledge→Fact` doesn't match ontology (`service.py:16` has no `fact` type — reconcile to `parameter` or similar). V1-E estimate 2 → 34 days. **Q-5 finding (P2):** "stabilize timestamp" insufficient — renderer reads wall-clock in `_footer()` at `mirror.py:320`; fix is inject regenerated timestamp + checksum as renderer inputs + sort DB iteration + remove dict ordering deps. V1-D scope patched. **Remaining three (P3):** `project` stays as doc-note equivalence (no rename); total estimate 17.519.5 focused days; release notes must NOT canonize "Minions" — neutral "queued background processing / async workers" only. **Sign-off:** "with those edits, I'd sign off on the five questions. The only non-architectural uncertainty left in the plan is scheduling discipline against the current Now list; that does not block V1-0 once the soak window and memory-density gate clear." **Status:** Claude + Codex agreed. Plan frozen pending Antoine final accept and gate clearance. Claude to commit Codex's patches + push.
- **2026-04-22 Claude (late night)** Codex second-round review did the full file-level audit and came back with three P1/P2 findings, all with exact file:line refs. Verified each against current code before revising. (1) **F-1 not clean**: `Entity` dataclass at `service.py:67` and `entities` table schema are missing the `extractor_version` and `canonical_home` shared-header fields required by `engineering-v1-acceptance.md:45`; `project` field is the project identifier but not named `project_id` as spec writes (spec wording "fields equivalent to" allows the naming, but needs explicit doc note). V1-0 scope now includes adding both missing fields via additive `_apply_migrations` pattern. (2) **F-2 needed exact statuses, not guesses**: per-function audit gave ground truth — 9 of 20 v1-required queries done, 1 partial (Q-001 returns project-wide tree not subsystem-scoped expand=contains per `engineering-query-catalog.md:71`), 10 missing. V1-A scope shrank to Q-001 shape fix + Q-6 integration (most pillar queries already implemented); V1-C closes the 8 net-new queries + Q-020 to V1-D. (3) **F-5 misframed**: the generic `conflicts` + `conflict_members` schema is ALREADY spec-compliant at `database.py:190`; divergence is detector body at `conflicts.py:36` (per-type dispatch needs generalization) + route path (`/admin/conflicts/*` needs `/conflicts/*` alias). V1-F no longer includes a schema migration; detector generalization + route alignment only. Totals revised to 16.517.5 days, ~60 tests (down from 1217 / 65 because V1-A and V1-F scopes both shrank after audit). Three of the eight open questions resolved. Remaining open: F-7 graduation depth, mirror determinism, `project` naming, velocity calibration, minions-as-V2 naming. No code changes this session — plan + ledger only. Next: commit + push revised plan, then await Antoine+Codex joint sign-off before V1-0 starts.
- **2026-04-22 Claude (night)** Codex first-round review of the V1 Completion Plan summary came back with four findings. Three substantive, one workspace-sync: (1) "5070% built" too loose — replaced with per-criterion table, global framing withdrawn; (2) phase order backward — provenance-at-write (F-8) and conflict hooks (F-5 minimal) depend-upon by every later phase but were in V1-E; new V1-0 prerequisite phase inserted to establish write-time invariants, and V1-A shrunk to a minimum query slice (four pillars Q-001/Q-005/Q-006/Q-017 + Q-6 integration) rather than full catalog closure; (3) "parallel with Now list / disjoint surfaces" too strong — real collisions listed explicitly (V1-0 provenance + memory extractor write path, V1-E graduation + memory module, V1-F conflicts migration + memory promote); schedule shifted ~4 weeks, V1-0 cannot start during pipeline soak; (4) Codex's Playground workspace can't see the plan file or the `src/atocore/engineering/` code — added a Workspace note to the plan directing per-file audit at the Windows canonical dev tree (`C:\Users\antoi\ATOCore`) and noting the three visible file paths (`docs/plans/engineering-v1-completion-plan.md`, `docs/decisions/2026-04-22-gbrain-plan-rejection.md`, `DEV-LEDGER.md`). Revised plan estimate: 1217 days across 7 phases (up from 1114 / 6), ~65 tests added (up from ~50). V1-0 is a hard prerequisite; no later phase starts until it lands. Pending Antoine decision on workspace sync (commit+push vs paste-to-Codex) so Codex can do the file-level audit. No code changes this session.
- **2026-04-22 Claude (late eve)** After the rejection, read the four core V1 architecture docs end-to-end (`engineering-ontology-v1.md`, `engineering-query-catalog.md`, `memory-vs-entities.md`, `engineering-v1-acceptance.md`) plus the four supporting docs (`promotion-rules.md`, `conflict-model.md`, `human-mirror-rules.md`, `tool-handoff-boundaries.md`). Cross-referenced against current code in `src/atocore/engineering/`. **Key finding:** V1 is already 5070% built — entity types (16, superset of V1's 12), all 18 V1 relationship types, 4-state lifecycle, CRUD + supersede + invalidate + PATCH, queries module with most killer-correctness queries (orphan_requirements, risky_decisions, unsupported_claims, impact_analysis, evidence_chain), conflicts module scaffolded, mirror scaffolded, graduation endpoint scaffolded. Recent commits e147ab2/b94f9df/081c058/069d155/b1a3dd0 are all V1 entity-layer work. Drafted `docs/plans/engineering-v1-completion-plan.md` reframing the work as **V1 completion, not V1 start**. Six sequential phases V1-A through V1-F, estimated 1114 days, ~50 new tests (533 → ~580). Phases run in parallel with the Now list (pipeline soak + density + multi-model triage + p04-constraints) because surfaces are disjoint. Plan explicitly defers the minions/queue mechanic per acceptance-doc negative list. Pending Codex audit of the plan itself — especially the F-2 query gap list (Claude didn't read each query function end-to-end), F-5 conflicts schema divergence (per-type detectors vs spec's generic slot-keyed shape), and F-7 graduation depth. No code changes this session.
- **2026-04-22 Claude (eve)** gbrain review session. Antoine surfaced https://github.com/garrytan/gbrain for compare/contrast. Claude drafted a "Phase 8 Minions + typed edges" plan pairing a durable job queue with a 6-predicate edge upgrade over wikilinks. Codex reviewed and rejected as packaged: (1) sequencing leapfrogged the `master-plan-status.md` Now list (pipeline soak → 100+ memories → multi-model triage → p04-constraints fix); (2) 6 predicates vs V1's 17 across Structural/Intent/Validation/Provenance families — would have been schema debt on day one per `engineering-ontology-v1.md:112-137`; (3) "edges over wikilinks" bypassed the V1 canonical entity + promotion contract in `memory-vs-entities.md`. Claude verified each high finding against the cited files and concurred. The underlying mechanic (durable background jobs + typed relationship graph) is still a valid future direction, but its correct home is the Engineering V1 sprint under **Next** in `master-plan-status.md:179`, not a leapfrog phase. Decision record: `docs/decisions/2026-04-22-gbrain-plan-rejection.md`. No code changes this session. Next (pending Antoine ack): read the four V1 architecture docs end-to-end, then draft an Engineering V1 foundation plan that follows the existing contract, not a new external reference. Phase 8 (OpenClaw) name remains untouched — Claude's misuse of "Phase 8" in the rejected plan was a naming collision, not a renaming.
- **2026-04-22 Claude (pm)** Issue B (wiki redlinks) landed — last remaining P2 from Antoine's sprint plan. `_wikilink_transform(text, current_project)` in `src/atocore/engineering/wiki.py` replaces `[[Name]]` / `[[Name|Display]]` tokens (pre-markdown) with HTML anchors. Resolution order: same-project exact-name match → live `wikilink`; other-project match → live link with `(in project X)` scope indicator (`wikilink-cross`); no match → `redlink` pointing at `/wiki/new?name=<quoted>&project=<current>`. New route `GET /wiki/new` renders a pre-filled "create this entity" form that POSTs to `/v1/entities` via a minimal inline fetch() and redirects to the new entity's wiki page on success. Transform applied in `render_project` (over the mirror markdown) and `render_entity` (over the description body). CSS: dashed-underline accent for live wikilinks, red italic + dashed for redlinks. 12 new tests including the regression from the spec (entity A references `[[EntityB]]` → initial render has `class="redlink"`; after EntityB is created, re-render no longer has redlink and includes `/wiki/entities/{b.id}`). Tests 521 → 533. All 6 acceptance criteria from the sprint plan ("daily-usable") now green: retract/supersede, edit without cloning, cross-project has a home, visual evidence, wiki readable, AKC can capture reliably. - **2026-04-22 Claude (pm)** Issue B (wiki redlinks) landed — last remaining P2 from Antoine's sprint plan. `_wikilink_transform(text, current_project)` in `src/atocore/engineering/wiki.py` replaces `[[Name]]` / `[[Name|Display]]` tokens (pre-markdown) with HTML anchors. Resolution order: same-project exact-name match → live `wikilink`; other-project match → live link with `(in project X)` scope indicator (`wikilink-cross`); no match → `redlink` pointing at `/wiki/new?name=<quoted>&project=<current>`. New route `GET /wiki/new` renders a pre-filled "create this entity" form that POSTs to `/v1/entities` via a minimal inline fetch() and redirects to the new entity's wiki page on success. Transform applied in `render_project` (over the mirror markdown) and `render_entity` (over the description body). CSS: dashed-underline accent for live wikilinks, red italic + dashed for redlinks. 12 new tests including the regression from the spec (entity A references `[[EntityB]]` → initial render has `class="redlink"`; after EntityB is created, re-render no longer has redlink and includes `/wiki/entities/{b.id}`). Tests 521 → 533. All 6 acceptance criteria from the sprint plan ("daily-usable") now green: retract/supersede, edit without cloning, cross-project has a home, visual evidence, wiki readable, AKC can capture reliably.
- **2026-04-22 Claude** PATCH `/entities/{id}` + Issue D (/v1/engineering/* aliases) landed. New `update_entity()` in `src/atocore/engineering/service.py` supports partial updates to description (replace), properties (shallow merge — `null` value deletes a key), confidence (0..1, 400 on bounds violation), source_refs (append + dedup). Writes an `updated` audit row with full before/after snapshots. Forbidden via this path: entity_type / project / name / status — those require supersede+create or the dedicated status endpoints, by design. New route `PATCH /entities/{id}` aliased under `/v1`. Issue D: all 10 `/engineering/*` query paths (decisions, systems, components/{id}/requirements, changes, gaps + sub-paths, impact, evidence) added to the `/v1` allowlist. 12 new PATCH tests (merge, null-delete, confidence bounds, source_refs dedup, 404, audit row, v1 alias). Tests 509 → 521. Next: commit + deploy, then Issue B (wiki redlinks) as the last remaining P2 per Antoine's sprint order. - **2026-04-22 Claude** PATCH `/entities/{id}` + Issue D (/v1/engineering/* aliases) landed. New `update_entity()` in `src/atocore/engineering/service.py` supports partial updates to description (replace), properties (shallow merge — `null` value deletes a key), confidence (0..1, 400 on bounds violation), source_refs (append + dedup). Writes an `updated` audit row with full before/after snapshots. Forbidden via this path: entity_type / project / name / status — those require supersede+create or the dedicated status endpoints, by design. New route `PATCH /entities/{id}` aliased under `/v1`. Issue D: all 10 `/engineering/*` query paths (decisions, systems, components/{id}/requirements, changes, gaps + sub-paths, impact, evidence) added to the `/v1` allowlist. 12 new PATCH tests (merge, null-delete, confidence bounds, source_refs dedup, 404, audit row, v1 alias). Tests 509 → 521. Next: commit + deploy, then Issue B (wiki redlinks) as the last remaining P2 per Antoine's sprint order.

View File

@@ -159,6 +159,17 @@ Every major object should support fields equivalent to:
- `created_at` - `created_at`
- `updated_at` - `updated_at`
- `notes` (optional) - `notes` (optional)
- `extractor_version` (V1-0)
- `canonical_home` (V1-0)
**Naming note (V1-0, 2026-04-22).** The AtoCore `entities` table and
`Entity` dataclass name the project-identifier field `project`, not
`project_id`. This doc's "fields equivalent to" wording allows that
naming flexibility — the `project` field on entity rows IS the
`project_id` per spec. No storage rename is planned; downstream readers
should treat `entity.project` as the project identifier. This was
resolved in Codex's third-round audit of the V1 Completion Plan (see
`docs/plans/engineering-v1-completion-plan.md`).
## Suggested Status Lifecycle ## Suggested Status Lifecycle

View File

@@ -1,6 +1,31 @@
# AtoCore — Current State (2026-04-19) # AtoCore — Current State (2026-04-22)
Live deploy: `877b97e` · Dalidou health: ok · Harness: 17/18. Live deploy: `2712c5d` · Dalidou health: ok · Harness: 17/18 · Tests: 547 passing.
## V1-0 landed 2026-04-22
Engineering V1 completion track has started. **V1-0 write-time invariants**
merged and deployed: F-1 shared-header fields (`extractor_version`,
`canonical_home`, `hand_authored`) added to `entities`, F-8 provenance
enforcement at both `create_entity` and `promote_entity`, F-5 synchronous
conflict-detection hook on every active-entity write path (create, promote,
supersede) with Q-3 fail-open. Prod backfill ran cleanly — 31 legacy
active/superseded entities flagged `hand_authored=1`, follow-up dry-run
returned 0 remaining rows. Test count 533 → 547 (+14).
R14 (P2, non-blocking): `POST /entities/{id}/promote` route fix translates
the new `ValueError` into 400. Branch `claude/r14-promote-400` pending
Codex review + squash-merge.
**Next in the V1 track:** V1-A (minimal query slice + Q-6 killer-correctness
integration). Gated on pipeline soak (~2026-04-26) + 100+ active memory
density target. See `docs/plans/engineering-v1-completion-plan.md` for
the full 7-phase roadmap and `docs/plans/v1-resume-state.md` for the
"you are here" map.
---
## Snapshot from previous update (2026-04-19)
## The numbers ## The numbers

View File

@@ -0,0 +1,113 @@
# Decision record: gbrain-inspired "Phase 8 Minions + typed edges" plan rejected
**Date:** 2026-04-22
**Author of plan:** Claude
**Reviewer:** Codex
**Ratified by:** Antoine
**Status:** Rejected as packaged. Underlying mechanic (durable background jobs + typed relationships) deferred to its correct home.
## Context
Antoine surfaced https://github.com/garrytan/gbrain and asked for a compare/contrast and a
plan to improve AtoCore. Claude proposed a "Phase 8" plan pairing:
1. A Minion-style durable job queue replacing the nightly cron pipeline
2. A typed-edge upgrade over existing wikilinks, with a six-predicate set
(`mentions`, `decided_by`, `supersedes`, `evidences`, `part_of`, `blocks`)
Codex reviewed and rejected the plan as packaged. This record captures what went wrong,
what was right, and where the ideas should actually land.
## What Codex flagged (verified against repo)
### High — wrong sequencing
`docs/master-plan-status.md` defines the **Now** list:
1. Observe the enhanced pipeline for a week
2. Knowledge density — batch-extract over all 234 interactions, target 100+ memories
3. Multi-model triage (Phase 11 entry)
4. Fix p04-constraints harness failure
Engineering V1 appears under **Next** (line 179) as
"Engineering V1 implementation sprint — once knowledge density is sufficient and the
pipeline feels boring and dependable."
Claude's plan jumped over all four **Now** items. That was the primary sequencing error.
### High — wrong predicate set
`docs/architecture/engineering-ontology-v1.md` already defines a 17-predicate V1
ontology across four families:
- **Structural:** `CONTAINS`, `PART_OF`, `INTERFACES_WITH`
- **Intent / logic:** `SATISFIES`, `CONSTRAINED_BY`, `BASED_ON_ASSUMPTION`,
`AFFECTED_BY_DECISION`, `SUPERSEDES`
- **Validation:** `ANALYZED_BY`, `VALIDATED_BY`, `SUPPORTS`, `CONFLICTS_WITH`,
`DEPENDS_ON`
- **Artifact / provenance:** `DESCRIBED_BY`, `UPDATED_BY_SESSION`, `EVIDENCED_BY`,
`SUMMARIZED_IN`
Claude's six-predicate set was a gbrain-shaped subset that could not express the V1
example statements at lines 141147 of that doc. Shipping it first would have been
schema debt on day one.
### High — wrong canonical boundary
`docs/architecture/memory-vs-entities.md` and
`docs/architecture/engineering-v1-acceptance.md` establish that V1 is **typed
entities plus typed relationships**, with one canonical home per concept, a shared
candidate-review / promotion flow, provenance, conflict handling, and mirror
generation. Claude's "typed edges on top of wikilinks" framing bypassed the canonical
entity contract — it would have produced labelled links over notes without the
promotion / canonicalization machinery V1 actually requires.
### Medium — overstated problem
Claude described the nightly pipeline as a "monolithic bash script" that needed to be
replaced. The actual runtime is API-driven (`src/atocore/api/routes.py:516`,
`src/atocore/interactions/service.py:55`), SQLite is already in WAL with a busy
timeout (`src/atocore/models/database.py:151`), and the reflection loop is explicit
capture / reinforce / extract. The queue argument overstated the current shape.
## What was right
- gbrain is genuine validation of the general pattern: **durable background jobs +
typed relationship graph compound value**. The gbrain v0.12.0 graph release and
Minions benchmark (both 2026-04-18) are evidence, not just inspiration.
- Async-ification of extraction with retries, per-job visibility, and SLOs remains a
real future win for AtoCore — but **additively, behind flags, after V1**, not as a
replacement for the current explicit endpoints.
## What we will do instead
1. **Keep to the `master-plan-status.md` Now list.** No leapfrog. Observe the
pipeline (including the confidence-decay Step F4 first real run), land knowledge
density via full-backlog batch extract, progress multi-model triage, fix
p04-constraints.
2. **When Engineering V1 is ready to start** (criterion: pipeline feels boring and
dependable, knowledge density ≥ 100 active memories), write a V1 foundation plan
that follows `engineering-ontology-v1.md`, `engineering-query-catalog.md`,
`memory-vs-entities.md`, and `engineering-v1-acceptance.md` — entities +
relationships + memory-to-entity bridge + mirror / query surfaces, in that order.
3. **Async workerization is optional and later.** Only after V1 is working, and only
if observed contention or latency warrants it. Jobs stay in the primary SQLite
(WAL already in place). No separate DB unless contention is measured.
## Lesson for future plans
A plan built from a **new external reference** (gbrain) without reading the
repository's own architecture docs will mis-specify predicates, boundaries, and
sequencing — even when the underlying mechanic is valid. Read the four V1
architecture docs end-to-end before proposing schema work.
## References
- https://github.com/garrytan/gbrain
- `docs/master-plan-status.md` (Now / Next / Later)
- `docs/architecture/engineering-ontology-v1.md`
- `docs/architecture/engineering-query-catalog.md`
- `docs/architecture/memory-vs-entities.md`
- `docs/architecture/engineering-v1-acceptance.md`
- `docs/architecture/llm-client-integration.md`
- `docs/architecture/human-mirror-rules.md`

View File

@@ -168,16 +168,40 @@ These are the current practical priorities.
"Zerodur" for p04 constraint queries. Investigate if it's a missing "Zerodur" for p04 constraint queries. Investigate if it's a missing
memory or retrieval ranking issue. memory or retrieval ranking issue.
## Active — Engineering V1 Completion Track (started 2026-04-22)
The Engineering V1 sprint moved from **Next** to **Active** on 2026-04-22.
The discovery from the gbrain review was that V1 entity infrastructure
had been built incrementally already; the sprint is a **completion** plan
against `engineering-v1-acceptance.md`, not a greenfield build. Full plan:
`docs/plans/engineering-v1-completion-plan.md`. "You are here" single-page
map: `docs/plans/v1-resume-state.md`.
Seven phases, ~17.519.5 focused days, runs in parallel with the Now list
where surfaces are disjoint, pauses when they collide.
| Phase | Scope | Status |
|---|---|---|
| V1-0 | Write-time invariants: F-1 header fields + F-8 provenance enforcement + F-5 hook on every active-entity write + Q-3 flag-never-block | ✅ done 2026-04-22 (`2712c5d`) |
| V1-A | Minimum query slice: Q-001 subsystem-scoped variant + Q-6 killer-correctness integration test on p05-interferometer | 🟡 gated — starts when soak (~2026-04-26) + density (100+ active memories) gates clear |
| V1-B | KB-CAD + KB-FEM ingest (`POST /ingest/kb-cad/export`, `POST /ingest/kb-fem/export`) + D-2 schema docs | pending V1-A |
| V1-C | Close the remaining 8 queries (Q-002/003/007/010/012/014/018/019; Q-020 to V1-D) | pending V1-B |
| V1-D | Full mirror surface (3 spec routes + regenerate + determinism + disputed + curated markers) + Q-5 golden file | pending V1-C |
| V1-E | Memory→entity graduation end-to-end + remaining Q-4 trust tests | pending V1-D (note: collides with memory extractor; pauses for multi-model triage work) |
| V1-F | F-5 detector generalization + route alias + O-1/O-2/O-3 operational + D-1/D-3/D-4 docs | finish line |
R14 (P2, non-blocking): `POST /entities/{id}/promote` route returns 500
on the new V1-0 `ValueError` instead of 400. Fix on branch
`claude/r14-promote-400`, pending Codex review.
## Next ## Next
These are the next major layers after the current stabilization pass. These are the next major layers after V1 and the current stabilization pass.
1. Phase 6 AtoDrive — clarify Google Drive as a trusted operational 1. Phase 6 AtoDrive — clarify Google Drive as a trusted operational
source and ingest from it source and ingest from it
2. Phase 13 Hardening — Chroma backup policy, monitoring, alerting, 2. Phase 13 Hardening — Chroma backup policy, monitoring, alerting,
failure visibility beyond log files failure visibility beyond log files
3. Engineering V1 implementation sprint — once knowledge density is
sufficient and the pipeline feels boring and dependable
## Later ## Later

View File

@@ -0,0 +1,617 @@
# Engineering V1 Completion Plan
**Date:** 2026-04-22
**Author:** Claude (after reading the four V1 architecture docs + promotion-rules,
conflict-model, human-mirror-rules, tool-handoff-boundaries end-to-end)
**Status:** Draft, pending Codex review
**Replaces:** the rejected "Phase 8 Minions + typed edges" plan (see
`docs/decisions/2026-04-22-gbrain-plan-rejection.md`)
## Position
This is **not** a plan to start Engineering V1. It is a plan to **finish** V1.
**Against what criterion?** Each F/Q/O/D item in `engineering-v1-acceptance.md`
gets scored individually in the Gap audit table below with exact code/test/doc
references. No global percentage. The headline framing from the first draft
("5070% built") is withdrawn — it's either done per-criterion or it isn't.
The relevant observation is narrower: the entity schema, the full
relationship type set, the 4-state lifecycle, basic CRUD and most of the
killer-correctness query functions are already implemented in
`src/atocore/engineering/*.py` in the Windows working tree at
`C:\Users\antoi\ATOCore` (the canonical dev workspace, per
`CLAUDE.md`). The recent commits e147ab2, b94f9df, 081c058, 069d155, b1a3dd0
are V1 entity-layer work. **Codex auditors working in a different
workspace / branch should sync from the canonical dev tree before
per-file review** — see the "Workspace note" at the end of this doc.
The question this plan answers: given the current code state, in what
order should the remaining V1 acceptance criteria be closed so that
every phase builds on invariants the earlier phases already enforced?
## Corrected sequencing principle (post-Codex review 2026-04-22)
The first draft ordered phases F-1 → F-2 → F-3 → F-4 → F-5 → F-6 → F-7 → F-8
following the acceptance doc's suggested reading order. Codex rejected
that ordering. The correct dependency order, which this revision adopts, is:
1. **Write-time invariants come first.** Every later phase creates active
entities. Provenance-at-write (F-8) and synchronous conflict-detection
hooks (F-5 minimal) must be enforced **before** any phase that writes
entities at scale (ingest, graduation, or broad query coverage that
depends on the model being populated).
2. **Query closure sits on top of the schema + invariants**, not ahead of
them. A minimum query slice that proves the model is fine early. The
full catalog closure waits until after the write paths are invariant-safe.
3. **Mirror is a derived consumer** of the entity layer, not a midstream
milestone. It comes after the entity layer produces enforced, correct data.
4. **Graduation and full conflict-spec compliance** are finishing work that
depend on everything above being stable.
The acceptance criteria are unchanged. Only the order of closing them changes.
## How this plan respects the rejected-plan lessons
- **No new predicates.** The V1 ontology in `engineering-ontology-v1.md:112-137`
already defines 18 relationship types; `service.py:38-62` already implements
them. Nothing added, nothing reshaped.
- **No new canonical boundary.** Typed entities + typed relationships with
promotion-based candidate flow per `memory-vs-entities.md`. Not
edges-over-wikilinks.
- **No leapfrog of `master-plan-status.md` Now list.** This plan is **in
parallel** with (not ahead of) the Now items because V1 entity work is
already happening alongside them. The sequencing section below is explicit.
- **Queue/worker infrastructure is explicitly out of scope.** The "flag it for
later" note at the end of this doc is the only mention, per
`engineering-v1-acceptance.md:378` negative list.
---
## Gap audit against `engineering-v1-acceptance.md`
Each criterion marked: ✅ done / 🟡 partial / ❌ missing. "Partial" means the
capability exists but does not yet match spec shape or coverage.
### Functional (F-1 through F-8)
| ID | Criterion | Status | Evidence |
|----|-----------|--------|----------|
| F-1 | 12 V1 entity types, 4 relationship families, shared header fields, 4-state lifecycle | 🟡 partial (per Codex 2026-04-22 audit) | `service.py:16-36` has 16 types (superset of V1's 12), `service.py:38-62` has 18 relationship types, `service.py:64` statuses, `Entity` dataclass at line 67. **Gaps vs `engineering-v1-acceptance.md:45`**: `extractor_version` missing from dataclass and `entities` table; `canonical_home` missing from dataclass and table; `project` field is the project identifier but not named `project_id` as spec uses — spec says "fields equivalent to" so naming flexibility is allowed but needs an explicit doc note. Remediation lands in V1-0 |
| F-2 | All v1-required Q-001 through Q-020 implemented, with provenance where required | 🟡 partial (per Codex 2026-04-22 per-function audit) | **Ground truth from per-function read of `queries.py` + `routes.py:2092+`:** Q-001 partial (`system_map()` returns project-wide tree, not the catalog's subsystem-scoped `GET /entities/Subsystem/<id>?expand=contains` shape per `engineering-query-catalog.md:71`); Q-002 missing; Q-003 missing; Q-004 done (covered by `system_map()`); Q-005 done (`requirements_for()`); Q-006 done (`orphan_requirements()`); Q-007 missing; Q-008 done (`decisions_affecting()`); Q-009 done (`risky_decisions()`); Q-010 missing; Q-011 done (`unsupported_claims()`); Q-012 missing; Q-013 done (`recent_changes()`); Q-014 missing; Q-016 done (`impact_analysis()`); Q-017 done (`evidence_chain()`); Q-018 missing; Q-019 missing; Q-020 missing (mirror route in spec shape). **Net: 9 of 20 v1-required queries done, 1 partial (Q-001), 10 missing.** Q-015 is v1-stretch, out of scope |
| F-3 | `POST /ingest/kb-cad/export` and `POST /ingest/kb-fem/export` | ❌ missing | No `/ingest/kb-cad` or `/ingest/kb-fem` route in `api/routes.py`. No schema doc under `docs/architecture/` |
| F-4 | Candidate review queue end-to-end (list/promote/reject/edit) | 🟡 partial for entities | Memory side shipped in Phase 9 Commit C. Entity side has `promote_entity`, `supersede_entity`, `invalidate_active_entity` but reject path and editable-before-promote may not match spec shape. Need to verify `GET /entities?status=candidate` returns spec shape |
| F-5 | Conflict detector fires synchronously; `POST /conflicts/{id}/resolve` + dismiss | 🟡 partial (per Codex 2026-04-22 audit — schema present, detector+routes divergent) | **Schema is already spec-shaped**: `database.py:190` defines the generic `conflicts` + `conflict_members` tables per `conflict-model.md`; `conflicts.py:154` persists through them. **Divergences are in detection and API, not schema**: (1) `conflicts.py:36` dispatches per-type detectors only (`_check_component_conflicts`, `_check_requirement_conflicts`) — needs generalization to slot-key-driven detection; (2) routes live at `/admin/conflicts/*`, spec says `/conflicts/*` — needs alias + deprecation. **No schema migration needed** |
| F-6 | Mirror: `/mirror/{project}/overview`, `/decisions`, `/subsystems/{id}`, `/regenerate`; files under `/srv/storage/atocore/data/mirror/`; disputed + curated markers; deterministic output | 🟡 partial | `mirror.py` has `generate_project_overview` with header/state/system/decisions/requirements/materials/vendors/memories/footer sections. API at `/projects/{project_name}/mirror` and `.html`. **Gaps**: no separate `/mirror/{project}/decisions` or `/mirror/{project}/subsystems/{id}` routes, no `POST /regenerate` endpoint, no debounced-async-on-write, no daily refresh, no `⚠ disputed` markers wired to conflicts, no `(curated)` override annotations verified, no golden-file test for determinism |
| F-7 | Memory→entity graduation: `POST /memory/{id}/graduate` + `graduated` status + forward pointer + original preserved | 🟡 partial (per Codex 2026-04-22 third-round audit) | `_graduation_prompt.py` exists; `scripts/graduate_memories.py` creates entity candidates from active memories; `database.py:143-146` adds `graduated_to_entity_id`; `memory.service` already has a `graduated` status; `service.py:354-356,389-451` preserves the original memory and marks it `graduated` with a forward pointer on entity promote; `tests/test_engineering_v1_phase5.py:67-90` covers that flow. **Gaps vs spec**: no direct `POST /memory/{id}/graduate` route yet (current surface is batch/admin-driven via `/admin/graduation/request`); no explicit acceptance tests yet for `adaptation→decision` and `project→requirement`; spec wording `knowledge→Fact` does not match the current ontology (there is no `fact` entity type in `service.py` / `_graduation_prompt.py`) and should be reconciled to an actual V1 type such as `parameter` or another ontology-defined entity. |
| F-8 | Every active entity has `source_refs`; Q-017 returns ≥1 row for every active entity | 🟡 partial | `Entity.source_refs` field exists; Q-017 (`evidence_chain`) exists. **Gap**: is provenance enforced at write time (not NULL), or just encouraged? Per spec it must be mandatory |
### Quality (Q-1 through Q-6)
| ID | Criterion | Status | Evidence |
|----|-----------|--------|----------|
| Q-1 | All pre-V1 tests still pass | ✅ presumed | 533 tests passing per DEV-LEDGER line 12 |
| Q-2 | Each F criterion has happy-path + error-path test, <10s each, <30s total | 🟡 partial | 16 + 15 + 15 + 12 = 58 tests in engineering/queries/v1-phase5/patch files. Need to verify coverage of each F criterion one-for-one |
| Q-3 | Conflict invariants enforced by tests (contradictory imports produce conflict, can't promote both, flag-never-block) | 🟡 partial | Tests likely exist in `test_engineering_v1_phase5.py` — verify explicit coverage of the three invariants |
| Q-4 | Trust hierarchy enforced by tests (candidates never in context, active-only reinforcement, no auto-project-state writes) | 🟡 partial | Phase 9 Commit B covered the memory side; verify entity side has equivalent tests |
| Q-5 | Mirror has golden-file test, deterministic output | ❌ missing | No golden file seen; mirror output reads wall-clock time inside `_footer()` (`mirror.py:320-327`). Determinism should come from injecting the regenerated timestamp/checksum as inputs to the renderer and pinning them in the golden-file test, not from calling `datetime.now()` inside render code |
| Q-6 | Killer correctness queries pass against seeded real-ish data (5 seed cases per Q-006/Q-009/Q-011) | ❌ likely missing | No fixture file named for this seen. The three queries exist but there's no evidence of the single integration test described in Q-6 |
### Operational (O-1 through O-5)
| ID | Criterion | Status | Evidence |
|----|-----------|--------|----------|
| O-1 | Schema migration additive, idempotent, tested against fresh + prod-copy DB | 🟡 presumed | `_apply_migrations` pattern is in use per CLAUDE.md sessions; tables exist. Need one confirmation run against a Dalidou backup copy |
| O-2 | Backup includes new tables; full restore drill passes; post-restore Q-001 works | ❌ not done | No evidence a restore drill has been run on V1 entity state. `docs/backup-restore-procedure.md` exists but drill is an explicit V1 prerequisite |
| O-3 | Performance bounds: write <100ms p99, query <500ms p99 at 1000 entities, mirror <5s per project | 🟡 unmeasured | 35 entities in system — bounds unmeasured at scale. Spec says "sanity-checked, not benchmarked", so this is a one-off manual check |
| O-4 | No new manual ops burden | 🟡 | Mirror regen auto-triggers not wired yet (see F-6 gap) — they must be wired for O-4 to pass |
| O-5 | Phase 9 reflection loop unchanged for identity/preference/episodic | ✅ presumed | `memory-vs-entities.md` says these three types don't interact with engineering layer. No recent change to memory extractor for these types |
### Documentation (D-1 through D-4)
| ID | Criterion | Status | Evidence |
|----|-----------|--------|----------|
| D-1 | 12 per-entity-type spec docs under `docs/architecture/entities/` | ❌ missing | No `docs/architecture/entities/` folder |
| D-2 | `kb-cad-export-schema.md` + `kb-fem-export-schema.md` | ❌ missing | No such files in `docs/architecture/` |
| D-3 | `docs/v1-release-notes.md` | ❌ missing | Not written yet (appropriately — it's written when V1 is done) |
| D-4 | `master-plan-status.md` + `current-state.md` updated with V1 completion | ❌ not yet | `master-plan-status.md:179` still has V1 under **Next** |
### Summary (revised per Codex 2026-04-22 per-file audit)
- **Functional:** 0/8 ✅, 7/8 🟡 partial (F-1 downgraded from ✅ — two header fields missing; F-2 through F-7 partial), 1/8 ❌ missing (F-3 ingest endpoints) → the entity layer shape is real but not yet spec-clean; write-time invariants come first, then everything builds on stable invariants
- **F-2 detail:** 9 of 20 v1-required queries done, 1 partial (Q-001 needs subsystem-scoped variant), 10 missing
- **F-5 detail:** generic `conflicts` + `conflict_members` schema already present (no migration needed); detector body + routes diverge from spec
- **Quality:** 1/6 ✅, 3/6 🟡 partial, 2/6 ❌ missing → golden file + killer-correctness integration test are the two clear gaps
- **Operational:** 0/5 ✅ (none fully verified), 3/5 🟡, 1/5 ❌ → backup drill is the one hard blocker here
- **Documentation:** 0/4 ✅, 4/4 ❌ → all 4 docs need writing
---
## Proposed completion order (revised post-Codex review)
Seven phases instead of six. The new V1-0 establishes the write-time
invariants (provenance enforcement F-8 + synchronous conflict hooks F-5
minimal) that every later phase depends on. V1-A becomes a **minimal query
slice** that proves the model on one project, not a full catalog closure.
Full query catalog closure moves to V1-C. Full F-5 spec compliance (the
generic `conflicts`/`conflict_members` slot-key schema) stays in V1-F
because that's the final shape, but the *minimal hooks* that fire
synchronously on writes land in V1-0.
Skipped by construction: F-1 core schema (already implemented) and O-5
(identity/preference/episodic don't touch the engineering layer).
### Phase V1-0: Write-time invariants (F-8 + F-5 minimal + F-1 audit)
**Scope:**
- **F-1 remediation (Codex audit 2026-04-22 already completed).** Add
the two missing shared-header fields to the `Entity` dataclass
(`service.py:67`) and the `entities` table schema:
- `extractor_version TEXT` — semver-ish string carrying the extractor
module version per `promotion-rules.md:268`. Backfill existing rows
with `"0.0.0"` or `NULL` flagged as unknown. Every future
write carries the current `EXTRACTOR_VERSION` constant.
- `canonical_home TEXT` — which layer is canonical for this concept.
For entities, value is always `"entity"`. For future graduation
records it may be `"memory"` (frozen pointer). Backfill active
rows with `"entity"`.
- Additive migration via the existing `_apply_migrations` pattern,
idempotent, safe on replay.
- Add doc note in `engineering-ontology-v1.md` clarifying that the
`project` field IS the `project_id` per spec — "fields equivalent
to" wording in the spec allows this, but make it explicit so
future readers don't trip on the naming.
- **F-8 provenance enforcement.** Add a NOT-NULL invariant at
`create_entity` and `promote_entity` that `source_refs` is non-empty
OR an explicit `hand_authored=True` flag is set (per
`promotion-rules.md:253`). Backfill any existing active entities that
fail the invariant — either attach provenance, flag as hand-authored,
or invalidate. Every future active entity has provenance by schema,
not by discipline.
- **F-5 minimal hooks.** Wire synchronous conflict detection into every
active-entity write path (`create_entity` with status=active,
`promote_entity`, `supersede_entity`). The *detector* can stay in its
current per-type form (`_check_component_conflicts`,
`_check_requirement_conflicts`); the *hook* must fire on every write.
Full generic slot-keyed schema lands in V1-F; the hook shape must be
generic enough that V1-F is a detector-body swap, not an API refactor.
- **Q-3 "flag never block" test.** The hook must return conflict-id in
the response body but never 4xx-block the write. One test per write
path demonstrating this.
- **Q-4 trust-hierarchy test for candidates.** One test: entity
candidates never appear in `/context/build` output. (Full trust tests
land in V1-E; this is the one that V1-0 can cover without graduation
being ready.)
**Acceptance:** F-1 ✅ (after `extractor_version` + `canonical_home`
land + doc note on `project` naming), F-8 ✅, F-5 hooks ✅, Q-3 ✅,
partial Q-4 ✅.
**Estimated size:** 3 days (two small schema additions + invariant
patches + hook wiring + tests; no audit overhead — Codex already did
that part).
**Tests added:** ~10.
**Why first:** every later phase writes entities. Without F-8 + F-5
hooks, V1-A through V1-F can leak invalid state into the store that
must then be cleaned up.
### Phase V1-A: Minimal query slice that proves the model (partial F-2 + Q-6)
**Scope:**
- Pick the **four pillar queries**: Q-001 (subsystem contents),
Q-005 (component satisfies requirements), Q-006 (orphan requirements —
killer correctness), Q-017 (evidence chain). These exercise structural +
intent + killer-correctness + provenance.
- **Q-001 needs a shape fix**: Codex's audit confirms the existing
`system_map()` returns a project-wide tree, not the spec's
subsystem-scoped `GET /entities/Subsystem/<id>?expand=contains`.
Add a subsystem-scoped variant (the existing project-wide route stays
for Q-004). This is the only shape fix in V1-A; larger query additions
move to V1-C.
- Q-005, Q-006, Q-017 are already implemented per Codex audit. V1-A
verifies them against seeded data; no code changes expected.
- Seed p05-interferometer with Q-6 integration data (one satisfying
Component + one orphan Requirement + one Decision on flagged
Assumption + one supported ValidationClaim + one unsupported
ValidationClaim).
- All three killer-correctness queries (Q-006, Q-009, Q-011) are
**already implemented** per Codex audit. V1-A runs them as a single
integration test against the seed data.
**Acceptance:** Q-001 subsystem-scoped variant + Q-6 integration test.
Partial F-2 (remaining 10 missing + 1 partial queries land in V1-C).
**Estimated size:** 1.5 days (scope shrunk — most pillar queries already
work per Codex audit; only Q-001 shape fix + seed data + integration
test required).
**Tests added:** ~4.
**Why second:** proves the entity layer shape works end-to-end on real
data before we start bolting ingest, graduation, or mirror onto it. If
the four pillar queries don't work, stopping here is cheap.
### Phase V1-B: KB-CAD / KB-FEM ingest (F-3) + D-2 schema docs
**Scope:**
- Write `docs/architecture/kb-cad-export-schema.md` and
`kb-fem-export-schema.md` (matches D-2).
- Implement `POST /ingest/kb-cad/export` and `POST /ingest/kb-fem/export`
per `tool-handoff-boundaries.md` sketches. Validator + entity-candidate
producer + provenance population **using the F-8 invariant from V1-0**.
- Hand-craft one real KB-CAD export for p05-interferometer and
round-trip it: export → candidate queue → reviewer promotes → queryable
via V1-A's four pillar queries.
- Tests: valid export → candidates created; invalid export → 400;
duplicate re-export → no duplicate candidates; re-export with changed
value → new candidate + conflict row (exercises V1-0's F-5 hook on a
real workload).
**Acceptance:** F-3 ✅, D-2 ✅.
**Estimated size:** 2 days.
**Tests added:** ~8.
**Why third:** ingest is the first real stress test of the V1-0
invariants. A re-import that creates a conflict must trigger the V1-0
hook; if it doesn't, V1-0 is incomplete and we catch it before going
further.
### Phase V1-C: Close the rest of the query catalog (remaining F-2)
**Scope:** close the 10 missing queries per Codex's audit. Already-done
queries (Q-004/Q-005/Q-006/Q-008/Q-009/Q-011/Q-013/Q-016/Q-017) are
verified but not rewritten.
- Q-002 (component → parents, inverse of CONTAINS)
- Q-003 (subsystem interfaces, Interface as simple string label)
- Q-007 (component → constraints via CONSTRAINED_BY)
- Q-010 (ValidationClaim → supporting results + AnalysisModel trace)
- Q-012 (conflicting results on same claim — exercises V1-0's F-5 hook)
- Q-014 (decision log ordered + superseded chain)
- Q-018 (`include=superseded` for supersession chains)
- Q-019 (Material → components, derived from Component.material field
per `engineering-query-catalog.md:266`, no edge needed)
- Q-020 (project overview mirror route) — deferred to V1-D where the
mirror lands in full.
**Acceptance:** F-2 ✅ (all 19 of 20 v1-required queries; Q-020 in V1-D).
**Estimated size:** 2 days (eight new query functions + routes +
per-query happy-path tests).
**Tests added:** ~12.
**Why fourth:** with the model proven (V1-A) and ingest exercising the
write invariants (V1-B), filling in the remaining queries is mechanical.
They all sit on top of the same entity store and V1-0 invariants.
### Phase V1-D: Full Mirror surface (F-6) + determinism golden file (Q-5) + Q-020
**Scope:**
- Split the single `/projects/{project_name}/mirror` route into the three
spec routes: `/mirror/{project}/overview` (= Q-020),
`/mirror/{project}/decisions`, `/mirror/{project}/subsystems/{subsystem}`.
- Add `POST /mirror/{project}/regenerate`.
- Move generated files to `/srv/storage/atocore/data/mirror/{project}/`.
- **Deterministic output:** inject regenerated timestamp + checksum as
renderer inputs (pinned by golden tests), sort every iteration, and
remove `dict` / database ordering dependencies. The renderer should
not call wall-clock time directly.
- `⚠ disputed` markers inline wherever an open conflict touches a
rendered field (uses V1-0's F-5 hook output).
- `(curated)` annotations where project_state overrides entity state.
- Regeneration triggers: synchronous on regenerate, debounced async on
entity write (30s window), daily scheduled refresh via existing
nightly cron (one new cron line, not a new queue).
- `mirror_regeneration_failures` table.
- Golden-file test: fixture project state → render → bytes equal.
**Acceptance:** F-6 ✅, Q-5 ✅, Q-020 ✅, O-4 moves toward ✅.
**Estimated size:** 34 days.
**Tests added:** ~15.
**Why fifth:** mirror is a derived consumer. It cannot be correct
before the entity store + queries + conflict hooks are correct. It
lands after everything it depends on is stable.
### Phase V1-E: Memory→entity graduation end-to-end (F-7) + remaining Q-4
**Scope:**
- Verify and close F-7 spec gaps:
- Add the missing direct `POST /memory/{id}/graduate` route, reusing the
same prompt/parser as the batch graduation path.
- Keep `/admin/graduation/request` as the bulk lane; direct route is the
per-memory acceptance surface.
- Preserve current behavior where promote marks source memories
`status="graduated"` and sets `graduated_to_entity_id`.
- Flow tested for `adaptation` → Decision and `project` → Requirement.
- Reconcile the spec's `knowledge` → Fact wording with the actual V1
ontology (no `fact` entity type exists today). Prefer doc alignment to
an existing typed entity such as `parameter`, rather than adding a vague
catch-all `Fact` type late in V1.
- Schema is mostly already in place: `graduated` status exists in memory
service, `graduated_to_entity_id` column + index exist, and promote
preserves the original memory. Remaining work is route surface,
ontology/spec reconciliation, and targeted end-to-end tests.
- **Q-4 full trust-hierarchy tests**: no auto-write to project_state
from any promote path; active-only reinforcement for entities; etc.
(The entity-candidates-excluded-from-context test shipped in V1-0.)
**Acceptance:** F-7 ✅, Q-4 ✅.
**Estimated size:** 34 days.
**Tests added:** ~8.
**Why sixth:** graduation touches memory-layer semantics (adds a
`graduated` status, flows memory→entity, requires memory-module changes).
Doing it after the entity layer is fully invariant-safe + query-complete
+ mirror-derived means the memory side only has to deal with one shape:
a stable, tested entity layer.
### Phase V1-F: Full F-5 spec compliance + O-1/O-2/O-3 + D-1/D-3/D-4
**Scope:**
- **F-5 full spec compliance** (Codex 2026-04-22 audit already confirmed
the gap shape — schema is spec-compliant, divergence is in detector +
routes only).
- **Detector generalization.** Replace the per-type dispatch at
`conflicts.py:36` (`_check_component_conflicts`,
`_check_requirement_conflicts`) with a slot-key-driven generic
detector that reads the per-entity-type conflict slot from a
registry and queries the already-generic `conflicts` +
`conflict_members` tables. The V1-0 hook shape was chosen to make
this a detector-body swap, not an API change.
- **Route alignment.** Add `/conflicts/*` routes as the canonical
surface per `conflict-model.md:187`. Keep `/admin/conflicts/*` as
aliases for one release, deprecate in D-3 release notes, remove
in V1.1.
- **No schema migration needed** (the tables at `database.py:190`
already match the spec).
- **O-1:** Run the full migration against a Dalidou backup copy.
Confirm additive, idempotent, safe to run twice.
- **O-2:** Run a full restore drill on the test project per
`docs/backup-restore-procedure.md`. Post-restore, Q-001 returns
correct shape. `POST /admin/backup` snapshot includes the new tables.
- **O-3:** Manual sanity-check of the three performance bounds.
- **D-1:** Write 12 short spec docs under `docs/architecture/entities/`
(one per V1 entity type).
- **D-3:** Write `docs/v1-release-notes.md`.
- **D-4:** Update `master-plan-status.md` and `current-state.md`
move engineering V1 from **Next** to **What Is Real Today**.
**Acceptance:** F-5 ✅, O-1 ✅, O-2 ✅, O-3 ✅, D-1 ✅, D-3 ✅, D-4 ✅ →
**V1 is done.**
**Estimated size:** 3 days (F-5 migration if needed is the main unknown;
D-1 entity docs at ~30 min each ≈ 6 hours; verification is fast).
**Tests added:** ~6 (F-5 spec-shape tests; verification adds no automated
tests).
### Total (revised after Codex 2026-04-22 audit)
- Phase budgets: V1-0 (3) + V1-A (1.5) + V1-B (2) + V1-C (2) + V1-D (3-4)
+ V1-E (3-4) + V1-F (3) ≈ **17.519.5 days of focused work**. This is a
realistic engineering-effort estimate, but a single-operator calendar
plan should still carry context-switch / soak / review buffer on top.
- Adds roughly **60 tests** (533 → ~593).
- Branch strategy: one branch per phase (V1-0 → V1-F), each squash-merged
to main after Codex review. Phases sequential because each builds on
the previous. **V1-0 is a hard prerequisite for all later phases**
nothing starts until V1-0 lands.
---
## Sequencing with the `master-plan-status.md` Now list
The **Now** list from master-plan-status.md:159-169 is:
1. Observe the enhanced pipeline (1 week soak — first F4 confidence decay
run was 2026-04-19 per Trusted State, so soak window ends ~2026-04-26)
2. Knowledge density — batch extract over 234 interactions, target 100+
active memories (currently 84)
3. Multi-model triage (Phase 11 entry)
4. Fix p04-constraints harness failure
**Principle (revised per Codex review):** V1 work and the Now list are
**less disjoint than the first draft claimed**. Real collision points:
| V1 phase | Collides with Now list at |
|---|---|
| V1-0 provenance enforcement | memory extractor write path if it shares helper functions; context assembly for the Q-4 partial trust test |
| V1-0 F-5 hooks | any write path that creates active rows (limited collision; entity writes are separate from memory writes) |
| V1-B KB-CAD/FEM ingest | none on the Now list, but adds an ingest surface that becomes operational burden (ties to O-4 "no new manual ops") |
| V1-D mirror regen triggers | scheduling / ops behavior that intersects with "boring and dependable" gate — mirror regen failures become an observable that the pipeline soak must accommodate |
| V1-E graduation | memory module (new `graduated` status, memory→entity flow); direct collision with memory extractor + triage |
| V1-F F-5 migration | conflicts.py touches the write path shared with memory promotion |
**Recommended schedule (revised):**
- **This week (2026-04-22 to 2026-04-26):** Pipeline soak continues.
Density batch-extract continues. V1 work **waits** — V1-0 would start
touching write paths, which is explicitly something we should not do
during a soak window. Density target (100+ active memories) and the
pipeline soak complete first.
- **Week of 2026-04-27:** If soak is clean and density reached, V1-0
starts. V1-0 is a hard prerequisite and cannot be skipped or parallelized.
- **Weeks of 2026-05-04 and 2026-05-11:** V1-A through V1-D in order.
Multi-model triage work (Now list item 3) continues in parallel only
if its touch-surface is triage-path-only (memory side). Any memory
extractor change pauses V1-E.
- **Week of 2026-05-18 approx:** V1-E (graduation). **This phase must
not run in parallel with memory extractor changes** — it directly
modifies memory module semantics. Multi-model triage should be settled
before V1-E starts.
- **Week of 2026-05-25:** V1-F.
- **End date target:** ~2026-06-01, four weeks later than the first
draft's 2026-05-18 soft target. The shift is deliberate — the first
draft's "parallel / disjoint" claim understated the real collisions.
**Pause points (explicit):**
- Any Now-list item that regresses the pipeline → V1 pauses immediately.
- Memory extractor changes in flight → V1-E pauses until they land and
soak.
- p04-constraints fix requires retrieval ranking changes → V1 does not
pause (retrieval is genuinely disjoint from entities).
- Multi-model triage work touching the entity extractor path (if one
gets prototyped) → V1-0 pauses until the triage decision settles.
---
## Test project
Per `engineering-v1-acceptance.md:379`, the recommended test bed is
**p05-interferometer** — "the optical/structural domain has the cleanest
entity model". I agree. Every F-2, F-3, F-6 criterion asserts against this
project.
p06-polisher is the backup test bed if p05 turns out to have data gaps
(polisher suite is actively worked and has more content).
---
## What V1 completion does NOT include
Per the negative list in `engineering-v1-acceptance.md:351-373`, all of the
following are **explicitly out of scope** for this plan:
- LLM extractor for entities (rule-based is V1)
- Auto-promotion of candidates (human-only in V1)
- Write-back to KB-CAD / KB-FEM
- Multi-user auth
- Real-time UI (API + Mirror markdown only)
- Cross-project rollups
- Time-travel queries (Q-015 stays stretch)
- Nightly conflict sweep (synchronous only)
- Incremental Chroma snapshots
- Retention cleanup script
- Backup encryption
- Off-Dalidou backup target (already exists at clawdbot per ledger, but
not a V1 criterion)
- **Async job queue / minions pattern** (the rejected plan's centerpiece —
explicitly deferred to post-V1 per the negative list)
---
## Open questions for Codex (post-second-round revision)
Three of the original eight questions (F-1 field audit, F-2 per-query
audit, F-5 schema divergence) were answered by Codex's 2026-04-22 audit
and folded into the plan. One open question remains; the rest are now
resolved in-plan:
1. **Parallel schedule vs Now list.** The first-round review correctly
softened this from "fully parallel" to "less disjoint than claimed".
Is the revised collision table + pause-points section enough, or
should specific Now-list items gate specific V1 phases more strictly?
2. **F-7 graduation gap depth.** Resolved by Codex audit. The schema and
preserve-original-memory hook are already in place, so V1-E is not a
greenfield build. But the direct `/memory/{id}/graduate` route and the
ontology/spec mismatch around `knowledge``Fact` are still open, so
V1-E is closer to **34 days** than 2.
3. **Mirror determinism — where does `now` go?** Resolved. Keep the
regenerated timestamp in the rendered output if desired, but pass it
into the renderer as an input value. Golden-file tests pin that input;
render code must not read the clock directly.
4. **`project` field naming.** Resolved. Keep the existing `project`
field; add the explicit doc note that it is the project identifier for
V1 acceptance purposes. No storage rename needed.
5. **Velocity calibration.** Resolved. **17.519.5 focused days** is a
fair engineering-effort estimate after the F-7 audit. For an actual
operator schedule, keep additional buffer for context switching, soak,
and review rounds.
6. **Minions/queue as V2 item in D-3.** Resolved. Do not name the
rejected "Minions" plan in V1 release notes. If D-3 includes a future
work section, refer to it neutrally as "queued background processing /
async workers" rather than canonizing a V2 codename before V2 is
designed.
---
## Risks
| Risk | Mitigation |
|---|---|
| V1 work slows the Now list | V1 pauses on any Now-list blocker. Codex veto on any V1 PR that touches memory extractor, retrieval ranking, or triage paths |
| F-5 detector generalization harder than estimated | Codex audit confirmed schema is already spec-compliant; only detector body + routes need work. If detector generalization still slips, keep per-type detectors and document as a V1.1 cleanup (detection correctness is unaffected, only code organization) |
| Mirror determinism regresses existing mirror output | Keep `/projects/{project_name}/mirror` alias returning the current shape; new `/mirror/{project}/overview` is the spec-compliant one. Deprecate old in V1 release notes |
| Golden file churn as templates evolve | Standard workflow: updating a golden file is a normal part of template work, documented in V1-C commit message |
| Backup drill on Dalidou is disruptive | Run against a clone of the Dalidou DB at a safe hour; no production drill required for V1 acceptance |
| p05-interferometer data gaps | Fall back to p06-polisher per this plan's test-project section |
| Scope creep during V1-A query audit | Any query that isn't in the v1-required set (Q-021 onward) is out of scope, period |
---
## What this plan is **for**
1. A checklist Claude can follow to close V1.
2. A review target for Codex — every phase has explicit acceptance
criteria tied to the acceptance doc.
3. A communication artifact for Antoine — "here's what's left, here's why,
here's the order, here's the risk."
## What this plan is **not**
1. Not a commitment to start tomorrow. Pipeline soak + density density
come first in parallel; V1-A can start this week only because it's
zero-risk additive work.
2. Not a rewrite. Every phase builds on existing code.
3. Not an ontology debate. The ontology is fixed in
`engineering-ontology-v1.md`. Any desire to change it is V2 material.
## Workspace note (for Codex audit)
Codex's first-round review (2026-04-22) flagged that
`docs/plans/engineering-v1-completion-plan.md` and `DEV-LEDGER.md` were
**not visible** in the Playground workspace they were running against,
and that `src/atocore/engineering/` appeared empty there.
The canonical dev workspace for AtoCore is the Windows path
`C:\Users\antoi\ATOCore` (per `CLAUDE.md`). The engineering layer code
(`src/atocore/engineering/service.py`, `queries.py`, `conflicts.py`,
`mirror.py`, `_graduation_prompt.py`, `wiki.py`, `triage_ui.py`) exists
there and is what the recent commits (e147ab2, b94f9df, 081c058, 069d155,
b1a3dd0) touched. The Windows working tree is what this plan was written
against.
Before the file-level audit:
1. Confirm which branch / SHA Codex is reviewing. The Windows working
tree has uncommitted changes to this plan + DEV-LEDGER as of
2026-04-22; commit will be made only after Antoine approves sync.
2. If Codex is reviewing `ATOCore-clean` or a Playground snapshot, that
tree may lag the canonical dev tree. Sync or re-clone from the
Windows working tree / current `origin/main` before per-file audit.
3. The three visible-to-Codex file paths for this plan are:
- `docs/plans/engineering-v1-completion-plan.md` (this file)
- `docs/decisions/2026-04-22-gbrain-plan-rejection.md` (prior decision)
- `DEV-LEDGER.md` (Recent Decisions + Session Log entries 2026-04-22)
## References
- `docs/architecture/engineering-ontology-v1.md`
- `docs/architecture/engineering-query-catalog.md`
- `docs/architecture/memory-vs-entities.md`
- `docs/architecture/engineering-v1-acceptance.md`
- `docs/architecture/promotion-rules.md`
- `docs/architecture/conflict-model.md`
- `docs/architecture/human-mirror-rules.md`
- `docs/architecture/tool-handoff-boundaries.md`
- `docs/master-plan-status.md` (Now/Next/Later list)
- `docs/decisions/2026-04-22-gbrain-plan-rejection.md` (the rejected plan)
- `src/atocore/engineering/service.py` (current V1 entity service)
- `src/atocore/engineering/queries.py` (current V1 query implementations)
- `src/atocore/engineering/conflicts.py` (current conflicts module)
- `src/atocore/engineering/mirror.py` (current mirror module)

View File

@@ -0,0 +1,161 @@
# V1 Completion — Resume State
**Last updated:** 2026-04-22 (after V1-0 landed + R14 branch pushed)
**Purpose:** single-page "you are here" so any future session can pick up
the V1 completion sprint without re-reading the full plan history.
## State of play
- **V1-0 is DONE.** Merged to main as `2712c5d`, deployed to Dalidou,
prod backfill ran cleanly (31 legacy entities flagged
`hand_authored=1`, zero violations remaining).
- **R14 is on a branch.** `claude/r14-promote-400` at `3888db9`
HTTP promote route returns 400 instead of 500 on V1-0 `ValueError`.
Pending Codex review + squash-merge. Non-blocking for V1-A.
- **V1-A is next but GATED.** Doesn't start until both gates clear.
## Start-gates for V1-A
| Gate | Condition | Status as of 2026-04-22 |
|---|---|---|
| Soak | Four clean nightly cycles since F4 confidence-decay first real run 2026-04-19 | Day 3 of 4 — expected clear around **2026-04-26** |
| Density | 100+ active memories | 84 active as of last ledger update — need +16. Lever: `scripts/batch_llm_extract_live.py` against 234-interaction backlog |
**When both are green, start V1-A.** If only one is green, hold.
## Pre-flight checklist when resuming
Before opening the V1-A branch, run through this in order:
1. `git checkout main && git pull` — make sure you're at the tip
2. Check `DEV-LEDGER.md` **Orientation** for current `live_sha`, `test_count`, `active_memories`
3. Check `/health` on Dalidou returns the same `build_sha` as Orientation
4. Check the dashboard for pipeline health: http://dalidou:8100/admin/dashboard
5. Confirm R14 branch status — either merged or explicitly deferred
6. Re-read the two core plan docs:
- `docs/plans/engineering-v1-completion-plan.md` — the full 7-phase plan
- `docs/architecture/engineering-v1-acceptance.md` — the acceptance contract
7. Skim the relevant spec docs for the phase you're about to start:
- V1-A: `engineering-query-catalog.md` (Q-001 + Q-006/Q-009/Q-011 killer queries)
- V1-B: `tool-handoff-boundaries.md` (KB-CAD/KB-FEM export shapes)
- V1-C: `engineering-query-catalog.md` (all remaining v1-required queries)
- V1-D: `human-mirror-rules.md` (mirror spec end-to-end)
- V1-E: `memory-vs-entities.md` (graduation flow)
- V1-F: `conflict-model.md` (generic slot-key detector)
## What V1-A looks like when started
**Branch:** `claude/v1-a-pillar-queries`
**Scope (~1.5 days):**
- **Q-001 shape fix.** Add a subsystem-scoped variant of `system_map()`
matching `GET /entities/Subsystem/<id>?expand=contains` per
`engineering-query-catalog.md:71`. The project-wide version stays
(it serves Q-004).
- **Q-6 integration test.** Seed p05-interferometer with five cases:
1 satisfying Component, 1 orphan Requirement, 1 Decision on flagged
Assumption, 1 supported ValidationClaim, 1 unsupported ValidationClaim.
One test asserting Q-006 / Q-009 / Q-011 return exactly the expected
members.
- The four "pillar" queries (Q-001, Q-005, Q-006, Q-017) already work
per Codex's 2026-04-22 audit. V1-A does NOT re-implement them —
V1-A verifies them on seeded data.
**Acceptance:** Q-001 subsystem-scoped variant + Q-6 integration test both
green. F-2 moves from 🟡 partial to slightly-less-partial.
**Estimated tests added:** ~4 (not ~12 — V1-A scope shrank after Codex
confirmed most queries already work).
## Map of the remaining phases
```
V1-0 ✅ write-time invariants landed 2026-04-22 (2712c5d)
V1-A 🟡 minimum query slice gated on soak + density (~1.5d when started)
V1-B KB-CAD/KB-FEM ingest + D-2 ~2d
V1-C close 8 remaining queries ~2d
V1-D full mirror + determinism ~3-4d (biggest phase)
V1-E graduation + trust tests ~3-4d (pauses for multi-model triage)
V1-F F-5 generalization + ops + docs ~3d — V1 done
```
## Parallel work that can run WITHOUT touching V1
These are genuinely disjoint surfaces; pick any of them during the gate
pause or as scheduling allows:
- **Density batch-extract** — *required* to unblock V1-A. Not optional.
- **p04-constraints harness fix** — retrieval-ranking change, fully
disjoint from entities. Safe to do anywhere in the V1 track.
- **Multi-model triage (Phase 11 entry)** — memory-side work, disjoint
from V1-A/B/C/D. **Pause before V1-E starts** because V1-E touches
memory module semantics.
## What NOT to do
- Don't start V1-A until both gates are green.
- Don't touch the memory extractor write path while V1-E is open.
- Don't name the rejected "Minions" plan in any doc — neutral wording
only ("queued background processing / async workers") per Codex
sign-off.
- Don't rename the `project` field to `project_id` — Codex + Antoine
agreed it stays as `project`, with a doc note in
`engineering-ontology-v1.md` that this IS the project_id per spec.
## Open review findings
| id | severity | summary | status |
|---|---|---|---|
| R14 | P2 | `POST /entities/{id}/promote` returns 500 on V1-0 `ValueError` instead of 400 | fixed on branch `claude/r14-promote-400`, pending Codex review |
Closed V1-0 findings: P1 "promote path allows provenance-less legacy
candidates" (service.py:365-379), P1 "supersede path missing F-5 hook"
(service.py:581-591), P2 "`--invalidate-instead` backfill too broad"
(v1_0_backfill_provenance.py:52-63). All three patched and approved in
the squash-merge to `2712c5d`.
## How agreement between Claude + Codex has worked so far
Three review rounds before V1-0 started + three during implementation:
1. **Rejection round.** Claude drafted a gbrain-inspired "Phase 8
Minions + typed edges" plan; Codex rejected as wrong-packaging.
Record: `docs/decisions/2026-04-22-gbrain-plan-rejection.md`.
2. **Completion-plan rewrite.** Claude rewrote against
`engineering-v1-acceptance.md`. Codex first-round review fixed the
phase order (provenance-first).
3. **Per-file audit.** Codex's second-round audit found F-1 / F-2 /
F-5 gaps, all folded in.
4. **Sign-off round.** Codex's third-round review resolved the five
remaining open questions inline and signed off: *"with those edits,
I'd sign off on the five questions."*
5. **V1-0 review.** Codex found two P1 gaps (promote re-check missing,
supersede hook missing) + one P2 (backfill scope too broad). All
three patched. Codex re-ran probes + regression suites, approved,
squash-merged.
6. **V1-0 deploy + prod backfill.** Codex deployed + ran backfill,
logged R14 as P2 residual.
Protocol has been: Claude writes, Codex audits, human Antoine ratifies.
Continue this for V1-A onward.
## References
- `docs/plans/engineering-v1-completion-plan.md` — full 7-phase plan
- `docs/decisions/2026-04-22-gbrain-plan-rejection.md` — prior rejection
- `docs/architecture/engineering-ontology-v1.md` — V1 ontology (18 predicates)
- `docs/architecture/engineering-query-catalog.md` — Q-001 through Q-020 spec
- `docs/architecture/engineering-v1-acceptance.md` — F/Q/O/D acceptance table
- `docs/architecture/promotion-rules.md` — candidate → active flow
- `docs/architecture/conflict-model.md` — F-5 spec
- `docs/architecture/human-mirror-rules.md` — V1-D spec
- `docs/architecture/memory-vs-entities.md` — V1-E spec
- `docs/architecture/tool-handoff-boundaries.md` — V1-B KB-CAD/KB-FEM
- `docs/master-plan-status.md` — Now / Active / Next / Later
- `DEV-LEDGER.md` — Orientation + Open Review Findings + Session Log

View File

@@ -0,0 +1,167 @@
"""V1-0 one-time backfill: flag existing active entities that have no
provenance (empty source_refs) as hand_authored=1 so they stop failing
the F-8 invariant.
Runs against the live AtoCore DB. Idempotent: a second run after the
first touches nothing because the flagged rows already have
hand_authored=1.
Per the Engineering V1 Completion Plan (V1-0 scope), the three options
for an existing active entity without provenance are:
1. Attach provenance — impossible without human review, not automatable
2. Flag hand-authored — safe, additive, this script's default
3. Invalidate — destructive, requires operator sign-off
This script picks option (2) by default. Add --dry-run to see what
would change without writing. Add --invalidate-instead to pick option
(3) for all rows (not recommended for first run).
Usage:
python scripts/v1_0_backfill_provenance.py --base-url http://dalidou:8100 --dry-run
python scripts/v1_0_backfill_provenance.py --base-url http://dalidou:8100
"""
from __future__ import annotations
import argparse
import json
import sqlite3
import sys
from pathlib import Path
def run(db_path: Path, dry_run: bool, invalidate_instead: bool) -> int:
if not db_path.exists():
print(f"ERROR: db not found: {db_path}", file=sys.stderr)
return 2
conn = sqlite3.connect(str(db_path))
conn.row_factory = sqlite3.Row
# Verify the V1-0 migration ran: if hand_authored column is missing
# the operator hasn't deployed V1-0 yet, and running this script
# would crash. Fail loud rather than attempt the ALTER here.
cols = {r["name"] for r in conn.execute("PRAGMA table_info(entities)").fetchall()}
if "hand_authored" not in cols:
print(
"ERROR: entities table lacks the hand_authored column. "
"Deploy V1-0 migrations first (init_db + init_engineering_schema).",
file=sys.stderr,
)
return 2
# Scope differs by mode:
# - Default (flag hand_authored=1): safe/additive, applies to active
# AND superseded rows so the historical trail is consistent.
# - --invalidate-instead: destructive — scope to ACTIVE rows only.
# Invalidating already-superseded history would collapse the audit
# trail, which the plan's remediation scope never intended
# (V1-0 talks about existing active no-provenance entities).
if invalidate_instead:
scope_sql = "status = 'active'"
else:
scope_sql = "status IN ('active', 'superseded')"
rows = conn.execute(
f"SELECT id, entity_type, name, project, status, source_refs, hand_authored "
f"FROM entities WHERE {scope_sql} AND hand_authored = 0"
).fetchall()
needs_fix = []
for row in rows:
refs_raw = row["source_refs"] or "[]"
try:
refs = json.loads(refs_raw)
except Exception:
refs = []
if not refs:
needs_fix.append(row)
print(f"found {len(needs_fix)} active/superseded entities with no provenance")
for row in needs_fix:
print(
f" - {row['id'][:8]} [{row['entity_type']}] "
f"{row['name']!r} project={row['project']!r} status={row['status']}"
)
if dry_run:
print("--dry-run: no changes written")
return 0
if not needs_fix:
print("nothing to do")
return 0
action = "invalidate" if invalidate_instead else "flag hand_authored=1"
print(f"applying: {action}")
cur = conn.cursor()
for row in needs_fix:
if invalidate_instead:
cur.execute(
"UPDATE entities SET status = 'invalid', "
"updated_at = CURRENT_TIMESTAMP WHERE id = ?",
(row["id"],),
)
cur.execute(
"INSERT INTO memory_audit "
"(id, memory_id, action, actor, before_json, after_json, note, entity_kind) "
"VALUES (?, ?, 'invalidated', 'v1_0_backfill', ?, ?, ?, 'entity')",
(
f"v10bf-{row['id'][:8]}-inv",
row["id"],
json.dumps({"status": row["status"]}),
json.dumps({"status": "invalid"}),
"V1-0 backfill: invalidated, no provenance",
),
)
else:
cur.execute(
"UPDATE entities SET hand_authored = 1, "
"updated_at = CURRENT_TIMESTAMP WHERE id = ?",
(row["id"],),
)
cur.execute(
"INSERT INTO memory_audit "
"(id, memory_id, action, actor, before_json, after_json, note, entity_kind) "
"VALUES (?, ?, 'hand_authored_flagged', 'v1_0_backfill', ?, ?, ?, 'entity')",
(
f"v10bf-{row['id'][:8]}-ha",
row["id"],
json.dumps({"hand_authored": False}),
json.dumps({"hand_authored": True}),
"V1-0 backfill: flagged hand_authored since source_refs empty",
),
)
conn.commit()
print(f"done: updated {len(needs_fix)} entities")
return 0
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument(
"--db",
type=Path,
default=Path("data/db/atocore.db"),
help="Path to the SQLite database (default: data/db/atocore.db)",
)
parser.add_argument("--dry-run", action="store_true", help="Report only; no writes")
parser.add_argument(
"--invalidate-instead",
action="store_true",
help=(
"DESTRUCTIVE. Invalidate active rows with no provenance instead "
"of flagging them hand_authored. Scoped to status='active' only "
"(superseded rows are left alone to preserve audit history). "
"Not recommended for first run — start with --dry-run, then "
"the default hand_authored flag path."
),
)
args = parser.parse_args()
return run(args.db, args.dry_run, args.invalidate_instead)
if __name__ == "__main__":
sys.exit(main())

View File

@@ -1457,6 +1457,11 @@ class EntityCreateRequest(BaseModel):
status: str = "active" status: str = "active"
confidence: float = 1.0 confidence: float = 1.0
source_refs: list[str] | None = None source_refs: list[str] | None = None
# V1-0 provenance enforcement (F-8). Clients must either pass
# non-empty source_refs or set hand_authored=true. The service layer
# raises ValueError otherwise, surfaced here as 400.
hand_authored: bool = False
extractor_version: str | None = None
class EntityPromoteRequest(BaseModel): class EntityPromoteRequest(BaseModel):
@@ -1486,6 +1491,8 @@ def api_create_entity(req: EntityCreateRequest) -> dict:
confidence=req.confidence, confidence=req.confidence,
source_refs=req.source_refs, source_refs=req.source_refs,
actor="api-http", actor="api-http",
hand_authored=req.hand_authored,
extractor_version=req.extractor_version,
) )
except ValueError as e: except ValueError as e:
raise HTTPException(status_code=400, detail=str(e)) raise HTTPException(status_code=400, detail=str(e))

View File

@@ -63,6 +63,12 @@ RELATIONSHIP_TYPES = [
ENTITY_STATUSES = ["candidate", "active", "superseded", "invalid"] ENTITY_STATUSES = ["candidate", "active", "superseded", "invalid"]
# V1-0: extractor version this module writes into new entity rows.
# Per promotion-rules.md:268, every candidate must record the version of
# the extractor that produced it so later re-evaluation is auditable.
# Bump this when extraction logic materially changes.
EXTRACTOR_VERSION = "v1.0.0"
@dataclass @dataclass
class Entity: class Entity:
@@ -77,6 +83,10 @@ class Entity:
source_refs: list[str] = field(default_factory=list) source_refs: list[str] = field(default_factory=list)
created_at: str = "" created_at: str = ""
updated_at: str = "" updated_at: str = ""
# V1-0 shared-header fields per engineering-v1-acceptance.md:45.
extractor_version: str = ""
canonical_home: str = "entity"
hand_authored: bool = False
@dataclass @dataclass
@@ -103,10 +113,25 @@ def init_engineering_schema() -> None:
status TEXT NOT NULL DEFAULT 'active', status TEXT NOT NULL DEFAULT 'active',
confidence REAL NOT NULL DEFAULT 1.0, confidence REAL NOT NULL DEFAULT 1.0,
source_refs TEXT NOT NULL DEFAULT '[]', source_refs TEXT NOT NULL DEFAULT '[]',
extractor_version TEXT NOT NULL DEFAULT '',
canonical_home TEXT NOT NULL DEFAULT 'entity',
hand_authored INTEGER NOT NULL DEFAULT 0,
created_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP, created_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP updated_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP
) )
""") """)
# V1-0 (Engineering V1 completion): the three shared-header fields
# per engineering-v1-acceptance.md:45. Idempotent ALTERs for
# databases created before V1-0 land these columns without a full
# migration. Fresh DBs get them via the CREATE TABLE above; the
# ALTERs below are a no-op there.
from atocore.models.database import _column_exists # late import; avoids cycle
if not _column_exists(conn, "entities", "extractor_version"):
conn.execute("ALTER TABLE entities ADD COLUMN extractor_version TEXT DEFAULT ''")
if not _column_exists(conn, "entities", "canonical_home"):
conn.execute("ALTER TABLE entities ADD COLUMN canonical_home TEXT DEFAULT 'entity'")
if not _column_exists(conn, "entities", "hand_authored"):
conn.execute("ALTER TABLE entities ADD COLUMN hand_authored INTEGER DEFAULT 0")
conn.execute(""" conn.execute("""
CREATE TABLE IF NOT EXISTS relationships ( CREATE TABLE IF NOT EXISTS relationships (
id TEXT PRIMARY KEY, id TEXT PRIMARY KEY,
@@ -149,6 +174,8 @@ def create_entity(
confidence: float = 1.0, confidence: float = 1.0,
source_refs: list[str] | None = None, source_refs: list[str] | None = None,
actor: str = "api", actor: str = "api",
hand_authored: bool = False,
extractor_version: str | None = None,
) -> Entity: ) -> Entity:
if entity_type not in ENTITY_TYPES: if entity_type not in ENTITY_TYPES:
raise ValueError(f"Invalid entity type: {entity_type}. Must be one of {ENTITY_TYPES}") raise ValueError(f"Invalid entity type: {entity_type}. Must be one of {ENTITY_TYPES}")
@@ -157,6 +184,21 @@ def create_entity(
if not name or not name.strip(): if not name or not name.strip():
raise ValueError("Entity name must be non-empty") raise ValueError("Entity name must be non-empty")
refs = list(source_refs) if source_refs else []
# V1-0 (F-8 provenance enforcement, engineering-v1-acceptance.md:147):
# every new entity row must carry non-empty source_refs OR be explicitly
# flagged hand_authored. This is the non-negotiable invariant every
# later V1 phase depends on — without it, active entities can escape
# into the graph with no traceable origin. Raises at the write seam so
# the bug is impossible to introduce silently.
if not refs and not hand_authored:
raise ValueError(
"source_refs required: every entity must carry provenance "
"(source_chunk_id / source_interaction_id / kb_cad_export_id / ...) "
"or set hand_authored=True to explicitly flag a direct human write"
)
# Phase 5: enforce project canonicalization contract at the write seam. # Phase 5: enforce project canonicalization contract at the write seam.
# Aliases like "p04" become "p04-gigabit" so downstream reads stay # Aliases like "p04" become "p04-gigabit" so downstream reads stay
# consistent with the registry. # consistent with the registry.
@@ -165,18 +207,22 @@ def create_entity(
entity_id = str(uuid.uuid4()) entity_id = str(uuid.uuid4())
now = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M:%S") now = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M:%S")
props = properties or {} props = properties or {}
refs = source_refs or [] ev = extractor_version if extractor_version is not None else EXTRACTOR_VERSION
with get_connection() as conn: with get_connection() as conn:
conn.execute( conn.execute(
"""INSERT INTO entities """INSERT INTO entities
(id, entity_type, name, project, description, properties, (id, entity_type, name, project, description, properties,
status, confidence, source_refs, created_at, updated_at) status, confidence, source_refs,
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)""", extractor_version, canonical_home, hand_authored,
created_at, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)""",
( (
entity_id, entity_type, name.strip(), project, entity_id, entity_type, name.strip(), project,
description, json.dumps(props), status, confidence, description, json.dumps(props), status, confidence,
json.dumps(refs), now, now, json.dumps(refs),
ev, "entity", 1 if hand_authored else 0,
now, now,
), ),
) )
@@ -194,14 +240,31 @@ def create_entity(
"project": project, "project": project,
"status": status, "status": status,
"confidence": confidence, "confidence": confidence,
"hand_authored": hand_authored,
"extractor_version": ev,
}, },
) )
# V1-0 (F-5 hook, engineering-v1-acceptance.md:99): synchronous
# conflict detection on any active-entity write. The promote path
# already had this hook (see promote_entity below); V1-0 adds it to
# direct-active creates so every active row — however it got that
# way — is checked. Fail-open per "flag, never block" rule in
# conflict-model.md:256: detector errors log but never fail the write.
if status == "active":
try:
from atocore.engineering.conflicts import detect_conflicts_for_entity
detect_conflicts_for_entity(entity_id)
except Exception as e:
log.warning("conflict_detection_failed", entity_id=entity_id, error=str(e))
return Entity( return Entity(
id=entity_id, entity_type=entity_type, name=name.strip(), id=entity_id, entity_type=entity_type, name=name.strip(),
project=project, description=description, properties=props, project=project, description=description, properties=props,
status=status, confidence=confidence, source_refs=refs, status=status, confidence=confidence, source_refs=refs,
created_at=now, updated_at=now, created_at=now, updated_at=now,
extractor_version=ev, canonical_home="entity",
hand_authored=hand_authored,
) )
@@ -361,6 +424,20 @@ def promote_entity(
if entity is None or entity.status != "candidate": if entity is None or entity.status != "candidate":
return False return False
# V1-0 (F-8 provenance re-check at promote). The invariant must hold at
# BOTH create_entity AND promote_entity per the plan, because candidate
# rows can exist in the DB from before V1-0 (no enforcement at their
# create time) or can be inserted by code paths that bypass the service
# layer. Block any candidate with empty source_refs that is NOT flagged
# hand_authored from ever becoming active. Same error shape as the
# create-side check for symmetry.
if not (entity.source_refs or []) and not entity.hand_authored:
raise ValueError(
"source_refs required: cannot promote a candidate with no "
"provenance. Attach source_refs via PATCH /entities/{id}, "
"or flag hand_authored=true before promoting."
)
if target_project is not None: if target_project is not None:
new_project = ( new_project = (
resolve_project_name(target_project) if target_project else "" resolve_project_name(target_project) if target_project else ""
@@ -503,6 +580,22 @@ def supersede_entity(
superseded_by=superseded_by, superseded_by=superseded_by,
error=str(e), error=str(e),
) )
# V1-0 (F-5 hook on supersede, per plan's "every active-entity
# write path"). Supersede demotes `entity_id` AND adds a
# `supersedes` relationship rooted at the already-active
# `superseded_by`. That new edge can create a conflict the
# detector should catch synchronously. Fail-open per
# conflict-model.md:256.
try:
from atocore.engineering.conflicts import detect_conflicts_for_entity
detect_conflicts_for_entity(superseded_by)
except Exception as e:
log.warning(
"conflict_detection_failed",
entity_id=superseded_by,
error=str(e),
)
return True return True
@@ -774,6 +867,15 @@ def get_entity_with_context(entity_id: str) -> dict | None:
def _row_to_entity(row) -> Entity: def _row_to_entity(row) -> Entity:
# V1-0 shared-header fields are optional on read — rows that predate
# V1-0 migration have NULL / missing values, so defaults kick in and
# older tests that build Entity() without the new fields keep passing.
# `row.keys()` lets us tolerate SQLite rows that lack the columns
# entirely (pre-migration sqlite3.Row).
keys = set(row.keys())
extractor_version = (row["extractor_version"] or "") if "extractor_version" in keys else ""
canonical_home = (row["canonical_home"] or "entity") if "canonical_home" in keys else "entity"
hand_authored = bool(row["hand_authored"]) if "hand_authored" in keys and row["hand_authored"] is not None else False
return Entity( return Entity(
id=row["id"], id=row["id"],
entity_type=row["entity_type"], entity_type=row["entity_type"],
@@ -786,6 +888,9 @@ def _row_to_entity(row) -> Entity:
source_refs=json.loads(row["source_refs"] or "[]"), source_refs=json.loads(row["source_refs"] or "[]"),
created_at=row["created_at"] or "", created_at=row["created_at"] or "",
updated_at=row["updated_at"] or "", updated_at=row["updated_at"] or "",
extractor_version=extractor_version,
canonical_home=canonical_home,
hand_authored=hand_authored,
) )

View File

@@ -240,30 +240,10 @@ def render_homepage() -> str:
# Quick stats # Quick stats
all_entities = get_entities(limit=500) all_entities = get_entities(limit=500)
all_memories_raw = get_memories(active_only=True, limit=500) all_memories = get_memories(active_only=True, limit=500)
# Partition real knowledge from ambient provenance so counts are honest.
# Each memory lands in exactly one bucket (low-signal takes priority).
all_memories: list = []
akc_session_count = 0
low_signal_count = 0
for _m in all_memories_raw:
if _is_low_signal_memory(_m):
low_signal_count += 1
elif _is_akc_session_memory(_m):
akc_session_count += 1
else:
all_memories.append(_m)
pending = get_memories(status="candidate", limit=500) pending = get_memories(status="candidate", limit=500)
lines.append('<h2>System</h2>') lines.append('<h2>System</h2>')
lines.append( lines.append(f'<p>{len(all_entities)} entities · {len(all_memories)} active memories · {len(projects)} projects</p>')
f'<p>{len(all_entities)} entities · {len(all_memories)} memories · '
f'{len(projects)} projects'
+ (f' · <span style="color:#888;">{akc_session_count} AKC session snapshots'
+ (f", {low_signal_count} low-signal hidden" if low_signal_count else "")
+ '</span>'
if akc_session_count or low_signal_count else '')
+ '</p>'
)
# Triage queue prompt — surfaced prominently if non-empty # Triage queue prompt — surfaced prominently if non-empty
if pending: if pending:
@@ -306,44 +286,6 @@ import re as _re
_WIKILINK_PATTERN = _re.compile(r"\[\[([^\[\]|]+?)(?:\|([^\[\]]+?))?\]\]") _WIKILINK_PATTERN = _re.compile(r"\[\[([^\[\]|]+?)(?:\|([^\[\]]+?))?\]\]")
# ----------------------------------------------------------------- signal/noise
# Memories with these patterns are low-signal ambient artefacts — they
# inflate lists on the homepage and domain pages without being informative.
# They remain in the DB (for provenance / audit) but are filtered from
# default browsing surfaces. Pass `include_low_signal=True` on a page query
# param to surface them.
_LOW_SIGNAL_CONTENT_PATTERNS = (
"(no transcript)", # silent-mic AKC sessions
"synthetic AKC integration", # E2E test pollution
"AKC-E2E-", # E2E test prefix in content
"AKC-IMG-TEST-", # image-upload test prefix
"IMG integration test — synthetic", # E2E narrative header
)
# AKC voice-session ambient memories follow this pattern — they're
# provenance records, not knowledge. Collapse them behind a link on domain
# pages instead of rendering each inline.
_AKC_SESSION_HEADER = "AKC voice session "
def _is_low_signal_memory(mem) -> bool:
"""True for memories whose content is known ambient/test pollution."""
content = (getattr(mem, "content", "") or "")
if not content:
return True
return any(p in content for p in _LOW_SIGNAL_CONTENT_PATTERNS)
def _is_akc_session_memory(mem) -> bool:
"""True for AKC voice-session ambient snapshots (have value as provenance,
but shouldn't clutter topical listings)."""
content = (getattr(mem, "content", "") or "")
tags = getattr(mem, "domain_tags", None) or []
if any(t in ("session", "akc") for t in tags) and "voice" in tags:
return True
return content.startswith(_AKC_SESSION_HEADER)
def _resolve_wikilink(target: str, current_project: str | None) -> tuple[str, str, str]: def _resolve_wikilink(target: str, current_project: str | None) -> tuple[str, str, str]:
"""Resolve a ``[[Name]]`` target to ``(href, css_class, extra_suffix)``. """Resolve a ``[[Name]]`` target to ``(href, css_class, extra_suffix)``.
@@ -449,6 +391,8 @@ def render_new_entity_form(name: str = "", project: str = "") -> str:
entity_type: fd.get('entity_type'), entity_type: fd.get('entity_type'),
project: fd.get('project') || '', project: fd.get('project') || '',
description: fd.get('description') || '', description: fd.get('description') || '',
// V1-0: human writes via the wiki form are hand_authored by definition.
hand_authored: true,
}; };
try { try {
const r = await fetch('/v1/entities', { const r = await fetch('/v1/entities', {
@@ -932,22 +876,8 @@ def render_domain(tag: str) -> str:
breadcrumbs=[("Wiki", "/wiki"), ("Domains", "")]) breadcrumbs=[("Wiki", "/wiki"), ("Domains", "")])
all_mems = get_memories(active_only=True, limit=500) all_mems = get_memories(active_only=True, limit=500)
matching_all = [m for m in all_mems matching = [m for m in all_mems
if any((t or "").lower() == tag for t in (m.domain_tags or []))] if any((t or "").lower() == tag for t in (m.domain_tags or []))]
# Partition: low-signal test pollution is hidden entirely, ambient AKC
# session memories are collapsed (shown as a count + link to
# /wiki/activity). Priority: low-signal > session > real.
matching: list = []
akc_sessions: list = []
hidden_low_signal = 0
for m in matching_all:
if _is_low_signal_memory(m):
hidden_low_signal += 1
elif _is_akc_session_memory(m):
akc_sessions.append(m)
else:
matching.append(m)
# Group by project # Group by project
by_project: dict[str, list] = {} by_project: dict[str, list] = {}
@@ -956,18 +886,6 @@ def render_domain(tag: str) -> str:
lines = [f'<h1>Domain: <code>{tag}</code></h1>'] lines = [f'<h1>Domain: <code>{tag}</code></h1>']
lines.append(f'<p class="meta">{len(matching)} active memories across {len(by_project)} projects</p>') lines.append(f'<p class="meta">{len(matching)} active memories across {len(by_project)} projects</p>')
if akc_sessions or hidden_low_signal:
noise_bits = []
if akc_sessions:
noise_bits.append(
f'<a href="/wiki/activity">{len(akc_sessions)} AKC voice session snapshots</a>'
)
if hidden_low_signal:
noise_bits.append(f"{hidden_low_signal} low-signal memories hidden")
lines.append(
f'<p class="meta" style="font-size:0.85rem; color:#888;">'
f'Ambient provenance not listed: {" · ".join(noise_bits)}.</p>'
)
if not matching: if not matching:
lines.append( lines.append(

View File

@@ -146,6 +146,28 @@ def _apply_migrations(conn: sqlite3.Connection) -> None:
"CREATE INDEX IF NOT EXISTS idx_memories_graduated ON memories(graduated_to_entity_id)" "CREATE INDEX IF NOT EXISTS idx_memories_graduated ON memories(graduated_to_entity_id)"
) )
# V1-0 (Engineering V1 completion): shared header fields per
# engineering-v1-acceptance.md:45. Three columns on `entities`:
# - extractor_version: which extractor produced this row. Lets old
# candidates be re-evaluated with a newer extractor per
# promotion-rules.md:268.
# - canonical_home: which layer holds the canonical record. Always
# "entity" for rows written via create_entity; reserved for future
# cross-layer bookkeeping.
# - hand_authored: 1 when the row was created directly by a human
# without source provenance. Enforced by the write path so every
# non-hand-authored row must carry non-empty source_refs (F-8).
# The entities table itself is created by init_engineering_schema
# (see engineering/service.py); these ALTERs cover existing DBs
# where the original CREATE TABLE predates V1-0.
if _table_exists(conn, "entities"):
if not _column_exists(conn, "entities", "extractor_version"):
conn.execute("ALTER TABLE entities ADD COLUMN extractor_version TEXT DEFAULT ''")
if not _column_exists(conn, "entities", "canonical_home"):
conn.execute("ALTER TABLE entities ADD COLUMN canonical_home TEXT DEFAULT 'entity'")
if not _column_exists(conn, "entities", "hand_authored"):
conn.execute("ALTER TABLE entities ADD COLUMN hand_authored INTEGER DEFAULT 0")
# Phase 4 (Robustness V1): append-only audit log for memory mutations. # Phase 4 (Robustness V1): append-only audit log for memory mutations.
# Every create/update/promote/reject/supersede/invalidate/reinforce/expire/ # Every create/update/promote/reject/supersede/invalidate/reinforce/expire/
# auto_promote writes one row here. before/after are JSON snapshots of the # auto_promote writes one row here. before/after are JSON snapshots of the
@@ -352,6 +374,14 @@ def _column_exists(conn: sqlite3.Connection, table: str, column: str) -> bool:
return any(row["name"] == column for row in rows) return any(row["name"] == column for row in rows)
def _table_exists(conn: sqlite3.Connection, table: str) -> bool:
row = conn.execute(
"SELECT name FROM sqlite_master WHERE type='table' AND name=?",
(table,),
).fetchone()
return row is not None
@contextmanager @contextmanager
def get_connection() -> Generator[sqlite3.Connection, None, None]: def get_connection() -> Generator[sqlite3.Connection, None, None]:
"""Get a database connection with row factory.""" """Get a database connection with row factory."""

View File

@@ -16,6 +16,36 @@ os.environ["ATOCORE_DATA_DIR"] = _default_test_dir
os.environ["ATOCORE_DEBUG"] = "true" os.environ["ATOCORE_DEBUG"] = "true"
# V1-0: every entity created in a test is "hand authored" by the test
# author — fixture data, not extracted content. Rather than rewrite 100+
# existing test call sites, wrap create_entity so that tests which don't
# provide source_refs get hand_authored=True automatically. Tests that
# explicitly pass source_refs or hand_authored are unaffected. This keeps
# the F-8 invariant enforced in production (the API, the wiki form, and
# graduation scripts all go through the unwrapped function) while leaving
# the existing test corpus intact.
def _patch_create_entity_for_tests():
from atocore.engineering import service as _svc
_original = _svc.create_entity
def _create_entity_test(*args, **kwargs):
# Only auto-flag when hand_authored isn't explicitly specified.
# Tests that want to exercise the F-8 raise path pass
# hand_authored=False explicitly and should hit the error.
if (
not kwargs.get("source_refs")
and "hand_authored" not in kwargs
):
kwargs["hand_authored"] = True
return _original(*args, **kwargs)
_svc.create_entity = _create_entity_test
_patch_create_entity_for_tests()
@pytest.fixture @pytest.fixture
def tmp_data_dir(tmp_path): def tmp_data_dir(tmp_path):
"""Provide a temporary data directory for tests.""" """Provide a temporary data directory for tests."""

View File

@@ -143,8 +143,11 @@ def test_requirement_name_conflict_detected(tmp_data_dir):
r2 = create_entity("requirement", "Surface figure < 25nm", r2 = create_entity("requirement", "Surface figure < 25nm",
project="p-test", description="Different interpretation") project="p-test", description="Different interpretation")
detected = detect_conflicts_for_entity(r2.id) # V1-0 synchronous hook: the conflict is already detected at r2's
assert len(detected) == 1 # create-time, so a redundant detect call returns [] due to
# _record_conflict dedup. Assert on list_open_conflicts instead —
# that's what the intent of this test really tests: duplicate
# active requirements surface as an open conflict.
conflicts = list_open_conflicts(project="p-test") conflicts = list_open_conflicts(project="p-test")
assert any(c["slot_kind"] == "requirement.name" for c in conflicts) assert any(c["slot_kind"] == "requirement.name" for c in conflicts)
@@ -191,8 +194,12 @@ def test_conflict_resolution_dismiss_leaves_entities_alone(tmp_data_dir):
description="first meaning") description="first meaning")
r2 = create_entity("requirement", "Dup req", project="p-test", r2 = create_entity("requirement", "Dup req", project="p-test",
description="second meaning") description="second meaning")
detected = detect_conflicts_for_entity(r2.id) # V1-0 synchronous hook already recorded the conflict at r2's
conflict_id = detected[0] # create-time. Look it up via list_open_conflicts rather than
# calling the detector again (which returns [] due to dedup).
open_list = list_open_conflicts(project="p-test")
assert open_list, "expected conflict recorded by create-time hook"
conflict_id = open_list[0]["id"]
assert resolve_conflict(conflict_id, "dismiss") assert resolve_conflict(conflict_id, "dismiss")
# Both still active — dismiss just clears the conflict marker # Both still active — dismiss just clears the conflict marker

View File

@@ -132,6 +132,7 @@ def test_api_post_entity_with_null_project_stores_global(seeded_db):
"entity_type": "material", "entity_type": "material",
"name": "Titanium", "name": "Titanium",
"project": None, "project": None,
"hand_authored": True, # V1-0 F-8: test fixture, no source_refs
}) })
assert r.status_code == 200 assert r.status_code == 200

View File

@@ -0,0 +1,362 @@
"""V1-0 write-time invariant tests.
Covers the Engineering V1 completion plan Phase V1-0 acceptance:
- F-1 shared-header fields: extractor_version + canonical_home + hand_authored
land in the entities table with working defaults
- F-8 provenance enforcement: create_entity raises without source_refs
unless hand_authored=True
- F-5 synchronous conflict-detection hook on any active-entity write
(create_entity with status="active" + the pre-existing promote_entity
path); fail-open per conflict-model.md:256
- Q-3 "flag, never block": a conflict never 4xx-blocks the write
- Q-4 partial trust: get_entities scope_only filters candidates out
Plan: docs/plans/engineering-v1-completion-plan.md
Spec: docs/architecture/engineering-v1-acceptance.md
"""
from __future__ import annotations
import pytest
from atocore.engineering.service import (
EXTRACTOR_VERSION,
create_entity,
create_relationship,
get_entities,
get_entity,
init_engineering_schema,
promote_entity,
supersede_entity,
)
from atocore.models.database import get_connection, init_db
# ---------- F-1: shared-header fields ----------
def test_entity_row_has_shared_header_fields(tmp_data_dir):
init_db()
init_engineering_schema()
with get_connection() as conn:
cols = {row["name"] for row in conn.execute("PRAGMA table_info(entities)").fetchall()}
assert "extractor_version" in cols
assert "canonical_home" in cols
assert "hand_authored" in cols
def test_created_entity_has_default_extractor_version_and_canonical_home(tmp_data_dir):
init_db()
init_engineering_schema()
e = create_entity(
entity_type="component",
name="Pivot Pin",
project="p04-gigabit",
source_refs=["test:fixture"],
)
assert e.extractor_version == EXTRACTOR_VERSION
assert e.canonical_home == "entity"
assert e.hand_authored is False
# round-trip through get_entity to confirm the row mapper returns
# the same values (not just the return-by-construct path)
got = get_entity(e.id)
assert got is not None
assert got.extractor_version == EXTRACTOR_VERSION
assert got.canonical_home == "entity"
assert got.hand_authored is False
def test_explicit_extractor_version_is_persisted(tmp_data_dir):
init_db()
init_engineering_schema()
e = create_entity(
entity_type="decision",
name="Pick GF-PTFE pads",
project="p04-gigabit",
source_refs=["interaction:abc"],
extractor_version="custom-v2.3",
)
got = get_entity(e.id)
assert got.extractor_version == "custom-v2.3"
# ---------- F-8: provenance enforcement ----------
def test_create_entity_without_provenance_raises(tmp_data_dir):
init_db()
init_engineering_schema()
with pytest.raises(ValueError, match="source_refs required"):
create_entity(
entity_type="component",
name="No Provenance",
project="p04-gigabit",
hand_authored=False, # explicit — bypasses the test-conftest auto-flag
)
def test_create_entity_with_hand_authored_needs_no_source_refs(tmp_data_dir):
init_db()
init_engineering_schema()
e = create_entity(
entity_type="component",
name="Human Entry",
project="p04-gigabit",
hand_authored=True,
)
assert e.hand_authored is True
got = get_entity(e.id)
assert got.hand_authored is True
# source_refs stays empty — the hand_authored flag IS the provenance
assert got.source_refs == []
def test_create_entity_with_empty_source_refs_list_is_treated_as_missing(tmp_data_dir):
init_db()
init_engineering_schema()
with pytest.raises(ValueError, match="source_refs required"):
create_entity(
entity_type="component",
name="Empty Refs",
project="p04-gigabit",
source_refs=[],
hand_authored=False,
)
def test_promote_rejects_legacy_candidate_without_provenance(tmp_data_dir):
"""Regression (Codex V1-0 probe): candidate rows can exist in the DB
from before V1-0 enforcement (or from paths that bypass create_entity).
promote_entity must re-check the invariant and refuse to flip a
no-provenance candidate to active. Without this check, the active
store can leak F-8 violations in from legacy data."""
init_db()
init_engineering_schema()
# Simulate a pre-V1-0 candidate by inserting directly into the table,
# bypassing the service-layer invariant. Real legacy rows look exactly
# like this: empty source_refs, hand_authored=0.
import uuid as _uuid
entity_id = str(_uuid.uuid4())
with get_connection() as conn:
conn.execute(
"INSERT INTO entities (id, entity_type, name, project, "
"description, properties, status, confidence, source_refs, "
"extractor_version, canonical_home, hand_authored, "
"created_at, updated_at) "
"VALUES (?, 'component', 'Legacy Orphan', 'p04-gigabit', "
"'', '{}', 'candidate', 1.0, '[]', '', 'entity', 0, "
"CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)",
(entity_id,),
)
with pytest.raises(ValueError, match="source_refs required"):
promote_entity(entity_id)
# And the row stays a candidate — no half-transition.
got = get_entity(entity_id)
assert got is not None
assert got.status == "candidate"
def test_promote_accepts_candidate_flagged_hand_authored(tmp_data_dir):
"""The other side of the promote re-check: hand_authored=1 with
empty source_refs still lets promote succeed, matching
create_entity's symmetry."""
init_db()
init_engineering_schema()
import uuid as _uuid
entity_id = str(_uuid.uuid4())
with get_connection() as conn:
conn.execute(
"INSERT INTO entities (id, entity_type, name, project, "
"description, properties, status, confidence, source_refs, "
"extractor_version, canonical_home, hand_authored, "
"created_at, updated_at) "
"VALUES (?, 'component', 'Hand Authored Candidate', "
"'p04-gigabit', '', '{}', 'candidate', 1.0, '[]', '', "
"'entity', 1, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)",
(entity_id,),
)
assert promote_entity(entity_id) is True
assert get_entity(entity_id).status == "active"
# ---------- F-5: synchronous conflict-detection hook ----------
def test_active_create_runs_conflict_detection_hook(tmp_data_dir, monkeypatch):
"""status=active writes trigger detect_conflicts_for_entity."""
init_db()
init_engineering_schema()
called_with: list[str] = []
def _fake_detect(entity_id: str):
called_with.append(entity_id)
return []
import atocore.engineering.conflicts as conflicts_mod
monkeypatch.setattr(conflicts_mod, "detect_conflicts_for_entity", _fake_detect)
e = create_entity(
entity_type="component",
name="Active With Hook",
project="p04-gigabit",
source_refs=["test:hook"],
status="active",
)
assert called_with == [e.id]
def test_supersede_runs_conflict_detection_on_new_active(tmp_data_dir, monkeypatch):
"""Regression (Codex V1-0 probe): per plan's 'every active-entity
write path', supersede_entity must trigger synchronous conflict
detection. The subject is the `superseded_by` entity — the one
whose graph state just changed because a new `supersedes` edge was
rooted at it."""
init_db()
init_engineering_schema()
old = create_entity(
entity_type="component",
name="Old Pad",
project="p04-gigabit",
source_refs=["test:old"],
status="active",
)
new = create_entity(
entity_type="component",
name="New Pad",
project="p04-gigabit",
source_refs=["test:new"],
status="active",
)
called_with: list[str] = []
def _fake_detect(entity_id: str):
called_with.append(entity_id)
return []
import atocore.engineering.conflicts as conflicts_mod
monkeypatch.setattr(conflicts_mod, "detect_conflicts_for_entity", _fake_detect)
assert supersede_entity(old.id, superseded_by=new.id) is True
# The detector fires on the `superseded_by` entity — the one whose
# edges just grew a new `supersedes` relationship.
assert new.id in called_with
def test_supersede_hook_fails_open(tmp_data_dir, monkeypatch):
"""Supersede must survive a broken detector per Q-3 flag-never-block."""
init_db()
init_engineering_schema()
old = create_entity(
entity_type="component", name="Old2", project="p04-gigabit",
source_refs=["test:old"], status="active",
)
new = create_entity(
entity_type="component", name="New2", project="p04-gigabit",
source_refs=["test:new"], status="active",
)
def _boom(entity_id: str):
raise RuntimeError("synthetic detector failure")
import atocore.engineering.conflicts as conflicts_mod
monkeypatch.setattr(conflicts_mod, "detect_conflicts_for_entity", _boom)
# The supersede still succeeds despite the detector blowing up.
assert supersede_entity(old.id, superseded_by=new.id) is True
assert get_entity(old.id).status == "superseded"
def test_candidate_create_does_not_run_conflict_hook(tmp_data_dir, monkeypatch):
"""status=candidate writes do NOT trigger detection — the hook is
for active rows only, per V1-0 scope. Candidates are checked at
promote time."""
init_db()
init_engineering_schema()
called: list[str] = []
def _fake_detect(entity_id: str):
called.append(entity_id)
return []
import atocore.engineering.conflicts as conflicts_mod
monkeypatch.setattr(conflicts_mod, "detect_conflicts_for_entity", _fake_detect)
create_entity(
entity_type="component",
name="Candidate No Hook",
project="p04-gigabit",
source_refs=["test:cand"],
status="candidate",
)
assert called == []
# ---------- Q-3: flag, never block ----------
def test_conflict_detector_failure_does_not_block_write(tmp_data_dir, monkeypatch):
"""Per conflict-model.md:256: detection errors must not fail the
write. The entity is still created; only a warning is logged."""
init_db()
init_engineering_schema()
def _boom(entity_id: str):
raise RuntimeError("synthetic detector failure")
import atocore.engineering.conflicts as conflicts_mod
monkeypatch.setattr(conflicts_mod, "detect_conflicts_for_entity", _boom)
# The write still succeeds — no exception propagates.
e = create_entity(
entity_type="component",
name="Hook Fails Open",
project="p04-gigabit",
source_refs=["test:failopen"],
status="active",
)
assert get_entity(e.id) is not None
# ---------- Q-4 (partial): trust-hierarchy — scope_only filters candidates ----------
def test_scope_only_active_does_not_return_candidates(tmp_data_dir):
"""V1-0 partial Q-4: active-scoped listing never returns candidates.
Full trust-hierarchy coverage (no-auto-project-state, etc.) ships in
V1-E per plan."""
init_db()
init_engineering_schema()
active = create_entity(
entity_type="component",
name="Active Alpha",
project="p04-gigabit",
source_refs=["test:alpha"],
status="active",
)
candidate = create_entity(
entity_type="component",
name="Candidate Beta",
project="p04-gigabit",
source_refs=["test:beta"],
status="candidate",
)
listed = get_entities(project="p04-gigabit", status="active", scope_only=True)
ids = {e.id for e in listed}
assert active.id in ids
assert candidate.id not in ids

View File

@@ -161,103 +161,3 @@ def test_memory_detail_shows_superseded_sources(tmp_data_dir):
assert html1 is not None assert html1 is not None
assert "superseded" in html1 assert "superseded" in html1
assert "auto-dedup-tier1" in html1 # audit trail shows who merged assert "auto-dedup-tier1" in html1 # audit trail shows who merged
# -------------------------------------------------- low-signal wiki filters
# Ambient AKC session memories and test pollution shouldn't dominate domain
# pages / homepage counts. These tests lock the partitioning behaviour.
def test_domain_page_hides_empty_transcript_sessions(tmp_data_dir):
"""Silent-mic AKC sessions (content has '(no transcript)') are ambient
noise — they go into the hidden count, not the main list."""
_init_all()
# One real knowledge memory with tag "optics"
create_memory(
"knowledge",
"CGH null corrector supports F/1.2 asphere testing",
project="p05", confidence=0.9, domain_tags=["optics", "cgh"],
)
# One silent AKC session with the same tag — should NOT appear
create_memory(
"episodic",
"AKC voice session abc (gen-002)\nDuration: 60s, 2 captures\n"
"\n## Transcript\n(no transcript)\n",
project="p05", confidence=0.7,
domain_tags=["optics", "session", "akc", "voice"],
)
html = render_domain("optics")
assert "CGH null corrector" in html
# The hidden-count banner should be present
assert "low-signal" in html or "Ambient provenance" in html
# And the empty-transcript content itself is not rendered inline
assert "(no transcript)" not in html
def test_domain_page_collapses_akc_session_snapshots(tmp_data_dir):
"""AKC voice-session memories are provenance records — count them as
a single collapsed link, don't inline every one."""
_init_all()
for i in range(5):
create_memory(
"episodic",
f"AKC voice session session-{i} (gen-00{i})\nDuration: 120s, 3 captures\n"
f"\n## Transcript\nReal transcript number {i}",
project="p05", confidence=0.7,
domain_tags=["optics", "session", "akc", "voice"],
)
html = render_domain("optics")
# Inline count should mention AKC session snapshots
assert "AKC voice session snapshots" in html
# None of the session transcripts should be pasted inline on the domain
# page (they're provenance, linked via /wiki/activity)
assert "Real transcript number 0" not in html
def test_homepage_stats_exclude_ambient_memory(tmp_data_dir):
"""Homepage system-stats line shows real memory count, pushes ambient
counts into a dimmed sub-segment."""
_init_all()
# 2 real memories + 3 ambient sessions + 1 silent junk
create_memory("knowledge", "Real fact 1", project="p05", confidence=0.8)
create_memory("knowledge", "Real fact 2", project="p05", confidence=0.8)
for i in range(3):
create_memory(
"episodic",
f"AKC voice session s{i} (gen-00{i})\nReal transcript x",
project="p05", confidence=0.7,
domain_tags=["session", "akc", "voice"],
)
create_memory(
"episodic",
"AKC voice session silent (gen-099)\nDuration: 30s, 0 captures\n"
"\n## Transcript\n(no transcript)\n",
project="p05", confidence=0.7,
domain_tags=["session", "akc", "voice"],
)
html = render_homepage()
assert "3 AKC session snapshots" in html
assert "low-signal hidden" in html
# Main count reflects only real knowledge
assert "2 memories" in html
def test_low_signal_predicate_catches_known_patterns():
from atocore.engineering.wiki import _is_low_signal_memory, _is_akc_session_memory
from dataclasses import dataclass
@dataclass
class M:
content: str = ""
domain_tags: list = None
# Explicit empty-transcript — low signal
assert _is_low_signal_memory(M(content="AKC voice session x\n## Transcript\n(no transcript)\n"))
# E2E test pollution — low signal
assert _is_low_signal_memory(M(content="IMG integration test — synthetic session"))
assert _is_low_signal_memory(M(content="synthetic AKC integration session"))
# Real knowledge — NOT low signal
assert not _is_low_signal_memory(M(content="The CGH is mounted to the fold mirror via…"))
# AKC session tag predicate
assert _is_akc_session_memory(M(content="anything", domain_tags=["session", "akc", "voice"]))
assert _is_akc_session_memory(M(content="AKC voice session abc"))
assert not _is_akc_session_memory(M(content="Real fact", domain_tags=["optics"]))