Files
ATOCore/docs/architecture/conflict-model.md

333 lines
14 KiB
Markdown
Raw Permalink Normal View History

docs(arch): memory-vs-entities, promotion-rules, conflict-model Three planning docs that answer the architectural questions the engineering query catalog raised. Together with the catalog they form roughly half of the pre-implementation planning sprint. docs/architecture/memory-vs-entities.md --------------------------------------- Resolves the central question blocking every other engineering layer doc: is a Decision a memory or an entity? Key decisions: - memories stay the canonical home for identity, preference, and episodic facts - entities become the canonical home for project, knowledge, and adaptation facts once the engineering layer V1 ships - no concept lives in both layers at full fidelity; one canonical home per concept - a "graduation" flow lets active memories upgrade into entities (memory stays as a frozen historical pointer, never deleted) - one shared candidate review queue across both layers - context builder budget gains a 15% slot for engineering entities, slotted between identity/preference memories and retrieved chunks - the Phase 9 memory extractor's structural cues (decision heading, constraint heading, requirement heading) are explicitly an intentional temporary overlap, cleanly migrated via graduation when the entity extractor ships docs/architecture/promotion-rules.md ------------------------------------ Defines the full Layer 0 → Layer 2 pipeline: - four layers: L0 raw source, L1 memory candidate/active, L2 entity candidate/active, L3 trusted project state - three extraction triggers: on interaction capture (existing), on ingestion wave (new, batched per wave), on explicit request - per-rule prior confidence tuned at write time by structural signal (echoes the retriever's high/low signal hints) and freshness bonus - batch cap of 50 candidates per pass to protect the reviewer - full provenance requirements: every candidate carries rule id, source_chunk_id, source_interaction_id, and extractor_version - reversibility matrix for every promotion step - explicit no-auto-promotion-in-V1 stance with the schema designed so auto-promotion policies can be added later without migration - the hard invariant: nothing ever moves into L3 automatically - ingestion-wave extraction produces a report artifact under data/extraction-reports/<wave-id>/ docs/architecture/conflict-model.md ----------------------------------- Defines how AtoCore handles contradictory facts without violating the "bad memory is worse than no memory" rule. - conflict = two or more active rows claiming the same slot with incompatible values - per-type "slot key" tuples for both memory and entity types - cross-layer conflict detection respects the trust hierarchy: trusted project state > active entities > active memories - new conflicts and conflict_members tables (schema proposal) - detection at two latencies: synchronous at write time, asynchronous nightly sweep - "flag, never block" rule: writes always succeed, conflicts are surfaced via /conflicts, /health open_conflicts_count, per-row response bodies, and the Human Mirror's disputed marker - resolution is always human: promote-winner + supersede-others, or dismiss-as-not-a-real-conflict, both with audit trail - explicitly out of scope for V1: cross-project conflicts, temporal-overlap conflicts, tolerance-aware numeric comparisons Also updates: - master-plan-status.md: Phase 9 moved from "started" to "baseline complete" now that Commits A, B, C are all landed - master-plan-status.md: adds a "Engineering Layer Planning Sprint" section listing the doc wave so far and the remaining docs (tool-handoff-boundaries, human-mirror-rules, representation-authority, engineering-v1-acceptance) - current-state.md: Phase 9 moved from "not started" to "baseline complete" with the A/B/C annotation This is pure doc work. No code changes, no schema changes, no behavior changes. Per the working rule in master-plan-status.md: the architecture docs shape decisions, they do not force premature schema work.
2026-04-06 21:30:35 -04:00
# Conflict Model (how AtoCore handles contradictory facts)
## Why this document exists
Any system that accumulates facts from multiple sources — interactions,
ingested documents, repo history, PKM notes — will eventually see
contradictory facts about the same thing. AtoCore's operating model
already has the hard rule:
> **Bad memory is worse than no memory.**
The practical consequence of that rule is: AtoCore must never
silently merge contradictory facts, never silently pick a winner,
and never silently discard evidence. Every conflict must be
surfaced to a human reviewer with full audit context.
This document defines what "conflict" means in AtoCore, how
conflicts are detected, how they are represented, how they are
surfaced, and how they are resolved.
## What counts as a conflict
A conflict exists when two or more facts in the system claim
incompatible values for the same conceptual slot. More precisely:
A conflict is a set of two or more **active** rows (across memories,
entities, project_state) such that:
1. They share the same **target identity** — same entity type and
same semantic key
2. Their **claimed values** are incompatible
3. They are all in an **active** status (not superseded, not
invalid, not candidate)
Examples that are conflicts:
- Two active `Decision` entities affecting the same `Subsystem`
with contradictory values for the same decided field (e.g.
lateral support material = GF-PTFE vs lateral support material = PEEK)
- An active `preference` memory "prefers rebase workflow" and an
active `preference` memory "prefers merge-commit workflow"
- A `project_state` entry `p05 / decision / lateral_support_material = GF-PTFE`
and an active `Decision` entity also claiming the lateral support
material is PEEK (cross-layer conflict)
Examples that are NOT conflicts:
- Two active memories both saying "prefers small diffs" — same
meaning, not contradictory
- An active memory saying X and a candidate memory saying Y —
candidates are not active, so this is part of the review queue
flow, not the conflict flow
- A superseded `Decision` saying X and an active `Decision` saying Y
— supersession is a resolved history, not a conflict
- Two active `Requirement` entities each constraining the same
component in different but compatible ways (e.g. one caps mass,
one caps heat flux) — different fields, no contradiction
## Detection triggers
Conflict detection must fire at every write that could create a new
active fact. That means the following hook points:
1. **`POST /memory` creating an active memory** (legacy path)
2. **`POST /memory/{id}/promote`** (candidate → active)
3. **`POST /entities` creating an active entity** (future)
4. **`POST /entities/{id}/promote`** (candidate → active, future)
5. **`POST /project/state`** (curating trusted state directly)
6. **`POST /memory/{id}/graduate`** (memory → entity graduation,
future — the resulting entity could conflict with something)
Extraction passes do NOT trigger conflict detection at candidate
write time. Candidates are allowed to sit in the queue in an
apparently-conflicting state; the reviewer will see them during
promotion and decision-detection fires at that moment.
## Detection strategy per layer
### Memory layer
For identity / preference / episodic memories (the ones that stay
in the memory layer):
- Matching key: `(memory_type, project, normalized_content_family)`
- `normalized_content_family` is not a hash of the content — that
would require exact equality — but a slot identifier extracted
by a small per-type rule set:
- identity: slot is "role" / "background" / "credentials"
- preference: slot is the first content word after "prefers" / "uses" / "likes"
normalized to a lowercase noun stem, OR the rule id that extracted it
- episodic: no slot — episodic entries are intrinsically tied to
a moment in time and rarely conflict
A conflict is flagged when two active memories share a
`(memory_type, project, slot)` but have different content bodies.
### Entity layer (V1)
For each V1 entity type, the conflict key is a short tuple that
uniquely identifies the "slot" that entity is claiming:
| Entity type | Conflict slot |
|-------------------|-------------------------------------------------------|
| Project | `(project_id)` |
| Subsystem | `(project_id, subsystem_name)` |
| Component | `(project_id, subsystem_name, component_name)` |
| Requirement | `(project_id, requirement_key)` |
| Constraint | `(project_id, constraint_target, constraint_kind)` |
| Decision | `(project_id, decision_target, decision_field)` |
| Material | `(project_id, component_id)` |
| Parameter | `(project_id, parameter_scope, parameter_name)` |
| AnalysisModel | `(project_id, subsystem_id, model_name)` |
| Result | `(project_id, analysis_model_id, result_key)` |
| ValidationClaim | `(project_id, claim_key)` |
| Artifact | no conflict detection — artifacts are additive |
A conflict is two active entities with the same slot but
different structural values. The exact "which fields count as
structural" list is per-type and lives in the entity schema doc
(not yet written — tracked as future `engineering-ontology-v1.md`
updates).
### Cross-layer (memory vs entity vs trusted project state)
Trusted project state trumps active entities trumps active
memories. This is the trust hierarchy from the operating model.
Cross-layer conflict detection works by a nightly job that walks
the three layers and flags any slot that has entries in more than
one layer with incompatible values:
- If trusted project state and an entity disagree: the entity is
flagged; trusted state is assumed correct
- If an entity and a memory disagree: the memory is flagged; the
entity is assumed correct
- If trusted state and a memory disagree: the memory is flagged;
trusted state is assumed correct
In all three cases the lower-trust row gets a `conflicts_with`
reference pointing at the higher-trust row but does NOT auto-move
to superseded. The flag is an alert, not an action.
## Representation
Conflicts are represented as rows in a new `conflicts` table
(V1 schema, not yet shipped):
```sql
CREATE TABLE conflicts (
id TEXT PRIMARY KEY,
detected_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
slot_kind TEXT NOT NULL, -- "memory_slot" or "entity_slot" or "cross_layer"
slot_key TEXT NOT NULL, -- JSON-encoded tuple identifying the slot
project TEXT DEFAULT '',
status TEXT NOT NULL DEFAULT 'open', -- open | resolved | dismissed
resolved_at DATETIME,
resolution TEXT DEFAULT '', -- free text from the reviewer
-- links to conflicting rows live in conflict_members
UNIQUE(slot_kind, slot_key, status) -- ensures only one open conflict per slot
);
CREATE TABLE conflict_members (
conflict_id TEXT NOT NULL REFERENCES conflicts(id) ON DELETE CASCADE,
member_kind TEXT NOT NULL, -- "memory" | "entity" | "project_state"
member_id TEXT NOT NULL,
member_layer_trust INTEGER NOT NULL,-- 1=memory, 2=entity, 3=project_state
PRIMARY KEY (conflict_id, member_kind, member_id)
);
```
Constraint rationale:
- `UNIQUE(slot_kind, slot_key, status)` where status='open' prevents
duplicate "conflict already open for this slot" rows. At most one
open conflict exists per slot at a time; new conflicting rows are
added as members to the existing conflict, not as a new conflict.
- `conflict_members.member_layer_trust` is denormalized so the
conflict resolution UI can sort conflicting rows by trust tier
without re-querying.
- `status='dismissed'` exists separately from `resolved` because
"the reviewer looked at this and declared it not a real conflict"
is a valid distinct outcome (the two rows really do describe
different things and the detector was overfitting).
## API shape
```
GET /conflicts list open conflicts
GET /conflicts?status=resolved list resolved conflicts
GET /conflicts?project=p05-interferometer scope by project
GET /conflicts/{id} full detail including all members
POST /conflicts/{id}/resolve mark resolved with notes
body: {
"resolution_notes": "...",
"winner_member_id": "...", # optional: if specified,
# other members are auto-superseded
"action": "supersede_others" # or "no_action" if reviewer
# wants to resolve without touching rows
}
POST /conflicts/{id}/dismiss mark dismissed ("not a real conflict")
body: {
"reason": "..."
}
```
Conflict detection must also surface in existing endpoints:
- `GET /memory/{id}` — response includes a `conflicts` array if
the memory is a member of any open conflict
- `GET /entities/{type}/{id}` (future) — same
- `GET /health` — includes `open_conflicts_count` so the operator
sees at a glance that review is pending
## Supersession as a conflict resolution tool
When the reviewer resolves a conflict with `action: "supersede_others"`,
the winner stays active and every other member is flipped to
status="superseded" with a `superseded_by` pointer to the winner.
This is the normal path: "we used to think X, now we know Y, flag
X as superseded so the audit trail keeps X visible but X no longer
influences context".
The conflict resolution audit record links back to all superseded
members, so the conflict history itself is queryable:
- "Show me every conflict that touched Subsystem X"
- "Show me every Decision that superseded another Decision because
of a conflict"
These are entries in the V1 query catalog (see Q-014 decision history).
## Detection latency
Conflict detection runs at two latencies:
1. **Synchronous (at write time)** — every create/promote/update of
an active row in a conflict-enabled type runs a synchronous
same-layer detector. If a conflict is detected the write still
succeeds but a row is inserted into `conflicts` and the API
response includes a `conflict_id` field so the caller knows
immediately.
2. **Asynchronous (nightly sweep)** — a scheduled job walks all
three layers looking for cross-layer conflicts that slipped
past write-time detection (e.g. a memory that was already
active before an entity with the same slot was promoted). The
sweep also looks for slot overlaps that the synchronous
detector can't see because the slot key extraction rules have
improved since the row was written.
Both paths write to the same `conflicts` table and both are
surfaced in the same review queue.
## The "flag, never block" rule
Detection **never** blocks writes. The operating rule is:
- If the write is otherwise valid (schema, permissions, trust
hierarchy), accept it
- Log the conflict
- Surface it to the reviewer
- Let the system keep functioning with the conflict in place
The alternative — blocking writes on conflict — would mean that
one stale fact could prevent all future writes until manually
resolved, which in practice makes the system unusable for normal
work. The "flag, never block" rule keeps AtoCore responsive while
still making conflicts impossible to ignore (the `/health`
endpoint's `open_conflicts_count` makes them loud).
The one exception: writing to `project_state` (layer 3) when an
open conflict already exists on that slot will return a warning
in the response body. The write still happens, but the reviewer
is explicitly told "you just wrote to a slot that has an open
conflict". This is the highest-trust layer so we want extra
friction there without actually blocking.
## Showing conflicts in the Human Mirror
When the Human Mirror template renders a project overview, any
open conflict in that project shows as a **"⚠ disputed"** marker
next to the affected field, with a link to the conflict detail.
This makes conflicts visible to anyone reading the derived
human-facing pages, not just to reviewers who think to check the
`/conflicts` endpoint.
The Human Mirror render rules (not yet written — tracked as future
`human-mirror-rules.md`) will specify exactly where and how the
disputed marker appears.
## What this document does NOT solve
1. **Automatic conflict resolution.** No policy will ever
automatically promote one conflict member over another. The
trust hierarchy is an *alert ordering* for reviewers, not an
auto-resolve rule. The human signs off on every resolution.
2. **Cross-project conflicts.** If p04 and p06 both have
entities claiming conflicting things about a shared component,
that is currently out of scope because the V1 slot keys all
include `project_id`. Cross-project conflict detection is a
future concern that needs its own slot key strategy.
3. **Temporal conflicts with partial overlap.** If a fact was
true during a time window and another fact is true in a
different time window, that is not a conflict — it's history.
Representing time-bounded facts is deferred to a future
temporal-entities doc.
4. **Probabilistic "soft" conflicts.** If two entities claim the
same slot with slightly different values (e.g. "4.8 kg" vs
"4.82 kg"), is that a conflict? For V1, yes — the string
values are unequal so they're flagged. Tolerance-aware
numeric comparisons are a V2 concern.
## TL;DR
- Conflicts = two or more active rows claiming the same slot with
incompatible values
- Detection fires on every active write AND in a nightly sweep
- Conflicts are stored in a dedicated `conflicts` table with a
`conflict_members` join
- Resolution is always human (promote-winner / supersede-others
/ dismiss-as-not-a-conflict)
- "Flag, never block" — writes always succeed, conflicts are
surfaced via `/conflicts`, `/health`, per-entity responses, and
the Human Mirror
- Trusted project state is the top of the trust hierarchy and is
assumed correct in any cross-layer conflict until the reviewer
says otherwise