docs(arch): memory-vs-entities, promotion-rules, conflict-model
Three planning docs that answer the architectural questions the engineering query catalog raised. Together with the catalog they form roughly half of the pre-implementation planning sprint. docs/architecture/memory-vs-entities.md --------------------------------------- Resolves the central question blocking every other engineering layer doc: is a Decision a memory or an entity? Key decisions: - memories stay the canonical home for identity, preference, and episodic facts - entities become the canonical home for project, knowledge, and adaptation facts once the engineering layer V1 ships - no concept lives in both layers at full fidelity; one canonical home per concept - a "graduation" flow lets active memories upgrade into entities (memory stays as a frozen historical pointer, never deleted) - one shared candidate review queue across both layers - context builder budget gains a 15% slot for engineering entities, slotted between identity/preference memories and retrieved chunks - the Phase 9 memory extractor's structural cues (decision heading, constraint heading, requirement heading) are explicitly an intentional temporary overlap, cleanly migrated via graduation when the entity extractor ships docs/architecture/promotion-rules.md ------------------------------------ Defines the full Layer 0 → Layer 2 pipeline: - four layers: L0 raw source, L1 memory candidate/active, L2 entity candidate/active, L3 trusted project state - three extraction triggers: on interaction capture (existing), on ingestion wave (new, batched per wave), on explicit request - per-rule prior confidence tuned at write time by structural signal (echoes the retriever's high/low signal hints) and freshness bonus - batch cap of 50 candidates per pass to protect the reviewer - full provenance requirements: every candidate carries rule id, source_chunk_id, source_interaction_id, and extractor_version - reversibility matrix for every promotion step - explicit no-auto-promotion-in-V1 stance with the schema designed so auto-promotion policies can be added later without migration - the hard invariant: nothing ever moves into L3 automatically - ingestion-wave extraction produces a report artifact under data/extraction-reports/<wave-id>/ docs/architecture/conflict-model.md ----------------------------------- Defines how AtoCore handles contradictory facts without violating the "bad memory is worse than no memory" rule. - conflict = two or more active rows claiming the same slot with incompatible values - per-type "slot key" tuples for both memory and entity types - cross-layer conflict detection respects the trust hierarchy: trusted project state > active entities > active memories - new conflicts and conflict_members tables (schema proposal) - detection at two latencies: synchronous at write time, asynchronous nightly sweep - "flag, never block" rule: writes always succeed, conflicts are surfaced via /conflicts, /health open_conflicts_count, per-row response bodies, and the Human Mirror's disputed marker - resolution is always human: promote-winner + supersede-others, or dismiss-as-not-a-real-conflict, both with audit trail - explicitly out of scope for V1: cross-project conflicts, temporal-overlap conflicts, tolerance-aware numeric comparisons Also updates: - master-plan-status.md: Phase 9 moved from "started" to "baseline complete" now that Commits A, B, C are all landed - master-plan-status.md: adds a "Engineering Layer Planning Sprint" section listing the doc wave so far and the remaining docs (tool-handoff-boundaries, human-mirror-rules, representation-authority, engineering-v1-acceptance) - current-state.md: Phase 9 moved from "not started" to "baseline complete" with the A/B/C annotation This is pure doc work. No code changes, no schema changes, no behavior changes. Per the working rule in master-plan-status.md: the architecture docs shape decisions, they do not force premature schema work.
This commit is contained in:
332
docs/architecture/conflict-model.md
Normal file
332
docs/architecture/conflict-model.md
Normal file
@@ -0,0 +1,332 @@
|
||||
# Conflict Model (how AtoCore handles contradictory facts)
|
||||
|
||||
## Why this document exists
|
||||
|
||||
Any system that accumulates facts from multiple sources — interactions,
|
||||
ingested documents, repo history, PKM notes — will eventually see
|
||||
contradictory facts about the same thing. AtoCore's operating model
|
||||
already has the hard rule:
|
||||
|
||||
> **Bad memory is worse than no memory.**
|
||||
|
||||
The practical consequence of that rule is: AtoCore must never
|
||||
silently merge contradictory facts, never silently pick a winner,
|
||||
and never silently discard evidence. Every conflict must be
|
||||
surfaced to a human reviewer with full audit context.
|
||||
|
||||
This document defines what "conflict" means in AtoCore, how
|
||||
conflicts are detected, how they are represented, how they are
|
||||
surfaced, and how they are resolved.
|
||||
|
||||
## What counts as a conflict
|
||||
|
||||
A conflict exists when two or more facts in the system claim
|
||||
incompatible values for the same conceptual slot. More precisely:
|
||||
|
||||
A conflict is a set of two or more **active** rows (across memories,
|
||||
entities, project_state) such that:
|
||||
|
||||
1. They share the same **target identity** — same entity type and
|
||||
same semantic key
|
||||
2. Their **claimed values** are incompatible
|
||||
3. They are all in an **active** status (not superseded, not
|
||||
invalid, not candidate)
|
||||
|
||||
Examples that are conflicts:
|
||||
|
||||
- Two active `Decision` entities affecting the same `Subsystem`
|
||||
with contradictory values for the same decided field (e.g.
|
||||
lateral support material = GF-PTFE vs lateral support material = PEEK)
|
||||
- An active `preference` memory "prefers rebase workflow" and an
|
||||
active `preference` memory "prefers merge-commit workflow"
|
||||
- A `project_state` entry `p05 / decision / lateral_support_material = GF-PTFE`
|
||||
and an active `Decision` entity also claiming the lateral support
|
||||
material is PEEK (cross-layer conflict)
|
||||
|
||||
Examples that are NOT conflicts:
|
||||
|
||||
- Two active memories both saying "prefers small diffs" — same
|
||||
meaning, not contradictory
|
||||
- An active memory saying X and a candidate memory saying Y —
|
||||
candidates are not active, so this is part of the review queue
|
||||
flow, not the conflict flow
|
||||
- A superseded `Decision` saying X and an active `Decision` saying Y
|
||||
— supersession is a resolved history, not a conflict
|
||||
- Two active `Requirement` entities each constraining the same
|
||||
component in different but compatible ways (e.g. one caps mass,
|
||||
one caps heat flux) — different fields, no contradiction
|
||||
|
||||
## Detection triggers
|
||||
|
||||
Conflict detection must fire at every write that could create a new
|
||||
active fact. That means the following hook points:
|
||||
|
||||
1. **`POST /memory` creating an active memory** (legacy path)
|
||||
2. **`POST /memory/{id}/promote`** (candidate → active)
|
||||
3. **`POST /entities` creating an active entity** (future)
|
||||
4. **`POST /entities/{id}/promote`** (candidate → active, future)
|
||||
5. **`POST /project/state`** (curating trusted state directly)
|
||||
6. **`POST /memory/{id}/graduate`** (memory → entity graduation,
|
||||
future — the resulting entity could conflict with something)
|
||||
|
||||
Extraction passes do NOT trigger conflict detection at candidate
|
||||
write time. Candidates are allowed to sit in the queue in an
|
||||
apparently-conflicting state; the reviewer will see them during
|
||||
promotion and decision-detection fires at that moment.
|
||||
|
||||
## Detection strategy per layer
|
||||
|
||||
### Memory layer
|
||||
|
||||
For identity / preference / episodic memories (the ones that stay
|
||||
in the memory layer):
|
||||
|
||||
- Matching key: `(memory_type, project, normalized_content_family)`
|
||||
- `normalized_content_family` is not a hash of the content — that
|
||||
would require exact equality — but a slot identifier extracted
|
||||
by a small per-type rule set:
|
||||
- identity: slot is "role" / "background" / "credentials"
|
||||
- preference: slot is the first content word after "prefers" / "uses" / "likes"
|
||||
normalized to a lowercase noun stem, OR the rule id that extracted it
|
||||
- episodic: no slot — episodic entries are intrinsically tied to
|
||||
a moment in time and rarely conflict
|
||||
|
||||
A conflict is flagged when two active memories share a
|
||||
`(memory_type, project, slot)` but have different content bodies.
|
||||
|
||||
### Entity layer (V1)
|
||||
|
||||
For each V1 entity type, the conflict key is a short tuple that
|
||||
uniquely identifies the "slot" that entity is claiming:
|
||||
|
||||
| Entity type | Conflict slot |
|
||||
|-------------------|-------------------------------------------------------|
|
||||
| Project | `(project_id)` |
|
||||
| Subsystem | `(project_id, subsystem_name)` |
|
||||
| Component | `(project_id, subsystem_name, component_name)` |
|
||||
| Requirement | `(project_id, requirement_key)` |
|
||||
| Constraint | `(project_id, constraint_target, constraint_kind)` |
|
||||
| Decision | `(project_id, decision_target, decision_field)` |
|
||||
| Material | `(project_id, component_id)` |
|
||||
| Parameter | `(project_id, parameter_scope, parameter_name)` |
|
||||
| AnalysisModel | `(project_id, subsystem_id, model_name)` |
|
||||
| Result | `(project_id, analysis_model_id, result_key)` |
|
||||
| ValidationClaim | `(project_id, claim_key)` |
|
||||
| Artifact | no conflict detection — artifacts are additive |
|
||||
|
||||
A conflict is two active entities with the same slot but
|
||||
different structural values. The exact "which fields count as
|
||||
structural" list is per-type and lives in the entity schema doc
|
||||
(not yet written — tracked as future `engineering-ontology-v1.md`
|
||||
updates).
|
||||
|
||||
### Cross-layer (memory vs entity vs trusted project state)
|
||||
|
||||
Trusted project state trumps active entities trumps active
|
||||
memories. This is the trust hierarchy from the operating model.
|
||||
|
||||
Cross-layer conflict detection works by a nightly job that walks
|
||||
the three layers and flags any slot that has entries in more than
|
||||
one layer with incompatible values:
|
||||
|
||||
- If trusted project state and an entity disagree: the entity is
|
||||
flagged; trusted state is assumed correct
|
||||
- If an entity and a memory disagree: the memory is flagged; the
|
||||
entity is assumed correct
|
||||
- If trusted state and a memory disagree: the memory is flagged;
|
||||
trusted state is assumed correct
|
||||
|
||||
In all three cases the lower-trust row gets a `conflicts_with`
|
||||
reference pointing at the higher-trust row but does NOT auto-move
|
||||
to superseded. The flag is an alert, not an action.
|
||||
|
||||
## Representation
|
||||
|
||||
Conflicts are represented as rows in a new `conflicts` table
|
||||
(V1 schema, not yet shipped):
|
||||
|
||||
```sql
|
||||
CREATE TABLE conflicts (
|
||||
id TEXT PRIMARY KEY,
|
||||
detected_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
slot_kind TEXT NOT NULL, -- "memory_slot" or "entity_slot" or "cross_layer"
|
||||
slot_key TEXT NOT NULL, -- JSON-encoded tuple identifying the slot
|
||||
project TEXT DEFAULT '',
|
||||
status TEXT NOT NULL DEFAULT 'open', -- open | resolved | dismissed
|
||||
resolved_at DATETIME,
|
||||
resolution TEXT DEFAULT '', -- free text from the reviewer
|
||||
-- links to conflicting rows live in conflict_members
|
||||
UNIQUE(slot_kind, slot_key, status) -- ensures only one open conflict per slot
|
||||
);
|
||||
|
||||
CREATE TABLE conflict_members (
|
||||
conflict_id TEXT NOT NULL REFERENCES conflicts(id) ON DELETE CASCADE,
|
||||
member_kind TEXT NOT NULL, -- "memory" | "entity" | "project_state"
|
||||
member_id TEXT NOT NULL,
|
||||
member_layer_trust INTEGER NOT NULL,-- 1=memory, 2=entity, 3=project_state
|
||||
PRIMARY KEY (conflict_id, member_kind, member_id)
|
||||
);
|
||||
```
|
||||
|
||||
Constraint rationale:
|
||||
|
||||
- `UNIQUE(slot_kind, slot_key, status)` where status='open' prevents
|
||||
duplicate "conflict already open for this slot" rows. At most one
|
||||
open conflict exists per slot at a time; new conflicting rows are
|
||||
added as members to the existing conflict, not as a new conflict.
|
||||
- `conflict_members.member_layer_trust` is denormalized so the
|
||||
conflict resolution UI can sort conflicting rows by trust tier
|
||||
without re-querying.
|
||||
- `status='dismissed'` exists separately from `resolved` because
|
||||
"the reviewer looked at this and declared it not a real conflict"
|
||||
is a valid distinct outcome (the two rows really do describe
|
||||
different things and the detector was overfitting).
|
||||
|
||||
## API shape
|
||||
|
||||
```
|
||||
GET /conflicts list open conflicts
|
||||
GET /conflicts?status=resolved list resolved conflicts
|
||||
GET /conflicts?project=p05-interferometer scope by project
|
||||
GET /conflicts/{id} full detail including all members
|
||||
|
||||
POST /conflicts/{id}/resolve mark resolved with notes
|
||||
body: {
|
||||
"resolution_notes": "...",
|
||||
"winner_member_id": "...", # optional: if specified,
|
||||
# other members are auto-superseded
|
||||
"action": "supersede_others" # or "no_action" if reviewer
|
||||
# wants to resolve without touching rows
|
||||
}
|
||||
|
||||
POST /conflicts/{id}/dismiss mark dismissed ("not a real conflict")
|
||||
body: {
|
||||
"reason": "..."
|
||||
}
|
||||
```
|
||||
|
||||
Conflict detection must also surface in existing endpoints:
|
||||
|
||||
- `GET /memory/{id}` — response includes a `conflicts` array if
|
||||
the memory is a member of any open conflict
|
||||
- `GET /entities/{type}/{id}` (future) — same
|
||||
- `GET /health` — includes `open_conflicts_count` so the operator
|
||||
sees at a glance that review is pending
|
||||
|
||||
## Supersession as a conflict resolution tool
|
||||
|
||||
When the reviewer resolves a conflict with `action: "supersede_others"`,
|
||||
the winner stays active and every other member is flipped to
|
||||
status="superseded" with a `superseded_by` pointer to the winner.
|
||||
This is the normal path: "we used to think X, now we know Y, flag
|
||||
X as superseded so the audit trail keeps X visible but X no longer
|
||||
influences context".
|
||||
|
||||
The conflict resolution audit record links back to all superseded
|
||||
members, so the conflict history itself is queryable:
|
||||
|
||||
- "Show me every conflict that touched Subsystem X"
|
||||
- "Show me every Decision that superseded another Decision because
|
||||
of a conflict"
|
||||
|
||||
These are entries in the V1 query catalog (see Q-014 decision history).
|
||||
|
||||
## Detection latency
|
||||
|
||||
Conflict detection runs at two latencies:
|
||||
|
||||
1. **Synchronous (at write time)** — every create/promote/update of
|
||||
an active row in a conflict-enabled type runs a synchronous
|
||||
same-layer detector. If a conflict is detected the write still
|
||||
succeeds but a row is inserted into `conflicts` and the API
|
||||
response includes a `conflict_id` field so the caller knows
|
||||
immediately.
|
||||
|
||||
2. **Asynchronous (nightly sweep)** — a scheduled job walks all
|
||||
three layers looking for cross-layer conflicts that slipped
|
||||
past write-time detection (e.g. a memory that was already
|
||||
active before an entity with the same slot was promoted). The
|
||||
sweep also looks for slot overlaps that the synchronous
|
||||
detector can't see because the slot key extraction rules have
|
||||
improved since the row was written.
|
||||
|
||||
Both paths write to the same `conflicts` table and both are
|
||||
surfaced in the same review queue.
|
||||
|
||||
## The "flag, never block" rule
|
||||
|
||||
Detection **never** blocks writes. The operating rule is:
|
||||
|
||||
- If the write is otherwise valid (schema, permissions, trust
|
||||
hierarchy), accept it
|
||||
- Log the conflict
|
||||
- Surface it to the reviewer
|
||||
- Let the system keep functioning with the conflict in place
|
||||
|
||||
The alternative — blocking writes on conflict — would mean that
|
||||
one stale fact could prevent all future writes until manually
|
||||
resolved, which in practice makes the system unusable for normal
|
||||
work. The "flag, never block" rule keeps AtoCore responsive while
|
||||
still making conflicts impossible to ignore (the `/health`
|
||||
endpoint's `open_conflicts_count` makes them loud).
|
||||
|
||||
The one exception: writing to `project_state` (layer 3) when an
|
||||
open conflict already exists on that slot will return a warning
|
||||
in the response body. The write still happens, but the reviewer
|
||||
is explicitly told "you just wrote to a slot that has an open
|
||||
conflict". This is the highest-trust layer so we want extra
|
||||
friction there without actually blocking.
|
||||
|
||||
## Showing conflicts in the Human Mirror
|
||||
|
||||
When the Human Mirror template renders a project overview, any
|
||||
open conflict in that project shows as a **"⚠ disputed"** marker
|
||||
next to the affected field, with a link to the conflict detail.
|
||||
This makes conflicts visible to anyone reading the derived
|
||||
human-facing pages, not just to reviewers who think to check the
|
||||
`/conflicts` endpoint.
|
||||
|
||||
The Human Mirror render rules (not yet written — tracked as future
|
||||
`human-mirror-rules.md`) will specify exactly where and how the
|
||||
disputed marker appears.
|
||||
|
||||
## What this document does NOT solve
|
||||
|
||||
1. **Automatic conflict resolution.** No policy will ever
|
||||
automatically promote one conflict member over another. The
|
||||
trust hierarchy is an *alert ordering* for reviewers, not an
|
||||
auto-resolve rule. The human signs off on every resolution.
|
||||
|
||||
2. **Cross-project conflicts.** If p04 and p06 both have
|
||||
entities claiming conflicting things about a shared component,
|
||||
that is currently out of scope because the V1 slot keys all
|
||||
include `project_id`. Cross-project conflict detection is a
|
||||
future concern that needs its own slot key strategy.
|
||||
|
||||
3. **Temporal conflicts with partial overlap.** If a fact was
|
||||
true during a time window and another fact is true in a
|
||||
different time window, that is not a conflict — it's history.
|
||||
Representing time-bounded facts is deferred to a future
|
||||
temporal-entities doc.
|
||||
|
||||
4. **Probabilistic "soft" conflicts.** If two entities claim the
|
||||
same slot with slightly different values (e.g. "4.8 kg" vs
|
||||
"4.82 kg"), is that a conflict? For V1, yes — the string
|
||||
values are unequal so they're flagged. Tolerance-aware
|
||||
numeric comparisons are a V2 concern.
|
||||
|
||||
## TL;DR
|
||||
|
||||
- Conflicts = two or more active rows claiming the same slot with
|
||||
incompatible values
|
||||
- Detection fires on every active write AND in a nightly sweep
|
||||
- Conflicts are stored in a dedicated `conflicts` table with a
|
||||
`conflict_members` join
|
||||
- Resolution is always human (promote-winner / supersede-others
|
||||
/ dismiss-as-not-a-conflict)
|
||||
- "Flag, never block" — writes always succeed, conflicts are
|
||||
surfaced via `/conflicts`, `/health`, per-entity responses, and
|
||||
the Human Mirror
|
||||
- Trusted project state is the top of the trust hierarchy and is
|
||||
assumed correct in any cross-layer conflict until the reviewer
|
||||
says otherwise
|
||||
Reference in New Issue
Block a user