Three planning docs that answer the architectural questions the engineering query catalog raised. Together with the catalog they form roughly half of the pre-implementation planning sprint. docs/architecture/memory-vs-entities.md --------------------------------------- Resolves the central question blocking every other engineering layer doc: is a Decision a memory or an entity? Key decisions: - memories stay the canonical home for identity, preference, and episodic facts - entities become the canonical home for project, knowledge, and adaptation facts once the engineering layer V1 ships - no concept lives in both layers at full fidelity; one canonical home per concept - a "graduation" flow lets active memories upgrade into entities (memory stays as a frozen historical pointer, never deleted) - one shared candidate review queue across both layers - context builder budget gains a 15% slot for engineering entities, slotted between identity/preference memories and retrieved chunks - the Phase 9 memory extractor's structural cues (decision heading, constraint heading, requirement heading) are explicitly an intentional temporary overlap, cleanly migrated via graduation when the entity extractor ships docs/architecture/promotion-rules.md ------------------------------------ Defines the full Layer 0 → Layer 2 pipeline: - four layers: L0 raw source, L1 memory candidate/active, L2 entity candidate/active, L3 trusted project state - three extraction triggers: on interaction capture (existing), on ingestion wave (new, batched per wave), on explicit request - per-rule prior confidence tuned at write time by structural signal (echoes the retriever's high/low signal hints) and freshness bonus - batch cap of 50 candidates per pass to protect the reviewer - full provenance requirements: every candidate carries rule id, source_chunk_id, source_interaction_id, and extractor_version - reversibility matrix for every promotion step - explicit no-auto-promotion-in-V1 stance with the schema designed so auto-promotion policies can be added later without migration - the hard invariant: nothing ever moves into L3 automatically - ingestion-wave extraction produces a report artifact under data/extraction-reports/<wave-id>/ docs/architecture/conflict-model.md ----------------------------------- Defines how AtoCore handles contradictory facts without violating the "bad memory is worse than no memory" rule. - conflict = two or more active rows claiming the same slot with incompatible values - per-type "slot key" tuples for both memory and entity types - cross-layer conflict detection respects the trust hierarchy: trusted project state > active entities > active memories - new conflicts and conflict_members tables (schema proposal) - detection at two latencies: synchronous at write time, asynchronous nightly sweep - "flag, never block" rule: writes always succeed, conflicts are surfaced via /conflicts, /health open_conflicts_count, per-row response bodies, and the Human Mirror's disputed marker - resolution is always human: promote-winner + supersede-others, or dismiss-as-not-a-real-conflict, both with audit trail - explicitly out of scope for V1: cross-project conflicts, temporal-overlap conflicts, tolerance-aware numeric comparisons Also updates: - master-plan-status.md: Phase 9 moved from "started" to "baseline complete" now that Commits A, B, C are all landed - master-plan-status.md: adds a "Engineering Layer Planning Sprint" section listing the doc wave so far and the remaining docs (tool-handoff-boundaries, human-mirror-rules, representation-authority, engineering-v1-acceptance) - current-state.md: Phase 9 moved from "not started" to "baseline complete" with the A/B/C annotation This is pure doc work. No code changes, no schema changes, no behavior changes. Per the working rule in master-plan-status.md: the architecture docs shape decisions, they do not force premature schema work.
333 lines
14 KiB
Markdown
333 lines
14 KiB
Markdown
# Conflict Model (how AtoCore handles contradictory facts)
|
|
|
|
## Why this document exists
|
|
|
|
Any system that accumulates facts from multiple sources — interactions,
|
|
ingested documents, repo history, PKM notes — will eventually see
|
|
contradictory facts about the same thing. AtoCore's operating model
|
|
already has the hard rule:
|
|
|
|
> **Bad memory is worse than no memory.**
|
|
|
|
The practical consequence of that rule is: AtoCore must never
|
|
silently merge contradictory facts, never silently pick a winner,
|
|
and never silently discard evidence. Every conflict must be
|
|
surfaced to a human reviewer with full audit context.
|
|
|
|
This document defines what "conflict" means in AtoCore, how
|
|
conflicts are detected, how they are represented, how they are
|
|
surfaced, and how they are resolved.
|
|
|
|
## What counts as a conflict
|
|
|
|
A conflict exists when two or more facts in the system claim
|
|
incompatible values for the same conceptual slot. More precisely:
|
|
|
|
A conflict is a set of two or more **active** rows (across memories,
|
|
entities, project_state) such that:
|
|
|
|
1. They share the same **target identity** — same entity type and
|
|
same semantic key
|
|
2. Their **claimed values** are incompatible
|
|
3. They are all in an **active** status (not superseded, not
|
|
invalid, not candidate)
|
|
|
|
Examples that are conflicts:
|
|
|
|
- Two active `Decision` entities affecting the same `Subsystem`
|
|
with contradictory values for the same decided field (e.g.
|
|
lateral support material = GF-PTFE vs lateral support material = PEEK)
|
|
- An active `preference` memory "prefers rebase workflow" and an
|
|
active `preference` memory "prefers merge-commit workflow"
|
|
- A `project_state` entry `p05 / decision / lateral_support_material = GF-PTFE`
|
|
and an active `Decision` entity also claiming the lateral support
|
|
material is PEEK (cross-layer conflict)
|
|
|
|
Examples that are NOT conflicts:
|
|
|
|
- Two active memories both saying "prefers small diffs" — same
|
|
meaning, not contradictory
|
|
- An active memory saying X and a candidate memory saying Y —
|
|
candidates are not active, so this is part of the review queue
|
|
flow, not the conflict flow
|
|
- A superseded `Decision` saying X and an active `Decision` saying Y
|
|
— supersession is a resolved history, not a conflict
|
|
- Two active `Requirement` entities each constraining the same
|
|
component in different but compatible ways (e.g. one caps mass,
|
|
one caps heat flux) — different fields, no contradiction
|
|
|
|
## Detection triggers
|
|
|
|
Conflict detection must fire at every write that could create a new
|
|
active fact. That means the following hook points:
|
|
|
|
1. **`POST /memory` creating an active memory** (legacy path)
|
|
2. **`POST /memory/{id}/promote`** (candidate → active)
|
|
3. **`POST /entities` creating an active entity** (future)
|
|
4. **`POST /entities/{id}/promote`** (candidate → active, future)
|
|
5. **`POST /project/state`** (curating trusted state directly)
|
|
6. **`POST /memory/{id}/graduate`** (memory → entity graduation,
|
|
future — the resulting entity could conflict with something)
|
|
|
|
Extraction passes do NOT trigger conflict detection at candidate
|
|
write time. Candidates are allowed to sit in the queue in an
|
|
apparently-conflicting state; the reviewer will see them during
|
|
promotion and decision-detection fires at that moment.
|
|
|
|
## Detection strategy per layer
|
|
|
|
### Memory layer
|
|
|
|
For identity / preference / episodic memories (the ones that stay
|
|
in the memory layer):
|
|
|
|
- Matching key: `(memory_type, project, normalized_content_family)`
|
|
- `normalized_content_family` is not a hash of the content — that
|
|
would require exact equality — but a slot identifier extracted
|
|
by a small per-type rule set:
|
|
- identity: slot is "role" / "background" / "credentials"
|
|
- preference: slot is the first content word after "prefers" / "uses" / "likes"
|
|
normalized to a lowercase noun stem, OR the rule id that extracted it
|
|
- episodic: no slot — episodic entries are intrinsically tied to
|
|
a moment in time and rarely conflict
|
|
|
|
A conflict is flagged when two active memories share a
|
|
`(memory_type, project, slot)` but have different content bodies.
|
|
|
|
### Entity layer (V1)
|
|
|
|
For each V1 entity type, the conflict key is a short tuple that
|
|
uniquely identifies the "slot" that entity is claiming:
|
|
|
|
| Entity type | Conflict slot |
|
|
|-------------------|-------------------------------------------------------|
|
|
| Project | `(project_id)` |
|
|
| Subsystem | `(project_id, subsystem_name)` |
|
|
| Component | `(project_id, subsystem_name, component_name)` |
|
|
| Requirement | `(project_id, requirement_key)` |
|
|
| Constraint | `(project_id, constraint_target, constraint_kind)` |
|
|
| Decision | `(project_id, decision_target, decision_field)` |
|
|
| Material | `(project_id, component_id)` |
|
|
| Parameter | `(project_id, parameter_scope, parameter_name)` |
|
|
| AnalysisModel | `(project_id, subsystem_id, model_name)` |
|
|
| Result | `(project_id, analysis_model_id, result_key)` |
|
|
| ValidationClaim | `(project_id, claim_key)` |
|
|
| Artifact | no conflict detection — artifacts are additive |
|
|
|
|
A conflict is two active entities with the same slot but
|
|
different structural values. The exact "which fields count as
|
|
structural" list is per-type and lives in the entity schema doc
|
|
(not yet written — tracked as future `engineering-ontology-v1.md`
|
|
updates).
|
|
|
|
### Cross-layer (memory vs entity vs trusted project state)
|
|
|
|
Trusted project state trumps active entities trumps active
|
|
memories. This is the trust hierarchy from the operating model.
|
|
|
|
Cross-layer conflict detection works by a nightly job that walks
|
|
the three layers and flags any slot that has entries in more than
|
|
one layer with incompatible values:
|
|
|
|
- If trusted project state and an entity disagree: the entity is
|
|
flagged; trusted state is assumed correct
|
|
- If an entity and a memory disagree: the memory is flagged; the
|
|
entity is assumed correct
|
|
- If trusted state and a memory disagree: the memory is flagged;
|
|
trusted state is assumed correct
|
|
|
|
In all three cases the lower-trust row gets a `conflicts_with`
|
|
reference pointing at the higher-trust row but does NOT auto-move
|
|
to superseded. The flag is an alert, not an action.
|
|
|
|
## Representation
|
|
|
|
Conflicts are represented as rows in a new `conflicts` table
|
|
(V1 schema, not yet shipped):
|
|
|
|
```sql
|
|
CREATE TABLE conflicts (
|
|
id TEXT PRIMARY KEY,
|
|
detected_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
slot_kind TEXT NOT NULL, -- "memory_slot" or "entity_slot" or "cross_layer"
|
|
slot_key TEXT NOT NULL, -- JSON-encoded tuple identifying the slot
|
|
project TEXT DEFAULT '',
|
|
status TEXT NOT NULL DEFAULT 'open', -- open | resolved | dismissed
|
|
resolved_at DATETIME,
|
|
resolution TEXT DEFAULT '', -- free text from the reviewer
|
|
-- links to conflicting rows live in conflict_members
|
|
UNIQUE(slot_kind, slot_key, status) -- ensures only one open conflict per slot
|
|
);
|
|
|
|
CREATE TABLE conflict_members (
|
|
conflict_id TEXT NOT NULL REFERENCES conflicts(id) ON DELETE CASCADE,
|
|
member_kind TEXT NOT NULL, -- "memory" | "entity" | "project_state"
|
|
member_id TEXT NOT NULL,
|
|
member_layer_trust INTEGER NOT NULL,-- 1=memory, 2=entity, 3=project_state
|
|
PRIMARY KEY (conflict_id, member_kind, member_id)
|
|
);
|
|
```
|
|
|
|
Constraint rationale:
|
|
|
|
- `UNIQUE(slot_kind, slot_key, status)` where status='open' prevents
|
|
duplicate "conflict already open for this slot" rows. At most one
|
|
open conflict exists per slot at a time; new conflicting rows are
|
|
added as members to the existing conflict, not as a new conflict.
|
|
- `conflict_members.member_layer_trust` is denormalized so the
|
|
conflict resolution UI can sort conflicting rows by trust tier
|
|
without re-querying.
|
|
- `status='dismissed'` exists separately from `resolved` because
|
|
"the reviewer looked at this and declared it not a real conflict"
|
|
is a valid distinct outcome (the two rows really do describe
|
|
different things and the detector was overfitting).
|
|
|
|
## API shape
|
|
|
|
```
|
|
GET /conflicts list open conflicts
|
|
GET /conflicts?status=resolved list resolved conflicts
|
|
GET /conflicts?project=p05-interferometer scope by project
|
|
GET /conflicts/{id} full detail including all members
|
|
|
|
POST /conflicts/{id}/resolve mark resolved with notes
|
|
body: {
|
|
"resolution_notes": "...",
|
|
"winner_member_id": "...", # optional: if specified,
|
|
# other members are auto-superseded
|
|
"action": "supersede_others" # or "no_action" if reviewer
|
|
# wants to resolve without touching rows
|
|
}
|
|
|
|
POST /conflicts/{id}/dismiss mark dismissed ("not a real conflict")
|
|
body: {
|
|
"reason": "..."
|
|
}
|
|
```
|
|
|
|
Conflict detection must also surface in existing endpoints:
|
|
|
|
- `GET /memory/{id}` — response includes a `conflicts` array if
|
|
the memory is a member of any open conflict
|
|
- `GET /entities/{type}/{id}` (future) — same
|
|
- `GET /health` — includes `open_conflicts_count` so the operator
|
|
sees at a glance that review is pending
|
|
|
|
## Supersession as a conflict resolution tool
|
|
|
|
When the reviewer resolves a conflict with `action: "supersede_others"`,
|
|
the winner stays active and every other member is flipped to
|
|
status="superseded" with a `superseded_by` pointer to the winner.
|
|
This is the normal path: "we used to think X, now we know Y, flag
|
|
X as superseded so the audit trail keeps X visible but X no longer
|
|
influences context".
|
|
|
|
The conflict resolution audit record links back to all superseded
|
|
members, so the conflict history itself is queryable:
|
|
|
|
- "Show me every conflict that touched Subsystem X"
|
|
- "Show me every Decision that superseded another Decision because
|
|
of a conflict"
|
|
|
|
These are entries in the V1 query catalog (see Q-014 decision history).
|
|
|
|
## Detection latency
|
|
|
|
Conflict detection runs at two latencies:
|
|
|
|
1. **Synchronous (at write time)** — every create/promote/update of
|
|
an active row in a conflict-enabled type runs a synchronous
|
|
same-layer detector. If a conflict is detected the write still
|
|
succeeds but a row is inserted into `conflicts` and the API
|
|
response includes a `conflict_id` field so the caller knows
|
|
immediately.
|
|
|
|
2. **Asynchronous (nightly sweep)** — a scheduled job walks all
|
|
three layers looking for cross-layer conflicts that slipped
|
|
past write-time detection (e.g. a memory that was already
|
|
active before an entity with the same slot was promoted). The
|
|
sweep also looks for slot overlaps that the synchronous
|
|
detector can't see because the slot key extraction rules have
|
|
improved since the row was written.
|
|
|
|
Both paths write to the same `conflicts` table and both are
|
|
surfaced in the same review queue.
|
|
|
|
## The "flag, never block" rule
|
|
|
|
Detection **never** blocks writes. The operating rule is:
|
|
|
|
- If the write is otherwise valid (schema, permissions, trust
|
|
hierarchy), accept it
|
|
- Log the conflict
|
|
- Surface it to the reviewer
|
|
- Let the system keep functioning with the conflict in place
|
|
|
|
The alternative — blocking writes on conflict — would mean that
|
|
one stale fact could prevent all future writes until manually
|
|
resolved, which in practice makes the system unusable for normal
|
|
work. The "flag, never block" rule keeps AtoCore responsive while
|
|
still making conflicts impossible to ignore (the `/health`
|
|
endpoint's `open_conflicts_count` makes them loud).
|
|
|
|
The one exception: writing to `project_state` (layer 3) when an
|
|
open conflict already exists on that slot will return a warning
|
|
in the response body. The write still happens, but the reviewer
|
|
is explicitly told "you just wrote to a slot that has an open
|
|
conflict". This is the highest-trust layer so we want extra
|
|
friction there without actually blocking.
|
|
|
|
## Showing conflicts in the Human Mirror
|
|
|
|
When the Human Mirror template renders a project overview, any
|
|
open conflict in that project shows as a **"⚠ disputed"** marker
|
|
next to the affected field, with a link to the conflict detail.
|
|
This makes conflicts visible to anyone reading the derived
|
|
human-facing pages, not just to reviewers who think to check the
|
|
`/conflicts` endpoint.
|
|
|
|
The Human Mirror render rules (not yet written — tracked as future
|
|
`human-mirror-rules.md`) will specify exactly where and how the
|
|
disputed marker appears.
|
|
|
|
## What this document does NOT solve
|
|
|
|
1. **Automatic conflict resolution.** No policy will ever
|
|
automatically promote one conflict member over another. The
|
|
trust hierarchy is an *alert ordering* for reviewers, not an
|
|
auto-resolve rule. The human signs off on every resolution.
|
|
|
|
2. **Cross-project conflicts.** If p04 and p06 both have
|
|
entities claiming conflicting things about a shared component,
|
|
that is currently out of scope because the V1 slot keys all
|
|
include `project_id`. Cross-project conflict detection is a
|
|
future concern that needs its own slot key strategy.
|
|
|
|
3. **Temporal conflicts with partial overlap.** If a fact was
|
|
true during a time window and another fact is true in a
|
|
different time window, that is not a conflict — it's history.
|
|
Representing time-bounded facts is deferred to a future
|
|
temporal-entities doc.
|
|
|
|
4. **Probabilistic "soft" conflicts.** If two entities claim the
|
|
same slot with slightly different values (e.g. "4.8 kg" vs
|
|
"4.82 kg"), is that a conflict? For V1, yes — the string
|
|
values are unequal so they're flagged. Tolerance-aware
|
|
numeric comparisons are a V2 concern.
|
|
|
|
## TL;DR
|
|
|
|
- Conflicts = two or more active rows claiming the same slot with
|
|
incompatible values
|
|
- Detection fires on every active write AND in a nightly sweep
|
|
- Conflicts are stored in a dedicated `conflicts` table with a
|
|
`conflict_members` join
|
|
- Resolution is always human (promote-winner / supersede-others
|
|
/ dismiss-as-not-a-conflict)
|
|
- "Flag, never block" — writes always succeed, conflicts are
|
|
surfaced via `/conflicts`, `/health`, per-entity responses, and
|
|
the Human Mirror
|
|
- Trusted project state is the top of the trust hierarchy and is
|
|
assumed correct in any cross-layer conflict until the reviewer
|
|
says otherwise
|