Commit Graph

7 Commits

Author SHA1 Message Date
dc20033a93 feat: Engineering Knowledge Layer V1 — entities + relationships
Layer 2 of the AtoCore architecture. Adds typed engineering entities
with relationships on top of the flat memory/state/chunk substrate.

Schema:
- entities table: id, entity_type, name, project, description,
  properties (JSON), status, confidence, source_refs, timestamps
- relationships table: source_entity_id, target_entity_id,
  relationship_type, confidence, source_refs

15 entity types: project, system, subsystem, component, interface,
requirement, constraint, decision, material, parameter,
analysis_model, result, validation_claim, vendor, process

12 relationship types: contains, part_of, interfaces_with,
satisfies, constrained_by, affected_by_decision, analyzed_by,
validated_by, depends_on, uses_material, described_by, supersedes

Service layer: full CRUD + get_entity_with_context (returns an
entity with its relationships and all related entities in one call).

API endpoints:
- POST /entities — create entity
- GET /entities — list/filter by type, project, status, name
- GET /entities/{id} — entity + relationships + related entities
- POST /relationships — create relationship

Schema auto-initialized on app startup via init_engineering_schema().

7 tests covering entity CRUD, relationships, context traversal,
filtering, name search, and validation.

Test count: 290 -> 297.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 09:50:58 -04:00
e877e5b8ff deploy: version-visible /health + deploy.sh + update runbook
Dalidou Claude's validation run against the live service exposed a
structural gap: the deployment at /srv/storage/atocore/app has no
git connection, the running container was built from pre-Phase-9
source, and /health hardcoded 'version: 0.1.0' so drift is
invisible. Weeks of work have been shipping to Gitea but never
reaching the live service.

This commit fixes both the drift-invisibility problem and the
absence of an update workflow, so the next deploy to Dalidou can
go live cleanly and future drifts surface immediately.

Layer 1: deployment drift is now visible via /health
----------------------------------------------------
- src/atocore/__init__.py: __version__ bumped from 0.1.0 to 0.2.0
  and documented as the source of truth for the deployed code
  version, with a history block explaining when each bump happens
  (API surface change, schema change, user-visible behavior change)
- src/atocore/main.py: FastAPI constructor now uses __version__
  instead of the hardcoded '0.1.0' string, so the OpenAPI docs
  reflect the actual code version
- src/atocore/api/routes.py: /health now reads from __version__
  dynamically. Both the existing 'version' field and a new
  'code_version' field report the same value for backwards compat.
  A new docstring explains that comparing this to the main
  branch's __version__ is the fastest way to detect drift.
- pyproject.toml: version bumped to 0.2.0 to stay in sync

The comparison is now:
  curl /health -> "code_version": "0.2.0"
  grep __version__ src/atocore/__init__.py -> "0.2.0"
If those differ, the deployment is stale. Concrete, unambiguous.

Layer 2: deploy.sh as the canonical update path
-----------------------------------------------
New file: deploy/dalidou/deploy.sh

One-shot bash script that handles both the first-time deploy
(where /srv/storage/atocore/app may not be a git repo yet) and
the ongoing update case. Steps:

1. If app dir is not a git checkout, back it up as
   <dir>.pre-git-<utc-stamp> and re-clone from Gitea.
   If it IS a checkout, fetch + reset --hard origin/<branch>.
2. Report the deployable commit SHA
3. Check that deploy/dalidou/.env exists (hard fail if missing
   with a clear message pointing at .env.example)
4. docker compose up -d --build — rebuilds the image from
   current source, restarts the container
5. Poll /health for up to 30 seconds; on failure, print the
   last 50 lines of container logs and exit non-zero
6. Parse /health.code_version and compare to the __version__
   in the freshly-pulled source. If they differ, exit non-zero
   with a message suggesting docker compose down && up
7. On success, report commit + code_version + "health: ok"

Configurable via env vars:
- ATOCORE_APP_DIR (default /srv/storage/atocore/app)
- ATOCORE_GIT_REMOTE (default http://dalidou:3000/Antoine/ATOCore.git)
- ATOCORE_BRANCH (default main)
- ATOCORE_HEALTH_URL (default http://127.0.0.1:8100/health)
- ATOCORE_DEPLOY_DRY_RUN=1 for preview-only mode

Explicit non-goals documented in the script header:
- does not manage secrets (.env is the caller's responsibility)
- does not take a pre-deploy backup (call /admin/backup first
  if you want one)
- does not roll back on failure (redeploy a known-good commit
  to recover)
- does not touch the DB directly — schema migrations run at
  service startup via the lifespan handler, and all existing
  _apply_migrations ALTERs are idempotent ADD COLUMN operations

Layer 3: updated docs/dalidou-deployment.md
-------------------------------------------
- First-time deployment steps now explicitly say "git clone", not
  "place the repository", so future first-time deploys don't end
  up as static snapshots again
- New "Updating a running deployment" section covering deploy.sh
  usage with all three modes (normal / branch override / dry-run)
- New "Deployment drift detection" section with the one-liner
  comparison between /health code_version and the repo's
  __version__
- New "Schema migrations on redeploy" section enumerating the
  exact ALTER TABLE statements that run on a pre-0.2.0 -> 0.2.0
  upgrade, confirming they are additive-only and safe, and
  recommending a backup via /admin/backup before any redeploy

Full suite: 215 passing, 1 warning. No test was hardcoded to the
old version string, so the version bump was safe without test
changes.

What this commit does NOT do
----------------------------
- Does NOT execute the deploy on the live Dalidou instance. That
  requires Dalidou access and is the next step. A ready-to-paste
  prompt for Dalidou Claude will be provided separately.
- Does NOT add CI/CD, webhook-based auto-deploy, or reverse
  proxy. Those remain in the 'deferred' section of the
  deployment doc.
- Does NOT change the Dockerfile. The existing 'COPY source at
  build time' pattern is what deploy.sh relies on — rebuilding
  the image picks up new code.
- Does NOT modify the database schema. The Phase 9 migrations
  that Dalidou's DB needs will be applied automatically on next
  service startup via the existing _apply_migrations path.
2026-04-08 18:08:49 -04:00
b9da5b6d84 phase9 first-real-use validation + small hygiene wins
Session 1 of the four-session plan. Empirically exercises the Phase 9
loop (capture -> reinforce -> extract) for the first time and lands
three small hygiene fixes.

Validation script + report
--------------------------
scripts/phase9_first_real_use.py — reproducible script that:
  - sets up an isolated SQLite + Chroma store under
    data/validation/phase9-first-use (gitignored)
  - seeds 3 active memories
  - runs 8 sample interactions through capture + reinforce + extract
  - prints what each step produced and reinforcement state at the end
  - supports --json output for downstream tooling

docs/phase9-first-real-use.md — narrative report of the run with:
  - extraction results table (8/8 expectations met exactly)
  - the empirical finding that REINFORCEMENT MATCHED ZERO seeds
    despite sample 5 clearly echoing the rebase preference memory
  - root cause analysis: the substring matcher is too brittle for
    natural paraphrases (e.g. "prefers" vs "I prefer", "history"
    vs "the history")
  - recommended fix: replace substring matcher with a token-overlap
    matcher (>=70% of memory tokens present in response, with
    light stemming and a small stop list)
  - explicit note that the fix is queued as a follow-up commit, not
    bundled into the report — keeps the audit trail clean

Key extraction results from the run:
  - all 7 heading/sentence rules fired correctly
  - 0 false positives on the prose-only sample (the most important
    sanity check)
  - long content preserved without truncation
  - dedup correctly kept three distinct cues from one interaction
  - project scoping flowed cleanly through the pipeline

Hygiene 1: FastAPI lifespan migration (src/atocore/main.py)
- Replaced @app.on_event("startup") with the modern @asynccontextmanager
  lifespan handler
- Same setup work (setup_logging, ensure_runtime_dirs, init_db,
  init_project_state_schema, startup_ready log)
- Removes the two on_event deprecation warnings from every test run
- Test suite now shows 1 warning instead of 3

Hygiene 2: EXTRACTOR_VERSION constant (src/atocore/memory/extractor.py)
- Added EXTRACTOR_VERSION = "0.1.0" with a versioned change log comment
- MemoryCandidate dataclass carries extractor_version on every candidate
- POST /interactions/{id}/extract response now includes extractor_version
  on both the top level (current run) and on each candidate
- Implements the versioning requirement called out in
  docs/architecture/promotion-rules.md so old candidates can be
  identified and re-evaluated when the rule set evolves

Hygiene 3: ~/.git-credentials cleanup (out-of-tree, not committed)
- Removed the dead OAUTH_USER:<jwt> line for dalidou:3000 that was
  being silently rewritten by the system credential manager on every
  push attempt
- Configured credential.http://dalidou:3000.helper with the empty-string
  sentinel pattern so the URL-specific helper chain is exactly
  ["", store] instead of inheriting the system-level "manager" helper
  that ships with Git for Windows
- Same fix for the 100.80.199.40 (Tailscale) entry
- Verified end to end: a fresh push using only the cleaned credentials
  file (no embedded URL) authenticates as Antoine and lands cleanly

Full suite: 160 passing (no change from previous), 1 warning
(was 3) thanks to the lifespan migration.
2026-04-07 06:16:35 -04:00
6bfa1fcc37 Add Dalidou storage foundation and deployment prep 2026-04-05 18:33:52 -04:00
b0889b3925 Stabilize core correctness and sync project plan state 2026-04-05 17:53:23 -04:00
531c560db7 feat: Phase 1 ingestion hardening + Phase 5 Trusted Project State
Phase 1 - Ingestion hardening:
- Encoding fallback (UTF-8/UTF-8-sig/Latin-1/CP1252)
- Delete detection: purge DB/vector entries for removed files
- Ingestion stats endpoint (GET /stats)

Phase 5 - Trusted Project State:
- project_state table with categories (status, decision, requirement, contact, milestone, fact, config)
- CRUD API: POST/GET/DELETE /project/state
- Upsert semantics, invalidation (supersede) support
- Context builder integrates project state at highest trust precedence
- Project state gets 20% budget allocation, appears first in context
- Trust precedence: Project State > Retrieved Chunks (per Master Plan)

33/33 tests passing. Validated end-to-end with GigaBIT M1 project data.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-05 09:41:59 -04:00
b4afbbb53a feat: implement AtoCore Phase 0 + Phase 0.5 (foundation + PoC)
Complete implementation of the personal context engine foundation:
- FastAPI server with 5 endpoints (ingest, query, context/build, health, debug)
- SQLite database with 5 tables (documents, chunks, memories, projects, interactions)
- Heading-aware markdown chunker (800 char max, recursive splitting)
- Multilingual embeddings via sentence-transformers (EN/FR)
- ChromaDB vector store with cosine similarity retrieval
- Context builder with project boosting, dedup, and budget enforcement
- CLI scripts for batch ingestion and test prompt evaluation
- 19 unit tests passing, 79% coverage
- Validated on 482 real project files (8383 chunks, 0 errors)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-05 09:21:27 -04:00