auto: daily sync

war-room-migration-plan.md

@@ -0,0 +1,206 @@
+# War-Room Migration Plan (Prioritized, Code-Level)
+
+Date: 2026-02-20
+Goal: Stabilize entrypoints and create a coherent, testable architecture from the current code reality.
+
+## Phase 0 - Stop The Bleeding (P0, 1-2 days)
+Risk: High if skipped, Low implementation risk
+
+Moves:
+- Fix broken module import paths in runtime code:
+  - `optimization_engine/future/llm_optimization_runner.py`
+  - `optimization_engine/config/setup_wizard.py`
+- Make `optimization_engine/run_optimization.py` importable/executable.
+- Fix plugin directory resolution in all loaders:
+  - `core/runner.py`
+  - `future/llm_optimization_runner.py`
+  - `config/setup_wizard.py`
+
+Interface changes:
+- None to user CLI shape.
+- Internal module paths become explicit (`optimization_engine.future.*` or extracted to a new stable package).
+
+Dependency order:
+1. Fix import paths.
+2. Fix plugin discovery path.
+3. Add smoke tests for `atomizer.py --help` and `optimization_engine/run_optimization.py --help`.
+
+Concrete refactor steps:
+1. Replace missing imports with real module paths:
+   - `optimization_engine.extractor_orchestrator` -> `optimization_engine.future.extractor_orchestrator`
+   - `optimization_engine.inline_code_generator` -> `optimization_engine.future.inline_code_generator`
+   - `optimization_engine.hook_generator` -> `optimization_engine.future.hook_generator`
+2. In hook loaders, resolve plugin root to `optimization_engine/plugins` (shared constant helper).
+3. Add assertion/telemetry when no hooks are loaded to avoid silent no-op behavior.
+
+## Phase 1 - Entrypoint Contract Unification (P0, 2-4 days)
+Risk: Medium
+
+Moves:
+- Define a single canonical optimization runner API used by both:
+  - `atomizer.py` workflows
+  - `optimization_engine/run_optimization.py`
+- Remove dead import (`OptimizationRunner`) from unified runner unless actually used.
+- Implement manual mode or explicitly remove it behind feature flag.
+
+Interface changes:
+- Introduce a stable internal entry function, e.g. `optimization_engine.app.run(spec)`.
+- Keep CLI flags backward-compatible in this phase.
+
+Dependency order:
+1. Stabilize Phase 0.
+2. Create adapter layer from old study scripts to new runner API.
+3. Route `run_optimization.py` through this API.
+
+Concrete refactor steps:
+1. Create `optimization_engine/app/runner_service.py` with:
+   - config parsing
+   - model update/solve/extract loop
+   - hook execution
+2. Convert `run_optimization.py` to thin CLI adapter.
+3. Add compatibility wrapper for existing study scripts.
+
+## Phase 2 - Study Script Decoupling (P1, 1-2 weeks)
+Risk: High (behavioral)
+
+Moves:
+- Stop subprocess dependence on arbitrary `studies/<study>/run_optimization.py` from `atomizer.py`.
+- Replace with deterministic “load study spec + run canonical engine”.
+
+Interface changes:
+- `atomizer neural-optimize --study X` resolves study metadata and calls canonical engine directly.
+- Study-local scripts become optional wrappers, not required runtime dependency.
+
+Dependency order:
+1. Canonical runner available (Phase 1).
+2. Study resolver module.
+3. Migrate one or two representative studies first (UAV + M1 mirror).
+
+Concrete refactor steps:
+1. Implement study resolver for `1_setup/optimization_config.json` and/or `atomizer_spec.json`.
+2. Add `atomizer` path for direct in-process execution.
+3. Keep fallback to legacy script for unmigrated studies with explicit warning.
+
+## Phase 3 - Hooks/Plugins Hardening (P1, 3-5 days)
+Risk: Medium
+
+Moves:
+- Consolidate lifecycle hook system (`plugins`) and clarify that NX Open wrappers (`hooks`) are separate.
+- Fix mismatches:
+  - `custom_objective` hook point vs `custom_objectives` folder naming.
+  - broken plugin registration patterns (e.g. return list without registering).
+
+Interface changes:
+- Standard plugin authoring contract:
+  - `register_hooks(hook_manager) -> None`
+  - must call `hook_manager.register_hook(...)`
+
+Dependency order:
+1. Shared plugin root helper.
+2. Plugin validation lint.
+3. Hook contract docs + tests.
+
+Concrete refactor steps:
+1. Add plugin linter test to fail on non-registered plugins.
+2. Normalize folder naming to enum values.
+3. Add runtime summary printout for loaded/active hooks at each hook point.
+
+## Phase 4 - Extractor Surface Rationalization (P1, 1 week)
+Risk: Medium
+
+Moves:
+- Classify extractors into tiers:
+  - Tier A (active runtime): displacement/stress/mass/frequency basics
+  - Tier B (domain-specific): zernike, optical, special studies
+  - Tier C (candidate deprecate): no references (`field_data_extractor.py`, `zernike_helpers.py`)
+- Move tier C to `archive/` or mark deprecated.
+
+Interface changes:
+- New extractor registry metadata: status (`active`, `experimental`, `deprecated`).
+
+Dependency order:
+1. Build reference report snapshot.
+2. Mark deprecated modules.
+3. Remove only after one release cycle.
+
+Concrete refactor steps:
+1. Add machine-readable registry file for extractors.
+2. Enforce registration for runtime-eligible extractors.
+3. Update callers to consume registry, not free-form imports.
+
+## Phase 5 - Template/Study Generation Repair (P1, 2-4 days)
+Risk: Medium
+
+Moves:
+- Fix template loader roots and generated script imports.
+- Current issues:
+  - templates resolve to `optimization_engine/templates` instead of repo `templates/` JSON set.
+  - studies root points to missing `optimization_engine/studies`.
+  - generated scripts import missing `optimization_engine.study_runner`.
+
+Interface changes:
+- `atomizer create-study` should generate runnable study directly against canonical runner API.
+
+Dependency order:
+1. Canonical runner available (Phase 1).
+2. Replace generated script template.
+3. Add integration test for create+validate flow.
+
+Concrete refactor steps:
+1. Correct template/study root resolution to repo-level paths.
+2. Replace `study_runner` import with canonical service call.
+3. Validate generated study with `atomizer validate` in CI smoke test.
+
+## Phase 6 - Future Folder Re-homing (P2, 1 week)
+Risk: Low/Medium
+
+Moves:
+- Split `optimization_engine/future/` into:
+  - `experimental/` (not production-wired)
+  - `runtime/` (production-wired)
+- Remove deprecation aliases from `optimization_engine/__init__.py` that point to non-runtime pieces unless intentionally supported.
+
+Interface changes:
+- Explicit import policy: production code cannot import `experimental` without feature flag.
+
+Dependency order:
+1. Tag modules by runtime usage.
+2. Move files and add shims.
+3. Remove shims after transition window.
+
+Concrete refactor steps:
+1. Create `optimization_engine/experimental/`.
+2. Move low-usage research modules.
+3. Keep only proven runtime modules in primary namespace.
+
+## Phase 7 - Verification and Guardrails (P0 ongoing)
+Risk: High if omitted
+
+Moves:
+- Add CI checks that would have caught current breakage:
+  - import smoke for both entrypoints
+  - plugin load count assertions
+  - create-study runnable check
+
+Concrete refactor steps:
+1. Add tests:
+   - `test_entrypoint_imports.py`
+   - `test_plugin_discovery_paths.py`
+   - `test_template_loader_paths.py`
+2. Add static scan that fails on unresolved `optimization_engine.<module>` imports.
+3. Add architecture decision record (ADR) for canonical runner + plugin model.
+
+## Dependency-Critical Ordering Summary
+1. Phase 0 (import/plugin path fixes)
+2. Phase 1 (canonical runner contract)
+3. Phase 2 and 5 (atomizer/study generation migration)
+4. Phase 3 (hooks hardening)
+5. Phase 4 and 6 (extractor/future rationalization)
+6. Phase 7 continuously
+
+## Quick Wins (do immediately)
+1. Fix `run_optimization.py` import crash.
+2. Fix all three plugin loader paths.
+3. Remove or wire `OptimizationRunner` import in unified runner.
+4. Fix `template_loader` roots and `study_runner` reference.
+