Atomizer/war-room-migration-plan.md

# War-Room Migration Plan (Prioritized, Code-Level)

Date: 2026-02-20
Goal: Stabilize entrypoints and create a coherent, testable architecture from the current code reality.

## Phase 0 - Stop The Bleeding (P0, 1-2 days)
Risk: High if skipped, Low implementation risk

Moves:
- Fix broken module import paths in runtime code:
  - `optimization_engine/future/llm_optimization_runner.py`
  - `optimization_engine/config/setup_wizard.py`
- Make `optimization_engine/run_optimization.py` importable/executable.
- Fix plugin directory resolution in all loaders:
  - `core/runner.py`
  - `future/llm_optimization_runner.py`
  - `config/setup_wizard.py`

Interface changes:
- None to user CLI shape.
- Internal module paths become explicit (`optimization_engine.future.*` or extracted to a new stable package).

Dependency order:
1. Fix import paths.
2. Fix plugin discovery path.
3. Add smoke tests for `atomizer.py --help` and `optimization_engine/run_optimization.py --help`.

Concrete refactor steps:
1. Replace missing imports with real module paths:
   - `optimization_engine.extractor_orchestrator` -> `optimization_engine.future.extractor_orchestrator`
   - `optimization_engine.inline_code_generator` -> `optimization_engine.future.inline_code_generator`
   - `optimization_engine.hook_generator` -> `optimization_engine.future.hook_generator`
2. In hook loaders, resolve plugin root to `optimization_engine/plugins` (shared constant helper).
3. Add assertion/telemetry when no hooks are loaded to avoid silent no-op behavior.

## Phase 1 - Entrypoint Contract Unification (P0, 2-4 days)
Risk: Medium

Moves:
- Define a single canonical optimization runner API used by both:
  - `atomizer.py` workflows
  - `optimization_engine/run_optimization.py`
- Remove dead import (`OptimizationRunner`) from unified runner unless actually used.
- Implement manual mode or explicitly remove it behind feature flag.

Interface changes:
- Introduce a stable internal entry function, e.g. `optimization_engine.app.run(spec)`.
- Keep CLI flags backward-compatible in this phase.

Dependency order:
1. Stabilize Phase 0.
2. Create adapter layer from old study scripts to new runner API.
3. Route `run_optimization.py` through this API.

Concrete refactor steps:
1. Create `optimization_engine/app/runner_service.py` with:
   - config parsing
   - model update/solve/extract loop
   - hook execution
2. Convert `run_optimization.py` to thin CLI adapter.
3. Add compatibility wrapper for existing study scripts.

## Phase 2 - Study Script Decoupling (P1, 1-2 weeks)
Risk: High (behavioral)

Moves:
- Stop subprocess dependence on arbitrary `studies/<study>/run_optimization.py` from `atomizer.py`.
- Replace with deterministic “load study spec + run canonical engine”.

Interface changes:
- `atomizer neural-optimize --study X` resolves study metadata and calls canonical engine directly.
- Study-local scripts become optional wrappers, not required runtime dependency.

Dependency order:
1. Canonical runner available (Phase 1).
2. Study resolver module.
3. Migrate one or two representative studies first (UAV + M1 mirror).

Concrete refactor steps:
1. Implement study resolver for `1_setup/optimization_config.json` and/or `atomizer_spec.json`.
2. Add `atomizer` path for direct in-process execution.
3. Keep fallback to legacy script for unmigrated studies with explicit warning.

## Phase 3 - Hooks/Plugins Hardening (P1, 3-5 days)
Risk: Medium

Moves:
- Consolidate lifecycle hook system (`plugins`) and clarify that NX Open wrappers (`hooks`) are separate.
- Fix mismatches:
  - `custom_objective` hook point vs `custom_objectives` folder naming.
  - broken plugin registration patterns (e.g. return list without registering).

Interface changes:
- Standard plugin authoring contract:
  - `register_hooks(hook_manager) -> None`
  - must call `hook_manager.register_hook(...)`

Dependency order:
1. Shared plugin root helper.
2. Plugin validation lint.
3. Hook contract docs + tests.

Concrete refactor steps:
1. Add plugin linter test to fail on non-registered plugins.
2. Normalize folder naming to enum values.
3. Add runtime summary printout for loaded/active hooks at each hook point.

## Phase 4 - Extractor Surface Rationalization (P1, 1 week)
Risk: Medium

Moves:
- Classify extractors into tiers:
  - Tier A (active runtime): displacement/stress/mass/frequency basics
  - Tier B (domain-specific): zernike, optical, special studies
  - Tier C (candidate deprecate): no references (`field_data_extractor.py`, `zernike_helpers.py`)
- Move tier C to `archive/` or mark deprecated.

Interface changes:
- New extractor registry metadata: status (`active`, `experimental`, `deprecated`).

Dependency order:
1. Build reference report snapshot.
2. Mark deprecated modules.
3. Remove only after one release cycle.

Concrete refactor steps:
1. Add machine-readable registry file for extractors.
2. Enforce registration for runtime-eligible extractors.
3. Update callers to consume registry, not free-form imports.

## Phase 5 - Template/Study Generation Repair (P1, 2-4 days)
Risk: Medium

Moves:
- Fix template loader roots and generated script imports.
- Current issues:
  - templates resolve to `optimization_engine/templates` instead of repo `templates/` JSON set.
  - studies root points to missing `optimization_engine/studies`.
  - generated scripts import missing `optimization_engine.study_runner`.

Interface changes:
- `atomizer create-study` should generate runnable study directly against canonical runner API.

Dependency order:
1. Canonical runner available (Phase 1).
2. Replace generated script template.
3. Add integration test for create+validate flow.

Concrete refactor steps:
1. Correct template/study root resolution to repo-level paths.
2. Replace `study_runner` import with canonical service call.
3. Validate generated study with `atomizer validate` in CI smoke test.

## Phase 6 - Future Folder Re-homing (P2, 1 week)
Risk: Low/Medium

Moves:
- Split `optimization_engine/future/` into:
  - `experimental/` (not production-wired)
  - `runtime/` (production-wired)
- Remove deprecation aliases from `optimization_engine/__init__.py` that point to non-runtime pieces unless intentionally supported.

Interface changes:
- Explicit import policy: production code cannot import `experimental` without feature flag.

Dependency order:
1. Tag modules by runtime usage.
2. Move files and add shims.
3. Remove shims after transition window.

Concrete refactor steps:
1. Create `optimization_engine/experimental/`.
2. Move low-usage research modules.
3. Keep only proven runtime modules in primary namespace.

## Phase 7 - Verification and Guardrails (P0 ongoing)
Risk: High if omitted

Moves:
- Add CI checks that would have caught current breakage:
  - import smoke for both entrypoints
  - plugin load count assertions
  - create-study runnable check

Concrete refactor steps:
1. Add tests:
   - `test_entrypoint_imports.py`
   - `test_plugin_discovery_paths.py`
   - `test_template_loader_paths.py`
2. Add static scan that fails on unresolved `optimization_engine.<module>` imports.
3. Add architecture decision record (ADR) for canonical runner + plugin model.

## Dependency-Critical Ordering Summary
1. Phase 0 (import/plugin path fixes)
2. Phase 1 (canonical runner contract)
3. Phase 2 and 5 (atomizer/study generation migration)
4. Phase 3 (hooks hardening)
5. Phase 4 and 6 (extractor/future rationalization)
6. Phase 7 continuously

## Quick Wins (do immediately)
1. Fix `run_optimization.py` import crash.
2. Fix all three plugin loader paths.
3. Remove or wire `OptimizationRunner` import in unified runner.
4. Fix `template_loader` roots and `study_runner` reference.