Atomizer/war-room-migration-plan.md

War-Room Migration Plan (Prioritized, Code-Level)

Date: 2026-02-20 Goal: Stabilize entrypoints and create a coherent, testable architecture from the current code reality.

Phase 0 - Stop The Bleeding (P0, 1-2 days)

Risk: High if skipped, Low implementation risk

Moves:

• Fix broken module import paths in runtime code:
  • optimization_engine/future/llm_optimization_runner.py
  • optimization_engine/config/setup_wizard.py
• Make optimization_engine/run_optimization.py importable/executable.
• Fix plugin directory resolution in all loaders:
  • core/runner.py
  • future/llm_optimization_runner.py
  • config/setup_wizard.py

Interface changes:

• None to user CLI shape.
• Internal module paths become explicit (optimization_engine.future.* or extracted to a new stable package).

Dependency order:

1. Fix import paths.
2. Fix plugin discovery path.
3. Add smoke tests for atomizer.py --help and optimization_engine/run_optimization.py --help.

Concrete refactor steps:

1. Replace missing imports with real module paths:
   • optimization_engine.extractor_orchestrator -> optimization_engine.future.extractor_orchestrator
   • optimization_engine.inline_code_generator -> optimization_engine.future.inline_code_generator
   • optimization_engine.hook_generator -> optimization_engine.future.hook_generator
2. In hook loaders, resolve plugin root to optimization_engine/plugins (shared constant helper).
3. Add assertion/telemetry when no hooks are loaded to avoid silent no-op behavior.

Phase 1 - Entrypoint Contract Unification (P0, 2-4 days)

Risk: Medium

Moves:

• Define a single canonical optimization runner API used by both:
  • atomizer.py workflows
  • optimization_engine/run_optimization.py
• Remove dead import (OptimizationRunner) from unified runner unless actually used.
• Implement manual mode or explicitly remove it behind feature flag.

Interface changes:

• Introduce a stable internal entry function, e.g. optimization_engine.app.run(spec).
• Keep CLI flags backward-compatible in this phase.

Dependency order:

1. Stabilize Phase 0.
2. Create adapter layer from old study scripts to new runner API.
3. Route run_optimization.py through this API.

Concrete refactor steps:

1. Create optimization_engine/app/runner_service.py with:
   • config parsing
   • model update/solve/extract loop
   • hook execution
2. Convert run_optimization.py to thin CLI adapter.
3. Add compatibility wrapper for existing study scripts.

Phase 2 - Study Script Decoupling (P1, 1-2 weeks)

Risk: High (behavioral)

Moves:

• Stop subprocess dependence on arbitrary studies/<study>/run_optimization.py from atomizer.py.
• Replace with deterministic “load study spec + run canonical engine”.

Interface changes:

• atomizer neural-optimize --study X resolves study metadata and calls canonical engine directly.
• Study-local scripts become optional wrappers, not required runtime dependency.

Dependency order:

1. Canonical runner available (Phase 1).
2. Study resolver module.
3. Migrate one or two representative studies first (UAV + M1 mirror).

Concrete refactor steps:

1. Implement study resolver for 1_setup/optimization_config.json and/or atomizer_spec.json.
2. Add atomizer path for direct in-process execution.
3. Keep fallback to legacy script for unmigrated studies with explicit warning.

Phase 3 - Hooks/Plugins Hardening (P1, 3-5 days)

Risk: Medium

Moves:

• Consolidate lifecycle hook system (plugins) and clarify that NX Open wrappers (hooks) are separate.
• Fix mismatches:
  • custom_objective hook point vs custom_objectives folder naming.
  • broken plugin registration patterns (e.g. return list without registering).

Interface changes:

• Standard plugin authoring contract:
  • register_hooks(hook_manager) -> None
  • must call hook_manager.register_hook(...)

Dependency order:

1. Shared plugin root helper.
2. Plugin validation lint.
3. Hook contract docs + tests.

Concrete refactor steps:

1. Add plugin linter test to fail on non-registered plugins.
2. Normalize folder naming to enum values.
3. Add runtime summary printout for loaded/active hooks at each hook point.

Phase 4 - Extractor Surface Rationalization (P1, 1 week)

Risk: Medium

Moves:

• Classify extractors into tiers:
  • Tier A (active runtime): displacement/stress/mass/frequency basics
  • Tier B (domain-specific): zernike, optical, special studies
  • Tier C (candidate deprecate): no references (field_data_extractor.py, zernike_helpers.py)
• Move tier C to archive/ or mark deprecated.

Interface changes:

• New extractor registry metadata: status (active, experimental, deprecated).

Dependency order:

1. Build reference report snapshot.
2. Mark deprecated modules.
3. Remove only after one release cycle.

Concrete refactor steps:

1. Add machine-readable registry file for extractors.
2. Enforce registration for runtime-eligible extractors.
3. Update callers to consume registry, not free-form imports.

Phase 5 - Template/Study Generation Repair (P1, 2-4 days)

Risk: Medium

Moves:

• Fix template loader roots and generated script imports.
• Current issues:
  • templates resolve to optimization_engine/templates instead of repo templates/ JSON set.
  • studies root points to missing optimization_engine/studies.
  • generated scripts import missing optimization_engine.study_runner.

Interface changes:

• atomizer create-study should generate runnable study directly against canonical runner API.

Dependency order:

1. Canonical runner available (Phase 1).
2. Replace generated script template.
3. Add integration test for create+validate flow.

Concrete refactor steps:

1. Correct template/study root resolution to repo-level paths.
2. Replace study_runner import with canonical service call.
3. Validate generated study with atomizer validate in CI smoke test.

Phase 6 - Future Folder Re-homing (P2, 1 week)

Risk: Low/Medium

Moves:

• Split optimization_engine/future/ into:
  • experimental/ (not production-wired)
  • runtime/ (production-wired)
• Remove deprecation aliases from optimization_engine/__init__.py that point to non-runtime pieces unless intentionally supported.

Interface changes:

• Explicit import policy: production code cannot import experimental without feature flag.

Dependency order:

1. Tag modules by runtime usage.
2. Move files and add shims.
3. Remove shims after transition window.

Concrete refactor steps:

1. Create optimization_engine/experimental/.
2. Move low-usage research modules.
3. Keep only proven runtime modules in primary namespace.

Phase 7 - Verification and Guardrails (P0 ongoing)

Risk: High if omitted

Moves:

• Add CI checks that would have caught current breakage:
  • import smoke for both entrypoints
  • plugin load count assertions
  • create-study runnable check

Concrete refactor steps:

1. Add tests:
   • test_entrypoint_imports.py
   • test_plugin_discovery_paths.py
   • test_template_loader_paths.py
2. Add static scan that fails on unresolved optimization_engine.<module> imports.
3. Add architecture decision record (ADR) for canonical runner + plugin model.

Dependency-Critical Ordering Summary

1. Phase 0 (import/plugin path fixes)
2. Phase 1 (canonical runner contract)
3. Phase 2 and 5 (atomizer/study generation migration)
4. Phase 3 (hooks hardening)
5. Phase 4 and 6 (extractor/future rationalization)
6. Phase 7 continuously

Quick Wins (do immediately)

1. Fix run_optimization.py import crash.
2. Fix all three plugin loader paths.
3. Remove or wire OptimizationRunner import in unified runner.
4. Fix template_loader roots and study_runner reference.