Atomizer/DEVELOPMENT.md

# Atomizer Development Guide

**Last Updated**: 2025-11-21
**Current Phase**: Phase 3.2 - Integration Sprint + Documentation
**Status**: 🟢 Core Complete (100%) | ✅ Protocols 10/11/13 Active (100%) | 🎯 Dashboard Live (95%) | 📚 Documentation Reorganized

📘 **Quick Links**:
- [Protocol Specifications](docs/PROTOCOLS.md) - All active protocols consolidated
- [Documentation Index](docs/00_INDEX.md) - Complete documentation navigation
- [README](README.md) - Project overview and quick start

---

## Table of Contents

1. [Current Phase](#current-phase)
2. [Completed Features](#completed-features)
3. [Active Development](#active-development)
4. [Known Issues](#known-issues)
5. [Testing Status](#testing-status)
6. [Phase-by-Phase Progress](#phase-by-phase-progress)

---

## Current Phase

### Phase 3.2: Integration Sprint (🎯 TOP PRIORITY)

**Goal**: Connect LLM intelligence components to production workflow

**Timeline**: 2-4 weeks (Started 2025-11-17)

**Status**: LLM components built and tested individually (85% complete). Need to wire them into production runner.

📋 **Detailed Plan**: [docs/PHASE_3_2_INTEGRATION_PLAN.md](docs/PHASE_3_2_INTEGRATION_PLAN.md)

**Critical Path**:

#### Week 1: Make LLM Mode Accessible (16 hours)
- [ ] **1.1** Create unified entry point `optimization_engine/run_optimization.py` (4h)
  - Add `--llm` flag for natural language mode
  - Add `--request` parameter for natural language input
  - Support both LLM and traditional JSON modes
  - Preserve backward compatibility

- [ ] **1.2** Wire LLMOptimizationRunner to production (8h)
  - Connect LLMWorkflowAnalyzer to entry point
  - Bridge LLMOptimizationRunner → OptimizationRunner
  - Pass model updater and simulation runner callables
  - Integrate with existing hook system

- [ ] **1.3** Create minimal example (2h)
  - Create `examples/llm_mode_demo.py`
  - Show natural language → optimization results
  - Compare traditional (100 lines) vs LLM (3 lines)

- [ ] **1.4** End-to-end integration test (2h)
  - Test with simple_beam_optimization study
  - Verify extractors generated correctly
  - Validate output matches manual mode

#### Week 2: Robustness & Safety (16 hours)
- [ ] **2.1** Code validation pipeline (6h)
  - Create `optimization_engine/code_validator.py`
  - Implement syntax validation (ast.parse)
  - Implement security scanning (whitelist imports)
  - Implement test execution on example OP2
  - Add retry with LLM feedback on failure

- [ ] **2.2** Graceful fallback mechanisms (4h)
  - Wrap all LLM calls in try/except
  - Provide clear error messages
  - Offer fallback to manual mode
  - Never crash on LLM failure

- [ ] **2.3** LLM audit trail (3h)
  - Create `optimization_engine/llm_audit.py`
  - Log all LLM requests and responses
  - Log generated code with prompts
  - Create `llm_audit.json` in study output

- [ ] **2.4** Failure scenario testing (3h)
  - Test invalid natural language request
  - Test LLM unavailable
  - Test generated code syntax errors
  - Test validation failures

#### Week 3: Learning System (12 hours)
- [ ] **3.1** Knowledge base implementation (4h)
  - Create `optimization_engine/knowledge_base.py`
  - Implement `save_session()` - Save successful workflows
  - Implement `search_templates()` - Find similar patterns
  - Add confidence scoring

- [ ] **3.2** Template extraction (4h)
  - Extract reusable patterns from generated code
  - Parameterize variable parts
  - Save templates with usage examples
  - Implement template application to new requests

- [ ] **3.3** ResearchAgent integration (4h)
  - Complete ResearchAgent implementation
  - Integrate into ExtractorOrchestrator error handling
  - Add user example collection workflow
  - Save learned knowledge to knowledge base

#### Week 4: Documentation & Discoverability (8 hours)
- [ ] **4.1** Update README (2h)
  - Add "🤖 LLM-Powered Mode" section
  - Show example command with natural language
  - Link to detailed docs

- [ ] **4.2** Create LLM mode documentation (3h)
  - Create `docs/LLM_MODE.md`
  - Explain how LLM mode works
  - Provide usage examples
  - Add troubleshooting guide

- [ ] **4.3** Create demo video/GIF (1h)
  - Record terminal session
  - Show before/after (100 lines → 3 lines)
  - Create animated GIF for README

- [ ] **4.4** Update all planning docs (2h)
  - Update DEVELOPMENT.md status
  - Update DEVELOPMENT_GUIDANCE.md (80-90% → 90-95%)
  - Mark Phase 3.2 as ✅ Complete

---

## Completed Features

### ✅ Live Dashboard System (Completed 2025-11-21)

#### Backend (FastAPI + WebSocket)
- [x] **FastAPI Backend** ([atomizer-dashboard/backend/](atomizer-dashboard/backend/))
  - REST API endpoints for study management
  - WebSocket streaming with file watching (Watchdog)
  - Real-time updates (<100ms latency)
  - CORS configured for local development

- [x] **REST API Endpoints** ([backend/api/routes/optimization.py](atomizer-dashboard/backend/api/routes/optimization.py))
  - `GET /api/optimization/studies` - List all studies
  - `GET /api/optimization/studies/{id}/status` - Get study status
  - `GET /api/optimization/studies/{id}/history` - Get trial history
  - `GET /api/optimization/studies/{id}/pruning` - Get pruning diagnostics

- [x] **WebSocket Streaming** ([backend/api/websocket/optimization_stream.py](atomizer-dashboard/backend/api/websocket/optimization_stream.py))
  - File watching on `optimization_history_incremental.json`
  - Real-time trial updates via WebSocket
  - Pruning alerts and progress updates
  - Automatic observer lifecycle management

#### Frontend (HTML + Chart.js)
- [x] **Enhanced Live Dashboard** ([atomizer-dashboard/dashboard-enhanced.html](atomizer-dashboard/dashboard-enhanced.html))
  - Real-time WebSocket updates
  - Interactive convergence chart (Chart.js)
  - Parameter space scatter plot
  - Pruning alerts (toast notifications)
  - Data export (JSON/CSV)
  - Study auto-discovery and selection
  - Metric dashboard (trials, best value, pruned count)

#### React Frontend (In Progress)
- [x] **Project Configuration** ([atomizer-dashboard/frontend/](atomizer-dashboard/frontend/))
  - React 18 + Vite 5 + TypeScript 5.2
  - TailwindCSS 3.3 for styling
  - Recharts 2.10 for charts
  - Complete build configuration

- [x] **TypeScript Types** ([frontend/src/types/](atomizer-dashboard/frontend/src/types/))
  - Complete type definitions for API data
  - WebSocket message types
  - Chart data structures

- [x] **Custom Hooks** ([frontend/src/hooks/useWebSocket.ts](atomizer-dashboard/frontend/src/hooks/useWebSocket.ts))
  - WebSocket connection management
  - Auto-reconnection with exponential backoff
  - Type-safe message routing

- [x] **Reusable Components** ([frontend/src/components/](atomizer-dashboard/frontend/src/components/))
  - Card, MetricCard, Badge, StudyCard components
  - TailwindCSS styling with dark theme

- [ ] **Dashboard Page** (Pending manual completion)
  - Need to run `npm install`
  - Create main.tsx, App.tsx, Dashboard.tsx
  - Integrate Recharts for charts
  - Test end-to-end

#### Documentation
- [x] **Dashboard Master Plan** ([docs/DASHBOARD_MASTER_PLAN.md](docs/DASHBOARD_MASTER_PLAN.md))
  - Complete 3-page architecture (Configurator, Live Dashboard, Results Viewer)
  - Tech stack recommendations
  - Implementation phases

- [x] **Implementation Status** ([docs/DASHBOARD_IMPLEMENTATION_STATUS.md](docs/DASHBOARD_IMPLEMENTATION_STATUS.md))
  - Current progress tracking
  - Testing instructions
  - Next steps

- [x] **React Implementation Guide** ([docs/DASHBOARD_REACT_IMPLEMENTATION.md](docs/DASHBOARD_REACT_IMPLEMENTATION.md))
  - Complete templates for remaining components
  - Recharts integration examples
  - Troubleshooting guide

- [x] **Session Summary** ([docs/DASHBOARD_SESSION_SUMMARY.md](docs/DASHBOARD_SESSION_SUMMARY.md))
  - Features demonstrated
  - How to use the dashboard
  - Architecture explanation

### ✅ Phase 1: Plugin System & Infrastructure (Completed 2025-01-16)

#### Core Architecture
- [x] **Hook Manager** ([optimization_engine/plugins/hook_manager.py](optimization_engine/plugins/hook_manager.py))
  - Hook registration with priority-based execution
  - Auto-discovery from plugin directories
  - Context passing to all hooks
  - Execution history tracking

- [x] **Lifecycle Hooks**
  - `pre_solve`: Execute before solver launch
  - `post_solve`: Execute after solve, before extraction
  - `post_extraction`: Execute after result extraction

#### Logging Infrastructure
- [x] **Detailed Trial Logs** ([detailed_logger.py](optimization_engine/plugins/pre_solve/detailed_logger.py))
  - Per-trial log files in `optimization_results/trial_logs/`
  - Complete iteration trace with timestamps
  - Design variables, configuration, timeline
  - Extracted results and constraint evaluations

- [x] **High-Level Optimization Log** ([optimization_logger.py](optimization_engine/plugins/pre_solve/optimization_logger.py))
  - `optimization.log` file tracking overall progress
  - Configuration summary header
  - Compact START/COMPLETE entries per trial
  - Easy to scan format for monitoring

- [x] **Result Appenders**
  - [log_solve_complete.py](optimization_engine/plugins/post_solve/log_solve_complete.py) - Appends solve completion to trial logs
  - [log_results.py](optimization_engine/plugins/post_extraction/log_results.py) - Appends extracted results to trial logs
  - [optimization_logger_results.py](optimization_engine/plugins/post_extraction/optimization_logger_results.py) - Appends results to optimization.log

#### Project Organization
- [x] **Studies Structure** ([studies/](studies/))
  - Standardized folder layout with `model/`, `optimization_results/`, `analysis/`
  - Comprehensive documentation in [studies/README.md](studies/README.md)
  - Example study: [bracket_stress_minimization/](studies/bracket_stress_minimization/)
  - Template structure for future studies

- [x] **Path Resolution** ([atomizer_paths.py](atomizer_paths.py))
  - Intelligent project root detection using marker files
  - Helper functions: `root()`, `optimization_engine()`, `studies()`, `tests()`
  - `ensure_imports()` for robust module imports
  - Works regardless of script location

#### Testing
- [x] **Hook Validation Test** ([test_hooks_with_bracket.py](tests/test_hooks_with_bracket.py))
  - Verifies hook loading and execution
  - Tests 3 trials with dummy data
  - Checks hook execution history

- [x] **Integration Tests**
  - [run_5trial_test.py](tests/run_5trial_test.py) - Quick 5-trial optimization
  - [test_journal_optimization.py](tests/test_journal_optimization.py) - Full optimization test

#### Runner Enhancements
- [x] **Context Passing** ([runner.py:332,365,412](optimization_engine/runner.py))
  - `output_dir` passed to all hook contexts
  - Trial number, design variables, extracted results
  - Configuration dictionary available to hooks

### ✅ Core Engine (Pre-Phase 1)
- [x] Optuna integration with TPE sampler
- [x] Multi-objective optimization support
- [x] NX journal execution ([nx_solver.py](optimization_engine/nx_solver.py))
- [x] Expression updates ([nx_updater.py](optimization_engine/nx_updater.py))
- [x] OP2 result extraction (stress, displacement)
- [x] Study management with resume capability
- [x] Web dashboard (real-time monitoring)
- [x] Precision control (4-decimal rounding)

---

## Active Development

### In Progress - Dashboard (High Priority)
- [x] Backend API complete (FastAPI + WebSocket)
- [x] HTML dashboard with Chart.js complete
- [x] React project structure and configuration complete
- [ ] **Complete React frontend** (Awaiting manual npm install)
  - [ ] Run `npm install` in frontend directory
  - [ ] Create main.tsx and App.tsx
  - [ ] Create Dashboard.tsx with Recharts
  - [ ] Test end-to-end with live optimization

### Up Next - Dashboard (Next Session)
- [ ] Study Configurator page (React)
- [ ] Results Report Viewer page (React)
- [ ] LLM chat interface integration (future)
- [ ] Docker deployment configuration

### In Progress - Phase 3.2 Integration
- [ ] Feature registry creation (Phase 2, Week 1)
- [ ] Claude skill definition (Phase 2, Week 1)

### Up Next (Phase 2, Week 2)
- [ ] Natural language parser
- [ ] Intent classification system
- [ ] Entity extraction for optimization parameters
- [ ] Conversational workflow manager

### Backlog (Phase 3+)
- [ ] Custom function generator (RSS, weighted objectives)
- [ ] Journal script generator
- [ ] Code validation pipeline
- [ ] Result analyzer with statistical analysis
- [ ] Surrogate quality checker
- [ ] HTML/PDF report generator

---

## Known Issues

### Critical
- None currently

### Minor
- [ ] `.claude/settings.local.json` modified during development (contains user-specific settings)
- [ ] Some old bash background processes still running from previous tests

### Documentation
- [ ] Need to add examples of custom hooks to studies/README.md
- [ ] Missing API documentation for hook_manager methods
- [ ] No developer guide for creating new plugins

---

## Testing Status

### Automated Tests
- ✅ **Hook system** - `test_hooks_with_bracket.py` passing
- ✅ **5-trial integration** - `run_5trial_test.py` working
- ✅ **Full optimization** - `test_journal_optimization.py` functional
- ⏳ **Unit tests** - Need to create for individual modules
- ⏳ **CI/CD pipeline** - Not yet set up

### Manual Testing
- ✅ Bracket optimization (50 trials)
- ✅ Log file generation in correct locations
- ✅ Hook execution at all lifecycle points
- ✅ Path resolution across different script locations
- ✅ **Dashboard backend** - REST API and WebSocket tested successfully
- ✅ **HTML dashboard** - Live updates working with Chart.js
- ⏳ **React dashboard** - Pending npm install and completion
- ⏳ Resume functionality with config validation

### Test Coverage
- Hook manager: ~80% (core functionality tested)
- Logging plugins: 100% (tested via integration tests)
- Path resolution: 100% (tested in all scripts)
- Result extractors: ~70% (basic tests exist)
- **Dashboard backend**: ~90% (REST endpoints and WebSocket tested)
- **Dashboard frontend**: ~60% (HTML version tested, React pending)
- Overall: ~65% estimated

---

## Phase-by-Phase Progress

### Phase 1: Plugin System ✅ (100% Complete)

**Completed** (2025-01-16):
- [x] Hook system for optimization lifecycle
- [x] Plugin auto-discovery and registration
- [x] Hook manager with priority-based execution
- [x] Detailed per-trial logs (`trial_logs/`)
- [x] High-level optimization log (`optimization.log`)
- [x] Context passing system for hooks
- [x] Studies folder structure
- [x] Comprehensive studies documentation
- [x] Model file organization (`model/` folder)
- [x] Intelligent path resolution
- [x] Test suite for hook system

**Deferred to Future Phases**:
- Feature registry → Phase 2 (with LLM interface)
- `pre_mesh` and `post_mesh` hooks → Future (not needed for current workflow)
- Custom objective/constraint registration → Phase 3 (Code Generation)

---

### Phase 2: LLM Integration 🟡 (0% Complete)

**Target**: 2 weeks (Started 2025-01-16)

#### Week 1 Todos (Feature Registry & Claude Skill)
- [ ] Create `optimization_engine/feature_registry.json`
- [ ] Extract all current capabilities
- [ ] Draft `.claude/skills/atomizer.md`
- [ ] Test LLM's ability to navigate codebase

#### Week 2 Todos (Natural Language Interface)
- [ ] Implement intent classifier
- [ ] Build entity extractor
- [ ] Create workflow manager
- [ ] Test end-to-end: "Create a stress minimization study"

**Success Criteria**:
- [ ] LLM can create optimization from natural language in <5 turns
- [ ] 90% of user requests understood correctly
- [ ] Zero manual JSON editing required

---

### Phase 3: Code Generation ⏳ (Not Started)

**Target**: 3 weeks

**Key Deliverables**:
- [ ] Custom function generator
  - [ ] RSS (Root Sum Square) template
  - [ ] Weighted objectives template
  - [ ] Custom constraints template
- [ ] Journal script generator
- [ ] Code validation pipeline
- [ ] Safe execution environment

**Success Criteria**:
- [ ] LLM generates 10+ custom functions with zero errors
- [ ] All generated code passes safety validation
- [ ] Users save 50% time vs. manual coding

---

### Phase 4: Analysis & Decision Support ⏳ (Not Started)

**Target**: 3 weeks

**Key Deliverables**:
- [ ] Result analyzer (convergence, sensitivity, outliers)
- [ ] Surrogate model quality checker (R², CV score, confidence intervals)
- [ ] Decision assistant (trade-offs, what-if analysis, recommendations)

**Success Criteria**:
- [ ] Surrogate quality detection 95% accurate
- [ ] Recommendations lead to 30% faster convergence
- [ ] Users report higher confidence in results

---

### Phase 5: Automated Reporting ⏳ (Not Started)

**Target**: 2 weeks

**Key Deliverables**:
- [ ] Report generator with Jinja2 templates
- [ ] Multi-format export (HTML, PDF, Markdown, JSON)
- [ ] LLM-written narrative explanations

**Success Criteria**:
- [ ] Reports generated in <30 seconds
- [ ] Narrative quality rated 4/5 by engineers
- [ ] 80% of reports used without manual editing

---

### Phase 6: NX MCP Enhancement ⏳ (Not Started)

**Target**: 4 weeks

**Key Deliverables**:
- [ ] NX documentation MCP server
- [ ] Advanced NX operations library
- [ ] Feature bank with 50+ pre-built operations

**Success Criteria**:
- [ ] NX MCP answers 95% of API questions correctly
- [ ] Feature bank covers 80% of common workflows
- [ ] Users write 50% less manual journal code

---

### Phase 7: Self-Improving System ⏳ (Not Started)

**Target**: 4 weeks

**Key Deliverables**:
- [ ] Feature learning system
- [ ] Best practices database
- [ ] Continuous documentation generation

**Success Criteria**:
- [ ] 20+ user-contributed features in library
- [ ] Pattern recognition identifies 10+ best practices
- [ ] Documentation auto-updates with zero manual effort

---

## Development Commands

### Running Dashboard
```bash
# Start backend server
cd atomizer-dashboard/backend
python -m uvicorn api.main:app --reload --host 0.0.0.0 --port 8000

# Access HTML dashboard (current)
# Open browser: http://localhost:8000

# Start React frontend (when ready)
cd atomizer-dashboard/frontend
npm install  # First time only
npm run dev  # Starts on http://localhost:3000
```

### Running Tests
```bash
# Hook validation (3 trials, fast)
python tests/test_hooks_with_bracket.py

# Quick integration test (5 trials)
python tests/run_5trial_test.py

# Full optimization test
python tests/test_journal_optimization.py
```

### Code Quality
```bash
# Run linter (when available)
# pylint optimization_engine/

# Run type checker (when available)
# mypy optimization_engine/

# Run all tests (when test suite is complete)
# pytest tests/
```

### Git Workflow
```bash
# Stage all changes
git add .

# Commit with conventional commits format
git commit -m "feat: description"  # New feature
git commit -m "fix: description"   # Bug fix
git commit -m "docs: description"  # Documentation
git commit -m "test: description"  # Tests
git commit -m "refactor: description"  # Code refactoring

# Push to GitHub
git push origin main
```

---

## Documentation

### For Developers
- [DEVELOPMENT_ROADMAP.md](DEVELOPMENT_ROADMAP.md) - Strategic vision and phases
- [studies/README.md](studies/README.md) - Studies folder organization
- [CHANGELOG.md](CHANGELOG.md) - Version history

### Dashboard Documentation
- [docs/DASHBOARD_MASTER_PLAN.md](docs/DASHBOARD_MASTER_PLAN.md) - Complete architecture blueprint
- [docs/DASHBOARD_IMPLEMENTATION_STATUS.md](docs/DASHBOARD_IMPLEMENTATION_STATUS.md) - Current progress
- [docs/DASHBOARD_REACT_IMPLEMENTATION.md](docs/DASHBOARD_REACT_IMPLEMENTATION.md) - React implementation guide
- [docs/DASHBOARD_SESSION_SUMMARY.md](docs/DASHBOARD_SESSION_SUMMARY.md) - Session summary
- [atomizer-dashboard/README.md](atomizer-dashboard/README.md) - Dashboard quick start
- [atomizer-dashboard/backend/README.md](atomizer-dashboard/backend/README.md) - Backend API docs
- [atomizer-dashboard/frontend/README.md](atomizer-dashboard/frontend/README.md) - Frontend setup guide

### For Users
- [README.md](README.md) - Project overview and quick start
- [docs/INDEX.md](docs/INDEX.md) - Complete documentation index
- [docs/](docs/) - Additional documentation

---

## Notes

### Architecture Decisions
- **Hook system**: Chose priority-based execution to allow precise control of plugin order
- **Path resolution**: Used marker files instead of environment variables for simplicity
- **Logging**: Two-tier system (detailed trial logs + high-level optimization.log) for different use cases

### Performance Considerations
- Hook execution adds <1s overhead per trial (acceptable for FEA simulations)
- Path resolution caching could improve startup time (future optimization)
- Log file sizes grow linearly with trials (~10KB per trial)

### Future Considerations
- Consider moving to structured logging (JSON) for easier parsing
- May need database for storing hook execution history (currently in-memory)
- Dashboard integration will require WebSocket for real-time log streaming

---

**Last Updated**: 2025-11-21
**Maintained by**: Antoine Polvé (antoine@atomaste.com)
**Repository**: [GitHub - Atomizer](https://github.com/yourusername/Atomizer)

---

## Recent Updates (November 21, 2025)

### Dashboard System Implementation ✅
- **Backend**: FastAPI + WebSocket with real-time file watching complete
- **HTML Dashboard**: Functional dashboard with Chart.js, data export, pruning alerts
- **React Setup**: Complete project configuration, types, hooks, components
- **Documentation**: 5 comprehensive markdown documents covering architecture, implementation, and usage

### Next Immediate Steps
1. Run `npm install` in `atomizer-dashboard/frontend`
2. Create `main.tsx`, `App.tsx`, and `Dashboard.tsx` using provided templates
3. Test React dashboard with live optimization
4. Build Study Configurator page (next major feature)