feat: Major update with validators, skills, dashboard, and docs reorganization

- Add validation framework (config, model, results, study validators) - Add Claude Code skills (create-study, run-optimization, generate-report, troubleshoot, analyze-model) - Add Atomizer Dashboard (React frontend + FastAPI backend) - Reorganize docs into structured directories (00-09) - Add neural surrogate modules and training infrastructure - Add multi-objective optimization support 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-25 19:23:58 -05:00
parent 74a92803b7
commit e3bdb08a22
155 changed files with 52729 additions and 37 deletions
--- a/.claude/skills/analyze-model.md
+++ b/.claude/skills/analyze-model.md
@@ -0,0 +1,269 @@
+# Analyze Model Skill
+
+**Last Updated**: November 25, 2025
+**Version**: 1.0 - Model Analysis and Feature Extraction
+
+You are helping the user understand their NX model's structure and identify optimization opportunities.
+
+## Purpose
+
+Extract and present information about an NX model to help the user:
+1. Identify available parametric expressions (potential design variables)
+2. Understand the simulation setup (analysis types, boundary conditions)
+3. Discover material properties
+4. Recommend optimization strategies based on model characteristics
+
+## Triggers
+
+- "analyze this model"
+- "what can I optimize"
+- "show me the expressions"
+- "look at my NX model"
+- "what parameters are available"
+
+## Prerequisites
+
+- User must provide path to NX model files (.prt, .sim, .fem)
+- NX must be available on the system (configured in config.py)
+- Model files must be valid NX format
+
+## Information Gathering
+
+Ask these questions if not already provided:
+
+1. **Model Location**:
+   - "Where is your NX model? (path to .prt file)"
+   - Default: Look in `studies/*/1_setup/model/`
+
+2. **Analysis Interest**:
+   - "What type of optimization are you considering?" (optional)
+   - This helps focus the analysis on relevant aspects
+
+## Execution Steps
+
+### Step 1: Validate Model Files
+
+Check that required files exist:
+
+```python
+from pathlib import Path
+
+def validate_model_files(model_path: Path) -> dict:
+    """Validate NX model files exist."""
+    prt_file = model_path
+    base_name = prt_file.stem
+    parent_dir = prt_file.parent
+
+    result = {
+        'prt_exists': prt_file.exists(),
+        'sim_file': None,
+        'fem_file': None,
+        'errors': []
+    }
+
+    # Look for simulation file
+    sim_patterns = [
+        f"{base_name}_sim1.sim",
+        f"{base_name}_sim.sim",
+        f"{base_name}.sim"
+    ]
+    for pattern in sim_patterns:
+        sim_path = parent_dir / pattern
+        if sim_path.exists():
+            result['sim_file'] = sim_path
+            break
+
+    # Look for FEM file
+    fem_patterns = [
+        f"{base_name}_fem1.fem",
+        f"{base_name}_fem.fem",
+        f"{base_name}.fem"
+    ]
+    for pattern in fem_patterns:
+        fem_path = parent_dir / pattern
+        if fem_path.exists():
+            result['fem_file'] = fem_path
+            break
+
+    if not result['prt_exists']:
+        result['errors'].append(f"Part file not found: {prt_file}")
+    if not result['sim_file']:
+        result['errors'].append("No simulation file (.sim) found")
+    if not result['fem_file']:
+        result['errors'].append("No FEM file (.fem) found - will be created on first solve")
+
+    return result
+```
+
+### Step 2: Extract Expressions
+
+Use NX Python API to extract all parametric expressions:
+
+```python
+# This requires running a journal inside NX
+# Use the expression extractor from optimization_engine
+
+from optimization_engine.extractors.expression_extractor import extract_all_expressions
+
+expressions = extract_all_expressions(prt_file)
+# Returns: [{'name': 'thickness', 'value': 2.0, 'unit': 'mm', 'formula': None}, ...]
+```
+
+**Manual Extraction Method** (if NX API not available):
+1. Read the .prt file header for expression metadata
+2. Look for common parameter naming patterns
+3. Ask user to provide expression names from NX
+
+### Step 3: Classify Expressions
+
+Categorize expressions by likely purpose:
+
+| Pattern | Category | Optimization Relevance |
+|---------|----------|----------------------|
+| `*_thickness`, `*_t`, `wall_*` | Structural | High - affects mass & stiffness |
+| `*_diameter`, `*_d`, `hole_*` | Geometric | High - affects mass & stress |
+| `*_radius`, `*_r`, `fillet_*` | Geometric | Medium - affects stress concentration |
+| `*_count`, `n_*`, `num_*` | Discrete | Medium - affects mass & complexity |
+| `*_length`, `*_l`, `span_*` | Dimensional | Context-dependent |
+| `*_angle`, `*_deg` | Orientation | Low-Medium |
+| `material_*`, `mat_*` | Material | Special handling required |
+
+### Step 4: Analyze Simulation Setup
+
+If .sim file exists, analyze:
+
+1. **Solution Types**:
+   - SOL 101 (Static) → Displacement, Stress
+   - SOL 103 (Modal) → Natural Frequencies
+   - SOL 105 (Buckling) → Buckling Load Factor
+   - SOL 106 (Nonlinear Static) → Large deformation
+   - SOL 112 (Transient) → Dynamic response
+
+2. **Load Cases**: Identify applied loads and boundary conditions
+
+3. **Material Properties**: Extract material definitions
+
+### Step 5: Generate Recommendations
+
+Based on analysis, recommend:
+
+1. **Design Variables**: Which expressions are good candidates
+2. **Objectives**: What can be optimized
+3. **Constraints**: What should be bounded
+4. **Protocol**: Which optimization protocol fits best
+
+## Output Format
+
+Present analysis in structured format:
+
+```
+MODEL ANALYSIS REPORT
+=====================
+
+Model: {model_name}
+Location: {model_path}
+
+FILES FOUND
+-----------
+✓ Part file: {prt_file}
+✓ Simulation: {sim_file}
+✓ FEM mesh: {fem_file}
+
+PARAMETRIC EXPRESSIONS
+----------------------
+| Name | Current Value | Unit | Category | Optimization Candidate |
+|------|---------------|------|----------|----------------------|
+| thickness | 2.0 | mm | Structural | ✓ High |
+| hole_diameter | 10.0 | mm | Geometric | ✓ High |
+| fillet_radius | 3.0 | mm | Geometric | ✓ Medium |
+| length | 100.0 | mm | Dimensional | ? Check constraints |
+
+SIMULATION SETUP
+----------------
+Analysis Types: Static (SOL 101), Modal (SOL 103)
+Material: Aluminum 6061-T6 (E=68.9 GPa, ρ=2700 kg/m³)
+Loads:
+  - Force: 500 N at tip
+  - Constraint: Fixed at root
+
+RECOMMENDATIONS
+---------------
+Suggested Objectives:
+  - Minimize mass (extract from p173 expression or FEM)
+  - Maximize first natural frequency
+
+Suggested Constraints:
+  - Max von Mises stress < 276 MPa (Al 6061 yield)
+  - Max displacement < {user to specify}
+
+Recommended Protocol: Protocol 11 (Multi-Objective NSGA-II)
+  - Reason: Multiple competing objectives (mass vs frequency)
+
+Ready to create optimization study? Say "create study" to proceed.
+```
+
+## Error Handling
+
+| Error | Recovery |
+|-------|----------|
+| File not found | Ask user for correct path |
+| Invalid NX format | Suggest checking NX version compatibility |
+| No expressions found | Guide user to create expressions in NX |
+| Simulation incomplete | List missing components |
+
+## Integration with Other Skills
+
+After analysis, can chain to:
+- `/create-study` - Create optimization based on findings
+- `/configure-surrogate` - Setup NN if large-scale optimization needed
+
+## Example Conversation
+
+```
+USER: Look at my NX model at studies/uav_arm_optimization/1_setup/model/Beam.prt
+
+CLAUDE: I'll analyze your UAV arm model. Let me examine its structure...
+
+[Runs analysis]
+
+MODEL ANALYSIS REPORT
+=====================
+Model: Beam
+Location: studies/uav_arm_optimization/1_setup/model/
+
+PARAMETRIC EXPRESSIONS
+----------------------
+Found 4 controllable parameters:
+| Name | Value | Bounds Suggestion |
+|------|-------|------------------|
+| beam_half_core_thickness | 15.0 mm | [5, 25] |
+| beam_face_thickness | 2.0 mm | [1, 5] |
+| holes_diameter | 30.0 mm | [10, 50] |
+| hole_count | 8 | [4, 12] |
+
+SIMULATION SETUP
+----------------
+✓ Static Analysis (SOL 101) - for stress/displacement
+✓ Modal Analysis (SOL 103) - for natural frequencies
+Material: Aluminum sandwich panel
+
+RECOMMENDATION
+--------------
+This model is well-suited for multi-objective optimization:
+- Objective 1: Minimize mass
+- Objective 2: Maximize first natural frequency
+- Constraint: Keep stress below yield strength
+
+Would you like me to create an optimization study with these settings?
+
+USER: Yes, but add a displacement constraint of 5mm max
+
+CLAUDE: [Invokes create-study skill with analyzed parameters + user constraint]
+```
+
+## Notes
+
+- Expression extraction requires NX to be running or using file parsing
+- Some expressions may be formula-driven (not directly editable)
+- Material properties may need separate handling
+- Consider mesh quality when recommending optimization bounds
--- a/.claude/skills/create-study.md
+++ b/.claude/skills/create-study.md
@@ -1,7 +1,7 @@
 # Create Optimization Study Skill

-**Last Updated**: November 24, 2025
-**Version**: 1.0 - Complete Study Scaffolding
+**Last Updated**: November 25, 2025
+**Version**: 1.1 - Complete Study Scaffolding with Validator Integration

 You are helping the user create a complete Atomizer optimization study from a natural language description.

@@ -18,6 +18,8 @@ Guide the user through an interactive conversation to:

 A complete Atomizer study has this structure:

+**CRITICAL**: All study files, including README.md and results, MUST be located within the study directory. NEVER create study documentation at the project root.
+
 ```
 studies/{study_name}/
 ├── 1_setup/
@@ -33,7 +35,7 @@ studies/{study_name}/
 │   └── [various analysis files]
 ├── run_optimization.py              # YOU GENERATE THIS
 ├── reset_study.py                   # YOU GENERATE THIS
-├── README.md                        # YOU GENERATE THIS
+├── README.md                        # YOU GENERATE THIS (INSIDE study directory!)
 └── NX_FILE_MODIFICATIONS_REQUIRED.md  # YOU GENERATE THIS (if needed)
 ```

@@ -275,6 +277,10 @@ except Exception as e:

 ### 5. README.md

+**CRITICAL: ALWAYS place README.md INSIDE the study directory at `studies/{study_name}/README.md`**
+
+Never create study documentation at the project root. All study-specific documentation must be centralized within the study directory structure.
+
 Comprehensive documentation including:
 - Engineering scenario and context
 - Problem statement with real-world constraints
@@ -289,6 +295,7 @@ Comprehensive documentation including:
 - Comparison with other studies
 - Technical notes

+**Location**: `studies/{study_name}/README.md` (NOT at project root)
 **Template**: Use [studies/drone_gimbal_arm_optimization/README.md](../studies/drone_gimbal_arm_optimization/README.md:1) as reference

 ### 6. NX_FILE_MODIFICATIONS_REQUIRED.md (if needed)
@@ -441,6 +448,91 @@ Extractors: extract_solid_stress, extract_displacement
 Multi-Solution: No (static only)
 ```

+## Validation Integration
+
+After generating files, always validate the study setup using the validator system:
+
+### Config Validation
+
+```python
+from optimization_engine.validators import validate_config_file
+
+result = validate_config_file("studies/{study_name}/1_setup/optimization_config.json")
+if result.is_valid:
+    print("[OK] Configuration is valid!")
+else:
+    for error in result.errors:
+        print(f"[ERROR] {error}")
+```
+
+### Model Validation
+
+```python
+from optimization_engine.validators import validate_study_model
+
+result = validate_study_model("{study_name}")
+if result.is_valid:
+    print(f"[OK] Model files valid!")
+    print(f"    Part: {result.prt_file.name}")
+    print(f"    Simulation: {result.sim_file.name}")
+else:
+    for error in result.errors:
+        print(f"[ERROR] {error}")
+```
+
+### Complete Study Validation
+
+```python
+from optimization_engine.validators import validate_study
+
+result = validate_study("{study_name}")
+print(result)  # Shows complete health check
+```
+
+### Validation Checklist for Generated Studies
+
+Before declaring a study complete, ensure:
+
+1. **Config Validation Passes**:
+   - All design variables have valid bounds (min < max)
+   - All objectives have proper extraction methods
+   - All constraints have thresholds defined
+   - Protocol matches objective count
+
+2. **Model Files Ready** (user must provide):
+   - Part file (.prt) exists in model directory
+   - Simulation file (.sim) exists
+   - FEM file (.fem) will be auto-generated
+
+3. **Run Script Works**:
+   - Test with `python run_optimization.py --trials 1`
+   - Verify imports resolve correctly
+   - Verify NX solver can be reached
+
+### Automated Pre-Flight Check
+
+Add this to run_optimization.py:
+
+```python
+def preflight_check():
+    """Validate study setup before running."""
+    from optimization_engine.validators import validate_study
+
+    result = validate_study(STUDY_NAME)
+
+    if not result.is_ready_to_run:
+        print("[X] Study validation failed!")
+        print(result)
+        sys.exit(1)
+
+    print("[OK] Pre-flight check passed!")
+    return True
+
+if __name__ == "__main__":
+    preflight_check()
+    # ... rest of optimization
+```
+
 ## Critical Reminders

 ### Multi-Objective Return Format
--- a/.claude/skills/generate-report.md
+++ b/.claude/skills/generate-report.md
@@ -0,0 +1,399 @@
+# Generate Report Skill
+
+**Last Updated**: November 25, 2025
+**Version**: 1.0 - Optimization Results Analysis and Reporting
+
+You are helping the user understand and communicate optimization results.
+
+## Purpose
+
+Generate comprehensive reports from completed optimization studies:
+1. Pareto front analysis and visualization
+2. Design space exploration insights
+3. Constraint satisfaction analysis
+4. Engineering recommendations
+5. Export-ready summaries
+
+## Triggers
+
+- "generate report"
+- "show results"
+- "analyze the optimization"
+- "what did we find?"
+- "summarize the study"
+- "export results"
+
+## Prerequisites
+
+- Completed optimization study with `study.db`
+- At least 10 completed trials (for meaningful analysis)
+- Optional: `optimization_config.json` for context
+
+## Report Types
+
+### 1. Quick Summary
+Fast overview of key findings (default).
+
+### 2. Full Technical Report
+Comprehensive analysis with all visualizations.
+
+### 3. Executive Summary
+High-level findings for stakeholders.
+
+### 4. Export Package
+Data files for external analysis.
+
+## Execution Steps
+
+### Step 1: Load Study Data
+
+```python
+import optuna
+import json
+from pathlib import Path
+
+def load_study_data(study_name: str):
+    """Load all optimization data for analysis."""
+    study_dir = Path(f"studies/{study_name}")
+
+    # Load Optuna study
+    storage = f"sqlite:///{study_dir / '2_results' / 'study.db'}"
+    study = optuna.load_study(study_name=study_name, storage=storage)
+
+    # Load config for context
+    config_path = study_dir / "1_setup" / "optimization_config.json"
+    config = json.loads(config_path.read_text()) if config_path.exists() else {}
+
+    return {
+        'study': study,
+        'config': config,
+        'n_trials': len(study.trials),
+        'n_completed': len([t for t in study.trials if t.state == optuna.trial.TrialState.COMPLETE]),
+        'is_multi_objective': len(study.directions) > 1
+    }
+```
+
+### Step 2: Analyze Results
+
+#### For Multi-Objective (Pareto Analysis)
+
+```python
+def analyze_pareto_front(study):
+    """Extract and analyze Pareto-optimal solutions."""
+    pareto_trials = study.best_trials
+
+    results = {
+        'n_pareto': len(pareto_trials),
+        'designs': [],
+        'ranges': {},
+        'trade_offs': []
+    }
+
+    for trial in pareto_trials:
+        design = {
+            'trial_number': trial.number,
+            'objectives': trial.values,
+            'parameters': trial.params,
+            'user_attrs': trial.user_attrs
+        }
+        results['designs'].append(design)
+
+    # Calculate objective ranges on Pareto front
+    if pareto_trials:
+        obj_names = ['mass', 'frequency']  # From config
+        for i, name in enumerate(obj_names):
+            values = [t.values[i] for t in pareto_trials]
+            results['ranges'][name] = {
+                'min': min(values),
+                'max': max(values),
+                'spread': max(values) - min(values)
+            }
+
+    return results
+```
+
+#### For Single-Objective
+
+```python
+def analyze_single_objective(study):
+    """Analyze single-objective optimization results."""
+    best = study.best_trial
+
+    return {
+        'best_value': best.value,
+        'best_params': best.params,
+        'convergence': get_convergence_history(study),
+        'improvement_rate': calculate_improvement_rate(study)
+    }
+```
+
+### Step 3: Constraint Analysis
+
+```python
+def analyze_constraints(study, config):
+    """Analyze constraint satisfaction across trials."""
+    constraints = config.get('constraints', [])
+
+    results = {
+        'total_trials': len(study.trials),
+        'feasible': 0,
+        'infeasible': 0,
+        'violations': {}
+    }
+
+    for constraint in constraints:
+        results['violations'][constraint['name']] = 0
+
+    for trial in study.trials:
+        if trial.state != optuna.trial.TrialState.COMPLETE:
+            continue
+
+        is_feasible = trial.user_attrs.get('feasible', True)
+        if is_feasible:
+            results['feasible'] += 1
+        else:
+            results['infeasible'] += 1
+            # Track which constraints were violated
+            for constraint in constraints:
+                if constraint['name'] in trial.user_attrs.get('violated_constraints', []):
+                    results['violations'][constraint['name']] += 1
+
+    results['feasibility_rate'] = results['feasible'] / results['total_trials'] * 100
+
+    return results
+```
+
+### Step 4: Generate Visualizations
+
+Describe what plots to generate (actual generation done via Python scripts):
+
+1. **Pareto Front Plot**: Mass vs Frequency with feasibility coloring
+2. **Parallel Coordinates**: All design variables with objective coloring
+3. **Parameter Importance**: Which variables most affect objectives
+4. **Convergence History**: Best value over trials
+5. **Design Space Coverage**: Parameter distributions
+
+### Step 5: Generate Report Text
+
+## Output Format: Quick Summary
+
+```
+OPTIMIZATION RESULTS: {study_name}
+===================================
+
+Study Info:
+  Protocol: Protocol 11 (Multi-Objective NSGA-II)
+  Trials: 30 completed
+  Duration: 14m 23s
+  Feasibility Rate: 80%
+
+PARETO FRONT (8 optimal designs)
+--------------------------------
+
+| # | Mass (g) | Freq (Hz) | Feasible | Notes |
+|---|----------|-----------|----------|-------|
+| 1 | 231      | 118       | Yes      | Lightest |
+| 2 | 248      | 128       | Yes      | |
+| 3 | 265      | 138       | Yes      | Balanced |
+| 4 | 282      | 148       | Yes      | |
+| 5 | 298      | 156       | Yes      | Stiffest |
+
+Trade-off Summary:
+  - Mass range: 231g - 298g (67g spread)
+  - Frequency range: 118Hz - 156Hz (38Hz spread)
+  - Trade-off rate: ~1.7g per Hz
+
+BEST DESIGNS BY OBJECTIVE
+-------------------------
+
+Lightest Design (#1):
+  Mass: 231g, Frequency: 118Hz
+  Parameters:
+    beam_half_core_thickness: 8.2mm
+    beam_face_thickness: 1.1mm
+    holes_diameter: 42mm
+    hole_count: 10
+
+Stiffest Design (#5):
+  Mass: 298g, Frequency: 156Hz
+  Parameters:
+    beam_half_core_thickness: 18.5mm
+    beam_face_thickness: 2.8mm
+    holes_diameter: 22mm
+    hole_count: 6
+
+CONSTRAINT ANALYSIS
+-------------------
+
+Constraints checked:
+  - max_displacement < 5mm: 24/30 satisfied (80%)
+  - max_stress < 200 MPa: 28/30 satisfied (93%)
+
+Most common violation: Displacement (6 trials)
+  Occurred when: holes_diameter > 40mm AND face_thickness < 1.5mm
+
+PARAMETER INSIGHTS
+------------------
+
+Most influential on mass:
+  1. holes_diameter (negative correlation)
+  2. beam_face_thickness (positive)
+  3. beam_half_core_thickness (positive)
+
+Most influential on frequency:
+  1. beam_half_core_thickness (positive)
+  2. holes_diameter (negative)
+  3. beam_face_thickness (moderate positive)
+
+RECOMMENDATIONS
+---------------
+
+1. For lightweight priority:
+   - Use larger holes (40-45mm)
+   - Minimize face thickness (1.0-1.5mm)
+   - Accept frequency around 120Hz
+
+2. For stiffness priority:
+   - Use smaller holes (20-25mm)
+   - Increase core thickness (18-20mm)
+   - Accept mass around 300g
+
+3. Balanced recommendation:
+   - Design #3 offers good compromise
+   - 265g mass with 138Hz frequency
+   - 13% lighter than stiffest, 17% stiffer than lightest
+
+NEXT STEPS
+----------
+
+1. Export selected designs: python export_pareto.py --designs 1,3,5
+2. Validate in NX: Open model with exported parameters
+3. Continue optimization: python run_optimization.py --trials 50 --resume
+4. Enable NN acceleration: Add --enable-nn flag for faster exploration
+```
+
+## Output Format: Full Technical Report
+
+Generate markdown file with:
+
+```markdown
+# {study_name} Optimization Report
+
+Generated: {timestamp}
+Protocol: {protocol}
+
+## Executive Summary
+[2-3 paragraph overview]
+
+## Study Configuration
+### Design Variables
+[Table of variables, bounds, types]
+
+### Objectives
+[Table of objectives, goals, targets]
+
+### Constraints
+[Table of constraints, thresholds]
+
+## Results Analysis
+
+### Pareto Front
+[Detailed table of all Pareto-optimal designs]
+
+![Pareto Front](pareto_front.png)
+
+### Design Space Exploration
+[Parallel coordinates plot]
+
+![Parallel Coordinates](parallel_coords.png)
+
+### Convergence History
+[Plot of best values over trials]
+
+### Parameter Importance
+[SHAP or importance analysis]
+
+## Constraint Satisfaction
+[Detailed constraint analysis]
+
+## Engineering Recommendations
+[Specific design recommendations]
+
+## Appendix
+### All Trial Data
+[Exportable CSV reference]
+
+### Methodology Notes
+[Protocol details, sampler settings]
+```
+
+## Export Package
+
+Generate files:
+```
+{study_name}_export/
+├── pareto_designs.csv       # All Pareto-optimal designs
+├── all_trials.csv           # Complete trial history
+├── optimization_summary.json # Machine-readable summary
+├── pareto_front.png         # Visualization
+├── parallel_coords.png      # Visualization
+└── README.md                # Export documentation
+```
+
+## Error Handling
+
+| Error | Recovery |
+|-------|----------|
+| No completed trials | Suggest running more trials |
+| No feasible designs | Analyze constraint violations |
+| Study not found | List available studies |
+| Incomplete data | Note missing fields |
+
+## Integration with Other Skills
+
+- After report → `/export-results` for specific designs
+- If poor results → `/troubleshoot` for analysis
+- To continue → `/run-optimization` with --resume
+
+## Example Conversation
+
+```
+USER: Show me the results for uav_arm_optimization
+
+CLAUDE: I'll generate a report for the UAV arm optimization study...
+
+[Loads study data]
+
+OPTIMIZATION RESULTS: uav_arm_optimization
+==========================================
+
+Found 30 completed trials with 8 Pareto-optimal designs.
+
+PARETO FRONT SUMMARY
+--------------------
+| Design | Mass (g) | Freq (Hz) | Trade-off |
+|--------|----------|-----------|-----------|
+| #1     | 231      | 118       | Lightest  |
+| #3     | 265      | 138       | Balanced  |
+| #5     | 298      | 156       | Stiffest  |
+
+Key Finding: You can save 67g (23%) by accepting 38Hz (24%)
+lower frequency.
+
+RECOMMENDATION: Design #3 (265g, 138Hz) offers the best balance
+for a camera gimbal application where both vibration and weight
+matter.
+
+Would you like me to:
+1. Generate a full technical report with visualizations?
+2. Export the top designs for validation in NX?
+3. Continue optimization to explore more designs?
+```
+
+## Notes
+
+- Reports should be actionable, not just data dumps
+- Always provide engineering context and recommendations
+- Consider the user's stated goals when highlighting results
+- Visualizations should be generated via Python scripts
+- Export formats should be compatible with common tools (Excel, etc.)
--- a/.claude/skills/run-optimization.md
+++ b/.claude/skills/run-optimization.md
@@ -0,0 +1,371 @@
+# Run Optimization Skill
+
+**Last Updated**: November 25, 2025
+**Version**: 1.0 - Optimization Execution and Monitoring
+
+You are helping the user run and monitor Atomizer optimization studies.
+
+## Purpose
+
+Execute optimization studies with proper:
+1. Pre-flight validation
+2. Resource management
+3. Progress monitoring
+4. Error recovery
+5. Dashboard integration
+
+## Triggers
+
+- "run optimization"
+- "start the study"
+- "run {study_name}"
+- "execute optimization"
+- "begin the optimization"
+
+## Prerequisites
+
+- Study must exist in `studies/{study_name}/`
+- `optimization_config.json` must be present and valid
+- `run_optimization.py` must exist
+- NX model files must be in place
+
+## Pre-Flight Checklist
+
+Before running, verify:
+
+### 1. Study Structure
+```
+studies/{study_name}/
+├── 1_setup/
+│   ├── model/
+│   │   ├── {Model}.prt          ✓ Required
+│   │   ├── {Model}_sim1.sim     ✓ Required
+│   │   └── {Model}_fem1.fem     ? Optional (created by NX)
+│   ├── optimization_config.json ✓ Required
+│   └── workflow_config.json     ? Optional
+├── 2_results/                   ? Created automatically
+└── run_optimization.py          ✓ Required
+```
+
+### 2. Configuration Validation
+```python
+from optimization_engine.validators.config_validator import validate_config
+
+result = validate_config(study_dir / "1_setup" / "optimization_config.json")
+if result.errors:
+    # STOP - fix errors first
+    for error in result.errors:
+        print(f"ERROR: {error}")
+if result.warnings:
+    # WARN but can continue
+    for warning in result.warnings:
+        print(f"WARNING: {warning}")
+```
+
+### 3. NX Environment
+- Verify NX is installed (check `config.py` for `NX_VERSION`)
+- Verify Nastran solver is available
+- Check for any running NX processes that might conflict
+
+## Execution Modes
+
+### Mode 1: Quick Test (3-5 trials)
+```bash
+cd studies/{study_name}
+python run_optimization.py --trials 3
+```
+**Use when**: First time running, testing configuration
+
+### Mode 2: Standard Run
+```bash
+cd studies/{study_name}
+python run_optimization.py --trials 30
+```
+**Use when**: Production optimization with FEA only
+
+### Mode 3: Extended with NN Surrogate
+```bash
+cd studies/{study_name}
+python run_optimization.py --trials 200 --enable-nn
+```
+**Use when**: Large-scale optimization with trained surrogate
+
+### Mode 4: Resume Interrupted
+```bash
+cd studies/{study_name}
+python run_optimization.py --trials 30 --resume
+```
+**Use when**: Optimization was interrupted
+
+## Execution Steps
+
+### Step 1: Validate Study
+
+Run pre-flight checks and present status:
+
+```
+PRE-FLIGHT CHECK: {study_name}
+===============================
+
+Configuration:
+  ✓ optimization_config.json valid
+  ✓ 4 design variables defined
+  ✓ 2 objectives (multi-objective)
+  ✓ 2 constraints configured
+
+Model Files:
+  ✓ Beam.prt exists (3.2 MB)
+  ✓ Beam_sim1.sim exists (1.1 MB)
+  ✓ Beam_fem1.fem exists (0.8 MB)
+
+Environment:
+  ✓ NX 2412 detected
+  ✓ Nastran solver available
+  ? No NX processes running (clean slate)
+
+Estimated Runtime:
+  - 30 trials × ~30s/trial = ~15 minutes
+  - With NN: 200 trials × ~2s/trial = ~7 minutes (after training)
+
+Ready to run. Proceed? (Y/n)
+```
+
+### Step 2: Start Optimization
+
+```python
+import subprocess
+import sys
+from pathlib import Path
+
+def run_optimization(study_name: str, trials: int = 30,
+                     enable_nn: bool = False, resume: bool = False):
+    """Start optimization as background process."""
+    study_dir = Path(f"studies/{study_name}")
+
+    cmd = [
+        sys.executable,
+        str(study_dir / "run_optimization.py"),
+        "--trials", str(trials)
+    ]
+
+    if enable_nn:
+        cmd.append("--enable-nn")
+    if resume:
+        cmd.append("--resume")
+
+    # Run in background
+    process = subprocess.Popen(
+        cmd,
+        cwd=study_dir,
+        stdout=subprocess.PIPE,
+        stderr=subprocess.STDOUT
+    )
+
+    return process.pid
+```
+
+### Step 3: Monitor Progress
+
+Provide real-time updates:
+
+```
+OPTIMIZATION RUNNING: {study_name}
+==================================
+
+Progress: [████████░░░░░░░░░░░░] 12/30 trials (40%)
+Elapsed:  5m 32s
+ETA:      ~8 minutes
+
+Current Trial #12:
+  Parameters: thickness=2.3, diameter=15.2, ...
+  Status: Running FEA...
+
+Best So Far (Pareto Front):
+  #1: mass=245g, freq=125Hz (feasible)
+  #2: mass=280g, freq=142Hz (feasible)
+  #3: mass=310g, freq=158Hz (feasible)
+
+Constraint Violations: 3/12 trials (25%)
+  Common: max_stress exceeded
+
+Dashboard: http://localhost:3003
+Optuna Dashboard: http://localhost:8081
+```
+
+### Step 4: Handle Completion/Errors
+
+**On Success**:
+```
+OPTIMIZATION COMPLETE
+=====================
+
+Study: {study_name}
+Duration: 14m 23s
+Trials: 30/30
+
+Results:
+  Feasible designs: 24/30 (80%)
+  Pareto-optimal: 8 designs
+
+Best Designs:
+  #1: mass=231g, freq=118Hz ← Lightest
+  #2: mass=298g, freq=156Hz ← Stiffest
+  #3: mass=265g, freq=138Hz ← Balanced
+
+Next Steps:
+  1. View results: /generate-report {study_name}
+  2. Continue optimization: python run_optimization.py --trials 50 --resume
+  3. Export designs: python export_pareto.py
+```
+
+**On Error**:
+```
+OPTIMIZATION ERROR
+==================
+
+Trial #15 failed with:
+  Error: NX simulation timeout after 600s
+  Design: thickness=1.2, diameter=45, hole_count=12
+
+Possible causes:
+  1. Mesh quality issues with extreme parameters
+  2. Convergence problems in solver
+  3. NX process locked/crashed
+
+Recovery options:
+  1. [Recommended] Resume with --resume flag (skips failed)
+  2. Narrow design variable bounds
+  3. Check NX manually with these parameters
+
+Resume command:
+  python run_optimization.py --trials 30 --resume
+```
+
+## Dashboard Integration
+
+Always inform user about monitoring options:
+
+```
+MONITORING OPTIONS
+==================
+
+1. Atomizer Dashboard (recommended):
+   cd atomizer-dashboard/backend && python -m uvicorn api.main:app --port 8000
+   cd atomizer-dashboard/frontend && npm run dev
+   → http://localhost:3003
+
+2. Optuna Dashboard:
+   python -c "import optuna; from optuna_dashboard import run_server; ..."
+   → http://localhost:8081
+
+3. Command Line:
+   Watch study.db for updates
+   Tail the log file
+```
+
+## Resource Management
+
+### Before Running
+- Check available RAM (recommend 8GB+ free)
+- Check disk space (OP2 files can be large)
+- Close unnecessary NX sessions
+
+### During Run
+- Monitor CPU usage (Nastran is CPU-intensive)
+- Watch for disk space issues
+- Handle NX license timeouts
+
+### Cleanup After
+- Option to remove intermediate OP2 files
+- Compress results for archival
+- Clean worker directories
+
+## Error Handling
+
+| Error | Cause | Recovery |
+|-------|-------|----------|
+| NX not found | NX not installed/configured | Check config.py settings |
+| Simulation timeout | Complex mesh/convergence | Increase timeout or simplify |
+| License error | NX license unavailable | Wait for license or use queue |
+| Database locked | Multiple processes | Stop conflicting processes |
+| Out of memory | Large mesh | Reduce mesh density |
+| OP2 parse error | Corrupted output | Re-run trial |
+
+## Parallel Execution (Advanced)
+
+For large studies, can run parallel workers:
+
+```python
+# Warning: Requires careful NX session management
+# Each worker needs isolated model directory
+
+python run_training_fea.py --study {study_name} --workers 2
+```
+
+## Output Format
+
+After starting optimization:
+
+```
+OPTIMIZATION STARTED
+====================
+
+Study: {study_name}
+Trials: 30
+Mode: Multi-objective (NSGA-II)
+Started: 2025-11-25 15:30:00
+
+Process ID: 12345
+Log File: studies/{study_name}/2_results/optimization.log
+
+Monitoring:
+  Dashboard: http://localhost:3003
+  Database: studies/{study_name}/2_results/study.db
+
+Commands:
+  Check progress: "how's the optimization going?"
+  Stop: Ctrl+C in terminal or kill process 12345
+  Resume: python run_optimization.py --resume
+```
+
+## Integration with Other Skills
+
+- After completion → `/generate-report`
+- On error → `/troubleshoot`
+- For more trials → `/configure-surrogate` (enable NN)
+
+## Example Conversation
+
+```
+USER: Run the uav_arm optimization
+
+CLAUDE: I'll start the UAV arm optimization. Let me run pre-flight checks...
+
+PRE-FLIGHT CHECK: uav_arm_optimization
+======================================
+✓ Configuration valid (Protocol 11, 30 trials)
+✓ Model files present
+✓ NX 2412 available
+
+Ready to start. This will take approximately 15 minutes.
+Starting now...
+
+OPTIMIZATION STARTED
+====================
+Process running in background.
+Monitor at: http://localhost:3003
+
+I'll let you know when it completes. You can ask "how's it going?" anytime.
+
+[... 15 minutes later ...]
+
+OPTIMIZATION COMPLETE
+=====================
+30/30 trials completed successfully.
+Found 8 Pareto-optimal designs.
+
+Best lightweight: 231g @ 118Hz
+Best stiff: 298g @ 156Hz
+
+Would you like me to generate a detailed report?
+```
--- a/.claude/skills/troubleshoot.md
+++ b/.claude/skills/troubleshoot.md
@@ -0,0 +1,358 @@
+# Troubleshoot Skill
+
+**Last Updated**: November 25, 2025
+**Version**: 1.0 - Debug Common Issues and Error Recovery
+
+You are helping the user diagnose and fix problems with Atomizer optimization studies.
+
+## Purpose
+
+Diagnose and resolve common issues:
+1. Configuration validation failures
+2. Model file problems
+3. NX solver errors
+4. Optuna/database issues
+5. Result extraction failures
+6. Constraint violation patterns
+
+## Triggers
+
+- "troubleshoot"
+- "debug"
+- "error"
+- "failed"
+- "not working"
+- "what's wrong"
+- "fix"
+
+## Prerequisites
+
+- Study must exist in `studies/{study_name}/`
+- User should describe the error or symptom
+
+## Diagnostic Process
+
+### Step 1: Run Full Validation
+
+```python
+from optimization_engine.validators import validate_study
+
+result = validate_study("{study_name}")
+print(result)
+```
+
+This provides a complete health check covering:
+- Configuration validity
+- Model file presence
+- Results integrity (if any)
+
+### Step 2: Identify Error Category
+
+Classify the issue into one of these categories:
+
+| Category | Symptoms | First Check |
+|----------|----------|-------------|
+| Config | "Invalid config", validation errors | `validate_config_file()` |
+| Model | "File not found", NX errors | `validate_study_model()` |
+| Solver | "Simulation failed", timeout | NX logs, OP2 files |
+| Database | "Study not found", lock errors | `study.db` file, Optuna |
+| Extraction | "Cannot extract", NaN values | OP2 file validity |
+| Constraints | All trials infeasible | Constraint thresholds |
+
+## Common Issues & Solutions
+
+### Issue 1: Configuration Validation Fails
+
+**Symptoms**:
+```
+[ERROR] [DESIGN_VAR_BOUNDS] beam_thickness: min (5) >= max (3)
+```
+
+**Diagnosis**:
+```python
+from optimization_engine.validators import validate_config_file
+
+result = validate_config_file("studies/{study_name}/1_setup/optimization_config.json")
+for error in result.errors:
+    print(error)
+```
+
+**Solutions**:
+| Error Code | Cause | Fix |
+|------------|-------|-----|
+| DESIGN_VAR_BOUNDS | Bounds inverted | Swap min/max values |
+| MISSING_OBJECTIVES | No objectives defined | Add objectives array |
+| INVALID_DIRECTION | Wrong goal value | Use "minimize" or "maximize" |
+| PROTOCOL_MISMATCH | Wrong protocol for objectives | Match protocol to # objectives |
+
+### Issue 2: Model Files Missing
+
+**Symptoms**:
+```
+[ERROR] No part file (.prt) found in model directory
+```
+
+**Diagnosis**:
+```python
+from optimization_engine.validators import validate_study_model
+
+result = validate_study_model("{study_name}")
+print(f"Part: {result.prt_file}")
+print(f"Sim: {result.sim_file}")
+print(f"FEM: {result.fem_file}")
+```
+
+**Solutions**:
+1. Ensure files are in `studies/{study_name}/1_setup/model/`
+2. Check file naming convention (e.g., `Beam.prt`, `Beam_sim1.sim`)
+3. FEM file auto-generates on first solve (not required initially)
+
+### Issue 3: NX Solver Fails
+
+**Symptoms**:
+```
+[NX SOLVER] Error: Simulation timeout after 600s
+[NX SOLVER] Error: Unable to open simulation file
+```
+
+**Diagnosis**:
+1. Check NX is installed and configured:
+   ```python
+   from config import NX_VERSION, NX_INSTALL_PATH
+   print(f"NX Version: {NX_VERSION}")
+   print(f"Path: {NX_INSTALL_PATH}")
+   ```
+
+2. Check for running NX processes:
+   ```bash
+   tasklist | findstr "ugraf"
+   ```
+
+3. Read NX journal output:
+   ```
+   studies/{study_name}/1_setup/model/_temp_solve_journal.py
+   ```
+
+**Solutions**:
+| Error | Cause | Fix |
+|-------|-------|-----|
+| Timeout | Complex mesh or bad parameters | Increase timeout or simplify design |
+| License error | NX license unavailable | Wait or check license server |
+| File locked | Another NX process has file open | Close NX and retry |
+| Expression not found | NX expression name mismatch | Verify expression names in NX |
+
+### Issue 4: Database Errors
+
+**Symptoms**:
+```
+[ERROR] Study 'my_study' not found in storage
+[ERROR] database is locked
+```
+
+**Diagnosis**:
+```python
+import optuna
+storage = "sqlite:///studies/{study_name}/2_results/study.db"
+studies = optuna.study.get_all_study_summaries(storage)
+print([s.study_name for s in studies])
+```
+
+**Solutions**:
+| Error | Cause | Fix |
+|-------|-------|-----|
+| Study not found | Wrong study name | Check exact name in database |
+| Database locked | Multiple processes | Kill other optimization processes |
+| Corrupted DB | Interrupted write | Delete and restart (backup first) |
+
+### Issue 5: Result Extraction Fails
+
+**Symptoms**:
+```
+[ERROR] Cannot extract displacement from OP2
+[ERROR] NaN values in objectives
+```
+
+**Diagnosis**:
+1. Check OP2 file exists:
+   ```bash
+   dir studies\{study_name}\1_setup\model\*.op2
+   ```
+
+2. Validate OP2 contents:
+   ```python
+   from pyNastran.op2.op2 import OP2
+   op2 = OP2()
+   op2.read_op2("path/to/file.op2")
+   print(op2.get_result_table_names())
+   ```
+
+3. Check extraction config matches OP2:
+   ```json
+   {
+     "extraction": {
+       "params": {
+         "subcase": 1,
+         "result_type": "displacement"
+       }
+     }
+   }
+   ```
+
+**Solutions**:
+| Error | Cause | Fix |
+|-------|-------|-----|
+| No OP2 file | Solve didn't run | Check NX solver output |
+| Wrong subcase | Subcase ID mismatch | Match subcase to solution |
+| Missing result | Result not requested | Enable output in NX |
+
+### Issue 6: All Trials Infeasible
+
+**Symptoms**:
+```
+Feasibility rate: 0%
+All trials violating constraints
+```
+
+**Diagnosis**:
+```python
+from optimization_engine.validators import validate_results
+
+result = validate_results("studies/{study_name}/2_results/study.db")
+print(f"Feasibility: {result.info.feasibility_rate}%")
+```
+
+Check constraint violations in Optuna dashboard or:
+```python
+import optuna
+study = optuna.load_study(...)
+for trial in study.trials:
+    if trial.user_attrs.get('feasible') == False:
+        print(f"Trial {trial.number}: {trial.user_attrs.get('violated_constraints')}")
+```
+
+**Solutions**:
+| Issue | Cause | Fix |
+|-------|-------|-----|
+| All trials fail constraint | Threshold too tight | Relax constraint threshold |
+| Single constraint always fails | Wrong extraction | Check constraint extraction |
+| Bounds cause violations | Design space infeasible | Expand design variable bounds |
+
+## Quick Diagnostic Commands
+
+### Validate Everything
+```bash
+python -m optimization_engine.validators.study_validator {study_name}
+```
+
+### Check Results
+```bash
+python -m optimization_engine.validators.results_validator {study_name}
+```
+
+### List All Studies
+```bash
+python -m optimization_engine.validators.study_validator
+```
+
+### Check Optuna Database
+```python
+import optuna
+storage = "sqlite:///studies/{study_name}/2_results/study.db"
+study = optuna.load_study(study_name="{study_name}", storage=storage)
+print(f"Trials: {len(study.trials)}")
+print(f"Completed: {len([t for t in study.trials if t.state == optuna.trial.TrialState.COMPLETE])}")
+print(f"Failed: {len([t for t in study.trials if t.state == optuna.trial.TrialState.FAIL])}")
+```
+
+## Recovery Actions
+
+### Reset Study Database
+```python
+import optuna
+storage = "sqlite:///studies/{study_name}/2_results/study.db"
+optuna.delete_study(study_name="{study_name}", storage=storage)
+```
+
+Or use the reset script:
+```bash
+python studies/{study_name}/reset_study.py
+```
+
+### Resume Interrupted Study
+```bash
+python studies/{study_name}/run_optimization.py --trials 30 --resume
+```
+
+### Clean Worker Directories
+```bash
+# Remove temp files from worker dirs
+del /S /Q studies\{study_name}\1_setup\worker_*
+```
+
+### Backup and Restore Database
+```bash
+# Backup
+copy studies\{study_name}\2_results\study.db studies\{study_name}\2_results\study_backup.db
+
+# Restore
+copy studies\{study_name}\2_results\study_backup.db studies\{study_name}\2_results\study.db
+```
+
+## Error Message Reference
+
+| Error Message | Category | Quick Fix |
+|---------------|----------|-----------|
+| "min >= max" | Config | Swap bounds |
+| "No part file found" | Model | Add .prt file |
+| "Simulation timeout" | Solver | Increase timeout or check NX |
+| "database is locked" | Database | Kill other processes |
+| "Cannot extract" | Extraction | Check OP2 and extraction config |
+| "All trials infeasible" | Constraints | Relax thresholds |
+
+## Integration with Other Skills
+
+- After fixing config -> `/run-optimization` to restart
+- After debugging results -> `/generate-report` to analyze
+- If model needs changes -> Check `NX_FILE_MODIFICATIONS_REQUIRED.md`
+
+## Example Conversation
+
+```
+USER: My optimization is failing with "Cannot extract displacement"
+
+CLAUDE: Let me diagnose the displacement extraction issue.
+
+First, I'll validate the study:
+
+[Runs validate_study()]
+
+The validation shows model files are present. Let me check if the OP2 file exists
+and contains displacement data...
+
+[Checks OP2 file]
+
+I found the issue: Your extraction config specifies subcase=1, but the displacement
+results are in subcase=2 (the static analysis solution).
+
+Fix: Update optimization_config.json:
+```json
+{
+  "extraction": {
+    "params": {
+      "subcase": 2  // Was 1, should be 2
+    }
+  }
+}
+```
+
+After this change, run again with:
+  python run_optimization.py --trials 30 --resume
+```
+
+## Notes
+
+- Always run full validation before investigating specific issues
+- Most problems are configuration or file path issues
+- NX solver errors often require checking NX directly
+- Keep backups before modifying study.db
+- Use --resume to continue after fixing issues