feat: Add Protocol 13 adaptive optimization, Plotly charts, and dashboard improvements
## Protocol 13: Adaptive Multi-Objective Optimization - Iterative FEA + Neural Network surrogate workflow - Initial FEA sampling, NN training, NN-accelerated search - FEA validation of top NN predictions, retraining loop - adaptive_state.json tracks iteration history and best values - M1 mirror study (V11) with 103 FEA, 3000 NN trials ## Dashboard Visualization Enhancements - Added Plotly.js interactive charts (parallel coords, Pareto, convergence) - Lazy loading with React.lazy() for performance - Code splitting: plotly.js-basic-dist (~1MB vs 3.5MB) - Chart library toggle (Recharts default, Plotly on-demand) - ExpandableChart component for full-screen modal views - ConsoleOutput component for real-time log viewing ## Documentation - Protocol 13 detailed documentation - Dashboard visualization guide - Plotly components README - Updated run-optimization skill with Mode 5 (adaptive) ## Bug Fixes - Fixed TypeScript errors in dashboard components - Fixed Card component to accept ReactNode title - Removed unused imports across components 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Antoine
@@ -24,6 +24,9 @@ This section defines ALL available components. When generating `run_optimization
|
||||
| E5 | **CAD Expression Mass** | `optimization_engine.extractors.extract_mass_from_expression` | `extract_mass_from_expression(prt_file, expression_name='p173')` | `.prt` + `_temp_mass.txt` | kg | `float` (mass in kg) |
|
||||
| E6 | **Field Data** | `optimization_engine.extractors.field_data_extractor` | `FieldDataExtractor(field_file, result_column, aggregation)` | `.fld`/`.csv` | varies | `{'value': float, 'stats': dict}` |
|
||||
| E7 | **Stiffness** | `optimization_engine.extractors.stiffness_calculator` | `StiffnessCalculator(field_file, op2_file, force_component, displacement_component)` | `.fld` + `.op2` | N/mm | `{'stiffness': float, 'displacement': float, 'force': float}` |
|
||||
| E8 | **Zernike WFE** | `optimization_engine.extractors.extract_zernike` | `extract_zernike_from_op2(op2_file, bdf_file, subcase)` | `.op2` + `.bdf` | nm | `{'global_rms_nm': float, 'filtered_rms_nm': float, 'coefficients': list, ...}` |
|
||||
| E9 | **Zernike Relative** | `optimization_engine.extractors.extract_zernike` | `extract_zernike_relative_rms(op2_file, bdf_file, target_subcase, ref_subcase)` | `.op2` + `.bdf` | nm | `{'relative_filtered_rms_nm': float, 'delta_coefficients': list, ...}` |
|
||||
| E10 | **Zernike Helpers** | `optimization_engine.extractors.zernike_helpers` | `create_zernike_objective(op2_finder, subcase, metric)` | `.op2` | nm | Callable returning metric value |
|
||||
|
||||
### PR.2 Extractor Code Snippets (COPY-PASTE)
|
||||
|
||||
@@ -76,6 +79,59 @@ applied_force = 1000.0 # N - MUST MATCH YOUR MODEL'S APPLIED LOAD
|
||||
stiffness = applied_force / max(abs(max_displacement), 1e-6) # N/mm
|
||||
```
|
||||
|
||||
**E8: Zernike Wavefront Error Extraction (Telescope Mirrors)**
|
||||
```python
|
||||
from optimization_engine.extractors.extract_zernike import extract_zernike_from_op2
|
||||
|
||||
# Extract Zernike coefficients and RMS metrics for a single subcase
|
||||
result = extract_zernike_from_op2(
|
||||
op2_file,
|
||||
bdf_file=None, # Auto-detect from op2 location
|
||||
subcase="20", # Subcase label (e.g., "20" = 20 deg elevation)
|
||||
displacement_unit="mm"
|
||||
)
|
||||
global_rms = result['global_rms_nm'] # Total surface RMS in nm
|
||||
filtered_rms = result['filtered_rms_nm'] # RMS with low orders (piston, tip, tilt, defocus) removed
|
||||
coefficients = result['coefficients'] # List of 50 Zernike coefficients
|
||||
```
|
||||
|
||||
**E9: Zernike Relative RMS (Between Subcases)**
|
||||
```python
|
||||
from optimization_engine.extractors.extract_zernike import extract_zernike_relative_rms
|
||||
|
||||
# Compare wavefront error between subcases (e.g., 40 deg vs 20 deg reference)
|
||||
result = extract_zernike_relative_rms(
|
||||
op2_file,
|
||||
bdf_file=None,
|
||||
target_subcase="40", # Target orientation
|
||||
reference_subcase="20", # Reference (usually polishing orientation)
|
||||
displacement_unit="mm"
|
||||
)
|
||||
relative_rms = result['relative_filtered_rms_nm'] # Differential WFE in nm
|
||||
delta_coeffs = result['delta_coefficients'] # Coefficient differences
|
||||
```
|
||||
|
||||
**E10: Zernike Objective Builder (Multi-Subcase Optimization)**
|
||||
```python
|
||||
from optimization_engine.extractors.zernike_helpers import ZernikeObjectiveBuilder
|
||||
|
||||
# Build objectives for multiple subcases in one extractor
|
||||
builder = ZernikeObjectiveBuilder(
|
||||
op2_finder=lambda: model_dir / "ASSY_M1-solution_1.op2"
|
||||
)
|
||||
|
||||
# Add relative objectives (target vs reference)
|
||||
builder.add_relative_objective("40", "20", metric="relative_filtered_rms_nm", weight=5.0)
|
||||
builder.add_relative_objective("60", "20", metric="relative_filtered_rms_nm", weight=5.0)
|
||||
|
||||
# Add absolute objective for polishing orientation
|
||||
builder.add_subcase_objective("90", metric="rms_filter_j1to3", weight=1.0)
|
||||
|
||||
# Evaluate all at once (efficient - parses OP2 only once)
|
||||
results = builder.evaluate_all()
|
||||
# Returns: {'rel_40_vs_20': 4.2, 'rel_60_vs_20': 8.7, 'rms_90': 15.3}
|
||||
```
|
||||
|
||||
### PR.3 NXSolver Interface
|
||||
|
||||
**Module**: `optimization_engine.nx_solver`
|
||||
@@ -430,6 +486,136 @@ if __name__ == "__main__":
|
||||
main()
|
||||
```
|
||||
|
||||
### PR.11 Adding New Features to Atomizer Framework (REUSABILITY)
|
||||
|
||||
**CRITICAL: When developing extractors, calculators, or post-processing logic for a study, ALWAYS add them to the Atomizer framework for reuse!**
|
||||
|
||||
#### Why Reusability Matters
|
||||
|
||||
Each study should build on the framework, not duplicate code:
|
||||
- **WRONG**: Embed 500 lines of Zernike analysis in `run_optimization.py`
|
||||
- **CORRECT**: Create `extract_zernike.py` in `optimization_engine/extractors/` and import it
|
||||
|
||||
#### When to Create a New Extractor
|
||||
|
||||
Create a new extractor when:
|
||||
1. The study needs result extraction not covered by existing extractors (E1-E10)
|
||||
2. The logic is reusable across different studies
|
||||
3. The extraction involves non-trivial calculations (>20 lines of code)
|
||||
|
||||
#### Workflow for Adding New Extractors
|
||||
|
||||
```
|
||||
STEP 1: Check existing extractors in PR.1 Catalog
|
||||
├── If exists → IMPORT and USE it (done!)
|
||||
└── If missing → Continue to STEP 2
|
||||
|
||||
STEP 2: Create extractor in optimization_engine/extractors/
|
||||
├── File: extract_{feature}.py
|
||||
├── Follow existing extractor patterns
|
||||
└── Include comprehensive docstrings
|
||||
|
||||
STEP 3: Add to __init__.py
|
||||
└── Export functions in optimization_engine/extractors/__init__.py
|
||||
|
||||
STEP 4: Update this skill (create-study.md)
|
||||
├── Add to PR.1 Extractor Catalog table
|
||||
└── Add code snippet to PR.2
|
||||
|
||||
STEP 5: Document in CLAUDE.md (if major feature)
|
||||
└── Add to Available Extractors table
|
||||
```
|
||||
|
||||
#### New Extractor Template
|
||||
|
||||
```python
|
||||
"""
|
||||
{Feature} Extractor for Atomizer Optimization
|
||||
=============================================
|
||||
|
||||
Extract {description} from {input_type} files.
|
||||
|
||||
Usage:
|
||||
from optimization_engine.extractors.extract_{feature} import extract_{feature}
|
||||
|
||||
result = extract_{feature}(input_file, **params)
|
||||
"""
|
||||
|
||||
from pathlib import Path
|
||||
from typing import Dict, Any, Optional
|
||||
import logging
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def extract_{feature}(
|
||||
input_file: Path,
|
||||
param1: str = "default",
|
||||
**kwargs
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Extract {feature} from {input_type}.
|
||||
|
||||
Args:
|
||||
input_file: Path to input file
|
||||
param1: Description of param
|
||||
**kwargs: Additional parameters
|
||||
|
||||
Returns:
|
||||
Dict with keys:
|
||||
- primary_result: Main extracted value
|
||||
- metadata: Additional extraction info
|
||||
|
||||
Example:
|
||||
>>> result = extract_{feature}("model.op2")
|
||||
>>> print(result['primary_result'])
|
||||
"""
|
||||
# Implementation here
|
||||
pass
|
||||
|
||||
|
||||
# Export for __init__.py
|
||||
__all__ = ['extract_{feature}']
|
||||
```
|
||||
|
||||
#### Example: Adding Thermal Gradient Extractor
|
||||
|
||||
If a study needs thermal gradient analysis:
|
||||
|
||||
1. **Create**: `optimization_engine/extractors/extract_thermal_gradient.py`
|
||||
2. **Implement**: Functions for parsing thermal OP2 data
|
||||
3. **Export**: Add to `__init__.py`
|
||||
4. **Document**: Add E11 to catalog here
|
||||
5. **Use**: Import in `run_optimization.py`
|
||||
|
||||
```python
|
||||
# In run_optimization.py - CORRECT
|
||||
from optimization_engine.extractors.extract_thermal_gradient import extract_thermal_gradient
|
||||
|
||||
result = extract_thermal_gradient(op2_file, subcase=1)
|
||||
max_gradient = result['max_gradient_K_per_mm']
|
||||
```
|
||||
|
||||
#### NEVER Do This
|
||||
|
||||
```python
|
||||
# In run_optimization.py - WRONG!
|
||||
def calculate_thermal_gradient(op2_file, subcase):
|
||||
"""200 lines of thermal gradient calculation..."""
|
||||
# This should be in optimization_engine/extractors/!
|
||||
pass
|
||||
|
||||
result = calculate_thermal_gradient(op2_file, 1) # Not reusable!
|
||||
```
|
||||
|
||||
#### Updating This Skill After Adding Extractor
|
||||
|
||||
When you add a new extractor to the framework:
|
||||
|
||||
1. **PR.1**: Add row to Extractor Catalog table with ID, name, module, function, input, output, returns
|
||||
2. **PR.2**: Add code snippet showing usage
|
||||
3. **Common Patterns**: Add new pattern if this creates a new optimization type
|
||||
|
||||
---
|
||||
|
||||
## Document Philosophy
|
||||
@@ -608,10 +794,11 @@ An Atomizer study is a self-contained optimization project that combines:
|
||||
```
|
||||
studies/{study_name}/
|
||||
├── 1_setup/ # INPUT: Configuration & Model
|
||||
│ ├── model/ # NX Files
|
||||
│ │ ├── {Model}.prt # Parametric part (USER PROVIDES)
|
||||
│ │ ├── {Model}_sim1.sim # Simulation setup (USER PROVIDES)
|
||||
│ ├── model/ # WORKING COPY of NX Files (AUTO-COPIED)
|
||||
│ │ ├── {Model}.prt # Parametric part (COPIED FROM SOURCE)
|
||||
│ │ ├── {Model}_sim1.sim # Simulation setup (COPIED FROM SOURCE)
|
||||
│ │ ├── {Model}_fem1.fem # FEM mesh (AUTO-GENERATED)
|
||||
│ │ ├── {Model}_fem1_i.prt # Idealized part (COPIED if Assembly FEM)
|
||||
│ │ └── *.dat, *.op2, *.f06 # Solver outputs (AUTO-GENERATED)
|
||||
│ ├── optimization_config.json # Study configuration (SKILL GENERATES)
|
||||
│ └── workflow_config.json # Workflow metadata (SKILL GENERATES)
|
||||
@@ -624,6 +811,80 @@ studies/{study_name}/
|
||||
└── README.md # Study documentation (SKILL GENERATES)
|
||||
```
|
||||
|
||||
### CRITICAL: Model File Protection
|
||||
|
||||
**NEVER modify the user's original/master model files.** Always work on copies.
|
||||
|
||||
**Why This Matters**:
|
||||
- Optimization iteratively modifies expressions, meshes, and geometry
|
||||
- NX saves changes automatically - corruption during iteration can damage master files
|
||||
- Broken geometry/mesh can make files unrecoverable
|
||||
- Users may have months of CAD work in master files
|
||||
|
||||
**Mandatory Workflow**:
|
||||
```
|
||||
User's Source Files Study Working Copy
|
||||
─────────────────────────────────────────────────────────────
|
||||
C:/Projects/M1-Gigabit/Latest/ studies/{study_name}/1_setup/model/
|
||||
├── M1_Blank.prt ──► ├── M1_Blank.prt
|
||||
├── M1_Blank_fem1.fem ──► ├── M1_Blank_fem1.fem
|
||||
├── M1_Blank_fem1_i.prt ──► ├── M1_Blank_fem1_i.prt
|
||||
├── ASSY_*.afm ──► ├── ASSY_*.afm
|
||||
└── *_sim1.sim ──► └── *_sim1.sim
|
||||
|
||||
↓ COPY ALL FILES ↓
|
||||
|
||||
OPTIMIZATION RUNS ON WORKING COPY ONLY
|
||||
|
||||
↓ IF CORRUPTION ↓
|
||||
|
||||
Delete working copy, re-copy from source
|
||||
No damage to master files
|
||||
```
|
||||
|
||||
**Copy Script (generated in run_optimization.py)**:
|
||||
```python
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
|
||||
def setup_working_copy(source_dir: Path, model_dir: Path, file_patterns: list):
|
||||
"""
|
||||
Copy model files from user's source to study working directory.
|
||||
|
||||
Args:
|
||||
source_dir: Path to user's original model files (NEVER MODIFY)
|
||||
model_dir: Path to study's 1_setup/model/ directory (WORKING COPY)
|
||||
file_patterns: List of glob patterns to copy (e.g., ['*.prt', '*.fem', '*.sim'])
|
||||
"""
|
||||
model_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
for pattern in file_patterns:
|
||||
for src_file in source_dir.glob(pattern):
|
||||
dst_file = model_dir / src_file.name
|
||||
if not dst_file.exists() or src_file.stat().st_mtime > dst_file.stat().st_mtime:
|
||||
print(f"Copying: {src_file.name}")
|
||||
shutil.copy2(src_file, dst_file)
|
||||
|
||||
print(f"Working copy ready in: {model_dir}")
|
||||
```
|
||||
|
||||
**Assembly FEM Files to Copy**:
|
||||
| File Pattern | Purpose | Required |
|
||||
|--------------|---------|----------|
|
||||
| `*.prt` | Parametric geometry parts | Yes |
|
||||
| `*_fem1.fem` | Component FEM meshes | Yes |
|
||||
| `*_fem1_i.prt` | Idealized parts (geometry link) | Yes (if exists) |
|
||||
| `*.afm` | Assembly FEM | Yes (if assembly) |
|
||||
| `*_sim1.sim` | Simulation setup | Yes |
|
||||
| `*.exp` | Expression files | If used |
|
||||
|
||||
**When User Provides Source Path**:
|
||||
1. **ASK**: "Where are your model files?"
|
||||
2. **STORE**: Record source path in `optimization_config.json` as `"source_model_dir"`
|
||||
3. **COPY**: All relevant NX files to `1_setup/model/`
|
||||
4. **NEVER**: Point optimization directly at source files
|
||||
5. **DOCUMENT**: In README, show both source and working paths
|
||||
|
||||
### Optimization Trial Loop
|
||||
|
||||
Each optimization trial follows this execution flow:
|
||||
@@ -1753,16 +2014,110 @@ Design Variables: {list}
|
||||
Multi-Solution: {Yes/No}
|
||||
```
|
||||
|
||||
2. **File Checklist**:
|
||||
2. **File Checklist** (ALL MANDATORY):
|
||||
```
|
||||
✓ studies/{study_name}/1_setup/optimization_config.json
|
||||
✓ studies/{study_name}/1_setup/workflow_config.json
|
||||
✓ studies/{study_name}/run_optimization.py
|
||||
✓ studies/{study_name}/reset_study.py
|
||||
✓ studies/{study_name}/README.md
|
||||
✓ studies/{study_name}/README.md # Engineering blueprint
|
||||
✓ studies/{study_name}/STUDY_REPORT.md # MANDATORY - Results report template
|
||||
[✓] studies/{study_name}/NX_FILE_MODIFICATIONS_REQUIRED.md (if needed)
|
||||
```
|
||||
|
||||
### STUDY_REPORT.md - MANDATORY FOR ALL STUDIES
|
||||
|
||||
**CRITICAL**: Every study MUST have a STUDY_REPORT.md file. This is the living results document that gets updated as optimization progresses. Create it at study setup time with placeholder sections.
|
||||
|
||||
**Location**: `studies/{study_name}/STUDY_REPORT.md`
|
||||
|
||||
**Purpose**:
|
||||
- Documents optimization results as they come in
|
||||
- Tracks best solutions found
|
||||
- Records convergence history
|
||||
- Compares FEA vs Neural predictions (if applicable)
|
||||
- Provides engineering recommendations
|
||||
|
||||
**Template**:
|
||||
```markdown
|
||||
# {Study Name} - Optimization Report
|
||||
|
||||
**Study**: {study_name}
|
||||
**Created**: {date}
|
||||
**Status**: 🔄 In Progress / ✅ Complete
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
| Metric | Value |
|
||||
|--------|-------|
|
||||
| Total Trials | - |
|
||||
| FEA Trials | - |
|
||||
| NN Trials | - |
|
||||
| Best {Objective1} | - |
|
||||
| Best {Objective2} | - |
|
||||
|
||||
*Summary will be updated as optimization progresses.*
|
||||
|
||||
---
|
||||
|
||||
## 1. Optimization Progress
|
||||
|
||||
### Trial History
|
||||
| Trial | {Obj1} | {Obj2} | Source | Status |
|
||||
|-------|--------|--------|--------|--------|
|
||||
| - | - | - | - | - |
|
||||
|
||||
### Convergence
|
||||
*Convergence plots will be added after sufficient trials.*
|
||||
|
||||
---
|
||||
|
||||
## 2. Best Designs Found
|
||||
|
||||
### Best {Objective1}
|
||||
| Parameter | Value |
|
||||
|-----------|-------|
|
||||
| - | - |
|
||||
|
||||
### Best {Objective2}
|
||||
| Parameter | Value |
|
||||
|-----------|-------|
|
||||
| - | - |
|
||||
|
||||
### Pareto Front (if multi-objective)
|
||||
*Pareto solutions will be listed here.*
|
||||
|
||||
---
|
||||
|
||||
## 3. Neural Surrogate Performance (if applicable)
|
||||
|
||||
| Metric | {Obj1} | {Obj2} |
|
||||
|--------|--------|--------|
|
||||
| R² Score | - | - |
|
||||
| MAE | - | - |
|
||||
| Prediction Time | - | - |
|
||||
|
||||
---
|
||||
|
||||
## 4. Engineering Recommendations
|
||||
|
||||
*Recommendations based on optimization results.*
|
||||
|
||||
---
|
||||
|
||||
## 5. Next Steps
|
||||
|
||||
- [ ] Continue optimization
|
||||
- [ ] Validate best design with detailed FEA
|
||||
- [ ] Manufacturing review
|
||||
|
||||
---
|
||||
|
||||
*Report auto-generated by Atomizer. Last updated: {date}*
|
||||
```
|
||||
|
||||
3. **Next Steps** (as shown earlier)
|
||||
|
||||
## Remember
|
||||
|
||||
Reference in New Issue
Block a user