feat: Major update - Physics docs, Zernike OPD, insights, NX journals, tools
Documentation: - Add docs/06_PHYSICS/ with Zernike fundamentals and OPD method docs - Add docs/guides/CMA-ES_EXPLAINED.md optimization guide - Update CLAUDE.md and ATOMIZER_CONTEXT.md with current architecture - Update OP_01_CREATE_STUDY protocol Planning: - Add DYNAMIC_RESPONSE plans for random vibration/PSD support - Add OPTIMIZATION_ENGINE_MIGRATION_PLAN for code reorganization Insights System: - Update design_space, modal_analysis, stress_field, thermal_field insights - Improve error handling and data validation NX Journals: - Add analyze_wfe_zernike.py for Zernike WFE analysis - Add capture_study_images.py for automated screenshots - Add extract_expressions.py and introspect_part.py utilities - Add user_generated_journals/journal_top_view_image_taking.py Tests & Tools: - Add comprehensive Zernike OPD test suite - Add audit_v10 tests for WFE validation - Add tools for Pareto graphs and mirror data extraction - Add migrate_studies_to_topics.py utility Knowledge Base: - Initialize LAC (Learning Atomizer Core) with failure/success patterns Dashboard: - Update Setup.tsx and launch_dashboard.py - Add restart-dev.bat helper script 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
59
docs/06_PHYSICS/00_INDEX.md
Normal file
59
docs/06_PHYSICS/00_INDEX.md
Normal file
@@ -0,0 +1,59 @@
|
||||
# Physics Documentation Index
|
||||
|
||||
This folder contains detailed physics and domain-specific documentation for Atomizer's analysis capabilities. These documents explain the **engineering and scientific foundations** behind the extractors and insights.
|
||||
|
||||
---
|
||||
|
||||
## Document Catalog
|
||||
|
||||
| Document | Topic | When to Read |
|
||||
|----------|-------|--------------|
|
||||
| [ZERNIKE_FUNDAMENTALS.md](ZERNIKE_FUNDAMENTALS.md) | Zernike polynomial basics, RMS calculation, multi-subcase analysis | Setting up mirror optimization, understanding WFE metrics |
|
||||
| [ZERNIKE_OPD_METHOD.md](ZERNIKE_OPD_METHOD.md) | **Rigorous OPD method** for lateral displacement correction | Lateral support optimization, validating WFE accuracy |
|
||||
|
||||
---
|
||||
|
||||
## Quick Navigation
|
||||
|
||||
### For Mirror/Optics Optimization
|
||||
|
||||
1. **New to Zernike?** Start with [ZERNIKE_FUNDAMENTALS.md](ZERNIKE_FUNDAMENTALS.md)
|
||||
2. **Lateral support optimization?** Read [ZERNIKE_OPD_METHOD.md](ZERNIKE_OPD_METHOD.md) - **critical**
|
||||
3. **Mid-spatial frequency analysis?** See SYS_16 (msf_zernike insight)
|
||||
|
||||
### For Structural Optimization
|
||||
|
||||
- Stress extraction: See `SYS_12_EXTRACTOR_LIBRARY.md` (E3, E12)
|
||||
- Strain energy: See `SYS_12_EXTRACTOR_LIBRARY.md` (E13)
|
||||
|
||||
### For Thermal Analysis
|
||||
|
||||
- Temperature extraction: See `SYS_12_EXTRACTOR_LIBRARY.md` (E15-E17)
|
||||
|
||||
---
|
||||
|
||||
## Related Documentation
|
||||
|
||||
| Location | Content |
|
||||
|----------|---------|
|
||||
| `.claude/skills/modules/extractors-catalog.md` | Quick extractor lookup |
|
||||
| `.claude/skills/modules/insights-catalog.md` | Quick insight lookup |
|
||||
| `docs/protocols/system/SYS_12_EXTRACTOR_LIBRARY.md` | Extractor specifications |
|
||||
| `docs/protocols/system/SYS_16_STUDY_INSIGHTS.md` | Insight specifications |
|
||||
|
||||
---
|
||||
|
||||
## Contributing Physics Documentation
|
||||
|
||||
When adding new physics documentation:
|
||||
|
||||
1. **Naming**: Use `{TOPIC}_{SUBTOPIC}.md` format (e.g., `THERMAL_GRADIENTS.md`)
|
||||
2. **Structure**: Follow the pattern in existing documents:
|
||||
- Executive Summary
|
||||
- Mathematical Formulation
|
||||
- When This Matters
|
||||
- Implementation Details
|
||||
- Usage Guide
|
||||
- Validation
|
||||
3. **Cross-reference**: Update this index and related skill modules
|
||||
4. **Link to code**: Reference the implementing extractors/insights
|
||||
@@ -308,6 +308,26 @@ studies/
|
||||
|
||||
## See Also
|
||||
|
||||
### Related Physics Documentation
|
||||
|
||||
- [ZERNIKE_OPD_METHOD.md](ZERNIKE_OPD_METHOD.md) - **Rigorous OPD method for lateral displacement correction** (critical for lateral support optimization)
|
||||
|
||||
### Protocol Documentation
|
||||
|
||||
- `docs/protocols/system/SYS_12_EXTRACTOR_LIBRARY.md` - Extractor specifications (E8-E10: Standard Zernike, E20-E21: OPD method)
|
||||
- `docs/protocols/system/SYS_16_STUDY_INSIGHTS.md` - Insight specifications (`zernike_wfe`, `zernike_opd_comparison`)
|
||||
|
||||
### Skill Modules (Quick Lookup)
|
||||
|
||||
- `.claude/skills/modules/extractors-catalog.md` - Quick extractor reference
|
||||
- `.claude/skills/modules/insights-catalog.md` - Quick insight reference
|
||||
|
||||
### Code Implementation
|
||||
|
||||
- [optimization_engine/extractors/extract_zernike.py](../../optimization_engine/extractors/extract_zernike.py) - Standard Zernike extractor
|
||||
- [optimization_engine/extractors/extract_zernike_opd.py](../../optimization_engine/extractors/extract_zernike_opd.py) - **OPD-based extractor** (use for lateral supports)
|
||||
- [optimization_engine/extractors/zernike_helpers.py](../../optimization_engine/extractors/zernike_helpers.py) - Helper functions and objective builders
|
||||
|
||||
### Example Configurations
|
||||
|
||||
- [examples/optimization_config_zernike_mirror.json](../examples/optimization_config_zernike_mirror.json) - Full example configuration
|
||||
- [optimization_engine/extractors/extract_zernike.py](../optimization_engine/extractors/extract_zernike.py) - Core implementation
|
||||
- [optimization_engine/extractors/zernike_helpers.py](../optimization_engine/extractors/zernike_helpers.py) - Helper functions
|
||||
579
docs/06_PHYSICS/ZERNIKE_OPD_METHOD.md
Normal file
579
docs/06_PHYSICS/ZERNIKE_OPD_METHOD.md
Normal file
@@ -0,0 +1,579 @@
|
||||
# Rigorous OPD-Based Zernike Analysis for Mirror Optimization
|
||||
|
||||
**Document Version**: 1.0
|
||||
**Created**: 2024-12-22
|
||||
**Author**: Atomizer Framework
|
||||
**Status**: Active
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
This document describes a **rigorous Optical Path Difference (OPD)** method for computing Zernike wavefront error that correctly accounts for **lateral (X, Y) displacements** in addition to axial (Z) displacements.
|
||||
|
||||
**The Problem**: Standard Zernike analysis uses only Z-displacement at the original (x, y) node positions. When supports pinch the mirror or lateral forces cause in-plane deformation, nodes shift in X and Y. The standard method is **blind to this**, potentially leading to:
|
||||
- Optimized designs that appear good but have poor actual optical performance
|
||||
- Optimizer convergence to non-optimal solutions that "cheat" by distorting laterally
|
||||
|
||||
**The Solution**: The OPD method computes the true surface error by accounting for the fact that a laterally-displaced node should be compared against the parabola height **at its new (x+dx, y+dy) position**, not its original position.
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [The Optical Physics Problem](#1-the-optical-physics-problem)
|
||||
2. [Mathematical Formulation](#2-mathematical-formulation)
|
||||
3. [When This Matters](#3-when-this-matters)
|
||||
4. [Implementation Details](#4-implementation-details)
|
||||
5. [Usage Guide](#5-usage-guide)
|
||||
6. [Validation and Testing](#6-validation-and-testing)
|
||||
7. [Migration Guide](#7-migration-guide)
|
||||
|
||||
---
|
||||
|
||||
## 1. The Optical Physics Problem
|
||||
|
||||
### 1.1 What Zernike Analysis Does
|
||||
|
||||
Zernike polynomials decompose a wavefront error surface into orthogonal modes:
|
||||
|
||||
```
|
||||
W(r, θ) = Σ cⱼ Zⱼ(r, θ)
|
||||
```
|
||||
|
||||
Where:
|
||||
- `W` = wavefront error (nm)
|
||||
- `cⱼ` = Zernike coefficient for mode j
|
||||
- `Zⱼ` = Zernike polynomial (Noll indexing)
|
||||
|
||||
For a reflective mirror, the wavefront error is **twice** the surface error:
|
||||
```
|
||||
WFE = 2 × surface_error
|
||||
```
|
||||
|
||||
### 1.2 Standard Method (Z-Only)
|
||||
|
||||
The standard approach:
|
||||
1. Read node original positions `(x₀, y₀, z₀)` from BDF/DAT
|
||||
2. Read displacement vector `(Δx, Δy, Δz)` from OP2
|
||||
3. Compute surface error = `Δz` (Z-displacement only)
|
||||
4. Compute WFE = `2 × Δz × nm_scale`
|
||||
5. Fit Zernike at original coordinates `(x₀, y₀)`
|
||||
|
||||
```python
|
||||
# Standard method (simplified)
|
||||
for nid, (dx, dy, dz) in displacements:
|
||||
x, y, z = original_coords[nid]
|
||||
wfe = dz * 2 * nm_scale # ONLY uses Z-displacement
|
||||
X.append(x) # Original X
|
||||
Y.append(y) # Original Y
|
||||
WFE.append(wfe)
|
||||
|
||||
coeffs = fit_zernike(X, Y, WFE)
|
||||
```
|
||||
|
||||
### 1.3 The Problem: Lateral Displacement is Ignored
|
||||
|
||||
Consider a node on a parabolic mirror:
|
||||
- **Original position**: `(x₀, y₀, z₀)` where `z₀ = -r₀²/(4f)` on the parabola
|
||||
- **Deformed position**: `(x₀+Δx, y₀+Δy, z₀+Δz)`
|
||||
|
||||
**Question**: What is the true surface error?
|
||||
|
||||
**Standard method says**: surface_error = `Δz`
|
||||
|
||||
**But this is wrong!** If the node moved laterally to a new `(x, y)`, the ideal parabola has a **different** Z at that location. The node should be compared against:
|
||||
|
||||
```
|
||||
z_expected = parabola(x₀+Δx, y₀+Δy) = -(x₀+Δx)² + (y₀+Δy)² / (4f)
|
||||
```
|
||||
|
||||
Not against `z₀ = parabola(x₀, y₀)`.
|
||||
|
||||
### 1.4 Visual Example
|
||||
|
||||
```
|
||||
Original parabola
|
||||
___
|
||||
_/ \_
|
||||
/ \
|
||||
/ *A \ A = original node at (x₀, y₀, z₀)
|
||||
/ ↗ ↘ \ B = deformed position (x₀+Δx, y₀+Δy, z₀+Δz)
|
||||
/ B C \ C = where node SHOULD be if staying on parabola
|
||||
/ \
|
||||
/_____________________\
|
||||
|
||||
Standard method: error = z_B - z_A = Δz
|
||||
(compares B to A vertically)
|
||||
|
||||
OPD method: error = z_B - z_C = Δz - Δz_parabola
|
||||
(compares B to where parabola is at B's (x,y))
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. Mathematical Formulation
|
||||
|
||||
### 2.1 Differential OPD Formulation
|
||||
|
||||
For a paraboloid with optical axis along Z:
|
||||
|
||||
```
|
||||
z = -r² / (4f) [concave mirror, vertex at origin]
|
||||
```
|
||||
|
||||
Where:
|
||||
- `r² = x² + y²`
|
||||
- `f` = focal length
|
||||
|
||||
**Key Insight**: We can compute the **change** in parabola Z due to lateral movement:
|
||||
|
||||
```
|
||||
Δz_parabola = z(x₀+Δx, y₀+Δy) - z(x₀, y₀)
|
||||
= -[(x₀+Δx)² + (y₀+Δy)²] / (4f) - [-( x₀² + y₀²) / (4f)]
|
||||
= -[r_def² - r₀²] / (4f)
|
||||
= -Δr² / (4f)
|
||||
```
|
||||
|
||||
Where:
|
||||
```
|
||||
Δr² = r_def² - r₀² = (x₀+Δx)² + (y₀+Δy)² - x₀² - y₀²
|
||||
= 2·x₀·Δx + Δx² + 2·y₀·Δy + Δy²
|
||||
```
|
||||
|
||||
### 2.2 True Surface Error
|
||||
|
||||
The true surface error is:
|
||||
|
||||
```
|
||||
surface_error = Δz - Δz_parabola
|
||||
= Δz - (-Δr² / 4f)
|
||||
= Δz + Δr² / (4f)
|
||||
```
|
||||
|
||||
**Interpretation**:
|
||||
- If a node moves **outward** (larger r), it should also move in **-Z** to stay on the concave parabola
|
||||
- If the FEA says it moved by `Δz`, but staying on the parabola requires `Δz_parabola`, the difference is the true error
|
||||
- This corrects for the "false error" that the standard method counts when nodes shift laterally
|
||||
|
||||
### 2.3 Wavefront Error
|
||||
|
||||
```
|
||||
WFE = 2 × surface_error × nm_scale
|
||||
= 2 × (Δz - Δz_parabola) × nm_scale
|
||||
```
|
||||
|
||||
### 2.4 Zernike Fitting Coordinates
|
||||
|
||||
Another subtlety: the Zernike fit should use the **deformed** coordinates `(x₀+Δx, y₀+Δy)` rather than the original coordinates. This is because the WFE surface represents the error at the positions where the nodes **actually are** after deformation.
|
||||
|
||||
```python
|
||||
# OPD method
|
||||
X_fit = x0 + dx # Deformed X
|
||||
Y_fit = y0 + dy # Deformed Y
|
||||
WFE = surface_error * 2 * nm_scale
|
||||
|
||||
coeffs = fit_zernike(X_fit, Y_fit, WFE)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. When This Matters
|
||||
|
||||
### 3.1 Magnitude Analysis
|
||||
|
||||
The correction term is:
|
||||
|
||||
```
|
||||
Δz_parabola = -Δr² / (4f) ≈ -(2·x₀·Δx + 2·y₀·Δy) / (4f) [ignoring Δx², Δy²]
|
||||
≈ -(x₀·Δx + y₀·Δy) / (2f)
|
||||
```
|
||||
|
||||
For a node at radius `r₀` with tangential displacement `Δ_tangential`:
|
||||
- The correction is approximately: `r₀ · Δ_lateral / (2f)`
|
||||
|
||||
**Example**: Mirror with f = 5000 mm, outer radius = 400 mm
|
||||
- Node at r = 400 mm shifts laterally by Δx = 0.001 mm (1 µm)
|
||||
- Correction: `400 × 0.001 / (2 × 5000) = 0.00004 mm = 40 nm`
|
||||
|
||||
This is **significant** when typical WFE is in the 10-100 nm range!
|
||||
|
||||
### 3.2 Classification by Load Case
|
||||
|
||||
| Load Case | Lateral Disp. | Method Impact |
|
||||
|-----------|--------------|---------------|
|
||||
| **Axial support** (gravity in Z) | Very small | Minimal - both methods similar |
|
||||
| **Lateral support** (gravity in X/Y) | **Large** | **Significant** - OPD method required |
|
||||
| **Clamp/fixture forces** | Can be large locally | May be significant at pinch points |
|
||||
| **Thermal** | Variable | Depends on thermal gradients |
|
||||
| **Mirror cell deflection** | Variable | Check lateral displacement magnitude |
|
||||
|
||||
### 3.3 Diagnostic Thresholds
|
||||
|
||||
The `ZernikeOPDExtractor` provides lateral displacement statistics:
|
||||
|
||||
| Max Lateral Disp. | Recommendation |
|
||||
|-------------------|----------------|
|
||||
| > 10 µm | **CRITICAL**: OPD method required |
|
||||
| 1 - 10 µm | **RECOMMENDED**: OPD method provides meaningful improvement |
|
||||
| 0.1 - 1 µm | **OPTIONAL**: OPD method provides minor improvement |
|
||||
| < 0.1 µm | **EQUIVALENT**: Both methods give essentially identical results |
|
||||
|
||||
---
|
||||
|
||||
## 4. Implementation Details
|
||||
|
||||
### 4.1 Module: `extract_zernike_opd.py`
|
||||
|
||||
Location: `optimization_engine/extractors/extract_zernike_opd.py`
|
||||
|
||||
**Key Functions**:
|
||||
|
||||
```python
|
||||
def compute_true_opd(x0, y0, z0, dx, dy, dz, focal_length, concave=True):
|
||||
"""
|
||||
Compute true surface error accounting for lateral displacement.
|
||||
|
||||
Returns:
|
||||
x_def: Deformed X coordinates
|
||||
y_def: Deformed Y coordinates
|
||||
surface_error: True surface error (not just Δz)
|
||||
lateral_magnitude: |Δx, Δy| for diagnostics
|
||||
"""
|
||||
```
|
||||
|
||||
```python
|
||||
def estimate_focal_length_from_geometry(x, y, z, concave=True):
|
||||
"""
|
||||
Estimate parabola focal length by fitting z = a·r² + b.
|
||||
Focal length = 1 / (4·|a|)
|
||||
"""
|
||||
```
|
||||
|
||||
**Main Class**:
|
||||
|
||||
```python
|
||||
class ZernikeOPDExtractor:
|
||||
"""
|
||||
Rigorous OPD-based Zernike extractor.
|
||||
|
||||
Key differences from ZernikeExtractor:
|
||||
- Uses deformed (x, y) coordinates for fitting
|
||||
- Computes surface error relative to parabola at deformed position
|
||||
- Provides lateral displacement diagnostics
|
||||
"""
|
||||
```
|
||||
|
||||
### 4.2 Algorithm Flow
|
||||
|
||||
```
|
||||
1. Load geometry (BDF) and displacements (OP2)
|
||||
|
||||
2. For each node:
|
||||
a. Get original position: (x₀, y₀, z₀)
|
||||
b. Get displacement: (Δx, Δy, Δz)
|
||||
c. Compute deformed position: (x_def, y_def) = (x₀+Δx, y₀+Δy)
|
||||
d. Compute Δr² = r_def² - r₀²
|
||||
e. Compute Δz_parabola = -Δr² / (4f) [for concave]
|
||||
f. Compute surface_error = Δz - Δz_parabola
|
||||
g. Store lateral_disp = √(Δx² + Δy²)
|
||||
|
||||
3. Convert to WFE: WFE = 2 × surface_error × nm_scale
|
||||
|
||||
4. Fit Zernike coefficients using (x_def, y_def, WFE)
|
||||
|
||||
5. Compute RMS metrics:
|
||||
- Global RMS = √(mean(WFE²))
|
||||
- Filtered RMS = √(mean((WFE - low_order_fit)²))
|
||||
```
|
||||
|
||||
### 4.3 Focal Length Handling
|
||||
|
||||
The extractor can:
|
||||
1. Use a **provided** focal length (most accurate)
|
||||
2. **Auto-estimate** from geometry by fitting `z = a·r² + b`
|
||||
|
||||
Auto-estimation works well for clean parabolic meshes but may need manual override for:
|
||||
- Off-axis parabolas
|
||||
- Aspheric surfaces
|
||||
- Meshes with significant manufacturing errors
|
||||
|
||||
```python
|
||||
# Explicit focal length
|
||||
extractor = ZernikeOPDExtractor(op2_file, focal_length=5000.0)
|
||||
|
||||
# Auto-estimate (default)
|
||||
extractor = ZernikeOPDExtractor(op2_file, auto_estimate_focal=True)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Usage Guide
|
||||
|
||||
### 5.1 Quick Comparison Test
|
||||
|
||||
Run the test script to see how much the methods differ for your data:
|
||||
|
||||
```bash
|
||||
conda activate atomizer
|
||||
python test_zernike_opd_comparison.py
|
||||
```
|
||||
|
||||
Output example:
|
||||
```
|
||||
--- Standard Method (Z-only) ---
|
||||
Global RMS: 171.65 nm
|
||||
Filtered RMS: 28.72 nm
|
||||
|
||||
--- Rigorous OPD Method ---
|
||||
Global RMS: 171.89 nm
|
||||
Filtered RMS: 29.15 nm
|
||||
|
||||
--- Difference (OPD - Standard) ---
|
||||
Filtered RMS: +0.43 nm (+1.5%)
|
||||
|
||||
--- Lateral Displacement ---
|
||||
Max: 0.156 µm
|
||||
RMS: 0.111 µm
|
||||
|
||||
>>> OPTIONAL: Small lateral displacements. OPD method provides minor improvement.
|
||||
```
|
||||
|
||||
### 5.2 Using in Optimization
|
||||
|
||||
**For new studies**, use the OPD extractor:
|
||||
|
||||
```python
|
||||
from optimization_engine.extractors import extract_zernike_opd_filtered_rms
|
||||
|
||||
def objective(trial):
|
||||
# ... parameter suggestion and FEA solve ...
|
||||
|
||||
# Use OPD method instead of standard
|
||||
rms = extract_zernike_opd_filtered_rms(
|
||||
op2_file,
|
||||
subcase='20',
|
||||
focal_length=5000.0 # Optional: specify or let it auto-estimate
|
||||
)
|
||||
return rms
|
||||
```
|
||||
|
||||
**In optimization config** (future enhancement):
|
||||
|
||||
```json
|
||||
{
|
||||
"objectives": [
|
||||
{
|
||||
"name": "filtered_rms",
|
||||
"extractor": "zernike_opd",
|
||||
"extractor_config": {
|
||||
"subcase": "20",
|
||||
"metric": "filtered_rms_nm",
|
||||
"focal_length": 5000.0
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### 5.3 Visualization with Insights
|
||||
|
||||
Generate the comparison insight for a study:
|
||||
|
||||
```bash
|
||||
python -m optimization_engine.insights generate studies/my_study --type zernike_opd_comparison
|
||||
```
|
||||
|
||||
This creates an HTML visualization showing:
|
||||
1. **Lateral displacement map** - Where pinching/lateral deformation occurs
|
||||
2. **WFE surface** - Using the rigorous OPD method
|
||||
3. **Comparison table** - Quantitative difference between methods
|
||||
4. **Recommendation** - Whether OPD method is needed for your study
|
||||
|
||||
### 5.4 API Reference
|
||||
|
||||
```python
|
||||
from optimization_engine.extractors import (
|
||||
# Main extractor class
|
||||
ZernikeOPDExtractor,
|
||||
|
||||
# Convenience functions
|
||||
extract_zernike_opd, # Full metrics dict
|
||||
extract_zernike_opd_filtered_rms, # Just the filtered RMS (float)
|
||||
compare_zernike_methods, # Compare standard vs OPD
|
||||
)
|
||||
|
||||
# Full extraction with all metrics
|
||||
result = extract_zernike_opd(op2_file, subcase='20')
|
||||
# Returns: {
|
||||
# 'filtered_rms_nm': float,
|
||||
# 'global_rms_nm': float,
|
||||
# 'max_lateral_disp_um': float,
|
||||
# 'rms_lateral_disp_um': float,
|
||||
# 'focal_length_used': float,
|
||||
# 'astigmatism_rms_nm': float,
|
||||
# 'coma_rms_nm': float,
|
||||
# ...
|
||||
# }
|
||||
|
||||
# Just the primary metric for optimization
|
||||
rms = extract_zernike_opd_filtered_rms(op2_file, subcase='20')
|
||||
|
||||
# Compare both methods
|
||||
comparison = compare_zernike_methods(op2_file, subcase='20')
|
||||
# Returns: {
|
||||
# 'standard_method': {'filtered_rms_nm': ...},
|
||||
# 'opd_method': {'filtered_rms_nm': ...},
|
||||
# 'delta': {'filtered_rms_nm': ..., 'percent_difference_filtered': ...},
|
||||
# 'lateral_displacement': {'max_um': ..., 'rms_um': ...},
|
||||
# 'recommendation': str
|
||||
# }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Validation and Testing
|
||||
|
||||
### 6.1 Analytical Test Case
|
||||
|
||||
For a simple test: apply a known lateral displacement and verify the correction.
|
||||
|
||||
**Setup**:
|
||||
- Parabola: f = 5000 mm
|
||||
- Node at (x₀, y₀) = (400, 0) mm, so r₀ = 400 mm
|
||||
- Apply uniform X-displacement: Δx = 0.01 mm, Δy = 0, Δz = 0
|
||||
|
||||
**Expected correction**:
|
||||
```
|
||||
Δr² = (400.01)² + 0² - 400² - 0² = 8.0001 mm²
|
||||
Δz_parabola = -8.0001 / (4 × 5000) = -0.0004 mm = -400 nm (surface)
|
||||
WFE_correction = 2 × 400 nm = 800 nm
|
||||
```
|
||||
|
||||
**Standard method**: WFE = 2 × Δz × 1e6 = 0 nm
|
||||
**OPD method**: WFE = 2 × (0 - (-0.0004)) × 1e6 = 800 nm
|
||||
|
||||
The OPD method correctly identifies that a purely lateral displacement **does** affect the wavefront!
|
||||
|
||||
### 6.2 Sanity Checks
|
||||
|
||||
The OPD method should:
|
||||
1. Give **identical** results to standard method when Δx = Δy = 0 everywhere
|
||||
2. Show **larger** WFE when nodes move outward laterally (positive Δr²)
|
||||
3. Show **smaller** WFE when nodes move inward laterally (negative Δr²)
|
||||
4. Scale with 1/f (larger effect for faster mirrors)
|
||||
|
||||
### 6.3 Running the Test
|
||||
|
||||
```bash
|
||||
conda activate atomizer
|
||||
python test_zernike_opd_comparison.py
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. Migration Guide
|
||||
|
||||
### 7.1 For Existing Studies
|
||||
|
||||
1. **Run comparison test** on a few representative iterations
|
||||
2. **Check the difference** - if > 5%, consider re-optimizing
|
||||
3. **For lateral support studies** - strongly recommend re-optimization with OPD method
|
||||
|
||||
### 7.2 For New Studies
|
||||
|
||||
1. **Use OPD method by default** - it's never worse than standard
|
||||
2. **Specify focal length** if known (more accurate than auto-estimate)
|
||||
3. **Monitor lateral displacement** in the insight reports
|
||||
|
||||
### 7.3 Code Changes
|
||||
|
||||
**Before** (standard method):
|
||||
```python
|
||||
from optimization_engine.extractors import extract_zernike_filtered_rms
|
||||
|
||||
rms = extract_zernike_filtered_rms(op2_file, subcase='20')
|
||||
```
|
||||
|
||||
**After** (OPD method):
|
||||
```python
|
||||
from optimization_engine.extractors import extract_zernike_opd_filtered_rms
|
||||
|
||||
rms = extract_zernike_opd_filtered_rms(op2_file, subcase='20', focal_length=5000.0)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Appendix A: Derivation Details
|
||||
|
||||
### A.1 Full Derivation of Δz_parabola
|
||||
|
||||
For a concave paraboloid: `z = -r²/(4f) = -(x² + y²)/(4f)`
|
||||
|
||||
Original position: `z₀ = -(x₀² + y₀²)/(4f)`
|
||||
Deformed position: `z_expected = -((x₀+Δx)² + (y₀+Δy)²)/(4f)`
|
||||
|
||||
```
|
||||
z_expected - z₀ = -[(x₀+Δx)² + (y₀+Δy)² - x₀² - y₀²] / (4f)
|
||||
= -[x₀² + 2x₀Δx + Δx² + y₀² + 2y₀Δy + Δy² - x₀² - y₀²] / (4f)
|
||||
= -[2x₀Δx + Δx² + 2y₀Δy + Δy²] / (4f)
|
||||
= -[r_def² - r₀²] / (4f)
|
||||
= -Δr² / (4f)
|
||||
```
|
||||
|
||||
This is `Δz_parabola` - the Z change required to stay on the ideal parabola.
|
||||
|
||||
### A.2 Sign Convention
|
||||
|
||||
For a **concave** mirror (typical telescope primary):
|
||||
- Surface curves toward -Z (vertex is the highest point)
|
||||
- `z = -r²/(4f)` (negative coefficient)
|
||||
- Moving outward (Δr² > 0) requires moving in -Z direction
|
||||
- `Δz_parabola = -Δr²/(4f)` is negative for outward movement
|
||||
|
||||
For a **convex** mirror:
|
||||
- Surface curves toward +Z
|
||||
- `z = +r²/(4f)` (positive coefficient)
|
||||
- `Δz_parabola = +Δr²/(4f)` is positive for outward movement
|
||||
|
||||
The `concave` parameter in the code handles this sign flip.
|
||||
|
||||
---
|
||||
|
||||
## Appendix B: Files Reference
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `optimization_engine/extractors/extract_zernike_opd.py` | Main OPD extractor implementation |
|
||||
| `optimization_engine/extractors/extract_zernike.py` | Standard (Z-only) extractor |
|
||||
| `optimization_engine/insights/zernike_opd_comparison.py` | Visualization insight |
|
||||
| `test_zernike_opd_comparison.py` | Quick test script |
|
||||
|
||||
### Related Documentation
|
||||
|
||||
| Document | Purpose |
|
||||
|----------|---------|
|
||||
| [ZERNIKE_FUNDAMENTALS.md](ZERNIKE_FUNDAMENTALS.md) | General Zernike usage, RMS calculation, multi-subcase analysis |
|
||||
| [00_INDEX.md](00_INDEX.md) | Physics documentation index |
|
||||
| `.claude/skills/modules/extractors-catalog.md` | Quick extractor lookup |
|
||||
| `.claude/skills/modules/insights-catalog.md` | Quick insight lookup |
|
||||
| `docs/protocols/system/SYS_12_EXTRACTOR_LIBRARY.md` | Extractor specifications (E8-E10, E20-E21) |
|
||||
| `docs/protocols/system/SYS_16_STUDY_INSIGHTS.md` | Insight specifications |
|
||||
|
||||
---
|
||||
|
||||
## Appendix C: Glossary
|
||||
|
||||
| Term | Definition |
|
||||
|------|------------|
|
||||
| **OPD** | Optical Path Difference - the path length difference experienced by light rays |
|
||||
| **WFE** | Wavefront Error - deviation of actual wavefront from ideal (WFE = 2 × surface error for reflection) |
|
||||
| **Zernike polynomials** | Orthogonal basis functions for representing wavefronts over a circular aperture |
|
||||
| **Noll index** | Standard optical indexing scheme for Zernike modes (j=1 is piston, j=4 is defocus, etc.) |
|
||||
| **Filtered RMS** | RMS after removing low-order modes (piston, tip, tilt, defocus) that can be corrected by alignment |
|
||||
| **Lateral displacement** | In-plane (X, Y) movement of nodes, as opposed to axial (Z) movement |
|
||||
| **Focal length** | Distance from vertex to focus for a parabola; f = R/(2) where R is vertex radius of curvature |
|
||||
|
||||
---
|
||||
|
||||
*Document maintained by Atomizer Framework. Last updated: 2024-12-22*
|
||||
212
docs/guides/CMA-ES_EXPLAINED.md
Normal file
212
docs/guides/CMA-ES_EXPLAINED.md
Normal file
@@ -0,0 +1,212 @@
|
||||
# CMA-ES Explained for Engineers
|
||||
|
||||
**CMA-ES** = **Covariance Matrix Adaptation Evolution Strategy**
|
||||
|
||||
A derivative-free optimization algorithm ideal for:
|
||||
- Local refinement around known good solutions
|
||||
- 4-10 dimensional problems
|
||||
- Smooth, continuous objective functions
|
||||
- Problems where gradient information is unavailable (like FEA)
|
||||
|
||||
---
|
||||
|
||||
## The Core Idea
|
||||
|
||||
Imagine searching for the lowest point in a hilly landscape while blindfolded:
|
||||
|
||||
1. **Throw darts** around your current best guess
|
||||
2. **Observe which darts land lower** (better objective)
|
||||
3. **Learn the shape of the valley** from those results
|
||||
4. **Adjust future throws** to follow the valley's direction
|
||||
|
||||
---
|
||||
|
||||
## Key Components
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ CMA-ES Components │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ 1. MEAN (μ) - Current best guess location │
|
||||
│ • Moves toward better solutions each generation │
|
||||
│ │
|
||||
│ 2. STEP SIZE (σ) - How far to throw darts │
|
||||
│ • Adapts: shrinks when close, grows when exploring │
|
||||
│ • sigma0=0.3 means 30% of parameter range initially │
|
||||
│ │
|
||||
│ 3. COVARIANCE MATRIX (C) - Shape of the search cloud │
|
||||
│ • Learns parameter correlations │
|
||||
│ • Stretches search along promising directions │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Visual: How the Search Evolves
|
||||
|
||||
```
|
||||
Generation 1 (Round search): Generation 10 (Learned shape):
|
||||
|
||||
x x x
|
||||
x x x x
|
||||
x ● x ──────► x ● x
|
||||
x x x x
|
||||
x x x
|
||||
|
||||
● = mean (center) Ellipse aligned with
|
||||
x = samples the valley direction
|
||||
```
|
||||
|
||||
CMA-ES learns that certain parameter combinations work well together and stretches its search cloud in that direction.
|
||||
|
||||
---
|
||||
|
||||
## The Algorithm (Simplified)
|
||||
|
||||
```python
|
||||
def cma_es_generation():
|
||||
# 1. SAMPLE: Generate λ candidates around the mean
|
||||
for i in range(population_size):
|
||||
candidates[i] = mean + sigma * sample_from_gaussian(covariance=C)
|
||||
|
||||
# 2. EVALUATE: Run FEA for each candidate
|
||||
for candidate in candidates:
|
||||
fitness[candidate] = run_simulation(candidate)
|
||||
|
||||
# 3. SELECT: Keep the best μ candidates
|
||||
selected = top_k(candidates, by=fitness, k=mu)
|
||||
|
||||
# 4. UPDATE MEAN: Move toward the best solutions
|
||||
new_mean = weighted_average(selected)
|
||||
|
||||
# 5. UPDATE COVARIANCE: Learn parameter correlations
|
||||
C = update_covariance(C, selected, mean, new_mean)
|
||||
|
||||
# 6. UPDATE STEP SIZE: Adapt exploration range
|
||||
sigma = adapt_step_size(sigma, evolution_path)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## The Covariance Matrix Magic
|
||||
|
||||
Consider 4 design variables:
|
||||
|
||||
```
|
||||
Covariance Matrix C (4x4):
|
||||
var1 var2 var3 var4
|
||||
var1 [ 1.0 0.3 -0.5 0.1 ]
|
||||
var2 [ 0.3 1.0 0.2 -0.2 ]
|
||||
var3 [-0.5 0.2 1.0 0.4 ]
|
||||
var4 [ 0.1 -0.2 0.4 1.0 ]
|
||||
```
|
||||
|
||||
**Reading the matrix:**
|
||||
- **Diagonal (1.0)**: Variance in each parameter
|
||||
- **Off-diagonal**: Correlations between parameters
|
||||
- **Positive (0.3)**: When var1 increases, var2 should increase
|
||||
- **Negative (-0.5)**: When var1 increases, var3 should decrease
|
||||
|
||||
CMA-ES **learns these correlations automatically** from simulation results!
|
||||
|
||||
---
|
||||
|
||||
## CMA-ES vs TPE
|
||||
|
||||
| Property | TPE | CMA-ES |
|
||||
|----------|-----|--------|
|
||||
| **Best for** | Global exploration | Local refinement |
|
||||
| **Starting point** | Random | Known baseline |
|
||||
| **Correlation learning** | None (independent) | Automatic |
|
||||
| **Step size** | Fixed ranges | Adaptive |
|
||||
| **Dimensionality** | Good for high-D | Best for 4-10D |
|
||||
| **Sample efficiency** | Good | Excellent (locally) |
|
||||
|
||||
---
|
||||
|
||||
## Optuna Configuration
|
||||
|
||||
```python
|
||||
from optuna.samplers import CmaEsSampler
|
||||
|
||||
# Baseline values (starting point)
|
||||
x0 = {
|
||||
'whiffle_min': 62.75,
|
||||
'whiffle_outer_to_vertical': 75.89,
|
||||
'whiffle_triangle_closeness': 65.65,
|
||||
'blank_backface_angle': 4.43
|
||||
}
|
||||
|
||||
sampler = CmaEsSampler(
|
||||
x0=x0, # Center of initial distribution
|
||||
sigma0=0.3, # Initial step size (30% of range)
|
||||
seed=42, # Reproducibility
|
||||
restart_strategy='ipop' # Increase population on restart
|
||||
)
|
||||
|
||||
study = optuna.create_study(sampler=sampler, direction="minimize")
|
||||
|
||||
# CRITICAL: Enqueue baseline as trial 0!
|
||||
# x0 only sets the CENTER, it doesn't evaluate the baseline
|
||||
study.enqueue_trial(x0)
|
||||
|
||||
study.optimize(objective, n_trials=200)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
### 1. Not Evaluating the Baseline
|
||||
|
||||
**Problem**: CMA-ES samples AROUND x0, but doesn't evaluate x0 itself.
|
||||
|
||||
**Solution**: Always enqueue the baseline:
|
||||
```python
|
||||
if len(study.trials) == 0:
|
||||
study.enqueue_trial(x0)
|
||||
```
|
||||
|
||||
### 2. sigma0 Too Large or Too Small
|
||||
|
||||
| sigma0 | Effect |
|
||||
|--------|--------|
|
||||
| **Too large (>0.5)** | Explores too far, misses local optimum |
|
||||
| **Too small (<0.1)** | Gets stuck, slow convergence |
|
||||
| **Recommended (0.2-0.3)** | Good balance for refinement |
|
||||
|
||||
### 3. Wrong Problem Type
|
||||
|
||||
CMA-ES struggles with:
|
||||
- Discrete/categorical variables
|
||||
- Very high dimensions (>20)
|
||||
- Multi-modal landscapes (use TPE first)
|
||||
- Noisy objectives (add regularization)
|
||||
|
||||
---
|
||||
|
||||
## When to Use CMA-ES in Atomizer
|
||||
|
||||
| Scenario | Use CMA-ES? |
|
||||
|----------|-------------|
|
||||
| First exploration of design space | No, use TPE |
|
||||
| Refining around known good design | **Yes** |
|
||||
| 4-10 continuous variables | **Yes** |
|
||||
| >15 variables | No, use TPE or NSGA-II |
|
||||
| Need to learn variable correlations | **Yes** |
|
||||
| Multi-objective optimization | No, use NSGA-II |
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- Hansen, N. (2016). The CMA Evolution Strategy: A Tutorial
|
||||
- Optuna CmaEsSampler: https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.CmaEsSampler.html
|
||||
- cmaes Python package: https://github.com/CyberAgentAILab/cmaes
|
||||
|
||||
---
|
||||
|
||||
*Created: 2025-12-19*
|
||||
*Atomizer Framework*
|
||||
@@ -71,6 +71,71 @@ When creating a new study:
|
||||
|
||||
---
|
||||
|
||||
## README Hierarchy (Parent-Child Documentation)
|
||||
|
||||
**Two-level documentation system**:
|
||||
|
||||
```
|
||||
studies/{geometry_type}/
|
||||
├── README.md # PARENT: Project-level context
|
||||
│ ├── Project overview # What is this geometry/component?
|
||||
│ ├── Physical system specs # Material, dimensions, constraints
|
||||
│ ├── Optical/mechanical specs # Domain-specific requirements
|
||||
│ ├── Design variables catalog # ALL possible variables with descriptions
|
||||
│ ├── Objectives catalog # ALL possible objectives
|
||||
│ ├── Campaign history # Summary of all sub-studies
|
||||
│ └── Sub-studies index # Links to each sub-study
|
||||
│
|
||||
├── sub_study_V1/
|
||||
│ └── README.md # CHILD: Study-specific details
|
||||
│ ├── Link to parent # "See ../README.md for context"
|
||||
│ ├── Study focus # What THIS study optimizes
|
||||
│ ├── Active variables # Which params enabled
|
||||
│ ├── Algorithm config # Sampler, trials, settings
|
||||
│ ├── Baseline/seeding # Starting point
|
||||
│ └── Results summary # Best trial, learnings
|
||||
│
|
||||
└── sub_study_V2/
|
||||
└── README.md # CHILD: References parent, adds specifics
|
||||
```
|
||||
|
||||
### Parent README Content (Geometry-Level)
|
||||
|
||||
| Section | Content |
|
||||
|---------|---------|
|
||||
| Project Overview | What the component is, purpose, context |
|
||||
| Physical System | Material, mass targets, loading conditions |
|
||||
| Domain Specs | Optical prescription (mirrors), structural limits (brackets) |
|
||||
| Design Variables | Complete catalog with ranges and descriptions |
|
||||
| Objectives | All possible metrics with formulas |
|
||||
| Campaign History | Evolution across sub-studies |
|
||||
| Sub-Studies Index | Table with links, status, best results |
|
||||
| Technical Notes | Domain-specific implementation details |
|
||||
|
||||
### Child README Content (Study-Level)
|
||||
|
||||
| Section | Content |
|
||||
|---------|---------|
|
||||
| Parent Reference | `> See [../README.md](../README.md) for project context` |
|
||||
| Study Focus | What differentiates THIS study |
|
||||
| Active Variables | Which parameters are enabled (subset of parent catalog) |
|
||||
| Algorithm Config | Sampler, n_trials, sigma, seed |
|
||||
| Baseline | Starting point (seeded from prior study or default) |
|
||||
| Results | Best trial, improvement metrics |
|
||||
| Key Learnings | What was discovered |
|
||||
|
||||
### When to Create Parent README
|
||||
|
||||
- **First study** for a geometry type → Create parent README immediately
|
||||
- **Subsequent studies** → Add to parent's sub-studies index
|
||||
- **New geometry type** → Create both parent and child READMEs
|
||||
|
||||
### Example Reference
|
||||
|
||||
See `studies/M1_Mirror/README.md` for a complete parent README example.
|
||||
|
||||
---
|
||||
|
||||
## Detailed Steps
|
||||
|
||||
### Step 1: Gather Requirements
|
||||
|
||||
Reference in New Issue
Block a user