Atomizer/docs/06_PROTOCOLS_DETAILED/ZERNIKE_MIRROR_OPTIMIZATION.md

# Zernike Mirror Optimization Protocol

## Overview

This document captures the learnings from the M1 mirror Zernike optimization studies (V1-V9), including the Assembly FEM (AFEM) workflow, subcase handling, and wavefront error metrics.

## Assembly FEM (AFEM) Structure

### NX File Organization

A typical telescope mirror assembly in NX consists of:

```
ASSY_M1.prt                      # Master assembly part
ASSY_M1_assyfem1.afm             # Assembly FEM container
ASSY_M1_assyfem1_sim1.sim        # Simulation file (this is what we solve)
M1_Blank.prt                     # Mirror blank part
M1_Blank_fem1.fem                # Mirror blank mesh
M1_Blank_fem1_i.prt              # Idealized geometry for FEM
M1_Vertical_Support_Skeleton.prt # Support structure part
M1_Vertical_Support_Skeleton_fem1.fem
M1_Vertical_Support_Skeleton_fem1_i.prt
```

### Key Relationships

1. **Assembly Part (.prt)** - Contains the CAD geometry and expressions (design parameters)
2. **Assembly FEM (.afm)** - Links component FEMs together, defines connections
3. **Simulation (.sim)** - Contains solutions, loads, boundary conditions, subcases
4. **Component FEMs (.fem)** - Individual meshes that get assembled

### Expression Propagation

Expressions defined in the master `.prt` propagate through the assembly:
- Modify expression in `ASSY_M1.prt`
- AFEM updates mesh connections automatically
- Solve via `.sim` file

## Multi-Subcase Analysis

### Telescope Gravity Orientations

For telescope mirrors, we analyze multiple gravity orientations (subcases):

| Subcase | Elevation Angle | Purpose |
|---------|-----------------|---------|
| 1 | 90 deg (zenith) | Polishing orientation - manufacturing reference |
| 2 | 20 deg | Low elevation - reference for relative metrics |
| 3 | 40 deg | Mid-low elevation |
| 4 | 60 deg | Mid-high elevation |

### Subcase Mapping

**Important**: NX subcase numbers don't always match angle labels!

```json
"subcase_labels": {
  "1": "90deg",   // Subcase 1 = 90 degrees
  "2": "20deg",   // Subcase 2 = 20 degrees (reference)
  "3": "40deg",   // Subcase 3 = 40 degrees
  "4": "60deg"    // Subcase 4 = 60 degrees
}
```

Always verify subcase-to-angle mapping by checking the NX simulation setup.

## Zernike Wavefront Error Analysis

### Optical Convention

For mirror surface deformation to wavefront error:
```
WFE = 2 * surface_displacement  (reflection doubles the path difference)
```

Unit conversion:
```python
NM_PER_MM = 1e6  # 1 mm displacement = 1e6 nm WFE contribution
wfe_nm = 2.0 * displacement_mm * 1e6
```

### Zernike Polynomial Indexing

We use **Noll indexing** (standard in optics):

| J | Name | (n,m) | Correctable? |
|---|------|-------|--------------|
| 1 | Piston | (0,0) | Yes - alignment |
| 2 | Tilt X | (1,-1) | Yes - alignment |
| 3 | Tilt Y | (1,1) | Yes - alignment |
| 4 | Defocus | (2,0) | Yes - focus adjustment |
| 5 | Astigmatism 45 | (2,-2) | Partially |
| 6 | Astigmatism 0 | (2,2) | Partially |
| 7 | Coma X | (3,-1) | No |
| 8 | Coma Y | (3,1) | No |
| 9 | Trefoil X | (3,-3) | No |
| 10 | Trefoil Y | (3,3) | No |
| 11 | Spherical | (4,0) | No |

### RMS Metrics

| Metric | Filter | Use Case |
|--------|--------|----------|
| `global_rms_nm` | None | Total surface error |
| `filtered_rms_nm` | J1-J4 removed | Uncorrectable error (optimization target) |
| `rms_filter_j1to3` | J1-J3 removed | Optician workload (keeps defocus) |

### Relative Metrics

For gravity-induced deformation, we compute relative WFE:
```
WFE_relative = WFE_target_orientation - WFE_reference_orientation
```

This removes the static (manufacturing) shape and isolates gravity effects.

Example: `rel_filtered_rms_40_vs_20` = filtered RMS at 40 deg relative to 20 deg reference

## Optimization Objectives

### Typical M1 Mirror Objectives

```json
"objectives": [
  {
    "name": "rel_filtered_rms_40_vs_20",
    "description": "Gravity-induced WFE at 40 deg vs 20 deg reference",
    "direction": "minimize",
    "weight": 5.0,
    "target": 4.0,
    "units": "nm"
  },
  {
    "name": "rel_filtered_rms_60_vs_20",
    "description": "Gravity-induced WFE at 60 deg vs 20 deg reference",
    "direction": "minimize",
    "weight": 5.0,
    "target": 10.0,
    "units": "nm"
  },
  {
    "name": "mfg_90_optician_workload",
    "description": "Polishing effort at zenith (J1-J3 filtered)",
    "direction": "minimize",
    "weight": 1.0,
    "target": 20.0,
    "units": "nm"
  }
]
```

### Weighted Sum Formulation

```python
weighted_objective = sum(weight_i * (value_i / target_i)) / sum(weight_i)
```

Targets normalize different metrics to comparable scales.

## Design Variables

### Typical Mirror Support Parameters

| Parameter | Description | Typical Range |
|-----------|-------------|---------------|
| `whiffle_min` | Whiffle tree minimum dimension | 35-55 mm |
| `whiffle_outer_to_vertical` | Whiffle arm angle | 68-80 deg |
| `whiffle_triangle_closeness` | Triangle geometry | 50-65 mm |
| `inner_circular_rib_dia` | Rib diameter | 480-620 mm |
| `lateral_inner_angle` | Lateral support angle | 25-28.5 deg |
| `blank_backface_angle` | Mirror blank geometry | 3.5-5.0 deg |

### Expression File Format (params.exp)

```
[mm]whiffle_min=42.49
[Degrees]whiffle_outer_to_vertical=79.41
[mm]inner_circular_rib_dia=582.48
```

## Iteration Folder Structure (V9)

```
study_name/
├── 1_setup/
│   ├── model/              # Master NX files (NEVER modify)
│   └── optimization_config.json
├── 2_iterations/
│   ├── iter0/              # Trial 0 (0-based to match Optuna)
│   │   ├── [all NX files]  # Fresh copy from master
│   │   ├── params.exp      # Expression updates for this trial
│   │   └── results/        # Processed outputs
│   ├── iter1/
│   └── ...
└── 3_results/
    └── study.db            # Optuna database
```

### Why 0-Based Iteration Folders?

Optuna uses 0-based trial numbers. Using `iter{trial.number}` ensures:
- Dashboard shows Trial 0 -> corresponds to folder iter0
- No confusion when cross-referencing results
- Consistent indexing throughout the system

## Lessons Learned

### 1. TPE Sampler Seed Issue

**Problem**: When resuming a study, re-initializing TPESampler with a fixed seed causes the sampler to restart its random sequence, generating duplicate parameters.

**Solution**: Only set seed for NEW studies:
```python
if is_new_study:
    sampler = TPESampler(seed=42, ...)
else:
    sampler = TPESampler(...)  # No seed for resume
```

### 2. Code Reuse Protocol

**Problem**: Embedding 500+ lines of Zernike code in `run_optimization.py` violates DRY principle.

**Solution**: Use centralized extractors:
```python
from optimization_engine.extractors import ZernikeExtractor

extractor = ZernikeExtractor(op2_file)
result = extractor.extract_relative("3", "2")
rms = result['relative_filtered_rms_nm']
```

### 3. Subcase Numbering

**Problem**: NX subcase numbers (1,2,3,4) don't match angle labels (20,40,60,90).

**Solution**: Use explicit mapping in config and translate:
```python
subcase_labels = {"1": "90deg", "2": "20deg", "3": "40deg", "4": "60deg"}
label_to_subcase = {v: k for k, v in subcase_labels.items()}
```

### 4. OP2 Data Validation

**Problem**: Corrupt OP2 files can have all-zero or unrealistic displacement values.

**Solution**: Validate before processing:
```python
unique_values = len(np.unique(disp_z))
if unique_values < 10:
    raise RuntimeError("CORRUPT OP2: insufficient unique values")

if np.abs(disp_z).max() > 1e6:
    raise RuntimeError("CORRUPT OP2: unrealistic displacement magnitude")
```

### 5. Reference Subcase for Relative Metrics

**Problem**: Which orientation to use as reference?

**Solution**: Use the lowest operational elevation (typically 20 deg) as reference. This makes higher elevations show positive relative WFE as gravity effects increase.

## ZernikeExtractor API Reference

### Basic Usage

```python
from optimization_engine.extractors import ZernikeExtractor

# Create extractor
extractor = ZernikeExtractor(
    op2_path="path/to/results.op2",
    bdf_path=None,  # Auto-detect from same folder
    displacement_unit="mm",
    n_modes=50,
    filter_orders=4
)

# Single subcase
result = extractor.extract_subcase("2")
# Returns: global_rms_nm, filtered_rms_nm, rms_filter_j1to3, aberrations...

# Relative between subcases
rel = extractor.extract_relative(target_subcase="3", reference_subcase="2")
# Returns: relative_filtered_rms_nm, relative_rms_filter_j1to3, ...

# All subcases with relative metrics
all_results = extractor.extract_all_subcases(reference_subcase="2")
```

### Available Metrics

| Method | Returns |
|--------|---------|
| `extract_subcase()` | global_rms_nm, filtered_rms_nm, rms_filter_j1to3, defocus_nm, astigmatism_rms_nm, coma_rms_nm, trefoil_rms_nm, spherical_nm |
| `extract_relative()` | relative_global_rms_nm, relative_filtered_rms_nm, relative_rms_filter_j1to3, relative aberrations |
| `extract_all_subcases()` | Dict of all subcases with both absolute and relative metrics |

## Configuration Template

```json
{
  "study_name": "m1_mirror_optimization",

  "design_variables": [
    {
      "name": "whiffle_min",
      "expression_name": "whiffle_min",
      "min": 35.0,
      "max": 55.0,
      "baseline": 40.55,
      "units": "mm",
      "enabled": true
    }
  ],

  "objectives": [
    {
      "name": "rel_filtered_rms_40_vs_20",
      "extractor": "zernike_relative",
      "extractor_config": {
        "target_subcase": "3",
        "reference_subcase": "2",
        "metric": "relative_filtered_rms_nm"
      },
      "direction": "minimize",
      "weight": 5.0,
      "target": 4.0
    }
  ],

  "zernike_settings": {
    "n_modes": 50,
    "filter_low_orders": 4,
    "displacement_unit": "mm",
    "subcases": ["1", "2", "3", "4"],
    "subcase_labels": {"1": "90deg", "2": "20deg", "3": "40deg", "4": "60deg"},
    "reference_subcase": "2"
  },

  "optimization_settings": {
    "sampler": "TPE",
    "seed": 42,
    "n_startup_trials": 15
  }
}
```

## Version History

| Version | Key Changes |
|---------|-------------|
| V1-V6 | Initial development, various folder structures |
| V7 | HEEDS-style iteration folders, fresh model copies |
| V8 | Autonomous NX session management, but had embedded Zernike code |
| V9 | Clean ZernikeExtractor integration, fixed sampler seed, 0-based folders |