Atomizer/studies/m1_mirror_zernike_optimization/README.md

# M1 Mirror Zernike Optimization

Multi-objective telescope primary mirror support structure optimization using Zernike wavefront error decomposition with neural network acceleration.

**Created**: 2025-11-28
**Protocol**: Protocol 12 (Hybrid FEA/Neural with Zernike)
**Status**: Setup Complete - Requires Expression Path Fix

---

## 1. Engineering Problem

### 1.1 Objective

Optimize the telescope primary mirror (M1) support structure to minimize wavefront error (WFE) across different gravity orientations (zenith angles), ensuring consistent optical performance from 20° to 90° elevation.

### 1.2 Physical System

- **Component**: M1 primary mirror assembly with whiffle tree support
- **Material**: Borosilicate glass (mirror blank), steel (support structure)
- **Loading**: Gravity at multiple zenith angles (20°, 40°, 60°, 90°)
- **Boundary Conditions**: Whiffle tree kinematic mount
- **Analysis Type**: Linear static multi-subcase (Nastran SOL 101)
- **Output**: Surface deformation → Zernike polynomial decomposition

---

## 2. Mathematical Formulation

### 2.1 Objectives

| Objective | Goal | Weight | Formula | Units | Target |
|-----------|------|--------|---------|-------|--------|
| rel_filtered_rms_40_vs_20 | minimize | 5.0 | $\sigma_{40/20} = \sqrt{\sum_{j=5}^{50} (Z_j^{rel})^2}$ | nm | 4 nm |
| rel_filtered_rms_60_vs_20 | minimize | 5.0 | $\sigma_{60/20} = \sqrt{\sum_{j=5}^{50} (Z_j^{rel})^2}$ | nm | 10 nm |
| mfg_90_optician_workload | minimize | 1.0 | $\sigma_{90}^{J4+} = \sqrt{\sum_{j=4}^{50} (Z_j^{rel})^2}$ | nm | 20 nm |

Where:
- $Z_j^{rel}$ = Relative Zernike coefficient (target subcase minus reference)
- Filtered RMS excludes J1-J4 (piston, tip, tilt, defocus) - correctable by alignment
- Manufacturing workload keeps J4 (defocus) since it represents optician correction effort

### 2.2 Zernike Decomposition

The wavefront error $W(r,\theta)$ is decomposed into Zernike polynomials:

$$W(r,\theta) = \sum_{j=1}^{50} Z_j \cdot P_j(r,\theta)$$

Where $P_j$ are Noll-indexed Zernike polynomials on the unit disk.

**WFE from Displacement**:
$$W_{nm} = 2 \cdot \delta_z \cdot 10^6$$

Where $\delta_z$ is the Z-displacement in mm (factor of 2 for reflection).

### 2.3 Design Variables

| Parameter | Symbol | Bounds | Baseline | Units | Description |
|-----------|--------|--------|----------|-------|-------------|
| whiffle_min | $w_{min}$ | [35, 55] | 40.55 | mm | Whiffle tree minimum parameter |
| whiffle_outer_to_vertical | $\alpha$ | [68, 80] | 75.67 | deg | Outer support angle to vertical |
| inner_circular_rib_dia | $D_{rib}$ | [480, 620] | 534.00 | mm | Inner circular rib diameter |

**Design Space**:
$$\mathbf{x} = [w_{min}, \alpha, D_{rib}]^T \in \mathbb{R}^3$$

**Additional Variables (Disabled)**:
- lateral_inner_angle, lateral_outer_angle (lateral support angles)
- lateral_outer_pivot, lateral_inner_pivot, lateral_middle_pivot (pivot positions)
- lateral_closeness (lateral support spacing)
- whiffle_triangle_closeness (whiffle tree geometry)
- blank_backface_angle (mirror blank geometry)

### 2.4 Objective Strategy

**Weighted Sum Minimization**:
$$J(\mathbf{x}) = \sum_{i=1}^{3} w_i \cdot \frac{f_i(\mathbf{x})}{t_i}$$

Where:
- $w_i$ = weight for objective $i$
- $f_i(\mathbf{x})$ = objective value
- $t_i$ = target value (normalization)

---

## 3. Optimization Algorithm

### 3.1 TPE Configuration

| Parameter | Value | Description |
|-----------|-------|-------------|
| Algorithm | TPE | Tree-structured Parzen Estimator |
| Sampler | `TPESampler` | Bayesian optimization |
| n_startup_trials | 15 | Random exploration before modeling |
| n_ei_candidates | 150 | Expected improvement candidates |
| multivariate | true | Model parameter correlations |
| Trials | 100 | 40 FEA + neural acceleration |
| Seed | 42 | Reproducibility |

**TPE Properties**:
- Models $p(x|y<y^*)$ and $p(x|y \geq y^*)$ separately
- Expected Improvement: $EI(x) = \int_{-\infty}^{y^*} (y^* - y) p(y|x) dy$
- Handles high-dimensional continuous spaces efficiently

### 3.2 Return Format

```python
def fea_objective(trial) -> Tuple[float, dict]:
    # ... simulation and Zernike extraction ...
    weighted_obj = compute_weighted_objective(objectives, config)
    return weighted_obj, trial_data
```

---

## 4. Simulation Pipeline

### 4.1 Trial Execution Flow

```
┌─────────────────────────────────────────────────────────────────────┐
│                         TRIAL n EXECUTION                            │
├─────────────────────────────────────────────────────────────────────┤
│                                                                      │
│  1. OPTUNA SAMPLES (TPE)                                            │
│     whiffle_min = trial.suggest_float("whiffle_min", 35, 55)        │
│     whiffle_outer_to_vertical = trial.suggest_float(..., 68, 80)    │
│     inner_circular_rib_dia = trial.suggest_float(..., 480, 620)     │
│                                                                      │
│  2. NX PARAMETER UPDATE                                             │
│     Module: optimization_engine/solve_simulation.py                  │
│     Target Part: M1_Blank.prt                                        │
│     Action: Update expressions with new design values                │
│                                                                      │
│  3. NX SIMULATION (Nastran SOL 101 - 4 Subcases)                    │
│     Module: optimization_engine/solve_simulation.py                  │
│     Input: ASSY_M1_assyfem1_sim1.sim                                │
│     Subcases: 1=20°, 2=40°, 3=60°, 4=90° zenith                     │
│     Output: .dat, .op2, .f06                                        │
│                                                                      │
│  4. ZERNIKE EXTRACTION (Displacement-Based)                          │
│     a. Read node coordinates from BDF/DAT                           │
│     b. Read Z-displacements from OP2 for each subcase               │
│     c. Compute RELATIVE displacement (subcase - reference)           │
│     d. Convert to WFE: W = 2 * Δδz * 10^6 nm                        │
│     e. Fit 50 Zernike coefficients via least-squares                │
│     f. Compute filtered RMS (exclude J1-J4)                         │
│                                                                      │
│  5. OBJECTIVE COMPUTATION                                           │
│     rel_filtered_rms_40_vs_20 ← Zernike RMS (subcase 2 - 1)        │
│     rel_filtered_rms_60_vs_20 ← Zernike RMS (subcase 3 - 1)        │
│     mfg_90_optician_workload ← Zernike RMS J4+ (subcase 4 - 1)     │
│                                                                      │
│  6. WEIGHTED SUM                                                    │
│     J = Σ (weight × objective / target)                             │
│                                                                      │
│  7. RETURN TO OPTUNA                                                │
│     return weighted_objective                                        │
│                                                                      │
└─────────────────────────────────────────────────────────────────────┘
```

### 4.2 Multi-Subcase Structure

| Subcase | Zenith Angle | Role | Description |
|---------|--------------|------|-------------|
| 1 | 20° | Reference | Near-zenith baseline orientation |
| 2 | 40° | Target | Mid-elevation performance |
| 3 | 60° | Target | Low-elevation performance |
| 4 | 90° | Polishing | Horizontal (manufacturing reference) |

---

## 5. Result Extraction Methods

### 5.1 Zernike Extraction (Displacement-Based Subtraction)

| Attribute | Value |
|-----------|-------|
| **Method** | `extract_zernike_with_relative()` |
| **Location** | `run_optimization.py` (inline) |
| **Geometry Source** | `.dat` (BDF format) |
| **Displacement Source** | `.op2` (OP2 binary) |
| **Output** | 50 Zernike coefficients per subcase |

**Algorithm (Correct Approach - Matches Original Script)**:

1. **Load Geometry**: Read node coordinates $(X_i, Y_i)$ from BDF
2. **Load Displacements**: Read $\delta_{z,i}$ from OP2 for each subcase
3. **Compute Relative Displacement** (node-by-node):
   $$\Delta\delta_{z,i} = \delta_{z,i}^{target} - \delta_{z,i}^{reference}$$
4. **Convert to WFE**:
   $$W_i = 2 \cdot \Delta\delta_{z,i} \cdot 10^6 \text{ nm}$$
5. **Fit Zernike** (least-squares on unit disk):
   $$\min_{\mathbf{Z}} \| \mathbf{W} - \mathbf{P} \mathbf{Z} \|^2$$
6. **Compute RMS**:
   $$\sigma_{filtered} = \sqrt{\sum_{j=5}^{50} Z_j^2}$$

**Critical Implementation Note**:
The relative calculation MUST subtract displacements first, then fit Zernike - NOT subtract Zernike coefficients directly. This matches the original `zernike_Post_Script_NX.py` implementation.

### 5.2 Code Pattern

```python
from pyNastran.op2.op2 import OP2
from pyNastran.bdf.bdf import BDF

# Read geometry
bdf = BDF()
bdf.read_bdf(str(bdf_path))
node_geo = {nid: node.get_position() for nid, node in bdf.nodes.items()}

# Read displacements
op2 = OP2()
op2.read_op2(str(op2_path))

# Compute relative displacement (node-by-node)
for i, nid in enumerate(node_ids):
    rel_dz = disp_z_target[i] - disp_z_reference[nid]

# Convert to WFE and fit Zernike
rel_wfe_nm = 2.0 * rel_disp_z * 1e6
coeffs, R_max = compute_zernike_from_wfe(X, Y, rel_wfe_nm, n_modes=50)
```

---

## 6. Neural Acceleration (AtomizerField)

### 6.1 Configuration

| Setting | Value | Description |
|---------|-------|-------------|
| `enabled` | `true` | Neural surrogate active |
| `model_type` | `ParametricZernikePredictor` | Predicts Zernike coefficients |
| `hidden_channels` | 128 | MLP width |
| `num_layers` | 4 | MLP depth |
| `learning_rate` | 0.001 | Adam optimizer |
| `epochs` | 200 | Training iterations |
| `batch_size` | 8 | Mini-batch size |
| `train_split` | 0.8 | Training fraction |

### 6.2 Surrogate Model

**Input**: $\mathbf{x} = [w_{min}, \alpha, D_{rib}]^T \in \mathbb{R}^3$

**Output**: $\hat{\mathbf{Z}} \in \mathbb{R}^{200}$ (50 coefficients × 4 subcases)

**Architecture**: Multi-Layer Perceptron
```
Input(3) → Linear(128) → ReLU → Linear(128) → ReLU →
Linear(128) → ReLU → Linear(128) → ReLU → Linear(200)
```

**Training Objective**:
$$\mathcal{L} = \frac{1}{N} \sum_{i=1}^{N} \| \mathbf{Z}_i - \hat{\mathbf{Z}}_i \|^2$$

### 6.3 Training Data Location

```
studies/m1_mirror_zernike_optimization/2_results/zernike_surrogate/
├── checkpoint_best.pt          # Best model weights
├── training_history.json       # Loss curves
└── validation_metrics.json     # R², MAE per coefficient
```

### 6.4 Expected Performance

| Metric | Value |
|--------|-------|
| FEA time per trial | 10-15 min |
| Neural time per trial | ~10 ms |
| Speedup | ~60,000x |
| Expected R² | > 0.95 (after 40 samples) |

---

## 7. Study File Structure

```
m1_mirror_zernike_optimization/
│
├── 1_setup/                              # INPUT CONFIGURATION
│   ├── model/                            # NX Model Files (symlinked/referenced)
│   │   └── → C:\Users\Antoine\CADTOMASTE\Atomizer\M1-Gigabit\Latest\
│   │       ├── ASSY_M1.prt               # Top-level assembly
│   │       ├── M1_Blank.prt              # Mirror blank (EXPRESSIONS HERE)
│   │       ├── ASSY_M1_assyfem1.afm      # Assembly FEM
│   │       ├── ASSY_M1_assyfem1_sim1.sim # Simulation file
│   │       └── assy_m1_assyfem1_sim1-solution_1.op2  # Results
│   │
│   └── optimization_config.json          # Study configuration
│
├── 2_results/                            # OUTPUT (auto-generated)
│   ├── study.db                          # Optuna SQLite database
│   ├── zernike_surrogate/                # Neural model checkpoints
│   └── reports/                          # Generated reports
│
├── run_optimization.py                   # Main entry point
├── DASHBOARD.md                          # Quick reference
└── README.md                             # This blueprint
```

---

## 8. Results Location

After optimization completes, results are stored in `2_results/`:

| File | Description | Format |
|------|-------------|--------|
| `study.db` | Optuna database with all trials | SQLite |
| `zernike_surrogate/checkpoint_best.pt` | Trained neural model | PyTorch |
| `reports/optimization_report.md` | Full results report | Markdown |

### 8.1 Results Report Contents

The generated report will contain:

1. **Optimization Summary** - Best WFE configurations found
2. **Zernike Analysis** - Coefficient distributions per subcase
3. **Parameter Sensitivity** - Design variable vs WFE relationships
4. **Convergence History** - Weighted objective over trials
5. **Neural Surrogate Performance** - R² per Zernike mode
6. **Recommended Configurations** - Top designs for production

### 8.2 Zernike-Specific Analysis

| Mode | Name | Physical Meaning |
|------|------|------------------|
| J1 | Piston | Constant offset (ignored) |
| J2, J3 | Tip/Tilt | Angular misalignment (correctable) |
| J4 | Defocus | Power error (correctable) |
| J5, J6 | Astigmatism | Cylindrical error |
| J7, J8 | Coma | Off-axis aberration |
| J9-J11 | Trefoil, Spherical | Higher-order terms |

---

## 9. Quick Start

### Staged Workflow (Recommended)

```bash
cd studies/m1_mirror_zernike_optimization

# Check current status
python run_optimization.py --status

# Run FEA trials (builds training data)
python run_optimization.py --run --trials 40

# Train neural surrogate
python run_optimization.py --train-surrogate

# Run neural-accelerated optimization
python run_optimization.py --run --trials 500 --enable-nn
```

### Stage Descriptions

| Stage | Command | Purpose | When to Use |
|-------|---------|---------|-------------|
| **STATUS** | `--status` | Check database, trial count | Anytime |
| **RUN** | `--run --trials N` | Run FEA optimization | Initial exploration |
| **TRAIN** | `--train-surrogate` | Train neural model | After ~40 FEA trials |
| **NEURAL** | `--run --enable-nn` | Fast neural trials | After training |

### Dashboard Access

| Dashboard | URL | Purpose |
|-----------|-----|---------|
| **Optuna Dashboard** | `optuna-dashboard sqlite:///2_results/study.db` | Trial history |

```bash
# Launch Optuna dashboard
cd studies/m1_mirror_zernike_optimization
optuna-dashboard sqlite:///2_results/study.db --port 8081
# Open http://localhost:8081
```

---

## 10. Configuration Reference

**File**: `1_setup/optimization_config.json`

| Section | Key | Description |
|---------|-----|-------------|
| `design_variables[]` | 11 parameters | 3 enabled, 8 disabled |
| `objectives[]` | 3 WFE metrics | Relative filtered RMS |
| `zernike_settings.n_modes` | 50 | Zernike polynomial count |
| `zernike_settings.filter_low_orders` | 4 | Exclude J1-J4 |
| `zernike_settings.subcases` | ["1","2","3","4"] | OP2 subcase IDs |
| `zernike_settings.reference_subcase` | "1" | 20° baseline |
| `optimization_settings.n_trials` | 100 | Total FEA trials |
| `surrogate_settings.model_type` | ParametricZernikePredictor | Neural architecture |
| `nx_settings.model_dir` | M1-Gigabit/Latest | NX model location |
| `nx_settings.sim_file` | ASSY_M1_assyfem1_sim1.sim | Simulation file |

---

## 11. Known Issues & Solutions

### 11.1 Expression Update Failure

**Issue**: NX journal cannot find expressions in assembly FEM.

**Cause**: Expressions are in component part `M1_Blank.prt`, not in `ASSY_M1_assyfem1`.

**Solution**: The `solve_simulation.py` journal now searches for `M1_Blank` part to update expressions. If still failing, verify:
1. `M1_Blank.prt` is loaded in the assembly
2. Expression names match exactly (case-sensitive)
3. Part is not read-only

### 11.2 Subcase Numbering

**Issue**: OP2 file uses numeric subcases (1,2,3,4) not angle labels (20,40,60,90).

**Solution**: Config uses `subcases: ["1","2","3","4"]` with `subcase_labels` mapping.

---

## 12. References

- **Noll, R.J.** (1976). Zernike polynomials and atmospheric turbulence. *JOSA*.
- **Wilson, R.N.** (2004). *Reflecting Telescope Optics I*. Springer.
- **pyNastran Documentation**: BDF/OP2 parsing for FEA post-processing
- **Optuna Documentation**: TPE sampler for black-box optimization