Atomizer/studies/m1_mirror_adaptive_V15/STUDY_REPORT.md

# M1 Mirror Adaptive V15 - NSGA-II Multi-Objective Optimization Report

**Study**: m1_mirror_adaptive_V15
**Algorithm**: NSGA-II (Multi-Objective Genetic Algorithm)
**Status**: Completed
**Created**: 2025-12-12
**Completed**: 2025-12-15

---

## Executive Summary

V15 applied NSGA-II multi-objective optimization to explore the Pareto front of design trade-offs, seeded with 494 valid trials from V14. The campaign completed 126 new FEA evaluations but **did not find improvements** over the V14-optimized designs.

| Metric | Value |
|--------|-------|
| Total Trials in DB | 644 |
| Seeded from V14 | 494 |
| New V15 FEA Trials | 126 |
| Failed Trials | 24 |
| Pareto Front Size | 2 |
| **Best Weighted Sum** | **121.72** (from V14 seed) |
| Best V15 FEA Weighted Sum | 127.35 |

**Key Finding**: NSGA-II exploration confirmed that V14 TPE optimization had already converged to the optimal region. No V15 FEA trial improved upon the seeded V14 data.

---

## 1. Optimization Configuration

### 1.1 Algorithm Settings

| Parameter | Value |
|-----------|-------|
| Algorithm | NSGA-II |
| Population Size | 50 |
| Crossover Probability | 0.9 |
| Mutation | Adaptive |
| Seed | 42 |

### 1.2 Design Variables (11 total)

| Variable | Min | Max | Baseline | Best Value | Units |
|----------|-----|-----|----------|------------|-------|
| `lateral_inner_angle` | 25.0 | 30.0 | 26.79 | **27.88** | deg |
| `lateral_outer_angle` | 11.0 | 17.0 | 14.64 | **13.19** | deg |
| `lateral_outer_pivot` | 9.0 | 12.0 | 10.40 | **11.53** | mm |
| `lateral_inner_pivot` | 5.0 | 12.0 | 10.07 | **7.41** | mm |
| `lateral_middle_pivot` | 15.0 | 27.0 | 20.73 | **22.95** | mm |
| `lateral_closeness` | 9.5 | 12.5 | 11.02 | **9.93** | mm |
| `whiffle_min` | 30.0 | 72.0 | 40.55 | **58.90** | mm |
| `whiffle_outer_to_vertical` | 60.0 | 80.0 | 75.67 | **77.84** | deg |
| `whiffle_triangle_closeness` | 50.0 | 80.0 | 60.00 | **66.88** | mm |
| `blank_backface_angle` | 4.1 | 4.5 | 4.15 | **4.30** | deg |
| `inner_circular_rib_dia` | 480.0 | 620.0 | 534.00 | **505.89** | mm |

### 1.3 Objectives

| Objective | Description | Target | Direction | Weight |
|-----------|-------------|--------|-----------|--------|
| `rel_filtered_rms_40_vs_20` | 40° tracking error vs 20° ref | 4.0 nm | minimize | 5 |
| `rel_filtered_rms_60_vs_20` | 60° tracking error vs 20° ref | 10.0 nm | minimize | 5 |
| `mfg_90_optician_workload` | Manufacturing deformation (J1-J3 filtered) | 20.0 nm | minimize | 1 |

**Weighted Sum Formula**: `5 × obj_40 + 5 × obj_60 + 1 × obj_mfg`

---

## 2. Results Analysis

### 2.1 Best Design (Global - from V14 Seed)

**Trial #445** (originally V14 Trial #725)

| Objective | Value | Target | Status |
|-----------|-------|--------|--------|
| 40° vs 20° tracking | **5.99 nm** | 4.0 nm | ❌ Above target |
| 60° vs 20° tracking | **13.10 nm** | 10.0 nm | ❌ Above target |
| Manufacturing (90°) | **26.28 nm** | 20.0 nm | ❌ Above target |
| **Weighted Sum** | **121.72** | - | Best achieved |

### 2.2 Best V15 FEA Design

**Trial #631** (New V15 FEA evaluation)

| Objective | Value | vs Best | Delta |
|-----------|-------|---------|-------|
| 40° vs 20° tracking | 5.92 nm | 5.99 nm | -1.2% ✓ |
| 60° vs 20° tracking | 13.78 nm | 13.10 nm | +5.2% ✗ |
| Manufacturing (90°) | 28.83 nm | 26.28 nm | +9.7% ✗ |
| **Weighted Sum** | **127.35** | **121.72** | +4.6% ✗ |

### 2.3 Top 10 Designs Overall

| Rank | Trial | Source | 40°/20° (nm) | 60°/20° (nm) | Mfg (nm) | WS |
|------|-------|--------|--------------|--------------|----------|-----|
| 1 | #445 | V14_FEA_725 | 5.99 | 13.10 | 26.28 | **121.72** |
| 2 | #444 | V14_FEA_724 | 6.05 | 13.17 | 26.42 | 122.54 |
| 3 | #274 | V14_FEA_550 | 5.76 | 13.27 | 27.53 | 122.69 |
| 4 | #440 | V14_FEA_720 | 6.09 | 13.34 | 26.96 | 124.12 |
| 5 | #438 | V14_FEA_716 | 6.08 | 13.37 | 27.17 | 124.42 |
| 6 | #271 | V14_FEA_547 | 5.81 | 13.43 | 28.36 | 124.57 |
| 7 | #290 | V14_FEA_566 | 5.85 | 13.48 | 28.03 | 124.66 |
| 8 | #275 | V14_FEA_551 | 5.83 | 13.50 | 28.06 | 124.67 |
| 9 | #487 | V14_FEA_778 | 6.24 | 13.41 | 26.55 | 124.77 |
| 10 | #364 | V14_FEA_642 | 5.89 | 13.50 | 27.95 | 124.91 |

**Observation**: All top 10 designs are from V14 seeded data. No V15 FEA trial made it into the top 10.

### 2.4 Top 5 V15 FEA Trials

| Rank | Trial | 40°/20° (nm) | 60°/20° (nm) | Mfg (nm) | WS |
|------|-------|--------------|--------------|----------|-----|
| 1 | #631 | 5.92 | 13.78 | 28.83 | 127.35 |
| 2 | #540 | 6.60 | 14.08 | 27.82 | 131.21 |
| 3 | #621 | 6.01 | 14.10 | 31.08 | 131.63 |
| 4 | #570 | 6.07 | 14.31 | 31.35 | 133.28 |
| 5 | #624 | 5.98 | 14.47 | 32.22 | 134.45 |

### 2.5 Pareto Front Analysis

The true Pareto front (non-dominated solutions) contains only **2 designs**:

| Trial | Source | 40°/20° (nm) | 60°/20° (nm) | Mfg (nm) | WS |
|-------|--------|--------------|--------------|----------|-----|
| #445 | V14_FEA_725 | 5.99 | 13.10 | 26.28 | 121.72 |
| #274 | V14_FEA_550 | 5.76 | 13.27 | 27.53 | 122.69 |

**Interpretation**: The small Pareto front indicates the objectives are not strongly conflicting in the optimal region. Trial #274 trades slightly worse 60° performance and manufacturing for better 40° performance.

---

## 3. Source Distribution

| Source | Count | Percentage |
|--------|-------|------------|
| V14 Seeded | 494 | 76.7% |
| V15 FEA (new) | 126 | 19.6% |
| V15 FEA (failed) | 24 | 3.7% |
| **Total** | **644** | 100% |

---

## 4. Why NSGA-II Did Not Improve

1. **V14 TPE Was Highly Effective**: The 785 TPE trials in V14 (including V11-V13 seeds) already explored the design space extensively and found a strong local/global optimum.

2. **Converged Design Space**: The best designs cluster tightly in parameter space, indicating convergence to an optimal region.

3. **NSGA-II Exploration vs Exploitation**: NSGA-II prioritizes Pareto front diversity over single-objective convergence. It sampled widely but the promising regions were already well-explored.

4. **Small Pareto Front**: With only 2 truly non-dominated solutions, there's limited trade-off space to explore.

---

## 5. Comparison with Prior Versions

| Version | Algorithm | FEA Trials | Best WS | Best 40° | Best 60° | Best Mfg |
|---------|-----------|------------|---------|----------|----------|----------|
| V11 | GNN + TuRBO | 107 | 129.33 | 6.34 | 13.81 | 28.54 |
| V12 | GNN + TuRBO | 5131* | 129.33 | 6.34 | 13.81 | 28.54 |
| V13 | TPE | 291 | 129.33 | 6.34 | 13.81 | 28.54 |
| V14 | TPE (adaptive) | 785 | **121.72** | **5.99** | **13.10** | **26.28** |
| V15 | NSGA-II | 126 | 121.72** | 5.99** | 13.10** | 26.28** |

\* V12 includes 5000+ surrogate predictions
\** Best from seeded V14 data, not new V15 FEA

**Campaign Improvement**: From V11 baseline to V14/V15 best:
- 40° tracking: 6.34 → 5.99 nm (**-5.5%**)
- 60° tracking: 13.81 → 13.10 nm (**-5.1%**)
- Manufacturing: 28.54 → 26.28 nm (**-7.9%**)
- Weighted Sum: 129.33 → 121.72 (**-5.9%**)

---

## 6. Best Design Summary

### 6.1 Recommended Configuration

The best design found across the entire M1 mirror optimization campaign (V11-V15):

```
Trial: V14 #725 (V15 #445)
Source: FEA Validated

Design Variables:
  lateral_inner_angle:        27.8846°
  lateral_outer_angle:        13.1862°
  lateral_outer_pivot:        11.5290 mm
  lateral_inner_pivot:         7.4063 mm
  lateral_middle_pivot:       22.9518 mm
  lateral_closeness:           9.9300 mm
  whiffle_min:                58.8986 mm
  whiffle_outer_to_vertical:  77.8388°
  whiffle_triangle_closeness: 66.8829 mm
  blank_backface_angle:        4.3030°
  inner_circular_rib_dia:    505.8876 mm

Performance:
  40° vs 20° tracking:     5.99 nm (target: 4.0 nm)
  60° vs 20° tracking:    13.10 nm (target: 10.0 nm)
  Manufacturing (90°):    26.28 nm (target: 20.0 nm)
  Weighted Sum:          121.72
```

### 6.2 Target Achievement Status

| Objective | Target | Achieved | Gap | Status |
|-----------|--------|----------|-----|--------|
| 40° tracking | 4.0 nm | 5.99 nm | +50% | ❌ Not met |
| 60° tracking | 10.0 nm | 13.10 nm | +31% | ❌ Not met |
| Manufacturing | 20.0 nm | 26.28 nm | +31% | ❌ Not met |

**Conclusion**: Current design space constraints cannot achieve the specified targets. Consider:
1. Expanding design variable bounds
2. Adding new design variables
3. Relaxing targets based on physical limitations
4. Design modifications (geometry changes, materials)

---

## 7. Conclusions and Recommendations

### 7.1 Key Conclusions

1. **Optimization Converged**: The M1 mirror optimization campaign has effectively converged after ~1,400 total FEA evaluations across V11-V15.

2. **V14 TPE Found Optimal**: The V14 TPE optimization found the best design; V15 NSGA-II exploration confirmed this optimum.

3. **Limited Trade-offs**: The small Pareto front (2 solutions) indicates the three objectives are not strongly conflicting in the optimal region.

4. **Targets Not Achievable**: Current design space cannot achieve the specified targets. The best achievable performance is ~50% above targets.

### 7.2 Recommendations

1. **Accept Current Best**: Trial #725 (V14) / #445 (V15) represents the optimal design within current constraints.

2. **Archive for Production**: Run `python tools/archive_best_design.py m1_mirror_adaptive_V15` to archive the best design.

3. **Future Work Options**:
   - Expand design space (new variables, wider bounds)
   - Alternative support structures
   - Material optimization
   - Active correction systems

4. **No Further Optimization Needed**: Additional trials are unlikely to find significant improvements without design space changes.

---

## Appendix A: File Locations

```
studies/m1_mirror_adaptive_V15/
├── 1_setup/
│   ├── optimization_config.json    # Study configuration
│   └── model/                      # NX model files
├── 2_iterations/
│   └── iter*/                      # FEA iteration folders
├── 3_results/
│   ├── study.db                    # Optuna database
│   ├── pareto_front.json           # Pareto front data
│   └── optimization.log            # Execution log
├── run_optimization.py             # Main optimization script
└── STUDY_REPORT.md                 # This report
```

---

## Appendix B: Zernike Analysis

### Subcase Configuration

| Subcase | Angle | Description |
|---------|-------|-------------|
| 1 | 90° | Manufacturing/polishing position |
| 2 | 20° | Reference position (minimum gravity) |
| 3 | 40° | Operational tracking position |
| 4 | 60° | Operational tracking position |

### Filtering Strategy

- **Tracking (40°, 60° vs 20°)**: Filter Zernike modes J0-J3 (piston, tip/tilt, focus)
- **Manufacturing (90° vs 20°)**: Filter only J1-J3 (tip/tilt, focus)
- **Number of Zernike modes**: 50

---

*Report generated by Atomizer. Last updated: 2025-12-15*