Atomizer/docs/physics/ZERNIKE_OPD_METHOD.md

Rigorous OPD-Based Zernike Analysis for Mirror Optimization

Document Version: 1.0 Created: 2024-12-22 Author: Atomizer Framework Status: Active

---

Executive Summary

This document describes a rigorous Optical Path Difference (OPD) method for computing Zernike wavefront error that correctly accounts for lateral (X, Y) displacements in addition to axial (Z) displacements.

The Problem: Standard Zernike analysis uses only Z-displacement at the original (x, y) node positions. When supports pinch the mirror or lateral forces cause in-plane deformation, nodes shift in X and Y. The standard method is blind to this, potentially leading to:

• Optimized designs that appear good but have poor actual optical performance
• Optimizer convergence to non-optimal solutions that "cheat" by distorting laterally

The Solution: The OPD method computes the true surface error by accounting for the fact that a laterally-displaced node should be compared against the parabola height at its new (x+dx, y+dy) position, not its original position.

---

Table of Contents

1. The Optical Physics Problem
2. Mathematical Formulation
3. When This Matters
4. Implementation Details
5. Usage Guide
6. Validation and Testing
7. Migration Guide

---

1. The Optical Physics Problem

1.1 What Zernike Analysis Does

Zernike polynomials decompose a wavefront error surface into orthogonal modes:

W(r, θ) = Σ cⱼ Zⱼ(r, θ)

Where:

• W = wavefront error (nm)
• cⱼ = Zernike coefficient for mode j
• Zⱼ = Zernike polynomial (Noll indexing)

For a reflective mirror, the wavefront error is twice the surface error:

WFE = 2 × surface_error

1.2 Standard Method (Z-Only)

The standard approach:

1. Read node original positions (x₀, y₀, z₀) from BDF/DAT
2. Read displacement vector (Δx, Δy, Δz) from OP2
3. Compute surface error = Δz (Z-displacement only)
4. Compute WFE = 2 × Δz × nm_scale
5. Fit Zernike at original coordinates (x₀, y₀)

# Standard method (simplified)
for nid, (dx, dy, dz) in displacements:
    x, y, z = original_coords[nid]
    wfe = dz * 2 * nm_scale  # ONLY uses Z-displacement
    X.append(x)              # Original X
    Y.append(y)              # Original Y
    WFE.append(wfe)

coeffs = fit_zernike(X, Y, WFE)

1.3 The Problem: Lateral Displacement is Ignored

Consider a node on a parabolic mirror:

• Original position: (x₀, y₀, z₀) where z₀ = -r₀²/(4f) on the parabola
• Deformed position: (x₀+Δx, y₀+Δy, z₀+Δz)

Question: What is the true surface error?

Standard method says: surface_error = Δz

But this is wrong! If the node moved laterally to a new (x, y), the ideal parabola has a different Z at that location. The node should be compared against:

z_expected = parabola(x₀+Δx, y₀+Δy) = -(x₀+Δx)² + (y₀+Δy)² / (4f)

Not against z₀ = parabola(x₀, y₀).

1.4 Visual Example

                    Original parabola
                         ___
                      _/     \_
                    /           \
                   /      *A     \      A = original node at (x₀, y₀, z₀)
                  /     ↗  ↘      \     B = deformed position (x₀+Δx, y₀+Δy, z₀+Δz)
                 /     B    C      \    C = where node SHOULD be if staying on parabola
                /                   \
               /_____________________\

Standard method: error = z_B - z_A = Δz
                 (compares B to A vertically)

OPD method: error = z_B - z_C = Δz - Δz_parabola
            (compares B to where parabola is at B's (x,y))

---

2. Mathematical Formulation

2.1 Differential OPD Formulation

For a paraboloid with optical axis along Z:

z = -r² / (4f)    [concave mirror, vertex at origin]

Where:

• r² = x² + y²
• f = focal length

Key Insight: We can compute the change in parabola Z due to lateral movement:

Δz_parabola = z(x₀+Δx, y₀+Δy) - z(x₀, y₀)
            = -[(x₀+Δx)² + (y₀+Δy)²] / (4f) - [-( x₀² + y₀²) / (4f)]
            = -[r_def² - r₀²] / (4f)
            = -Δr² / (4f)

Where:

Δr² = r_def² - r₀² = (x₀+Δx)² + (y₀+Δy)² - x₀² - y₀²
    = 2·x₀·Δx + Δx² + 2·y₀·Δy + Δy²

2.2 True Surface Error

The true surface error is:

surface_error = Δz - Δz_parabola
              = Δz - (-Δr² / 4f)
              = Δz + Δr² / (4f)

Interpretation:

• If a node moves outward (larger r), it should also move in -Z to stay on the concave parabola
• If the FEA says it moved by Δz, but staying on the parabola requires Δz_parabola, the difference is the true error
• This corrects for the "false error" that the standard method counts when nodes shift laterally

2.3 Wavefront Error

WFE = 2 × surface_error × nm_scale
    = 2 × (Δz - Δz_parabola) × nm_scale

2.4 Zernike Fitting Coordinates

Another subtlety: the Zernike fit should use the deformed coordinates (x₀+Δx, y₀+Δy) rather than the original coordinates. This is because the WFE surface represents the error at the positions where the nodes actually are after deformation.

# OPD method
X_fit = x0 + dx  # Deformed X
Y_fit = y0 + dy  # Deformed Y
WFE = surface_error * 2 * nm_scale

coeffs = fit_zernike(X_fit, Y_fit, WFE)

---

3. When This Matters

3.1 Magnitude Analysis

The correction term is:

Δz_parabola = -Δr² / (4f) ≈ -(2·x₀·Δx + 2·y₀·Δy) / (4f)    [ignoring Δx², Δy²]
            ≈ -(x₀·Δx + y₀·Δy) / (2f)

For a node at radius r₀ with tangential displacement Δ_tangential:

• The correction is approximately: r₀ · Δ_lateral / (2f)

Example: Mirror with f = 5000 mm, outer radius = 400 mm

• Node at r = 400 mm shifts laterally by Δx = 0.001 mm (1 µm)
• Correction: 400 × 0.001 / (2 × 5000) = 0.00004 mm = 40 nm

This is significant when typical WFE is in the 10-100 nm range!

3.2 Classification by Load Case

Load Case	Lateral Disp.	Method Impact
Axial support (gravity in Z)	Very small	Minimal - both methods similar
Lateral support (gravity in X/Y)	Large	Significant - OPD method required
Clamp/fixture forces	Can be large locally	May be significant at pinch points
Thermal	Variable	Depends on thermal gradients
Mirror cell deflection	Variable	Check lateral displacement magnitude

3.3 Diagnostic Thresholds

The ZernikeOPDExtractor provides lateral displacement statistics:

Max Lateral Disp.	Recommendation
> 10 µm	CRITICAL: OPD method required
1 - 10 µm	RECOMMENDED: OPD method provides meaningful improvement
0.1 - 1 µm	OPTIONAL: OPD method provides minor improvement
< 0.1 µm	EQUIVALENT: Both methods give essentially identical results

---

4. Implementation Details

4.1 Module: extract_zernike_opd.py

Location: optimization_engine/extractors/extract_zernike_opd.py

Key Functions:

def compute_true_opd(x0, y0, z0, dx, dy, dz, focal_length, concave=True):
    """
    Compute true surface error accounting for lateral displacement.

    Returns:
        x_def: Deformed X coordinates
        y_def: Deformed Y coordinates
        surface_error: True surface error (not just Δz)
        lateral_magnitude: |Δx, Δy| for diagnostics
    """

def estimate_focal_length_from_geometry(x, y, z, concave=True):
    """
    Estimate parabola focal length by fitting z = a·r² + b.
    Focal length = 1 / (4·|a|)
    """

Main Class:

class ZernikeOPDExtractor:
    """
    Rigorous OPD-based Zernike extractor.

    Key differences from ZernikeExtractor:
    - Uses deformed (x, y) coordinates for fitting
    - Computes surface error relative to parabola at deformed position
    - Provides lateral displacement diagnostics
    """

4.2 Algorithm Flow

1. Load geometry (BDF) and displacements (OP2)

2. For each node:
   a. Get original position: (x₀, y₀, z₀)
   b. Get displacement: (Δx, Δy, Δz)
   c. Compute deformed position: (x_def, y_def) = (x₀+Δx, y₀+Δy)
   d. Compute Δr² = r_def² - r₀²
   e. Compute Δz_parabola = -Δr² / (4f)   [for concave]
   f. Compute surface_error = Δz - Δz_parabola
   g. Store lateral_disp = √(Δx² + Δy²)

3. Convert to WFE: WFE = 2 × surface_error × nm_scale

4. Fit Zernike coefficients using (x_def, y_def, WFE)

5. Compute RMS metrics:
   - Global RMS = √(mean(WFE²))
   - Filtered RMS = √(mean((WFE - low_order_fit)²))

4.3 Focal Length Handling

The extractor can:

1. Use a provided focal length (most accurate)
2. Auto-estimate from geometry by fitting z = a·r² + b

Auto-estimation works well for clean parabolic meshes but may need manual override for:

• Off-axis parabolas
• Aspheric surfaces
• Meshes with significant manufacturing errors

# Explicit focal length
extractor = ZernikeOPDExtractor(op2_file, focal_length=5000.0)

# Auto-estimate (default)
extractor = ZernikeOPDExtractor(op2_file, auto_estimate_focal=True)

---

5. Usage Guide

5.1 Quick Comparison Test

Run the test script to see how much the methods differ for your data:

conda activate atomizer
python test_zernike_opd_comparison.py

Output example:

--- Standard Method (Z-only) ---
  Global RMS:   171.65 nm
  Filtered RMS: 28.72 nm

--- Rigorous OPD Method ---
  Global RMS:   171.89 nm
  Filtered RMS: 29.15 nm

--- Difference (OPD - Standard) ---
  Filtered RMS: +0.43 nm (+1.5%)

--- Lateral Displacement ---
  Max:  0.156 µm
  RMS:  0.111 µm

>>> OPTIONAL: Small lateral displacements. OPD method provides minor improvement.

5.2 Using in Optimization

For new studies, use the OPD extractor:

from optimization_engine.extractors import extract_zernike_opd_filtered_rms

def objective(trial):
    # ... parameter suggestion and FEA solve ...

    # Use OPD method instead of standard
    rms = extract_zernike_opd_filtered_rms(
        op2_file,
        subcase='20',
        focal_length=5000.0  # Optional: specify or let it auto-estimate
    )
    return rms

In optimization config (future enhancement):

{
  "objectives": [
    {
      "name": "filtered_rms",
      "extractor": "zernike_opd",
      "extractor_config": {
        "subcase": "20",
        "metric": "filtered_rms_nm",
        "focal_length": 5000.0
      }
    }
  ]
}

5.3 Visualization with Insights

Generate the comparison insight for a study:

python -m optimization_engine.insights generate studies/my_study --type zernike_opd_comparison

This creates an HTML visualization showing:

1. Lateral displacement map - Where pinching/lateral deformation occurs
2. WFE surface - Using the rigorous OPD method
3. Comparison table - Quantitative difference between methods
4. Recommendation - Whether OPD method is needed for your study

5.4 API Reference

from optimization_engine.extractors import (
    # Main extractor class
    ZernikeOPDExtractor,

    # Convenience functions
    extract_zernike_opd,           # Full metrics dict
    extract_zernike_opd_filtered_rms,  # Just the filtered RMS (float)
    compare_zernike_methods,       # Compare standard vs OPD
)

# Full extraction with all metrics
result = extract_zernike_opd(op2_file, subcase='20')
# Returns: {
#   'filtered_rms_nm': float,
#   'global_rms_nm': float,
#   'max_lateral_disp_um': float,
#   'rms_lateral_disp_um': float,
#   'focal_length_used': float,
#   'astigmatism_rms_nm': float,
#   'coma_rms_nm': float,
#   ...
# }

# Just the primary metric for optimization
rms = extract_zernike_opd_filtered_rms(op2_file, subcase='20')

# Compare both methods
comparison = compare_zernike_methods(op2_file, subcase='20')
# Returns: {
#   'standard_method': {'filtered_rms_nm': ...},
#   'opd_method': {'filtered_rms_nm': ...},
#   'delta': {'filtered_rms_nm': ..., 'percent_difference_filtered': ...},
#   'lateral_displacement': {'max_um': ..., 'rms_um': ...},
#   'recommendation': str
# }

---

6. Validation and Testing

6.1 Analytical Test Case

For a simple test: apply a known lateral displacement and verify the correction.

Setup:

• Parabola: f = 5000 mm
• Node at (x₀, y₀) = (400, 0) mm, so r₀ = 400 mm
• Apply uniform X-displacement: Δx = 0.01 mm, Δy = 0, Δz = 0

Expected correction:

Δr² = (400.01)² + 0² - 400² - 0² = 8.0001 mm²
Δz_parabola = -8.0001 / (4 × 5000) = -0.0004 mm = -400 nm (surface)
WFE_correction = 2 × 400 nm = 800 nm

Standard method: WFE = 2 × Δz × 1e6 = 0 nm OPD method: WFE = 2 × (0 - (-0.0004)) × 1e6 = 800 nm

The OPD method correctly identifies that a purely lateral displacement does affect the wavefront!

6.2 Sanity Checks

The OPD method should:

1. Give identical results to standard method when Δx = Δy = 0 everywhere
2. Show larger WFE when nodes move outward laterally (positive Δr²)
3. Show smaller WFE when nodes move inward laterally (negative Δr²)
4. Scale with 1/f (larger effect for faster mirrors)

6.3 Running the Test

conda activate atomizer
python test_zernike_opd_comparison.py

---

7. Migration Guide

7.1 For Existing Studies

1. Run comparison test on a few representative iterations
2. Check the difference - if > 5%, consider re-optimizing
3. For lateral support studies - strongly recommend re-optimization with OPD method

7.2 For New Studies

1. Use OPD method by default - it's never worse than standard
2. Specify focal length if known (more accurate than auto-estimate)
3. Monitor lateral displacement in the insight reports

7.3 Code Changes

Before (standard method):

from optimization_engine.extractors import extract_zernike_filtered_rms

rms = extract_zernike_filtered_rms(op2_file, subcase='20')

After (OPD method):

from optimization_engine.extractors import extract_zernike_opd_filtered_rms

rms = extract_zernike_opd_filtered_rms(op2_file, subcase='20', focal_length=5000.0)

---

Appendix A: Derivation Details

A.1 Full Derivation of Δz_parabola

For a concave paraboloid: z = -r²/(4f) = -(x² + y²)/(4f)

Original position: z₀ = -(x₀² + y₀²)/(4f) Deformed position: z_expected = -((x₀+Δx)² + (y₀+Δy)²)/(4f)

z_expected - z₀ = -[(x₀+Δx)² + (y₀+Δy)² - x₀² - y₀²] / (4f)
                = -[x₀² + 2x₀Δx + Δx² + y₀² + 2y₀Δy + Δy² - x₀² - y₀²] / (4f)
                = -[2x₀Δx + Δx² + 2y₀Δy + Δy²] / (4f)
                = -[r_def² - r₀²] / (4f)
                = -Δr² / (4f)

This is Δz_parabola - the Z change required to stay on the ideal parabola.

A.2 Sign Convention

For a concave mirror (typical telescope primary):

• Surface curves toward -Z (vertex is the highest point)
• z = -r²/(4f) (negative coefficient)
• Moving outward (Δr² > 0) requires moving in -Z direction
• Δz_parabola = -Δr²/(4f) is negative for outward movement

For a convex mirror:

• Surface curves toward +Z
• z = +r²/(4f) (positive coefficient)
• Δz_parabola = +Δr²/(4f) is positive for outward movement

The concave parameter in the code handles this sign flip.

---

Appendix B: Files Reference

File	Purpose
optimization_engine/extractors/extract_zernike_opd.py	Main OPD extractor implementation
optimization_engine/extractors/extract_zernike.py	Standard (Z-only) extractor
optimization_engine/insights/zernike_opd_comparison.py	Visualization insight
test_zernike_opd_comparison.py	Quick test script

Related Documentation

Document	Purpose
ZERNIKE_FUNDAMENTALS.md	General Zernike usage, RMS calculation, multi-subcase analysis
00_INDEX.md	Physics documentation index
.claude/skills/modules/extractors-catalog.md	Quick extractor lookup
.claude/skills/modules/insights-catalog.md	Quick insight lookup
docs/protocols/system/SYS_12_EXTRACTOR_LIBRARY.md	Extractor specifications (E8-E10, E20-E21)
docs/protocols/system/SYS_17_STUDY_INSIGHTS.md	Insight specifications

---

Appendix C: Glossary

Term	Definition
OPD	Optical Path Difference - the path length difference experienced by light rays
WFE	Wavefront Error - deviation of actual wavefront from ideal (WFE = 2 × surface error for reflection)
Zernike polynomials	Orthogonal basis functions for representing wavefronts over a circular aperture
Noll index	Standard optical indexing scheme for Zernike modes (j=1 is piston, j=4 is defocus, etc.)
Filtered RMS	RMS after removing low-order modes (piston, tip, tilt, defocus) that can be corrected by alignment
Lateral displacement	In-plane (X, Y) movement of nodes, as opposed to axial (Z) movement
Focal length	Distance from vertex to focus for a parabola; f = R/(2) where R is vertex radius of curvature

---

Document maintained by Atomizer Framework. Last updated: 2024-12-22