Atomizer/docs/protocols/system/SYS_16_STUDY_INSIGHTS.md

SYS_16: Study Insights

Version: 1.0.0 Status: Active Purpose: Physics-focused visualizations for FEA optimization results

---

Overview

Study Insights provide physics understanding of optimization results through interactive 3D visualizations. Unlike the Analysis page (which shows optimizer metrics like convergence and Pareto fronts), Insights answer the question: "What does this design actually look like?"

Analysis vs Insights

Aspect	Analysis	Insights
Focus	Optimization performance	Physics understanding
Questions	"Is the optimizer converging?"	"What does the best design look like?"
Data Source	study.db (trials, objectives)	Simulation outputs (OP2, mesh, fields)
Typical Plots	Convergence, Pareto, parameters	3D surfaces, stress contours, mode shapes
When Used	During/after optimization	After specific trial of interest

---

Available Insight Types

Type ID	Name	Applicable To	Data Required
zernike_dashboard	Zernike Dashboard (RECOMMENDED)	Mirror, optics	OP2 with displacement subcases
zernike_wfe	Zernike WFE Analysis	Mirror, optics	OP2 with displacement subcases
zernike_opd_comparison	Zernike OPD Method Comparison	Mirror, optics, lateral	OP2 with displacement subcases
msf_zernike	MSF Zernike Analysis	Mirror, optics	OP2 with displacement subcases
stress_field	Stress Distribution	Structural, bracket, beam	OP2 with stress results
modal	Modal Analysis	Vibration, dynamic	OP2 with eigenvalue/eigenvector
thermal	Thermal Analysis	Thermo-structural	OP2 with temperature results
design_space	Design Space Explorer	All optimization studies	study.db with 5+ trials

Zernike Method Comparison: Standard vs OPD

The Zernike insights now support two WFE computation methods:

Method	Description	When to Use
Standard (Z-only)	Uses only Z-displacement at original (x,y) coordinates	Quick analysis, negligible lateral displacement
OPD (X,Y,Z) ← RECOMMENDED	Accounts for lateral (X,Y) displacement via interpolation	Any surface with gravity loads, most rigorous

How OPD method works:

1. Builds interpolator from undeformed BDF mesh geometry
2. For each deformed node at (x+dx, y+dy, z+dz), interpolates Z_ideal at new XY position
3. Computes WFE = z_deformed - Z_ideal(x_def, y_def)
4. Fits Zernike polynomials to the surface error map

Typical difference: OPD method gives 8-11% higher WFE values than Standard (more conservative/accurate).

---

Architecture

Module Structure

optimization_engine/insights/
├── __init__.py               # Registry and public API
├── base.py                   # StudyInsight base class, InsightConfig, InsightResult
├── zernike_wfe.py            # Mirror wavefront error visualization (50 modes)
├── zernike_opd_comparison.py # OPD vs Standard method comparison (lateral disp. analysis)
├── msf_zernike.py            # MSF band decomposition (100 modes, LSF/MSF/HSF)
├── stress_field.py           # Stress contour visualization
├── modal_analysis.py         # Mode shape visualization
├── thermal_field.py      # Temperature distribution
└── design_space.py       # Parameter-objective exploration

Class Hierarchy

StudyInsight (ABC)
├── ZernikeDashboardInsight     # RECOMMENDED: Unified dashboard with all views
├── ZernikeWFEInsight           # Standard 50-mode WFE analysis (with OPD toggle)
├── ZernikeOPDComparisonInsight # OPD method comparison (lateral displacement)
├── MSFZernikeInsight           # 100-mode MSF band analysis
├── StressFieldInsight
├── ModalInsight
├── ThermalInsight
└── DesignSpaceInsight

Key Classes

StudyInsight (Base Class)

class StudyInsight(ABC):
    insight_type: str       # Unique identifier (e.g., 'zernike_wfe')
    name: str               # Human-readable name
    description: str        # What this insight shows
    applicable_to: List[str]  # Study types this applies to

    def can_generate(self) -> bool:
        """Check if required data exists."""

    def generate(self, config: InsightConfig) -> InsightResult:
        """Generate visualization."""

    def generate_html(self, trial_id=None, **kwargs) -> Path:
        """Generate standalone HTML file."""

    def get_plotly_data(self, trial_id=None, **kwargs) -> dict:
        """Get Plotly figure for dashboard embedding."""

InsightConfig

@dataclass
class InsightConfig:
    trial_id: Optional[int] = None     # Which trial to visualize
    colorscale: str = 'Turbo'          # Plotly colorscale
    amplification: float = 1.0         # Deformation scale factor
    lighting: bool = True              # 3D lighting effects
    output_dir: Optional[Path] = None  # Where to save HTML
    extra: Dict[str, Any] = {}         # Type-specific config

InsightResult

@dataclass
class InsightResult:
    success: bool
    html_path: Optional[Path] = None        # Generated HTML file
    plotly_figure: Optional[dict] = None    # Figure for dashboard
    summary: Optional[dict] = None          # Key metrics
    error: Optional[str] = None             # Error message if failed

---

Usage

Python API

from optimization_engine.insights import get_insight, list_available_insights
from pathlib import Path

study_path = Path("studies/my_mirror_study")

# List what's available
available = list_available_insights(study_path)
for info in available:
    print(f"{info['type']}: {info['name']}")

# Generate specific insight
insight = get_insight('zernike_wfe', study_path)
if insight and insight.can_generate():
    result = insight.generate()
    print(f"Generated: {result.html_path}")
    print(f"40-20 Filtered RMS: {result.summary['40_vs_20_filtered_rms']:.2f} nm")

CLI

# List all insight types
python -m optimization_engine.insights list

# Generate all available insights for a study
python -m optimization_engine.insights generate studies/my_study

# Generate specific insight
python -m optimization_engine.insights generate studies/my_study --type zernike_wfe

With Configuration

from optimization_engine.insights import get_insight, InsightConfig

insight = get_insight('stress_field', study_path)
config = InsightConfig(
    colorscale='Hot',
    extra={
        'yield_stress': 250,  # MPa
        'stress_unit': 'MPa'
    }
)
result = insight.generate(config)

---

Insight Type Details

0. Zernike Dashboard (zernike_dashboard) - RECOMMENDED

Purpose: Unified dashboard with all orientations (40°, 60°, 90°) and MSF band analysis on one page. Light theme, executive summary, and method comparison.

Generates: 1 comprehensive HTML file with:

• Executive summary with metric cards (40-20, 60-20, MFG workload)
• MSF band analysis (LSF/MSF/HSF decomposition)
• 3D surface plots for each orientation
• Zernike coefficient bar charts color-coded by band

Configuration:

config = InsightConfig(
    extra={
        'n_modes': 50,
        'filter_low_orders': 4,
        'theme': 'light',  # Light theme for reports
    }
)

Summary Output:

{
    '40_vs_20_filtered_rms': 6.53,   # nm (OPD method)
    '60_vs_20_filtered_rms': 14.21,  # nm (OPD method)
    '90_optician_workload': 26.34,   # nm (J1-J3 filtered)
    'msf_rss_40': 2.1,               # nm (MSF band contribution)
}

1. Zernike WFE Analysis (zernike_wfe)

Purpose: Visualize wavefront error for mirror optimization with Zernike polynomial decomposition. Now includes Standard/OPD method toggle and lateral displacement maps.

Generates: 6 HTML files

• zernike_*_40_vs_20.html - 40° vs 20° relative WFE (with method toggle)
• zernike_*_40_lateral.html - Lateral displacement map for 40°
• zernike_*_60_vs_20.html - 60° vs 20° relative WFE (with method toggle)
• zernike_*_60_lateral.html - Lateral displacement map for 60°
• zernike_*_90_mfg.html - 90° manufacturing (with method toggle)
• zernike_*_90_mfg_lateral.html - Lateral displacement map for 90°

Features:

• Toggle buttons to switch between Standard (Z-only) and OPD (X,Y,Z) methods
• Toggle between WFE view and ΔX, ΔY, ΔZ displacement components
• Metrics comparison table showing both methods side-by-side
• Lateral displacement statistics (Max, RMS in µm)

Configuration:

config = InsightConfig(
    amplification=0.5,  # Reduce deformation scaling
    colorscale='Turbo',
    extra={
        'n_modes': 50,
        'filter_low_orders': 4,  # Remove piston, tip, tilt, defocus
        'disp_unit': 'mm',
    }
)

Summary Output:

{
    '40_vs_20_filtered_rms_std': 6.01,   # nm (Standard method)
    '40_vs_20_filtered_rms_opd': 6.53,   # nm (OPD method)
    '60_vs_20_filtered_rms_std': 12.81,  # nm
    '60_vs_20_filtered_rms_opd': 14.21,  # nm
    '90_mfg_filtered_rms_std': 24.5,     # nm
    '90_mfg_filtered_rms_opd': 26.34,    # nm
    '90_optician_workload': 26.34,       # nm (J1-J3 filtered)
    'lateral_40_max_um': 0.234,          # µm max lateral displacement
    'lateral_60_max_um': 0.312,          # µm
    'lateral_90_max_um': 0.089,          # µm
}

2. MSF Zernike Analysis (msf_zernike)

Purpose: Detailed mid-spatial frequency analysis for telescope mirrors with gravity-induced support print-through.

Generates: 1 comprehensive HTML file with:

• Band decomposition table (LSF/MSF/HSF RSS metrics)
• MSF-only 3D surface visualization
• Coefficient bar chart color-coded by band
• Dominant MSF mode identification
• Mesh resolution analysis

Band Definitions (for 1.2m class mirror):

Band	Zernike Order	Feature Size	Physical Meaning
LSF	n ≤ 10	> 120 mm	M2 hexapod correctable
MSF	n = 11-50	24-109 mm	Support print-through
HSF	n > 50	< 24 mm	Near mesh resolution limit

Configuration:

config = InsightConfig(
    extra={
        'n_modes': 100,      # Higher than zernike_wfe (100 vs 50)
        'lsf_max': 10,       # n ≤ 10 is LSF
        'msf_max': 50,       # n = 11-50 is MSF
        'disp_unit': 'mm',
    }
)

Analyses Performed:

• Absolute WFE at each orientation (40°, 60°, 90°)
• Relative to 20° (operational reference)
• Relative to 90° (manufacturing/polishing reference)

Summary Output:

{
    'n_modes': 100,
    'lsf_max_order': 10,
    'msf_max_order': 50,
    'mesh_nodes': 78290,
    'mesh_spacing_mm': 4.1,
    'max_resolvable_order': 157,
    '40deg_vs_20deg_lsf_rss': 12.3,   # nm
    '40deg_vs_20deg_msf_rss': 8.7,    # nm - KEY METRIC
    '40deg_vs_20deg_total_rss': 15.2, # nm
    '40deg_vs_20deg_msf_pct': 33.0,   # % of total in MSF band
    # ... similar for 60deg, 90deg
}

When to Use:

• Analyzing support structure print-through
• Quantifying gravity-induced MSF content
• Comparing MSF at different orientations
• Validating mesh resolution is adequate for MSF capture

---

3. Stress Distribution (stress_field)

Purpose: Visualize Von Mises stress distribution with hot spot identification.

Configuration:

config = InsightConfig(
    colorscale='Hot',
    extra={
        'yield_stress': 250,    # MPa - shows safety factor
        'stress_unit': 'MPa',
    }
)

Summary Output:

{
    'max_stress': 187.5,      # MPa
    'mean_stress': 45.2,      # MPa
    'p95_stress': 120.3,      # 95th percentile
    'p99_stress': 165.8,      # 99th percentile
    'safety_factor': 1.33,    # If yield_stress provided
}

4. Modal Analysis (modal)

Purpose: Visualize natural frequencies and mode shapes.

Configuration:

config = InsightConfig(
    amplification=50.0,  # Mode shape scale
    extra={
        'n_modes': 20,      # Number of modes to show
        'show_mode': 1,     # Which mode shape to display
    }
)

Summary Output:

{
    'n_modes': 20,
    'first_frequency_hz': 125.4,
    'frequencies_hz': [125.4, 287.8, 312.5, ...],
}

5. Thermal Analysis (thermal)

Purpose: Visualize temperature distribution and gradients.

Configuration:

config = InsightConfig(
    colorscale='Thermal',
    extra={
        'temp_unit': 'K',  # or 'C', 'F'
    }
)

Summary Output:

{
    'max_temp': 423.5,    # K
    'min_temp': 293.0,    # K
    'mean_temp': 345.2,   # K
    'temp_range': 130.5,  # K
}

6. Design Space Explorer (design_space)

Purpose: Visualize parameter-objective relationships from optimization trials.

Configuration:

config = InsightConfig(
    extra={
        'primary_objective': 'filtered_rms',  # Color by this objective
    }
)

Summary Output:

{
    'n_trials': 100,
    'n_params': 4,
    'n_objectives': 2,
    'best_trial_id': 47,
    'best_params': {'p1': 0.5, 'p2': 1.2, ...},
    'best_values': {'filtered_rms': 45.2, 'mass': 2.34},
}

---

Output Directory

Insights are saved to {study}/3_insights/:

studies/my_study/
├── 1_setup/
├── 2_results/
└── 3_insights/               # Created by insights module
    ├── zernike_20241220_143022_40_vs_20.html
    ├── zernike_20241220_143022_60_vs_20.html
    ├── zernike_20241220_143022_90_mfg.html
    ├── stress_20241220_143025.html
    └── design_space_20241220_143030.html

---

Creating New Insight Types

To add a new insight type (power_user+):

1. Create the insight class

# optimization_engine/insights/my_insight.py

from .base import StudyInsight, InsightConfig, InsightResult, register_insight

@register_insight
class MyInsight(StudyInsight):
    insight_type = "my_insight"
    name = "My Custom Insight"
    description = "Description of what it shows"
    applicable_to = ["structural", "all"]

    def can_generate(self) -> bool:
        # Check if required data exists
        return self.results_path.exists()

    def _generate(self, config: InsightConfig) -> InsightResult:
        # Generate visualization
        # ... build Plotly figure ...

        html_path = config.output_dir / f"my_insight_{timestamp}.html"
        html_path.write_text(fig.to_html(...))

        return InsightResult(
            success=True,
            html_path=html_path,
            summary={'key_metric': value}
        )

2. Register in __init__.py

from .my_insight import MyInsight

3. Test

python -m optimization_engine.insights list
# Should show "my_insight" in the list

---

Dashboard Integration

The Insights tab in the Atomizer Dashboard provides a 3-step workflow:

Step 1: Select Iteration

• Lists all available iterations (iter1, iter2, etc.) and best_design_archive
• Shows OP2 file name and modification timestamp
• Auto-selects "Best Design (Recommended)" if available

Step 2: Choose Insight Type

• Groups insights by category (Optical, Structural, Thermal, etc.)
• Shows insight name and description
• Click to select, then "Generate Insight"

Step 3: View Result

• Displays summary metrics (RMS values, etc.)
• Embedded Plotly visualization (if available)
• "Open Full View" button for multi-file insights (like Zernike WFE)
• Fullscreen mode for detailed analysis

API Endpoints

GET  /api/insights/studies/{id}/iterations      # List available iterations
GET  /api/insights/studies/{id}/available       # List available insight types
GET  /api/insights/studies/{id}/generated       # List previously generated files
POST /api/insights/studies/{id}/generate/{type} # Generate insight for iteration
GET  /api/insights/studies/{id}/view/{type}     # View generated HTML

Generate Request Body

{
  "iteration": "best_design_archive",  // or "iter5", etc.
  "trial_id": null,                    // Optional specific trial
  "config": {}                         // Insight-specific config
}

---

Version History

Version	Date	Changes
1.3.0	2025-12-22	Added ZernikeDashboardInsight (unified view), OPD method toggle, lateral displacement maps
1.2.0	2024-12-22	Dashboard overhaul: 3-step workflow, iteration selection, faster loading
1.1.0	2024-12-21	Added MSF Zernike Analysis insight (6 insight types)
1.0.0	2024-12-20	Initial release with 5 insight types