Atomizer/docs/protocols/operations/OP_01_CREATE_STUDY.md

# OP_01: Create Optimization Study

<!--
PROTOCOL: Create Optimization Study
LAYER: Operations
VERSION: 1.0
STATUS: Active
LAST_UPDATED: 2025-12-05
PRIVILEGE: user
LOAD_WITH: [core/study-creation-core.md]
-->

## Overview

This protocol guides you through creating a complete Atomizer optimization study from scratch. It covers gathering requirements, generating configuration files, and validating setup.

**Skill to Load**: `.claude/skills/core/study-creation-core.md`

---

## When to Use

| Trigger | Action |
|---------|--------|
| "new study", "create study" | Follow this protocol |
| "set up optimization" | Follow this protocol |
| "optimize my design" | Follow this protocol |
| User provides NX model | Assess and follow this protocol |

---

## Quick Reference

**Required Outputs** (ALL MANDATORY - study is INCOMPLETE without these):
| File | Purpose | Location | Priority |
|------|---------|----------|----------|
| `optimization_config.json` | Design vars, objectives, constraints | `1_setup/` | 1 |
| `run_optimization.py` | Execution script | Study root | 2 |
| **`README.md`** | Engineering documentation | Study root | **3 - NEVER SKIP** |
| `STUDY_REPORT.md` | Results template | Study root | 4 |

**CRITICAL**: README.md is MANDATORY for every study. A study without README.md is INCOMPLETE.

**Study Structure**:
```
studies/{geometry_type}/{study_name}/
├── 1_setup/
│   ├── model/              # NX files (.prt, .sim, .fem)
│   └── optimization_config.json
├── 2_iterations/           # FEA trial folders (iter1, iter2, ...)
├── 3_results/              # Optimization outputs (study.db, logs)
├── README.md               # MANDATORY
├── STUDY_REPORT.md         # MANDATORY
└── run_optimization.py
```

**IMPORTANT: Studies are organized by geometry type**:
| Geometry Type | Folder | Examples |
|---------------|--------|----------|
| M1 Mirror | `studies/M1_Mirror/` | m1_mirror_adaptive_V14, m1_mirror_cost_reduction_V3 |
| Simple Bracket | `studies/Simple_Bracket/` | bracket_stiffness_optimization |
| UAV Arm | `studies/UAV_Arm/` | uav_arm_optimization |
| Drone Gimbal | `studies/Drone_Gimbal/` | drone_gimbal_arm_optimization |
| Simple Beam | `studies/Simple_Beam/` | simple_beam_optimization |
| Other/Test | `studies/_Other/` | training_data_export_test |

When creating a new study:
1. Identify the geometry type (mirror, bracket, beam, etc.)
2. Place study under the appropriate `studies/{geometry_type}/` folder
3. For new geometry types, create a new folder with descriptive name

---

## README Hierarchy (Parent-Child Documentation)

**Two-level documentation system**:

```
studies/{geometry_type}/
├── README.md                          # PARENT: Project-level context
│   ├── Project overview               # What is this geometry/component?
│   ├── Physical system specs          # Material, dimensions, constraints
│   ├── Optical/mechanical specs       # Domain-specific requirements
│   ├── Design variables catalog       # ALL possible variables with descriptions
│   ├── Objectives catalog             # ALL possible objectives
│   ├── Campaign history               # Summary of all sub-studies
│   └── Sub-studies index              # Links to each sub-study
│
├── sub_study_V1/
│   └── README.md                      # CHILD: Study-specific details
│       ├── Link to parent             # "See ../README.md for context"
│       ├── Study focus                # What THIS study optimizes
│       ├── Active variables           # Which params enabled
│       ├── Algorithm config           # Sampler, trials, settings
│       ├── Baseline/seeding           # Starting point
│       └── Results summary            # Best trial, learnings
│
└── sub_study_V2/
    └── README.md                      # CHILD: References parent, adds specifics
```

### Parent README Content (Geometry-Level)

| Section | Content |
|---------|---------|
| Project Overview | What the component is, purpose, context |
| Physical System | Material, mass targets, loading conditions |
| Domain Specs | Optical prescription (mirrors), structural limits (brackets) |
| Design Variables | Complete catalog with ranges and descriptions |
| Objectives | All possible metrics with formulas |
| Campaign History | Evolution across sub-studies |
| Sub-Studies Index | Table with links, status, best results |
| Technical Notes | Domain-specific implementation details |

### Child README Content (Study-Level)

| Section | Content |
|---------|---------|
| Parent Reference | `> See [../README.md](../README.md) for project context` |
| Study Focus | What differentiates THIS study |
| Active Variables | Which parameters are enabled (subset of parent catalog) |
| Algorithm Config | Sampler, n_trials, sigma, seed |
| Baseline | Starting point (seeded from prior study or default) |
| Results | Best trial, improvement metrics |
| Key Learnings | What was discovered |

### When to Create Parent README

- **First study** for a geometry type → Create parent README immediately
- **Subsequent studies** → Add to parent's sub-studies index
- **New geometry type** → Create both parent and child READMEs

### Example Reference

See `studies/M1_Mirror/README.md` for a complete parent README example.

---

## Interview Mode (DEFAULT)

**Study creation now uses Interview Mode by default.** This provides guided study creation with intelligent validation.

### Triggers (Any of These Start Interview Mode)

- "create a study", "new study", "set up study"
- "create a study for my bracket"
- "optimize this model"
- "I want to minimize mass"
- Any study creation request without "skip interview" or "manual"

### When to Skip Interview Mode (Manual)

Use manual mode only when:
- Power user who knows the exact configuration
- Recreating a known study configuration
- User explicitly says "skip interview", "quick setup", or "manual config"

### Starting Interview Mode

```python
from optimization_engine.interview import StudyInterviewEngine

engine = StudyInterviewEngine(study_path)

# Run introspection first (if model available)
introspection = {
    "expressions": [...],  # From part introspection
    "model_path": "...",
    "sim_path": "..."
}

session = engine.start_interview(study_name, introspection=introspection)
action = engine.get_first_question()

# Present action.message to user
# Process answers with: action = engine.process_answer(user_response)
```

### Interview Benefits

- **Material-aware validation**: Checks stress limits against yield
- **Anti-pattern detection**: Warns about mass minimization without constraints
- **Auto extractor mapping**: Maps goals to correct extractors (E1-E10)
- **State persistence**: Resume interrupted interviews
- **Blueprint generation**: Creates validated configuration

See `.claude/skills/modules/study-interview-mode.md` for full documentation.

---

## Detailed Steps (Manual Mode - Power Users Only)

### Step 1: Gather Requirements

**Ask the user**:
1. What are you trying to optimize? (objective)
2. What can you change? (design variables)
3. What limits must be respected? (constraints)
4. Where are your NX files?

**Example Dialog**:
```
User: "I want to optimize my bracket"
You: "What should I optimize for - minimum mass, maximum stiffness,
      target frequency, or something else?"
User: "Minimize mass while keeping stress below 250 MPa"
```

### Step 2: Analyze Model (Introspection)

**MANDATORY**: When user provides NX files, run comprehensive introspection:

```python
from optimization_engine.hooks.nx_cad.model_introspection import (
    introspect_part,
    introspect_simulation,
    introspect_op2,
    introspect_study
)

# Introspect the part file to get expressions, mass, features
part_info = introspect_part("C:/path/to/model.prt")

# Introspect the simulation to get solutions, BCs, loads
sim_info = introspect_simulation("C:/path/to/model.sim")

# If OP2 exists, check what results are available
op2_info = introspect_op2("C:/path/to/results.op2")

# Or introspect entire study directory at once
study_info = introspect_study("studies/my_study/")
```

**Introspection Report Contents**:

| Source | Information Extracted |
|--------|----------------------|
| `.prt` | Expressions (count, values, types), bodies, mass, material, features |
| `.sim` | Solutions, boundary conditions, loads, materials, mesh info, output requests |
| `.op2` | Available results (displacement, stress, strain, SPC forces, etc.), subcases |

**Generate Introspection Report** at study creation:
1. Save report to `studies/{study_name}/MODEL_INTROSPECTION.md`
2. Include summary of what's available for optimization
3. List potential design variables (expressions)
4. List extractable results (from OP2)

**Key Questions Answered by Introspection**:
- What expressions exist? (potential design variables)
- What solution types? (static, modal, etc.)
- What results are available in OP2? (displacement, stress, SPC forces)
- Multi-solution required? (static + modal = set `solution_name=None`)

### Step 3: Select Protocol

Based on objectives:

| Scenario | Protocol | Sampler |
|----------|----------|---------|
| Single objective | Protocol 10 (IMSO) | TPE, CMA-ES, or GP |
| 2-3 objectives | Protocol 11 | NSGA-II |
| >50 trials, need speed | Protocol 14 | + Neural acceleration |

See [SYS_10_IMSO](../system/SYS_10_IMSO.md), [SYS_11_MULTI_OBJECTIVE](../system/SYS_11_MULTI_OBJECTIVE.md).

### Step 4: Select Extractors

Match physics to extractors from [SYS_12_EXTRACTOR_LIBRARY](../system/SYS_12_EXTRACTOR_LIBRARY.md):

| Need | Extractor ID | Function |
|------|--------------|----------|
| Max displacement | E1 | `extract_displacement()` |
| Natural frequency | E2 | `extract_frequency()` |
| Von Mises stress | E3 | `extract_solid_stress()` |
| Mass from BDF | E4 | `extract_mass_from_bdf()` |
| Mass from NX | E5 | `extract_mass_from_expression()` |
| Wavefront error | E8-E10 | Zernike extractors |

### Step 5: Generate Configuration

Create `optimization_config.json`:

```json
{
  "study_name": "bracket_optimization",
  "description": "Minimize bracket mass while meeting stress constraint",

  "design_variables": [
    {
      "name": "thickness",
      "type": "continuous",
      "min": 2.0,
      "max": 10.0,
      "unit": "mm",
      "description": "Wall thickness"
    }
  ],

  "objectives": [
    {
      "name": "mass",
      "type": "minimize",
      "unit": "kg",
      "description": "Total bracket mass"
    }
  ],

  "constraints": [
    {
      "name": "max_stress",
      "type": "less_than",
      "value": 250.0,
      "unit": "MPa",
      "description": "Maximum allowable von Mises stress"
    }
  ],

  "simulation": {
    "model_file": "1_setup/model/bracket.prt",
    "sim_file": "1_setup/model/bracket.sim",
    "solver": "nastran",
    "solution_name": null
  },

  "optimization_settings": {
    "protocol": "protocol_10_single_objective",
    "sampler": "TPESampler",
    "n_trials": 50
  }
}
```

### Step 6: Generate run_optimization.py

**CRITICAL**: Always use the `FEARunner` class pattern with proper `NXSolver` initialization.

```python
#!/usr/bin/env python3
"""
{study_name} - Optimization Runner
Generated by Atomizer LLM
"""
import sys
import re
import json
from pathlib import Path
from typing import Dict, Optional, Any

# Add optimization engine to path
sys.path.insert(0, str(Path(__file__).parent.parent.parent))

import optuna
from optimization_engine.nx_solver import NXSolver
from optimization_engine.utils import ensure_nx_running
from optimization_engine.extractors import extract_solid_stress

# Paths
STUDY_DIR = Path(__file__).parent
SETUP_DIR = STUDY_DIR / "1_setup"
ITERATIONS_DIR = STUDY_DIR / "2_iterations"
RESULTS_DIR = STUDY_DIR / "3_results"
CONFIG_PATH = SETUP_DIR / "optimization_config.json"

# Ensure directories exist
ITERATIONS_DIR.mkdir(exist_ok=True)
RESULTS_DIR.mkdir(exist_ok=True)


class FEARunner:
    """Runs actual FEA simulations. Always use this pattern!"""

    def __init__(self, config: Dict[str, Any]):
        self.config = config
        self.nx_solver = None
        self.nx_manager = None
        self.master_model_dir = SETUP_DIR / "model"

    def setup(self):
        """Setup NX and solver. Called lazily on first use."""
        study_name = self.config.get('study_name', 'my_study')

        # Ensure NX is running
        self.nx_manager, nx_was_started = ensure_nx_running(
            session_id=study_name,
            auto_start=True,
            start_timeout=120
        )

        # CRITICAL: Initialize NXSolver with named parameters, NOT config dict
        nx_settings = self.config.get('nx_settings', {})
        nx_install_dir = nx_settings.get('nx_install_path', 'C:\\Program Files\\Siemens\\NX2506')

        # Extract version from path
        version_match = re.search(r'NX(\d+)', nx_install_dir)
        nastran_version = version_match.group(1) if version_match else "2506"

        self.nx_solver = NXSolver(
            master_model_dir=str(self.master_model_dir),
            nx_install_dir=nx_install_dir,
            nastran_version=nastran_version,
            timeout=nx_settings.get('simulation_timeout_s', 600),
            use_iteration_folders=True,
            study_name=study_name
        )

    def run_fea(self, params: Dict[str, float], iter_num: int) -> Optional[Dict]:
        """Run FEA simulation and extract results."""
        if self.nx_solver is None:
            self.setup()

        # Create expression updates
        expressions = {var['expression_name']: params[var['name']]
                       for var in self.config['design_variables']}

        # Create iteration folder with model copies
        iter_folder = self.nx_solver.create_iteration_folder(
            iterations_base_dir=ITERATIONS_DIR,
            iteration_number=iter_num,
            expression_updates=expressions
        )

        # Run simulation
        nx_settings = self.config.get('nx_settings', {})
        sim_file = iter_folder / nx_settings.get('sim_file', 'model.sim')

        result = self.nx_solver.run_simulation(
            sim_file=sim_file,
            working_dir=iter_folder,
            expression_updates=expressions,
            solution_name=nx_settings.get('solution_name', 'Solution 1'),
            cleanup=False
        )

        if not result['success']:
            return None

        # Extract results
        op2_file = result['op2_file']
        stress_result = extract_solid_stress(op2_file)

        return {
            'params': params,
            'max_stress': stress_result['max_von_mises'],
            'op2_file': op2_file
        }


# Optimizer class would use FEARunner...
# See m1_mirror_adaptive_V14/run_optimization.py for full example
```

**WRONG** - causes `TypeError: expected str, bytes or os.PathLike object, not dict`:
```python
self.nx_solver = NXSolver(self.config)  # ❌ NEVER DO THIS
```

**Reference implementations**:
- `studies/m1_mirror_adaptive_V14/run_optimization.py` (TPE single-objective)
- `studies/m1_mirror_adaptive_V15/run_optimization.py` (NSGA-II multi-objective)

### Step 7: Generate Documentation

**README.md** (11 sections required):
1. Engineering Problem
2. Mathematical Formulation
3. Optimization Algorithm
4. Simulation Pipeline
5. Result Extraction Methods
6. Neural Acceleration (if applicable)
7. Study File Structure
8. Results Location
9. Quick Start
10. Configuration Reference
11. References

**STUDY_REPORT.md** (template):
```markdown
# Study Report: {study_name}

## Executive Summary
- Trials completed: _pending_
- Best objective: _pending_
- Constraint satisfaction: _pending_

## Optimization Progress
_To be filled after run_

## Best Designs Found
_To be filled after run_

## Recommendations
_To be filled after analysis_
```

### Step 7b: Capture Baseline Geometry Images (Recommended)

For better documentation, capture images of the starting geometry using the NX journal:

```bash
# Capture baseline images for study documentation
"C:\Program Files\Siemens\DesigncenterNX2512\NXBIN\run_journal.exe" ^
  "C:\Users\antoi\Atomizer\nx_journals\capture_study_images.py" ^
  -args "path/to/model.prt" "1_setup/" "model_name"
```

This generates:
- `1_setup/{model_name}_Top.png` - Top view
- `1_setup/{model_name}_iso.png` - Isometric view

**Include in README.md**:
```markdown
## Baseline Geometry

![Model - Top View](1_setup/model_name_Top.png)
*Top view description*

![Model - Isometric View](1_setup/model_name_iso.png)
*Isometric view description*
```

**Journal location**: `nx_journals/capture_study_images.py`

### Step 8: Validate NX Model File Chain

**CRITICAL**: NX simulation files have parent-child dependencies. ALL linked files must be copied to the study folder.

**Required File Chain Check**:
```
.sim (Simulation)
 └── .fem (FEM)
      └── _i.prt (Idealized Part)  ← OFTEN MISSING!
           └── .prt (Geometry Part)
```

**Validation Steps**:
1. Open the `.sim` file in NX
2. Go to **Assemblies → Assembly Navigator** or check **Part Navigator**
3. Identify ALL child components (especially `*_i.prt` idealized parts)
4. Copy ALL linked files to `1_setup/model/`

**Common Issue**: The `_i.prt` (idealized part) is often forgotten. Without it:
- `UpdateFemodel()` runs but mesh doesn't change
- Geometry changes don't propagate to FEM
- All optimization trials produce identical results

**File Checklist**:
| File Pattern | Description | Required |
|--------------|-------------|----------|
| `*.prt` | Geometry part | ✅ Always |
| `*_i.prt` | Idealized part | ✅ If FEM uses idealization |
| `*.fem` | FEM file | ✅ Always |
| `*.sim` | Simulation file | ✅ Always |

**Introspection should report**:
- List of all parts referenced by .sim
- Warning if any referenced parts are missing from study folder

### Step 9: Final Validation Checklist

**CRITICAL**: Study is NOT complete until ALL items are checked:

- [ ] NX files exist in `1_setup/model/`
- [ ] **ALL child parts copied** (especially `*_i.prt`)
- [ ] Expression names match model
- [ ] Config validates (JSON schema)
- [ ] `run_optimization.py` has no syntax errors
- [ ] **README.md exists** (MANDATORY - study is incomplete without it!)
- [ ] README.md contains: Overview, Objectives, Constraints, Design Variables, Settings, Usage, Structure
- [ ] STUDY_REPORT.md template exists

**README.md Minimum Content**:
1. Overview/Purpose
2. Objectives with weights
3. Constraints (if any)
4. Design variables with ranges
5. Optimization settings
6. Usage commands
7. Directory structure

---

## Examples

### Example 1: Simple Bracket

```
User: "Optimize my bracket.prt for minimum mass, stress < 250 MPa"

Generated config:
- 1 design variable (thickness)
- 1 objective (minimize mass)
- 1 constraint (stress < 250)
- Protocol 10, TPE sampler
- 50 trials
```

### Example 2: Multi-Objective Beam

```
User: "Minimize mass AND maximize stiffness for my beam"

Generated config:
- 2 design variables (width, height)
- 2 objectives (minimize mass, maximize stiffness)
- Protocol 11, NSGA-II sampler
- 50 trials (Pareto front)
```

### Example 3: Telescope Mirror

```
User: "Minimize wavefront error at 40deg vs 20deg reference"

Generated config:
- Multiple design variables (mount positions)
- 1 objective (minimize relative WFE)
- Zernike extractor E9
- Protocol 10
```

---

## Troubleshooting

| Symptom | Cause | Solution |
|---------|-------|----------|
| "Expression not found" | Name mismatch | Verify expression names in NX |
| "No feasible designs" | Constraints too tight | Relax constraint values |
| Config validation fails | Missing required field | Check JSON schema |
| Import error | Wrong path | Check sys.path setup |

---

## Cross-References

- **Depends On**: [SYS_12_EXTRACTOR_LIBRARY](../system/SYS_12_EXTRACTOR_LIBRARY.md)
- **Next Step**: [OP_02_RUN_OPTIMIZATION](./OP_02_RUN_OPTIMIZATION.md)
- **Skill**: `.claude/skills/core/study-creation-core.md`

---

## Version History

| Version | Date | Changes |
|---------|------|---------|
| 1.1 | 2025-12-12 | Added FEARunner class pattern, NXSolver initialization warning |
| 1.0 | 2025-12-05 | Initial release |