Atomizer/docs/protocols/system/SYS_14_NEURAL_ACCELERATION.md

SYS_14: Neural Network Acceleration

Overview

Atomizer provides neural network surrogate acceleration enabling 100-1000x faster optimization by replacing expensive FEA evaluations with instant neural predictions.

Two approaches available:

1. MLP Surrogate (Simple, integrated) - 4-layer MLP trained on FEA data, runs within study
2. GNN Field Predictor (Advanced) - Graph neural network for full field predictions

Key Innovation: Train once on FEA data, then explore 5,000-50,000+ designs in the time it takes to run 50 FEA trials.

---

When to Use

Trigger	Action
>50 trials needed	Consider neural acceleration
"neural", "surrogate", "NN" mentioned	Load this protocol
"fast", "acceleration", "speed" needed	Suggest neural acceleration
Training data available	Enable surrogate

---

Quick Reference

Performance Comparison:

Metric	Traditional FEA	Neural Network	Improvement
Time per evaluation	10-30 minutes	4.5 milliseconds	2,000-500,000x
Trials per hour	2-6	800,000+	1000x
Design exploration	~50 designs	~50,000 designs	1000x

Model Types:

Model	Purpose	Use When
MLP Surrogate	Direct objective prediction	Simple studies, quick setup
Field Predictor GNN	Full displacement/stress fields	Need field visualization
Parametric Predictor GNN	Direct objective prediction	Complex geometry, need accuracy
Ensemble	Uncertainty quantification	Need confidence bounds

---

MLP Surrogate (Recommended for Quick Start)

Overview

The MLP (Multi-Layer Perceptron) surrogate is a simple but effective neural network that predicts objectives directly from design parameters. It's integrated into the study workflow via run_nn_optimization.py.

Architecture

Input Layer (N design variables)
    ↓
Linear(N, 64) + ReLU + BatchNorm + Dropout(0.1)
    ↓
Linear(64, 128) + ReLU + BatchNorm + Dropout(0.1)
    ↓
Linear(128, 128) + ReLU + BatchNorm + Dropout(0.1)
    ↓
Linear(128, 64) + ReLU + BatchNorm + Dropout(0.1)
    ↓
Linear(64, M objectives)

Parameters: ~34,000 trainable

Workflow Modes

1. Standard Hybrid Mode (--all)

Run all phases sequentially:

python run_nn_optimization.py --all

Phases:

1. Export: Extract training data from existing FEA trials
2. Train: Train MLP surrogate (300 epochs default)
3. NN-Optimize: Run 1000 NN trials with NSGA-II
4. Validate: Validate top 10 candidates with FEA

2. Hybrid Loop Mode (--hybrid-loop)

Iterative refinement:

python run_nn_optimization.py --hybrid-loop --iterations 5 --nn-trials 500

Each iteration:

1. Train/retrain surrogate from current FEA data
2. Run NN optimization
3. Validate top candidates with FEA
4. Add validated results to training set
5. Repeat until convergence (max error < 5%)

3. Turbo Mode (--turbo) ⚡ RECOMMENDED

Aggressive single-best validation:

python run_nn_optimization.py --turbo --nn-trials 5000 --batch-size 100 --retrain-every 10

Strategy:

• Run NN in small batches (100 trials)
• Validate ONLY the single best candidate with FEA
• Add to training data immediately
• Retrain surrogate every N FEA validations
• Repeat until total NN budget exhausted

Example: 5,000 NN trials with batch=100 → 50 FEA validations in ~12 minutes

Configuration

{
  "neural_acceleration": {
    "enabled": true,
    "min_training_points": 50,
    "auto_train": true,
    "epochs": 300,
    "validation_split": 0.2,
    "nn_trials": 1000,
    "validate_top_n": 10,
    "model_file": "surrogate_best.pt",
    "separate_nn_database": true
  }
}

Important: separate_nn_database: true stores NN trials in nn_study.db instead of study.db to avoid overloading the dashboard with thousands of NN-only results.

Typical Accuracy

Objective	Expected Error
Mass	1-5%
Stress	1-4%
Stiffness	5-15%

Output Files

2_results/
├── study.db                    # Main FEA + validated results (dashboard)
├── nn_study.db                 # NN-only results (not in dashboard)
├── surrogate_best.pt           # Trained model weights
├── training_data.json          # Normalized training data
├── nn_optimization_state.json  # NN optimization state
├── nn_pareto_front.json        # NN-predicted Pareto front
├── validation_report.json      # FEA validation results
└── turbo_report.json           # Turbo mode results (if used)

---

Zernike GNN (Mirror Optimization)

Overview

The Zernike GNN is a specialized Graph Neural Network for mirror surface optimization. Unlike the MLP surrogate that predicts objectives directly, the Zernike GNN predicts the full displacement field, then computes Zernike coefficients and objectives via differentiable layers.

Why GNN over MLP for Zernike?

1. Spatial awareness: GNN learns smooth deformation fields via message passing
2. Correct relative computation: Predicts fields, then subtracts (like FEA)
3. Multi-task learning: Field + objective supervision
4. Physics-informed: Edge structure respects mirror geometry

Architecture

Design Variables [11]
      │
      ▼
Design Encoder [11 → 128]
      │
      └──────────────────┐
                         │
Node Features            │
[r, θ, x, y]             │
      │                  │
      ▼                  │
Node Encoder             │
[4 → 128]                │
      │                  │
      └─────────┬────────┘
                │
                ▼
┌─────────────────────────────┐
│ Design-Conditioned          │
│ Message Passing (× 6)       │
│                             │
│ • Polar-aware edges         │
│ • Design modulates messages │
│ • Residual connections      │
└─────────────┬───────────────┘
              │
              ▼
Per-Node Decoder [128 → 4]
              │
              ▼
Z-Displacement Field [3000, 4]
(one value per node per subcase)
              │
              ▼
┌─────────────────────────────┐
│ DifferentiableZernikeFit    │
│ (GPU-accelerated)           │
└─────────────┬───────────────┘
              │
              ▼
Zernike Coefficients → Objectives

Module Structure

optimization_engine/gnn/
├── __init__.py              # Public API
├── polar_graph.py           # PolarMirrorGraph - fixed polar grid
├── zernike_gnn.py           # ZernikeGNN model (design-conditioned conv)
├── differentiable_zernike.py # GPU Zernike fitting & objective layers
├── extract_displacement_field.py # OP2 → HDF5 field extraction
├── train_zernike_gnn.py     # ZernikeGNNTrainer pipeline
├── gnn_optimizer.py         # ZernikeGNNOptimizer for turbo mode
└── backfill_field_data.py   # Extract fields from existing trials

Training Workflow

# Step 1: Extract displacement fields from FEA trials
python -m optimization_engine.gnn.backfill_field_data V11

# Step 2: Train GNN on extracted data
python -m optimization_engine.gnn.train_zernike_gnn V11 V12 --epochs 200

# Step 3: Run GNN-accelerated optimization
python run_gnn_turbo.py --trials 5000

Key Classes

Class	Purpose
PolarMirrorGraph	Fixed 3000-node polar grid for mirror surface
ZernikeGNN	Main model with design-conditioned message passing
DifferentiableZernikeFit	GPU-accelerated Zernike coefficient computation
ZernikeObjectiveLayer	Compute rel_rms objectives from coefficients
ZernikeGNNTrainer	Complete training pipeline with multi-task loss
ZernikeGNNOptimizer	Turbo optimization with GNN predictions

Calibration

GNN predictions require calibration against FEA ground truth. Use the full FEA dataset (not just validation samples) for robust calibration:

# compute_full_calibration.py
# Computes calibration factors: GNN_pred * factor ≈ FEA_truth
calibration_factors = {
    'rel_filtered_rms_40_vs_20': 1.15,  # GNN underpredicts by ~15%
    'rel_filtered_rms_60_vs_20': 1.08,
    'mfg_90_optician_workload': 0.95,   # GNN overpredicts by ~5%
}

Performance

Metric	FEA	Zernike GNN
Time per eval	8-10 min	4 ms
Trials per hour	6-7	900,000
Typical accuracy	Ground truth	5-15% error

---

GNN Field Predictor (Generic)

Core Components

Component	File	Purpose
BDF/OP2 Parser	neural_field_parser.py	Convert NX files to neural format
Data Validator	validate_parsed_data.py	Physics and quality checks
Field Predictor	field_predictor.py	GNN for full field prediction
Parametric Predictor	parametric_predictor.py	GNN for direct objectives
Physics Loss	physics_losses.py	Physics-informed training
Neural Surrogate	neural_surrogate.py	Integration with Atomizer
Neural Runner	runner_with_neural.py	Optimization with NN acceleration

Workflow Diagram

Traditional:
Design → NX Model → Mesh → Solve (30 min) → Results → Objective

Neural (after training):
Design → Neural Network (4.5 ms) → Results → Objective

---

Neural Model Types

1. Field Predictor GNN

Use Case: When you need full field predictions (stress distribution, deformation shape).

Input Features (12D per node):
├── Node coordinates (x, y, z)
├── Material properties (E, nu, rho)
├── Boundary conditions (fixed/free per DOF)
└── Load information (force magnitude, direction)

GNN Layers (6 message passing):
├── MeshGraphConv (custom for FEA topology)
├── Layer normalization
├── ReLU activation
└── Dropout (0.1)

Output (per node):
├── Displacement (6 DOF: Tx, Ty, Tz, Rx, Ry, Rz)
└── Von Mises stress (1 value)

Parameters: ~718,221 trainable

2. Parametric Predictor GNN (Recommended)

Use Case: Direct optimization objective prediction (fastest option).

Design Parameters (ND) → Design Encoder (MLP) → GNN Backbone → Scalar Heads

Output (objectives):
├── mass (grams)
├── frequency (Hz)
├── max_displacement (mm)
└── max_stress (MPa)

Parameters: ~500,000 trainable

3. Ensemble Models

Use Case: Uncertainty quantification.

1. Train 3-5 models with different random seeds
2. At inference, run all models
3. Use mean for prediction, std for uncertainty
4. High uncertainty → trigger FEA validation

---

Training Pipeline

Step 1: Collect Training Data

Enable export in workflow config:

{
  "training_data_export": {
    "enabled": true,
    "export_dir": "atomizer_field_training_data/my_study"
  }
}

Output structure:

atomizer_field_training_data/my_study/
├── trial_0001/
│   ├── input/model.bdf       # Nastran input
│   ├── output/model.op2      # Binary results
│   └── metadata.json         # Design params + objectives
├── trial_0002/
│   └── ...
└── study_summary.json

Recommended: 100-500 FEA samples for good generalization.

Step 2: Parse to Neural Format

cd atomizer-field
python batch_parser.py ../atomizer_field_training_data/my_study

Creates HDF5 + JSON files per trial.

Step 3: Train Model

Parametric Predictor (recommended):

python train_parametric.py \
  --train_dir ../training_data/parsed \
  --val_dir ../validation_data/parsed \
  --epochs 200 \
  --hidden_channels 128 \
  --num_layers 4

Field Predictor:

python train.py \
  --train_dir ../training_data/parsed \
  --epochs 200 \
  --model FieldPredictorGNN \
  --hidden_channels 128 \
  --num_layers 6 \
  --physics_loss_weight 0.3

Step 4: Validate

python validate.py --checkpoint runs/my_model/checkpoint_best.pt

Expected output:

Validation Results:
├── Mean Absolute Error: 2.3% (mass), 1.8% (frequency)
├── R² Score: 0.987
├── Inference Time: 4.5ms ± 0.8ms
└── Physics Violations: 0.2%

Step 5: Deploy

{
  "neural_surrogate": {
    "enabled": true,
    "model_checkpoint": "atomizer-field/runs/my_model/checkpoint_best.pt",
    "confidence_threshold": 0.85
  }
}

---

Configuration

Full Neural Configuration Example

{
  "study_name": "bracket_neural_optimization",

  "surrogate_settings": {
    "enabled": true,
    "model_type": "parametric_gnn",
    "model_path": "models/bracket_surrogate.pt",
    "confidence_threshold": 0.85,
    "validation_frequency": 10,
    "fallback_to_fea": true
  },

  "training_data_export": {
    "enabled": true,
    "export_dir": "atomizer_field_training_data/bracket_study",
    "export_bdf": true,
    "export_op2": true,
    "export_fields": ["displacement", "stress"]
  },

  "neural_optimization": {
    "initial_fea_trials": 50,
    "neural_trials": 5000,
    "retraining_interval": 500,
    "uncertainty_threshold": 0.15
  }
}

Configuration Parameters

Parameter	Type	Default	Description
enabled	bool	false	Enable neural surrogate
model_type	string	"parametric_gnn"	Model architecture
model_path	string	-	Path to trained model
confidence_threshold	float	0.85	Min confidence for predictions
validation_frequency	int	10	FEA validation every N trials
fallback_to_fea	bool	true	Use FEA when uncertain

---

Hybrid FEA/Neural Workflow

Phase 1: FEA Exploration (50-100 trials)

• Run standard FEA optimization
• Export training data automatically
• Build landscape understanding

Phase 2: Neural Training

• Parse collected data
• Train parametric predictor
• Validate accuracy

Phase 3: Neural Acceleration (1000s of trials)

• Use neural network for rapid exploration
• Periodic FEA validation
• Retrain if distribution shifts

Phase 4: FEA Refinement (10-20 trials)

• Validate top candidates with FEA
• Ensure results are physically accurate
• Generate final Pareto front

---

Adaptive Iteration Loop

For complex optimizations, use iterative refinement:

┌─────────────────────────────────────────────────────────────────┐
│  Iteration 1:                                                    │
│  ┌──────────────┐    ┌──────────────┐    ┌──────────────┐       │
│  │ Initial FEA  │ -> │ Train NN     │ -> │ NN Search    │       │
│  │ (50-100)     │    │ Surrogate    │    │ (1000 trials)│       │
│  └──────────────┘    └──────────────┘    └──────────────┘       │
│                                                 │                │
│  Iteration 2+:                                  ▼                │
│  ┌──────────────┐    ┌──────────────┐    ┌──────────────┐       │
│  │ Validate Top │ -> │ Retrain NN   │ -> │ NN Search    │       │
│  │ NN with FEA  │    │ with new data│    │ (1000 trials)│       │
│  └──────────────┘    └──────────────┘    └──────────────┘       │
└─────────────────────────────────────────────────────────────────┘

Adaptive Configuration

{
  "adaptive_settings": {
    "enabled": true,
    "initial_fea_trials": 50,
    "nn_trials_per_iteration": 1000,
    "fea_validation_per_iteration": 5,
    "max_iterations": 10,
    "convergence_threshold": 0.01,
    "retrain_epochs": 100
  }
}

Convergence Criteria

Stop when:

• No improvement for 2-3 consecutive iterations
• Reached FEA budget limit
• Objective improvement < 1% threshold

Output Files

studies/my_study/3_results/
├── adaptive_state.json      # Current iteration state
├── surrogate_model.pt       # Trained neural network
└── training_history.json    # NN training metrics

---

Loss Functions

Data Loss (MSE)

Standard prediction error:

data_loss = MSE(predicted, target)

Physics Loss

Enforce physical constraints:

physics_loss = (
    equilibrium_loss +      # Force balance
    boundary_loss +         # BC satisfaction
    compatibility_loss      # Strain compatibility
)

Combined Training

total_loss = data_loss + 0.3 * physics_loss

Physics loss weight typically 0.1-0.5.

---

Uncertainty Quantification

Ensemble Method

# Run N models
predictions = [model_i(x) for model_i in ensemble]

# Statistics
mean_prediction = np.mean(predictions)
uncertainty = np.std(predictions)

# Decision
if uncertainty > threshold:
    # Use FEA instead
    result = run_fea(x)
else:
    result = mean_prediction

Confidence Thresholds

Uncertainty	Action
< 5%	Use neural prediction
5-15%	Use neural, flag for validation
> 15%	Fall back to FEA

---

Troubleshooting

Symptom	Cause	Solution
High prediction error	Insufficient training data	Collect more FEA samples
Out-of-distribution warnings	Design outside training range	Retrain with expanded range
Slow inference	Large mesh	Use parametric predictor instead
Physics violations	Low physics loss weight	Increase physics_loss_weight

---

Cross-References

• Depends On: SYS_10_IMSO for optimization framework
• Used By: OP_02_RUN_OPTIMIZATION, OP_05_EXPORT_TRAINING_DATA
• See Also: modules/neural-acceleration.md

---

Implementation Files

atomizer-field/
├── neural_field_parser.py       # BDF/OP2 parsing
├── field_predictor.py           # Field GNN
├── parametric_predictor.py      # Parametric GNN
├── train.py                     # Field training
├── train_parametric.py          # Parametric training
├── validate.py                  # Model validation
├── physics_losses.py            # Physics-informed loss
└── batch_parser.py              # Batch data conversion

optimization_engine/
├── neural_surrogate.py          # Atomizer integration
└── runner_with_neural.py        # Neural runner

---

Self-Improving Turbo Optimization

Overview

The Self-Improving Turbo pattern combines MLP surrogate exploration with iterative FEA validation and surrogate retraining. This creates a closed-loop optimization where the surrogate continuously improves from its own mistakes.

Workflow

INITIALIZE:
  - Load pre-trained surrogate (from prior FEA data)
  - Load previous FEA params for diversity checking

REPEAT until converged or FEA budget exhausted:

  1. SURROGATE EXPLORE (~1 min)
     ├─ Run 5000 Optuna TPE trials with surrogate
     ├─ Quantize predictions to machining precision
     └─ Find diverse top candidates

  2. SELECT DIVERSE CANDIDATES
     ├─ Sort by weighted sum
     ├─ Select top 5 that are:
     │   ├─ At least 15% different from each other
     │   └─ At least 7.5% different from ALL previous FEA
     └─ Ensures exploration, not just exploitation

  3. FEA VALIDATE (~25 min for 5 candidates)
     ├─ For each candidate:
     │   ├─ Create iteration folder
     │   ├─ Update NX expressions
     │   ├─ Run Nastran solver
     │   ├─ Extract objectives (ZernikeOPD or other)
     │   └─ Log prediction error
     └─ Add results to training data

  4. RETRAIN SURROGATE (~2 min)
     ├─ Combine all FEA samples
     ├─ Retrain MLP for 100 epochs
     ├─ Save new checkpoint
     └─ Reload improved model

  5. CHECK CONVERGENCE
     ├─ Track best feasible objective
     ├─ If improved: reset patience counter
     └─ If no improvement for 3 iterations: STOP

Configuration Example

{
  "turbo_settings": {
    "surrogate_trials_per_iteration": 5000,
    "fea_validations_per_iteration": 5,
    "max_fea_validations": 100,
    "max_iterations": 30,
    "convergence_patience": 3,
    "retrain_frequency": "every_iteration",
    "min_samples_for_retrain": 20
  }
}

Key Parameters

Parameter	Typical Value	Description
surrogate_trials_per_iteration	5000	NN trials per iteration
fea_validations_per_iteration	5	FEA runs per iteration
max_fea_validations	100	Total FEA budget
convergence_patience	3	Stop after N no-improvement iterations
MIN_CANDIDATE_DISTANCE	0.15	15% of param range for diversity

Example Results (M1 Mirror Turbo V1)

Metric	Value
FEA Validations	45
Best WS Found	282.05
Baseline (V11)	284.19
Improvement	0.75%

---

Dashboard Integration for Neural Studies

Problem

Neural surrogate studies generate thousands of NN-only trials that would overwhelm the dashboard. Only FEA-validated trials should be visible.

Solution: Separate Optuna Study

Log FEA validation results to a separate Optuna study that the dashboard can read:

import optuna

# Create Optuna study for dashboard visibility
optuna_db_path = RESULTS_DIR / "study.db"
optuna_storage = f"sqlite:///{optuna_db_path}"
optuna_study = optuna.create_study(
    study_name=study_name,
    storage=optuna_storage,
    direction="minimize",
    load_if_exists=True,
)

# After each FEA validation:
trial = optuna_study.ask()

# Set parameters (using suggest_float with fixed bounds)
for var_name, var_val in result['params'].items():
    trial.suggest_float(var_name, var_val, var_val)

# Set objectives as user attributes
for obj_name, obj_val in result['objectives'].items():
    trial.set_user_attr(obj_name, obj_val)

# Log iteration metadata
trial.set_user_attr('turbo_iteration', turbo_iter)
trial.set_user_attr('prediction_error', abs(actual_ws - predicted_ws))
trial.set_user_attr('is_feasible', is_feasible)

# Report the objective value
optuna_study.tell(trial, result['weighted_sum'])

File Structure

3_results/
├── study.db              # Optuna format (for dashboard)
├── study_custom.db       # Custom SQLite (detailed turbo data)
├── checkpoints/
│   └── best_model.pt     # Surrogate model
├── turbo_logs/           # Per-iteration JSON logs
└── best_design_archive/  # Archived best designs

Backfilling Existing Data

If you have existing turbo runs without Optuna logging, use the backfill script:

# scripts/backfill_optuna.py
import optuna
import sqlite3
import json

# Read from custom database
conn = sqlite3.connect('study_custom.db')
c.execute('''
    SELECT iter_num, turbo_iteration, weighted_sum, surrogate_predicted_ws,
           params, objectives, is_feasible
    FROM trials ORDER BY iter_num
''')

# Create Optuna study
study = optuna.create_study(...)

# Backfill each trial
for row in rows:
    trial = study.ask()
    params = json.loads(row['params'])  # Stored as JSON
    objectives = json.loads(row['objectives'])

    for name, val in params.items():
        trial.suggest_float(name, float(val), float(val))
    for name, val in objectives.items():
        trial.set_user_attr(name, float(val))

    study.tell(trial, row['weighted_sum'])

Dashboard View

After integration, the dashboard shows:

• Only FEA-validated trials (not NN-only)
• Objective convergence over FEA iterations
• Parameter distributions from validated designs
• Prediction error trends (via user attributes)

---

L-BFGS Gradient Optimizer (v2.4)

Overview

The L-BFGS Gradient Optimizer exploits the differentiability of trained MLP surrogates to achieve 100-1000x faster convergence compared to derivative-free methods like TPE or CMA-ES.

Key insight: Your trained MLP is fully differentiable. L-BFGS computes exact gradients via backpropagation, enabling precise local optimization.

When to Use

Scenario	Use L-BFGS?
After turbo mode identifies promising regions	✓ Yes
To polish top 10-20 candidates before FEA	✓ Yes
For initial exploration (cold start)	✗ No - use TPE/grid first
Multi-modal problems (many local minima)	Use multi-start L-BFGS

Quick Start

# CLI usage
python -m optimization_engine.gradient_optimizer studies/my_study --n-starts 20

# Or per-study script
cd studies/M1_Mirror/m1_mirror_adaptive_V14
python run_lbfgs_polish.py --n-starts 20

Python API

from optimization_engine.gradient_optimizer import GradientOptimizer, run_lbfgs_polish
from optimization_engine.generic_surrogate import GenericSurrogate

# Method 1: Quick run from study directory
results = run_lbfgs_polish(
    study_dir="studies/my_study",
    n_starts=20,           # Starting points
    use_top_fea=True,      # Use top FEA results as starts
    n_iterations=100       # L-BFGS iterations per start
)

# Method 2: Full control
surrogate = GenericSurrogate(config)
surrogate.load("surrogate_best.pt")

optimizer = GradientOptimizer(
    surrogate=surrogate,
    objective_weights=[5.0, 5.0, 1.0],  # From config
    objective_directions=['minimize', 'minimize', 'minimize']
)

# Multi-start optimization
result = optimizer.optimize(
    starting_points=top_candidates,  # List of param dicts
    n_random_restarts=10,            # Additional random starts
    method='lbfgs',                  # 'lbfgs', 'adam', or 'sgd'
    n_iterations=100
)

# Access results
print(f"Best WS: {result.weighted_sum}")
print(f"Params: {result.params}")
print(f"Improvement: {result.improvement}")

Hybrid Grid + Gradient Mode

For problems with multiple local minima:

results = optimizer.grid_search_then_gradient(
    n_grid_samples=500,        # Random exploration
    n_top_for_gradient=20,     # Top candidates to polish
    n_iterations=100           # L-BFGS iterations
)

Integration with Turbo Mode

Recommended workflow:

1. FEA Exploration (50-100 trials) → Train initial surrogate
2. Turbo Mode (5000 NN trials) → Find promising regions
3. L-BFGS Polish (20 starts) → Precise local optima    ← NEW
4. FEA Validation (top 3-5) → Verify best designs

Output

Results saved to 3_results/lbfgs_results.json:

{
  "results": [
    {
      "params": {"rib_thickness": 10.42, ...},
      "objectives": {"wfe_40_20": 5.12, ...},
      "weighted_sum": 172.34,
      "converged": true,
      "improvement": 8.45
    }
  ]
}

Performance Comparison

Method	Evaluations to Converge	Time
TPE	200-500	30 min (surrogate)
CMA-ES	100-300	15 min (surrogate)
L-BFGS	20-50	<1 sec

Key Classes

Class	Purpose
GradientOptimizer	Main optimizer with L-BFGS/Adam/SGD
OptimizationResult	Result container with params, objectives, convergence info
run_lbfgs_polish()	Convenience function for study-level usage
MultiStartLBFGS	Simplified multi-start interface

Implementation Details

• Bounds handling: Projected gradient (clamp to bounds after each step)
• Normalization: Inherits from surrogate (design_mean/std, obj_mean/std)
• Convergence: Gradient norm < tolerance (default 1e-7)
• Line search: Strong Wolfe conditions for L-BFGS

---

Version History

Version	Date	Changes
2.4	2025-12-28	Added L-BFGS Gradient Optimizer for surrogate polish
2.3	2025-12-28	Added TrialManager, DashboardDB, proper trial_NNNN naming
2.2	2025-12-24	Added Self-Improving Turbo and Dashboard Integration sections
2.1	2025-12-10	Added Zernike GNN section for mirror optimization
2.0	2025-12-06	Added MLP Surrogate with Turbo Mode
1.0	2025-12-05	Initial consolidation from neural docs

---

New Trial Management System (v2.3)

Overview

The new trial management system provides:

1. Consistent trial naming: trial_NNNN/ folders (zero-padded, never reused)
2. Dashboard compatibility: Optuna-compatible SQLite schema
3. Clear separation: Surrogate predictions are ephemeral, only FEA results are trials

Key Components

Component	File	Purpose
TrialManager	optimization_engine/utils/trial_manager.py	Trial folder + DB management
DashboardDB	optimization_engine/utils/dashboard_db.py	Optuna-compatible database ops

Usage Pattern

from optimization_engine.utils.trial_manager import TrialManager

# Initialize
tm = TrialManager(study_dir, "my_study")

# Start trial (creates folder, reserves DB row)
trial = tm.new_trial(
    params={'rib_thickness': 10.5},
    source="turbo",
    metadata={'turbo_batch': 1, 'predicted_ws': 186.77}
)

# Run FEA...

# Complete trial (logs to DB)
tm.complete_trial(
    trial_number=trial['trial_number'],
    objectives={'wfe_40_20': 5.63, 'mass_kg': 118.67},
    weighted_sum=175.87,
    is_feasible=True,
    metadata={'solve_time': 211.7}
)

Trial Folder Structure

2_iterations/
├── trial_0001/
│   ├── params.json      # Input parameters
│   ├── params.exp       # NX expression format
│   ├── results.json     # Output objectives
│   ├── _meta.json       # Full metadata (source, timestamps, predictions)
│   └── *.op2, *.fem...  # FEA files
├── trial_0002/
└── ...

Database Schema

The DashboardDB class creates Optuna-compatible tables:

Table	Purpose
studies	Study metadata
trials	Trial info with state, number, study_id
trial_values	Objective values
trial_params	Parameter values
trial_user_attributes	Custom metadata (turbo_batch, predicted_ws, etc.)

Converting Legacy Databases

from optimization_engine.utils.dashboard_db import convert_custom_to_optuna

# Convert custom schema to Optuna format
convert_custom_to_optuna(
    db_path="3_results/study.db",
    study_name="my_study"
)

Key Principles

1. Surrogate predictions are NOT trials - only FEA-validated results are logged
2. Trial numbers never reset - monotonically increasing across all runs
3. Folders never overwritten - each trial gets a unique trial_NNNN/ directory
4. Metadata preserved - predictions stored for accuracy analysis