feat: Add MLP surrogate with Turbo Mode for 100x faster optimization
Neural Acceleration (MLP Surrogate): - Add run_nn_optimization.py with hybrid FEA/NN workflow - MLP architecture: 4-layer (64->128->128->64) with BatchNorm/Dropout - Three workflow modes: - --all: Sequential export->train->optimize->validate - --hybrid-loop: Iterative Train->NN->Validate->Retrain cycle - --turbo: Aggressive single-best validation (RECOMMENDED) - Turbo mode: 5000 NN trials + 50 FEA validations in ~12 minutes - Separate nn_study.db to avoid overloading dashboard Performance Results (bracket_pareto_3obj study): - NN prediction errors: mass 1-5%, stress 1-4%, stiffness 5-15% - Found minimum mass designs at boundary (angle~30deg, thick~30mm) - 100x speedup vs pure FEA exploration Protocol Operating System: - Add .claude/skills/ with Bootstrap, Cheatsheet, Context Loader - Add docs/protocols/ with operations (OP_01-06) and system (SYS_10-14) - Update SYS_14_NEURAL_ACCELERATION.md with MLP Turbo Mode docs NX Automation: - Add optimization_engine/hooks/ for NX CAD/CAE automation - Add study_wizard.py for guided study creation - Fix FEM mesh update: load idealized part before UpdateFemodel() New Study: - bracket_pareto_3obj: 3-objective Pareto (mass, stress, stiffness) - 167 FEA trials + 5000 NN trials completed - Demonstrates full hybrid workflow 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
Antoine
341
docs/protocols/system/SYS_10_IMSO.md
Normal file
341
docs/protocols/system/SYS_10_IMSO.md
Normal file
@@ -0,0 +1,341 @@
|
||||
# SYS_10: Intelligent Multi-Strategy Optimization (IMSO)
|
||||
|
||||
<!--
|
||||
PROTOCOL: Intelligent Multi-Strategy Optimization
|
||||
LAYER: System
|
||||
VERSION: 2.1
|
||||
STATUS: Active
|
||||
LAST_UPDATED: 2025-12-05
|
||||
PRIVILEGE: user
|
||||
LOAD_WITH: []
|
||||
-->
|
||||
|
||||
## Overview
|
||||
|
||||
Protocol 10 implements adaptive optimization that automatically characterizes the problem landscape and selects the best optimization algorithm. This two-phase approach combines automated landscape analysis with algorithm-specific optimization.
|
||||
|
||||
**Key Innovation**: Adaptive characterization phase that intelligently determines when enough exploration has been done, then transitions to the optimal algorithm.
|
||||
|
||||
---
|
||||
|
||||
## When to Use
|
||||
|
||||
| Trigger | Action |
|
||||
|---------|--------|
|
||||
| Single-objective optimization | Use this protocol |
|
||||
| "adaptive", "intelligent", "IMSO" mentioned | Load this protocol |
|
||||
| User unsure which algorithm to use | Recommend this protocol |
|
||||
| Complex landscape suspected | Use this protocol |
|
||||
|
||||
**Do NOT use when**: Multi-objective optimization needed (use SYS_11 instead)
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Parameter | Default | Range | Description |
|
||||
|-----------|---------|-------|-------------|
|
||||
| `min_trials` | 10 | 5-50 | Minimum characterization trials |
|
||||
| `max_trials` | 30 | 10-100 | Maximum characterization trials |
|
||||
| `confidence_threshold` | 0.85 | 0.0-1.0 | Stopping confidence level |
|
||||
| `check_interval` | 5 | 1-10 | Trials between checks |
|
||||
|
||||
**Landscape → Algorithm Mapping**:
|
||||
|
||||
| Landscape Type | Primary Strategy | Fallback |
|
||||
|----------------|------------------|----------|
|
||||
| smooth_unimodal | GP-BO | CMA-ES |
|
||||
| smooth_multimodal | GP-BO | TPE |
|
||||
| rugged_unimodal | TPE | CMA-ES |
|
||||
| rugged_multimodal | TPE | - |
|
||||
| noisy | TPE | - |
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ PHASE 1: ADAPTIVE CHARACTERIZATION STUDY │
|
||||
│ ───────────────────────────────────────────────────────── │
|
||||
│ Sampler: Random/Sobol (unbiased exploration) │
|
||||
│ Trials: 10-30 (adapts to problem complexity) │
|
||||
│ │
|
||||
│ Every 5 trials: │
|
||||
│ → Analyze landscape metrics │
|
||||
│ → Check metric convergence │
|
||||
│ → Calculate characterization confidence │
|
||||
│ → Decide if ready to stop │
|
||||
│ │
|
||||
│ Stop when: │
|
||||
│ ✓ Confidence ≥ 85% │
|
||||
│ ✓ OR max trials reached (30) │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ TRANSITION: LANDSCAPE ANALYSIS & STRATEGY SELECTION │
|
||||
│ ───────────────────────────────────────────────────────── │
|
||||
│ Analyze: │
|
||||
│ - Smoothness (0-1) │
|
||||
│ - Multimodality (number of modes) │
|
||||
│ - Parameter correlation │
|
||||
│ - Noise level │
|
||||
│ │
|
||||
│ Classify & Recommend: │
|
||||
│ smooth_unimodal → GP-BO (best) or CMA-ES │
|
||||
│ smooth_multimodal → GP-BO │
|
||||
│ rugged_multimodal → TPE │
|
||||
│ rugged_unimodal → TPE or CMA-ES │
|
||||
│ noisy → TPE (most robust) │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ PHASE 2: OPTIMIZATION STUDY │
|
||||
│ ───────────────────────────────────────────────────────── │
|
||||
│ Sampler: Recommended from Phase 1 │
|
||||
│ Warm Start: Initialize from best characterization point │
|
||||
│ Trials: User-specified (default 50) │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Core Components
|
||||
|
||||
### 1. Adaptive Characterization (`adaptive_characterization.py`)
|
||||
|
||||
**Confidence Calculation**:
|
||||
```python
|
||||
confidence = (
|
||||
0.40 * metric_stability_score + # Are metrics converging?
|
||||
0.30 * parameter_coverage_score + # Explored enough space?
|
||||
0.20 * sample_adequacy_score + # Enough samples for complexity?
|
||||
0.10 * landscape_clarity_score # Clear classification?
|
||||
)
|
||||
```
|
||||
|
||||
**Stopping Criteria**:
|
||||
- **Minimum trials**: 10 (baseline data requirement)
|
||||
- **Maximum trials**: 30 (prevent over-characterization)
|
||||
- **Confidence threshold**: 85% (high confidence required)
|
||||
- **Check interval**: Every 5 trials
|
||||
|
||||
**Adaptive Behavior**:
|
||||
```python
|
||||
# Simple problem (smooth, unimodal, low noise):
|
||||
if smoothness > 0.6 and unimodal and noise < 0.3:
|
||||
required_samples = 10 + dimensionality
|
||||
# Stops at ~10-15 trials
|
||||
|
||||
# Complex problem (multimodal with N modes):
|
||||
if multimodal and n_modes > 2:
|
||||
required_samples = 10 + 5 * n_modes + 2 * dimensionality
|
||||
# Continues to ~20-30 trials
|
||||
```
|
||||
|
||||
### 2. Landscape Analyzer (`landscape_analyzer.py`)
|
||||
|
||||
**Metrics Computed**:
|
||||
|
||||
| Metric | Method | Interpretation |
|
||||
|--------|--------|----------------|
|
||||
| Smoothness (0-1) | Spearman correlation | >0.6: Good for CMA-ES, GP-BO |
|
||||
| Multimodality | DBSCAN clustering | Detects distinct good regions |
|
||||
| Correlation | Parameter-objective correlation | Identifies influential params |
|
||||
| Noise (0-1) | Local consistency check | True simulation instability |
|
||||
|
||||
**Landscape Classifications**:
|
||||
- `smooth_unimodal`: Single smooth bowl
|
||||
- `smooth_multimodal`: Multiple smooth regions
|
||||
- `rugged_unimodal`: Single rugged region
|
||||
- `rugged_multimodal`: Multiple rugged regions
|
||||
- `noisy`: High noise level
|
||||
|
||||
### 3. Strategy Selector (`strategy_selector.py`)
|
||||
|
||||
**Algorithm Characteristics**:
|
||||
|
||||
**GP-BO (Gaussian Process Bayesian Optimization)**:
|
||||
- Best for: Smooth, expensive functions (like FEA)
|
||||
- Explicit surrogate model with uncertainty quantification
|
||||
- Acquisition function balances exploration/exploitation
|
||||
|
||||
**CMA-ES (Covariance Matrix Adaptation Evolution Strategy)**:
|
||||
- Best for: Smooth unimodal problems
|
||||
- Fast convergence to local optimum
|
||||
- Adapts search distribution to landscape
|
||||
|
||||
**TPE (Tree-structured Parzen Estimator)**:
|
||||
- Best for: Multimodal, rugged, or noisy problems
|
||||
- Robust to noise and discontinuities
|
||||
- Good global exploration
|
||||
|
||||
### 4. Intelligent Optimizer (`intelligent_optimizer.py`)
|
||||
|
||||
**Workflow**:
|
||||
1. Create characterization study (Random/Sobol sampler)
|
||||
2. Run adaptive characterization with stopping criterion
|
||||
3. Analyze final landscape
|
||||
4. Select optimal strategy
|
||||
5. Create optimization study with recommended sampler
|
||||
6. Warm-start from best characterization point
|
||||
7. Run optimization
|
||||
8. Generate intelligence report
|
||||
|
||||
---
|
||||
|
||||
## Configuration
|
||||
|
||||
Add to `optimization_config.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"intelligent_optimization": {
|
||||
"enabled": true,
|
||||
"characterization": {
|
||||
"min_trials": 10,
|
||||
"max_trials": 30,
|
||||
"confidence_threshold": 0.85,
|
||||
"check_interval": 5
|
||||
},
|
||||
"landscape_analysis": {
|
||||
"min_trials_for_analysis": 10
|
||||
},
|
||||
"strategy_selection": {
|
||||
"allow_cmaes": true,
|
||||
"allow_gpbo": true,
|
||||
"allow_tpe": true
|
||||
}
|
||||
},
|
||||
"trials": {
|
||||
"n_trials": 50
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Usage Example
|
||||
|
||||
```python
|
||||
from pathlib import Path
|
||||
from optimization_engine.intelligent_optimizer import IntelligentOptimizer
|
||||
|
||||
# Create optimizer
|
||||
optimizer = IntelligentOptimizer(
|
||||
study_name="my_optimization",
|
||||
study_dir=Path("studies/my_study/2_results"),
|
||||
config=optimization_config,
|
||||
verbose=True
|
||||
)
|
||||
|
||||
# Define design variables
|
||||
design_vars = {
|
||||
'parameter1': (lower_bound, upper_bound),
|
||||
'parameter2': (lower_bound, upper_bound)
|
||||
}
|
||||
|
||||
# Run Protocol 10
|
||||
results = optimizer.optimize(
|
||||
objective_function=my_objective,
|
||||
design_variables=design_vars,
|
||||
n_trials=50,
|
||||
target_value=target,
|
||||
tolerance=0.1
|
||||
)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Performance Benefits
|
||||
|
||||
**Efficiency**:
|
||||
- **Simple problems**: Early stop at ~10-15 trials (33% reduction)
|
||||
- **Complex problems**: Extended characterization at ~20-30 trials
|
||||
- **Right algorithm**: Uses optimal strategy for landscape type
|
||||
|
||||
**Example Performance** (Circular Plate Frequency Tuning):
|
||||
- TPE alone: ~95 trials to target
|
||||
- Random search: ~150+ trials
|
||||
- **Protocol 10**: ~56 trials (**41% reduction**)
|
||||
|
||||
---
|
||||
|
||||
## Intelligence Reports
|
||||
|
||||
Protocol 10 generates three tracking files:
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `characterization_progress.json` | Metric evolution, confidence progression, stopping decision |
|
||||
| `intelligence_report.json` | Final landscape classification, parameter correlations, strategy recommendation |
|
||||
| `strategy_transitions.json` | Phase transitions, algorithm switches, performance metrics |
|
||||
|
||||
**Location**: `studies/{study_name}/2_results/intelligent_optimizer/`
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Cause | Solution |
|
||||
|---------|-------|----------|
|
||||
| Characterization takes too long | Complex landscape | Increase `max_trials` or accept longer characterization |
|
||||
| Wrong algorithm selected | Insufficient exploration | Lower `confidence_threshold` or increase `min_trials` |
|
||||
| Poor convergence | Mismatch between landscape and algorithm | Review `intelligence_report.json`, consider manual override |
|
||||
| "No characterization data" | Study not using Protocol 10 | Enable `intelligent_optimization.enabled: true` |
|
||||
|
||||
---
|
||||
|
||||
## Cross-References
|
||||
|
||||
- **Depends On**: None
|
||||
- **Used By**: [OP_01_CREATE_STUDY](../operations/OP_01_CREATE_STUDY.md), [OP_02_RUN_OPTIMIZATION](../operations/OP_02_RUN_OPTIMIZATION.md)
|
||||
- **Integrates With**: [SYS_13_DASHBOARD_TRACKING](./SYS_13_DASHBOARD_TRACKING.md)
|
||||
- **See Also**: [SYS_11_MULTI_OBJECTIVE](./SYS_11_MULTI_OBJECTIVE.md) for multi-objective optimization
|
||||
|
||||
---
|
||||
|
||||
## Implementation Files
|
||||
|
||||
- `optimization_engine/intelligent_optimizer.py` - Main orchestrator
|
||||
- `optimization_engine/adaptive_characterization.py` - Stopping criterion
|
||||
- `optimization_engine/landscape_analyzer.py` - Landscape metrics
|
||||
- `optimization_engine/strategy_selector.py` - Algorithm recommendation
|
||||
|
||||
---
|
||||
|
||||
## Version History
|
||||
|
||||
| Version | Date | Changes |
|
||||
|---------|------|---------|
|
||||
| 2.1 | 2025-11-20 | Fixed strategy selector timing, multimodality detection, added simulation validation |
|
||||
| 2.0 | 2025-11-20 | Added adaptive characterization, two-study architecture |
|
||||
| 1.0 | 2025-11-19 | Initial implementation |
|
||||
|
||||
### Version 2.1 Bug Fixes Detail
|
||||
|
||||
**Fix #1: Strategy Selector - Use Characterization Trial Count**
|
||||
|
||||
*Problem*: Strategy selector used total trial count (including pruned) instead of characterization trial count, causing wrong algorithm selection after characterization.
|
||||
|
||||
*Solution* (`strategy_selector.py`): Use `char_trials = landscape.get('total_trials', trials_completed)` for decisions.
|
||||
|
||||
**Fix #2: Improved Multimodality Detection**
|
||||
|
||||
*Problem*: False multimodality detected on smooth continuous surfaces (2 modes detected when problem was unimodal).
|
||||
|
||||
*Solution* (`landscape_analyzer.py`): Added heuristic - if only 2 modes with smoothness > 0.6 and noise < 0.2, reclassify as unimodal (smooth continuous manifold).
|
||||
|
||||
**Fix #3: Simulation Validation**
|
||||
|
||||
*Problem*: 20% pruning rate due to extreme parameters causing mesh/solver failures.
|
||||
|
||||
*Solution*: Created `simulation_validator.py` with:
|
||||
- Hard limits (reject invalid parameters)
|
||||
- Soft limits (warn about risky parameters)
|
||||
- Aspect ratio checks
|
||||
- Model-specific validation rules
|
||||
|
||||
*Impact*: Reduced pruning rate from 20% to ~5%.
|
||||
Reference in New Issue
Block a user