Atomizer/docs/protocols/system/SYS_10_IMSO.md

SYS_10: Intelligent Multi-Strategy Optimization (IMSO)

Overview

Protocol 10 implements adaptive optimization that automatically characterizes the problem landscape and selects the best optimization algorithm. This two-phase approach combines automated landscape analysis with algorithm-specific optimization.

Key Innovation: Adaptive characterization phase that intelligently determines when enough exploration has been done, then transitions to the optimal algorithm.

---

When to Use

Trigger	Action
Single-objective optimization	Use this protocol
"adaptive", "intelligent", "IMSO" mentioned	Load this protocol
User unsure which algorithm to use	Recommend this protocol
Complex landscape suspected	Use this protocol

Do NOT use when: Multi-objective optimization needed (use SYS_11 instead)

---

Quick Reference

Parameter	Default	Range	Description
min_trials	10	5-50	Minimum characterization trials
max_trials	30	10-100	Maximum characterization trials
confidence_threshold	0.85	0.0-1.0	Stopping confidence level
check_interval	5	1-10	Trials between checks

Landscape → Algorithm Mapping:

Landscape Type	Primary Strategy	Fallback
smooth_unimodal	GP-BO	CMA-ES
smooth_multimodal	GP-BO	TPE
rugged_unimodal	TPE	CMA-ES
rugged_multimodal	TPE	-
noisy	TPE	-

---

Architecture

┌─────────────────────────────────────────────────────────────┐
│  PHASE 1: ADAPTIVE CHARACTERIZATION STUDY                   │
│  ─────────────────────────────────────────────────────────  │
│  Sampler: Random/Sobol (unbiased exploration)               │
│  Trials: 10-30 (adapts to problem complexity)               │
│                                                              │
│  Every 5 trials:                                            │
│    → Analyze landscape metrics                              │
│    → Check metric convergence                               │
│    → Calculate characterization confidence                  │
│    → Decide if ready to stop                                │
│                                                              │
│  Stop when:                                                 │
│    ✓ Confidence ≥ 85%                                       │
│    ✓ OR max trials reached (30)                             │
└─────────────────────────────────────────────────────────────┘
                            │
                            ▼
┌─────────────────────────────────────────────────────────────┐
│  TRANSITION: LANDSCAPE ANALYSIS & STRATEGY SELECTION        │
│  ─────────────────────────────────────────────────────────  │
│  Analyze:                                                   │
│    - Smoothness (0-1)                                       │
│    - Multimodality (number of modes)                        │
│    - Parameter correlation                                  │
│    - Noise level                                            │
│                                                              │
│  Classify & Recommend:                                      │
│    smooth_unimodal    → GP-BO (best) or CMA-ES             │
│    smooth_multimodal  → GP-BO                               │
│    rugged_multimodal  → TPE                                 │
│    rugged_unimodal    → TPE or CMA-ES                       │
│    noisy              → TPE (most robust)                   │
└─────────────────────────────────────────────────────────────┘
                            │
                            ▼
┌─────────────────────────────────────────────────────────────┐
│  PHASE 2: OPTIMIZATION STUDY                                │
│  ─────────────────────────────────────────────────────────  │
│  Sampler: Recommended from Phase 1                          │
│  Warm Start: Initialize from best characterization point    │
│  Trials: User-specified (default 50)                        │
└─────────────────────────────────────────────────────────────┘

---

Core Components

1. Adaptive Characterization (adaptive_characterization.py)

Confidence Calculation:

confidence = (
    0.40 * metric_stability_score +      # Are metrics converging?
    0.30 * parameter_coverage_score +    # Explored enough space?
    0.20 * sample_adequacy_score +       # Enough samples for complexity?
    0.10 * landscape_clarity_score       # Clear classification?
)

Stopping Criteria:

• Minimum trials: 10 (baseline data requirement)
• Maximum trials: 30 (prevent over-characterization)
• Confidence threshold: 85% (high confidence required)
• Check interval: Every 5 trials

Adaptive Behavior:

# Simple problem (smooth, unimodal, low noise):
if smoothness > 0.6 and unimodal and noise < 0.3:
    required_samples = 10 + dimensionality
    # Stops at ~10-15 trials

# Complex problem (multimodal with N modes):
if multimodal and n_modes > 2:
    required_samples = 10 + 5 * n_modes + 2 * dimensionality
    # Continues to ~20-30 trials

2. Landscape Analyzer (landscape_analyzer.py)

Metrics Computed:

Metric	Method	Interpretation
Smoothness (0-1)	Spearman correlation	>0.6: Good for CMA-ES, GP-BO
Multimodality	DBSCAN clustering	Detects distinct good regions
Correlation	Parameter-objective correlation	Identifies influential params
Noise (0-1)	Local consistency check	True simulation instability

Landscape Classifications:

• smooth_unimodal: Single smooth bowl
• smooth_multimodal: Multiple smooth regions
• rugged_unimodal: Single rugged region
• rugged_multimodal: Multiple rugged regions
• noisy: High noise level

3. Strategy Selector (strategy_selector.py)

Algorithm Characteristics:

GP-BO (Gaussian Process Bayesian Optimization):

• Best for: Smooth, expensive functions (like FEA)
• Explicit surrogate model with uncertainty quantification
• Acquisition function balances exploration/exploitation

CMA-ES (Covariance Matrix Adaptation Evolution Strategy):

• Best for: Smooth unimodal problems
• Fast convergence to local optimum
• Adapts search distribution to landscape

TPE (Tree-structured Parzen Estimator):

• Best for: Multimodal, rugged, or noisy problems
• Robust to noise and discontinuities
• Good global exploration

4. Intelligent Optimizer (intelligent_optimizer.py)

Workflow:

1. Create characterization study (Random/Sobol sampler)
2. Run adaptive characterization with stopping criterion
3. Analyze final landscape
4. Select optimal strategy
5. Create optimization study with recommended sampler
6. Warm-start from best characterization point
7. Run optimization
8. Generate intelligence report

---

Configuration

Add to optimization_config.json:

{
  "intelligent_optimization": {
    "enabled": true,
    "characterization": {
      "min_trials": 10,
      "max_trials": 30,
      "confidence_threshold": 0.85,
      "check_interval": 5
    },
    "landscape_analysis": {
      "min_trials_for_analysis": 10
    },
    "strategy_selection": {
      "allow_cmaes": true,
      "allow_gpbo": true,
      "allow_tpe": true
    }
  },
  "trials": {
    "n_trials": 50
  }
}

---

Usage Example

from pathlib import Path
from optimization_engine.intelligent_optimizer import IntelligentOptimizer

# Create optimizer
optimizer = IntelligentOptimizer(
    study_name="my_optimization",
    study_dir=Path("studies/my_study/2_results"),
    config=optimization_config,
    verbose=True
)

# Define design variables
design_vars = {
    'parameter1': (lower_bound, upper_bound),
    'parameter2': (lower_bound, upper_bound)
}

# Run Protocol 10
results = optimizer.optimize(
    objective_function=my_objective,
    design_variables=design_vars,
    n_trials=50,
    target_value=target,
    tolerance=0.1
)

---

Performance Benefits

Efficiency:

• Simple problems: Early stop at ~10-15 trials (33% reduction)
• Complex problems: Extended characterization at ~20-30 trials
• Right algorithm: Uses optimal strategy for landscape type

Example Performance (Circular Plate Frequency Tuning):

• TPE alone: ~95 trials to target
• Random search: ~150+ trials
• Protocol 10: ~56 trials (41% reduction)

---

Intelligence Reports

Protocol 10 generates three tracking files:

File	Purpose
characterization_progress.json	Metric evolution, confidence progression, stopping decision
intelligence_report.json	Final landscape classification, parameter correlations, strategy recommendation
strategy_transitions.json	Phase transitions, algorithm switches, performance metrics

Location: studies/{study_name}/2_results/intelligent_optimizer/

---

Troubleshooting

Symptom	Cause	Solution
Characterization takes too long	Complex landscape	Increase max_trials or accept longer characterization
Wrong algorithm selected	Insufficient exploration	Lower confidence_threshold or increase min_trials
Poor convergence	Mismatch between landscape and algorithm	Review intelligence_report.json, consider manual override
"No characterization data"	Study not using Protocol 10	Enable intelligent_optimization.enabled: true

---

Cross-References

• Depends On: None
• Used By: OP_01_CREATE_STUDY, OP_02_RUN_OPTIMIZATION
• Integrates With: SYS_13_DASHBOARD_TRACKING
• See Also: SYS_11_MULTI_OBJECTIVE for multi-objective optimization

---

Implementation Files

• optimization_engine/intelligent_optimizer.py - Main orchestrator
• optimization_engine/adaptive_characterization.py - Stopping criterion
• optimization_engine/landscape_analyzer.py - Landscape metrics
• optimization_engine/strategy_selector.py - Algorithm recommendation

---

Version History

Version	Date	Changes
2.1	2025-11-20	Fixed strategy selector timing, multimodality detection, added simulation validation
2.0	2025-11-20	Added adaptive characterization, two-study architecture
1.0	2025-11-19	Initial implementation

Version 2.1 Bug Fixes Detail

Fix #1: Strategy Selector - Use Characterization Trial Count

Problem: Strategy selector used total trial count (including pruned) instead of characterization trial count, causing wrong algorithm selection after characterization.

Solution (strategy_selector.py): Use char_trials = landscape.get('total_trials', trials_completed) for decisions.

Fix #2: Improved Multimodality Detection

Problem: False multimodality detected on smooth continuous surfaces (2 modes detected when problem was unimodal).

Solution (landscape_analyzer.py): Added heuristic - if only 2 modes with smoothness > 0.6 and noise < 0.2, reclassify as unimodal (smooth continuous manifold).

Fix #3: Simulation Validation

Problem: 20% pruning rate due to extreme parameters causing mesh/solver failures.

Solution: Created simulation_validator.py with:

• Hard limits (reject invalid parameters)
• Soft limits (warn about risky parameters)
• Aspect ratio checks
• Model-specific validation rules

Impact: Reduced pruning rate from 20% to ~5%.