feat: Add MLP surrogate with Turbo Mode for 100x faster optimization
Neural Acceleration (MLP Surrogate): - Add run_nn_optimization.py with hybrid FEA/NN workflow - MLP architecture: 4-layer (64->128->128->64) with BatchNorm/Dropout - Three workflow modes: - --all: Sequential export->train->optimize->validate - --hybrid-loop: Iterative Train->NN->Validate->Retrain cycle - --turbo: Aggressive single-best validation (RECOMMENDED) - Turbo mode: 5000 NN trials + 50 FEA validations in ~12 minutes - Separate nn_study.db to avoid overloading dashboard Performance Results (bracket_pareto_3obj study): - NN prediction errors: mass 1-5%, stress 1-4%, stiffness 5-15% - Found minimum mass designs at boundary (angle~30deg, thick~30mm) - 100x speedup vs pure FEA exploration Protocol Operating System: - Add .claude/skills/ with Bootstrap, Cheatsheet, Context Loader - Add docs/protocols/ with operations (OP_01-06) and system (SYS_10-14) - Update SYS_14_NEURAL_ACCELERATION.md with MLP Turbo Mode docs NX Automation: - Add optimization_engine/hooks/ for NX CAD/CAE automation - Add study_wizard.py for guided study creation - Fix FEM mesh update: load idealized part before UpdateFemodel() New Study: - bracket_pareto_3obj: 3-objective Pareto (mass, stress, stiffness) - 167 FEA trials + 5000 NN trials completed - Demonstrates full hybrid workflow 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
Antoine
403
docs/protocols/operations/OP_01_CREATE_STUDY.md
Normal file
403
docs/protocols/operations/OP_01_CREATE_STUDY.md
Normal file
@@ -0,0 +1,403 @@
|
||||
# OP_01: Create Optimization Study
|
||||
|
||||
<!--
|
||||
PROTOCOL: Create Optimization Study
|
||||
LAYER: Operations
|
||||
VERSION: 1.0
|
||||
STATUS: Active
|
||||
LAST_UPDATED: 2025-12-05
|
||||
PRIVILEGE: user
|
||||
LOAD_WITH: [core/study-creation-core.md]
|
||||
-->
|
||||
|
||||
## Overview
|
||||
|
||||
This protocol guides you through creating a complete Atomizer optimization study from scratch. It covers gathering requirements, generating configuration files, and validating setup.
|
||||
|
||||
**Skill to Load**: `.claude/skills/core/study-creation-core.md`
|
||||
|
||||
---
|
||||
|
||||
## When to Use
|
||||
|
||||
| Trigger | Action |
|
||||
|---------|--------|
|
||||
| "new study", "create study" | Follow this protocol |
|
||||
| "set up optimization" | Follow this protocol |
|
||||
| "optimize my design" | Follow this protocol |
|
||||
| User provides NX model | Assess and follow this protocol |
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference
|
||||
|
||||
**Required Outputs**:
|
||||
| File | Purpose | Location |
|
||||
|------|---------|----------|
|
||||
| `optimization_config.json` | Design vars, objectives, constraints | `1_setup/` |
|
||||
| `run_optimization.py` | Execution script | Study root |
|
||||
| `README.md` | Engineering documentation | Study root |
|
||||
| `STUDY_REPORT.md` | Results template | Study root |
|
||||
|
||||
**Study Structure**:
|
||||
```
|
||||
studies/{study_name}/
|
||||
├── 1_setup/
|
||||
│ ├── model/ # NX files (.prt, .sim, .fem)
|
||||
│ └── optimization_config.json
|
||||
├── 2_results/ # Created during run
|
||||
├── README.md # MANDATORY
|
||||
├── STUDY_REPORT.md # MANDATORY
|
||||
└── run_optimization.py
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Detailed Steps
|
||||
|
||||
### Step 1: Gather Requirements
|
||||
|
||||
**Ask the user**:
|
||||
1. What are you trying to optimize? (objective)
|
||||
2. What can you change? (design variables)
|
||||
3. What limits must be respected? (constraints)
|
||||
4. Where are your NX files?
|
||||
|
||||
**Example Dialog**:
|
||||
```
|
||||
User: "I want to optimize my bracket"
|
||||
You: "What should I optimize for - minimum mass, maximum stiffness,
|
||||
target frequency, or something else?"
|
||||
User: "Minimize mass while keeping stress below 250 MPa"
|
||||
```
|
||||
|
||||
### Step 2: Analyze Model (Introspection)
|
||||
|
||||
**MANDATORY**: When user provides NX files, run comprehensive introspection:
|
||||
|
||||
```python
|
||||
from optimization_engine.hooks.nx_cad.model_introspection import (
|
||||
introspect_part,
|
||||
introspect_simulation,
|
||||
introspect_op2,
|
||||
introspect_study
|
||||
)
|
||||
|
||||
# Introspect the part file to get expressions, mass, features
|
||||
part_info = introspect_part("C:/path/to/model.prt")
|
||||
|
||||
# Introspect the simulation to get solutions, BCs, loads
|
||||
sim_info = introspect_simulation("C:/path/to/model.sim")
|
||||
|
||||
# If OP2 exists, check what results are available
|
||||
op2_info = introspect_op2("C:/path/to/results.op2")
|
||||
|
||||
# Or introspect entire study directory at once
|
||||
study_info = introspect_study("studies/my_study/")
|
||||
```
|
||||
|
||||
**Introspection Report Contents**:
|
||||
|
||||
| Source | Information Extracted |
|
||||
|--------|----------------------|
|
||||
| `.prt` | Expressions (count, values, types), bodies, mass, material, features |
|
||||
| `.sim` | Solutions, boundary conditions, loads, materials, mesh info, output requests |
|
||||
| `.op2` | Available results (displacement, stress, strain, SPC forces, etc.), subcases |
|
||||
|
||||
**Generate Introspection Report** at study creation:
|
||||
1. Save report to `studies/{study_name}/MODEL_INTROSPECTION.md`
|
||||
2. Include summary of what's available for optimization
|
||||
3. List potential design variables (expressions)
|
||||
4. List extractable results (from OP2)
|
||||
|
||||
**Key Questions Answered by Introspection**:
|
||||
- What expressions exist? (potential design variables)
|
||||
- What solution types? (static, modal, etc.)
|
||||
- What results are available in OP2? (displacement, stress, SPC forces)
|
||||
- Multi-solution required? (static + modal = set `solution_name=None`)
|
||||
|
||||
### Step 3: Select Protocol
|
||||
|
||||
Based on objectives:
|
||||
|
||||
| Scenario | Protocol | Sampler |
|
||||
|----------|----------|---------|
|
||||
| Single objective | Protocol 10 (IMSO) | TPE, CMA-ES, or GP |
|
||||
| 2-3 objectives | Protocol 11 | NSGA-II |
|
||||
| >50 trials, need speed | Protocol 14 | + Neural acceleration |
|
||||
|
||||
See [SYS_10_IMSO](../system/SYS_10_IMSO.md), [SYS_11_MULTI_OBJECTIVE](../system/SYS_11_MULTI_OBJECTIVE.md).
|
||||
|
||||
### Step 4: Select Extractors
|
||||
|
||||
Match physics to extractors from [SYS_12_EXTRACTOR_LIBRARY](../system/SYS_12_EXTRACTOR_LIBRARY.md):
|
||||
|
||||
| Need | Extractor ID | Function |
|
||||
|------|--------------|----------|
|
||||
| Max displacement | E1 | `extract_displacement()` |
|
||||
| Natural frequency | E2 | `extract_frequency()` |
|
||||
| Von Mises stress | E3 | `extract_solid_stress()` |
|
||||
| Mass from BDF | E4 | `extract_mass_from_bdf()` |
|
||||
| Mass from NX | E5 | `extract_mass_from_expression()` |
|
||||
| Wavefront error | E8-E10 | Zernike extractors |
|
||||
|
||||
### Step 5: Generate Configuration
|
||||
|
||||
Create `optimization_config.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"study_name": "bracket_optimization",
|
||||
"description": "Minimize bracket mass while meeting stress constraint",
|
||||
|
||||
"design_variables": [
|
||||
{
|
||||
"name": "thickness",
|
||||
"type": "continuous",
|
||||
"min": 2.0,
|
||||
"max": 10.0,
|
||||
"unit": "mm",
|
||||
"description": "Wall thickness"
|
||||
}
|
||||
],
|
||||
|
||||
"objectives": [
|
||||
{
|
||||
"name": "mass",
|
||||
"type": "minimize",
|
||||
"unit": "kg",
|
||||
"description": "Total bracket mass"
|
||||
}
|
||||
],
|
||||
|
||||
"constraints": [
|
||||
{
|
||||
"name": "max_stress",
|
||||
"type": "less_than",
|
||||
"value": 250.0,
|
||||
"unit": "MPa",
|
||||
"description": "Maximum allowable von Mises stress"
|
||||
}
|
||||
],
|
||||
|
||||
"simulation": {
|
||||
"model_file": "1_setup/model/bracket.prt",
|
||||
"sim_file": "1_setup/model/bracket.sim",
|
||||
"solver": "nastran",
|
||||
"solution_name": null
|
||||
},
|
||||
|
||||
"optimization_settings": {
|
||||
"protocol": "protocol_10_single_objective",
|
||||
"sampler": "TPESampler",
|
||||
"n_trials": 50
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Step 6: Generate run_optimization.py
|
||||
|
||||
```python
|
||||
#!/usr/bin/env python
|
||||
"""
|
||||
{study_name} - Optimization Runner
|
||||
Generated by Atomizer LLM
|
||||
"""
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
# Add optimization engine to path
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent.parent))
|
||||
|
||||
from optimization_engine.nx_solver import NXSolver
|
||||
from optimization_engine.extractors import extract_displacement, extract_solid_stress
|
||||
|
||||
# Paths
|
||||
STUDY_DIR = Path(__file__).parent
|
||||
MODEL_DIR = STUDY_DIR / "1_setup" / "model"
|
||||
RESULTS_DIR = STUDY_DIR / "2_results"
|
||||
|
||||
def objective(trial):
|
||||
"""Optimization objective function."""
|
||||
# Sample design variables
|
||||
thickness = trial.suggest_float("thickness", 2.0, 10.0)
|
||||
|
||||
# Update NX model and solve
|
||||
nx_solver = NXSolver(...)
|
||||
result = nx_solver.run_simulation(
|
||||
sim_file=MODEL_DIR / "bracket.sim",
|
||||
working_dir=MODEL_DIR,
|
||||
expression_updates={"thickness": thickness}
|
||||
)
|
||||
|
||||
if not result['success']:
|
||||
raise optuna.TrialPruned("Simulation failed")
|
||||
|
||||
# Extract results using library extractors
|
||||
op2_file = result['op2_file']
|
||||
stress_result = extract_solid_stress(op2_file)
|
||||
max_stress = stress_result['max_von_mises']
|
||||
|
||||
# Check constraint
|
||||
if max_stress > 250.0:
|
||||
raise optuna.TrialPruned(f"Stress constraint violated: {max_stress} MPa")
|
||||
|
||||
# Return objective
|
||||
mass = extract_mass(...)
|
||||
return mass
|
||||
|
||||
if __name__ == "__main__":
|
||||
# Run optimization
|
||||
import optuna
|
||||
study = optuna.create_study(direction="minimize")
|
||||
study.optimize(objective, n_trials=50)
|
||||
```
|
||||
|
||||
### Step 7: Generate Documentation
|
||||
|
||||
**README.md** (11 sections required):
|
||||
1. Engineering Problem
|
||||
2. Mathematical Formulation
|
||||
3. Optimization Algorithm
|
||||
4. Simulation Pipeline
|
||||
5. Result Extraction Methods
|
||||
6. Neural Acceleration (if applicable)
|
||||
7. Study File Structure
|
||||
8. Results Location
|
||||
9. Quick Start
|
||||
10. Configuration Reference
|
||||
11. References
|
||||
|
||||
**STUDY_REPORT.md** (template):
|
||||
```markdown
|
||||
# Study Report: {study_name}
|
||||
|
||||
## Executive Summary
|
||||
- Trials completed: _pending_
|
||||
- Best objective: _pending_
|
||||
- Constraint satisfaction: _pending_
|
||||
|
||||
## Optimization Progress
|
||||
_To be filled after run_
|
||||
|
||||
## Best Designs Found
|
||||
_To be filled after run_
|
||||
|
||||
## Recommendations
|
||||
_To be filled after analysis_
|
||||
```
|
||||
|
||||
### Step 8: Validate NX Model File Chain
|
||||
|
||||
**CRITICAL**: NX simulation files have parent-child dependencies. ALL linked files must be copied to the study folder.
|
||||
|
||||
**Required File Chain Check**:
|
||||
```
|
||||
.sim (Simulation)
|
||||
└── .fem (FEM)
|
||||
└── _i.prt (Idealized Part) ← OFTEN MISSING!
|
||||
└── .prt (Geometry Part)
|
||||
```
|
||||
|
||||
**Validation Steps**:
|
||||
1. Open the `.sim` file in NX
|
||||
2. Go to **Assemblies → Assembly Navigator** or check **Part Navigator**
|
||||
3. Identify ALL child components (especially `*_i.prt` idealized parts)
|
||||
4. Copy ALL linked files to `1_setup/model/`
|
||||
|
||||
**Common Issue**: The `_i.prt` (idealized part) is often forgotten. Without it:
|
||||
- `UpdateFemodel()` runs but mesh doesn't change
|
||||
- Geometry changes don't propagate to FEM
|
||||
- All optimization trials produce identical results
|
||||
|
||||
**File Checklist**:
|
||||
| File Pattern | Description | Required |
|
||||
|--------------|-------------|----------|
|
||||
| `*.prt` | Geometry part | ✅ Always |
|
||||
| `*_i.prt` | Idealized part | ✅ If FEM uses idealization |
|
||||
| `*.fem` | FEM file | ✅ Always |
|
||||
| `*.sim` | Simulation file | ✅ Always |
|
||||
|
||||
**Introspection should report**:
|
||||
- List of all parts referenced by .sim
|
||||
- Warning if any referenced parts are missing from study folder
|
||||
|
||||
### Step 9: Final Validation Checklist
|
||||
|
||||
Before running:
|
||||
|
||||
- [ ] NX files exist in `1_setup/model/`
|
||||
- [ ] **ALL child parts copied** (especially `*_i.prt`)
|
||||
- [ ] Expression names match model
|
||||
- [ ] Config validates (JSON schema)
|
||||
- [ ] `run_optimization.py` has no syntax errors
|
||||
- [ ] README.md has all 11 sections
|
||||
- [ ] STUDY_REPORT.md template exists
|
||||
|
||||
---
|
||||
|
||||
## Examples
|
||||
|
||||
### Example 1: Simple Bracket
|
||||
|
||||
```
|
||||
User: "Optimize my bracket.prt for minimum mass, stress < 250 MPa"
|
||||
|
||||
Generated config:
|
||||
- 1 design variable (thickness)
|
||||
- 1 objective (minimize mass)
|
||||
- 1 constraint (stress < 250)
|
||||
- Protocol 10, TPE sampler
|
||||
- 50 trials
|
||||
```
|
||||
|
||||
### Example 2: Multi-Objective Beam
|
||||
|
||||
```
|
||||
User: "Minimize mass AND maximize stiffness for my beam"
|
||||
|
||||
Generated config:
|
||||
- 2 design variables (width, height)
|
||||
- 2 objectives (minimize mass, maximize stiffness)
|
||||
- Protocol 11, NSGA-II sampler
|
||||
- 50 trials (Pareto front)
|
||||
```
|
||||
|
||||
### Example 3: Telescope Mirror
|
||||
|
||||
```
|
||||
User: "Minimize wavefront error at 40deg vs 20deg reference"
|
||||
|
||||
Generated config:
|
||||
- Multiple design variables (mount positions)
|
||||
- 1 objective (minimize relative WFE)
|
||||
- Zernike extractor E9
|
||||
- Protocol 10
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Cause | Solution |
|
||||
|---------|-------|----------|
|
||||
| "Expression not found" | Name mismatch | Verify expression names in NX |
|
||||
| "No feasible designs" | Constraints too tight | Relax constraint values |
|
||||
| Config validation fails | Missing required field | Check JSON schema |
|
||||
| Import error | Wrong path | Check sys.path setup |
|
||||
|
||||
---
|
||||
|
||||
## Cross-References
|
||||
|
||||
- **Depends On**: [SYS_12_EXTRACTOR_LIBRARY](../system/SYS_12_EXTRACTOR_LIBRARY.md)
|
||||
- **Next Step**: [OP_02_RUN_OPTIMIZATION](./OP_02_RUN_OPTIMIZATION.md)
|
||||
- **Skill**: `.claude/skills/core/study-creation-core.md`
|
||||
|
||||
---
|
||||
|
||||
## Version History
|
||||
|
||||
| Version | Date | Changes |
|
||||
|---------|------|---------|
|
||||
| 1.0 | 2025-12-05 | Initial release |
|
||||
297
docs/protocols/operations/OP_02_RUN_OPTIMIZATION.md
Normal file
297
docs/protocols/operations/OP_02_RUN_OPTIMIZATION.md
Normal file
@@ -0,0 +1,297 @@
|
||||
# OP_02: Run Optimization
|
||||
|
||||
<!--
|
||||
PROTOCOL: Run Optimization
|
||||
LAYER: Operations
|
||||
VERSION: 1.0
|
||||
STATUS: Active
|
||||
LAST_UPDATED: 2025-12-05
|
||||
PRIVILEGE: user
|
||||
LOAD_WITH: []
|
||||
-->
|
||||
|
||||
## Overview
|
||||
|
||||
This protocol covers executing optimization runs, including pre-flight validation, execution modes, monitoring, and handling common issues.
|
||||
|
||||
---
|
||||
|
||||
## When to Use
|
||||
|
||||
| Trigger | Action |
|
||||
|---------|--------|
|
||||
| "start", "run", "execute" | Follow this protocol |
|
||||
| "begin optimization" | Follow this protocol |
|
||||
| Study setup complete | Execute this protocol |
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference
|
||||
|
||||
**Start Command**:
|
||||
```bash
|
||||
conda activate atomizer
|
||||
cd studies/{study_name}
|
||||
python run_optimization.py
|
||||
```
|
||||
|
||||
**Common Options**:
|
||||
| Flag | Purpose |
|
||||
|------|---------|
|
||||
| `--n-trials 100` | Override trial count |
|
||||
| `--resume` | Continue interrupted run |
|
||||
| `--test` | Run single trial for validation |
|
||||
| `--export-training` | Export data for neural training |
|
||||
|
||||
---
|
||||
|
||||
## Pre-Flight Checklist
|
||||
|
||||
Before running, verify:
|
||||
|
||||
- [ ] **Environment**: `conda activate atomizer`
|
||||
- [ ] **Config exists**: `1_setup/optimization_config.json`
|
||||
- [ ] **Script exists**: `run_optimization.py`
|
||||
- [ ] **Model files**: NX files in `1_setup/model/`
|
||||
- [ ] **No conflicts**: No other optimization running on same study
|
||||
- [ ] **Disk space**: Sufficient for results
|
||||
|
||||
**Quick Validation**:
|
||||
```bash
|
||||
python run_optimization.py --test
|
||||
```
|
||||
This runs a single trial to verify setup.
|
||||
|
||||
---
|
||||
|
||||
## Execution Modes
|
||||
|
||||
### 1. Standard Run
|
||||
|
||||
```bash
|
||||
python run_optimization.py
|
||||
```
|
||||
Uses settings from `optimization_config.json`.
|
||||
|
||||
### 2. Override Trials
|
||||
|
||||
```bash
|
||||
python run_optimization.py --n-trials 100
|
||||
```
|
||||
Override trial count from config.
|
||||
|
||||
### 3. Resume Interrupted
|
||||
|
||||
```bash
|
||||
python run_optimization.py --resume
|
||||
```
|
||||
Continues from last completed trial.
|
||||
|
||||
### 4. Neural Acceleration
|
||||
|
||||
```bash
|
||||
python run_optimization.py --neural
|
||||
```
|
||||
Requires trained surrogate model.
|
||||
|
||||
### 5. Export Training Data
|
||||
|
||||
```bash
|
||||
python run_optimization.py --export-training
|
||||
```
|
||||
Saves BDF/OP2 for neural network training.
|
||||
|
||||
---
|
||||
|
||||
## Monitoring Progress
|
||||
|
||||
### Option 1: Console Output
|
||||
The script prints progress:
|
||||
```
|
||||
Trial 15/50 complete. Best: 0.234 kg
|
||||
Trial 16/50 complete. Best: 0.234 kg
|
||||
```
|
||||
|
||||
### Option 2: Dashboard
|
||||
See [SYS_13_DASHBOARD_TRACKING](../system/SYS_13_DASHBOARD_TRACKING.md).
|
||||
|
||||
```bash
|
||||
# Start dashboard (separate terminal)
|
||||
cd atomizer-dashboard/backend && python -m uvicorn api.main:app --port 8000
|
||||
cd atomizer-dashboard/frontend && npm run dev
|
||||
|
||||
# Open browser
|
||||
http://localhost:3000
|
||||
```
|
||||
|
||||
### Option 3: Query Database
|
||||
|
||||
```bash
|
||||
python -c "
|
||||
import optuna
|
||||
study = optuna.load_study('study_name', 'sqlite:///2_results/study.db')
|
||||
print(f'Trials: {len(study.trials)}')
|
||||
print(f'Best value: {study.best_value}')
|
||||
"
|
||||
```
|
||||
|
||||
### Option 4: Optuna Dashboard
|
||||
|
||||
```bash
|
||||
optuna-dashboard sqlite:///2_results/study.db
|
||||
# Open http://localhost:8080
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## During Execution
|
||||
|
||||
### What Happens Per Trial
|
||||
|
||||
1. **Sample parameters**: Optuna suggests design variable values
|
||||
2. **Update model**: NX expressions updated via journal
|
||||
3. **Solve**: NX Nastran runs FEA simulation
|
||||
4. **Extract results**: Extractors read OP2 file
|
||||
5. **Evaluate**: Check constraints, compute objectives
|
||||
6. **Record**: Trial stored in Optuna database
|
||||
|
||||
### Normal Output
|
||||
|
||||
```
|
||||
[2025-12-05 10:15:30] Trial 1 started
|
||||
[2025-12-05 10:17:45] NX solve complete (135.2s)
|
||||
[2025-12-05 10:17:46] Extraction complete
|
||||
[2025-12-05 10:17:46] Trial 1 complete: mass=0.342 kg, stress=198.5 MPa
|
||||
|
||||
[2025-12-05 10:17:47] Trial 2 started
|
||||
...
|
||||
```
|
||||
|
||||
### Expected Timing
|
||||
|
||||
| Operation | Typical Time |
|
||||
|-----------|--------------|
|
||||
| NX solve | 30s - 30min |
|
||||
| Extraction | <1s |
|
||||
| Per trial total | 1-30 min |
|
||||
| 50 trials | 1-24 hours |
|
||||
|
||||
---
|
||||
|
||||
## Handling Issues
|
||||
|
||||
### Trial Failed / Pruned
|
||||
|
||||
```
|
||||
[WARNING] Trial 12 pruned: Stress constraint violated (312.5 MPa > 250 MPa)
|
||||
```
|
||||
**Normal behavior** - optimizer learns from failures.
|
||||
|
||||
### NX Session Timeout
|
||||
|
||||
```
|
||||
[ERROR] NX session timeout after 600s
|
||||
```
|
||||
**Solution**: Increase timeout in config or simplify model.
|
||||
|
||||
### Expression Not Found
|
||||
|
||||
```
|
||||
[ERROR] Expression 'thicknes' not found in model
|
||||
```
|
||||
**Solution**: Check spelling, verify expression exists in NX.
|
||||
|
||||
### OP2 File Missing
|
||||
|
||||
```
|
||||
[ERROR] OP2 file not found: model.op2
|
||||
```
|
||||
**Solution**: Check NX solve completed. Review NX log file.
|
||||
|
||||
### Database Locked
|
||||
|
||||
```
|
||||
[ERROR] Database is locked
|
||||
```
|
||||
**Solution**: Another process using database. Wait or kill stale process.
|
||||
|
||||
---
|
||||
|
||||
## Stopping and Resuming
|
||||
|
||||
### Graceful Stop
|
||||
Press `Ctrl+C` once. Current trial completes, then exits.
|
||||
|
||||
### Force Stop
|
||||
Press `Ctrl+C` twice. Immediate exit (may lose current trial).
|
||||
|
||||
### Resume
|
||||
```bash
|
||||
python run_optimization.py --resume
|
||||
```
|
||||
Continues from last completed trial. Same study database used.
|
||||
|
||||
---
|
||||
|
||||
## Post-Run Actions
|
||||
|
||||
After optimization completes:
|
||||
|
||||
1. **Check results**:
|
||||
```bash
|
||||
python -c "import optuna; s=optuna.load_study(...); print(s.best_params)"
|
||||
```
|
||||
|
||||
2. **View in dashboard**: `http://localhost:3000`
|
||||
|
||||
3. **Generate report**: See [OP_04_ANALYZE_RESULTS](./OP_04_ANALYZE_RESULTS.md)
|
||||
|
||||
4. **Update STUDY_REPORT.md**: Fill in results template
|
||||
|
||||
---
|
||||
|
||||
## Protocol Integration
|
||||
|
||||
### With Protocol 10 (IMSO)
|
||||
If enabled, optimization runs in two phases:
|
||||
1. Characterization (10-30 trials)
|
||||
2. Optimization (remaining trials)
|
||||
|
||||
Dashboard shows phase transitions.
|
||||
|
||||
### With Protocol 11 (Multi-Objective)
|
||||
If 2+ objectives, uses NSGA-II. Returns Pareto front, not single best.
|
||||
|
||||
### With Protocol 13 (Dashboard)
|
||||
Writes `optimizer_state.json` every trial for real-time updates.
|
||||
|
||||
### With Protocol 14 (Neural)
|
||||
If `--neural` flag, uses trained surrogate for fast evaluation.
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Cause | Solution |
|
||||
|---------|-------|----------|
|
||||
| "ModuleNotFoundError" | Wrong environment | `conda activate atomizer` |
|
||||
| All trials pruned | Constraints too tight | Relax constraints |
|
||||
| Very slow | Model too complex | Simplify mesh, increase timeout |
|
||||
| No improvement | Wrong sampler | Try different algorithm |
|
||||
| "NX license error" | License unavailable | Check NX license server |
|
||||
|
||||
---
|
||||
|
||||
## Cross-References
|
||||
|
||||
- **Preceded By**: [OP_01_CREATE_STUDY](./OP_01_CREATE_STUDY.md)
|
||||
- **Followed By**: [OP_03_MONITOR_PROGRESS](./OP_03_MONITOR_PROGRESS.md), [OP_04_ANALYZE_RESULTS](./OP_04_ANALYZE_RESULTS.md)
|
||||
- **Integrates With**: [SYS_10_IMSO](../system/SYS_10_IMSO.md), [SYS_13_DASHBOARD_TRACKING](../system/SYS_13_DASHBOARD_TRACKING.md)
|
||||
|
||||
---
|
||||
|
||||
## Version History
|
||||
|
||||
| Version | Date | Changes |
|
||||
|---------|------|---------|
|
||||
| 1.0 | 2025-12-05 | Initial release |
|
||||
246
docs/protocols/operations/OP_03_MONITOR_PROGRESS.md
Normal file
246
docs/protocols/operations/OP_03_MONITOR_PROGRESS.md
Normal file
@@ -0,0 +1,246 @@
|
||||
# OP_03: Monitor Progress
|
||||
|
||||
<!--
|
||||
PROTOCOL: Monitor Optimization Progress
|
||||
LAYER: Operations
|
||||
VERSION: 1.0
|
||||
STATUS: Active
|
||||
LAST_UPDATED: 2025-12-05
|
||||
PRIVILEGE: user
|
||||
LOAD_WITH: [SYS_13_DASHBOARD_TRACKING]
|
||||
-->
|
||||
|
||||
## Overview
|
||||
|
||||
This protocol covers monitoring optimization progress through console output, dashboard, database queries, and Optuna's built-in tools.
|
||||
|
||||
---
|
||||
|
||||
## When to Use
|
||||
|
||||
| Trigger | Action |
|
||||
|---------|--------|
|
||||
| "status", "progress" | Follow this protocol |
|
||||
| "how many trials" | Query database |
|
||||
| "what's happening" | Check console or dashboard |
|
||||
| "is it running" | Check process status |
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Method | Command/URL | Best For |
|
||||
|--------|-------------|----------|
|
||||
| Console | Watch terminal output | Quick check |
|
||||
| Dashboard | `http://localhost:3000` | Visual monitoring |
|
||||
| Database query | Python one-liner | Scripted checks |
|
||||
| Optuna Dashboard | `http://localhost:8080` | Detailed analysis |
|
||||
|
||||
---
|
||||
|
||||
## Monitoring Methods
|
||||
|
||||
### 1. Console Output
|
||||
|
||||
If running in foreground, watch terminal:
|
||||
```
|
||||
[10:15:30] Trial 15/50 started
|
||||
[10:17:45] Trial 15/50 complete: mass=0.234 kg (best: 0.212 kg)
|
||||
[10:17:46] Trial 16/50 started
|
||||
```
|
||||
|
||||
### 2. Atomizer Dashboard
|
||||
|
||||
**Start Dashboard** (if not running):
|
||||
```bash
|
||||
# Terminal 1: Backend
|
||||
cd atomizer-dashboard/backend
|
||||
python -m uvicorn api.main:app --reload --port 8000
|
||||
|
||||
# Terminal 2: Frontend
|
||||
cd atomizer-dashboard/frontend
|
||||
npm run dev
|
||||
```
|
||||
|
||||
**View at**: `http://localhost:3000`
|
||||
|
||||
**Features**:
|
||||
- Real-time trial progress bar
|
||||
- Current optimizer phase (if Protocol 10)
|
||||
- Pareto front visualization (if multi-objective)
|
||||
- Parallel coordinates plot
|
||||
- Convergence chart
|
||||
|
||||
### 3. Database Query
|
||||
|
||||
**Quick status**:
|
||||
```bash
|
||||
python -c "
|
||||
import optuna
|
||||
study = optuna.load_study(
|
||||
study_name='my_study',
|
||||
storage='sqlite:///studies/my_study/2_results/study.db'
|
||||
)
|
||||
print(f'Trials completed: {len(study.trials)}')
|
||||
print(f'Best value: {study.best_value}')
|
||||
print(f'Best params: {study.best_params}')
|
||||
"
|
||||
```
|
||||
|
||||
**Detailed status**:
|
||||
```python
|
||||
import optuna
|
||||
|
||||
study = optuna.load_study(
|
||||
study_name='my_study',
|
||||
storage='sqlite:///studies/my_study/2_results/study.db'
|
||||
)
|
||||
|
||||
# Trial counts by state
|
||||
from collections import Counter
|
||||
states = Counter(t.state.name for t in study.trials)
|
||||
print(f"Complete: {states.get('COMPLETE', 0)}")
|
||||
print(f"Pruned: {states.get('PRUNED', 0)}")
|
||||
print(f"Failed: {states.get('FAIL', 0)}")
|
||||
print(f"Running: {states.get('RUNNING', 0)}")
|
||||
|
||||
# Best trials
|
||||
if len(study.directions) > 1:
|
||||
print(f"Pareto front size: {len(study.best_trials)}")
|
||||
else:
|
||||
print(f"Best value: {study.best_value}")
|
||||
```
|
||||
|
||||
### 4. Optuna Dashboard
|
||||
|
||||
```bash
|
||||
optuna-dashboard sqlite:///studies/my_study/2_results/study.db
|
||||
# Open http://localhost:8080
|
||||
```
|
||||
|
||||
**Features**:
|
||||
- Trial history table
|
||||
- Parameter importance
|
||||
- Optimization history plot
|
||||
- Slice plot (parameter vs objective)
|
||||
|
||||
### 5. Check Running Processes
|
||||
|
||||
```bash
|
||||
# Linux/Mac
|
||||
ps aux | grep run_optimization
|
||||
|
||||
# Windows
|
||||
tasklist | findstr python
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Key Metrics to Monitor
|
||||
|
||||
### Trial Progress
|
||||
- Completed trials vs target
|
||||
- Completion rate (trials/hour)
|
||||
- Estimated time remaining
|
||||
|
||||
### Objective Improvement
|
||||
- Current best value
|
||||
- Improvement trend
|
||||
- Plateau detection
|
||||
|
||||
### Constraint Satisfaction
|
||||
- Feasibility rate (% passing constraints)
|
||||
- Most violated constraint
|
||||
|
||||
### For Protocol 10 (IMSO)
|
||||
- Current phase (Characterization vs Optimization)
|
||||
- Current strategy (TPE, GP, CMA-ES)
|
||||
- Characterization confidence
|
||||
|
||||
### For Protocol 11 (Multi-Objective)
|
||||
- Pareto front size
|
||||
- Hypervolume indicator
|
||||
- Spread of solutions
|
||||
|
||||
---
|
||||
|
||||
## Interpreting Results
|
||||
|
||||
### Healthy Optimization
|
||||
```
|
||||
Trial 45/50: mass=0.198 kg (best: 0.195 kg)
|
||||
Feasibility rate: 78%
|
||||
```
|
||||
- Progress toward target
|
||||
- Reasonable feasibility rate (60-90%)
|
||||
- Gradual improvement
|
||||
|
||||
### Potential Issues
|
||||
|
||||
**All Trials Pruned**:
|
||||
```
|
||||
Trial 20 pruned: constraint violated
|
||||
Trial 21 pruned: constraint violated
|
||||
...
|
||||
```
|
||||
→ Constraints too tight. Consider relaxing.
|
||||
|
||||
**No Improvement**:
|
||||
```
|
||||
Trial 30: best=0.234 (unchanged since trial 8)
|
||||
Trial 31: best=0.234 (unchanged since trial 8)
|
||||
```
|
||||
→ May have converged, or stuck in local minimum.
|
||||
|
||||
**High Failure Rate**:
|
||||
```
|
||||
Failed: 15/50 (30%)
|
||||
```
|
||||
→ Model issues. Check NX logs.
|
||||
|
||||
---
|
||||
|
||||
## Real-Time State File
|
||||
|
||||
If using Protocol 10, check:
|
||||
```bash
|
||||
cat studies/my_study/2_results/intelligent_optimizer/optimizer_state.json
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"timestamp": "2025-12-05T10:15:30",
|
||||
"trial_number": 29,
|
||||
"total_trials": 50,
|
||||
"current_phase": "adaptive_optimization",
|
||||
"current_strategy": "GP_UCB",
|
||||
"is_multi_objective": false
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Cause | Solution |
|
||||
|---------|-------|----------|
|
||||
| Dashboard shows old data | Backend not running | Start backend |
|
||||
| "No study found" | Wrong path | Check study name and path |
|
||||
| Trial count not increasing | Process stopped | Check if still running |
|
||||
| Dashboard not updating | Polling issue | Refresh browser |
|
||||
|
||||
---
|
||||
|
||||
## Cross-References
|
||||
|
||||
- **Preceded By**: [OP_02_RUN_OPTIMIZATION](./OP_02_RUN_OPTIMIZATION.md)
|
||||
- **Followed By**: [OP_04_ANALYZE_RESULTS](./OP_04_ANALYZE_RESULTS.md)
|
||||
- **Integrates With**: [SYS_13_DASHBOARD_TRACKING](../system/SYS_13_DASHBOARD_TRACKING.md)
|
||||
|
||||
---
|
||||
|
||||
## Version History
|
||||
|
||||
| Version | Date | Changes |
|
||||
|---------|------|---------|
|
||||
| 1.0 | 2025-12-05 | Initial release |
|
||||
302
docs/protocols/operations/OP_04_ANALYZE_RESULTS.md
Normal file
302
docs/protocols/operations/OP_04_ANALYZE_RESULTS.md
Normal file
@@ -0,0 +1,302 @@
|
||||
# OP_04: Analyze Results
|
||||
|
||||
<!--
|
||||
PROTOCOL: Analyze Optimization Results
|
||||
LAYER: Operations
|
||||
VERSION: 1.0
|
||||
STATUS: Active
|
||||
LAST_UPDATED: 2025-12-05
|
||||
PRIVILEGE: user
|
||||
LOAD_WITH: []
|
||||
-->
|
||||
|
||||
## Overview
|
||||
|
||||
This protocol covers analyzing optimization results, including extracting best solutions, generating reports, comparing designs, and interpreting Pareto fronts.
|
||||
|
||||
---
|
||||
|
||||
## When to Use
|
||||
|
||||
| Trigger | Action |
|
||||
|---------|--------|
|
||||
| "results", "what did we find" | Follow this protocol |
|
||||
| "best design" | Extract best trial |
|
||||
| "compare", "trade-off" | Pareto analysis |
|
||||
| "report" | Generate summary |
|
||||
| Optimization complete | Analyze and document |
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference
|
||||
|
||||
**Key Outputs**:
|
||||
| Output | Location | Purpose |
|
||||
|--------|----------|---------|
|
||||
| Best parameters | `study.best_params` | Optimal design |
|
||||
| Pareto front | `study.best_trials` | Trade-off solutions |
|
||||
| Trial history | `study.trials` | Full exploration |
|
||||
| Intelligence report | `intelligent_optimizer/` | Algorithm insights |
|
||||
|
||||
---
|
||||
|
||||
## Analysis Methods
|
||||
|
||||
### 1. Single-Objective Results
|
||||
|
||||
```python
|
||||
import optuna
|
||||
|
||||
study = optuna.load_study(
|
||||
study_name='my_study',
|
||||
storage='sqlite:///2_results/study.db'
|
||||
)
|
||||
|
||||
# Best result
|
||||
print(f"Best value: {study.best_value}")
|
||||
print(f"Best parameters: {study.best_params}")
|
||||
print(f"Best trial: #{study.best_trial.number}")
|
||||
|
||||
# Get full best trial details
|
||||
best = study.best_trial
|
||||
print(f"User attributes: {best.user_attrs}")
|
||||
```
|
||||
|
||||
### 2. Multi-Objective Results (Pareto Front)
|
||||
|
||||
```python
|
||||
import optuna
|
||||
|
||||
study = optuna.load_study(
|
||||
study_name='my_study',
|
||||
storage='sqlite:///2_results/study.db'
|
||||
)
|
||||
|
||||
# All Pareto-optimal solutions
|
||||
pareto_trials = study.best_trials
|
||||
print(f"Pareto front size: {len(pareto_trials)}")
|
||||
|
||||
# Print all Pareto solutions
|
||||
for trial in pareto_trials:
|
||||
print(f"Trial {trial.number}: {trial.values} - {trial.params}")
|
||||
|
||||
# Find extremes
|
||||
# Assuming objectives: [stiffness (max), mass (min)]
|
||||
best_stiffness = max(pareto_trials, key=lambda t: t.values[0])
|
||||
lightest = min(pareto_trials, key=lambda t: t.values[1])
|
||||
|
||||
print(f"Best stiffness: Trial {best_stiffness.number}")
|
||||
print(f"Lightest: Trial {lightest.number}")
|
||||
```
|
||||
|
||||
### 3. Parameter Importance
|
||||
|
||||
```python
|
||||
import optuna
|
||||
|
||||
study = optuna.load_study(...)
|
||||
|
||||
# Parameter importance (which parameters matter most)
|
||||
importance = optuna.importance.get_param_importances(study)
|
||||
for param, score in importance.items():
|
||||
print(f"{param}: {score:.3f}")
|
||||
```
|
||||
|
||||
### 4. Constraint Analysis
|
||||
|
||||
```python
|
||||
# Find feasibility rate
|
||||
completed = [t for t in study.trials if t.state == optuna.trial.TrialState.COMPLETE]
|
||||
pruned = [t for t in study.trials if t.state == optuna.trial.TrialState.PRUNED]
|
||||
|
||||
feasibility_rate = len(completed) / (len(completed) + len(pruned))
|
||||
print(f"Feasibility rate: {feasibility_rate:.1%}")
|
||||
|
||||
# Analyze why trials were pruned
|
||||
for trial in pruned[:5]: # First 5 pruned
|
||||
reason = trial.user_attrs.get('pruning_reason', 'Unknown')
|
||||
print(f"Trial {trial.number}: {reason}")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Visualization
|
||||
|
||||
### Using Optuna Dashboard
|
||||
|
||||
```bash
|
||||
optuna-dashboard sqlite:///2_results/study.db
|
||||
# Open http://localhost:8080
|
||||
```
|
||||
|
||||
**Available Plots**:
|
||||
- Optimization history
|
||||
- Parameter importance
|
||||
- Slice plot (parameter vs objective)
|
||||
- Parallel coordinates
|
||||
- Contour plot (2D parameter interaction)
|
||||
|
||||
### Using Atomizer Dashboard
|
||||
|
||||
Navigate to `http://localhost:3000` and select study.
|
||||
|
||||
**Features**:
|
||||
- Pareto front plot with normalization
|
||||
- Parallel coordinates with selection
|
||||
- Real-time convergence chart
|
||||
|
||||
### Custom Visualization
|
||||
|
||||
```python
|
||||
import matplotlib.pyplot as plt
|
||||
import optuna
|
||||
|
||||
study = optuna.load_study(...)
|
||||
|
||||
# Plot optimization history
|
||||
fig = optuna.visualization.plot_optimization_history(study)
|
||||
fig.show()
|
||||
|
||||
# Plot parameter importance
|
||||
fig = optuna.visualization.plot_param_importances(study)
|
||||
fig.show()
|
||||
|
||||
# Plot Pareto front (multi-objective)
|
||||
if len(study.directions) > 1:
|
||||
fig = optuna.visualization.plot_pareto_front(study)
|
||||
fig.show()
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Generate Reports
|
||||
|
||||
### Update STUDY_REPORT.md
|
||||
|
||||
After analysis, fill in the template:
|
||||
|
||||
```markdown
|
||||
# Study Report: bracket_optimization
|
||||
|
||||
## Executive Summary
|
||||
- **Trials completed**: 50
|
||||
- **Best mass**: 0.195 kg
|
||||
- **Best parameters**: thickness=4.2mm, width=25.8mm
|
||||
- **Constraint satisfaction**: All constraints met
|
||||
|
||||
## Optimization Progress
|
||||
- Initial best: 0.342 kg (trial 1)
|
||||
- Final best: 0.195 kg (trial 38)
|
||||
- Improvement: 43%
|
||||
|
||||
## Best Designs Found
|
||||
|
||||
### Design 1 (Overall Best)
|
||||
| Parameter | Value |
|
||||
|-----------|-------|
|
||||
| thickness | 4.2 mm |
|
||||
| width | 25.8 mm |
|
||||
|
||||
| Metric | Value | Constraint |
|
||||
|--------|-------|------------|
|
||||
| Mass | 0.195 kg | - |
|
||||
| Max stress | 238.5 MPa | < 250 MPa ✓ |
|
||||
|
||||
## Engineering Recommendations
|
||||
1. Recommended design: Trial 38 parameters
|
||||
2. Safety margin: 4.6% on stress constraint
|
||||
3. Consider manufacturing tolerance analysis
|
||||
```
|
||||
|
||||
### Export to CSV
|
||||
|
||||
```python
|
||||
import pandas as pd
|
||||
|
||||
# All trials to DataFrame
|
||||
trials_data = []
|
||||
for trial in study.trials:
|
||||
if trial.state == optuna.trial.TrialState.COMPLETE:
|
||||
row = {'trial': trial.number, 'value': trial.value}
|
||||
row.update(trial.params)
|
||||
trials_data.append(row)
|
||||
|
||||
df = pd.DataFrame(trials_data)
|
||||
df.to_csv('optimization_results.csv', index=False)
|
||||
```
|
||||
|
||||
### Export Best Design for FEA Validation
|
||||
|
||||
```python
|
||||
# Get best parameters
|
||||
best_params = study.best_params
|
||||
|
||||
# Format for NX expression update
|
||||
for name, value in best_params.items():
|
||||
print(f"{name} = {value}")
|
||||
|
||||
# Or save as JSON
|
||||
import json
|
||||
with open('best_design.json', 'w') as f:
|
||||
json.dump(best_params, f, indent=2)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Intelligence Report (Protocol 10)
|
||||
|
||||
If using Protocol 10, check intelligence files:
|
||||
|
||||
```bash
|
||||
# Landscape analysis
|
||||
cat 2_results/intelligent_optimizer/intelligence_report.json
|
||||
|
||||
# Characterization progress
|
||||
cat 2_results/intelligent_optimizer/characterization_progress.json
|
||||
```
|
||||
|
||||
**Key Insights**:
|
||||
- Landscape classification (smooth/rugged, unimodal/multimodal)
|
||||
- Algorithm recommendation rationale
|
||||
- Parameter correlations
|
||||
- Confidence metrics
|
||||
|
||||
---
|
||||
|
||||
## Validation Checklist
|
||||
|
||||
Before finalizing results:
|
||||
|
||||
- [ ] Best solution satisfies all constraints
|
||||
- [ ] Results are physically reasonable
|
||||
- [ ] Parameter values within manufacturing limits
|
||||
- [ ] Consider re-running FEA on best design to confirm
|
||||
- [ ] Document any anomalies or surprises
|
||||
- [ ] Update STUDY_REPORT.md
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Cause | Solution |
|
||||
|---------|-------|----------|
|
||||
| Best value seems wrong | Constraint not enforced | Check objective function |
|
||||
| No Pareto solutions | All trials failed | Check constraints |
|
||||
| Unexpected best params | Local minimum | Try different starting points |
|
||||
| Can't load study | Wrong path | Verify database location |
|
||||
|
||||
---
|
||||
|
||||
## Cross-References
|
||||
|
||||
- **Preceded By**: [OP_02_RUN_OPTIMIZATION](./OP_02_RUN_OPTIMIZATION.md), [OP_03_MONITOR_PROGRESS](./OP_03_MONITOR_PROGRESS.md)
|
||||
- **Related**: [SYS_11_MULTI_OBJECTIVE](../system/SYS_11_MULTI_OBJECTIVE.md) for Pareto analysis
|
||||
- **Skill**: `.claude/skills/generate-report.md`
|
||||
|
||||
---
|
||||
|
||||
## Version History
|
||||
|
||||
| Version | Date | Changes |
|
||||
|---------|------|---------|
|
||||
| 1.0 | 2025-12-05 | Initial release |
|
||||
294
docs/protocols/operations/OP_05_EXPORT_TRAINING_DATA.md
Normal file
294
docs/protocols/operations/OP_05_EXPORT_TRAINING_DATA.md
Normal file
@@ -0,0 +1,294 @@
|
||||
# OP_05: Export Training Data
|
||||
|
||||
<!--
|
||||
PROTOCOL: Export Neural Network Training Data
|
||||
LAYER: Operations
|
||||
VERSION: 1.0
|
||||
STATUS: Active
|
||||
LAST_UPDATED: 2025-12-05
|
||||
PRIVILEGE: user
|
||||
LOAD_WITH: [SYS_14_NEURAL_ACCELERATION]
|
||||
-->
|
||||
|
||||
## Overview
|
||||
|
||||
This protocol covers exporting FEA simulation data for training neural network surrogates. Proper data export enables Protocol 14 (Neural Acceleration).
|
||||
|
||||
---
|
||||
|
||||
## When to Use
|
||||
|
||||
| Trigger | Action |
|
||||
|---------|--------|
|
||||
| "export training data" | Follow this protocol |
|
||||
| "neural network data" | Follow this protocol |
|
||||
| Planning >50 trials | Consider export for acceleration |
|
||||
| Want to train surrogate | Follow this protocol |
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference
|
||||
|
||||
**Export Command**:
|
||||
```bash
|
||||
python run_optimization.py --export-training
|
||||
```
|
||||
|
||||
**Output Structure**:
|
||||
```
|
||||
atomizer_field_training_data/{study_name}/
|
||||
├── trial_0001/
|
||||
│ ├── input/model.bdf
|
||||
│ ├── output/model.op2
|
||||
│ └── metadata.json
|
||||
├── trial_0002/
|
||||
│ └── ...
|
||||
└── study_summary.json
|
||||
```
|
||||
|
||||
**Recommended Data Volume**:
|
||||
| Complexity | Training Samples | Validation Samples |
|
||||
|------------|-----------------|-------------------|
|
||||
| Simple (2-3 params) | 50-100 | 20-30 |
|
||||
| Medium (4-6 params) | 100-200 | 30-50 |
|
||||
| Complex (7+ params) | 200-500 | 50-100 |
|
||||
|
||||
---
|
||||
|
||||
## Configuration
|
||||
|
||||
### Enable Export in Config
|
||||
|
||||
Add to `optimization_config.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"training_data_export": {
|
||||
"enabled": true,
|
||||
"export_dir": "atomizer_field_training_data/my_study",
|
||||
"export_bdf": true,
|
||||
"export_op2": true,
|
||||
"export_fields": ["displacement", "stress"],
|
||||
"include_failed": false
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Configuration Options
|
||||
|
||||
| Option | Type | Default | Description |
|
||||
|--------|------|---------|-------------|
|
||||
| `enabled` | bool | false | Enable export |
|
||||
| `export_dir` | string | - | Output directory |
|
||||
| `export_bdf` | bool | true | Save Nastran input |
|
||||
| `export_op2` | bool | true | Save binary results |
|
||||
| `export_fields` | list | all | Which result fields |
|
||||
| `include_failed` | bool | false | Include failed trials |
|
||||
|
||||
---
|
||||
|
||||
## Export Workflow
|
||||
|
||||
### Step 1: Run with Export Enabled
|
||||
|
||||
```bash
|
||||
conda activate atomizer
|
||||
cd studies/my_study
|
||||
python run_optimization.py --export-training
|
||||
```
|
||||
|
||||
Or run standard optimization with config export enabled.
|
||||
|
||||
### Step 2: Verify Export
|
||||
|
||||
```bash
|
||||
ls atomizer_field_training_data/my_study/
|
||||
# Should see trial_0001/, trial_0002/, etc.
|
||||
|
||||
# Check a trial
|
||||
ls atomizer_field_training_data/my_study/trial_0001/
|
||||
# input/model.bdf
|
||||
# output/model.op2
|
||||
# metadata.json
|
||||
```
|
||||
|
||||
### Step 3: Check Metadata
|
||||
|
||||
```bash
|
||||
cat atomizer_field_training_data/my_study/trial_0001/metadata.json
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"trial_number": 1,
|
||||
"design_parameters": {
|
||||
"thickness": 5.2,
|
||||
"width": 30.0
|
||||
},
|
||||
"objectives": {
|
||||
"mass": 0.234,
|
||||
"max_stress": 198.5
|
||||
},
|
||||
"constraints_satisfied": true,
|
||||
"simulation_time": 145.2
|
||||
}
|
||||
```
|
||||
|
||||
### Step 4: Check Study Summary
|
||||
|
||||
```bash
|
||||
cat atomizer_field_training_data/my_study/study_summary.json
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"study_name": "my_study",
|
||||
"total_trials": 50,
|
||||
"successful_exports": 47,
|
||||
"failed_exports": 3,
|
||||
"design_parameters": ["thickness", "width"],
|
||||
"objectives": ["mass", "max_stress"],
|
||||
"export_timestamp": "2025-12-05T15:30:00"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Data Quality Checks
|
||||
|
||||
### Verify Sample Count
|
||||
|
||||
```python
|
||||
from pathlib import Path
|
||||
import json
|
||||
|
||||
export_dir = Path("atomizer_field_training_data/my_study")
|
||||
trials = list(export_dir.glob("trial_*"))
|
||||
print(f"Exported trials: {len(trials)}")
|
||||
|
||||
# Check for missing files
|
||||
for trial_dir in trials:
|
||||
bdf = trial_dir / "input" / "model.bdf"
|
||||
op2 = trial_dir / "output" / "model.op2"
|
||||
meta = trial_dir / "metadata.json"
|
||||
|
||||
if not all([bdf.exists(), op2.exists(), meta.exists()]):
|
||||
print(f"Missing files in {trial_dir}")
|
||||
```
|
||||
|
||||
### Check Parameter Coverage
|
||||
|
||||
```python
|
||||
import json
|
||||
import numpy as np
|
||||
|
||||
# Load all metadata
|
||||
params = []
|
||||
for trial_dir in export_dir.glob("trial_*"):
|
||||
with open(trial_dir / "metadata.json") as f:
|
||||
meta = json.load(f)
|
||||
params.append(meta["design_parameters"])
|
||||
|
||||
# Check coverage
|
||||
import pandas as pd
|
||||
df = pd.DataFrame(params)
|
||||
print(df.describe())
|
||||
|
||||
# Look for gaps
|
||||
for col in df.columns:
|
||||
print(f"{col}: min={df[col].min():.2f}, max={df[col].max():.2f}")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Space-Filling Sampling
|
||||
|
||||
For best neural network training, use space-filling designs:
|
||||
|
||||
### Latin Hypercube Sampling
|
||||
|
||||
```python
|
||||
from scipy.stats import qmc
|
||||
|
||||
# Generate space-filling samples
|
||||
n_samples = 100
|
||||
n_params = 4
|
||||
|
||||
sampler = qmc.LatinHypercube(d=n_params)
|
||||
samples = sampler.random(n=n_samples)
|
||||
|
||||
# Scale to parameter bounds
|
||||
lower = [2.0, 20.0, 5.0, 1.0]
|
||||
upper = [10.0, 50.0, 15.0, 5.0]
|
||||
scaled = qmc.scale(samples, lower, upper)
|
||||
```
|
||||
|
||||
### Sobol Sequence
|
||||
|
||||
```python
|
||||
sampler = qmc.Sobol(d=n_params)
|
||||
samples = sampler.random(n=n_samples)
|
||||
scaled = qmc.scale(samples, lower, upper)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Next Steps After Export
|
||||
|
||||
### 1. Parse to Neural Format
|
||||
|
||||
```bash
|
||||
cd atomizer-field
|
||||
python batch_parser.py ../atomizer_field_training_data/my_study
|
||||
```
|
||||
|
||||
### 2. Split Train/Validation
|
||||
|
||||
```python
|
||||
from sklearn.model_selection import train_test_split
|
||||
|
||||
# 80/20 split
|
||||
train_trials, val_trials = train_test_split(
|
||||
all_trials,
|
||||
test_size=0.2,
|
||||
random_state=42
|
||||
)
|
||||
```
|
||||
|
||||
### 3. Train Model
|
||||
|
||||
```bash
|
||||
python train_parametric.py \
|
||||
--train_dir ../training_data/parsed \
|
||||
--val_dir ../validation_data/parsed \
|
||||
--epochs 200
|
||||
```
|
||||
|
||||
See [SYS_14_NEURAL_ACCELERATION](../system/SYS_14_NEURAL_ACCELERATION.md) for full training workflow.
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Cause | Solution |
|
||||
|---------|-------|----------|
|
||||
| No export directory | Export not enabled | Add `training_data_export` to config |
|
||||
| Missing OP2 files | Solve failed | Check `include_failed: false` |
|
||||
| Incomplete metadata | Extraction error | Check extractor logs |
|
||||
| Low sample count | Too many failures | Relax constraints |
|
||||
|
||||
---
|
||||
|
||||
## Cross-References
|
||||
|
||||
- **Related**: [SYS_14_NEURAL_ACCELERATION](../system/SYS_14_NEURAL_ACCELERATION.md)
|
||||
- **Preceded By**: [OP_02_RUN_OPTIMIZATION](./OP_02_RUN_OPTIMIZATION.md)
|
||||
- **Skill**: `.claude/skills/modules/neural-acceleration.md`
|
||||
|
||||
---
|
||||
|
||||
## Version History
|
||||
|
||||
| Version | Date | Changes |
|
||||
|---------|------|---------|
|
||||
| 1.0 | 2025-12-05 | Initial release |
|
||||
437
docs/protocols/operations/OP_06_TROUBLESHOOT.md
Normal file
437
docs/protocols/operations/OP_06_TROUBLESHOOT.md
Normal file
@@ -0,0 +1,437 @@
|
||||
# OP_06: Troubleshoot
|
||||
|
||||
<!--
|
||||
PROTOCOL: Troubleshoot Optimization Issues
|
||||
LAYER: Operations
|
||||
VERSION: 1.0
|
||||
STATUS: Active
|
||||
LAST_UPDATED: 2025-12-05
|
||||
PRIVILEGE: user
|
||||
LOAD_WITH: []
|
||||
-->
|
||||
|
||||
## Overview
|
||||
|
||||
This protocol provides systematic troubleshooting for common optimization issues, covering NX errors, extraction failures, database problems, and performance issues.
|
||||
|
||||
---
|
||||
|
||||
## When to Use
|
||||
|
||||
| Trigger | Action |
|
||||
|---------|--------|
|
||||
| "error", "failed" | Follow this protocol |
|
||||
| "not working", "crashed" | Follow this protocol |
|
||||
| "help", "stuck" | Follow this protocol |
|
||||
| Unexpected behavior | Follow this protocol |
|
||||
|
||||
---
|
||||
|
||||
## Quick Diagnostic
|
||||
|
||||
```bash
|
||||
# 1. Check environment
|
||||
conda activate atomizer
|
||||
python --version # Should be 3.9+
|
||||
|
||||
# 2. Check study structure
|
||||
ls studies/my_study/
|
||||
# Should have: 1_setup/, run_optimization.py
|
||||
|
||||
# 3. Check model files
|
||||
ls studies/my_study/1_setup/model/
|
||||
# Should have: .prt, .sim files
|
||||
|
||||
# 4. Test single trial
|
||||
python run_optimization.py --test
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Error Categories
|
||||
|
||||
### 1. Environment Errors
|
||||
|
||||
#### "ModuleNotFoundError: No module named 'optuna'"
|
||||
|
||||
**Cause**: Wrong Python environment
|
||||
|
||||
**Solution**:
|
||||
```bash
|
||||
conda activate atomizer
|
||||
# Verify
|
||||
conda list | grep optuna
|
||||
```
|
||||
|
||||
#### "Python version mismatch"
|
||||
|
||||
**Cause**: Wrong Python version
|
||||
|
||||
**Solution**:
|
||||
```bash
|
||||
python --version # Need 3.9+
|
||||
conda activate atomizer
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 2. NX Model Setup Errors
|
||||
|
||||
#### "All optimization trials produce identical results"
|
||||
|
||||
**Cause**: Missing idealized part (`*_i.prt`) or broken file chain
|
||||
|
||||
**Symptoms**:
|
||||
- Journal shows "FE model updated" but results don't change
|
||||
- DAT files have same node coordinates with different expressions
|
||||
- OP2 file timestamps update but values are identical
|
||||
|
||||
**Root Cause**: NX simulation files have a parent-child hierarchy:
|
||||
```
|
||||
.sim → .fem → _i.prt → .prt (geometry)
|
||||
```
|
||||
|
||||
If the `_i.prt` (idealized part) is missing or not properly linked, `UpdateFemodel()` runs but the mesh doesn't regenerate because:
|
||||
- FEM mesh is tied to idealized geometry, not master geometry
|
||||
- Without idealized part updating, FEM has nothing new to mesh against
|
||||
|
||||
**Solution**:
|
||||
1. **Check file chain in NX**:
|
||||
- Open `.sim` file
|
||||
- Go to **Part Navigator** or **Assembly Navigator**
|
||||
- List ALL referenced parts
|
||||
|
||||
2. **Copy ALL linked files** to study folder:
|
||||
```bash
|
||||
# Typical file set needed:
|
||||
Model.prt # Geometry
|
||||
Model_fem1_i.prt # Idealized part ← OFTEN MISSING!
|
||||
Model_fem1.fem # FEM file
|
||||
Model_sim1.sim # Simulation file
|
||||
```
|
||||
|
||||
3. **Verify links are intact**:
|
||||
- Open model in NX after copying
|
||||
- Check that updates propagate: Geometry → Idealized → FEM → Sim
|
||||
|
||||
4. **CRITICAL CODE FIX** (already implemented in `solve_simulation.py`):
|
||||
The idealized part MUST be explicitly loaded before `UpdateFemodel()`:
|
||||
```python
|
||||
# Load idealized part BEFORE updating FEM
|
||||
for filename in os.listdir(working_dir):
|
||||
if '_i.prt' in filename.lower():
|
||||
idealized_part, status = theSession.Parts.Open(path)
|
||||
break
|
||||
|
||||
# Now UpdateFemodel() will work correctly
|
||||
feModel.UpdateFemodel()
|
||||
```
|
||||
Without loading the `_i.prt`, NX cannot propagate geometry changes to the mesh.
|
||||
|
||||
**Prevention**: Always use introspection to list all parts referenced by a simulation.
|
||||
|
||||
---
|
||||
|
||||
### 3. NX/Solver Errors
|
||||
|
||||
#### "NX session timeout after 600s"
|
||||
|
||||
**Cause**: Model too complex or NX stuck
|
||||
|
||||
**Solution**:
|
||||
1. Increase timeout in config:
|
||||
```json
|
||||
"simulation": {
|
||||
"timeout": 1200
|
||||
}
|
||||
```
|
||||
2. Simplify mesh if possible
|
||||
3. Check NX license availability
|
||||
|
||||
#### "Expression 'xxx' not found in model"
|
||||
|
||||
**Cause**: Expression name mismatch
|
||||
|
||||
**Solution**:
|
||||
1. Open model in NX
|
||||
2. Go to Tools → Expressions
|
||||
3. Verify exact expression name (case-sensitive)
|
||||
4. Update config to match
|
||||
|
||||
#### "NX license error"
|
||||
|
||||
**Cause**: License server unavailable
|
||||
|
||||
**Solution**:
|
||||
1. Check license server status
|
||||
2. Wait and retry
|
||||
3. Contact IT if persistent
|
||||
|
||||
#### "NX solve failed - check log"
|
||||
|
||||
**Cause**: Nastran solver error
|
||||
|
||||
**Solution**:
|
||||
1. Find log file: `1_setup/model/*.log` or `*.f06`
|
||||
2. Search for "FATAL" or "ERROR"
|
||||
3. Common causes:
|
||||
- Singular stiffness matrix (constraints issue)
|
||||
- Bad mesh (distorted elements)
|
||||
- Missing material properties
|
||||
|
||||
---
|
||||
|
||||
### 3. Extraction Errors
|
||||
|
||||
#### "OP2 file not found"
|
||||
|
||||
**Cause**: Solve didn't produce output
|
||||
|
||||
**Solution**:
|
||||
1. Check if solve completed
|
||||
2. Look for `.op2` file in model directory
|
||||
3. Check NX log for solve errors
|
||||
|
||||
#### "No displacement data for subcase X"
|
||||
|
||||
**Cause**: Wrong subcase number
|
||||
|
||||
**Solution**:
|
||||
1. Check available subcases in OP2:
|
||||
```python
|
||||
from pyNastran.op2.op2 import OP2
|
||||
op2 = OP2()
|
||||
op2.read_op2('model.op2')
|
||||
print(op2.displacements.keys())
|
||||
```
|
||||
2. Update subcase in extractor call
|
||||
|
||||
#### "Element type 'xxx' not supported"
|
||||
|
||||
**Cause**: Extractor doesn't support element type
|
||||
|
||||
**Solution**:
|
||||
1. Check available types in extractor
|
||||
2. Common types: `cquad4`, `ctria3`, `ctetra`, `chexa`
|
||||
3. May need different extractor
|
||||
|
||||
---
|
||||
|
||||
### 4. Database Errors
|
||||
|
||||
#### "Database is locked"
|
||||
|
||||
**Cause**: Another process using database
|
||||
|
||||
**Solution**:
|
||||
1. Check for running processes:
|
||||
```bash
|
||||
ps aux | grep run_optimization
|
||||
```
|
||||
2. Kill stale process if needed
|
||||
3. Wait for other optimization to finish
|
||||
|
||||
#### "Study 'xxx' not found"
|
||||
|
||||
**Cause**: Wrong study name or path
|
||||
|
||||
**Solution**:
|
||||
1. Check exact study name in database:
|
||||
```python
|
||||
import optuna
|
||||
storage = optuna.storages.RDBStorage('sqlite:///study.db')
|
||||
print(storage.get_all_study_summaries())
|
||||
```
|
||||
2. Use correct name when loading
|
||||
|
||||
#### "IntegrityError: UNIQUE constraint failed"
|
||||
|
||||
**Cause**: Duplicate trial number
|
||||
|
||||
**Solution**:
|
||||
1. Don't run multiple optimizations on same study simultaneously
|
||||
2. Use `--resume` flag for continuation
|
||||
|
||||
---
|
||||
|
||||
### 5. Constraint/Feasibility Errors
|
||||
|
||||
#### "All trials pruned"
|
||||
|
||||
**Cause**: No feasible region
|
||||
|
||||
**Solution**:
|
||||
1. Check constraint values:
|
||||
```python
|
||||
# In objective function, print constraint values
|
||||
print(f"Stress: {stress}, limit: 250")
|
||||
```
|
||||
2. Relax constraints
|
||||
3. Widen design variable bounds
|
||||
|
||||
#### "No improvement after N trials"
|
||||
|
||||
**Cause**: Stuck in local minimum or converged
|
||||
|
||||
**Solution**:
|
||||
1. Check if truly converged (good result)
|
||||
2. Try different starting region
|
||||
3. Use different sampler
|
||||
4. Increase exploration (lower `n_startup_trials`)
|
||||
|
||||
---
|
||||
|
||||
### 6. Performance Issues
|
||||
|
||||
#### "Trials running very slowly"
|
||||
|
||||
**Cause**: Complex model or inefficient extraction
|
||||
|
||||
**Solution**:
|
||||
1. Profile time per component:
|
||||
```python
|
||||
import time
|
||||
start = time.time()
|
||||
# ... operation ...
|
||||
print(f"Took: {time.time() - start:.1f}s")
|
||||
```
|
||||
2. Simplify mesh if NX is slow
|
||||
3. Check extraction isn't re-parsing OP2 multiple times
|
||||
|
||||
#### "Memory error"
|
||||
|
||||
**Cause**: Large OP2 file or many trials
|
||||
|
||||
**Solution**:
|
||||
1. Clear Python memory between trials
|
||||
2. Don't store all results in memory
|
||||
3. Use database for persistence
|
||||
|
||||
---
|
||||
|
||||
## Diagnostic Commands
|
||||
|
||||
### Quick Health Check
|
||||
|
||||
```bash
|
||||
# Environment
|
||||
conda activate atomizer
|
||||
python -c "import optuna; print('Optuna OK')"
|
||||
python -c "import pyNastran; print('pyNastran OK')"
|
||||
|
||||
# Study structure
|
||||
ls -la studies/my_study/
|
||||
|
||||
# Config validity
|
||||
python -c "
|
||||
import json
|
||||
with open('studies/my_study/1_setup/optimization_config.json') as f:
|
||||
config = json.load(f)
|
||||
print('Config OK')
|
||||
print(f'Objectives: {len(config.get(\"objectives\", []))}')
|
||||
"
|
||||
|
||||
# Database status
|
||||
python -c "
|
||||
import optuna
|
||||
study = optuna.load_study('my_study', 'sqlite:///studies/my_study/2_results/study.db')
|
||||
print(f'Trials: {len(study.trials)}')
|
||||
"
|
||||
```
|
||||
|
||||
### NX Log Analysis
|
||||
|
||||
```bash
|
||||
# Find latest log
|
||||
ls -lt studies/my_study/1_setup/model/*.log | head -1
|
||||
|
||||
# Search for errors
|
||||
grep -i "error\|fatal\|fail" studies/my_study/1_setup/model/*.log
|
||||
```
|
||||
|
||||
### Trial Failure Analysis
|
||||
|
||||
```python
|
||||
import optuna
|
||||
|
||||
study = optuna.load_study(...)
|
||||
|
||||
# Failed trials
|
||||
failed = [t for t in study.trials
|
||||
if t.state == optuna.trial.TrialState.FAIL]
|
||||
print(f"Failed: {len(failed)}")
|
||||
|
||||
for t in failed[:5]:
|
||||
print(f"Trial {t.number}: {t.user_attrs}")
|
||||
|
||||
# Pruned trials
|
||||
pruned = [t for t in study.trials
|
||||
if t.state == optuna.trial.TrialState.PRUNED]
|
||||
print(f"Pruned: {len(pruned)}")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Recovery Actions
|
||||
|
||||
### Reset Study (Start Fresh)
|
||||
|
||||
```bash
|
||||
# Backup first
|
||||
cp -r studies/my_study/2_results studies/my_study/2_results_backup
|
||||
|
||||
# Delete results
|
||||
rm -rf studies/my_study/2_results/*
|
||||
|
||||
# Run fresh
|
||||
python run_optimization.py
|
||||
```
|
||||
|
||||
### Resume Interrupted Study
|
||||
|
||||
```bash
|
||||
python run_optimization.py --resume
|
||||
```
|
||||
|
||||
### Restore from Backup
|
||||
|
||||
```bash
|
||||
cp -r studies/my_study/2_results_backup/* studies/my_study/2_results/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Getting Help
|
||||
|
||||
### Information to Provide
|
||||
|
||||
When asking for help, include:
|
||||
1. Error message (full traceback)
|
||||
2. Config file contents
|
||||
3. Study structure (`ls -la`)
|
||||
4. What you tried
|
||||
5. NX log excerpt (if NX error)
|
||||
|
||||
### Log Locations
|
||||
|
||||
| Log | Location |
|
||||
|-----|----------|
|
||||
| Optimization | Console output or redirect to file |
|
||||
| NX Solve | `1_setup/model/*.log`, `*.f06` |
|
||||
| Database | `2_results/study.db` (query with optuna) |
|
||||
| Intelligence | `2_results/intelligent_optimizer/*.json` |
|
||||
|
||||
---
|
||||
|
||||
## Cross-References
|
||||
|
||||
- **Related**: All operation protocols
|
||||
- **System**: [SYS_10_IMSO](../system/SYS_10_IMSO.md), [SYS_12_EXTRACTOR_LIBRARY](../system/SYS_12_EXTRACTOR_LIBRARY.md)
|
||||
|
||||
---
|
||||
|
||||
## Version History
|
||||
|
||||
| Version | Date | Changes |
|
||||
|---------|------|---------|
|
||||
| 1.0 | 2025-12-05 | Initial release |
|
||||
Reference in New Issue
Block a user