feat: Add MLP surrogate with Turbo Mode for 100x faster optimization

Neural Acceleration (MLP Surrogate):
- Add run_nn_optimization.py with hybrid FEA/NN workflow
- MLP architecture: 4-layer (64->128->128->64) with BatchNorm/Dropout
- Three workflow modes:
  - --all: Sequential export->train->optimize->validate
  - --hybrid-loop: Iterative Train->NN->Validate->Retrain cycle
  - --turbo: Aggressive single-best validation (RECOMMENDED)
- Turbo mode: 5000 NN trials + 50 FEA validations in ~12 minutes
- Separate nn_study.db to avoid overloading dashboard

Performance Results (bracket_pareto_3obj study):
- NN prediction errors: mass 1-5%, stress 1-4%, stiffness 5-15%
- Found minimum mass designs at boundary (angle~30deg, thick~30mm)
- 100x speedup vs pure FEA exploration

Protocol Operating System:
- Add .claude/skills/ with Bootstrap, Cheatsheet, Context Loader
- Add docs/protocols/ with operations (OP_01-06) and system (SYS_10-14)
- Update SYS_14_NEURAL_ACCELERATION.md with MLP Turbo Mode docs

NX Automation:
- Add optimization_engine/hooks/ for NX CAD/CAE automation
- Add study_wizard.py for guided study creation
- Fix FEM mesh update: load idealized part before UpdateFemodel()

New Study:
- bracket_pareto_3obj: 3-objective Pareto (mass, stress, stiffness)
- 167 FEA trials + 5000 NN trials completed
- Demonstrates full hybrid workflow

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
Antoine
2025-12-06 20:01:59 -05:00
parent 0cb2808c44
commit 602560c46a
70 changed files with 31018 additions and 289 deletions

View File

@@ -0,0 +1,395 @@
# EXT_01: Create New Extractor
<!--
PROTOCOL: Create New Physics Extractor
LAYER: Extensions
VERSION: 1.0
STATUS: Active
LAST_UPDATED: 2025-12-05
PRIVILEGE: power_user
LOAD_WITH: [SYS_12_EXTRACTOR_LIBRARY]
-->
## Overview
This protocol guides you through creating a new physics extractor for the centralized extractor library. Follow this when you need to extract results not covered by existing extractors.
**Privilege Required**: power_user or admin
---
## When to Use
| Trigger | Action |
|---------|--------|
| Need physics not in library | Follow this protocol |
| "create extractor", "new extractor" | Follow this protocol |
| Custom result extraction needed | Follow this protocol |
**First**: Check [SYS_12_EXTRACTOR_LIBRARY](../system/SYS_12_EXTRACTOR_LIBRARY.md) - the functionality may already exist!
---
## Quick Reference
**Create in**: `optimization_engine/extractors/`
**Export from**: `optimization_engine/extractors/__init__.py`
**Document in**: Update SYS_12 and this protocol
**Template location**: `docs/protocols/extensions/templates/extractor_template.py`
---
## Step-by-Step Guide
### Step 1: Verify Need
Before creating:
1. Check existing extractors in [SYS_12](../system/SYS_12_EXTRACTOR_LIBRARY.md)
2. Search codebase: `grep -r "your_physics" optimization_engine/`
3. Confirm no existing solution
### Step 1.5: Research NX Open APIs (REQUIRED for NX extractors)
**If the extractor needs NX Open APIs** (not just pyNastran OP2 parsing):
```
# 1. Search for relevant NX Open APIs
siemens_docs_search("inertia properties NXOpen")
siemens_docs_search("mass properties body NXOpen.CAE")
# 2. Fetch detailed documentation for promising classes
siemens_docs_fetch("NXOpen.MeasureManager")
siemens_docs_fetch("NXOpen.UF.UFWeight")
# 3. Get method signatures
siemens_docs_search("AskMassProperties NXOpen")
```
**When to use NX Open vs pyNastran:**
| Data Source | Tool | Example |
|-------------|------|---------|
| OP2 results (stress, disp, freq) | pyNastran | `extract_displacement()` |
| CAD properties (mass, inertia) | NX Open | New extractor with NXOpen API |
| BDF data (mesh, properties) | pyNastran | `extract_mass_from_bdf()` |
| NX expressions | NX Open | `extract_mass_from_expression()` |
| FEM model data | NX Open CAE | Needs `NXOpen.CAE.*` APIs |
**Document the APIs used** in the extractor docstring:
```python
def extract_inertia(part_file: Path) -> Dict[str, Any]:
"""
Extract mass and inertia properties from NX part.
NX Open APIs Used:
- NXOpen.MeasureManager.NewMassProperties()
- NXOpen.MeasureBodies.InformationUnit
- NXOpen.UF.UFWeight.AskProps()
See: docs.sw.siemens.com for full API reference
"""
```
### Step 2: Create Extractor File
Create `optimization_engine/extractors/extract_{physics}.py`:
```python
"""
Extract {Physics Name} from FEA results.
Author: {Your Name}
Created: {Date}
Version: 1.0
"""
from pathlib import Path
from typing import Dict, Any, Optional, Union
from pyNastran.op2.op2 import OP2
def extract_{physics}(
op2_file: Union[str, Path],
subcase: int = 1,
# Add other parameters as needed
) -> Dict[str, Any]:
"""
Extract {physics description} from OP2 file.
Args:
op2_file: Path to the OP2 results file
subcase: Subcase number to extract (default: 1)
Returns:
Dictionary containing:
- '{main_result}': The primary result value
- '{secondary}': Additional result info
- 'subcase': The subcase extracted
Raises:
FileNotFoundError: If OP2 file doesn't exist
KeyError: If subcase not found in results
ValueError: If result data is invalid
Example:
>>> result = extract_{physics}('model.op2', subcase=1)
>>> print(result['{main_result}'])
123.45
"""
op2_file = Path(op2_file)
if not op2_file.exists():
raise FileNotFoundError(f"OP2 file not found: {op2_file}")
# Read OP2 file
op2 = OP2()
op2.read_op2(str(op2_file))
# Extract your physics
# TODO: Implement extraction logic
# Example for displacement-like result:
if subcase not in op2.displacements:
raise KeyError(f"Subcase {subcase} not found in results")
data = op2.displacements[subcase]
# Process data...
return {
'{main_result}': computed_value,
'{secondary}': secondary_value,
'subcase': subcase,
}
# Optional: Class-based extractor for complex cases
class {Physics}Extractor:
"""
Class-based extractor for {physics} with state management.
Use when extraction requires multiple steps or configuration.
"""
def __init__(self, op2_file: Union[str, Path], **config):
self.op2_file = Path(op2_file)
self.config = config
self._op2 = None
def _load_op2(self):
"""Lazy load OP2 file."""
if self._op2 is None:
self._op2 = OP2()
self._op2.read_op2(str(self.op2_file))
return self._op2
def extract(self, subcase: int = 1) -> Dict[str, Any]:
"""Extract results for given subcase."""
op2 = self._load_op2()
# Implementation here
pass
```
### Step 3: Add to __init__.py
Edit `optimization_engine/extractors/__init__.py`:
```python
# Add import
from .extract_{physics} import extract_{physics}
# Or for class
from .extract_{physics} import {Physics}Extractor
# Add to __all__
__all__ = [
# ... existing exports ...
'extract_{physics}',
'{Physics}Extractor',
]
```
### Step 4: Write Tests
Create `tests/test_extract_{physics}.py`:
```python
"""Tests for {physics} extractor."""
import pytest
from pathlib import Path
from optimization_engine.extractors import extract_{physics}
class TestExtract{Physics}:
"""Test suite for {physics} extraction."""
@pytest.fixture
def sample_op2(self, tmp_path):
"""Create or copy sample OP2 for testing."""
# Either copy existing test file or create mock
pass
def test_basic_extraction(self, sample_op2):
"""Test basic extraction works."""
result = extract_{physics}(sample_op2)
assert '{main_result}' in result
assert isinstance(result['{main_result}'], float)
def test_file_not_found(self):
"""Test error handling for missing file."""
with pytest.raises(FileNotFoundError):
extract_{physics}('nonexistent.op2')
def test_invalid_subcase(self, sample_op2):
"""Test error handling for invalid subcase."""
with pytest.raises(KeyError):
extract_{physics}(sample_op2, subcase=999)
```
### Step 5: Document
#### Update SYS_12_EXTRACTOR_LIBRARY.md
Add to Quick Reference table:
```markdown
| E{N} | {Physics} | `extract_{physics}()` | .op2 | {unit} |
```
Add detailed section:
```markdown
### E{N}: {Physics} Extraction
**Module**: `optimization_engine.extractors.extract_{physics}`
\`\`\`python
from optimization_engine.extractors import extract_{physics}
result = extract_{physics}(op2_file, subcase=1)
{main_result} = result['{main_result}']
\`\`\`
```
#### Update skills/modules/extractors-catalog.md
Add entry following existing pattern.
### Step 6: Validate
```bash
# Run tests
pytest tests/test_extract_{physics}.py -v
# Test import
python -c "from optimization_engine.extractors import extract_{physics}; print('OK')"
# Test with real file
python -c "
from optimization_engine.extractors import extract_{physics}
result = extract_{physics}('path/to/test.op2')
print(result)
"
```
---
## Extractor Design Guidelines
### Do's
- Return dictionaries with clear keys
- Include metadata (subcase, units, etc.)
- Handle edge cases gracefully
- Provide clear error messages
- Document all parameters and returns
- Write tests
### Don'ts
- Don't re-parse OP2 multiple times in one call
- Don't hardcode paths
- Don't swallow exceptions silently
- Don't return raw pyNastran objects
- Don't modify input files
### Naming Conventions
| Type | Convention | Example |
|------|------------|---------|
| File | `extract_{physics}.py` | `extract_thermal.py` |
| Function | `extract_{physics}` | `extract_thermal` |
| Class | `{Physics}Extractor` | `ThermalExtractor` |
| Return key | lowercase_with_underscores | `max_temperature` |
---
## Examples
### Example: Thermal Gradient Extractor
```python
"""Extract thermal gradients from temperature results."""
from pathlib import Path
from typing import Dict, Any
from pyNastran.op2.op2 import OP2
import numpy as np
def extract_thermal_gradient(
op2_file: Path,
subcase: int = 1,
direction: str = 'magnitude'
) -> Dict[str, Any]:
"""
Extract thermal gradient from temperature field.
Args:
op2_file: Path to OP2 file
subcase: Subcase number
direction: 'magnitude', 'x', 'y', or 'z'
Returns:
Dictionary with gradient results
"""
op2 = OP2()
op2.read_op2(str(op2_file))
temps = op2.temperatures[subcase]
# Calculate gradient...
return {
'max_gradient': max_grad,
'mean_gradient': mean_grad,
'max_gradient_location': location,
'direction': direction,
'subcase': subcase,
'unit': 'K/mm'
}
```
---
## Troubleshooting
| Issue | Cause | Solution |
|-------|-------|----------|
| Import error | Not added to __init__.py | Add export |
| "No module" | Wrong file location | Check path |
| KeyError | Wrong OP2 data structure | Debug OP2 contents |
| Tests fail | Missing test data | Create fixtures |
---
## Cross-References
- **Reference**: [SYS_12_EXTRACTOR_LIBRARY](../system/SYS_12_EXTRACTOR_LIBRARY.md)
- **Template**: `templates/extractor_template.py`
- **Related**: [EXT_02_CREATE_HOOK](./EXT_02_CREATE_HOOK.md)
---
## Version History
| Version | Date | Changes |
|---------|------|---------|
| 1.0 | 2025-12-05 | Initial release |

View File

@@ -0,0 +1,366 @@
# EXT_02: Create Lifecycle Hook
<!--
PROTOCOL: Create Lifecycle Hook Plugin
LAYER: Extensions
VERSION: 1.0
STATUS: Active
LAST_UPDATED: 2025-12-05
PRIVILEGE: power_user
LOAD_WITH: []
-->
## Overview
This protocol guides you through creating lifecycle hooks that execute at specific points during optimization. Hooks enable custom logic injection without modifying core code.
**Privilege Required**: power_user or admin
---
## When to Use
| Trigger | Action |
|---------|--------|
| Need custom logic at specific point | Follow this protocol |
| "create hook", "callback" | Follow this protocol |
| Want to log/validate/modify at runtime | Follow this protocol |
---
## Quick Reference
**Hook Points Available**:
| Hook Point | When It Runs | Use Case |
|------------|--------------|----------|
| `pre_mesh` | Before meshing | Validate geometry |
| `post_mesh` | After meshing | Check mesh quality |
| `pre_solve` | Before solver | Log trial start |
| `post_solve` | After solver | Validate results |
| `post_extraction` | After extraction | Custom metrics |
| `post_calculation` | After objectives | Derived quantities |
| `custom_objective` | Custom objective | Complex objectives |
**Create in**: `optimization_engine/plugins/{hook_point}/`
---
## Step-by-Step Guide
### Step 1: Identify Hook Point
Choose the appropriate hook point:
```
Trial Flow:
├─► PRE_MESH → Validate model before meshing
├─► POST_MESH → Check mesh quality
├─► PRE_SOLVE → Log trial start, validate inputs
├─► POST_SOLVE → Check solve success, capture timing
├─► POST_EXTRACTION → Compute derived quantities
├─► POST_CALCULATION → Final validation, logging
└─► CUSTOM_OBJECTIVE → Custom objective functions
```
### Step 2: Create Hook File
Create `optimization_engine/plugins/{hook_point}/{hook_name}.py`:
```python
"""
{Hook Description}
Author: {Your Name}
Created: {Date}
Version: 1.0
Hook Point: {hook_point}
"""
from typing import Dict, Any
def {hook_name}_hook(context: Dict[str, Any]) -> Dict[str, Any]:
"""
{Description of what this hook does}.
Args:
context: Dictionary containing:
- trial_number: Current trial number
- design_params: Current design parameters
- results: Results so far (if post-extraction)
- config: Optimization config
- working_dir: Path to working directory
Returns:
Dictionary with computed values or modifications.
Return empty dict if no modifications needed.
Example:
>>> result = {hook_name}_hook({'trial_number': 1, ...})
>>> print(result)
{'{computed_key}': 123.45}
"""
# Access context
trial_num = context.get('trial_number')
design_params = context.get('design_params', {})
results = context.get('results', {})
# Your logic here
# ...
# Return computed values
return {
'{computed_key}': computed_value,
}
def register_hooks(hook_manager):
"""
Register this hook with the hook manager.
This function is called automatically when plugins are loaded.
Args:
hook_manager: The HookManager instance
"""
hook_manager.register_hook(
hook_point='{hook_point}',
function={hook_name}_hook,
name='{hook_name}_hook',
description='{Brief description}',
priority=100, # Lower = runs earlier
enabled=True
)
```
### Step 3: Test Hook
```python
# Test in isolation
from optimization_engine.plugins.{hook_point}.{hook_name} import {hook_name}_hook
test_context = {
'trial_number': 1,
'design_params': {'thickness': 5.0},
'results': {'max_stress': 200.0},
}
result = {hook_name}_hook(test_context)
print(result)
```
### Step 4: Enable Hook
Hooks are auto-discovered from the plugins directory. To verify:
```python
from optimization_engine.plugins.hook_manager import HookManager
manager = HookManager()
manager.discover_plugins()
print(manager.list_hooks())
```
---
## Hook Examples
### Example 1: Safety Factor Calculator (post_calculation)
```python
"""Calculate safety factor after stress extraction."""
def safety_factor_hook(context):
"""Calculate safety factor from stress results."""
results = context.get('results', {})
config = context.get('config', {})
max_stress = results.get('max_von_mises', 0)
yield_strength = config.get('material', {}).get('yield_strength', 250)
if max_stress > 0:
safety_factor = yield_strength / max_stress
else:
safety_factor = float('inf')
return {
'safety_factor': safety_factor,
'yield_strength': yield_strength,
}
def register_hooks(hook_manager):
hook_manager.register_hook(
hook_point='post_calculation',
function=safety_factor_hook,
name='safety_factor_hook',
description='Calculate safety factor from stress',
priority=100,
enabled=True
)
```
### Example 2: Trial Logger (pre_solve)
```python
"""Log trial information before solve."""
import json
from datetime import datetime
from pathlib import Path
def trial_logger_hook(context):
"""Log trial start information."""
trial_num = context.get('trial_number')
design_params = context.get('design_params', {})
working_dir = context.get('working_dir', Path('.'))
log_entry = {
'trial': trial_num,
'timestamp': datetime.now().isoformat(),
'params': design_params,
}
log_file = working_dir / 'trial_log.jsonl'
with open(log_file, 'a') as f:
f.write(json.dumps(log_entry) + '\n')
return {} # No modifications
def register_hooks(hook_manager):
hook_manager.register_hook(
hook_point='pre_solve',
function=trial_logger_hook,
name='trial_logger_hook',
description='Log trial parameters before solve',
priority=10, # Run early
enabled=True
)
```
### Example 3: Mesh Quality Check (post_mesh)
```python
"""Validate mesh quality after meshing."""
def mesh_quality_hook(context):
"""Check mesh quality metrics."""
mesh_file = context.get('mesh_file')
# Check quality metrics
quality_issues = []
# ... quality checks ...
if quality_issues:
context['warnings'] = context.get('warnings', []) + quality_issues
return {
'mesh_quality_passed': len(quality_issues) == 0,
'mesh_issues': quality_issues,
}
def register_hooks(hook_manager):
hook_manager.register_hook(
hook_point='post_mesh',
function=mesh_quality_hook,
name='mesh_quality_hook',
description='Validate mesh quality',
priority=50,
enabled=True
)
```
---
## Hook Context Reference
### Standard Context Keys
| Key | Type | Available At | Description |
|-----|------|--------------|-------------|
| `trial_number` | int | All | Current trial number |
| `design_params` | dict | All | Design parameter values |
| `config` | dict | All | Optimization config |
| `working_dir` | Path | All | Study working directory |
| `model_file` | Path | pre_mesh+ | NX model file path |
| `mesh_file` | Path | post_mesh+ | Mesh file path |
| `op2_file` | Path | post_solve+ | Results file path |
| `results` | dict | post_extraction+ | Extracted results |
| `objectives` | dict | post_calculation | Computed objectives |
### Priority Guidelines
| Priority Range | Use For |
|----------------|---------|
| 1-50 | Critical hooks that must run first |
| 50-100 | Standard hooks |
| 100-150 | Logging and monitoring |
| 150+ | Cleanup and finalization |
---
## Managing Hooks
### Enable/Disable at Runtime
```python
hook_manager.disable_hook('my_hook')
hook_manager.enable_hook('my_hook')
```
### Check Hook Status
```python
hooks = hook_manager.list_hooks()
for hook in hooks:
print(f"{hook['name']}: {'enabled' if hook['enabled'] else 'disabled'}")
```
### Hook Execution Order
Hooks at the same point run in priority order (lower first):
```
Priority 10: trial_logger_hook
Priority 50: mesh_quality_hook
Priority 100: safety_factor_hook
```
---
## Troubleshooting
| Issue | Cause | Solution |
|-------|-------|----------|
| Hook not running | Not registered | Check `register_hooks` function |
| Wrong hook point | Misnamed directory | Check directory name matches hook point |
| Context missing key | Wrong hook point | Use appropriate hook point for data needed |
| Hook error crashes trial | Unhandled exception | Add try/except in hook |
---
## Cross-References
- **Related**: [EXT_01_CREATE_EXTRACTOR](./EXT_01_CREATE_EXTRACTOR.md)
- **System**: `optimization_engine/plugins/hook_manager.py`
- **Template**: `templates/hook_template.py`
---
## Version History
| Version | Date | Changes |
|---------|------|---------|
| 1.0 | 2025-12-05 | Initial release |

View File

@@ -0,0 +1,263 @@
# EXT_03: Create New Protocol
<!--
PROTOCOL: Create New Protocol Document
LAYER: Extensions
VERSION: 1.0
STATUS: Active
LAST_UPDATED: 2025-12-05
PRIVILEGE: admin
LOAD_WITH: []
-->
## Overview
This protocol guides you through creating new protocol documents for the Atomizer Protocol Operating System (POS). Use this when adding significant new system capabilities.
**Privilege Required**: admin
---
## When to Use
| Trigger | Action |
|---------|--------|
| Adding major new system capability | Follow this protocol |
| "create protocol", "new protocol" | Follow this protocol |
| Need to document architectural pattern | Follow this protocol |
---
## Protocol Types
| Layer | Prefix | Purpose | Example |
|-------|--------|---------|---------|
| Operations | OP_ | How-to guides | OP_01_CREATE_STUDY |
| System | SYS_ | Core specifications | SYS_10_IMSO |
| Extensions | EXT_ | Extensibility guides | EXT_01_CREATE_EXTRACTOR |
---
## Step-by-Step Guide
### Step 1: Determine Protocol Type
- **Operations (OP_)**: User-facing procedures
- **System (SYS_)**: Technical specifications
- **Extensions (EXT_)**: Developer guides
### Step 2: Assign Protocol Number
**Operations**: Sequential (OP_01, OP_02, ...)
**System**: By feature area (SYS_10=optimization, SYS_11=multi-obj, etc.)
**Extensions**: Sequential (EXT_01, EXT_02, ...)
Check existing protocols to avoid conflicts.
### Step 3: Create Protocol File
Use the template from `templates/protocol_template.md`:
```markdown
# {LAYER}_{NUMBER}_{NAME}.md
<!--
PROTOCOL: {Full Name}
LAYER: {Operations|System|Extensions}
VERSION: 1.0
STATUS: Active
LAST_UPDATED: {YYYY-MM-DD}
PRIVILEGE: {user|power_user|admin}
LOAD_WITH: [{dependencies}]
-->
## Overview
{1-3 sentence description of what this protocol does}
---
## When to Use
| Trigger | Action |
|---------|--------|
| {keyword or condition} | Follow this protocol |
---
## Quick Reference
{Tables with key parameters, commands, or mappings}
---
## Detailed Specification
### Section 1: {Topic}
{Content}
### Section 2: {Topic}
{Content}
---
## Examples
### Example 1: {Scenario}
{Complete working example}
---
## Troubleshooting
| Symptom | Cause | Solution |
|---------|-------|----------|
| {error} | {why} | {fix} |
---
## Cross-References
- **Depends On**: [{protocol}]({path})
- **Used By**: [{protocol}]({path})
- **See Also**: [{related}]({path})
---
## Version History
| Version | Date | Changes |
|---------|------|---------|
| 1.0 | {DATE} | Initial release |
```
### Step 4: Write Content
**Required Sections**:
1. Overview - What does this protocol do?
2. When to Use - Trigger conditions
3. Quick Reference - Fast lookup
4. Detailed Specification - Full content
5. Examples - Working examples
6. Troubleshooting - Common issues
7. Cross-References - Related protocols
8. Version History - Changes over time
**Writing Guidelines**:
- Front-load important information
- Use tables for structured data
- Include complete code examples
- Provide troubleshooting for common issues
### Step 5: Update Navigation
**docs/protocols/README.md**:
```markdown
| {NUM} | {Name} | [{Layer}]({layer}/{filename}) |
```
**.claude/skills/01_CHEATSHEET.md**:
```markdown
| {task} | {LAYER}_{NUM} | {key info} |
```
**.claude/skills/02_CONTEXT_LOADER.md**:
Add loading rules if needed.
### Step 6: Update Cross-References
Add references in related protocols:
- "Depends On" in new protocol
- "Used By" or "See Also" in existing protocols
### Step 7: Validate
```bash
# Check markdown syntax
# Verify all links work
# Test code examples
# Ensure consistent formatting
```
---
## Protocol Metadata
### Header Comment Block
```markdown
<!--
PROTOCOL: Full Protocol Name
LAYER: Operations|System|Extensions
VERSION: Major.Minor
STATUS: Active|Draft|Deprecated
LAST_UPDATED: YYYY-MM-DD
PRIVILEGE: user|power_user|admin
LOAD_WITH: [SYS_10, SYS_11]
-->
```
### Status Values
| Status | Meaning |
|--------|---------|
| Draft | In development, not ready for use |
| Active | Production ready |
| Deprecated | Being phased out |
### Privilege Levels
| Level | Who Can Use |
|-------|-------------|
| user | All users |
| power_user | Developers who can extend |
| admin | Full system access |
---
## Versioning
### Semantic Versioning
- **Major (X.0)**: Breaking changes
- **Minor (1.X)**: New features, backward compatible
- **Patch (1.0.X)**: Bug fixes (usually omit for docs)
### Version History Format
```markdown
| Version | Date | Changes |
|---------|------|---------|
| 2.0 | 2025-12-15 | Redesigned architecture |
| 1.1 | 2025-12-05 | Added neural support |
| 1.0 | 2025-11-20 | Initial release |
```
---
## Troubleshooting
| Issue | Cause | Solution |
|-------|-------|----------|
| Protocol not found | Wrong path | Check location and README |
| LLM not loading | Missing from context loader | Update 02_CONTEXT_LOADER.md |
| Broken links | Path changed | Update cross-references |
---
## Cross-References
- **Template**: `templates/protocol_template.md`
- **Navigation**: `docs/protocols/README.md`
- **Context Loading**: `.claude/skills/02_CONTEXT_LOADER.md`
---
## Version History
| Version | Date | Changes |
|---------|------|---------|
| 1.0 | 2025-12-05 | Initial release |

View File

@@ -0,0 +1,331 @@
# EXT_04: Create New Skill
<!--
PROTOCOL: Create New Skill or Module
LAYER: Extensions
VERSION: 1.0
STATUS: Active
LAST_UPDATED: 2025-12-05
PRIVILEGE: admin
LOAD_WITH: []
-->
## Overview
This protocol guides you through creating new skills or skill modules for the LLM instruction system. Skills provide task-specific guidance to Claude sessions.
**Privilege Required**: admin
---
## When to Use
| Trigger | Action |
|---------|--------|
| Need new LLM capability | Follow this protocol |
| "create skill", "new skill" | Follow this protocol |
| Task pattern needs documentation | Follow this protocol |
---
## Skill Types
| Type | Location | Purpose | Example |
|------|----------|---------|---------|
| Bootstrap | `.claude/skills/0X_*.md` | LLM orientation | 00_BOOTSTRAP.md |
| Core | `.claude/skills/core/` | Always-load skills | study-creation-core.md |
| Module | `.claude/skills/modules/` | Optional, load-on-demand | extractors-catalog.md |
| Dev | `.claude/skills/DEV_*.md` | Developer workflows | DEV_DOCUMENTATION.md |
---
## Step-by-Step Guide
### Step 1: Determine Skill Type
**Bootstrap (0X_)**: System-level LLM guidance
- Task classification
- Context loading rules
- Execution patterns
**Core**: Essential task skills that are always loaded
- Study creation
- Run optimization (basic)
**Module**: Specialized skills loaded on demand
- Specific extractors
- Domain-specific (Zernike, neural)
- Advanced features
**Dev (DEV_)**: Developer-facing workflows
- Documentation maintenance
- Testing procedures
- Contribution guides
### Step 2: Create Skill File
#### For Core/Module Skills
```markdown
# {Skill Name}
**Version**: 1.0
**Purpose**: {One-line description}
---
## Overview
{What this skill enables Claude to do}
---
## When to Load
This skill should be loaded when:
- {Condition 1}
- {Condition 2}
---
## Quick Reference
{Tables with key patterns, commands}
---
## Detailed Instructions
### Pattern 1: {Name}
{Step-by-step instructions}
**Example**:
\`\`\`python
{code example}
\`\`\`
### Pattern 2: {Name}
{Step-by-step instructions}
---
## Code Templates
### Template 1: {Name}
\`\`\`python
{copy-paste ready code}
\`\`\`
---
## Validation
Before completing:
- [ ] {Check 1}
- [ ] {Check 2}
---
## Related
- **Protocol**: [{related}]({path})
- **Module**: [{related}]({path})
```
### Step 3: Register Skill
#### For Bootstrap Skills
Add to `00_BOOTSTRAP.md` task classification tree.
#### For Core Skills
Add to `02_CONTEXT_LOADER.md`:
```yaml
{TASK_TYPE}:
always_load:
- core/{skill_name}.md
```
#### For Modules
Add to `02_CONTEXT_LOADER.md`:
```yaml
{TASK_TYPE}:
load_if:
- modules/{skill_name}.md: "{condition}"
```
### Step 4: Update Navigation
Add to `01_CHEATSHEET.md` if relevant to common tasks.
### Step 5: Test
Test with fresh Claude session:
1. Start new conversation
2. Describe task that should trigger skill
3. Verify correct skill is loaded
4. Verify skill instructions are followed
---
## Skill Design Guidelines
### Structure
- **Front-load**: Most important info first
- **Tables**: Use for structured data
- **Code blocks**: Complete, copy-paste ready
- **Checklists**: For validation steps
### Content
- **Task-focused**: What should Claude DO?
- **Prescriptive**: Clear instructions, not options
- **Examples**: Show expected patterns
- **Validation**: How to verify success
### Length Guidelines
| Skill Type | Target Lines | Rationale |
|------------|--------------|-----------|
| Bootstrap | 100-200 | Quick orientation |
| Core | 500-1000 | Comprehensive task guide |
| Module | 150-400 | Focused specialization |
### Avoid
- Duplicating protocol content (reference instead)
- Vague instructions ("consider" → "do")
- Missing examples
- Untested code
---
## Module vs Protocol
**Skills** teach Claude HOW to interact:
- Conversation patterns
- Code templates
- Validation steps
- User interaction
**Protocols** document WHAT exists:
- Technical specifications
- Configuration options
- Architecture details
- Troubleshooting
Skills REFERENCE protocols, don't duplicate them.
---
## Examples
### Example: Domain-Specific Module
`modules/thermal-optimization.md`:
```markdown
# Thermal Optimization Module
**Version**: 1.0
**Purpose**: Specialized guidance for thermal FEA optimization
---
## When to Load
Load when:
- "thermal", "temperature", "heat" in user request
- Optimizing for thermal properties
---
## Quick Reference
| Physics | Extractor | Unit |
|---------|-----------|------|
| Max temp | E11 | K |
| Gradient | E12 | K/mm |
| Heat flux | E13 | W/m² |
---
## Objective Patterns
### Minimize Max Temperature
\`\`\`python
from optimization_engine.extractors import extract_temperature
def objective(trial):
# ... run simulation ...
temp_result = extract_temperature(op2_file)
return temp_result['max_temperature']
\`\`\`
### Minimize Thermal Gradient
\`\`\`python
from optimization_engine.extractors import extract_thermal_gradient
def objective(trial):
# ... run simulation ...
grad_result = extract_thermal_gradient(op2_file)
return grad_result['max_gradient']
\`\`\`
---
## Configuration Example
\`\`\`json
{
"objectives": [
{
"name": "max_temperature",
"type": "minimize",
"unit": "K",
"description": "Maximum temperature in component"
}
]
}
\`\`\`
---
## Related
- **Extractors**: E11, E12, E13 in SYS_12
- **Protocol**: See OP_01 for study creation
```
---
## Troubleshooting
| Issue | Cause | Solution |
|-------|-------|----------|
| Skill not loaded | Not in context loader | Add loading rule |
| Wrong skill loaded | Ambiguous triggers | Refine conditions |
| Instructions not followed | Too vague | Make prescriptive |
---
## Cross-References
- **Context Loader**: `.claude/skills/02_CONTEXT_LOADER.md`
- **Bootstrap**: `.claude/skills/00_BOOTSTRAP.md`
- **Related**: [EXT_03_CREATE_PROTOCOL](./EXT_03_CREATE_PROTOCOL.md)
---
## Version History
| Version | Date | Changes |
|---------|------|---------|
| 1.0 | 2025-12-05 | Initial release |

View File

@@ -0,0 +1,186 @@
"""
Extract {Physics Name} from FEA results.
This is a template for creating new physics extractors.
Copy this file to optimization_engine/extractors/extract_{physics}.py
and customize for your specific physics extraction.
Author: {Your Name}
Created: {Date}
Version: 1.0
"""
from pathlib import Path
from typing import Dict, Any, Optional, Union
from pyNastran.op2.op2 import OP2
def extract_{physics}(
op2_file: Union[str, Path],
subcase: int = 1,
# Add other parameters specific to your physics
) -> Dict[str, Any]:
"""
Extract {physics description} from OP2 file.
Args:
op2_file: Path to the OP2 results file
subcase: Subcase number to extract (default: 1)
# Document other parameters
Returns:
Dictionary containing:
- '{main_result}': The primary result value ({unit})
- '{secondary_result}': Secondary result info
- 'subcase': The subcase extracted
- 'unit': Unit of the result
Raises:
FileNotFoundError: If OP2 file doesn't exist
KeyError: If subcase not found in results
ValueError: If result data is invalid
Example:
>>> result = extract_{physics}('model.op2', subcase=1)
>>> print(result['{main_result}'])
123.45
>>> print(result['unit'])
'{unit}'
"""
# Convert to Path for consistency
op2_file = Path(op2_file)
# Validate file exists
if not op2_file.exists():
raise FileNotFoundError(f"OP2 file not found: {op2_file}")
# Read OP2 file
op2 = OP2()
op2.read_op2(str(op2_file))
# =========================================
# CUSTOMIZE: Your extraction logic here
# =========================================
# Example: Access displacement data
# if subcase not in op2.displacements:
# raise KeyError(f"Subcase {subcase} not found in displacement results")
# data = op2.displacements[subcase]
# Example: Access stress data
# if subcase not in op2.cquad4_stress:
# raise KeyError(f"Subcase {subcase} not found in stress results")
# stress_data = op2.cquad4_stress[subcase]
# Example: Process data
# values = data.data # numpy array
# max_value = values.max()
# max_index = values.argmax()
# =========================================
# Replace with your actual computation
# =========================================
main_result = 0.0 # TODO: Compute actual value
secondary_result = 0 # TODO: Compute actual value
return {
'{main_result}': main_result,
'{secondary_result}': secondary_result,
'subcase': subcase,
'unit': '{unit}',
}
# Optional: Class-based extractor for complex cases
class {Physics}Extractor:
"""
Class-based extractor for {physics} with state management.
Use this pattern when:
- Extraction requires multiple steps
- You need to cache the OP2 data
- Configuration is complex
Example:
>>> extractor = {Physics}Extractor('model.op2', config={'option': value})
>>> result = extractor.extract(subcase=1)
>>> print(result)
"""
def __init__(
self,
op2_file: Union[str, Path],
bdf_file: Optional[Union[str, Path]] = None,
**config
):
"""
Initialize the extractor.
Args:
op2_file: Path to OP2 results file
bdf_file: Optional path to BDF mesh file (for node coordinates)
**config: Additional configuration options
"""
self.op2_file = Path(op2_file)
self.bdf_file = Path(bdf_file) if bdf_file else None
self.config = config
self._op2 = None # Lazy-loaded
def _load_op2(self) -> OP2:
"""Lazy load OP2 file (caches result)."""
if self._op2 is None:
self._op2 = OP2()
self._op2.read_op2(str(self.op2_file))
return self._op2
def extract(self, subcase: int = 1) -> Dict[str, Any]:
"""
Extract results for given subcase.
Args:
subcase: Subcase number
Returns:
Dictionary with extraction results
"""
op2 = self._load_op2()
# TODO: Implement your extraction logic
# Use self.config for configuration options
return {
'{main_result}': 0.0,
'subcase': subcase,
}
def extract_all_subcases(self) -> Dict[int, Dict[str, Any]]:
"""
Extract results for all available subcases.
Returns:
Dictionary mapping subcase number to results
"""
op2 = self._load_op2()
# TODO: Find available subcases
# available_subcases = list(op2.displacements.keys())
results = {}
# for sc in available_subcases:
# results[sc] = self.extract(subcase=sc)
return results
# =========================================
# After creating your extractor:
# 1. Add to optimization_engine/extractors/__init__.py:
# from .extract_{physics} import extract_{physics}
# __all__ = [..., 'extract_{physics}']
#
# 2. Update docs/protocols/system/SYS_12_EXTRACTOR_LIBRARY.md
# - Add to Quick Reference table
# - Add detailed section with example
#
# 3. Create test file: tests/test_extract_{physics}.py
# =========================================

View File

@@ -0,0 +1,213 @@
"""
{Hook Name} - Lifecycle Hook Plugin
This is a template for creating new lifecycle hooks.
Copy this file to optimization_engine/plugins/{hook_point}/{hook_name}.py
Available hook points:
- pre_mesh: Before meshing
- post_mesh: After meshing
- pre_solve: Before solver execution
- post_solve: After solver completion
- post_extraction: After result extraction
- post_calculation: After objective calculation
- custom_objective: Custom objective functions
Author: {Your Name}
Created: {Date}
Version: 1.0
Hook Point: {hook_point}
"""
from typing import Dict, Any, Optional
from pathlib import Path
import json
from datetime import datetime
def {hook_name}_hook(context: Dict[str, Any]) -> Dict[str, Any]:
"""
{Description of what this hook does}.
This hook runs at the {hook_point} stage of the optimization trial.
Args:
context: Dictionary containing trial context:
- trial_number (int): Current trial number
- design_params (dict): Current design parameter values
- config (dict): Optimization configuration
- working_dir (Path): Study working directory
For post_solve and later:
- op2_file (Path): Path to OP2 results file
- solve_success (bool): Whether solve succeeded
- solve_time (float): Solve duration in seconds
For post_extraction and later:
- results (dict): Extracted results so far
For post_calculation:
- objectives (dict): Computed objective values
- constraints (dict): Constraint values
Returns:
Dictionary with computed values or modifications.
These values are added to the trial context.
Return empty dict {} if no modifications needed.
Raises:
Exception: Any exception will be logged but won't stop the trial
unless you want it to (raise optuna.TrialPruned instead)
Example:
>>> context = {'trial_number': 1, 'design_params': {'x': 5.0}}
>>> result = {hook_name}_hook(context)
>>> print(result)
{{'{computed_key}': 123.45}}
"""
# =========================================
# Access context values
# =========================================
trial_num = context.get('trial_number', 0)
design_params = context.get('design_params', {})
config = context.get('config', {})
working_dir = context.get('working_dir', Path('.'))
# For post_solve hooks and later:
# op2_file = context.get('op2_file')
# solve_success = context.get('solve_success', False)
# For post_extraction hooks and later:
# results = context.get('results', {})
# For post_calculation hooks:
# objectives = context.get('objectives', {})
# constraints = context.get('constraints', {})
# =========================================
# Your hook logic here
# =========================================
# Example: Log trial start (pre_solve hook)
# print(f"[Hook] Trial {trial_num} starting with params: {design_params}")
# Example: Compute derived quantity (post_extraction hook)
# max_stress = results.get('max_von_mises', 0)
# yield_strength = config.get('material', {}).get('yield_strength', 250)
# safety_factor = yield_strength / max(max_stress, 1e-6)
# Example: Write log file (post_calculation hook)
# log_entry = {
# 'trial': trial_num,
# 'timestamp': datetime.now().isoformat(),
# 'objectives': context.get('objectives', {}),
# }
# with open(working_dir / 'trial_log.jsonl', 'a') as f:
# f.write(json.dumps(log_entry) + '\n')
# =========================================
# Return computed values
# =========================================
# Values returned here are added to the context
# and can be accessed by later hooks or the optimizer
return {
# '{computed_key}': computed_value,
}
def register_hooks(hook_manager) -> None:
"""
Register this hook with the hook manager.
This function is called automatically when plugins are discovered.
It must be named exactly 'register_hooks' and take one argument.
Args:
hook_manager: The HookManager instance from optimization_engine
"""
hook_manager.register_hook(
hook_point='{hook_point}', # pre_mesh, post_mesh, pre_solve, etc.
function={hook_name}_hook,
name='{hook_name}_hook',
description='{Brief description of what this hook does}',
priority=100, # Lower number = runs earlier (1-200 typical range)
enabled=True # Set to False to disable by default
)
# =========================================
# Optional: Helper functions
# =========================================
def _helper_function(data: Any) -> Any:
"""
Private helper function for the hook.
Keep hook logic clean by extracting complex operations
into helper functions.
"""
pass
# =========================================
# After creating your hook:
#
# 1. Place in correct directory:
# optimization_engine/plugins/{hook_point}/{hook_name}.py
#
# 2. Hook is auto-discovered - no __init__.py changes needed
#
# 3. Test the hook:
# python -c "
# from optimization_engine.plugins.hook_manager import HookManager
# hm = HookManager()
# hm.discover_plugins()
# print(hm.list_hooks())
# "
#
# 4. Update documentation if significant:
# - Add to EXT_02_CREATE_HOOK.md examples section
# =========================================
# =========================================
# Example hooks for reference
# =========================================
def example_logger_hook(context: Dict[str, Any]) -> Dict[str, Any]:
"""Example: Simple trial logger for pre_solve."""
trial = context.get('trial_number', 0)
params = context.get('design_params', {})
print(f"[LOG] Trial {trial} starting: {params}")
return {}
def example_safety_factor_hook(context: Dict[str, Any]) -> Dict[str, Any]:
"""Example: Safety factor calculator for post_extraction."""
results = context.get('results', {})
max_stress = results.get('max_von_mises', 0)
if max_stress > 0:
safety_factor = 250.0 / max_stress # Assuming 250 MPa yield
else:
safety_factor = float('inf')
return {'safety_factor': safety_factor}
def example_validator_hook(context: Dict[str, Any]) -> Dict[str, Any]:
"""Example: Result validator for post_solve."""
import optuna
solve_success = context.get('solve_success', False)
op2_file = context.get('op2_file')
if not solve_success:
raise optuna.TrialPruned("Solve failed")
if op2_file and not Path(op2_file).exists():
raise optuna.TrialPruned("OP2 file not generated")
return {'validation_passed': True}

View File

@@ -0,0 +1,112 @@
# {LAYER}_{NUMBER}_{NAME}
<!--
PROTOCOL: {Full Protocol Name}
LAYER: {Operations|System|Extensions}
VERSION: 1.0
STATUS: Active
LAST_UPDATED: {YYYY-MM-DD}
PRIVILEGE: {user|power_user|admin}
LOAD_WITH: [{dependency_protocols}]
-->
## Overview
{1-3 sentence description of what this protocol does and why it exists.}
---
## When to Use
| Trigger | Action |
|---------|--------|
| {keyword or user intent} | Follow this protocol |
| {condition} | Follow this protocol |
---
## Quick Reference
{Key information in table format for fast lookup}
| Parameter | Default | Description |
|-----------|---------|-------------|
| {param} | {value} | {description} |
---
## Detailed Specification
### Section 1: {Topic}
{Detailed content}
```python
# Code example if applicable
```
### Section 2: {Topic}
{Detailed content}
---
## Configuration
{If applicable, show configuration examples}
```json
{
"setting": "value"
}
```
---
## Examples
### Example 1: {Scenario Name}
{Complete working example with context}
```python
# Full working code example
```
### Example 2: {Scenario Name}
{Another example showing different use case}
---
## Troubleshooting
| Symptom | Cause | Solution |
|---------|-------|----------|
| {error message or symptom} | {root cause} | {how to fix} |
| {symptom} | {cause} | {solution} |
---
## Cross-References
- **Depends On**: [{protocol_name}]({relative_path})
- **Used By**: [{protocol_name}]({relative_path})
- **See Also**: [{related_doc}]({path})
---
## Implementation Files
{If applicable, list the code files that implement this protocol}
- `path/to/file.py` - {description}
- `path/to/other.py` - {description}
---
## Version History
| Version | Date | Changes |
|---------|------|---------|
| 1.0 | {YYYY-MM-DD} | Initial release |