docs: Major documentation overhaul - restructure folders, update tagline, add Getting Started guide
- Restructure docs/ folder (remove numeric prefixes): - 04_USER_GUIDES -> guides/ - 05_API_REFERENCE -> api/ - 06_PHYSICS -> physics/ - 07_DEVELOPMENT -> development/ - 08_ARCHIVE -> archive/ - 09_DIAGRAMS -> diagrams/ - Replace tagline 'Talk, don't click' with 'LLM-driven optimization framework' in 9 files - Create comprehensive docs/GETTING_STARTED.md: - Prerequisites and quick setup - Project structure overview - First study tutorial (Claude or manual) - Dashboard usage guide - Neural acceleration introduction - Rewrite docs/00_INDEX.md with correct paths and modern structure - Archive obsolete files: - 01_PROTOCOLS.md -> archive/historical/01_PROTOCOLS_legacy.md - 03_GETTING_STARTED.md -> archive/historical/ - ATOMIZER_PODCAST_BRIEFING.md -> archive/marketing/ - Update timestamps to 2026-01-20 across all key files - Update .gitignore to exclude docs/generated/ - Version bump: ATOMIZER_CONTEXT v1.8 -> v2.0
This commit is contained in:
450
docs/guides/hybrid_mode.md
Normal file
450
docs/guides/hybrid_mode.md
Normal file
@@ -0,0 +1,450 @@
|
||||
# Hybrid LLM Mode Guide
|
||||
**Recommended Mode for Development** | Phase 3.2 Architecture | November 18, 2025
|
||||
|
||||
## What is Hybrid Mode?
|
||||
|
||||
Hybrid Mode (Mode 2) gives you **90% of the automation** with **10% of the complexity**. It's the sweet spot between manual configuration and full LLM autonomy.
|
||||
|
||||
### Why Hybrid Mode?
|
||||
|
||||
- ✅ **No API Key Required** - Use Claude Code/Desktop instead of Claude API
|
||||
- ✅ **90% Automation** - Auto-generates extractors, calculations, and hooks
|
||||
- ✅ **Full Transparency** - You see and approve the workflow JSON
|
||||
- ✅ **Production Ready** - Uses centralized library system
|
||||
- ✅ **Easy to Upgrade** - Can enable full API mode later
|
||||
|
||||
## How It Works
|
||||
|
||||
### The Workflow
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ HYBRID MODE - 90% Automation, No API Key │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
|
||||
1. YOU + CLAUDE CODE:
|
||||
├─ Describe optimization in natural language
|
||||
└─ Claude helps create workflow JSON
|
||||
↓
|
||||
2. SAVE workflow JSON:
|
||||
├─ llm_workflow_config.json
|
||||
└─ Contains: design vars, objectives, constraints
|
||||
↓
|
||||
3. LLMOptimizationRunner:
|
||||
├─ Auto-generates extractors (pyNastran)
|
||||
├─ Auto-generates calculations
|
||||
├─ Auto-generates hooks
|
||||
├─ Adds to core library (deduplication!)
|
||||
└─ Runs optimization loop (Optuna)
|
||||
↓
|
||||
4. RESULTS:
|
||||
├─ optimization_results.json (best design)
|
||||
├─ optimization_history.json (all trials)
|
||||
├─ extractors_manifest.json (what was used)
|
||||
└─ Study folder stays clean!
|
||||
```
|
||||
|
||||
## Step-by-Step: Your First Hybrid Optimization
|
||||
|
||||
### Step 1: Describe Your Optimization to Claude
|
||||
|
||||
**Example conversation with Claude Code:**
|
||||
|
||||
```
|
||||
YOU: I want to optimize a bracket design.
|
||||
- Design variables: wall_thickness (1-5mm), fillet_radius (2-8mm)
|
||||
- Objective: minimize mass
|
||||
- Constraints: max_stress < 200 MPa, max_displacement < 0.5mm
|
||||
- I have a Beam.prt file and Beam_sim1.sim file ready
|
||||
|
||||
CLAUDE: I'll help you create the workflow JSON...
|
||||
```
|
||||
|
||||
### Step 2: Claude Creates Workflow JSON
|
||||
|
||||
Claude will generate a file like this:
|
||||
|
||||
```json
|
||||
{
|
||||
"study_name": "bracket_optimization",
|
||||
"optimization_request": "Minimize mass while keeping stress below 200 MPa and displacement below 0.5mm",
|
||||
|
||||
"design_variables": [
|
||||
{
|
||||
"parameter": "wall_thickness",
|
||||
"bounds": [1, 5],
|
||||
"description": "Bracket wall thickness in mm"
|
||||
},
|
||||
{
|
||||
"parameter": "fillet_radius",
|
||||
"bounds": [2, 8],
|
||||
"description": "Fillet radius in mm"
|
||||
}
|
||||
],
|
||||
|
||||
"objectives": [
|
||||
{
|
||||
"name": "mass",
|
||||
"goal": "minimize",
|
||||
"weight": 1.0,
|
||||
"extraction": {
|
||||
"action": "extract_mass",
|
||||
"domain": "result_extraction",
|
||||
"params": {
|
||||
"result_type": "mass",
|
||||
"metric": "total"
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
|
||||
"constraints": [
|
||||
{
|
||||
"name": "max_stress_limit",
|
||||
"type": "less_than",
|
||||
"threshold": 200,
|
||||
"extraction": {
|
||||
"action": "extract_von_mises_stress",
|
||||
"domain": "result_extraction",
|
||||
"params": {
|
||||
"result_type": "stress",
|
||||
"metric": "max"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "max_displacement_limit",
|
||||
"type": "less_than",
|
||||
"threshold": 0.5,
|
||||
"extraction": {
|
||||
"action": "extract_displacement",
|
||||
"domain": "result_extraction",
|
||||
"params": {
|
||||
"result_type": "displacement",
|
||||
"metric": "max"
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Step 3: Save and Review
|
||||
|
||||
Save the JSON to your study directory:
|
||||
|
||||
```
|
||||
studies/
|
||||
bracket_optimization/
|
||||
1_setup/
|
||||
model/
|
||||
Bracket.prt # Your NX model
|
||||
Bracket_sim1.sim # Your FEM setup
|
||||
workflow_config.json # ← SAVE HERE
|
||||
```
|
||||
|
||||
**IMPORTANT**: Review the JSON before running! Check:
|
||||
- ✅ Design variable names match your NX expressions
|
||||
- ✅ Bounds are in correct units (mm not m!)
|
||||
- ✅ Extraction actions match available OP2 results
|
||||
|
||||
### Step 4: Run LLMOptimizationRunner
|
||||
|
||||
Now the magic happens - 90% automation kicks in:
|
||||
|
||||
```python
|
||||
from pathlib import Path
|
||||
from optimization_engine.llm_optimization_runner import LLMOptimizationRunner
|
||||
|
||||
# Point to your files
|
||||
study_dir = Path("studies/bracket_optimization")
|
||||
workflow_json = study_dir / "1_setup/workflow_config.json"
|
||||
prt_file = study_dir / "1_setup/model/Bracket.prt"
|
||||
sim_file = study_dir / "1_setup/model/Bracket_sim1.sim"
|
||||
output_dir = study_dir / "2_substudies/optimization_run_001"
|
||||
|
||||
# Create runner
|
||||
runner = LLMOptimizationRunner(
|
||||
llm_workflow_file=workflow_json,
|
||||
prt_file=prt_file,
|
||||
sim_file=sim_file,
|
||||
output_dir=output_dir,
|
||||
n_trials=20
|
||||
)
|
||||
|
||||
# Run optimization (this is where automation happens!)
|
||||
study = runner.run()
|
||||
|
||||
print(f"Best design found:")
|
||||
print(f" wall_thickness: {study.best_params['wall_thickness']:.2f} mm")
|
||||
print(f" fillet_radius: {study.best_params['fillet_radius']:.2f} mm")
|
||||
print(f" mass: {study.best_value:.4f} kg")
|
||||
```
|
||||
|
||||
### Step 5: What Gets Auto-Generated
|
||||
|
||||
During the run, the system automatically:
|
||||
|
||||
1. **Analyzes OP2 structure** (pyNastran research agent)
|
||||
2. **Generates extractors** and adds to core library:
|
||||
```
|
||||
optimization_engine/extractors/
|
||||
├── extract_mass.py ← Generated!
|
||||
├── extract_von_mises_stress.py ← Generated!
|
||||
└── extract_displacement.py ← Generated!
|
||||
```
|
||||
3. **Creates study manifest** (no code pollution!):
|
||||
```
|
||||
2_substudies/optimization_run_001/
|
||||
└── extractors_manifest.json ← References only
|
||||
```
|
||||
4. **Runs optimization loop** with Optuna
|
||||
5. **Saves full audit trail**:
|
||||
```
|
||||
2_substudies/optimization_run_001/
|
||||
├── llm_workflow_config.json ← What you specified
|
||||
├── extractors_manifest.json ← What was used
|
||||
├── optimization_results.json ← Best design
|
||||
└── optimization_history.json ← All trials
|
||||
```
|
||||
|
||||
## Real Example: Beam Optimization
|
||||
|
||||
Let's walk through the existing beam optimization:
|
||||
|
||||
### Natural Language Request
|
||||
"I want to optimize a sandwich beam. Design variables are core thickness (20-30mm), face thickness (1-3mm), hole diameter (180-280mm), and number of holes (8-14). Minimize weight while keeping displacement under 2mm."
|
||||
|
||||
### Claude Creates JSON
|
||||
```json
|
||||
{
|
||||
"study_name": "simple_beam_optimization",
|
||||
"optimization_request": "Minimize weight subject to max displacement < 2mm",
|
||||
|
||||
"design_variables": [
|
||||
{"parameter": "beam_half_core_thickness", "bounds": [20, 30]},
|
||||
{"parameter": "beam_face_thickness", "bounds": [1, 3]},
|
||||
{"parameter": "holes_diameter", "bounds": [180, 280]},
|
||||
{"parameter": "hole_count", "bounds": [8, 14]}
|
||||
],
|
||||
|
||||
"objectives": [
|
||||
{
|
||||
"name": "mass",
|
||||
"goal": "minimize",
|
||||
"weight": 1.0,
|
||||
"extraction": {
|
||||
"action": "extract_mass",
|
||||
"domain": "result_extraction",
|
||||
"params": {"result_type": "mass", "metric": "total"}
|
||||
}
|
||||
}
|
||||
],
|
||||
|
||||
"constraints": [
|
||||
{
|
||||
"name": "max_displacement_limit",
|
||||
"type": "less_than",
|
||||
"threshold": 2.0,
|
||||
"extraction": {
|
||||
"action": "extract_displacement",
|
||||
"domain": "result_extraction",
|
||||
"params": {"result_type": "displacement", "metric": "max"}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Run Script
|
||||
```python
|
||||
from pathlib import Path
|
||||
from optimization_engine.llm_optimization_runner import LLMOptimizationRunner
|
||||
|
||||
study_dir = Path("studies/simple_beam_optimization")
|
||||
runner = LLMOptimizationRunner(
|
||||
llm_workflow_file=study_dir / "1_setup/workflow_config.json",
|
||||
prt_file=study_dir / "1_setup/model/Beam.prt",
|
||||
sim_file=study_dir / "1_setup/model/Beam_sim1.sim",
|
||||
output_dir=study_dir / "2_substudies/test_run",
|
||||
n_trials=20
|
||||
)
|
||||
|
||||
study = runner.run()
|
||||
```
|
||||
|
||||
### Results After 20 Trials
|
||||
```
|
||||
Best design found:
|
||||
beam_half_core_thickness: 27.3 mm
|
||||
beam_face_thickness: 2.1 mm
|
||||
holes_diameter: 245.2 mm
|
||||
hole_count: 11
|
||||
mass: 1.234 kg (45% reduction!)
|
||||
max_displacement: 1.87 mm (within limit)
|
||||
```
|
||||
|
||||
### Study Folder (Clean!)
|
||||
```
|
||||
2_substudies/test_run/
|
||||
├── extractors_manifest.json # Just references
|
||||
├── llm_workflow_config.json # What you wanted
|
||||
├── optimization_results.json # Best design
|
||||
└── optimization_history.json # All 20 trials
|
||||
```
|
||||
|
||||
## Advanced: Extractor Library Reuse
|
||||
|
||||
The beauty of centralized library system:
|
||||
|
||||
### First Optimization Run
|
||||
```python
|
||||
# First beam optimization
|
||||
runner1 = LLMOptimizationRunner(...)
|
||||
runner1.run()
|
||||
|
||||
# Creates:
|
||||
# optimization_engine/extractors/extract_mass.py
|
||||
# optimization_engine/extractors/extract_displacement.py
|
||||
```
|
||||
|
||||
### Second Optimization Run (Different Study!)
|
||||
```python
|
||||
# Different bracket optimization (but same extractions!)
|
||||
runner2 = LLMOptimizationRunner(...)
|
||||
runner2.run()
|
||||
|
||||
# REUSES existing extractors!
|
||||
# No duplicate code generated
|
||||
# Study folder stays clean
|
||||
```
|
||||
|
||||
The system automatically detects identical extraction functionality and reuses code from the core library.
|
||||
|
||||
## Comparison: Three Modes
|
||||
|
||||
| Feature | Manual Mode | **Hybrid Mode** | Full LLM Mode |
|
||||
|---------|-------------|-----------------|---------------|
|
||||
| API Key Required | ❌ No | ❌ No | ✅ Yes |
|
||||
| Automation Level | 0% (you code) | 90% (auto-gen) | 100% (NL only) |
|
||||
| Extractor Generation | Manual | ✅ Auto | ✅ Auto |
|
||||
| Hook Generation | Manual | ✅ Auto | ✅ Auto |
|
||||
| Core Library | Manual | ✅ Auto | ✅ Auto |
|
||||
| Transparency | Full | Full | High |
|
||||
| Development Speed | Slow | **Fast** | Fastest |
|
||||
| Production Ready | ✅ Yes | ✅ Yes | ⚠️ Alpha |
|
||||
| **Recommended For** | Complex custom | **Most users** | Future |
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Issue: "Expression not found in NX model"
|
||||
**Problem**: Design variable name doesn't match NX expression name
|
||||
|
||||
**Solution**:
|
||||
1. Open your `.prt` file in NX
|
||||
2. Tools → Expression → check exact names
|
||||
3. Update workflow JSON with exact names
|
||||
|
||||
**Example**:
|
||||
```json
|
||||
// WRONG
|
||||
"parameter": "thickness"
|
||||
|
||||
// RIGHT (must match NX exactly)
|
||||
"parameter": "beam_half_core_thickness"
|
||||
```
|
||||
|
||||
### Issue: "No mass results in OP2"
|
||||
**Problem**: OP2 file doesn't contain mass data
|
||||
|
||||
**Solution**:
|
||||
1. Check what's actually in the OP2:
|
||||
```python
|
||||
from pyNastran.op2.op2 import OP2
|
||||
model = OP2()
|
||||
model.read_op2('path/to/results.op2')
|
||||
print(dir(model)) # See available results
|
||||
```
|
||||
|
||||
2. Use available result type instead (e.g., von Mises stress, displacement)
|
||||
|
||||
### Issue: "Extractor generation failed"
|
||||
**Problem**: pyNastran research agent couldn't figure out extraction pattern
|
||||
|
||||
**Solution**:
|
||||
1. Check `optimization_engine/knowledge_base/` for available patterns
|
||||
2. Manually create extractor in `optimization_engine/extractors/`
|
||||
3. Reference it in workflow JSON using existing action name
|
||||
|
||||
### Issue: "Parameter values too extreme"
|
||||
**Problem**: Design variables using wrong range (0.2-1.0 instead of 20-30)
|
||||
|
||||
**Fixed**: This was the bug we fixed on Nov 17! Make sure you're using latest code.
|
||||
|
||||
**Verify**:
|
||||
```python
|
||||
# Check bounds parsing in llm_optimization_runner.py
|
||||
if 'bounds' in var_config:
|
||||
var_min, var_max = var_config['bounds'] # Should use this!
|
||||
```
|
||||
|
||||
## Tips for Success
|
||||
|
||||
### 1. Start Small
|
||||
- First run: 5-10 trials to verify everything works
|
||||
- Check results, review auto-generated extractors
|
||||
- Then scale up to 50-100 trials
|
||||
|
||||
### 2. Verify Units
|
||||
- NX expressions: Check Tools → Expression
|
||||
- Workflow JSON: Match units exactly
|
||||
- Common mistake: mm vs m, kg vs g
|
||||
|
||||
### 3. Use Existing Examples
|
||||
- `studies/simple_beam_optimization/` - Working example
|
||||
- Copy the structure, modify workflow JSON
|
||||
- Reuse proven patterns
|
||||
|
||||
### 4. Review Auto-Generated Code
|
||||
```python
|
||||
# After first run, check what was generated:
|
||||
from optimization_engine.extractor_library import ExtractorLibrary
|
||||
|
||||
library = ExtractorLibrary()
|
||||
print(library.get_library_summary())
|
||||
```
|
||||
|
||||
### 5. Leverage Deduplication
|
||||
- Same extraction across studies? Library reuses code!
|
||||
- No need to regenerate extractors
|
||||
- Study folders stay clean automatically
|
||||
|
||||
## Next Steps
|
||||
|
||||
### Ready to Test?
|
||||
1. ✅ Read this guide
|
||||
2. ✅ Review beam optimization example
|
||||
3. ✅ Create your workflow JSON with Claude's help
|
||||
4. ✅ Run your first optimization!
|
||||
|
||||
### Want Full Automation?
|
||||
When you're ready for Full LLM Mode (Mode 3):
|
||||
1. Set up Claude API key
|
||||
2. Use natural language requests (no JSON needed!)
|
||||
3. System creates workflow JSON automatically
|
||||
4. Everything else identical to Hybrid Mode
|
||||
|
||||
### Questions?
|
||||
- Check `docs/ARCHITECTURE_REFACTOR_NOV17.md` for library system details
|
||||
- Review `optimization_engine/llm_optimization_runner.py` for implementation
|
||||
- Run E2E test: `python tests/test_phase_3_2_e2e.py`
|
||||
|
||||
---
|
||||
|
||||
**Status**: Production Ready ✅
|
||||
**Mode**: Hybrid (90% Automation)
|
||||
**API Required**: No
|
||||
**Testing**: E2E tests passing (18/18 checks)
|
||||
**Architecture**: Centralized library with deduplication
|
||||
|
||||
**Ready to revolutionize your optimization workflow!** 🚀
|
||||
Reference in New Issue
Block a user