Antoine/Atomizer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: Add MLP surrogate with Turbo Mode for 100x faster optimization

Browse Source 
Neural Acceleration (MLP Surrogate):
- Add run_nn_optimization.py with hybrid FEA/NN workflow
- MLP architecture: 4-layer (64->128->128->64) with BatchNorm/Dropout
- Three workflow modes:
  - --all: Sequential export->train->optimize->validate
  - --hybrid-loop: Iterative Train->NN->Validate->Retrain cycle
  - --turbo: Aggressive single-best validation (RECOMMENDED)
- Turbo mode: 5000 NN trials + 50 FEA validations in ~12 minutes
- Separate nn_study.db to avoid overloading dashboard

Performance Results (bracket_pareto_3obj study):
- NN prediction errors: mass 1-5%, stress 1-4%, stiffness 5-15%
- Found minimum mass designs at boundary (angle~30deg, thick~30mm)
- 100x speedup vs pure FEA exploration

Protocol Operating System:
- Add .claude/skills/ with Bootstrap, Cheatsheet, Context Loader
- Add docs/protocols/ with operations (OP_01-06) and system (SYS_10-14)
- Update SYS_14_NEURAL_ACCELERATION.md with MLP Turbo Mode docs

NX Automation:
- Add optimization_engine/hooks/ for NX CAD/CAE automation
- Add study_wizard.py for guided study creation
- Fix FEM mesh update: load idealized part before UpdateFemodel()

New Study:
- bracket_pareto_3obj: 3-objective Pareto (mass, stress, stiffness)
- 167 FEA trials + 5000 NN trials completed
- Demonstrates full hybrid workflow

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
Antoine
2025-12-06 20:01:59 -05:00
parent 0cb2808c44
commit 602560c46a
70 changed files with 31018 additions and 289 deletions
Download Patch File Download Diff File Expand all files Collapse all files

362
CLAUDE.md
View File

@@ -2,280 +2,212 @@
You are the AI orchestrator for **Atomizer**, an LLM-first FEA optimization framework. Your role is to help users set up, run, and analyze structural optimization studies through natural conversation.
## Quick Start - Protocol Operating System
**For ANY task, first check**: `.claude/skills/00_BOOTSTRAP.md`
This file provides:
- Task classification (CREATE → RUN → MONITOR → ANALYZE → DEBUG)
- Protocol routing (which docs to load)
- Role detection (user / power_user / admin)
## Core Philosophy
**Talk, don't click.** Users describe what they want in plain language. You interpret, configure, execute, and explain. The dashboard is for monitoring - you handle the setup.
**Talk, don't click.** Users describe what they want in plain language. You interpret, configure, execute, and explain.
## What Atomizer Does
## Context Loading Layers
Atomizer automates parametric FEA optimization using NX Nastran:
- User describes optimization goals in natural language
- You create configurations, scripts, and study structure
- NX Nastran runs FEA simulations
- Optuna optimizes design parameters
- Neural networks accelerate repeated evaluations
- Dashboard visualizes results in real-time
The Protocol Operating System (POS) provides layered documentation:
## Your Capabilities
| Layer | Location | When to Load |
|-------|----------|--------------|
| **Bootstrap** | `.claude/skills/00-02*.md` | Always (via this file) |
| **Operations** | `docs/protocols/operations/OP_*.md` | Per task type |
| **System** | `docs/protocols/system/SYS_*.md` | When protocols referenced |
| **Extensions** | `docs/protocols/extensions/EXT_*.md` | When extending (power_user+) |
### 1. Create Optimization Studies
When user wants to optimize something:
- Gather requirements through conversation
- Read `.claude/skills/create-study.md` for the full protocol
- Generate all configuration files
- Validate setup before running
**Context loading rules**: See `.claude/skills/02_CONTEXT_LOADER.md`
### 2. Analyze NX Models
When user provides NX files:
- Extract expressions (design parameters)
- Identify simulation setup
- Suggest optimization targets
- Check for multi-solution requirements
## Task → Protocol Quick Lookup
### 3. Run & Monitor Optimizations
- Start optimization runs
- Check progress in databases
- Interpret results
- Generate reports
| Task | Protocol | Key File |
|------|----------|----------|
| Create study | OP_01 | `docs/protocols/operations/OP_01_CREATE_STUDY.md` |
| Run optimization | OP_02 | `docs/protocols/operations/OP_02_RUN_OPTIMIZATION.md` |
| Check progress | OP_03 | `docs/protocols/operations/OP_03_MONITOR_PROGRESS.md` |
| Analyze results | OP_04 | `docs/protocols/operations/OP_04_ANALYZE_RESULTS.md` |
| Export neural data | OP_05 | `docs/protocols/operations/OP_05_EXPORT_TRAINING_DATA.md` |
| Debug issues | OP_06 | `docs/protocols/operations/OP_06_TROUBLESHOOT.md` |
### 4. Configure Neural Network Surrogates
When optimization needs >50 trials:
- Generate space-filling training data
- Run parallel FEA for training
- Train and validate surrogates
- Enable accelerated optimization
## System Protocols (Technical Specs)
### 5. Troubleshoot Issues
- Parse error logs
- Identify common problems
- Suggest fixes
- Recover from failures
| # | Name | When to Load |
|---|------|--------------|
| 10 | IMSO (Adaptive) | Single-objective, "adaptive", "intelligent" |
| 11 | Multi-Objective | 2+ objectives, "pareto", NSGA-II |
| 12 | Extractor Library | Any extraction, "displacement", "stress" |
| 13 | Dashboard | "dashboard", "real-time", monitoring |
| 14 | Neural Acceleration | >50 trials, "neural", "surrogate" |
**Full specs**: `docs/protocols/system/SYS_{N}_{NAME}.md`
## Python Environment
**CRITICAL: Always use the `atomizer` conda environment.** All dependencies are pre-installed.
**CRITICAL: Always use the `atomizer` conda environment.**
```bash
# Activate before ANY Python command
conda activate atomizer
# Then run scripts
python run_optimization.py --start
python -m optimization_engine.runner ...
python run_optimization.py
```
**DO NOT:**
- Install packages with pip/conda (everything is already installed)
- Install packages with pip/conda (everything is installed)
- Create new virtual environments
- Use system Python
**Pre-installed packages include:** optuna, numpy, scipy, pandas, matplotlib, pyNastran, torch, plotly, and all Atomizer dependencies.
## Key Files & Locations
## Key Directories
```
Atomizer/
├── .claude/
│ ├── skills/ # Skill instructions (READ THESE)
│ │ ├── create-study.md # Main study creation skill
│ │ └── analyze-workflow.md
│ └── settings.local.json
├── docs/
│ ├── 01_PROTOCOLS.md # Quick protocol reference
│ ├── 06_PROTOCOLS_DETAILED/ # Full protocol docs
│ └── 07_DEVELOPMENT/ # Development plans
├── .claude/skills/ # LLM skills (Bootstrap + Core + Modules)
├── docs/protocols/ # Protocol Operating System
│ ├── operations/ # OP_01 - OP_06
│ ├── system/ # SYS_10 - SYS_14
│ └── extensions/ # EXT_01 - EXT_04
├── optimization_engine/ # Core Python modules
│ ├── runner.py # Main optimizer
│ ├── nx_solver.py # NX interface
│ ├── extractors/ # Result extraction
│ └── validators/ # Config validation
├── studies/ # User studies live here
│ └── {study_name}/
│ ├── 1_setup/ # Config & model files
│ ├── 2_results/ # Optuna DB & outputs
│ └── run_optimization.py
│ └── extractors/ # Physics extraction library
├── studies/ # User studies
└── atomizer-dashboard/ # React dashboard
```
## Conversation Patterns
## CRITICAL: NX Open Development Protocol
### User: "I want to optimize this bracket"
1. Ask about model location, goals, constraints
2. Load skill: `.claude/skills/create-study.md`
3. Follow the interactive discovery process
4. Generate files, validate, confirm
### Always Use Official Documentation First
### User: "Run 200 trials with neural network"
1. Check if surrogate_settings needed
2. Modify config to enable NN
3. Explain the hybrid workflow stages
4. Start run, show monitoring options
**For ANY development involving NX, NX Open, or Siemens APIs:**
### User: "What's the status?"
1. Query database for trial counts
2. Check for running background processes
3. Summarize progress and best results
4. Suggest next steps
1. **FIRST** - Query the MCP Siemens docs tools:
- `mcp__siemens-docs__nxopen_get_class` - Get class documentation
- `mcp__siemens-docs__nxopen_get_index` - Browse class/function indexes
- `mcp__siemens-docs__siemens_docs_list` - List available resources
### User: "The optimization failed"
1. Read error logs
2. Check common failure modes
3. Suggest fixes
4. Offer to retry
2. **THEN** - Use secondary sources if needed:
- PyNastran documentation (for BDF/OP2 parsing)
- NXOpen TSE examples in `nx_journals/`
- Existing extractors in `optimization_engine/extractors/`
## Protocols Reference
3. **NEVER** - Guess NX Open API calls without checking documentation first
| Protocol | Use Case | Sampler |
|----------|----------|---------|
| Protocol 10 | Single objective + constraints | TPE/CMA-ES |
| Protocol 11 | Multi-objective (2-3 goals) | NSGA-II |
| Protocol 12 | Hybrid FEA/NN acceleration | NSGA-II + surrogate |
## Result Extraction
Use centralized extractors from `optimization_engine/extractors/`:
| Need | Extractor | Example |
|------|-----------|---------|
| Displacement | `extract_displacement` | Max tip deflection |
| Stress | `extract_solid_stress` | Max von Mises |
| Frequency | `extract_frequency` | 1st natural freq |
| Mass | `extract_mass_from_expression` | CAD mass property |
## Multi-Solution Detection
If user needs BOTH:
- Static results (stress, displacement)
- Modal results (frequency)
Then set `solution_name=None` to solve ALL solutions.
## Validation Before Action
Always validate before:
- Starting optimization (config validator)
- Generating files (check paths exist)
- Running FEA (check NX files present)
## Dashboard Integration
- Setup/Config: **You handle it**
- Real-time monitoring: **Dashboard at localhost:3000**
- Results analysis: **Both (you interpret, dashboard visualizes)**
## CRITICAL: Code Reuse Protocol (MUST FOLLOW)
### STOP! Before Writing ANY Code in run_optimization.py
**This is the #1 cause of code duplication. EVERY TIME you're about to write:**
- A function longer than 20 lines
- Any physics/math calculations (Zernike, RMS, stress, etc.)
- Any OP2/BDF parsing logic
- Any post-processing or extraction logic
**STOP and run this checklist:**
**Available NX Open Classes (quick lookup):**
| Class | Page ID | Description |
|-------|---------|-------------|
| Session | a03318.html | Main NX session object |
| Part | a02434.html | Part file operations |
| BasePart | a00266.html | Base class for parts |
| CaeSession | a10510.html | CAE/FEM session |
| PdmSession | a50542.html | PDM integration |
**Example workflow for NX journal development:**
```
□ Did I check optimization_engine/extractors/__init__.py?
□ Did I grep for similar function names in optimization_engine/?
□ Does this functionality exist somewhere else in the codebase?
1. User: "Extract mass from NX part"
2. Claude: Query nxopen_get_class("Part") to find mass-related methods
3. Claude: Query nxopen_get_class("Session") to understand part access
4. Claude: Check existing extractors for similar functionality
5. Claude: Write code using verified API calls
```
**MCP Server Setup:** See `mcp-server/README.md`
## CRITICAL: Code Reuse Protocol
### The 20-Line Rule
If you're writing a function longer than ~20 lines in `studies/*/run_optimization.py`:
If you're writing a function longer than ~20 lines in `run_optimization.py`:
1. **STOP** - This is a code smell
2. **SEARCH** - The functionality probably exists
3. **IMPORT** - Use the existing module
4. **Only if truly new** - Create in `optimization_engine/extractors/`, NOT in the study
2. **SEARCH** - Check `optimization_engine/extractors/`
3. **IMPORT** - Use existing extractor
4. **Only if truly new** - Follow EXT_01 to create new extractor
### Available Extractors (ALWAYS CHECK FIRST)
### Available Extractors
| Module | Functions | Use For |
|--------|-----------|---------|
| **`extract_zernike.py`** | `ZernikeExtractor`, `extract_zernike_from_op2`, `extract_zernike_filtered_rms`, `extract_zernike_relative_rms` | Telescope mirror WFE analysis - Noll indexing, RMS calculations, multi-subcase |
| **`zernike_helpers.py`** | `create_zernike_objective`, `ZernikeObjectiveBuilder`, `extract_zernike_for_trial` | Zernike optimization integration |
| **`extract_displacement.py`** | `extract_displacement` | Max/min displacement from OP2 |
| **`extract_von_mises_stress.py`** | `extract_solid_stress` | Von Mises stress extraction |
| **`extract_frequency.py`** | `extract_frequency` | Natural frequencies from OP2 |
| **`extract_mass.py`** | `extract_mass_from_expression` | CAD mass property |
| **`op2_extractor.py`** | Generic OP2 result extraction | Low-level OP2 access |
| **`field_data_extractor.py`** | Field data for neural networks | Training data generation |
| ID | Physics | Function |
|----|---------|----------|
| E1 | Displacement | `extract_displacement()` |
| E2 | Frequency | `extract_frequency()` |
| E3 | Stress | `extract_solid_stress()` |
| E4 | BDF Mass | `extract_mass_from_bdf()` |
| E5 | CAD Mass | `extract_mass_from_expression()` |
| E8-10 | Zernike | `extract_zernike_*()` |
### Correct Pattern: Zernike Example
**Full catalog**: `docs/protocols/system/SYS_12_EXTRACTOR_LIBRARY.md`
**❌ WRONG - What I did (and must NEVER do again):**
```python
# studies/m1_mirror/run_optimization.py
def noll_indices(j): # 30 lines
...
def zernike_radial(n, m, r): # 20 lines
...
def compute_zernike_coefficients(...): # 80 lines
...
def compute_rms_metrics(...): # 40 lines
...
# Total: 500+ lines of duplicated code
```
## Privilege Levels
**✅ CORRECT - What I should have done:**
```python
# studies/m1_mirror/run_optimization.py
from optimization_engine.extractors import (
ZernikeExtractor,
extract_zernike_for_trial
)
| Level | Operations | Extensions |
|-------|------------|------------|
| **user** | All OP_* | None |
| **power_user** | All OP_* | EXT_01, EXT_02 |
| **admin** | All | All |
# In objective function - 5 lines instead of 500
extractor = ZernikeExtractor(op2_file, bdf_file)
result = extractor.extract_relative(target_subcase="40", reference_subcase="20")
filtered_rms = result['relative_filtered_rms_nm']
```
### Creating New Extractors (Only When Truly Needed)
When functionality genuinely doesn't exist:
```
1. CREATE module in optimization_engine/extractors/new_feature.py
2. ADD exports to optimization_engine/extractors/__init__.py
3. UPDATE this table in CLAUDE.md
4. IMPORT in run_optimization.py (just the import, not the implementation)
```
### Why This Is Critical
| Embedding Code in Studies | Using Central Extractors |
|---------------------------|-------------------------|
| Bug fixes don't propagate | Fix once, applies everywhere |
| No unit tests | Tested in isolation |
| Hard to discover | Clear API in __init__.py |
| Copy-paste errors | Single source of truth |
| 500+ line studies | Clean, readable studies |
Default to `user` unless explicitly stated otherwise.
## Key Principles
1. **Conversation first** - Don't ask user to edit JSON manually
2. **Validate everything** - Catch errors before they cause failures
3. **Explain decisions** - Say why you chose a sampler/protocol
4. **Sensible defaults** - User only specifies what they care about
5. **Progressive disclosure** - Start simple, add complexity when needed
6. **NEVER modify master files** - Always copy model files to study working directory before optimization. User's source files must remain untouched. If corruption occurs during iteration, working copy can be deleted and re-copied.
7. **ALWAYS reuse existing code** - Check `optimization_engine/extractors/` BEFORE writing any new post-processing logic. Never duplicate functionality that already exists.
4. **NEVER modify master files** - Copy NX files to study directory
5. **ALWAYS reuse code** - Check extractors before writing new code
## Current State Awareness
## CRITICAL: NX FEM Mesh Update Requirements
Check these before suggesting actions:
- Running background processes: `/tasks` command
- Study databases: `studies/*/2_results/study.db`
- Model files: `studies/*/1_setup/model/`
- Dashboard status: Check if servers running
**When parametric optimization produces identical results, the mesh is NOT updating!**
### Required File Chain
```
.sim (Simulation)
└── .fem (FEM)
└── *_i.prt (Idealized Part) ← MUST EXIST AND BE LOADED!
└── .prt (Geometry Part)
```
### The Fix (Already Implemented in solve_simulation.py)
The idealized part (`*_i.prt`) MUST be explicitly loaded BEFORE calling `UpdateFemodel()`:
```python
# STEP 2: Load idealized part first (CRITICAL!)
for filename in os.listdir(working_dir):
if '_i.prt' in filename.lower():
idealized_part, status = theSession.Parts.Open(path)
break
# THEN update FEM - now it will actually regenerate the mesh
feModel.UpdateFemodel()
```
**Without loading the `_i.prt`, `UpdateFemodel()` runs but the mesh doesn't change!**
### Study Setup Checklist
When creating a new study, ensure ALL these files are copied:
- [ ] `Model.prt` - Geometry part
- [ ] `Model_fem1_i.prt` - Idealized part ← **OFTEN MISSING!**
- [ ] `Model_fem1.fem` - FEM file
- [ ] `Model_sim1.sim` - Simulation file
See `docs/protocols/operations/OP_06_TROUBLESHOOT.md` for full troubleshooting guide.
## Developer Documentation
**For developers maintaining Atomizer**:
- Read `.claude/skills/DEV_DOCUMENTATION.md`
- Use self-documenting commands: "Document the {feature} I added"
- Commit code + docs together
## When Uncertain
1. Read the relevant skill file
2. Check docs/06_PROTOCOLS_DETAILED/
3. Look at existing similar studies
1. Check `.claude/skills/00_BOOTSTRAP.md` for task routing
2. Check `.claude/skills/01_CHEATSHEET.md` for quick lookup
3. Load relevant protocol from `docs/protocols/`
4. Ask user for clarification
---