Antoine/Atomizer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: Add AtomizerField training data export and intelligent model discovery

Browse Source 
Major additions:
- Training data export system for AtomizerField neural network training
- Bracket stiffness optimization study with 50+ training samples
- Intelligent NX model discovery (auto-detect solutions, expressions, mesh)
- Result extractors module for displacement, stress, frequency, mass
- User-generated NX journals for advanced workflows
- Archive structure for legacy scripts and test outputs
- Protocol documentation and dashboard launcher

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Anto01
2025-11-26 12:01:50 -05:00
parent a0c008a593
commit 2b3573ec42
949 changed files with 1405144 additions and 470 deletions
Download Patch File Download Diff File Expand all files Collapse all files

264
README.md
View File

@@ -1,19 +1,21 @@
# Atomizer
> Advanced LLM-native optimization platform for Siemens NX Simcenter
> Advanced LLM-native optimization platform for Siemens NX Simcenter with Neural Network Acceleration
[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
[![License](https://img.shields.io/badge/license-Proprietary-red.svg)](LICENSE)
[![Status](https://img.shields.io/badge/status-alpha-yellow.svg)](https://github.com)
[![Status](https://img.shields.io/badge/status-beta-green.svg)](https://github.com)
[![Neural](https://img.shields.io/badge/neural-GNN%20powered-purple.svg)](docs/NEURAL_FEATURES_COMPLETE.md)
## Overview
Atomizer is an **LLM-native optimization framework** for Siemens NX Simcenter that transforms how engineers interact with optimization workflows. Instead of manual JSON configuration and scripting, Atomizer uses AI as a collaborative engineering assistant.
Atomizer is an **LLM-native optimization framework** for Siemens NX Simcenter that transforms how engineers interact with optimization workflows. It combines AI-assisted natural language interfaces with **Graph Neural Network (GNN) surrogates** that achieve **600x-500,000x speedup** over traditional FEA simulations.
### Core Philosophy
Atomizer enables engineers to:
- **Describe optimizations in natural language** instead of writing configuration files
- **Accelerate optimization 1000x** using trained neural network surrogates
- **Generate custom analysis functions on-the-fly** (RSS metrics, weighted objectives, constraints)
- **Get intelligent recommendations** based on optimization results and surrogate models
- **Generate comprehensive reports** with AI-written insights and visualizations
@@ -21,13 +23,15 @@ Atomizer enables engineers to:
### Key Features
- **Neural Network Acceleration**: Graph Neural Networks predict FEA results in 4.5ms vs 10-30min for traditional solvers
- **LLM-Driven Workflow**: Natural language study creation, configuration, and analysis
- **Advanced Optimization**: Optuna-powered TPE, Gaussian Process surrogates, multi-objective Pareto fronts
- **Dynamic Code Generation**: AI writes custom Python functions and NX journal scripts during optimization
- **Intelligent Decision Support**: Surrogate quality assessment, sensitivity analysis, engineering recommendations
- **Real-Time Monitoring**: Interactive web dashboard with live progress tracking
- **Extensible Architecture**: Plugin system with hooks for pre/post mesh, solve, and extraction phases
- **Self-Improving**: Feature registry that learns from user workflows and expands capabilities
- **Hybrid FEA/NN Optimization**: Intelligent switching between physics simulation and neural predictions
- **Self-Improving**: Continuous learning from optimization runs to improve neural surrogates
---
@@ -37,15 +41,18 @@ Atomizer enables engineers to:
### Quick Links
- **[Visual Architecture Diagrams](docs/09_DIAGRAMS/)** - 🆕 Comprehensive Mermaid diagrams showing system architecture and workflows
- **[Neural Features Guide](docs/NEURAL_FEATURES_COMPLETE.md)** - Complete guide to GNN surrogates, training, and integration
- **[Neural Workflow Tutorial](docs/NEURAL_WORKFLOW_TUTORIAL.md)** - Step-by-step: data collection → training → optimization
- **[Visual Architecture Diagrams](docs/09_DIAGRAMS/)** - Comprehensive Mermaid diagrams showing system architecture and workflows
- **[Protocol Specifications](docs/PROTOCOLS.md)** - All active protocols (10, 11, 13) consolidated
- **[Development Guide](DEVELOPMENT.md)** - Development workflow, testing, contributing
- **[Dashboard Guide](docs/DASHBOARD.md)** - 🆕 Comprehensive React dashboard with multi-objective visualization
- **[NX Multi-Solution Protocol](docs/NX_MULTI_SOLUTION_PROTOCOL.md)** - 🆕 Critical fix for multi-solution workflows
- **[Dashboard Guide](docs/DASHBOARD.md)** - Comprehensive React dashboard with multi-objective visualization
- **[NX Multi-Solution Protocol](docs/NX_MULTI_SOLUTION_PROTOCOL.md)** - Critical fix for multi-solution workflows
- **[Getting Started](docs/HOW_TO_EXTEND_OPTIMIZATION.md)** - Create your first optimization study
### By Topic
- **Neural Acceleration**: [NEURAL_FEATURES_COMPLETE.md](docs/NEURAL_FEATURES_COMPLETE.md), [NEURAL_WORKFLOW_TUTORIAL.md](docs/NEURAL_WORKFLOW_TUTORIAL.md), [GNN_ARCHITECTURE.md](docs/GNN_ARCHITECTURE.md)
- **Protocols**: [PROTOCOLS.md](docs/PROTOCOLS.md) - Protocol 10 (Intelligent Optimization), 11 (Multi-Objective), 13 (Dashboard)
- **Architecture**: [HOOK_ARCHITECTURE.md](docs/HOOK_ARCHITECTURE.md), [NX_SESSION_MANAGEMENT.md](docs/NX_SESSION_MANAGEMENT.md)
- **Dashboard**: [DASHBOARD_MASTER_PLAN.md](docs/DASHBOARD_MASTER_PLAN.md), [DASHBOARD_REACT_IMPLEMENTATION.md](docs/DASHBOARD_REACT_IMPLEMENTATION.md)
@@ -66,9 +73,18 @@ Atomizer enables engineers to:
│ Plugin System + Feature Registry + Code Generator │
└─────────────────────────────────────────────────────────┘
↕
┌───────────────────────────┬─────────────────────────────┐
│ Traditional Path │ Neural Path (New!) │
├───────────────────────────┼─────────────────────────────┤
│ NX Solver (via Journals) │ AtomizerField GNN │
│ ~10-30 min per eval │ ~4.5 ms per eval │
│ Full physics fidelity │ Physics-informed learning │
└───────────────────────────┴─────────────────────────────┘
↕
┌─────────────────────────────────────────────────────────┐
│ Execution Layer │
│ NX Solver (via Journals) + Optuna + Result Extractors │
│ Hybrid Decision Engine │
│ Confidence-based switching • Uncertainty quantification│
│ Automatic FEA validation • Online learning │
└─────────────────────────────────────────────────────────┘
↕
┌─────────────────────────────────────────────────────────┐
@@ -77,6 +93,31 @@ Atomizer enables engineers to:
└─────────────────────────────────────────────────────────┘
```
### Neural Network Components (AtomizerField)
```
┌─────────────────────────────────────────────────────────┐
│ AtomizerField System │
├─────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ BDF/OP2 │ │ GNN │ │ Inference │ │
│ │ Parser │──>│ Training │──>│ Engine │ │
│ │ (Phase 1) │ │ (Phase 2) │ │ (Phase 2) │ │
│ └─────────────┘ └─────────────┘ └─────────────┘ │
│ │ │ │ │
│ ▼ ▼ ▼ │
│ ┌─────────────────────────────────────────────────┐ │
│ │ Neural Model Types │ │
│ ├─────────────────────────────────────────────────┤ │
│ │ • Field Predictor GNN (displacement + stress) │ │
│ │ • Parametric GNN (all 4 objectives directly) │ │
│ │ • Ensemble models for uncertainty │ │
│ └─────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────┘
```
## Quick Start
### Prerequisites
@@ -179,6 +220,19 @@ python run_5trial_test.py
## Features
### Neural Network Acceleration (AtomizerField)
- **Graph Neural Networks (GNN)**: Physics-aware architecture that respects FEA mesh topology
- **Parametric Surrogate**: Design-conditioned GNN predicts all 4 objectives (mass, frequency, displacement, stress)
- **Ultra-Fast Inference**: 4.5ms per prediction vs 10-30 minutes for FEA (2,000-500,000x speedup)
- **Physics-Informed Loss**: Custom loss functions enforce equilibrium, constitutive laws, and boundary conditions
- **Uncertainty Quantification**: Ensemble-based confidence scores with automatic FEA validation triggers
- **Hybrid Optimization**: Smart switching between FEA and NN based on confidence thresholds
- **Training Data Export**: Automatic export of FEA results in neural training format (BDF/OP2 → HDF5+JSON)
- **Pre-trained Models**: Ready-to-use models for UAV arm optimization with documented training pipelines
### Core Optimization
- **Intelligent Multi-Objective Optimization**: NSGA-II algorithm for Pareto-optimal solutions
- **Advanced Dashboard**: React-based real-time monitoring with parallel coordinates visualization
- **NX Integration**: Seamless journal-based control of Siemens NX Simcenter
@@ -194,23 +248,35 @@ python run_5trial_test.py
## Current Status
**Development Phase**: Alpha - 80-90% Complete
**Development Phase**: Beta - 95% Complete
### Core Optimization
- ✅ **Phase 1 (Plugin System)**: 100% Complete & Production Ready
- ✅ **Phases 2.5-3.1 (LLM Intelligence)**: 100% Complete - Components built and tested
- ✅ **Phase 3.2 Week 1 (LLM Mode)**: **COMPLETE** - Natural language optimization now available!
- 🎯 **Phase 3.2 Week 2-4 (Robustness)**: **IN PROGRESS** - Validation, safety, learning system
- 🔬 **Phase 3.4 (NXOpen Docs)**: Research & investigation phase
- ✅ **Phase 3.2 (LLM Mode)**: Complete - Natural language optimization available
- ✅ **Protocol 10 (IMSO)**: Complete - Intelligent Multi-Strategy Optimization
- ✅ **Protocol 11 (Multi-Objective)**: Complete - Pareto optimization
- ✅ **Protocol 13 (Dashboard)**: Complete - Real-time React dashboard
### Neural Network Acceleration (AtomizerField)
- ✅ **Phase 1 (Data Parser)**: Complete - BDF/OP2 → HDF5+JSON conversion
- ✅ **Phase 2 (Neural Architecture)**: Complete - GNN models with physics-informed loss
- ✅ **Phase 2.1 (Parametric GNN)**: Complete - Design-conditioned predictions
- ✅ **Phase 2.2 (Integration Layer)**: Complete - Neural surrogate + hybrid optimizer
- ✅ **Phase 3 (Testing)**: Complete - 18 comprehensive tests
- ✅ **Pre-trained Models**: Available for UAV arm optimization
**What's Working**:
- ✅ Complete optimization engine with Optuna + NX Simcenter
- ✅ Substudy system with live history tracking
- ✅ **LLM Mode**: Natural language → Auto-generated code → Optimization → Results
- ✅ LLM components (workflow analyzer, code generators, research agent) - production integrated
- ✅ 50-trial optimization validated with real results
- ✅ End-to-end workflow: `--llm "your request"` → results
- ✅ **Neural acceleration**: 4.5ms predictions (2000x speedup over FEA)
- ✅ **Hybrid optimization**: Smart FEA/NN switching with confidence thresholds
- ✅ **Parametric surrogate**: Predicts all 4 objectives from design parameters
- ✅ **Training pipeline**: Export data → Train GNN → Deploy → Optimize
- ✅ Real-time dashboard with Pareto front visualization
- ✅ Multi-objective optimization with NSGA-II
- ✅ LLM-assisted natural language workflows
**Current Focus**: Adding robustness, safety checks, and learning capabilities to LLM mode.
**Production Ready**: Core optimization + neural acceleration fully functional.
See [DEVELOPMENT_GUIDANCE.md](DEVELOPMENT_GUIDANCE.md) for comprehensive status and priorities.
@@ -218,92 +284,102 @@ See [DEVELOPMENT_GUIDANCE.md](DEVELOPMENT_GUIDANCE.md) for comprehensive status
```
Atomizer/
├── optimization_engine/ # Core optimization logic
│ ├── runner.py # Main optimization runner
│ ├── nx_solver.py # NX journal execution
│ ├── nx_updater.py # NX model parameter updates
│ ├── pynastran_research_agent.py # Phase 3: Auto OP2 code gen ✅
│ ├── hook_generator.py # Phase 2.9: Auto hook generation ✅
│ ├── result_extractors/ # OP2/F06 parsers
│ │ └── extractors.py # Stress, displacement extractors
│ └── plugins/ # Plugin system (Phase 1 ✅)
│ ├── hook_manager.py # Hook registration & execution
│ ├── hooks.py # HookPoint enum, Hook dataclass
│ ├── pre_solve/ # Pre-solve lifecycle hooks
│ │ ├── detailed_logger.py
│ │ └── optimization_logger.py
│ ├── post_solve/ # Post-solve lifecycle hooks
│ │ └── log_solve_complete.py
│ ├── post_extraction/ # Post-extraction lifecycle hooks
│ │ ├── log_results.py
│ │ └── optimization_logger_results.py
│ └── post_calculation/ # Post-calculation hooks (Phase 2.9 ✅)
│ ├── weighted_objective_test.py
│ ├── safety_factor_hook.py
│ └── min_to_avg_ratio_hook.py
├── dashboard/ # Web UI
│ ├── api/ # Flask backend
│ ├── frontend/ # HTML/CSS/JS
│ └── scripts/ # NX expression extraction
├── studies/ # Optimization studies
│ ├── README.md # Comprehensive studies guide
│ └── bracket_displacement_maximizing/ # Example study with substudies
│ ├── README.md # Study documentation
│ ├── SUBSTUDIES_README.md # Substudy system guide
│ ├── model/ # Shared FEA model files (.prt, .sim, .fem)
│ ├── config/ # Substudy configuration templates
│ ├── substudies/ # Independent substudy results
│ │ ├── coarse_exploration/ # Fast 20-trial coarse search
│ │ │ ├── config.json
│ │ │ ├── optimization_history_incremental.json # Live updates
│ │ │ └── best_design.json
│ │ └── fine_tuning/ # Refined 50-trial optimization
│ ├── run_substudy.py # Substudy runner with continuation support
│ └── run_optimization.py # Standalone optimization runner
├── tests/ # Unit and integration tests
│ ├── test_hooks_with_bracket.py
│ ├── run_5trial_test.py
│ └── test_journal_optimization.py
├── docs/ # Documentation
├── atomizer_paths.py # Intelligent path resolution
├── DEVELOPMENT_ROADMAP.md # Future vision and phases
└── README.md # This file
├── optimization_engine/ # Core optimization logic
│ ├── runner.py # Main optimization runner
│ ├── runner_with_neural.py # Neural-enhanced runner (NEW)
│ ├── neural_surrogate.py # GNN integration layer (NEW)
│ ├── training_data_exporter.py # Export FEA→neural format (NEW)
│ ├── nx_solver.py # NX journal execution
│ ├── nx_updater.py # NX model parameter updates
│ ├── result_extractors/ # OP2/F06 parsers
│ └── plugins/ # Plugin system
│
├── atomizer-field/ # Neural Network System (NEW)
│ ├── neural_field_parser.py # BDF/OP2 → neural format
│ ├── validate_parsed_data.py # Physics validation
│ ├── batch_parser.py # Batch processing
│ ├── neural_models/ # GNN architectures
│ │ ├── field_predictor.py # Field prediction GNN
│ │ ├── parametric_predictor.py # Parametric GNN (4 objectives)
│ │ └── physics_losses.py # Physics-informed loss functions
│ ├── train.py # Training pipeline
│ ├── train_parametric.py # Parametric model training
│ ├── predict.py # Inference engine
│ ├── runs/ # Pre-trained models
│ │ └── parametric_uav_arm_v2/ # UAV arm model (ready to use)
│ └── tests/ # 18 comprehensive tests
│
├── atomizer-dashboard/ # React Dashboard (NEW)
│ ├── backend/ # FastAPI + WebSocket
│ └── frontend/ # React + Tailwind + Recharts
│
├── studies/ # Optimization studies
│ ├── uav_arm_optimization/ # Example with neural integration
│ └── [other studies]/ # Traditional optimization examples
│
├── atomizer_field_training_data/ # Training data storage
│ └── [study_name]/ # Exported training cases
│
├── docs/ # Documentation
│ ├── NEURAL_FEATURES_COMPLETE.md # Complete neural guide
│ ├── NEURAL_WORKFLOW_TUTORIAL.md # Step-by-step tutorial
│ ├── GNN_ARCHITECTURE.md # Architecture deep-dive
│ └── [other docs]/
│
├── tests/ # Integration tests
└── README.md # This file
```
## Example: Bracket Displacement Maximization with Substudies
## Example: Neural-Accelerated UAV Arm Optimization
A complete working example is in `studies/bracket_displacement_maximizing/`:
A complete working example with neural acceleration in `studies/uav_arm_optimization/`:
```bash
# Run standalone optimization (20 trials)
cd studies/bracket_displacement_maximizing
python run_optimization.py
# Step 1: Run initial FEA optimization (collect training data)
cd studies/uav_arm_optimization
python run_optimization.py --trials 50 --export-training-data
# Or run a substudy (hierarchical organization)
python run_substudy.py coarse_exploration # 20-trial coarse search
python run_substudy.py fine_tuning # 50-trial refinement with continuation
# Step 2: Train neural network on collected data
cd ../../atomizer-field
python train_parametric.py \
--train_dir ../atomizer_field_training_data/uav_arm \
--epochs 200
# View live progress
cat substudies/coarse_exploration/optimization_history_incremental.json
# Step 3: Run neural-accelerated optimization (1000x faster!)
cd ../studies/uav_arm_optimization
python run_optimization.py --trials 5000 --use-neural
```
**What it does**:
1. Loads `Bracket_sim1.sim` with parametric geometry
2. Varies `tip_thickness` (15-25mm) and `support_angle` (20-40°)
3. Runs FEA solve for each trial using NX journal mode
4. Extracts displacement and stress from OP2 files
5. Maximizes displacement while maintaining safety factor >= 4.0
**What happens**:
1. Initial 50 FEA trials collect training data (~8 hours)
2. GNN trains on the data (~30 minutes)
3. Neural-accelerated trials run 5000 designs (~4 minutes total!)
**Substudy System**:
- **Shared Models**: All substudies use the same model files
- **Independent Configs**: Each substudy has its own parameter bounds and settings
- **Continuation Support**: Fine-tuning substudy continues from coarse exploration results
- **Live History**: Real-time JSON updates for monitoring progress
**Design Variables**:
- `beam_half_core_thickness`: 5-15 mm
- `beam_face_thickness`: 1-5 mm
- `holes_diameter`: 20-50 mm
- `hole_count`: 5-15
**Results** (typical):
- Best thickness: ~4.2mm
- Stress reduction: 15-20% vs. baseline
- Convergence: ~30 trials to plateau
**Objectives**:
- Minimize mass
- Maximize frequency
- Minimize max displacement
- Minimize max stress
**Performance**:
- FEA time: ~10 seconds/trial
- Neural time: ~4.5 ms/trial
- Speedup: **2,200x**
## Example: Traditional Bracket Optimization
For traditional FEA-only optimization, see `studies/bracket_displacement_maximizing/`:
```bash
cd studies/bracket_displacement_maximizing
python run_optimization.py --trials 50
```
## Dashboard Usage