Atomizer/README.md

# Atomizer

> Advanced LLM-native optimization platform for Siemens NX Simcenter with Neural Network Acceleration

[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
[![License](https://img.shields.io/badge/license-Proprietary-red.svg)](LICENSE)
[![Status](https://img.shields.io/badge/status-beta-green.svg)](https://github.com)
[![Neural](https://img.shields.io/badge/neural-GNN%20powered-purple.svg)](docs/NEURAL_FEATURES_COMPLETE.md)

## Overview

Atomizer is an **LLM-native optimization framework** for Siemens NX Simcenter that transforms how engineers interact with optimization workflows. It combines AI-assisted natural language interfaces with **Graph Neural Network (GNN) surrogates** that achieve **600x-500,000x speedup** over traditional FEA simulations.

### Core Philosophy

Atomizer enables engineers to:
- **Describe optimizations in natural language** instead of writing configuration files
- **Accelerate optimization 1000x** using trained neural network surrogates
- **Generate custom analysis functions on-the-fly** (RSS metrics, weighted objectives, constraints)
- **Get intelligent recommendations** based on optimization results and surrogate models
- **Generate comprehensive reports** with AI-written insights and visualizations
- **Extend the framework autonomously** through LLM-driven code generation

### Key Features

- **Neural Network Acceleration**: Graph Neural Networks predict FEA results in 4.5ms vs 10-30min for traditional solvers
- **LLM-Driven Workflow**: Natural language study creation, configuration, and analysis
- **Advanced Optimization**: Optuna-powered TPE, Gaussian Process surrogates, multi-objective Pareto fronts
- **Dynamic Code Generation**: AI writes custom Python functions and NX journal scripts during optimization
- **Intelligent Decision Support**: Surrogate quality assessment, sensitivity analysis, engineering recommendations
- **Real-Time Monitoring**: Interactive web dashboard with live progress tracking
- **Extensible Architecture**: Plugin system with hooks for pre/post mesh, solve, and extraction phases
- **Hybrid FEA/NN Optimization**: Intelligent switching between physics simulation and neural predictions
- **Self-Improving**: Continuous learning from optimization runs to improve neural surrogates

---

## Documentation

📚 **[Complete Documentation Index](docs/00_INDEX.md)** - Start here for all documentation

### Quick Links

- **[Neural Features Guide](docs/NEURAL_FEATURES_COMPLETE.md)** - Complete guide to GNN surrogates, training, and integration
- **[Neural Workflow Tutorial](docs/NEURAL_WORKFLOW_TUTORIAL.md)** - Step-by-step: data collection → training → optimization
- **[Visual Architecture Diagrams](docs/09_DIAGRAMS/)** - Comprehensive Mermaid diagrams showing system architecture and workflows
- **[Protocol Specifications](docs/PROTOCOLS.md)** - All active protocols (10, 11, 13) consolidated
- **[Development Guide](DEVELOPMENT.md)** - Development workflow, testing, contributing
- **[Dashboard Guide](docs/DASHBOARD.md)** - Comprehensive React dashboard with multi-objective visualization
- **[NX Multi-Solution Protocol](docs/NX_MULTI_SOLUTION_PROTOCOL.md)** - Critical fix for multi-solution workflows
- **[Getting Started](docs/HOW_TO_EXTEND_OPTIMIZATION.md)** - Create your first optimization study

### By Topic

- **Neural Acceleration**: [NEURAL_FEATURES_COMPLETE.md](docs/NEURAL_FEATURES_COMPLETE.md), [NEURAL_WORKFLOW_TUTORIAL.md](docs/NEURAL_WORKFLOW_TUTORIAL.md), [GNN_ARCHITECTURE.md](docs/GNN_ARCHITECTURE.md)
- **Protocols**: [PROTOCOLS.md](docs/PROTOCOLS.md) - Protocol 10 (Intelligent Optimization), 11 (Multi-Objective), 13 (Dashboard)
- **Architecture**: [HOOK_ARCHITECTURE.md](docs/HOOK_ARCHITECTURE.md), [NX_SESSION_MANAGEMENT.md](docs/NX_SESSION_MANAGEMENT.md)
- **Dashboard**: [DASHBOARD_MASTER_PLAN.md](docs/DASHBOARD_MASTER_PLAN.md), [DASHBOARD_REACT_IMPLEMENTATION.md](docs/DASHBOARD_REACT_IMPLEMENTATION.md)
- **Advanced**: [HYBRID_MODE_GUIDE.md](docs/HYBRID_MODE_GUIDE.md) - LLM-assisted workflows

---

## Architecture

```
┌─────────────────────────────────────────────────────────┐
│                 LLM Interface Layer                     │
│  Claude Skill + Natural Language Parser + Workflow Mgr  │
└─────────────────────────────────────────────────────────┘
                          ↕
┌─────────────────────────────────────────────────────────┐
│              Optimization Engine Core                   │
│  Plugin System + Feature Registry + Code Generator      │
└─────────────────────────────────────────────────────────┘
                          ↕
┌───────────────────────────┬─────────────────────────────┐
│     Traditional Path      │      Neural Path (New!)     │
├───────────────────────────┼─────────────────────────────┤
│  NX Solver (via Journals) │   AtomizerField GNN         │
│  ~10-30 min per eval      │   ~4.5 ms per eval          │
│  Full physics fidelity    │   Physics-informed learning │
└───────────────────────────┴─────────────────────────────┘
                          ↕
┌─────────────────────────────────────────────────────────┐
│         Hybrid Decision Engine                          │
│  Confidence-based switching • Uncertainty quantification│
│  Automatic FEA validation • Online learning             │
└─────────────────────────────────────────────────────────┘
                          ↕
┌─────────────────────────────────────────────────────────┐
│              Analysis & Reporting                       │
│  Surrogate Quality + Sensitivity + Report Generator     │
└─────────────────────────────────────────────────────────┘
```

### Neural Network Components (AtomizerField)

```
┌─────────────────────────────────────────────────────────┐
│                  AtomizerField System                   │
├─────────────────────────────────────────────────────────┤
│                                                         │
│  ┌─────────────┐   ┌─────────────┐   ┌─────────────┐  │
│  │ BDF/OP2     │   │    GNN      │   │  Inference  │  │
│  │ Parser      │──>│  Training   │──>│   Engine    │  │
│  │ (Phase 1)   │   │  (Phase 2)  │   │  (Phase 2)  │  │
│  └─────────────┘   └─────────────┘   └─────────────┘  │
│        │                 │                  │          │
│        ▼                 ▼                  ▼          │
│  ┌─────────────────────────────────────────────────┐  │
│  │              Neural Model Types                  │  │
│  ├─────────────────────────────────────────────────┤  │
│  │ • Field Predictor GNN (displacement + stress)   │  │
│  │ • Parametric GNN (all 4 objectives directly)    │  │
│  │ • Ensemble models for uncertainty               │  │
│  └─────────────────────────────────────────────────┘  │
│                                                         │
└─────────────────────────────────────────────────────────┘
```

## Quick Start

### Prerequisites

- **Siemens NX 2412** with NX Nastran solver
- **Python 3.10+** (recommend Anaconda)
- **Git** for version control

### Installation

1. **Clone the repository**:
   ```bash
   git clone https://github.com/yourusername/Atomizer.git
   cd Atomizer
   ```

2. **Create Python environment**:
   ```bash
   conda create -n atomizer python=3.10
   conda activate atomizer
   ```

3. **Install dependencies**:
   ```bash
   pip install -r requirements.txt
   ```

4. **Configure NX path** (edit if needed):
   - Default NX path: `C:\Program Files\Siemens\NX2412\NXBIN\run_journal.exe`
   - Update in `optimization_engine/nx_solver.py` if different

### Basic Usage

#### Example 1: Natural Language Optimization (LLM Mode - Available Now!)

**New in Phase 3.2**: Describe your optimization in natural language - no JSON config needed!

```bash
python optimization_engine/run_optimization.py \
  --llm "Minimize displacement and mass while keeping stress below 200 MPa. \
        Design variables: beam_half_core_thickness (15-30 mm), \
        beam_face_thickness (15-30 mm). Run 10 trials using TPE." \
  --prt studies/simple_beam_optimization/1_setup/model/Beam.prt \
  --sim studies/simple_beam_optimization/1_setup/model/Beam_sim1.sim \
  --trials 10
```

**What happens automatically:**
- ✅ LLM parses your natural language request
- ✅ Auto-generates result extractors (displacement, stress, mass)
- ✅ Auto-generates inline calculations (safety factor, RSS objectives)
- ✅ Auto-generates post-processing hooks (plotting, reporting)
- ✅ Runs optimization with Optuna
- ✅ Saves results, plots, and best design

**Example**: See [examples/llm_mode_simple_example.py](examples/llm_mode_simple_example.py) for a complete walkthrough.

**Requirements**: Claude Code integration (no API key needed) or provide `--api-key` for Anthropic API.

#### Example 2: Current JSON Configuration

Create `studies/my_study/config.json`:

```json
{
  "sim_file": "studies/bracket_stress_minimization/model/Bracket_sim1.sim",
  "design_variables": [
    {
      "name": "wall_thickness",
      "expression_name": "wall_thickness",
      "min": 3.0,
      "max": 8.0,
      "units": "mm"
    }
  ],
  "objectives": [
    {
      "name": "max_stress",
      "extractor": "stress_extractor",
      "metric": "max_von_mises",
      "direction": "minimize",
      "weight": 1.0,
      "units": "MPa"
    }
  ],
  "optimization_settings": {
    "n_trials": 50,
    "sampler": "TPE",
    "n_startup_trials": 20
  }
}
```

Run optimization:
```bash
python tests/test_journal_optimization.py
# Or use the quick 5-trial test:
python run_5trial_test.py
```

## Features

### Neural Network Acceleration (AtomizerField)

- **Graph Neural Networks (GNN)**: Physics-aware architecture that respects FEA mesh topology
- **Parametric Surrogate**: Design-conditioned GNN predicts all 4 objectives (mass, frequency, displacement, stress)
- **Ultra-Fast Inference**: 4.5ms per prediction vs 10-30 minutes for FEA (2,000-500,000x speedup)
- **Physics-Informed Loss**: Custom loss functions enforce equilibrium, constitutive laws, and boundary conditions
- **Uncertainty Quantification**: Ensemble-based confidence scores with automatic FEA validation triggers
- **Hybrid Optimization**: Smart switching between FEA and NN based on confidence thresholds
- **Training Data Export**: Automatic export of FEA results in neural training format (BDF/OP2 → HDF5+JSON)
- **Pre-trained Models**: Ready-to-use models for UAV arm optimization with documented training pipelines

### Core Optimization

- **Intelligent Multi-Objective Optimization**: NSGA-II algorithm for Pareto-optimal solutions
- **Advanced Dashboard**: React-based real-time monitoring with parallel coordinates visualization
- **NX Integration**: Seamless journal-based control of Siemens NX Simcenter
- **Multi-Solution Support**: Automatic handling of combined analysis types (static + modal, thermal + structural)
- **Smart Logging**: Detailed per-trial logs + high-level optimization progress tracking
- **Plugin System**: Extensible hooks at pre-solve, post-solve, and post-extraction points
- **Study Management**: Isolated study folders with automatic result organization
- **Substudy System**: NX-like hierarchical studies with shared models and independent configurations
- **Live History Tracking**: Real-time incremental JSON updates for monitoring progress
- **Resume Capability**: Interrupt and resume optimizations without data loss
- **Pareto Front Analysis**: Automatic extraction and visualization of non-dominated solutions
- **Parallel Coordinates Plot**: Research-grade multi-dimensional visualization with interactive selection

## Current Status

**Development Phase**: Beta - 95% Complete

### Core Optimization
- ✅ **Phase 1 (Plugin System)**: 100% Complete & Production Ready
- ✅ **Phases 2.5-3.1 (LLM Intelligence)**: 100% Complete - Components built and tested
- ✅ **Phase 3.2 (LLM Mode)**: Complete - Natural language optimization available
- ✅ **Protocol 10 (IMSO)**: Complete - Intelligent Multi-Strategy Optimization
- ✅ **Protocol 11 (Multi-Objective)**: Complete - Pareto optimization
- ✅ **Protocol 13 (Dashboard)**: Complete - Real-time React dashboard

### Neural Network Acceleration (AtomizerField)
- ✅ **Phase 1 (Data Parser)**: Complete - BDF/OP2 → HDF5+JSON conversion
- ✅ **Phase 2 (Neural Architecture)**: Complete - GNN models with physics-informed loss
- ✅ **Phase 2.1 (Parametric GNN)**: Complete - Design-conditioned predictions
- ✅ **Phase 2.2 (Integration Layer)**: Complete - Neural surrogate + hybrid optimizer
- ✅ **Phase 3 (Testing)**: Complete - 18 comprehensive tests
- ✅ **Pre-trained Models**: Available for UAV arm optimization

**What's Working**:
- ✅ Complete optimization engine with Optuna + NX Simcenter
- ✅ **Neural acceleration**: 4.5ms predictions (2000x speedup over FEA)
- ✅ **Hybrid optimization**: Smart FEA/NN switching with confidence thresholds
- ✅ **Parametric surrogate**: Predicts all 4 objectives from design parameters
- ✅ **Training pipeline**: Export data → Train GNN → Deploy → Optimize
- ✅ Real-time dashboard with Pareto front visualization
- ✅ Multi-objective optimization with NSGA-II
- ✅ LLM-assisted natural language workflows

**Production Ready**: Core optimization + neural acceleration fully functional.

See [DEVELOPMENT_GUIDANCE.md](DEVELOPMENT_GUIDANCE.md) for comprehensive status and priorities.

## Project Structure

```
Atomizer/
├── optimization_engine/           # Core optimization logic
│   ├── runner.py                 # Main optimization runner
│   ├── runner_with_neural.py     # Neural-enhanced runner (NEW)
│   ├── neural_surrogate.py       # GNN integration layer (NEW)
│   ├── training_data_exporter.py # Export FEA→neural format (NEW)
│   ├── nx_solver.py              # NX journal execution
│   ├── nx_updater.py             # NX model parameter updates
│   ├── result_extractors/        # OP2/F06 parsers
│   └── plugins/                  # Plugin system
│
├── atomizer-field/               # Neural Network System (NEW)
│   ├── neural_field_parser.py    # BDF/OP2 → neural format
│   ├── validate_parsed_data.py   # Physics validation
│   ├── batch_parser.py           # Batch processing
│   ├── neural_models/            # GNN architectures
│   │   ├── field_predictor.py    # Field prediction GNN
│   │   ├── parametric_predictor.py # Parametric GNN (4 objectives)
│   │   └── physics_losses.py     # Physics-informed loss functions
│   ├── train.py                  # Training pipeline
│   ├── train_parametric.py       # Parametric model training
│   ├── predict.py                # Inference engine
│   ├── runs/                     # Pre-trained models
│   │   └── parametric_uav_arm_v2/  # UAV arm model (ready to use)
│   └── tests/                    # 18 comprehensive tests
│
├── atomizer-dashboard/           # React Dashboard (NEW)
│   ├── backend/                  # FastAPI + WebSocket
│   └── frontend/                 # React + Tailwind + Recharts
│
├── studies/                      # Optimization studies
│   ├── uav_arm_optimization/     # Example with neural integration
│   └── [other studies]/          # Traditional optimization examples
│
├── atomizer_field_training_data/ # Training data storage
│   └── [study_name]/             # Exported training cases
│
├── docs/                         # Documentation
│   ├── NEURAL_FEATURES_COMPLETE.md  # Complete neural guide
│   ├── NEURAL_WORKFLOW_TUTORIAL.md  # Step-by-step tutorial
│   ├── GNN_ARCHITECTURE.md          # Architecture deep-dive
│   └── [other docs]/
│
├── tests/                        # Integration tests
└── README.md                     # This file
```

## Example: Neural-Accelerated UAV Arm Optimization

A complete working example with neural acceleration in `studies/uav_arm_optimization/`:

```bash
# Step 1: Run initial FEA optimization (collect training data)
cd studies/uav_arm_optimization
python run_optimization.py --trials 50 --export-training-data

# Step 2: Train neural network on collected data
cd ../../atomizer-field
python train_parametric.py \
  --train_dir ../atomizer_field_training_data/uav_arm \
  --epochs 200

# Step 3: Run neural-accelerated optimization (1000x faster!)
cd ../studies/uav_arm_optimization
python run_optimization.py --trials 5000 --use-neural
```

**What happens**:
1. Initial 50 FEA trials collect training data (~8 hours)
2. GNN trains on the data (~30 minutes)
3. Neural-accelerated trials run 5000 designs (~4 minutes total!)

**Design Variables**:
- `beam_half_core_thickness`: 5-15 mm
- `beam_face_thickness`: 1-5 mm
- `holes_diameter`: 20-50 mm
- `hole_count`: 5-15

**Objectives**:
- Minimize mass
- Maximize frequency
- Minimize max displacement
- Minimize max stress

**Performance**:
- FEA time: ~10 seconds/trial
- Neural time: ~4.5 ms/trial
- Speedup: **2,200x**

## Example: Traditional Bracket Optimization

For traditional FEA-only optimization, see `studies/bracket_displacement_maximizing/`:

```bash
cd studies/bracket_displacement_maximizing
python run_optimization.py --trials 50
```

## Dashboard Usage

Start the dashboard:
```bash
python dashboard/start_dashboard.py
```

Features:
- **Create studies** with folder structure (sim/, results/, config.json)
- **Drop .sim/.prt files** into study folders
- **Explore .sim files** to extract expressions via NX
- **Configure optimization** with 5-step wizard:
  1. Simulation files
  2. Design variables
  3. Objectives
  4. Constraints
  5. Optimization settings
- **Monitor progress** with real-time charts
- **View results** with trial history and best parameters

## Vision: LLM-Native Engineering Assistant

Atomizer is evolving into a comprehensive AI-powered engineering platform. See [DEVELOPMENT_ROADMAP.md](DEVELOPMENT_ROADMAP.md) for details on:

- **Phase 1-7 development plan** with timelines and deliverables
- **Example use cases** demonstrating natural language workflows
- **Architecture diagrams** showing plugin system and LLM integration
- **Success metrics** for each phase

### Future Capabilities

```
User: "Add RSS function combining stress and displacement"
→ LLM: Writes Python function, validates, registers as custom objective

User: "Use surrogate to predict these 10 parameter sets"
→ LLM: Checks surrogate R² > 0.9, runs predictions with confidence intervals

User: "Make an optimization report"
→ LLM: Generates HTML with plots, insights, recommendations (30 seconds)

User: "Why did trial #34 perform best?"
→ LLM: "Trial #34 had optimal stress distribution due to thickness 4.2mm
       creating uniform load paths. Fillet radius 3.1mm reduced stress
       concentration by 18%. This combination is Pareto-optimal."
```

## Development Status

### Completed Phases

- [x] **Phase 1**: Core optimization engine & Plugin system ✅
  - NX journal integration
  - Web dashboard
  - Lifecycle hooks (pre-solve, post-solve, post-extraction)

- [x] **Phase 2.5**: Intelligent Codebase-Aware Gap Detection ✅
  - Scans existing capabilities before requesting examples
  - Matches workflow steps to implemented features
  - 80-90% accuracy on complex optimization requests

- [x] **Phase 2.6**: Intelligent Step Classification ✅
  - Distinguishes engineering features from inline calculations
  - Identifies post-processing hooks vs FEA operations
  - Foundation for smart code generation

- [x] **Phase 2.7**: LLM-Powered Workflow Intelligence ✅
  - Replaces static regex with Claude AI analysis
  - Detects ALL intermediate calculation steps
  - Understands engineering context (PCOMP, CBAR, element forces, etc.)
  - 95%+ expected accuracy with full nuance detection

- [x] **Phase 2.8**: Inline Code Generation ✅
  - LLM-generates Python code for simple math operations
  - Handles avg/min/max, normalization, percentage calculations
  - Direct integration with Phase 2.7 LLM output
  - Optional automated code generation for calculations

- [x] **Phase 2.9**: Post-Processing Hook Generation ✅
  - LLM-generates standalone Python middleware scripts
  - Integrated with Phase 1 lifecycle hook system
  - Handles weighted objectives, custom formulas, constraints, comparisons
  - Complete JSON-based I/O for optimization loops
  - Optional automated scripting for post-processing operations

- [x] **Phase 3**: pyNastran Documentation Integration ✅
  - LLM-enhanced OP2 extraction code generation
  - Documentation research via WebFetch
  - 3 core extraction patterns (displacement, stress, force)
  - Knowledge base for learned patterns
  - Successfully tested on real OP2 files
  - Optional automated code generation for result extraction!

- [x] **Phase 3.1**: LLM-Enhanced Automation Pipeline ✅
  - Extractor orchestrator integrates Phase 2.7 + Phase 3.0
  - Optional automatic extractor generation from LLM output
  - Dynamic loading and execution on real OP2 files
  - End-to-end test passed: Request → Code → Execution → Objective
  - LLM-enhanced workflow with user flexibility achieved!

### Next Priorities

- [ ] **Phase 3.2**: Optimization runner integration with orchestrator
- [ ] **Phase 3.5**: NXOpen introspection & pattern curation
- [ ] **Phase 4**: Code generation for complex FEA features
- [ ] **Phase 5**: Analysis & decision support
- [ ] **Phase 6**: Automated reporting

**For Developers**:
- [DEVELOPMENT.md](DEVELOPMENT.md) - Current status, todos, and active development
- [DEVELOPMENT_ROADMAP.md](DEVELOPMENT_ROADMAP.md) - Strategic vision and long-term plan
- [CHANGELOG.md](CHANGELOG.md) - Version history and changes

## License

Proprietary - Atomaste © 2025

## Support

- **Documentation**: [docs/](docs/)
- **Studies**: [studies/](studies/) - Optimization study templates and examples
- **Development Roadmap**: [DEVELOPMENT_ROADMAP.md](DEVELOPMENT_ROADMAP.md)
- **Email**: antoine@atomaste.com

## Resources

### NXOpen References
- **Official API Docs**: [Siemens NXOpen Documentation](https://docs.sw.siemens.com/en-US/doc/209349590/)
- **NXOpenTSE**: [The Scripting Engineer's Guide](https://nxopentsedocumentation.thescriptingengineer.com/)
- **Our Guide**: [NXOpen Resources](docs/NXOPEN_RESOURCES.md)

### Optimization
- **Optuna Documentation**: [optuna.readthedocs.io](https://optuna.readthedocs.io/)
- **pyNastran**: [github.com/SteveDoyle2/pyNastran](https://github.com/SteveDoyle2/pyNastran)

---

**Built with ❤️ by Atomaste** | Powered by Optuna, NXOpen, and Claude