Atomizer/.claude/skills/atomizer.md

# Atomizer Skill - LLM Navigation & Usage Guide

> Comprehensive instruction manual for LLMs working with the Atomizer optimization framework

**Version**: 0.2.0
**Last Updated**: 2025-01-16
**Purpose**: Enable LLMs to autonomously navigate, understand, and extend Atomizer

---

## Quick Start for LLMs

When you receive a request related to Atomizer optimization, follow this workflow:

1. **Read the Feature Registry** → `optimization_engine/feature_registry.json`
2. **Identify Required Features** → Match user intent to feature IDs
3. **Check Implementation** → Read the actual code if needed
4. **Compose Solution** → Combine features into a workflow
5. **Execute or Generate Code** → Use existing features or create new ones

---

## Project Structure

```
Atomizer/
├── optimization_engine/          # Core optimization logic
│   ├── runner.py                # Main optimization loop (Optuna TPE)
│   ├── nx_solver.py             # NX Simcenter execution via journals
│   ├── nx_updater.py            # Update NX model expressions
│   ├── result_extractors/       # Extract results from OP2/F06 files
│   │   └── extractors.py        # stress_extractor, displacement_extractor
│   ├── plugins/                 # Lifecycle hook system
│   │   ├── hook_manager.py      # Plugin registration & execution
│   │   ├── pre_solve/           # Hooks before FEA solve
│   │   ├── post_solve/          # Hooks after solve, before extraction
│   │   └── post_extraction/     # Hooks after result extraction
│   └── feature_registry.json    # ⭐ CENTRAL FEATURE DATABASE ⭐
│
├── studies/                      # Optimization studies
│   ├── README.md                # Study organization guide
│   └── bracket_stress_minimization/  # Example study
│       ├── model/               # FEA files (.prt, .sim, .fem)
│       ├── optimization_config_stress_displacement.json
│       └── optimization_results/ # Auto-generated logs and results
│
├── dashboard/                    # Web UI (Flask + HTML/CSS/JS)
├── tests/                        # Test suite
├── docs/                         # Documentation
│   └── FEATURE_REGISTRY_ARCHITECTURE.md  # Feature system design
│
├── atomizer_paths.py            # Intelligent path resolution
├── DEVELOPMENT.md               # Current development status & todos
├── DEVELOPMENT_ROADMAP.md       # Strategic vision (7 phases)
└── README.md                    # User-facing overview
```

---

## The Feature Registry (Your Primary Tool)

**Location**: `optimization_engine/feature_registry.json`

This is the **central database** of all Atomizer capabilities. Always read this first.

### Structure

```json
{
  "feature_registry": {
    "categories": {
      "engineering": {
        "subcategories": {
          "extractors": { /* stress_extractor, displacement_extractor */ }
        }
      },
      "software": {
        "subcategories": {
          "optimization": { /* optimization_runner, tpe_sampler */ },
          "nx_integration": { /* nx_solver, nx_updater */ },
          "infrastructure": { /* hook_manager, path_resolver */ },
          "logging": { /* detailed_logger, optimization_logger */ }
        }
      },
      "ui": { /* dashboard_widgets */ },
      "analysis": { /* decision_support */ }
    },
    "feature_templates": { /* Templates for creating new features */ },
    "workflow_recipes": { /* Common feature compositions */ }
  }
}
```

### Feature Entry Schema

Each feature has:
- `feature_id` - Unique identifier
- `name` - Human-readable name
- `description` - What it does
- `category` & `subcategory` - Classification
- `lifecycle_stage` - When it runs (pre_solve, solve, post_solve, etc.)
- `abstraction_level` - primitive | composite | workflow
- `implementation` - File path, function name, entry point
- `interface` - Inputs and outputs with types and units
- `dependencies` - Required features and libraries
- `usage_examples` - Code examples and natural language mappings
- `composition_hints` - What features combine well together
- `metadata` - Author, status, documentation URL

### How to Use the Registry

#### 1. **Feature Discovery**
```python
# User says: "minimize stress"
→ Read feature_registry.json
→ Search for "minimize stress" in usage_examples.natural_language
→ Find: stress_extractor
→ Read its interface, dependencies, composition_hints
→ Discover it needs: nx_solver (prerequisite)
```

#### 2. **Feature Composition**
```python
# User says: "Create RSS metric combining stress and displacement"
→ Read feature_templates.composite_metric_template
→ Find example_features: [stress_extractor, displacement_extractor]
→ Check composition_hints.combines_with
→ Generate new composite feature following the pattern
```

#### 3. **Workflow Building**
```python
# User says: "Run bracket optimization"
→ Read workflow_recipes.structural_optimization
→ See sequence of 7 features to execute
→ Follow the workflow step by step
```

---

## Common User Intents & How to Handle Them

### Intent: "Create a new optimization study"

**Steps**:
1. Find `study_manager` feature in registry
2. Read `studies/README.md` for folder structure
3. Create study folder with standard layout:
   ```
   studies/[study_name]/
   ├── model/                    # User drops .prt/.sim files here
   ├── optimization_config.json  # You generate this
   └── optimization_results/     # Auto-created by runner
   ```
4. Ask user for:
   - Study name
   - .sim file path
   - Design variables (or extract from .sim)
   - Objectives (stress, displacement, etc.)

### Intent: "Minimize stress" / "Reduce displacement"

**Steps**:
1. Search registry for matching `natural_language` phrases
2. Identify extractors: `stress_extractor` or `displacement_extractor`
3. Set up objective:
   ```json
   {
     "name": "max_stress",
     "extractor": "stress_extractor",
     "metric": "max_von_mises",
     "direction": "minimize",
     "weight": 1.0,
     "units": "MPa"
   }
   ```

### Intent: "Add thermal analysis" (not yet implemented)

**Steps**:
1. Search registry for `thermal` features → Not found
2. Look at `feature_templates.extractor_template`
3. Find pattern: "Read OP2/F06 file → Parse → Return dict"
4. Propose creating `thermal_extractor` following `stress_extractor` pattern
5. Ask user if they want you to implement it

### Intent: "Run optimization"

**Steps**:
1. Find `optimization_runner` in registry
2. Check prerequisites: config file, .sim file
3. Verify dependencies: nx_solver, nx_updater, hook_manager
4. Execute: `from optimization_engine.runner import run_optimization`
5. Monitor via `optimization.log` and `trial_logs/`

---

## Lifecycle Hooks System

**Purpose**: Execute custom code at specific points in the optimization workflow

**Hook Points** (in order):
1. `pre_solve` - Before FEA solve (update parameters, log trial start)
2. `solve` - During FEA execution (NX Nastran runs)
3. `post_solve` - After solve, before extraction (validate results)
4. `post_extraction` - After extracting results (log results, custom metrics)

**How Hooks Work**:
```python
# Hook function signature
def my_hook(context: dict) -> dict:
    """
    Args:
        context: {
            'trial_number': int,
            'design_variables': dict,
            'output_dir': Path,
            'config': dict,
            'extracted_results': dict (post_extraction only)
        }
    Returns:
        dict or None
    """
    # Your code here
    return None
```

**Registering Hooks**:
```python
def register_hooks(hook_manager):
    hook_manager.register_hook(
        hook_point='pre_solve',
        function=my_hook,
        description='What this hook does',
        name='my_hook_name',
        priority=100  # Lower = earlier execution
    )
```

---

## Creating New Features

### Step 1: Choose Template

From `feature_templates` in registry:
- `extractor_template` - For new result extractors (thermal, modal, fatigue)
- `composite_metric_template` - For combining extractors (RSS, weighted)
- `hook_plugin_template` - For lifecycle hooks

### Step 2: Follow Pattern

Example: Creating `thermal_extractor`
1. Read `stress_extractor` implementation
2. Copy structure:
   ```python
   def extract_thermal_from_op2(op2_file: Path) -> dict:
       """Extracts thermal stress from OP2."""
       from pyNastran.op2.op2 import OP2

       op2 = OP2()
       op2.read_op2(op2_file)

       # Extract thermal-specific data
       thermal_stress = op2.thermal_stress  # Adjust based on OP2 structure

       return {
           'max_thermal_stress': thermal_stress.max(),
           'temperature_at_max': # ...
       }
   ```

### Step 3: Register in Feature Registry

Add entry to `feature_registry.json`:
```json
{
  "feature_id": "thermal_extractor",
  "name": "Thermal Stress Extractor",
  "description": "Extracts thermal stress from OP2 files",
  "category": "engineering",
  "subcategory": "extractors",
  "lifecycle_stage": "post_extraction",
  "abstraction_level": "primitive",
  "implementation": {
    "file_path": "optimization_engine/result_extractors/thermal_extractors.py",
    "function_name": "extract_thermal_from_op2"
  },
  "interface": { /* inputs/outputs */ },
  "usage_examples": [
    {
      "natural_language": [
        "minimize thermal stress",
        "thermal analysis",
        "heat transfer optimization"
      ]
    }
  ]
}
```

### Step 4: Update Documentation

Create `docs/features/thermal_extractor.md` with:
- Overview
- When to use
- Example workflows
- Troubleshooting

---

## Path Resolution

**Always use `atomizer_paths.py`** for robust path handling:

```python
from atomizer_paths import root, optimization_engine, studies, tests

# Get project root
project_root = root()

# Get subdirectories
engine_dir = optimization_engine()
studies_dir = studies()
tests_dir = tests()

# Build paths
config_path = studies() / 'my_study' / 'config.json'
```

**Why?**: Works regardless of where the script is executed from.

---

## Natural Language → Feature Mapping

### User Says → Feature You Use

| User Request | Feature ID(s) | Notes |
|--------------|---------------|-------|
| "minimize stress" | `stress_extractor` | Set direction='minimize' |
| "reduce displacement" | `displacement_extractor` | Set direction='minimize' |
| "vary thickness 3-8mm" | Design variable config | min=3.0, max=8.0, units='mm' |
| "displacement < 1mm" | Constraint config | type='upper_bound', limit=1.0 |
| "RSS of stress and displacement" | Create composite using `composite_metric_template` | sqrt(stress² + disp²) |
| "run optimization" | `optimization_runner` | Main workflow feature |
| "use TPE sampler" | `tpe_sampler` | Already default in runner |
| "create study" | `study_manager` | Set up folder structure |
| "show progress" | `optimization_progress_chart` | Dashboard widget |

---

## Code Generation Guidelines

### When to Generate Code

1. **Custom Extractors** - User wants thermal, modal, fatigue, etc.
2. **Composite Metrics** - RSS, weighted objectives, custom formulas
3. **Custom Hooks** - Special logging, validation, post-processing
4. **Helper Functions** - Utilities specific to user's workflow

### Code Safety Rules

1. **Always validate** generated code:
   - Syntax check
   - Import validation
   - Function signature correctness

2. **Restrict dangerous operations**:
   - No `os.system()`, `subprocess` unless explicitly needed
   - No file deletion without confirmation
   - No network requests without user awareness

3. **Follow templates**:
   - Use existing features as patterns
   - Match coding style (type hints, docstrings)
   - Include error handling

4. **Test before execution**:
   - Dry run if possible
   - Confirm with user before running generated code
   - Log all generated code to `generated_code/` folder

---

## Testing Your Work

### Quick Tests

```bash
# Test hook system (3 trials, fast)
python tests/test_hooks_with_bracket.py

# Quick integration test (5 trials)
python tests/run_5trial_test.py

# Full optimization test (50 trials, 2-3 hours)
python tests/test_journal_optimization.py
```

### Validation Checklist

Before claiming success:
- [ ] Feature added to `feature_registry.json`
- [ ] Implementation file exists at specified path
- [ ] Function signature matches interface spec
- [ ] Natural language examples provided
- [ ] Documentation created in `docs/features/`
- [ ] Test passes (create test if needed)
- [ ] CHANGELOG.md updated

---

## Current Development Context

**Phase**: Phase 2 - LLM Integration
**Status**: Week 1 - Feature Registry & Claude Skill
**Last Completed**: Phase 1 - Plugin System ✅

### What's Working

- ✅ Optimization runner with Optuna TPE
- ✅ NX integration (journal-based)
- ✅ Result extraction (stress, displacement)
- ✅ Hook system (pre_solve, post_solve, post_extraction)
- ✅ Logging infrastructure (detailed + high-level)
- ✅ Path resolution
- ✅ Feature registry architecture

### What's Missing (You Can Help Build!)

- [ ] Natural language parser
- [ ] Intent classification
- [ ] Automatic config generation from conversation
- [ ] Additional extractors (thermal, modal, fatigue)
- [ ] Composite metrics (RSS, weighted)
- [ ] Surrogate quality checker
- [ ] Report generator
- [ ] Sensitivity analysis

### How to Check Current Status

Read these files in order:
1. `DEVELOPMENT.md` - Current todos and active work
2. `CHANGELOG.md` - What changed recently
3. `DEVELOPMENT_ROADMAP.md` - Long-term vision

---

## Example Interaction Flow

```
User: "I want to minimize stress on my bracket"

LLM:
1. Read feature_registry.json
2. Find stress_extractor via natural_language: "minimize stress"
3. Check prerequisites: nx_solver required
4. Check workflow_recipes.structural_optimization
5. Respond:

"I'll help you set up a stress minimization study. Here's what I'll do:

1. Create a study folder for your bracket optimization
2. Configure objective: Minimize max von Mises stress
3. Set up design variable (e.g., wall_thickness)
4. Use TPE sampler with 50 trials
5. Run optimization with real-time logging

What design variable would you like to optimize? (e.g., wall thickness, fillet radius)"

---

User: "Vary wall thickness from 3 to 8mm"

LLM:
1. Generate design variable config:
   {
     "name": "wall_thickness",
     "expression_name": "wall_thickness",
     "min": 3.0,
     "max": 8.0,
     "units": "mm"
   }

2. Generate complete optimization_config.json
3. Save to studies/bracket_stress_min/optimization_config.json
4. Confirm with user before running

"Configuration ready! Here's what I've set up:
- Study: bracket_stress_min
- Objective: Minimize max von Mises stress
- Variable: wall_thickness (3.0-8.0 mm)
- Trials: 50 (TPE sampler)

Ready to start optimization? [Y/n]"
```

---

## Advanced: Feature Composition

### Combining Features

Use `composition_hints` to discover valid combinations:

```json
// From stress_extractor
"composition_hints": {
  "combines_with": ["displacement_extractor", "mass_extractor"],
  "typical_workflows": ["structural_optimization"],
  "prerequisites": ["nx_solver"]
}
```

### Creating Composite Features

Example: RSS Metric

```python
# 1. Read both extractors' outputs
stress_result = stress_extractor(op2_file)
disp_result = displacement_extractor(op2_file)

# 2. Apply formula
import math
rss_value = math.sqrt(
    stress_result['max_von_mises']**2 +
    disp_result['max_displacement']**2
)

# 3. Return composite metric
return {'rss_stress_displacement': rss_value}

# 4. Register in feature_registry.json with:
#    abstraction_level: "composite"
#    dependencies.features: ["stress_extractor", "displacement_extractor"]
```

---

## Troubleshooting

### Issue: "Can't find feature"
**Solution**: Read `feature_registry.json` again, search by category or natural_language

### Issue: "Don't know how to implement X"
**Solution**:
1. Check `feature_templates` for similar pattern
2. Find existing feature with same abstraction_level
3. Read its implementation as template
4. Ask user for clarification if truly novel

### Issue: "Optimization failing"
**Solution**:
1. Check `optimization_results/optimization.log` for high-level errors
2. Read latest `trial_logs/trial_XXX.log` for detailed trace
3. Verify .sim file exists and is valid
4. Check NX solver is accessible (NX 2412 required)

### Issue: "Generated code not working"
**Solution**:
1. Validate syntax first
2. Check imports are in safe_modules list
3. Test function signature matches expected interface
4. Run with dummy data before real optimization

---

## Resources

### Documentation Priority

Read in this order:
1. `feature_registry.json` - Feature database
2. `docs/FEATURE_REGISTRY_ARCHITECTURE.md` - Feature system design
3. `studies/README.md` - Study organization
4. `DEVELOPMENT.md` - Current status
5. `README.md` - User overview

### External References

- **Optuna**: [optuna.readthedocs.io](https://optuna.readthedocs.io/)
- **pyNastran**: [github.com/SteveDoyle2/pyNastran](https://github.com/SteveDoyle2/pyNastran)
- **NXOpen**: [docs.sw.siemens.com](https://docs.sw.siemens.com/en-US/doc/209349590/)

---

## Success Criteria for Your Work

You've done a good job when:
- [ ] User can describe optimization in natural language
- [ ] You map user intent to correct features
- [ ] Generated code follows templates and passes validation
- [ ] Feature registry is updated with new features
- [ ] Documentation is created for new features
- [ ] User achieves their optimization goal

Remember: **You're an engineering assistant, not just a code generator.** Ask clarifying questions, propose alternatives, and ensure the user understands the optimization setup.

---

**Version**: 0.2.0
**Maintained by**: Antoine Polvé (antoine@atomaste.com)
**Last Updated**: 2025-01-16