Atomizer/docs/PHASE_2_7_LLM_INTEGRATION.md

# Phase 2.7: LLM-Powered Workflow Intelligence

## Problem: Static Regex vs. Dynamic Intelligence

**Previous Approach (Phase 2.5-2.6):**
- ❌ Dumb regex patterns to extract workflow steps
- ❌ Static rules for step classification
- ❌ Missed intermediate calculations
- ❌ Couldn't understand nuance (CBUSH vs CBAR, element forces vs reaction forces)

**New Approach (Phase 2.7):**
- ✅ **Use Claude LLM to analyze user requests**
- ✅ **Understand engineering context dynamically**
- ✅ **Detect ALL intermediate steps intelligently**
- ✅ **Distinguish subtle differences (element types, directions, metrics)**

## Architecture

```
User Request
     ↓
LLM Analyzer (Claude)
     ↓
Structured JSON Analysis
     ↓
┌────────────────────────────────────┐
│ Engineering Features (FEA)         │
│ Inline Calculations (Math)         │
│ Post-Processing Hooks (Custom)     │
│ Optimization Config                │
└────────────────────────────────────┘
     ↓
Phase 2.5 Capability Matching
     ↓
Research Plan / Code Generation
```

## Example: CBAR Optimization Request

**User Input:**
```
I want to extract forces in direction Z of all the 1D elements and find the average of it,
then find the minimum value and compare it to the average, then assign it to a objective
metric that needs to be minimized.

I want to iterate on the FEA properties of the Cbar element stiffness in X to make the
objective function minimized.

I want to use genetic algorithm to iterate and optimize this
```

**LLM Analysis Output:**
```json
{
  "engineering_features": [
    {
      "action": "extract_1d_element_forces",
      "domain": "result_extraction",
      "description": "Extract element forces from CBAR in Z direction from OP2",
      "params": {
        "element_types": ["CBAR"],
        "result_type": "element_force",
        "direction": "Z"
      }
    },
    {
      "action": "update_cbar_stiffness",
      "domain": "fea_properties",
      "description": "Modify CBAR stiffness in X direction",
      "params": {
        "element_type": "CBAR",
        "property": "stiffness_x"
      }
    }
  ],
  "inline_calculations": [
    {
      "action": "calculate_average",
      "params": {"input": "forces_z", "operation": "mean"},
      "code_hint": "avg = sum(forces_z) / len(forces_z)"
    },
    {
      "action": "find_minimum",
      "params": {"input": "forces_z", "operation": "min"},
      "code_hint": "min_val = min(forces_z)"
    }
  ],
  "post_processing_hooks": [
    {
      "action": "custom_objective_metric",
      "description": "Compare min to average",
      "params": {
        "inputs": ["min_force", "avg_force"],
        "formula": "min_force / avg_force",
        "objective": "minimize"
      }
    }
  ],
  "optimization": {
    "algorithm": "genetic_algorithm",
    "design_variables": [
      {"parameter": "cbar_stiffness_x", "type": "FEA_property"}
    ]
  }
}
```

## Key Intelligence Improvements

### 1. Detects Intermediate Steps
**Old (Regex):**
- ❌ Only saw "extract forces" and "optimize"
- ❌ Missed average, minimum, comparison

**New (LLM):**
- ✅ Identifies: extract → average → min → compare → optimize
- ✅ Classifies each as engineering vs. simple math

### 2. Understands Engineering Context
**Old (Regex):**
- ❌ "forces" → generic "reaction_force" extraction
- ❌ Didn't distinguish CBUSH from CBAR

**New (LLM):**
- ✅ "1D element forces" → element forces (not reaction forces)
- ✅ "CBAR stiffness in X" → specific property in specific direction
- ✅ Understands these come from different sources (OP2 vs property cards)

### 3. Smart Classification
**Old (Regex):**
```python
if 'average' in text:
    return 'simple_calculation'  # Dumb!
```

**New (LLM):**
```python
# LLM reasoning:
# - "average of forces" → simple Python (sum/len)
# - "extract forces from OP2" → engineering (pyNastran)
# - "compare min to avg for objective" → hook (custom logic)
```

### 4. Generates Actionable Code Hints
**Old:** Just action names like "calculate_average"

**New:** Includes code hints for auto-generation:
```json
{
  "action": "calculate_average",
  "code_hint": "avg = sum(forces_z) / len(forces_z)"
}
```

## Integration with Existing Phases

### Phase 2.5 (Capability Matching)
LLM output feeds directly into existing capability matcher:
- Engineering features → check if implemented
- If missing → create research plan
- If similar → adapt existing code

### Phase 2.6 (Step Classification)
Now **replaced by LLM** for better accuracy:
- No more static rules
- Context-aware classification
- Understands subtle differences

## Implementation

**File:** `optimization_engine/llm_workflow_analyzer.py`

**Key Function:**
```python
analyzer = LLMWorkflowAnalyzer(api_key=os.getenv('ANTHROPIC_API_KEY'))
analysis = analyzer.analyze_request(user_request)

# Returns structured JSON with:
# - engineering_features
# - inline_calculations
# - post_processing_hooks
# - optimization config
```

## Benefits

1. **Accurate**: Understands engineering nuance
2. **Complete**: Detects ALL steps, including intermediate ones
3. **Dynamic**: No hardcoded patterns to maintain
4. **Extensible**: Automatically handles new request types
5. **Actionable**: Provides code hints for auto-generation

## LLM Integration Modes

### Development Mode (Recommended)
For development within Claude Code:
- Use Claude Code directly for interactive workflow analysis
- No API consumption or costs
- Real-time feedback and iteration
- Perfect for testing and refinement

### Production Mode (Future)
For standalone Atomizer execution:
- Optional Anthropic API integration
- Set `ANTHROPIC_API_KEY` environment variable
- Falls back to heuristics if no key provided
- Useful for automated batch processing

**Current Status**: llm_workflow_analyzer.py supports both modes. For development, continue using Claude Code interactively.

## Next Steps

1. ✅ Install anthropic package
2. ✅ Create LLM analyzer module
3. ✅ Document integration modes
4. ⏳ Integrate with Phase 2.5 capability matcher
5. ⏳ Test with diverse optimization requests via Claude Code
6. ⏳ Build code generator for inline calculations
7. ⏳ Build hook generator for post-processing

## Success Criteria

**Input:**
"Extract 1D forces, find average, find minimum, compare to average, optimize CBAR stiffness"

**Output:**
```
Engineering Features: 2 (need research)
  - extract_1d_element_forces
  - update_cbar_stiffness

Inline Calculations: 2 (auto-generate)
  - calculate_average
  - find_minimum

Post-Processing: 1 (generate hook)
  - custom_objective_metric (min/avg ratio)

Optimization: 1
  - genetic_algorithm

✅ All steps detected
✅ Correctly classified
✅ Ready for implementation
```