Atomizer/docs/PHASE_2_7_LLM_INTEGRATION.md

Phase 2.7: LLM-Powered Workflow Intelligence

Problem: Static Regex vs. Dynamic Intelligence

Previous Approach (Phase 2.5-2.6):

• ❌ Dumb regex patterns to extract workflow steps
• ❌ Static rules for step classification
• ❌ Missed intermediate calculations
• ❌ Couldn't understand nuance (CBUSH vs CBAR, element forces vs reaction forces)

New Approach (Phase 2.7):

• ✅ Use Claude LLM to analyze user requests
• ✅ Understand engineering context dynamically
• ✅ Detect ALL intermediate steps intelligently
• ✅ Distinguish subtle differences (element types, directions, metrics)

Architecture

User Request
     ↓
LLM Analyzer (Claude)
     ↓
Structured JSON Analysis
     ↓
┌────────────────────────────────────┐
│ Engineering Features (FEA)         │
│ Inline Calculations (Math)         │
│ Post-Processing Hooks (Custom)     │
│ Optimization Config                │
└────────────────────────────────────┘
     ↓
Phase 2.5 Capability Matching
     ↓
Research Plan / Code Generation

Example: CBAR Optimization Request

User Input:

I want to extract forces in direction Z of all the 1D elements and find the average of it,
then find the minimum value and compare it to the average, then assign it to a objective
metric that needs to be minimized.

I want to iterate on the FEA properties of the Cbar element stiffness in X to make the
objective function minimized.

I want to use genetic algorithm to iterate and optimize this

LLM Analysis Output:

{
  "engineering_features": [
    {
      "action": "extract_1d_element_forces",
      "domain": "result_extraction",
      "description": "Extract element forces from CBAR in Z direction from OP2",
      "params": {
        "element_types": ["CBAR"],
        "result_type": "element_force",
        "direction": "Z"
      }
    },
    {
      "action": "update_cbar_stiffness",
      "domain": "fea_properties",
      "description": "Modify CBAR stiffness in X direction",
      "params": {
        "element_type": "CBAR",
        "property": "stiffness_x"
      }
    }
  ],
  "inline_calculations": [
    {
      "action": "calculate_average",
      "params": {"input": "forces_z", "operation": "mean"},
      "code_hint": "avg = sum(forces_z) / len(forces_z)"
    },
    {
      "action": "find_minimum",
      "params": {"input": "forces_z", "operation": "min"},
      "code_hint": "min_val = min(forces_z)"
    }
  ],
  "post_processing_hooks": [
    {
      "action": "custom_objective_metric",
      "description": "Compare min to average",
      "params": {
        "inputs": ["min_force", "avg_force"],
        "formula": "min_force / avg_force",
        "objective": "minimize"
      }
    }
  ],
  "optimization": {
    "algorithm": "genetic_algorithm",
    "design_variables": [
      {"parameter": "cbar_stiffness_x", "type": "FEA_property"}
    ]
  }
}

Key Intelligence Improvements

1. Detects Intermediate Steps

Old (Regex):

• ❌ Only saw "extract forces" and "optimize"
• ❌ Missed average, minimum, comparison

New (LLM):

• ✅ Identifies: extract → average → min → compare → optimize
• ✅ Classifies each as engineering vs. simple math

2. Understands Engineering Context

Old (Regex):

• ❌ "forces" → generic "reaction_force" extraction
• ❌ Didn't distinguish CBUSH from CBAR

New (LLM):

• ✅ "1D element forces" → element forces (not reaction forces)
• ✅ "CBAR stiffness in X" → specific property in specific direction
• ✅ Understands these come from different sources (OP2 vs property cards)

3. Smart Classification

Old (Regex):

if 'average' in text:
    return 'simple_calculation'  # Dumb!

New (LLM):

# LLM reasoning:
# - "average of forces" → simple Python (sum/len)
# - "extract forces from OP2" → engineering (pyNastran)
# - "compare min to avg for objective" → hook (custom logic)

4. Generates Actionable Code Hints

Old: Just action names like "calculate_average"

New: Includes code hints for auto-generation:

{
  "action": "calculate_average",
  "code_hint": "avg = sum(forces_z) / len(forces_z)"
}

Integration with Existing Phases

Phase 2.5 (Capability Matching)

LLM output feeds directly into existing capability matcher:

• Engineering features → check if implemented
• If missing → create research plan
• If similar → adapt existing code

Phase 2.6 (Step Classification)

Now replaced by LLM for better accuracy:

• No more static rules
• Context-aware classification
• Understands subtle differences

Implementation

File: optimization_engine/llm_workflow_analyzer.py

Key Function:

analyzer = LLMWorkflowAnalyzer(api_key=os.getenv('ANTHROPIC_API_KEY'))
analysis = analyzer.analyze_request(user_request)

# Returns structured JSON with:
# - engineering_features
# - inline_calculations
# - post_processing_hooks
# - optimization config

Benefits

1. Accurate: Understands engineering nuance
2. Complete: Detects ALL steps, including intermediate ones
3. Dynamic: No hardcoded patterns to maintain
4. Extensible: Automatically handles new request types
5. Actionable: Provides code hints for auto-generation

LLM Integration Modes

Development Mode (Recommended)

For development within Claude Code:

• Use Claude Code directly for interactive workflow analysis
• No API consumption or costs
• Real-time feedback and iteration
• Perfect for testing and refinement

Production Mode (Future)

For standalone Atomizer execution:

• Optional Anthropic API integration
• Set ANTHROPIC_API_KEY environment variable
• Falls back to heuristics if no key provided
• Useful for automated batch processing

Current Status: llm_workflow_analyzer.py supports both modes. For development, continue using Claude Code interactively.

Next Steps

1. ✅ Install anthropic package
2. ✅ Create LLM analyzer module
3. ✅ Document integration modes
4. ⏳ Integrate with Phase 2.5 capability matcher
5. ⏳ Test with diverse optimization requests via Claude Code
6. ⏳ Build code generator for inline calculations
7. ⏳ Build hook generator for post-processing

Success Criteria

Input: "Extract 1D forces, find average, find minimum, compare to average, optimize CBAR stiffness"

Output:

Engineering Features: 2 (need research)
  - extract_1d_element_forces
  - update_cbar_stiffness

Inline Calculations: 2 (auto-generate)
  - calculate_average
  - find_minimum

Post-Processing: 1 (generate hook)
  - custom_objective_metric (min/avg ratio)

Optimization: 1
  - genetic_algorithm

✅ All steps detected
✅ Correctly classified
✅ Ready for implementation