Files
Atomizer/examples/README_INTERACTIVE_SESSION.md
Anto01 0a7cca9c6a feat: Complete Phase 2.5-2.7 - Intelligent LLM-Powered Workflow Analysis
This commit implements three major architectural improvements to transform
Atomizer from static pattern matching to intelligent AI-powered analysis.

## Phase 2.5: Intelligent Codebase-Aware Gap Detection 

Created intelligent system that understands existing capabilities before
requesting examples:

**New Files:**
- optimization_engine/codebase_analyzer.py (379 lines)
  Scans Atomizer codebase for existing FEA/CAE capabilities

- optimization_engine/workflow_decomposer.py (507 lines, v0.2.0)
  Breaks user requests into atomic workflow steps
  Complete rewrite with multi-objective, constraints, subcase targeting

- optimization_engine/capability_matcher.py (312 lines)
  Matches workflow steps to existing code implementations

- optimization_engine/targeted_research_planner.py (259 lines)
  Creates focused research plans for only missing capabilities

**Results:**
- 80-90% coverage on complex optimization requests
- 87-93% confidence in capability matching
- Fixed expression reading misclassification (geometry vs result_extraction)

## Phase 2.6: Intelligent Step Classification 

Distinguishes engineering features from simple math operations:

**New Files:**
- optimization_engine/step_classifier.py (335 lines)

**Classification Types:**
1. Engineering Features - Complex FEA/CAE needing research
2. Inline Calculations - Simple math to auto-generate
3. Post-Processing Hooks - Middleware between FEA steps

## Phase 2.7: LLM-Powered Workflow Intelligence 

Replaces static regex patterns with Claude AI analysis:

**New Files:**
- optimization_engine/llm_workflow_analyzer.py (395 lines)
  Uses Claude API for intelligent request analysis
  Supports both Claude Code (dev) and API (production) modes

- .claude/skills/analyze-workflow.md
  Skill template for LLM workflow analysis integration

**Key Breakthrough:**
- Detects ALL intermediate steps (avg, min, normalization, etc.)
- Understands engineering context (CBUSH vs CBAR, directions, metrics)
- Distinguishes OP2 extraction from part expression reading
- Expected 95%+ accuracy with full nuance detection

## Test Coverage

**New Test Files:**
- tests/test_phase_2_5_intelligent_gap_detection.py (335 lines)
- tests/test_complex_multiobj_request.py (130 lines)
- tests/test_cbush_optimization.py (130 lines)
- tests/test_cbar_genetic_algorithm.py (150 lines)
- tests/test_step_classifier.py (140 lines)
- tests/test_llm_complex_request.py (387 lines)

All tests include:
- UTF-8 encoding for Windows console
- atomizer environment (not test_env)
- Comprehensive validation checks

## Documentation

**New Documentation:**
- docs/PHASE_2_5_INTELLIGENT_GAP_DETECTION.md (254 lines)
- docs/PHASE_2_7_LLM_INTEGRATION.md (227 lines)
- docs/SESSION_SUMMARY_PHASE_2_5_TO_2_7.md (252 lines)

**Updated:**
- README.md - Added Phase 2.5-2.7 completion status
- DEVELOPMENT_ROADMAP.md - Updated phase progress

## Critical Fixes

1. **Expression Reading Misclassification** (lines cited in session summary)
   - Updated codebase_analyzer.py pattern detection
   - Fixed workflow_decomposer.py domain classification
   - Added capability_matcher.py read_expression mapping

2. **Environment Standardization**
   - All code now uses 'atomizer' conda environment
   - Removed test_env references throughout

3. **Multi-Objective Support**
   - WorkflowDecomposer v0.2.0 handles multiple objectives
   - Constraint extraction and validation
   - Subcase and direction targeting

## Architecture Evolution

**Before (Static & Dumb):**
User Request → Regex Patterns → Hardcoded Rules → Missed Steps 

**After (LLM-Powered & Intelligent):**
User Request → Claude AI Analysis → Structured JSON →
├─ Engineering (research needed)
├─ Inline (auto-generate Python)
├─ Hooks (middleware scripts)
└─ Optimization (config) 

## LLM Integration Strategy

**Development Mode (Current):**
- Use Claude Code directly for interactive analysis
- No API consumption or costs
- Perfect for iterative development

**Production Mode (Future):**
- Optional Anthropic API integration
- Falls back to heuristics if no API key
- For standalone batch processing

## Next Steps

- Phase 2.8: Inline Code Generation
- Phase 2.9: Post-Processing Hook Generation
- Phase 3: MCP Integration for automated documentation research

🚀 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-16 13:35:41 -05:00

8.6 KiB

Interactive Research Agent Session

Overview

The Interactive Research Agent allows you to interact with the AI-powered Research Agent through a conversational CLI interface. The agent can learn from examples you provide and automatically generate code for new optimization features.

Quick Start

Run the Interactive Session

python examples/interactive_research_session.py

Try the Demo

When the session starts, type demo to see an automated demonstration:

💬 Your request: demo

The demo will show:

  1. Learning from Example: Agent learns XML material structure from a steel example
  2. Code Generation: Automatically generates Python code (81 lines)
  3. Knowledge Reuse: Second request reuses learned knowledge (no example needed!)

How to Use

Making Requests

Simply type your request in natural language:

💬 Your request: Create an NX material XML generator for aluminum

The agent will:

  1. Analyze what it knows and what's missing
  2. Ask for examples if it needs to learn something new
  3. Search its knowledge base for existing patterns
  4. Generate code from learned templates
  5. Save the generated feature to a file

Providing Examples

When the agent asks for an example, you have 3 options:

  1. Provide a file path:

    Your choice: examples/my_example.xml
    
  2. Paste content directly:

    Your choice: <?xml version="1.0"?>
    <MyExample>...</MyExample>
    
  3. Skip (if you don't have an example):

    Your choice: skip
    

Understanding the Output

The agent provides visual feedback at each step:

  • 🔍 Knowledge Gap Analysis: Shows what's missing and confidence level
  • 📋 Research Plan: Steps the agent will take to gather knowledge
  • 🧠 Knowledge Synthesized: What the agent learned (schemas, patterns)
  • 💻 Code Generation: Preview of generated Python code
  • 💾 Files Created: Where the generated code was saved

Confidence Levels

  • < 50%: New domain - Learning required (will ask for examples)
  • 50-80%: Partial knowledge - Some research needed
  • > 80%: Known domain - Can reuse existing knowledge

Example Session

================================================================================
🤖 Interactive Research Agent Session
================================================================================

  Welcome! I'm your Research Agent. I can learn from examples and
  generate code for optimization features.

  Commands:
    • Type your request in natural language
    • Type 'demo' for a demonstration
    • Type 'quit' to exit

💬 Your request: Create NX material XML for titanium Ti-6Al-4V

--------------------------------------------------------------------------------
[Step 1] Analyzing Knowledge Gap
--------------------------------------------------------------------------------

  🔍 Knowledge Gap Analysis:

  Missing Features (1):
    • new_feature_required

  Missing Knowledge (1):
    • material

  Confidence Level: 80%
  📊 Status: Known domain - Can reuse existing knowledge

--------------------------------------------------------------------------------
[Step 2] Executing Research Plan
--------------------------------------------------------------------------------

  📋 Research Plan Created:

  I'll gather knowledge in 2 steps:

  1. 📚 Search Knowledge Base
     Expected confidence: 80%
     Search query: "material XML NX"

  2. 👤 Ask User For Example
     Expected confidence: 95%
     What I'll ask: "Could you provide an example of an NX material XML file?"

  ⚡ Executing Step 1/2: Search Knowledge Base
  ----------------------------------------------------------------------------
  🔍 Searching knowledge base for: "material XML NX"
  ✓ Found existing knowledge! Session: 2025-11-16_nx_materials_demo
    Confidence: 95%, Relevance: 85%

  ⚡ Executing Step 2/2: Ask User For Example
  ----------------------------------------------------------------------------
  ⊘ Skipping - Already have high confidence from knowledge base

--------------------------------------------------------------------------------
[Step 3] Synthesizing Knowledge
--------------------------------------------------------------------------------

  🧠 Knowledge Synthesized:

  Overall Confidence: 95%

  📄 Learned XML Structure:
    Root element: <PhysicalMaterial>
    Attributes: {'name': 'Steel_AISI_1020', 'version': '1.0'}
    Required fields (5):
      • Density
      • YoungModulus
      • PoissonRatio
      • ThermalExpansion
      • YieldStrength

--------------------------------------------------------------------------------
[Step 4] Generating Feature Code
--------------------------------------------------------------------------------

  🔨 Designing feature: create_nx_material_xml_for_t
    Category: engineering
    Lifecycle stage: all
    Input parameters: 5

  💻 Generating Python code...
    Generated 2327 characters (81 lines)
    ✓ Code is syntactically valid Python

  💾 Saved to: optimization_engine/custom_functions/create_nx_material_xml_for_t.py

================================================================================
✓ Request Completed Successfully!
================================================================================

  Generated file: optimization_engine/custom_functions/create_nx_material_xml_for_t.py
  Knowledge confidence: 95%
  Session saved: 2025-11-16_create_nx_material_xml_for_t

💬 Your request: quit

  👋 Goodbye! Session ended.

Key Features

1. Knowledge Accumulation

  • Agent remembers what it learns across sessions
  • Second similar request doesn't require re-learning
  • Knowledge base grows over time

2. Intelligent Research Planning

  • Prioritizes reliable sources (user examples > MCP > web)
  • Creates step-by-step research plan
  • Explains what it will do before doing it

3. Pattern Recognition

  • Extracts XML schemas from examples
  • Identifies Python code patterns (functions, classes, imports)
  • Learns relationships between inputs and outputs

4. Code Generation

  • Generates complete Python modules with:
    • Docstrings and documentation
    • Type hints for all parameters
    • Example usage code
    • Error handling
  • Code is syntactically validated before saving

5. Session Documentation

  • Every research session is automatically documented
  • Includes: user question, sources, findings, decisions
  • Searchable for future knowledge retrieval

Advanced Usage

Auto Mode (for Testing)

For automated testing, you can run the session in auto-mode:

from examples.interactive_research_session import InteractiveResearchSession

session = InteractiveResearchSession(auto_mode=True)
session.run_demo()  # Runs without user input prompts

Programmatic Usage

You can also use the Research Agent programmatically:

from optimization_engine.research_agent import ResearchAgent

agent = ResearchAgent()

# Identify what's missing
gap = agent.identify_knowledge_gap("Create NX modal analysis")

# Search existing knowledge
existing = agent.search_knowledge_base("modal analysis")

# Create research plan
plan = agent.create_research_plan(gap)

# ... execute plan and synthesize knowledge

Troubleshooting

"No matching session found"

  • This is normal for new domains the agent hasn't seen before
  • The agent will ask for an example to learn from

"Confidence too low to generate code"

  • Provide more detailed examples
  • Try providing multiple examples of the same pattern
  • Check that your example files are well-formed

"Generated code has syntax errors"

  • This is rare and indicates a bug in code generation
  • Please report this with the example that caused it

What's Next

The interactive session currently includes:

  • Knowledge gap detection
  • Knowledge base search and retrieval
  • Learning from user examples
  • Python code generation
  • Session documentation

Coming in future phases:

  • 🔜 MCP server integration (query NX documentation)
  • 🔜 Web search integration (search online resources)
  • 🔜 Multi-turn conversations with context
  • 🔜 Code refinement based on feedback
  • 🔜 Feature validation and testing

Testing

Run the automated test:

python tests/test_interactive_session.py

This will demonstrate the complete workflow including:

  • Learning from an example (steel material XML)
  • Generating working Python code
  • Reusing knowledge for a second request
  • All without user interaction

Support

For issues or questions:

  • Check the existing research sessions in knowledge_base/research_sessions/
  • Review generated code in optimization_engine/custom_functions/
  • See test examples in tests/test_*.py