Atomizer/knowledge_base/README.md

Knowledge Base

Persistent storage of learned patterns, schemas, and research findings for autonomous feature generation

Purpose: Enable Atomizer to learn from user examples, documentation, and research sessions, building a growing repository of knowledge that makes future feature generation faster and more accurate.

---

Folder Structure

knowledge_base/
├── nx_research/           # NX-specific learned patterns and schemas
│   ├── material_xml_schema.md
│   ├── journal_script_patterns.md
│   ├── load_bc_patterns.md
│   └── best_practices.md
├── research_sessions/     # Detailed logs of each research session
│   └── [YYYY-MM-DD]_[topic]/
│       ├── user_question.txt        # Original user request
│       ├── sources_consulted.txt    # Where information came from
│       ├── findings.md              # What was learned
│       └── decision_rationale.md    # Why this approach was chosen
└── templates/             # Reusable code patterns learned from research
    ├── xml_generation_template.py
    ├── journal_script_template.py
    └── custom_extractor_template.py

---

Research Workflow

1. Knowledge Gap Detection

When an LLM encounters a request it cannot fulfill:

# Search feature registry
gap = research_agent.identify_knowledge_gap("Create NX material XML")
# Returns: {'missing_features': ['material_generator'], 'confidence': 0.2}

2. Research Plan Creation

Prioritize sources: User Examples > NX MCP > Web Documentation

plan = research_agent.create_research_plan(gap)
# Returns: [
#   {'step': 1, 'action': 'ask_user_for_example', 'priority': 'high'},
#   {'step': 2, 'action': 'query_nx_mcp', 'priority': 'medium'},
#   {'step': 3, 'action': 'web_search', 'query': 'NX material XML', 'priority': 'low'}
# ]

3. Interactive Research

Ask user first for concrete examples:

LLM: "I don't have a feature for NX material XMLs yet.
     Do you have an example .xml file I can learn from?"

User: [uploads steel_material.xml]

LLM: [Analyzes structure, extracts schema, identifies patterns]

4. Knowledge Synthesis

Combine findings from multiple sources:

findings = {
    'user_example': 'steel_material.xml',
    'nx_mcp_docs': 'PhysicalMaterial schema',
    'web_docs': 'NXOpen material properties API'
}

knowledge = research_agent.synthesize_knowledge(findings)
# Returns: {
#   'schema': {...},
#   'patterns': [...],
#   'confidence': 0.85
# }

5. Feature Generation

Create new feature following learned patterns:

feature_spec = research_agent.design_feature(knowledge)
# Generates:
# - optimization_engine/custom_functions/nx_material_generator.py
# - knowledge_base/nx_research/material_xml_schema.md
# - knowledge_base/templates/xml_generation_template.py

6. Documentation & Integration

Save research session and update registries:

research_agent.document_session(
    topic='nx_materials',
    findings=findings,
    generated_files=['nx_material_generator.py'],
    confidence=0.85
)
# Creates: knowledge_base/research_sessions/2025-01-16_nx_materials/

---

Confidence Tracking

Knowledge is tagged with confidence scores based on source:

Source	Confidence	Reliability
User-validated example	0.95	Highest - user confirmed it works
NX MCP (official docs)	0.85	High - authoritative source
NXOpenTSE (community)	0.70	Medium - community-verified
Web search (generic)	0.50	Low - needs validation

Rule: Only generate code if combined confidence > 0.70

---

Knowledge Retrieval

Before starting new research, search existing knowledge base:

# Check if we already know about this topic
existing = research_agent.search_knowledge_base("material XML")
if existing and existing['confidence'] > 0.8:
    # Use existing template
    template = load_template(existing['template_path'])
else:
    # Start new research session
    research_agent.execute_research(topic="material XML")

---

Best Practices

For NX Research

• Always save journal script patterns with comments explaining NXOpen API calls
• Document version compatibility (e.g., "Tested on NX 2412")
• Include error handling patterns (common NX exceptions)
• Store unit conversion patterns (mm/m, MPa/Pa, etc.)

For Research Sessions

• Save user's original question verbatim
• Document ALL sources consulted (with URLs or file paths)
• Explain decision rationale (why this approach over alternatives)
• Include confidence assessment with justification

For Templates

• Make templates parameterizable (use Jinja2 or similar)
• Include type hints and docstrings
• Add validation logic (check inputs before execution)
• Document expected inputs/outputs

---

Example Research Session

Session: 2025-01-16_nx_materials

User Question:

"Please create a new material XML for NX with titanium Ti-6Al-4V properties"

Sources Consulted:

1. User provided: steel_material.xml (existing NX material)
2. NX MCP query: "PhysicalMaterial XML schema"
3. Web search: "Titanium Ti-6Al-4V material properties"

Findings:

• XML schema learned from user example
• Material properties from web search
• Validation: User confirmed generated XML loads in NX

Generated Files:

1. optimization_engine/custom_functions/nx_material_generator.py
2. knowledge_base/nx_research/material_xml_schema.md
3. knowledge_base/templates/xml_generation_template.py

Confidence: 0.90 (user-validated)

Decision Rationale: Chose XML generation over direct NXOpen API because:

• XML is version-agnostic (works across NX versions)
• User already had XML workflow established
• Easier for user to inspect/validate generated files

---

Future Enhancements

Phase 2 (Current)

• Interactive research workflow
• Knowledge base structure
• Basic pattern learning

Phase 3-4

• Multi-source synthesis (combine user + MCP + web)
• Automatic template extraction from code
• Pattern recognition across sessions

Phase 7-8

• Community knowledge sharing
• Pattern evolution (refine templates based on usage)
• Predictive research (anticipate knowledge gaps)

---

Last Updated: 2025-01-16 Related Docs: DEVELOPMENT_ROADMAP.md, FEATURE_REGISTRY_ARCHITECTURE.md