Atomizer/docs/09_DIAGRAMS/architecture_overview.md

Atomizer System Architecture

Last Updated: 2025-11-21 Version: 1.0 Status: ✅ Complete

---

Overview

This document provides comprehensive visual documentation of Atomizer's architecture, showing how components interact to provide intelligent, multi-strategy structural optimization powered by NX Nastran.

---

High-Level System Architecture

graph TB
    subgraph User["👤 User Interface"]
        CLI[Command Line Interface]
        Dashboard[React Dashboard<br/>Real-time Monitoring]
        Config[JSON Configuration<br/>optimization_config.json]
    end

    subgraph Core["🧠 Atomizer Core (Python)"]
        direction TB

        subgraph Engine["Optimization Engine"]
            IntelOpt[IntelligentOptimizer<br/>Protocol 10: IMSO]
            Optuna[Optuna Study<br/>Multi-objective Support]
            Strategy[Strategy Portfolio<br/>TPE/NSGA-II/CMA-ES/...]
            Landscape[Landscape Analyzer<br/>Adaptive Characterization]
        end

        subgraph Tracking["Real-time Tracking"]
            RTTracker[RealtimeTracker<br/>Protocol 13]
            TrialLog[trial_log.json]
            OptimizerState[optimizer_state.json]
            Intelligence[intelligence_report.json]
        end

        subgraph Extractors["Extractor Library<br/>Protocol 12"]
            FreqExt[Frequency Extractor]
            MassExt[Mass Extractor]
            StressExt[Stress Extractor]
            CustomExt[Custom Extractors]
        end
    end

    subgraph NX["⚙️ NX Nastran Integration"]
        SessionMgr[NX Session Manager<br/>Session Pooling]
        ModelUpdate[Model Updater<br/>Design Variable Injection]
        Solver[NX Solver<br/>FEA Execution]
        Results[Result Extraction<br/>Extractor Execution]
    end

    subgraph Storage["💾 Storage Layer"]
        StudyDB[(Optuna SQLite DB<br/>study.db)]
        ModelFiles[(CAD Models<br/>.prt, .sim, .fem)]
        Reports[(Reports & Logs<br/>Markdown, JSON)]
    end

    %% User interactions
    CLI --> Config
    Config --> IntelOpt
    Dashboard -.->|WebSocket Updates| RTTracker

    %% Core optimization flow
    IntelOpt --> Landscape
    Landscape --> Strategy
    Strategy --> Optuna
    IntelOpt --> RTTracker

    %% Optimization loop
    Optuna -->|Trial Parameters| ModelUpdate
    ModelUpdate --> SessionMgr
    SessionMgr --> Solver
    Solver --> Results
    Results --> Extractors
    Extractors -->|Objectives| Optuna

    %% Tracking
    RTTracker --> TrialLog
    RTTracker --> OptimizerState
    Landscape --> Intelligence

    %% Storage
    Optuna --> StudyDB
    SessionMgr --> ModelFiles
    IntelOpt --> Reports

    %% Dashboard reads
    Dashboard -.->|REST API| StudyDB
    Dashboard -.->|File Polling| TrialLog

    %% Styling
    classDef userClass fill:#E3F2FD,stroke:#1976D2,stroke-width:2px
    classDef coreClass fill:#E8F5E9,stroke:#388E3C,stroke-width:2px
    classDef nxClass fill:#FFF3E0,stroke:#F57C00,stroke-width:2px
    classDef storageClass fill:#F3E5F5,stroke:#7B1FA2,stroke-width:2px

    class CLI,Dashboard,Config userClass
    class IntelOpt,Optuna,Strategy,Landscape,RTTracker,FreqExt,MassExt,StressExt,CustomExt coreClass
    class SessionMgr,ModelUpdate,Solver,Results nxClass
    class StudyDB,ModelFiles,Reports storageClass

---

Component Details

1. Optimization Engine Core

graph LR
    subgraph IntelligentOptimizer["IntelligentOptimizer (Protocol 10)"]
        Init[Initialize Study]
        Config[Load Configuration]
        Phase[Determine Phase]
        Execute[Execute Trials]
        Finalize[Generate Report]
    end

    subgraph Protocols["Protocol Support"]
        P10[Protocol 10: IMSO<br/>Adaptive Strategy]
        P11[Protocol 11: Multi-Obj<br/>NSGA-II Support]
        P13[Protocol 13: Tracking<br/>Real-time Updates]
    end

    Init --> Config
    Config --> Phase
    Phase -->|Single-Objective| P10
    Phase -->|Multi-Objective| P11
    P10 --> Execute
    P11 --> Execute
    Execute --> P13
    P13 --> Finalize

    classDef protocolClass fill:#E1BEE7,stroke:#6A1B9A,stroke-width:2px
    class P10,P11,P13 protocolClass

2. NX Integration Architecture

graph TB
    subgraph SessionManagement["NX Session Manager"]
        Pool[Session Pool<br/>Reusable Sessions]
        Create[Create Session]
        Reuse[Reuse Session]
        Cleanup[Cleanup on Error]
    end

    subgraph ModelOperation["Model Operations"]
        Load[Load CAD Model<br/>.prt file]
        Update[Update Parameters<br/>Design Variables]
        Solve[Run FEA Solver<br/>.sim file]
        Extract[Extract Results<br/>Custom Extractors]
    end

    subgraph ErrorHandling["Error Handling"]
        Timeout[Timeout Detection]
        Retry[Retry Logic]
        FallbackCreate[Fallback: New Session]
    end

    Pool --> Reuse
    Pool -->|No Available| Create
    Reuse --> Load
    Create --> Load
    Load --> Update
    Update --> Solve
    Solve --> Extract

    Solve -.->|Error| Timeout
    Timeout --> Retry
    Retry -->|Fail| FallbackCreate
    FallbackCreate --> Cleanup

    classDef sessionClass fill:#FFEBEE,stroke:#C62828,stroke-width:2px
    classDef modelClass fill:#E0F2F1,stroke:#00695C,stroke-width:2px

    class Pool,Create,Reuse,Cleanup sessionClass
    class Load,Update,Solve,Extract modelClass

3. Data Flow Architecture

sequenceDiagram
    participant User
    participant IntelOpt as IntelligentOptimizer
    participant Optuna
    participant NX as NX Solver
    participant Extractor
    participant Tracker as RealtimeTracker
    participant Dashboard

    User->>IntelOpt: Start Optimization
    IntelOpt->>Optuna: Create Study

    loop For each trial
        Optuna->>Optuna: Sample Design Variables
        Optuna->>NX: Trial Parameters
        NX->>NX: Update CAD Model
        NX->>NX: Run FEA Solver
        NX->>Extractor: Simulation Results
        Extractor->>Extractor: Calculate Objectives
        Extractor->>Optuna: Objective Values

        Optuna->>Tracker: Trial Complete
        Tracker->>Tracker: Write trial_log.json
        Tracker->>Tracker: Write optimizer_state.json
        Tracker-->>Dashboard: File Update Signal
        Dashboard-->>User: Live UI Update
    end

    Optuna->>IntelOpt: Study Complete
    IntelOpt->>IntelOpt: Generate Report
    IntelOpt->>User: Optimization Results

---

Protocol Integration

Protocol 10: Intelligent Multi-Strategy Optimization (IMSO)

stateDiagram-v2
    [*] --> Initialization

    Initialization --> TypeCheck: Load Config

    TypeCheck --> SingleObj: Single Objective
    TypeCheck --> MultiObj: Multi-Objective

    SingleObj --> Characterization: Protocol 10 Active
    Characterization --> LandscapeAnalysis: Min trials reached
    LandscapeAnalysis --> StrategySelection: Confidence ≥ 0.85
    StrategySelection --> Optimization: Strategy Selected

    MultiObj --> NSGAIISetup: Protocol 11 Active
    NSGAIISetup --> Optimization: NSGA-II Configured

    Optimization --> TrialExecution: Run Trial
    TrialExecution --> RealtimeTracking: Protocol 13
    RealtimeTracking --> TrialExecution: Next Trial
    TrialExecution --> Finalization: All trials complete

    Finalization --> [*]: Report Generated

    note right of Characterization
        Adaptive phase
        10-30 trials
        Random/Sobol sampling
    end note

    note right of LandscapeAnalysis
        Analyzes:
        - Smoothness
        - Multimodality
        - Separability
        - Noise level
    end note

    note right of NSGAIISetup
        Multi-objective:
        - Skips characterization
        - Direct NSGA-II
        - Pareto front optimization
    end note

Protocol 11: Multi-Objective Support

graph TB
    subgraph Detection["Multi-Objective Detection"]
        CheckObj[Check len objectives]
        Single{objectives == 1?}
        Multi{objectives > 1?}
    end

    subgraph SinglePath["Single-Objective Path"]
        BestValue[study.best_value]
        BestTrial[study.best_trial]
        TrialValue[trial.value]
    end

    subgraph MultiPath["Multi-Objective Path"]
        BestTrials[study.best_trials<br/>Pareto Front]
        TrialValues[trial.values<br/>Multiple objectives]
        NSGAII[NSGA-II Sampler]
    end

    subgraph Validation["Protocol 11 Compliance"]
        CheckAPI{API call type?}
        UseSingle[Use .value]
        UseMulti[Use .values]
        SkipChar[Skip Characterization]
        SkipStag[Skip Stagnation]
    end

    CheckObj --> Single
    CheckObj --> Multi

    Single --> SinglePath
    Multi --> MultiPath

    SinglePath --> CheckAPI
    MultiPath --> CheckAPI

    CheckAPI -->|Single| UseSingle
    CheckAPI -->|Multi| UseMulti

    UseMulti --> SkipChar
    UseMulti --> SkipStag

    classDef errorClass fill:#FFCDD2,stroke:#C62828,stroke-width:3px
    classDef safeClass fill:#C8E6C9,stroke:#2E7D32,stroke-width:3px

    class UseSingle,UseMulti safeClass

Protocol 13: Real-Time Dashboard Tracking

sequenceDiagram
    participant Trial as Optuna Trial
    participant Callback as Optuna Callback
    participant Tracker as RealtimeTracker
    participant FS as File System
    participant Dashboard

    Trial->>Callback: Trial Complete
    Callback->>Tracker: after_trial()

    activate Tracker
    Tracker->>Tracker: Prepare Trial Data
    Tracker->>FS: Write trial_log.json
    Tracker->>FS: Write optimizer_state.json

    alt Characterization Phase
        Tracker->>Tracker: Update Progress
        Tracker->>FS: Write characterization_progress.json
    end

    alt Landscape Analysis
        Tracker->>Tracker: Analyze Landscape
        Tracker->>FS: Write intelligence_report.json
    end

    alt Strategy Transition
        Tracker->>Tracker: Log Transition
        Tracker->>FS: Write strategy_transitions.json
    end

    deactivate Tracker

    FS-->>Dashboard: File Watch Event
    Dashboard->>FS: Read JSON Files
    Dashboard->>Dashboard: Update UI
    Dashboard-->>Dashboard: Render Charts

---

Extractor Library (Protocol 12)

graph TB
    subgraph ExtractorLibrary["Extractor Library (optimization_engine/extractors/)"]
        Base[base_extractor.py<br/>BaseExtractor ABC]

        subgraph Builtin["Built-in Extractors"]
            Freq[frequency_extractor.py<br/>FrequencyExtractor]
            Mass[mass_extractor.py<br/>MassExtractor]
            Stress[stress_extractor.py<br/>StressExtractor]
            Disp[displacement_extractor.py<br/>DisplacementExtractor]
        end

        Custom[Custom User Extractors<br/>Extend BaseExtractor]
    end

    subgraph Usage["Extractor Usage Flow"]
        Registry[Extractor Registry<br/>Map name -> class]
        Load[Load by Name]
        Execute[Execute Extraction]
        Return[Return Value]
    end

    Base --> Freq
    Base --> Mass
    Base --> Stress
    Base --> Disp
    Base --> Custom

    Freq --> Registry
    Mass --> Registry
    Stress --> Registry
    Disp --> Registry
    Custom --> Registry

    Registry --> Load
    Load --> Execute
    Execute --> Return

    classDef extractorClass fill:#FFF9C4,stroke:#F57F17,stroke-width:2px
    class Freq,Mass,Stress,Disp,Custom extractorClass

---

Study Directory Structure

graph TB
    subgraph StudyRoot["studies/my_optimization/"]
        Setup["1_setup/"]
        Results["2_results/"]
    end

    subgraph SetupContents["1_setup/"]
        Model["model/<br/>.prt, .sim, .fem files"]
        OptConfig["optimization_config.json"]
        RunScript["run_optimization.py"]
    end

    subgraph ResultsContents["2_results/"]
        StudyDB["study.db<br/>Optuna SQLite"]
        Summary["optimization_summary.json"]

        subgraph IntelFolder["intelligent_optimizer/"]
            TrialLog["trial_log.json"]
            OptimizerState["optimizer_state.json"]
            CharProgress["characterization_progress.json"]
            IntelReport["intelligence_report.json"]
            StratTrans["strategy_transitions.json"]
        end

        Report["final_report.md<br/>Markdown summary"]
    end

    StudyRoot --> Setup
    StudyRoot --> Results

    Setup --> Model
    Setup --> OptConfig
    Setup --> RunScript

    Results --> StudyDB
    Results --> Summary
    Results --> IntelFolder
    Results --> Report

    IntelFolder --> TrialLog
    IntelFolder --> OptimizerState
    IntelFolder --> CharProgress
    IntelFolder --> IntelReport
    IntelFolder --> StratTrans

---

Philosophy & Design Principles

1. Separation of Concerns

graph LR
    subgraph Philosophy["Atomizer Philosophy"]
        Engine[Optimization Engine<br/>Pure algorithmic logic]
        NX[NX Integration<br/>CAD/FEA operations]
        Track[Tracking System<br/>Observability]
        User[User Interface<br/>Configuration & Monitoring]
    end

    Engine -.->|Clean API| NX
    Engine -.->|Events| Track
    Track -.->|Data| User
    User -.->|Config| Engine

    note1[No tight coupling<br/>Each layer independent]
    note2[Swap NX for Ansys?<br/>Just change integration layer]
    note3[Add new optimizer?<br/>Implement strategy interface]

    Engine -.-> note2
    NX -.-> note2
    Track -.-> note3

2. Protocol-Based Architecture

mindmap
    root((Protocols))
        Protocol 10: IMSO
            Adaptive Characterization
            Landscape Analysis
            Strategy Selection
            Dynamic Switching
        Protocol 11: Multi-Obj
            Pareto Front
            NSGA-II
            API Consistency
            Backward Compatible
        Protocol 13: Tracking
            Real-time Updates
            JSON File Writes
            Dashboard Integration
            Observability
        Future Protocols
            Protocol 14: ?
            Protocol 15: ?
            Extensible System

3. Extensibility Points

graph TB
    subgraph Core["Atomizer Core<br/>(Unchangeable)"]
        CoreEngine[Optimization Engine]
        CoreProtocols[Protocol System]
    end

    subgraph Extension["Extension Points"]
        CustomExtractor[Custom Extractors<br/>BaseExtractor ABC]
        CustomStrategy[Custom Strategies<br/>Optuna Sampler]
        CustomSolver[Custom Solvers<br/>Solver Interface]
        CustomCallback[Custom Callbacks<br/>Optuna Callback]
    end

    subgraph User["User Code"]
        MyExtractor[MyCustomExtractor]
        MyStrategy[MyCustomSampler]
        MySolver[MyFEASolver]
    end

    CoreEngine --> CustomExtractor
    CoreEngine --> CustomStrategy
    CoreEngine --> CustomSolver
    CoreEngine --> CustomCallback

    CustomExtractor -.->|Implement| MyExtractor
    CustomStrategy -.->|Implement| MyStrategy
    CustomSolver -.->|Implement| MySolver

    classDef coreClass fill:#BBDEFB,stroke:#1565C0,stroke-width:3px
    classDef extClass fill:#C8E6C9,stroke:#2E7D32,stroke-width:2px
    classDef userClass fill:#FFE082,stroke:#F57F00,stroke-width:2px

    class CoreEngine,CoreProtocols coreClass
    class CustomExtractor,CustomStrategy,CustomSolver,CustomCallback extClass
    class MyExtractor,MyStrategy,MySolver userClass

---

Technology Stack

graph TB
    subgraph Frontend["Frontend Layer"]
        React[React 18<br/>TypeScript]
        Vite[Vite<br/>Build Tool]
        Recharts[Recharts<br/>Visualization]
    end

    subgraph Backend["Backend Layer"]
        FastAPI[FastAPI<br/>REST API]
        Uvicorn[Uvicorn<br/>ASGI Server]
    end

    subgraph Core["Core Engine"]
        Python[Python 3.8+]
        Optuna[Optuna 3.x<br/>Hyperparameter Optimization]
        NumPy[NumPy/SciPy<br/>Numerical Computing]
    end

    subgraph Integration["Integration"]
        NXOpen[NX Open API<br/>Python Journals]
        SQLite[SQLite<br/>Study Storage]
    end

    React --> FastAPI
    FastAPI --> Optuna
    Optuna --> SQLite
    Optuna --> NXOpen

    classDef frontClass fill:#E1F5FE,stroke:#0277BD
    classDef backClass fill:#F3E5F5,stroke:#6A1B9A
    classDef coreClass fill:#E8F5E9,stroke:#2E7D32
    classDef integClass fill:#FFF3E0,stroke:#E65100

    class React,Vite,Recharts frontClass
    class FastAPI,Uvicorn backClass
    class Python,Optuna,NumPy coreClass
    class NXOpen,SQLite integClass

---

Performance Characteristics

graph LR
    subgraph Timing["Typical Trial Timing (100 trials)"]
        T1[Trial Overhead<br/>~0.5s]
        T2[NX Model Update<br/>~1s]
        T3[FEA Solve<br/>~5-10s]
        T4[Result Extraction<br/>~0.5s]
        T5[Optuna Update<br/>~0.1s]
    end

    T1 --> T2 --> T3 --> T4 --> T5

    Total[Total: ~7-12s per trial<br/>100 trials = 12-20 minutes]

    T5 -.-> Total

    note1[Parallelization possible<br/>Multiple NX sessions]
    note2[Session reuse saves<br/>~2s per trial]

    T3 -.-> note1
    T2 -.-> note2

---

Next Steps

For protocol-specific workflows, see:

• Protocol Workflows - Detailed protocol execution diagrams
• Optimization Lifecycle - Trial-by-trial flow

For implementation details, see:

• Main Documentation Index
• Protocol Specifications