docs: Major documentation overhaul - restructure folders, update tagline, add Getting Started guide
- Restructure docs/ folder (remove numeric prefixes): - 04_USER_GUIDES -> guides/ - 05_API_REFERENCE -> api/ - 06_PHYSICS -> physics/ - 07_DEVELOPMENT -> development/ - 08_ARCHIVE -> archive/ - 09_DIAGRAMS -> diagrams/ - Replace tagline 'Talk, don't click' with 'LLM-driven optimization framework' in 9 files - Create comprehensive docs/GETTING_STARTED.md: - Prerequisites and quick setup - Project structure overview - First study tutorial (Claude or manual) - Dashboard usage guide - Neural acceleration introduction - Rewrite docs/00_INDEX.md with correct paths and modern structure - Archive obsolete files: - 01_PROTOCOLS.md -> archive/historical/01_PROTOCOLS_legacy.md - 03_GETTING_STARTED.md -> archive/historical/ - ATOMIZER_PODCAST_BRIEFING.md -> archive/marketing/ - Update timestamps to 2026-01-20 across all key files - Update .gitignore to exclude docs/generated/ - Version bump: ATOMIZER_CONTEXT v1.8 -> v2.0
This commit is contained in:
144
docs/diagrams/00_INDEX.md
Normal file
144
docs/diagrams/00_INDEX.md
Normal file
@@ -0,0 +1,144 @@
|
||||
# Atomizer Diagrams Index
|
||||
|
||||
**Last Updated**: 2025-11-21
|
||||
**Purpose**: Visual documentation of Atomizer's architecture, workflows, and protocols
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
This directory contains comprehensive visual documentation using Mermaid diagrams. These diagrams are designed to be:
|
||||
- **Rendered automatically** in GitHub, VS Code, and documentation sites
|
||||
- **Easy to maintain** as plain text with version control
|
||||
- **Comprehensive** covering architecture, workflows, and protocol details
|
||||
|
||||
---
|
||||
|
||||
## Available Diagrams
|
||||
|
||||
### 1. System Architecture
|
||||
**File**: [architecture_overview.md](architecture_overview.md)
|
||||
|
||||
Complete system architecture showing:
|
||||
- Core components and their relationships
|
||||
- Data flow between optimization engine, NX solver, and dashboard
|
||||
- Plugin system and extensibility points
|
||||
- Protocol integration layers
|
||||
|
||||
### 2. Protocol Workflows
|
||||
**File**: [protocol_workflows.md](protocol_workflows.md)
|
||||
|
||||
Detailed protocol execution flows:
|
||||
- Protocol 10: Intelligent Multi-Strategy Optimization (IMSO)
|
||||
- Protocol 11: Multi-Objective Support
|
||||
- Protocol 13: Real-Time Dashboard Tracking
|
||||
- LLM-assisted optimization workflow (Hybrid Mode)
|
||||
|
||||
### 3. Optimization Lifecycle
|
||||
**File**: [optimization_lifecycle.md](optimization_lifecycle.md)
|
||||
|
||||
Trial-by-trial execution flow:
|
||||
- Study initialization
|
||||
- Design variable sampling
|
||||
- NX model update and solving
|
||||
- Result extraction and objective calculation
|
||||
- Optuna feedback loop
|
||||
|
||||
---
|
||||
|
||||
## Viewing Diagrams
|
||||
|
||||
### GitHub
|
||||
Diagrams render automatically when viewing `.md` files on GitHub.
|
||||
|
||||
### VS Code
|
||||
Install the **Markdown Preview Mermaid Support** extension:
|
||||
```
|
||||
code --install-extension bierner.markdown-mermaid
|
||||
```
|
||||
|
||||
### Documentation Sites
|
||||
Mermaid diagrams are supported by:
|
||||
- MkDocs (with pymdown-extensions)
|
||||
- Docusaurus
|
||||
- GitBook
|
||||
- Most modern documentation generators
|
||||
|
||||
### Online Viewer
|
||||
Copy diagram code to: https://mermaid.live/
|
||||
|
||||
---
|
||||
|
||||
## Diagram Conventions
|
||||
|
||||
### Color Coding
|
||||
- **Blue (#4A90E2)**: Core optimization components
|
||||
- **Green (#7CB342)**: NX/Solver integration
|
||||
- **Orange (#FF9800)**: Dashboard/Tracking
|
||||
- **Purple (#9C27B0)**: Protocol logic
|
||||
- **Red (#E53935)**: LLM/Hybrid features
|
||||
- **Gray (#757575)**: Data/Configuration
|
||||
|
||||
### Node Shapes
|
||||
- **Rectangles**: Processes/Components
|
||||
- **Rounded Rectangles**: Modules/Systems
|
||||
- **Diamonds**: Decision points
|
||||
- **Cylinders**: Databases/Storage
|
||||
- **Circles**: Entry/Exit points
|
||||
|
||||
### Arrow Types
|
||||
- **Solid arrow (→)**: Data flow
|
||||
- **Dashed arrow (-->)**: Control flow
|
||||
- **Thick arrow (==>)**: Critical path
|
||||
- **Dotted arrow (-.)**: Optional/Conditional
|
||||
|
||||
---
|
||||
|
||||
## Maintenance
|
||||
|
||||
### Updating Diagrams
|
||||
|
||||
When code changes affect architecture:
|
||||
1. Update relevant diagram file
|
||||
2. Test rendering in VS Code or GitHub preview
|
||||
3. Update "Last Updated" date in diagram header
|
||||
4. Commit with descriptive message: `docs: Update architecture diagram for [feature]`
|
||||
|
||||
### Creating New Diagrams
|
||||
|
||||
For new features or protocols:
|
||||
1. Create new `.md` file in this directory
|
||||
2. Use existing diagrams as templates
|
||||
3. Follow naming convention: `feature_name.md`
|
||||
4. Add entry to this index
|
||||
5. Link from relevant protocol documentation
|
||||
|
||||
### Testing Diagrams
|
||||
|
||||
Before committing:
|
||||
```bash
|
||||
# Check syntax with Mermaid CLI (if installed)
|
||||
mmdc -i diagram.md -o test.svg
|
||||
|
||||
# Or test in browser
|
||||
# Copy code to https://mermaid.live/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Quick Links
|
||||
|
||||
### Documentation
|
||||
- [Main Index](../00_INDEX.md)
|
||||
- [Protocols](../01_PROTOCOLS.md)
|
||||
- [Architecture](../02_ARCHITECTURE.md)
|
||||
|
||||
### Implementation
|
||||
- [Optimization Engine](../../optimization_engine/)
|
||||
- [NX Integration](../../nx_journals/)
|
||||
- [Dashboard](../../atomizer-dashboard/)
|
||||
|
||||
---
|
||||
|
||||
**Maintained By**: Atomizer Development Team
|
||||
**Feedback**: Open GitHub issue with "documentation" label
|
||||
640
docs/diagrams/architecture_overview.md
Normal file
640
docs/diagrams/architecture_overview.md
Normal file
@@ -0,0 +1,640 @@
|
||||
# Atomizer System Architecture
|
||||
|
||||
**Last Updated**: 2025-11-21
|
||||
**Version**: 1.0
|
||||
**Status**: ✅ Complete
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
This document provides comprehensive visual documentation of Atomizer's architecture, showing how components interact to provide intelligent, multi-strategy structural optimization powered by NX Nastran.
|
||||
|
||||
---
|
||||
|
||||
## High-Level System Architecture
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph User["👤 User Interface"]
|
||||
CLI[Command Line Interface]
|
||||
Dashboard[React Dashboard<br/>Real-time Monitoring]
|
||||
Config[JSON Configuration<br/>optimization_config.json]
|
||||
end
|
||||
|
||||
subgraph Core["🧠 Atomizer Core (Python)"]
|
||||
direction TB
|
||||
|
||||
subgraph Engine["Optimization Engine"]
|
||||
IntelOpt[IntelligentOptimizer<br/>Protocol 10: IMSO]
|
||||
Optuna[Optuna Study<br/>Multi-objective Support]
|
||||
Strategy[Strategy Portfolio<br/>TPE/NSGA-II/CMA-ES/...]
|
||||
Landscape[Landscape Analyzer<br/>Adaptive Characterization]
|
||||
end
|
||||
|
||||
subgraph Tracking["Real-time Tracking"]
|
||||
RTTracker[RealtimeTracker<br/>Protocol 13]
|
||||
TrialLog[trial_log.json]
|
||||
OptimizerState[optimizer_state.json]
|
||||
Intelligence[intelligence_report.json]
|
||||
end
|
||||
|
||||
subgraph Extractors["Extractor Library<br/>Protocol 12"]
|
||||
FreqExt[Frequency Extractor]
|
||||
MassExt[Mass Extractor]
|
||||
StressExt[Stress Extractor]
|
||||
CustomExt[Custom Extractors]
|
||||
end
|
||||
end
|
||||
|
||||
subgraph NX["⚙️ NX Nastran Integration"]
|
||||
SessionMgr[NX Session Manager<br/>Session Pooling]
|
||||
ModelUpdate[Model Updater<br/>Design Variable Injection]
|
||||
Solver[NX Solver<br/>FEA Execution]
|
||||
Results[Result Extraction<br/>Extractor Execution]
|
||||
end
|
||||
|
||||
subgraph Storage["💾 Storage Layer"]
|
||||
StudyDB[(Optuna SQLite DB<br/>study.db)]
|
||||
ModelFiles[(CAD Models<br/>.prt, .sim, .fem)]
|
||||
Reports[(Reports & Logs<br/>Markdown, JSON)]
|
||||
end
|
||||
|
||||
%% User interactions
|
||||
CLI --> Config
|
||||
Config --> IntelOpt
|
||||
Dashboard -.->|WebSocket Updates| RTTracker
|
||||
|
||||
%% Core optimization flow
|
||||
IntelOpt --> Landscape
|
||||
Landscape --> Strategy
|
||||
Strategy --> Optuna
|
||||
IntelOpt --> RTTracker
|
||||
|
||||
%% Optimization loop
|
||||
Optuna -->|Trial Parameters| ModelUpdate
|
||||
ModelUpdate --> SessionMgr
|
||||
SessionMgr --> Solver
|
||||
Solver --> Results
|
||||
Results --> Extractors
|
||||
Extractors -->|Objectives| Optuna
|
||||
|
||||
%% Tracking
|
||||
RTTracker --> TrialLog
|
||||
RTTracker --> OptimizerState
|
||||
Landscape --> Intelligence
|
||||
|
||||
%% Storage
|
||||
Optuna --> StudyDB
|
||||
SessionMgr --> ModelFiles
|
||||
IntelOpt --> Reports
|
||||
|
||||
%% Dashboard reads
|
||||
Dashboard -.->|REST API| StudyDB
|
||||
Dashboard -.->|File Polling| TrialLog
|
||||
|
||||
%% Styling
|
||||
classDef userClass fill:#E3F2FD,stroke:#1976D2,stroke-width:2px
|
||||
classDef coreClass fill:#E8F5E9,stroke:#388E3C,stroke-width:2px
|
||||
classDef nxClass fill:#FFF3E0,stroke:#F57C00,stroke-width:2px
|
||||
classDef storageClass fill:#F3E5F5,stroke:#7B1FA2,stroke-width:2px
|
||||
|
||||
class CLI,Dashboard,Config userClass
|
||||
class IntelOpt,Optuna,Strategy,Landscape,RTTracker,FreqExt,MassExt,StressExt,CustomExt coreClass
|
||||
class SessionMgr,ModelUpdate,Solver,Results nxClass
|
||||
class StudyDB,ModelFiles,Reports storageClass
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Component Details
|
||||
|
||||
### 1. Optimization Engine Core
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
subgraph IntelligentOptimizer["IntelligentOptimizer (Protocol 10)"]
|
||||
Init[Initialize Study]
|
||||
Config[Load Configuration]
|
||||
Phase[Determine Phase]
|
||||
Execute[Execute Trials]
|
||||
Finalize[Generate Report]
|
||||
end
|
||||
|
||||
subgraph Protocols["Protocol Support"]
|
||||
P10[Protocol 10: IMSO<br/>Adaptive Strategy]
|
||||
P11[Protocol 11: Multi-Obj<br/>NSGA-II Support]
|
||||
P13[Protocol 13: Tracking<br/>Real-time Updates]
|
||||
end
|
||||
|
||||
Init --> Config
|
||||
Config --> Phase
|
||||
Phase -->|Single-Objective| P10
|
||||
Phase -->|Multi-Objective| P11
|
||||
P10 --> Execute
|
||||
P11 --> Execute
|
||||
Execute --> P13
|
||||
P13 --> Finalize
|
||||
|
||||
classDef protocolClass fill:#E1BEE7,stroke:#6A1B9A,stroke-width:2px
|
||||
class P10,P11,P13 protocolClass
|
||||
```
|
||||
|
||||
### 2. NX Integration Architecture
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph SessionManagement["NX Session Manager"]
|
||||
Pool[Session Pool<br/>Reusable Sessions]
|
||||
Create[Create Session]
|
||||
Reuse[Reuse Session]
|
||||
Cleanup[Cleanup on Error]
|
||||
end
|
||||
|
||||
subgraph ModelOperation["Model Operations"]
|
||||
Load[Load CAD Model<br/>.prt file]
|
||||
Update[Update Parameters<br/>Design Variables]
|
||||
Solve[Run FEA Solver<br/>.sim file]
|
||||
Extract[Extract Results<br/>Custom Extractors]
|
||||
end
|
||||
|
||||
subgraph ErrorHandling["Error Handling"]
|
||||
Timeout[Timeout Detection]
|
||||
Retry[Retry Logic]
|
||||
FallbackCreate[Fallback: New Session]
|
||||
end
|
||||
|
||||
Pool --> Reuse
|
||||
Pool -->|No Available| Create
|
||||
Reuse --> Load
|
||||
Create --> Load
|
||||
Load --> Update
|
||||
Update --> Solve
|
||||
Solve --> Extract
|
||||
|
||||
Solve -.->|Error| Timeout
|
||||
Timeout --> Retry
|
||||
Retry -->|Fail| FallbackCreate
|
||||
FallbackCreate --> Cleanup
|
||||
|
||||
classDef sessionClass fill:#FFEBEE,stroke:#C62828,stroke-width:2px
|
||||
classDef modelClass fill:#E0F2F1,stroke:#00695C,stroke-width:2px
|
||||
|
||||
class Pool,Create,Reuse,Cleanup sessionClass
|
||||
class Load,Update,Solve,Extract modelClass
|
||||
```
|
||||
|
||||
### 3. Data Flow Architecture
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant User
|
||||
participant IntelOpt as IntelligentOptimizer
|
||||
participant Optuna
|
||||
participant NX as NX Solver
|
||||
participant Extractor
|
||||
participant Tracker as RealtimeTracker
|
||||
participant Dashboard
|
||||
|
||||
User->>IntelOpt: Start Optimization
|
||||
IntelOpt->>Optuna: Create Study
|
||||
|
||||
loop For each trial
|
||||
Optuna->>Optuna: Sample Design Variables
|
||||
Optuna->>NX: Trial Parameters
|
||||
NX->>NX: Update CAD Model
|
||||
NX->>NX: Run FEA Solver
|
||||
NX->>Extractor: Simulation Results
|
||||
Extractor->>Extractor: Calculate Objectives
|
||||
Extractor->>Optuna: Objective Values
|
||||
|
||||
Optuna->>Tracker: Trial Complete
|
||||
Tracker->>Tracker: Write trial_log.json
|
||||
Tracker->>Tracker: Write optimizer_state.json
|
||||
Tracker-->>Dashboard: File Update Signal
|
||||
Dashboard-->>User: Live UI Update
|
||||
end
|
||||
|
||||
Optuna->>IntelOpt: Study Complete
|
||||
IntelOpt->>IntelOpt: Generate Report
|
||||
IntelOpt->>User: Optimization Results
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Protocol Integration
|
||||
|
||||
### Protocol 10: Intelligent Multi-Strategy Optimization (IMSO)
|
||||
|
||||
```mermaid
|
||||
stateDiagram-v2
|
||||
[*] --> Initialization
|
||||
|
||||
Initialization --> TypeCheck: Load Config
|
||||
|
||||
TypeCheck --> SingleObj: Single Objective
|
||||
TypeCheck --> MultiObj: Multi-Objective
|
||||
|
||||
SingleObj --> Characterization: Protocol 10 Active
|
||||
Characterization --> LandscapeAnalysis: Min trials reached
|
||||
LandscapeAnalysis --> StrategySelection: Confidence ≥ 0.85
|
||||
StrategySelection --> Optimization: Strategy Selected
|
||||
|
||||
MultiObj --> NSGAIISetup: Protocol 11 Active
|
||||
NSGAIISetup --> Optimization: NSGA-II Configured
|
||||
|
||||
Optimization --> TrialExecution: Run Trial
|
||||
TrialExecution --> RealtimeTracking: Protocol 13
|
||||
RealtimeTracking --> TrialExecution: Next Trial
|
||||
TrialExecution --> Finalization: All trials complete
|
||||
|
||||
Finalization --> [*]: Report Generated
|
||||
|
||||
note right of Characterization
|
||||
Adaptive phase
|
||||
10-30 trials
|
||||
Random/Sobol sampling
|
||||
end note
|
||||
|
||||
note right of LandscapeAnalysis
|
||||
Analyzes:
|
||||
- Smoothness
|
||||
- Multimodality
|
||||
- Separability
|
||||
- Noise level
|
||||
end note
|
||||
|
||||
note right of NSGAIISetup
|
||||
Multi-objective:
|
||||
- Skips characterization
|
||||
- Direct NSGA-II
|
||||
- Pareto front optimization
|
||||
end note
|
||||
```
|
||||
|
||||
### Protocol 11: Multi-Objective Support
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph Detection["Multi-Objective Detection"]
|
||||
CheckObj[Check len objectives]
|
||||
Single{objectives == 1?}
|
||||
Multi{objectives > 1?}
|
||||
end
|
||||
|
||||
subgraph SinglePath["Single-Objective Path"]
|
||||
BestValue[study.best_value]
|
||||
BestTrial[study.best_trial]
|
||||
TrialValue[trial.value]
|
||||
end
|
||||
|
||||
subgraph MultiPath["Multi-Objective Path"]
|
||||
BestTrials[study.best_trials<br/>Pareto Front]
|
||||
TrialValues[trial.values<br/>Multiple objectives]
|
||||
NSGAII[NSGA-II Sampler]
|
||||
end
|
||||
|
||||
subgraph Validation["Protocol 11 Compliance"]
|
||||
CheckAPI{API call type?}
|
||||
UseSingle[Use .value]
|
||||
UseMulti[Use .values]
|
||||
SkipChar[Skip Characterization]
|
||||
SkipStag[Skip Stagnation]
|
||||
end
|
||||
|
||||
CheckObj --> Single
|
||||
CheckObj --> Multi
|
||||
|
||||
Single --> SinglePath
|
||||
Multi --> MultiPath
|
||||
|
||||
SinglePath --> CheckAPI
|
||||
MultiPath --> CheckAPI
|
||||
|
||||
CheckAPI -->|Single| UseSingle
|
||||
CheckAPI -->|Multi| UseMulti
|
||||
|
||||
UseMulti --> SkipChar
|
||||
UseMulti --> SkipStag
|
||||
|
||||
classDef errorClass fill:#FFCDD2,stroke:#C62828,stroke-width:3px
|
||||
classDef safeClass fill:#C8E6C9,stroke:#2E7D32,stroke-width:3px
|
||||
|
||||
class UseSingle,UseMulti safeClass
|
||||
```
|
||||
|
||||
### Protocol 13: Real-Time Dashboard Tracking
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant Trial as Optuna Trial
|
||||
participant Callback as Optuna Callback
|
||||
participant Tracker as RealtimeTracker
|
||||
participant FS as File System
|
||||
participant Dashboard
|
||||
|
||||
Trial->>Callback: Trial Complete
|
||||
Callback->>Tracker: after_trial()
|
||||
|
||||
activate Tracker
|
||||
Tracker->>Tracker: Prepare Trial Data
|
||||
Tracker->>FS: Write trial_log.json
|
||||
Tracker->>FS: Write optimizer_state.json
|
||||
|
||||
alt Characterization Phase
|
||||
Tracker->>Tracker: Update Progress
|
||||
Tracker->>FS: Write characterization_progress.json
|
||||
end
|
||||
|
||||
alt Landscape Analysis
|
||||
Tracker->>Tracker: Analyze Landscape
|
||||
Tracker->>FS: Write intelligence_report.json
|
||||
end
|
||||
|
||||
alt Strategy Transition
|
||||
Tracker->>Tracker: Log Transition
|
||||
Tracker->>FS: Write strategy_transitions.json
|
||||
end
|
||||
|
||||
deactivate Tracker
|
||||
|
||||
FS-->>Dashboard: File Watch Event
|
||||
Dashboard->>FS: Read JSON Files
|
||||
Dashboard->>Dashboard: Update UI
|
||||
Dashboard-->>Dashboard: Render Charts
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Extractor Library (Protocol 12)
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph ExtractorLibrary["Extractor Library (optimization_engine/extractors/)"]
|
||||
Base[base_extractor.py<br/>BaseExtractor ABC]
|
||||
|
||||
subgraph Builtin["Built-in Extractors"]
|
||||
Freq[frequency_extractor.py<br/>FrequencyExtractor]
|
||||
Mass[mass_extractor.py<br/>MassExtractor]
|
||||
Stress[stress_extractor.py<br/>StressExtractor]
|
||||
Disp[displacement_extractor.py<br/>DisplacementExtractor]
|
||||
end
|
||||
|
||||
Custom[Custom User Extractors<br/>Extend BaseExtractor]
|
||||
end
|
||||
|
||||
subgraph Usage["Extractor Usage Flow"]
|
||||
Registry[Extractor Registry<br/>Map name -> class]
|
||||
Load[Load by Name]
|
||||
Execute[Execute Extraction]
|
||||
Return[Return Value]
|
||||
end
|
||||
|
||||
Base --> Freq
|
||||
Base --> Mass
|
||||
Base --> Stress
|
||||
Base --> Disp
|
||||
Base --> Custom
|
||||
|
||||
Freq --> Registry
|
||||
Mass --> Registry
|
||||
Stress --> Registry
|
||||
Disp --> Registry
|
||||
Custom --> Registry
|
||||
|
||||
Registry --> Load
|
||||
Load --> Execute
|
||||
Execute --> Return
|
||||
|
||||
classDef extractorClass fill:#FFF9C4,stroke:#F57F17,stroke-width:2px
|
||||
class Freq,Mass,Stress,Disp,Custom extractorClass
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Study Directory Structure
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph StudyRoot["studies/my_optimization/"]
|
||||
Setup["1_setup/"]
|
||||
Results["2_results/"]
|
||||
end
|
||||
|
||||
subgraph SetupContents["1_setup/"]
|
||||
Model["model/<br/>.prt, .sim, .fem files"]
|
||||
OptConfig["optimization_config.json"]
|
||||
RunScript["run_optimization.py"]
|
||||
end
|
||||
|
||||
subgraph ResultsContents["2_results/"]
|
||||
StudyDB["study.db<br/>Optuna SQLite"]
|
||||
Summary["optimization_summary.json"]
|
||||
|
||||
subgraph IntelFolder["intelligent_optimizer/"]
|
||||
TrialLog["trial_log.json"]
|
||||
OptimizerState["optimizer_state.json"]
|
||||
CharProgress["characterization_progress.json"]
|
||||
IntelReport["intelligence_report.json"]
|
||||
StratTrans["strategy_transitions.json"]
|
||||
end
|
||||
|
||||
Report["final_report.md<br/>Markdown summary"]
|
||||
end
|
||||
|
||||
StudyRoot --> Setup
|
||||
StudyRoot --> Results
|
||||
|
||||
Setup --> Model
|
||||
Setup --> OptConfig
|
||||
Setup --> RunScript
|
||||
|
||||
Results --> StudyDB
|
||||
Results --> Summary
|
||||
Results --> IntelFolder
|
||||
Results --> Report
|
||||
|
||||
IntelFolder --> TrialLog
|
||||
IntelFolder --> OptimizerState
|
||||
IntelFolder --> CharProgress
|
||||
IntelFolder --> IntelReport
|
||||
IntelFolder --> StratTrans
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Philosophy & Design Principles
|
||||
|
||||
### 1. Separation of Concerns
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
subgraph Philosophy["Atomizer Philosophy"]
|
||||
Engine[Optimization Engine<br/>Pure algorithmic logic]
|
||||
NX[NX Integration<br/>CAD/FEA operations]
|
||||
Track[Tracking System<br/>Observability]
|
||||
User[User Interface<br/>Configuration & Monitoring]
|
||||
end
|
||||
|
||||
Engine -.->|Clean API| NX
|
||||
Engine -.->|Events| Track
|
||||
Track -.->|Data| User
|
||||
User -.->|Config| Engine
|
||||
|
||||
note1[No tight coupling<br/>Each layer independent]
|
||||
note2[Swap NX for Ansys?<br/>Just change integration layer]
|
||||
note3[Add new optimizer?<br/>Implement strategy interface]
|
||||
|
||||
Engine -.-> note2
|
||||
NX -.-> note2
|
||||
Track -.-> note3
|
||||
```
|
||||
|
||||
### 2. Protocol-Based Architecture
|
||||
|
||||
```mermaid
|
||||
mindmap
|
||||
root((Protocols))
|
||||
Protocol 10: IMSO
|
||||
Adaptive Characterization
|
||||
Landscape Analysis
|
||||
Strategy Selection
|
||||
Dynamic Switching
|
||||
Protocol 11: Multi-Obj
|
||||
Pareto Front
|
||||
NSGA-II
|
||||
API Consistency
|
||||
Backward Compatible
|
||||
Protocol 13: Tracking
|
||||
Real-time Updates
|
||||
JSON File Writes
|
||||
Dashboard Integration
|
||||
Observability
|
||||
Future Protocols
|
||||
Protocol 14: ?
|
||||
Protocol 15: ?
|
||||
Extensible System
|
||||
```
|
||||
|
||||
### 3. Extensibility Points
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph Core["Atomizer Core<br/>(Unchangeable)"]
|
||||
CoreEngine[Optimization Engine]
|
||||
CoreProtocols[Protocol System]
|
||||
end
|
||||
|
||||
subgraph Extension["Extension Points"]
|
||||
CustomExtractor[Custom Extractors<br/>BaseExtractor ABC]
|
||||
CustomStrategy[Custom Strategies<br/>Optuna Sampler]
|
||||
CustomSolver[Custom Solvers<br/>Solver Interface]
|
||||
CustomCallback[Custom Callbacks<br/>Optuna Callback]
|
||||
end
|
||||
|
||||
subgraph User["User Code"]
|
||||
MyExtractor[MyCustomExtractor]
|
||||
MyStrategy[MyCustomSampler]
|
||||
MySolver[MyFEASolver]
|
||||
end
|
||||
|
||||
CoreEngine --> CustomExtractor
|
||||
CoreEngine --> CustomStrategy
|
||||
CoreEngine --> CustomSolver
|
||||
CoreEngine --> CustomCallback
|
||||
|
||||
CustomExtractor -.->|Implement| MyExtractor
|
||||
CustomStrategy -.->|Implement| MyStrategy
|
||||
CustomSolver -.->|Implement| MySolver
|
||||
|
||||
classDef coreClass fill:#BBDEFB,stroke:#1565C0,stroke-width:3px
|
||||
classDef extClass fill:#C8E6C9,stroke:#2E7D32,stroke-width:2px
|
||||
classDef userClass fill:#FFE082,stroke:#F57F00,stroke-width:2px
|
||||
|
||||
class CoreEngine,CoreProtocols coreClass
|
||||
class CustomExtractor,CustomStrategy,CustomSolver,CustomCallback extClass
|
||||
class MyExtractor,MyStrategy,MySolver userClass
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Technology Stack
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph Frontend["Frontend Layer"]
|
||||
React[React 18<br/>TypeScript]
|
||||
Vite[Vite<br/>Build Tool]
|
||||
Recharts[Recharts<br/>Visualization]
|
||||
end
|
||||
|
||||
subgraph Backend["Backend Layer"]
|
||||
FastAPI[FastAPI<br/>REST API]
|
||||
Uvicorn[Uvicorn<br/>ASGI Server]
|
||||
end
|
||||
|
||||
subgraph Core["Core Engine"]
|
||||
Python[Python 3.8+]
|
||||
Optuna[Optuna 3.x<br/>Hyperparameter Optimization]
|
||||
NumPy[NumPy/SciPy<br/>Numerical Computing]
|
||||
end
|
||||
|
||||
subgraph Integration["Integration"]
|
||||
NXOpen[NX Open API<br/>Python Journals]
|
||||
SQLite[SQLite<br/>Study Storage]
|
||||
end
|
||||
|
||||
React --> FastAPI
|
||||
FastAPI --> Optuna
|
||||
Optuna --> SQLite
|
||||
Optuna --> NXOpen
|
||||
|
||||
classDef frontClass fill:#E1F5FE,stroke:#0277BD
|
||||
classDef backClass fill:#F3E5F5,stroke:#6A1B9A
|
||||
classDef coreClass fill:#E8F5E9,stroke:#2E7D32
|
||||
classDef integClass fill:#FFF3E0,stroke:#E65100
|
||||
|
||||
class React,Vite,Recharts frontClass
|
||||
class FastAPI,Uvicorn backClass
|
||||
class Python,Optuna,NumPy coreClass
|
||||
class NXOpen,SQLite integClass
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Performance Characteristics
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
subgraph Timing["Typical Trial Timing (100 trials)"]
|
||||
T1[Trial Overhead<br/>~0.5s]
|
||||
T2[NX Model Update<br/>~1s]
|
||||
T3[FEA Solve<br/>~5-10s]
|
||||
T4[Result Extraction<br/>~0.5s]
|
||||
T5[Optuna Update<br/>~0.1s]
|
||||
end
|
||||
|
||||
T1 --> T2 --> T3 --> T4 --> T5
|
||||
|
||||
Total[Total: ~7-12s per trial<br/>100 trials = 12-20 minutes]
|
||||
|
||||
T5 -.-> Total
|
||||
|
||||
note1[Parallelization possible<br/>Multiple NX sessions]
|
||||
note2[Session reuse saves<br/>~2s per trial]
|
||||
|
||||
T3 -.-> note1
|
||||
T2 -.-> note2
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
For protocol-specific workflows, see:
|
||||
- [Protocol Workflows](protocol_workflows.md) - Detailed protocol execution diagrams
|
||||
- [Optimization Lifecycle](optimization_lifecycle.md) - Trial-by-trial flow
|
||||
|
||||
For implementation details, see:
|
||||
- [Main Documentation Index](../00_INDEX.md)
|
||||
- [Protocol Specifications](../01_PROTOCOLS.md)
|
||||
676
docs/diagrams/protocol_workflows.md
Normal file
676
docs/diagrams/protocol_workflows.md
Normal file
@@ -0,0 +1,676 @@
|
||||
# Protocol Workflows
|
||||
|
||||
**Last Updated**: 2025-11-21
|
||||
**Version**: 1.0
|
||||
**Status**: ✅ Complete
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
This document provides detailed workflow diagrams for each Atomizer protocol, showing exactly how the system processes optimizations from configuration to completion.
|
||||
|
||||
---
|
||||
|
||||
## Protocol 10: Intelligent Multi-Strategy Optimization (IMSO)
|
||||
|
||||
### Complete IMSO Workflow
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
Start([User Starts Optimization]) --> LoadConfig[Load optimization_config.json]
|
||||
LoadConfig --> CheckMultiObj{Multi-objective?}
|
||||
|
||||
%% Multi-objective path
|
||||
CheckMultiObj -->|Yes| MultiObjPath[Protocol 11 Active]
|
||||
MultiObjPath --> SkipChar[Skip Characterization<br/>Protocol 10 Not Used]
|
||||
SkipChar --> SetNSGAII[Set Sampler: NSGA-II]
|
||||
SetNSGAII --> RunTrialsMulti[Run All Trials<br/>NSGA-II Only]
|
||||
|
||||
%% Single-objective path
|
||||
CheckMultiObj -->|No| SingleObjPath[Protocol 10 Active]
|
||||
SingleObjPath --> InitChar[Initialize Characterization]
|
||||
|
||||
%% Characterization Phase
|
||||
subgraph CharPhase["Characterization Phase (Protocol 10)"]
|
||||
InitChar --> RandomSampling[Random/Sobol Sampling]
|
||||
RandomSampling --> RunCharTrial[Run Trial]
|
||||
RunCharTrial --> CharCount{Trial count ≥ min_trials?}
|
||||
CharCount -->|No| RandomSampling
|
||||
CharCount -->|Yes| CheckInterval{Check interval<br/>reached?}
|
||||
CheckInterval -->|No| RandomSampling
|
||||
CheckInterval -->|Yes| CalcConfidence[Calculate Confidence Score]
|
||||
|
||||
CalcConfidence --> ConfCheck{Confidence ≥ 0.85?}
|
||||
ConfCheck -->|No| MaxCheck{Reached max_trials?}
|
||||
MaxCheck -->|No| RandomSampling
|
||||
MaxCheck -->|Yes| ForceAnalysis[Force Landscape Analysis]
|
||||
|
||||
ConfCheck -->|Yes| ReadyForAnalysis[Ready for Analysis]
|
||||
ForceAnalysis --> ReadyForAnalysis
|
||||
end
|
||||
|
||||
%% Landscape Analysis
|
||||
ReadyForAnalysis --> LandscapeAnalysis
|
||||
|
||||
subgraph Analysis["Landscape Analysis"]
|
||||
LandscapeAnalysis[Analyze Landscape] --> Smoothness[Compute Smoothness]
|
||||
Smoothness --> Multimodal[Detect Multimodality]
|
||||
Multimodal --> Separability[Measure Separability]
|
||||
Separability --> Noise[Estimate Noise Level]
|
||||
|
||||
Noise --> Classify{Landscape Type}
|
||||
|
||||
Classify -->|Smooth + Unimodal| RecommendTPE[Recommend: TPE]
|
||||
Classify -->|Smooth + Multimodal| RecommendCMAES[Recommend: CMA-ES]
|
||||
Classify -->|Noisy| RecommendGP[Recommend: GP]
|
||||
Classify -->|Complex| RecommendNSGAII[Recommend: NSGA-II]
|
||||
Classify -->|Unknown| RecommendRandom[Recommend: Random]
|
||||
end
|
||||
|
||||
%% Strategy Selection
|
||||
RecommendTPE --> SelectStrategy[Select Strategy]
|
||||
RecommendCMAES --> SelectStrategy
|
||||
RecommendGP --> SelectStrategy
|
||||
RecommendNSGAII --> SelectStrategy
|
||||
RecommendRandom --> SelectStrategy
|
||||
|
||||
SelectStrategy --> LogTransition[Log Strategy Transition<br/>Protocol 13]
|
||||
LogTransition --> RunOptimization
|
||||
|
||||
%% Optimization Phase
|
||||
subgraph OptPhase["Optimization Phase"]
|
||||
RunOptimization[Run Optimization] --> RunTrial[Execute Trial]
|
||||
RunTrial --> CheckStagnation{Stagnation<br/>detected?}
|
||||
CheckStagnation -->|No| MoreTrials{More trials?}
|
||||
CheckStagnation -->|Yes| SwitchStrategy[Switch Strategy]
|
||||
SwitchStrategy --> LogSwitch[Log Transition]
|
||||
LogSwitch --> RunTrial
|
||||
|
||||
MoreTrials -->|Yes| RunTrial
|
||||
MoreTrials -->|No| OptComplete[Optimization Complete]
|
||||
end
|
||||
|
||||
RunTrialsMulti --> Complete
|
||||
OptComplete --> Complete
|
||||
|
||||
Complete[Generate Final Report] --> End([Optimization Complete])
|
||||
|
||||
%% Styling
|
||||
classDef phaseClass fill:#E8F5E9,stroke:#2E7D32,stroke-width:3px
|
||||
classDef decisionClass fill:#FFF9C4,stroke:#F57F17,stroke-width:2px
|
||||
classDef protocolClass fill:#E1BEE7,stroke:#6A1B9A,stroke-width:2px
|
||||
|
||||
class CharPhase,OptPhase,Analysis phaseClass
|
||||
class CheckMultiObj,CharCount,CheckInterval,ConfCheck,MaxCheck,Classify,CheckStagnation,MoreTrials decisionClass
|
||||
class MultiObjPath,SingleObjPath,LogTransition,LogSwitch protocolClass
|
||||
```
|
||||
|
||||
### Confidence Calculation Details
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph ConfidenceCalc["Confidence Score Calculation"]
|
||||
Start[Recent Trials<br/>Last 10-20 trials] --> Convergence[Convergence Score<br/>Improvement rate]
|
||||
Start --> Coverage[Exploration Coverage<br/>Design space sampling]
|
||||
Start --> Stability[Prediction Stability<br/>Surrogate variance]
|
||||
|
||||
Convergence --> Weight1[× 0.4]
|
||||
Coverage --> Weight2[× 0.3]
|
||||
Stability --> Weight3[× 0.3]
|
||||
|
||||
Weight1 --> Sum[Σ Weighted Scores]
|
||||
Weight2 --> Sum
|
||||
Weight3 --> Sum
|
||||
|
||||
Sum --> Overall[Overall Confidence<br/>0.0 - 1.0]
|
||||
|
||||
Overall --> Threshold{≥ 0.85?}
|
||||
Threshold -->|Yes| Ready[Ready for Exploitation]
|
||||
Threshold -->|No| Continue[Continue Characterization]
|
||||
end
|
||||
|
||||
subgraph Metrics["Individual Metrics"]
|
||||
Conv[Convergence:<br/>Recent improvement / Initial range]
|
||||
Cov[Coverage:<br/>Unique regions explored / Total regions]
|
||||
Stab[Stability:<br/>1 - (prediction_std / value_range)]
|
||||
end
|
||||
|
||||
Convergence -.-> Conv
|
||||
Coverage -.-> Cov
|
||||
Stability -.-> Stab
|
||||
```
|
||||
|
||||
### Landscape Analysis Algorithm
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph LandscapeFeatures["Landscape Feature Extraction"]
|
||||
direction TB
|
||||
|
||||
Trials[Completed Trials] --> BuildSurrogate[Build GP Surrogate]
|
||||
|
||||
BuildSurrogate --> F1[Feature 1:<br/>Smoothness]
|
||||
BuildSurrogate --> F2[Feature 2:<br/>Multimodality]
|
||||
BuildSurrogate --> F3[Feature 3:<br/>Separability]
|
||||
BuildSurrogate --> F4[Feature 4:<br/>Noise Level]
|
||||
|
||||
F1 --> Smooth[Gradient Variance<br/>Low = Smooth]
|
||||
F2 --> Modes[Local Optima Count<br/>High = Multimodal]
|
||||
F3 --> Sep[Variable Interaction<br/>Low = Separable]
|
||||
F4 --> NoiseEst[Residual Variance<br/>High = Noisy]
|
||||
end
|
||||
|
||||
subgraph Classification["Strategy Classification"]
|
||||
direction TB
|
||||
|
||||
Smooth --> C1{Smoothness<br/>< 0.3?}
|
||||
Modes --> C2{Modes > 3?}
|
||||
Sep --> C3{Separability<br/>< 0.5?}
|
||||
NoiseEst --> C4{Noise > 0.2?}
|
||||
|
||||
C1 -->|Yes| TPEScore[TPE Score +2]
|
||||
C1 -->|No| TPEScore
|
||||
|
||||
C2 -->|Yes| CMAScore[CMA-ES Score +2]
|
||||
C2 -->|No| CMAScore
|
||||
|
||||
C3 -->|Yes| GPScore[GP Score +1]
|
||||
C3 -->|No| GPScore
|
||||
|
||||
C4 -->|Yes| NSGAScore[NSGA-II Score +2]
|
||||
C4 -->|No| NSGAScore
|
||||
|
||||
TPEScore --> MaxScore{Max score?}
|
||||
CMAScore --> MaxScore
|
||||
GPScore --> MaxScore
|
||||
NSGAScore --> MaxScore
|
||||
|
||||
MaxScore --> Winner[Winning Strategy]
|
||||
end
|
||||
|
||||
LandscapeFeatures --> Classification
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Protocol 11: Multi-Objective Support
|
||||
|
||||
### Multi-Objective Decision Tree
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
Start[Component Initialization] --> CheckObj{Check Study<br/>Objectives}
|
||||
|
||||
CheckObj -->|len == 1| SingleObj[Single-Objective Mode]
|
||||
CheckObj -->|len > 1| MultiObj[Multi-Objective Mode]
|
||||
|
||||
%% Single-objective path
|
||||
subgraph SinglePath["Single-Objective Operations"]
|
||||
SingleObj --> API1[Use Singular API]
|
||||
API1 --> UseBestValue[study.best_value]
|
||||
API1 --> UseBestTrial[study.best_trial]
|
||||
API1 --> UseTrialValue[trial.value]
|
||||
API1 --> EnableChar[Enable Characterization]
|
||||
API1 --> EnableStag[Enable Stagnation Detection]
|
||||
end
|
||||
|
||||
%% Multi-objective path
|
||||
subgraph MultiPath["Multi-Objective Operations"]
|
||||
MultiObj --> API2[Use Plural API]
|
||||
API2 --> UseBestTrials[study.best_trials<br/>Pareto Front]
|
||||
API2 --> UseTrialValues[trial.values<br/>List of objectives]
|
||||
API2 --> SkipChar[Skip Characterization<br/>Return confidence=1.0]
|
||||
API2 --> SkipStag[Skip Stagnation Detection<br/>Return False]
|
||||
API2 --> UseSampler[Force NSGA-II Sampler]
|
||||
end
|
||||
|
||||
%% Component checks
|
||||
subgraph Components["Component-Level Checks"]
|
||||
Comp1[adaptive_surrogate.py] --> CompCheck1{Multi-obj?}
|
||||
CompCheck1 -->|Yes| Return1[Return max confidence]
|
||||
CompCheck1 -->|No| Calc1[Calculate confidence]
|
||||
|
||||
Comp2[strategy_portfolio.py] --> CompCheck2{Multi-obj?}
|
||||
CompCheck2 -->|Yes| Return2[Skip stagnation]
|
||||
CompCheck2 -->|No| Calc2[Detect stagnation]
|
||||
|
||||
Comp3[realtime_tracking.py] --> CompCheck3{Multi-obj?}
|
||||
CompCheck3 -->|Yes| Write1[Write trial.values]
|
||||
CompCheck3 -->|No| Write2[Write trial.value]
|
||||
end
|
||||
|
||||
SinglePath --> Components
|
||||
MultiPath --> Components
|
||||
|
||||
Components --> End[Component Execution]
|
||||
|
||||
%% Styling
|
||||
classDef singleClass fill:#C8E6C9,stroke:#2E7D32,stroke-width:2px
|
||||
classDef multiClass fill:#BBDEFB,stroke:#1565C0,stroke-width:2px
|
||||
classDef checkClass fill:#FFF9C4,stroke:#F57F17,stroke-width:2px
|
||||
|
||||
class SinglePath,API1,UseBestValue,UseBestTrial,UseTrialValue,EnableChar,EnableStag singleClass
|
||||
class MultiPath,API2,UseBestTrials,UseTrialValues,SkipChar,SkipStag,UseSampler multiClass
|
||||
class CheckObj,CompCheck1,CompCheck2,CompCheck3 checkClass
|
||||
```
|
||||
|
||||
### Protocol 11 Compliance Checklist
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
subgraph Detection["1. Detect Objective Count"]
|
||||
D1[len study.directions] --> D2{Count?}
|
||||
D2 -->|1| SetSingle[is_multi_objective = False]
|
||||
D2 -->|>1| SetMulti[is_multi_objective = True]
|
||||
end
|
||||
|
||||
subgraph APIUsage["2. Use Correct API"]
|
||||
SetSingle --> API1[✓ study.best_value<br/>✓ study.best_trial<br/>✓ trial.value]
|
||||
SetMulti --> API2[✓ study.best_trials<br/>✓ trial.values<br/>✗ trial.value = ERROR]
|
||||
end
|
||||
|
||||
subgraph Features["3. Feature Compatibility"]
|
||||
API1 --> F1[✓ Characterization<br/>✓ Landscape Analysis<br/>✓ Stagnation Detection<br/>✓ All Strategies]
|
||||
API2 --> F2[✗ Characterization Skipped<br/>✗ Landscape Analysis Skipped<br/>✗ Stagnation Detection Skipped<br/>✓ NSGA-II Only]
|
||||
end
|
||||
|
||||
subgraph Tracking["4. Tracking Compliance"]
|
||||
F1 --> T1[Write trial.value<br/>Write best_value]
|
||||
F2 --> T2[Write trial.values<br/>Write pareto_front_size]
|
||||
end
|
||||
|
||||
subgraph Validation["5. Pre-deployment Validation"]
|
||||
T1 --> V1{Test both modes}
|
||||
T2 --> V1
|
||||
V1 --> V2[✓ Single-obj test passes]
|
||||
V1 --> V3[✓ Multi-obj test passes]
|
||||
V2 --> Pass[Protocol 11 Compliant]
|
||||
V3 --> Pass
|
||||
end
|
||||
|
||||
classDef errorClass fill:#FFCDD2,stroke:#C62828,stroke-width:2px
|
||||
classDef successClass fill:#C8E6C9,stroke:#2E7D32,stroke-width:2px
|
||||
|
||||
class API2 errorClass
|
||||
class Pass successClass
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Protocol 13: Real-Time Dashboard Tracking
|
||||
|
||||
### Tracking Workflow
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
autonumber
|
||||
participant Opt as IntelligentOptimizer
|
||||
participant Optuna
|
||||
participant Callback as RealtimeTracker<br/>(Optuna Callback)
|
||||
participant FS as File System
|
||||
participant Dashboard
|
||||
|
||||
%% Study initialization
|
||||
Opt->>Callback: Initialize Tracker
|
||||
Callback->>FS: Create intelligent_optimizer/<br/>directory
|
||||
|
||||
%% Trial execution
|
||||
loop For each trial
|
||||
Optuna->>Optuna: Sample Parameters
|
||||
Optuna->>Opt: Execute Objective Function
|
||||
Opt->>Opt: Run NX Solver
|
||||
|
||||
%% Trial completion
|
||||
Optuna->>Callback: after_trial(study, trial)
|
||||
|
||||
activate Callback
|
||||
|
||||
%% Write trial log
|
||||
Callback->>Callback: Extract trial data
|
||||
Callback->>FS: Append to trial_log.json
|
||||
Note over FS: {trial_number, params,<br/>values, state, timestamp}
|
||||
|
||||
%% Write optimizer state
|
||||
Callback->>Callback: Get current strategy/phase
|
||||
Callback->>FS: Write optimizer_state.json
|
||||
Note over FS: {current_strategy,<br/>current_phase, trial_number}
|
||||
|
||||
%% Conditional writes
|
||||
alt Characterization Phase
|
||||
Callback->>Callback: Calculate progress
|
||||
Callback->>FS: Write characterization_progress.json
|
||||
Note over FS: {confidence, convergence,<br/>exploration, trials_complete}
|
||||
end
|
||||
|
||||
alt Landscape Analysis Complete
|
||||
Callback->>Callback: Get analysis results
|
||||
Callback->>FS: Write intelligence_report.json
|
||||
Note over FS: {landscape_features,<br/>strategy_recommendation,<br/>confidence}
|
||||
end
|
||||
|
||||
alt Strategy Transition
|
||||
Callback->>Callback: Log transition
|
||||
Callback->>FS: Write strategy_transitions.json
|
||||
Note over FS: {from_strategy, to_strategy,<br/>reason, trial_number}
|
||||
end
|
||||
|
||||
deactivate Callback
|
||||
|
||||
%% Dashboard update
|
||||
FS-->>Dashboard: File Watch Event
|
||||
Dashboard->>FS: Read JSON Files
|
||||
Dashboard->>Dashboard: Parse Data
|
||||
Dashboard->>Dashboard: Update UI
|
||||
Dashboard->>Dashboard: Render Charts
|
||||
|
||||
end
|
||||
|
||||
%% Finalization
|
||||
Opt->>FS: Write optimization_summary.json
|
||||
Opt->>FS: Write final_report.md
|
||||
Dashboard->>Dashboard: Show Final Results
|
||||
```
|
||||
|
||||
### File Write Patterns
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph FileWrites["Protocol 13 File Writes"]
|
||||
direction TB
|
||||
|
||||
subgraph EveryTrial["Every Trial (after_trial)"]
|
||||
T1[trial_log.json<br/>APPEND mode]
|
||||
T2[optimizer_state.json<br/>OVERWRITE mode]
|
||||
end
|
||||
|
||||
subgraph Conditional["Conditional Writes"]
|
||||
C1{In characterization?}
|
||||
C1 -->|Yes| W1[characterization_progress.json]
|
||||
C1 -->|No| Skip1[Skip]
|
||||
|
||||
C2{Landscape analyzed?}
|
||||
C2 -->|Yes| W2[intelligence_report.json]
|
||||
C2 -->|No| Skip2[Skip]
|
||||
|
||||
C3{Strategy changed?}
|
||||
C3 -->|Yes| W3[strategy_transitions.json<br/>APPEND mode]
|
||||
C3 -->|No| Skip3[Skip]
|
||||
end
|
||||
|
||||
subgraph OnComplete["On Completion"]
|
||||
F1[optimization_summary.json]
|
||||
F2[final_report.md]
|
||||
end
|
||||
end
|
||||
|
||||
subgraph Format["JSON Format Examples"]
|
||||
direction LR
|
||||
|
||||
Ex1["trial_log.json:<br/>[{trial: 0, ...},<br/> {trial: 1, ...}]"]
|
||||
Ex2["optimizer_state.json:<br/>{strategy: 'TPE',<br/> phase: 'optimization'}"]
|
||||
Ex3["characterization_progress.json:<br/>{confidence: 0.72,<br/> trials: 15/30}"]
|
||||
end
|
||||
|
||||
EveryTrial -.-> Ex1
|
||||
EveryTrial -.-> Ex2
|
||||
Conditional -.-> Ex3
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## LLM-Assisted Workflow (Hybrid Mode)
|
||||
|
||||
### Complete LLM Workflow
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
Start([User Provides Requirements]) --> LLMStart[LLM Receives Request]
|
||||
|
||||
LLMStart --> Parse[Parse Requirements]
|
||||
Parse --> Extract[Extract Key Info:<br/>- Objectives<br/>- Design variables<br/>- Constraints<br/>- Preferences]
|
||||
|
||||
Extract --> DesignChoice{Design<br/>Decisions<br/>Needed?}
|
||||
|
||||
%% Interactive clarification
|
||||
DesignChoice -->|Yes| AskUser[Ask User for Clarification]
|
||||
AskUser --> UserResponse[User Provides Details]
|
||||
UserResponse --> Extract
|
||||
|
||||
%% Generate configuration
|
||||
DesignChoice -->|No| GenerateConfig[Generate Config]
|
||||
|
||||
subgraph ConfigGen["Configuration Generation"]
|
||||
GenerateConfig --> SetObjectives[Define Objectives<br/>with extractors]
|
||||
SetObjectives --> SetDesignVars[Define Design Variables<br/>with bounds]
|
||||
SetDesignVars --> SetConstraints[Define Constraints]
|
||||
SetConstraints --> SetProtocols[Configure Protocols]
|
||||
|
||||
SetProtocols --> P10Config{Enable<br/>Protocol 10?}
|
||||
P10Config -->|Yes| CharConfig[Set characterization params]
|
||||
P10Config -->|No| SkipP10[Use default sampler]
|
||||
|
||||
CharConfig --> FinalConfig
|
||||
SkipP10 --> FinalConfig[Assemble JSON]
|
||||
end
|
||||
|
||||
FinalConfig --> ValidateConfig{Valid<br/>Configuration?}
|
||||
|
||||
ValidateConfig -->|No| FixConfig[Fix Issues]
|
||||
FixConfig --> FinalConfig
|
||||
|
||||
ValidateConfig -->|Yes| WriteConfig[Write optimization_config.json]
|
||||
|
||||
%% Create extractor if needed
|
||||
WriteConfig --> NeedExtractor{Custom<br/>Extractor<br/>Needed?}
|
||||
|
||||
NeedExtractor -->|Yes| CreateExtractor
|
||||
|
||||
subgraph ExtractorGen["Extractor Generation"]
|
||||
CreateExtractor[Generate Extractor Code] --> BaseClass[Inherit from BaseExtractor]
|
||||
BaseClass --> ExtractMethod[Implement extract method]
|
||||
ExtractMethod --> RegisterExt[Register in library]
|
||||
end
|
||||
|
||||
RegisterExt --> WriteExtractor[Write extractor.py]
|
||||
|
||||
NeedExtractor -->|No| SkipExtractor[Use built-in extractors]
|
||||
|
||||
WriteExtractor --> SetupStudy
|
||||
SkipExtractor --> SetupStudy
|
||||
|
||||
%% Study setup
|
||||
subgraph StudySetup["Study Setup"]
|
||||
SetupStudy[Create Study Directory] --> CreateFolders[Create 1_setup/ & 2_results/]
|
||||
CreateFolders --> CopyModel[Copy CAD model to 1_setup/model/]
|
||||
CopyModel --> CopyConfig[Copy config to 1_setup/]
|
||||
CopyConfig --> CreateRunner[Create run_optimization.py]
|
||||
end
|
||||
|
||||
CreateRunner --> ReadyToRun[Study Ready to Run]
|
||||
|
||||
ReadyToRun --> UserConfirm{User<br/>Approves?}
|
||||
|
||||
UserConfirm -->|No| Revise[User Provides Feedback]
|
||||
Revise --> Parse
|
||||
|
||||
UserConfirm -->|Yes| RunOptimization[Execute Optimization]
|
||||
|
||||
%% Optimization execution
|
||||
RunOptimization --> Protocol10Flow[Follow Protocol 10 Workflow]
|
||||
Protocol10Flow --> Protocol13Track[Protocol 13 Tracking Active]
|
||||
Protocol13Track --> Monitor[LLM Can Monitor Progress]
|
||||
|
||||
Monitor --> Complete{Optimization<br/>Complete?}
|
||||
Complete -->|No| Monitor
|
||||
Complete -->|Yes| Analyze[Analyze Results]
|
||||
|
||||
%% Result analysis
|
||||
subgraph ResultAnalysis["Result Analysis"]
|
||||
Analyze --> ReadDB[Read study.db]
|
||||
ReadDB --> ReadReports[Read JSON reports]
|
||||
ReadReports --> ExtractInsights[Extract Insights:<br/>- Best solution<br/>- Trade-offs<br/>- Landscape type<br/>- Convergence]
|
||||
end
|
||||
|
||||
ExtractInsights --> PresentResults[Present Results to User]
|
||||
PresentResults --> End([Workflow Complete])
|
||||
|
||||
%% Styling
|
||||
classDef llmClass fill:#FFEBEE,stroke:#C62828,stroke-width:2px
|
||||
classDef userClass fill:#E3F2FD,stroke:#1976D2,stroke-width:2px
|
||||
classDef processClass fill:#E8F5E9,stroke:#2E7D32,stroke-width:2px
|
||||
|
||||
class LLMStart,Parse,Extract,GenerateConfig,CreateExtractor,Analyze,ExtractInsights,PresentResults llmClass
|
||||
class Start,AskUser,UserResponse,UserConfirm,Revise,End userClass
|
||||
class ConfigGen,ExtractorGen,StudySetup,ResultAnalysis processClass
|
||||
```
|
||||
|
||||
### LLM Decision Points
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph Decisions["LLM Decision Points During Setup"]
|
||||
direction TB
|
||||
|
||||
D1[Objective Type]
|
||||
D1 --> D1A{Minimize/Maximize?}
|
||||
D1A --> D1B{Target value?}
|
||||
D1B --> D1C{Single/Multi-objective?}
|
||||
|
||||
D2[Design Variables]
|
||||
D2 --> D2A{Continuous/Integer/Categorical?}
|
||||
D2A --> D2B{Physical bounds?}
|
||||
D2B --> D2C{Engineering constraints?}
|
||||
|
||||
D3[Extractor Selection]
|
||||
D3 --> D3A{Built-in available?}
|
||||
D3A -->|No| D3B[Generate custom]
|
||||
D3A -->|Yes| D3C[Use built-in]
|
||||
|
||||
D4[Protocol Configuration]
|
||||
D4 --> D4A{Enable IMSO?}
|
||||
D4A -->|Yes| D4B[Set characterization<br/>min/max trials]
|
||||
D4A -->|No| D4C[Use default sampler]
|
||||
|
||||
D5[Constraint Handling]
|
||||
D5 --> D5A{Hard constraints?}
|
||||
D5A -->|Yes| D5B[Pruning callbacks]
|
||||
D5A -->|No| D5C[Soft penalties]
|
||||
end
|
||||
|
||||
subgraph Clarification["When LLM Asks User"]
|
||||
Q1[Ambiguous objective<br/>'Optimize performance']
|
||||
Q2[Unknown bounds<br/>'What is max thickness?']
|
||||
Q3[Missing constraints<br/>'Any mass limits?']
|
||||
Q4[Extractor uncertainty<br/>'How to extract stress?']
|
||||
end
|
||||
|
||||
D1C -.->|Unclear| Q1
|
||||
D2B -.->|Unknown| Q2
|
||||
D2C -.->|Unclear| Q3
|
||||
D3A -.->|Unsure| Q4
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Integration: All Protocols Together
|
||||
|
||||
### Complete Optimization Flow
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
Start([User Starts]) --> Config[Load Configuration]
|
||||
|
||||
Config --> DetectMode{LLM-assisted?}
|
||||
|
||||
%% LLM path
|
||||
DetectMode -->|Yes| LLMSetup[LLM generates config<br/>and extractors]
|
||||
LLMSetup --> InitOpt
|
||||
|
||||
%% Manual path
|
||||
DetectMode -->|No| ManualConfig[User-written config]
|
||||
ManualConfig --> InitOpt
|
||||
|
||||
%% Initialization
|
||||
InitOpt[Initialize IntelligentOptimizer] --> P13Init[Protocol 13: Initialize Tracking]
|
||||
P13Init --> DetectObj{Multi-objective?}
|
||||
|
||||
%% Single-objective path
|
||||
DetectObj -->|No| P10Start[Protocol 10: Start IMSO]
|
||||
|
||||
subgraph P10["Protocol 10 Execution"]
|
||||
P10Start --> CharPhase[Characterization Phase<br/>Random/Sobol sampling]
|
||||
CharPhase --> CharProgress{Confidence ≥ 0.85?}
|
||||
CharProgress -->|No| CharPhase
|
||||
CharProgress -->|Yes| P13Char[P13: Write characterization_progress.json]
|
||||
|
||||
P13Char --> Landscape[Landscape Analysis]
|
||||
Landscape --> P13Intel[P13: Write intelligence_report.json]
|
||||
P13Intel --> SelectStrat[Select Strategy]
|
||||
SelectStrat --> P13Trans[P13: Write strategy_transitions.json]
|
||||
|
||||
P13Trans --> OptPhase[Optimization Phase<br/>Selected strategy]
|
||||
OptPhase --> StagnCheck{Stagnation?}
|
||||
StagnCheck -->|Yes| SwitchStrat[Switch Strategy]
|
||||
SwitchStrat --> P13Trans
|
||||
StagnCheck -->|No| OptPhase
|
||||
end
|
||||
|
||||
%% Multi-objective path
|
||||
DetectObj -->|Yes| P11Start[Protocol 11: Multi-Obj Mode]
|
||||
|
||||
subgraph P11["Protocol 11 Execution"]
|
||||
P11Start --> SkipChar[Skip Characterization]
|
||||
SkipChar --> SetNSGAII[Set NSGA-II Sampler]
|
||||
SetNSGAII --> OptPhaseMulti[Optimization Phase<br/>NSGA-II only]
|
||||
end
|
||||
|
||||
%% Trial execution (common)
|
||||
OptPhase --> RunTrial
|
||||
OptPhaseMulti --> RunTrial
|
||||
|
||||
subgraph TrialExec["Trial Execution"]
|
||||
RunTrial[Execute Trial] --> NXUpdate[Update NX Model]
|
||||
NXUpdate --> NXSolve[Run FEA Solver]
|
||||
NXSolve --> ExtractResults[Extract Results<br/>Using Extractors]
|
||||
ExtractResults --> CalcObj[Calculate Objectives]
|
||||
CalcObj --> OptunaUpdate[Update Optuna Study]
|
||||
end
|
||||
|
||||
%% Tracking after trial
|
||||
OptunaUpdate --> P13Trial[P13: after_trial callback]
|
||||
|
||||
subgraph P13Track["Protocol 13 Tracking"]
|
||||
P13Trial --> WriteLog[Append trial_log.json]
|
||||
WriteLog --> WriteState[Write optimizer_state.json]
|
||||
WriteState --> DashUpdate[Dashboard Auto-Updates]
|
||||
end
|
||||
|
||||
DashUpdate --> MoreTrials{More trials?}
|
||||
MoreTrials -->|Yes| RunTrial
|
||||
MoreTrials -->|No| Finalize
|
||||
|
||||
%% Finalization
|
||||
Finalize[Generate Final Report] --> P13Final[P13: Write optimization_summary.json]
|
||||
P13Final --> End([Complete])
|
||||
|
||||
%% Styling
|
||||
classDef p10Class fill:#E8F5E9,stroke:#2E7D32,stroke-width:2px
|
||||
classDef p11Class fill:#BBDEFB,stroke:#1565C0,stroke-width:2px
|
||||
classDef p13Class fill:#FFE082,stroke:#F57F00,stroke-width:2px
|
||||
|
||||
class P10,P10Start,CharPhase,CharProgress,Landscape,SelectStrat,OptPhase,StagnCheck,SwitchStrat p10Class
|
||||
class P11,P11Start,SkipChar,SetNSGAII,OptPhaseMulti p11Class
|
||||
class P13Init,P13Char,P13Intel,P13Trans,P13Trial,P13Track,P13Final,DashUpdate p13Class
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
For detailed architecture diagrams, see:
|
||||
- [Architecture Overview](architecture_overview.md) - System components and data flow
|
||||
|
||||
For implementation details, see:
|
||||
- [Protocol Specifications](../01_PROTOCOLS.md)
|
||||
- [Documentation Index](../00_INDEX.md)
|
||||
Reference in New Issue
Block a user