Phase 1-7 of Canvas V4 Ralph Loop implementation:
Backend:
- Add /api/files routes for browsing model files
- Add /api/nx routes for NX model introspection
- Add NXIntrospector service to discover expressions and extractors
- Add health check with database status
Frontend:
- Add FileBrowser component for selecting .sim/.prt/.fem files
- Add IntrospectionPanel to discover expressions and extractors
- Update NodeConfigPanel with browse and introspect buttons
- Update schema with NODE_HANDLES for proper flow direction
- Update validation for correct DesignVar -> Model -> Solver flow
- Update useCanvasStore.addNode() to accept custom data
Flow correction: Design Variables now connect TO Model (as source),
not FROM Model. This matches the actual data flow in optimization.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Major changes:
- Dashboard: WebSocket-based chat with session management
- Dashboard: New chat components (ChatPane, ChatInput, ModeToggle)
- Dashboard: Enhanced UI with parallel coordinates chart
- MCP Server: New atomizer-tools server for Claude integration
- Extractors: Enhanced Zernike OPD extractor
- Reports: Improved report generator
New studies (configs and scripts only):
- M1 Mirror: Cost reduction campaign studies
- Simple Beam, Simple Bracket, UAV Arm studies
Note: Large iteration data (2_iterations/, best_design_archive/)
excluded via .gitignore - kept on local Gitea only.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Update README.md with LLM assistant section
- Create optimization_memory JSONL structure
- Move implementation plans from skills/modules to docs/plans
- Verify all imports work correctly
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Study Interview Mode is now the DEFAULT for all study creation requests.
This intelligent Q&A system guides users through optimization setup with:
- 7-phase interview flow: introspection → objectives → constraints → design_variables → validation → review → complete
- Material-aware validation with 12 materials and fuzzy name matching
- Anti-pattern detection for 12 common mistakes (mass-no-constraint, stress-over-yield, etc.)
- Auto extractor mapping E1-E24 based on goal keywords
- State persistence with JSON serialization and backup rotation
- StudyBlueprint generation with full validation
Triggers: "create a study", "new study", "optimize this", any study creation intent
Skip with: "skip interview", "quick setup", "manual config"
Components:
- StudyInterviewEngine: Main orchestrator
- QuestionEngine: Conditional logic evaluation
- EngineeringValidator: MaterialsDatabase + AntiPatternDetector
- InterviewPresenter: Markdown formatting for Claude
- StudyBlueprint: Validated configuration output
- InterviewState: Persistent state management
All 129 tests passing.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Self-Aware Turbo v3 optimization validated on M1 Mirror flat back:
- Best WS: 205.58 (12% better than previous best 218.26)
- 100% feasibility rate, 100% unique designs
- Uses 556 training samples from V5-V8 campaign data
Key innovations in V9:
- Adaptive exploration schedule (15% → 8% → 3%)
- Mass threshold at 118 kg (optimal sweet spot)
- 70% exploitation near best design
- Seeded with best known design from V7
- Ensemble surrogate with R²=0.99
Updated documentation:
- SYS_16: SAT protocol updated to v3.0 VALIDATED
- Cheatsheet: Added SAT v3 as recommended method
- Context: Updated protocol overview
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Added new PART 8: Self-Aware Turbo (SAT) - Validated Breakthrough
- Explains ensemble surrogate with epistemic uncertainty
- Documents OOD detection and adaptive exploration schedule
- Includes V9 results: WS=205.58 (best ever)
- Added SAT sound bites for podcast
- Updated document to 12 sections
Generated with Claude Code
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Key changes based on feedback:
- Reposition as "optimizer & NX configurator" not "LLM-first"
- Add Part 2: Study Characterization & Performance Learning
- Add Part 3: Protocol Evolution workflow (Research → Review → Approve)
- Add Part 4: MCP-first development approach with documentation hierarchy
- Emphasize simulation optimization over CAD/mesh concerns
- Add LAC knowledge accumulation for parameter-performance relationships
- Add privilege levels for protocol approval (user/power_user/admin)
- Update sound bites and core messaging
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add TrialManager (trial_manager.py) for consistent trial_NNNN naming
- Add DashboardDB (dashboard_db.py) for Optuna-compatible database schema
- Update CLAUDE.md with trial management documentation
- Update ATOMIZER_CONTEXT.md with v1.8 trial system
- Update cheatsheet v2.2 with new utilities
- Update SYS_14 protocol to v2.3 with TrialManager integration
- Add LAC learnings for trial management patterns
- Add archive/README.md for deprecated code policy
Key principles:
- Trial numbers NEVER reset (monotonic)
- Folders NEVER get overwritten
- Database always synced with filesystem
- Surrogate predictions are NOT trials (only FEA results)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Major improvements to Zernike WFE visualization:
- Add ZernikeDashboardInsight: Unified dashboard with all orientations (40°, 60°, 90°)
on one page with light theme and executive summary
- Add OPD method toggle: Switch between Standard (Z-only) and OPD (X,Y,Z) methods
in ZernikeWFEInsight with interactive buttons
- Add lateral displacement maps: Visualize X,Y displacement for each orientation
- Add displacement component views: Toggle between WFE, ΔX, ΔY, ΔZ in relative views
- Add metrics comparison table showing both methods side-by-side
New extractors:
- extract_zernike_figure.py: ZernikeOPDExtractor using BDF geometry interpolation
- extract_zernike_opd.py: Parabola-based OPD with focal length
Key finding: OPD method gives 8-11% higher WFE values than Standard method
(more conservative/accurate for surfaces with lateral displacement under gravity)
Documentation updates:
- SYS_12: Added E22 ZernikeOPD as recommended method
- SYS_16: Added ZernikeDashboard, updated ZernikeWFE with OPD features
- Cheatsheet: Added Zernike method comparison table
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
New Tools (tools/):
- analyze_study.py: Generate comprehensive optimization reports
- find_best_iteration.py: Find best iteration folder, optionally copy it
- archive_best_design.py: Archive best design to 3_results/best_design_archive/<timestamp>/
Protocol Updates:
- OP_02_RUN_OPTIMIZATION.md v1.1: Add mandatory archive_best_design step
in Post-Run Actions. This MUST be done after every optimization run.
V14 Updates:
- run_optimization.py: Auto-archive best design at end of optimization
- optimization_config.json: Expand bounds for V14 continuation
- lateral_outer_angle: min 13->11 deg (was at 4.7%)
- lateral_inner_pivot: min 7->5 mm (was at 8.1%)
- lateral_middle_pivot: max 23->27 mm (was at 99.4%)
- whiffle_min: max 60->72 mm (was at 96.3%)
Usage:
python tools/analyze_study.py m1_mirror_adaptive_V14
python tools/find_best_iteration.py m1_mirror_adaptive_V14
python tools/archive_best_design.py m1_mirror_adaptive_V14
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
New E11 Part Mass Extractor:
- Add nx_journals/extract_part_mass_material.py - NX journal using
NXOpen.MeasureManager.NewMassProperties() for accurate geometry-based mass
- Add optimization_engine/extractors/extract_part_mass_material.py - Python
wrapper that reads JSON output from journal
- Add E11 entry to extractors/catalog.json
Documentation Updates:
- SYS_12_EXTRACTOR_LIBRARY.md: Add mass accuracy warning noting pyNastran
get_mass_breakdown() under-reports ~7% on hex-dominant meshes with
tet/pyramid fill elements. E11 (geometry .prt) should be preferred over
E4 (BDF) unless material is overridden at FEM level.
- 01_CHEATSHEET.md: Add mass extraction tip
V14 Config:
- Expand design variable bounds (blank_backface_angle max 4.5°,
whiffle_triangle_closeness max 80mm, whiffle_min max 60mm)
Testing showed:
- E11 from .prt: 97.66 kg (accurate - matches NX GUI)
- E4 pyNastran get_mass_breakdown(): 90.73 kg (~7% under-reported)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Comprehensive guide teaching users how to interact with Atomizer so that
the learning system evolves correctly. Covers:
- The right mindset (colleague, not tool)
- Starting sessions with proper context
- Communicating goals, constraints, preferences
- Creating and running optimization studies
- Analyzing and validating results
- Reporting errors effectively
- Contributing to LAC (recording insights, outcomes, workarounds)
- Ending sessions properly to capture learnings
Includes:
- Mermaid diagrams for learning loop and flows
- Good vs bad examples for every interaction type
- Complete example session transcript
- Quick reference card for common patterns
- Golden rules summary
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add persistent knowledge system that enables Atomizer to learn from every
session and improve over time.
## New Files
- knowledge_base/lac.py: LAC class with optimization memory, session insights,
and skill evolution tracking
- knowledge_base/__init__.py: Package initialization
- .claude/skills/modules/learning-atomizer-core.md: Full LAC skill documentation
- docs/07_DEVELOPMENT/ATOMIZER_CLAUDE_CODE_INSTRUCTIONS.md: Master instructions
## Updated Files
- CLAUDE.md: Added LAC section, communication style, AVERVS execution framework,
error classification, and "Atomizer Claude" identity
- 00_BOOTSTRAP.md: Added session startup/closing checklists with LAC integration
- 01_CHEATSHEET.md: Added LAC CLI and Python API quick reference
- 02_CONTEXT_LOADER.md: Added LAC query section and anti-pattern
## LAC Features
- Query similar past optimizations before starting new ones
- Record insights (failures, success patterns, workarounds)
- Record optimization outcomes for future reference
- Suggest protocol improvements based on discoveries
- Simple JSONL storage (no database required)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The Method Selector now uses relative accuracy thresholds to assess
NN suitability by comparing NN error to problem variability (CV ratio).
NNQualityAssessor features:
- Physics-based objective classification (linear, smooth, nonlinear, chaotic)
- CV ratio computation: nn_error / coefficient_of_variation
- Turbo suitability score based on relative thresholds
- Data collection from validation_report.json, turbo_report.json, and study.db
Quality thresholds by objective type:
- Linear (mass, volume): max 2% error, CV ratio < 0.5
- Smooth (frequency): max 5% error, CV ratio < 1.0
- Nonlinear (stress, stiffness): max 10% error, CV ratio < 2.0
- Chaotic (contact, buckling): max 20% error, CV ratio < 3.0
CLI output now includes:
- Per-objective NN quality table with error, CV, ratio, and quality indicator
- Turbo suitability and hybrid suitability percentages
- Warnings when NN error exceeds physics-based thresholds
Updated SYS_15_METHOD_SELECTOR.md to v2.0 with full NN Quality Assessment documentation.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Phase 2 - Structural Analysis:
- extract_principal_stress: σ1, σ2, σ3 principal stresses from OP2
- extract_strain_energy: Element and total strain energy
- extract_spc_forces: Reaction forces at boundary conditions
Phase 3 - Multi-Physics:
- extract_temperature: Nodal temperatures from thermal OP2 (SOL 153/159)
- extract_temperature_gradient: Thermal gradient approximation
- extract_heat_flux: Element heat flux from thermal analysis
- extract_modal_mass: Modal effective mass from F06 (SOL 103)
- get_first_frequency: Convenience function for first natural frequency
Documentation:
- Updated SYS_12_EXTRACTOR_LIBRARY.md with E12-E18 specifications
- Updated NX_OPEN_AUTOMATION_ROADMAP.md marking Phase 3 complete
- Added test_phase3_extractors.py for validation
All extractors follow consistent API pattern returning Dict with
success, data, and error fields for robust error handling.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Dashboard enhancements:
- Add Analysis page with tabs: Overview, Parameters, Pareto, Correlations, Constraints, Surrogate, Runs
- Add PlotlyCorrelationHeatmap for parameter-objective correlation analysis
- Add PlotlyFeasibilityChart for constraint satisfaction visualization
- Add PlotlySurrogateQuality for FEA vs NN prediction comparison
- Add PlotlyRunComparison for comparing optimization runs within a study
Real-time improvements:
- Replace watchdog file-watching with SQLite database polling for better Windows reliability
- Add DatabasePoller class with 2-second polling interval
- Enhanced WebSocket messages: trial_completed, new_best, pareto_update, progress
Desktop notifications:
- Add useNotifications hook using Web Notifications API
- Add NotificationSettings toggle component
- Notify users when new best solutions are found
Config editor:
- Add PUT /studies/{study_id}/config endpoint with auto-backup
- Add ConfigEditor modal with tabs: General, Variables, Objectives, Settings, JSON
- Prevents editing while optimization is running
Enhanced Pareto visualization:
- Add dark mode styling with transparent backgrounds
- Add stats bar showing Pareto, FEA, NN, and infeasible counts
- Add Pareto front connecting line for 2D view
- Add table showing top 10 Pareto-optimal solutions
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add embedded Claude Code terminal with xterm.js for full CLI experience
- Create WebSocket PTY backend for real-time terminal communication
- Add terminal status endpoint to check CLI availability
- Update dashboard to use Claude Code terminal instead of API chat
- Add optimization control panel with start/stop/validate actions
- Add study context provider for global state management
- Update frontend with new dependencies (xterm.js addons)
- Comprehensive README documentation for all new features
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add ConvergencePlot component with running best, statistics, gradient fill
- Add ParameterImportanceChart with Pearson correlation analysis
- Add StudyReportViewer with KaTeX math rendering and full markdown support
- Update pruning endpoint to query Optuna database directly
- Add /report endpoint for STUDY_REPORT.md files
- Fix chart data transformation for single/multi-objective studies
- Update Protocol 13 documentation with new components
- Update generate-report skill with dashboard integration
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Problem:
When running optimization studies with multiple solutions (e.g., static + modal),
NX opens solution monitor windows for each trial. These windows superpose and cause
usability issues during long optimization runs.
Solution:
- Automatically disable solution monitor when solving all solutions (solution_name=None)
- Loop through all solutions and set "solution monitor" property to False
- Implemented in solve_simulation.py before solve execution (lines 271-295)
- Includes error handling with graceful fallback
Benefits:
- No monitor window pile-up during optimization studies
- Better performance (no GUI overhead)
- No user configuration required - works automatically
- Based on user-recorded journal (journal_monitor_window_off.py)
Documentation:
- Updated docs/NX_MULTI_SOLUTION_PROTOCOL.md with solution monitor control section
- Added implementation details and when the feature activates
- Cross-referenced user's recorded journal
Implementation: optimization_engine/solve_simulation.py
Documentation: docs/NX_MULTI_SOLUTION_PROTOCOL.md
Reference: nx_journals/user_generated_journals/journal_monitor_window_off.py
Migrate drone_gimbal_arm study as reference implementation for Phase 1.3 logging system.
Changes:
- Replace all print() statements with logger calls throughout run_optimization.py
- Add logger.trial_start() and logger.trial_complete() for structured trial logging
- Use logger.trial_failed() for error handling with full tracebacks
- Add logger.study_start() and logger.study_complete() for lifecycle logging
- Replace constraint violation prints with logger.warning()
- Create comprehensive LOGGING_MIGRATION_GUIDE.md with before/after examples
Benefits:
- Color-coded console output (green INFO, yellow WARNING, red ERROR)
- Automatic file logging to 2_results/optimization.log with rotation (50MB, 3 backups)
- Structured format with timestamps for dashboard integration
- Professional error handling with exc_info=True
- Reference implementation for migrating remaining studies
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Implements JSON Schema validation for optimization configurations to ensure
consistency across all studies and prevent configuration errors.
Added:
- optimization_engine/schemas/optimization_config_schema.json
- Comprehensive schema for Protocol 10 & 11 configurations
- Validates objectives, constraints, design variables, simulation settings
- Enforces standard field names (goal, bounds, parameter, threshold)
- optimization_engine/config_manager.py
- ConfigManager class with schema validation
- CLI tool: python config_manager.py <config.json>
- Type-safe accessor methods for config elements
- Custom validations: bounds check, multi-objective consistency, location check
- optimization_engine/schemas/README.md
- Complete documentation of standard configuration format
- Validation examples and common error fixes
- Migration guidance for legacy configs
- docs/07_DEVELOPMENT/Phase_1_2_Implementation_Plan.md
- Detailed implementation plan for remaining Phase 1.2 tasks
- Migration tool design, integration guide, testing plan
Testing:
- Validated drone_gimbal_arm_optimization config successfully
- ConfigManager works with drone_gimbal format (new standard)
- Identifies legacy format issues in bracket studies
Standards Established:
- Configuration location: studies/{name}/1_setup/
- Objective direction: "goal" not "type"
- Design var bounds: "bounds": [min, max] not "min"/"max"
- Design var name: "parameter" not "name"
- Constraint threshold: "threshold" not "value"
Next Steps (Phase 1.2.1+):
- Config migration tool for legacy studies
- Integration with run_optimization.py
- Update create-study Claude skill with schema reference
- Migrate bracket studies to new format
Relates to: Phase 1.2 MVP Development Plan
🤖 Generated with Claude Code
Co-Authored-By: Claude <noreply@anthropic.com>
MAJOR ARCHITECTURE REFACTOR - Clean Study Folders
Problem Identified by User:
"My study folder is a mess, why? I want some order and real structure to develop
an insanely good engineering software that evolve with time."
- Every substudy was generating duplicate extractor code
- Study folders polluted with reusable library code (generated_extractors/, generated_hooks/)
- No code reuse across studies
- Not production-grade architecture
Solution - Centralized Library System:
Implemented smart library with signature-based deduplication:
- Core extractors in optimization_engine/extractors/
- Studies only store metadata (extractors_manifest.json)
- Clean separation: studies = data, core = code
Architecture:
BEFORE (BAD):
studies/my_study/
generated_extractors/ ❌ Code pollution!
extract_displacement.py
extract_von_mises_stress.py
generated_hooks/ ❌ Code pollution!
llm_workflow_config.json
results.json
AFTER (GOOD):
optimization_engine/extractors/ ✓ Core library
extract_displacement.py
extract_stress.py
catalog.json
studies/my_study/
extractors_manifest.json ✓ Just references!
llm_workflow_config.json ✓ Config
optimization_results.json ✓ Results
New Components:
1. ExtractorLibrary (extractor_library.py)
- Signature-based deduplication
- Centralized catalog (catalog.json)
- Study manifest generation
- Reusability across all studies
2. Updated ExtractorOrchestrator
- Uses core library instead of per-study generation
- Creates manifest instead of copying code
- Backward compatible (legacy mode available)
3. Updated LLMOptimizationRunner
- Removed generated_extractors/ directory creation
- Removed generated_hooks/ directory creation
- Uses core library exclusively
4. Updated Tests
- Verifies extractors_manifest.json exists
- Checks for clean study folder structure
- All 18/18 checks pass
Results:
Study folders NOW ONLY contain:
✓ extractors_manifest.json - references to core library
✓ llm_workflow_config.json - study configuration
✓ optimization_results.json - optimization results
✓ optimization_history.json - trial history
✓ .db file - Optuna database
Core library contains:
✓ extract_displacement.py - reusable across ALL studies
✓ extract_von_mises_stress.py - reusable across ALL studies
✓ extract_mass.py - reusable across ALL studies
✓ catalog.json - tracks all extractors with signatures
Benefits:
- Clean, professional study folder structure
- Code reuse eliminates duplication
- Library grows over time, studies stay clean
- Production-grade architecture
- "Insanely good engineering software that evolves with time"
Testing:
E2E test passes with clean folder structure
- No generated_extractors/ pollution
- Manifest correctly references library
- Core library populated with reusable extractors
- Study folder professional and minimal
Documentation:
- Added comprehensive architecture doc (docs/ARCHITECTURE_REFACTOR_NOV17.md)
- Includes migration guide
- Documents future work (hooks library, versioning, CLI tools)
Next Steps:
- Apply same architecture to hooks library
- Add auto-generated documentation for library
- Implement versioning for reproducibility
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
