Commit Graph

88 Commits

Author SHA1 Message Date
b448ca6268 auto: daily sync 2026-02-25 08:00:14 +00:00
6d443df3ec Remap channels: project-dashboard→feed, add #reports channel 2026-02-17 02:08:56 +00:00
97fe055b8d Add plan 13: Taskboard/Kanban Dynamic Project Orchestration 2026-02-17 01:27:54 +00:00
3184eb0d0e Add doc 12: Context lifecycle management — condensation, threads, staleness 2026-02-16 02:26:19 +00:00
7086f9fbdf Add doc 11: HQ improvements plan from Bhanu video analysis 2026-02-16 01:19:27 +00:00
cf82de4f06 docs: add HQ multi-agent framework documentation from PKM
- Project plan, agent roster, architecture, roadmap
- Decision log, full system plan, Discord setup/migration guides
- System implementation status (as-built)
- Cluster pivot history
- Orchestration engine plan (Phases 1-4)
- Webster and Auditor reviews
2026-02-15 21:44:07 +00:00
d6a1d6eee1 auto: daily sync 2026-02-15 08:00:21 +00:00
57130ccfbc docs: add nightly memory digestion methodology 2026-02-12 14:20:57 +00:00
04f06766a0 docs: Atomizer HQ Dashboard — full plan (CEO-requested)
Five-pane architecture:
- Project Blueprint (CONTEXT.md → live view)
- Study Tracker (enhanced real-time monitoring)
- Command Center (remote NX execution from browser)
- Agent Console (interact with HQ agents)
- Reports & Export (PDF/HTML generation)

Phased implementation: D1-D5 (7-12 weeks total, MVP at D3)
Extends existing atomizer-dashboard (no rewrite)
Progressive: file-based bridge → WebSocket → NX MCP
2026-02-11 18:32:54 +00:00
857c01e7ca chore: major repo cleanup - remove dead code and cruft
Remove ~24K lines of dead code for a lean rebuild foundation:

- Remove atomizer-field/ (neural field predictor experiment, concept archived in docs)
- Remove generated_extractors/, generated_hooks/ (legacy generator outputs)
- Remove optimization_validation/ (empty skeleton)
- Remove reports/ (superseded by optimization_engine/reporting/)
- Remove root-level stale files: DEVELOPMENT.md, INSTALL_INSTRUCTIONS.md,
  config.py, atomizer_paths.py, optimization_config.json, train_neural.bat,
  generate_training_data.py, run_training_fea.py, migrate_imports.py
- Update .gitignore for introspection caches and insight outputs

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-09 14:26:37 -05:00
8d9d55356c docs: Archive stale docs and create Atomizer-HQ agent documentation
Archive Management:
- Moved RALPH_LOOP, CANVAS, and dashboard implementation plans to archive/review/ for CEO review
- Moved completed restructuring plan and protocol v1 to archive/historical/
- Moved old session summaries to archive/review/

New HQ Documentation (docs/hq/):
- README.md: Overview of Atomizer-HQ multi-agent optimization team
- PROJECT_STRUCTURE.md: Standard KB-integrated project layout with Hydrotech reference
- KB_CONVENTIONS.md: Knowledge Base accumulation principles with generation tracking
- AGENT_WORKFLOWS.md: Project lifecycle phases and agent handoffs (OP_09 integration)
- STUDY_CONVENTIONS.md: Technical study execution standards and atomizer_spec.json format

Index Update:
- Reorganized docs/00_INDEX.md with HQ docs prominent
- Updated structure to reflect new agent-focused organization
- Maintained core documentation access for engineers

No files deleted, only moved to appropriate archive locations.
2026-02-09 02:48:35 +00:00
af195c3a75 docs: add handoff document for trajectory optimization setup 2026-01-29 16:46:55 +00:00
5d69b3bd10 docs: add Zernike trajectory method documentation + example config 2026-01-29 16:32:05 +00:00
b3f3329c79 docs: update status + next sprint focus (Draft+Publish, Create Wizard) 2026-01-29 03:10:07 +00:00
bb27f3fb00 docs: add QUICK_REF + workflow OS + 2026Q1 roadmap 2026-01-29 02:28:02 +00:00
a26914bbe8 feat: Add Studio UI, intake system, and extractor improvements
Dashboard:
- Add Studio page with drag-drop model upload and Claude chat
- Add intake system for study creation workflow
- Improve session manager and context builder
- Add intake API routes and frontend components

Optimization Engine:
- Add CLI module for command-line operations
- Add intake module for study preprocessing
- Add validation module with gate checks
- Improve Zernike extractor documentation
- Update spec models with better validation
- Enhance solve_simulation robustness

Documentation:
- Add ATOMIZER_STUDIO.md planning doc
- Add ATOMIZER_UX_SYSTEM.md for UX patterns
- Update extractor library docs
- Add study-readme-generator skill

Tools:
- Add test scripts for extraction validation
- Add Zernike recentering test

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 12:02:30 -05:00
3193831340 feat: Add DevLoop automation and HTML Reports
## DevLoop - Closed-Loop Development System
- Orchestrator for plan → build → test → analyze cycle
- Gemini planning via OpenCode CLI
- Claude implementation via CLI bridge
- Playwright browser testing integration
- Test runner with API, filesystem, and browser tests
- Persistent state in .devloop/ directory
- CLI tool: tools/devloop_cli.py

Usage:
  python tools/devloop_cli.py start 'Create new feature'
  python tools/devloop_cli.py plan 'Fix bug in X'
  python tools/devloop_cli.py test --study support_arm
  python tools/devloop_cli.py browser --level full

## HTML Reports (optimization_engine/reporting/)
- Interactive Plotly-based reports
- Convergence plot, Pareto front, parallel coordinates
- Parameter importance analysis
- Self-contained HTML (offline-capable)
- Tailwind CSS styling

## Playwright E2E Tests
- Home page tests
- Test results in test-results/

## LAC Knowledge Base Updates
- Session insights (failures, workarounds, patterns)
- Optimization memory for arm support study
2026-01-24 21:18:18 -05:00
a3f18dc377 chore: Project cleanup and Canvas UX improvements (Phase 7-9)
## Cleanup (v0.5.0)
- Delete 102+ orphaned MCP session temp files
- Remove build artifacts (htmlcov, dist, __pycache__)
- Archive superseded plan docs (RALPH_LOOP V2/V3, CANVAS V3, etc.)
- Move debug/analysis scripts from tests/ to tools/analysis/
- Archive redundant NX journals to archive/nx_journals/
- Archive monolithic PROTOCOL.md to docs/archive/
- Update .gitignore with missing patterns
- Clean old study files (optimization_log_old.txt, run_optimization_old.py)

## Canvas UX (Phases 7-9)
- Phase 7: Resizable panels with localStorage persistence
  - Left sidebar: 200-400px, Right panel: 280-600px
  - New useResizablePanel hook and ResizeHandle component
- Phase 8: Enable all palette items
  - All 8 node types now draggable
  - Singleton logic for model/solver/algorithm/surrogate
- Phase 9: Solver configuration
  - Add SolverEngine type (nxnastran, mscnastran, python, etc.)
  - Add NastranSolutionType (SOL101-SOL200)
  - Engine/solution dropdowns in config panel
  - Python script path support

## Documentation
- Update CHANGELOG.md with recent versions
- Update docs/00_INDEX.md
- Create examples/README.md
- Add docs/plans/CANVAS_UX_IMPROVEMENTS.md
2026-01-24 15:17:34 -05:00
c224b16ac3 feat: Add panel management, validation, and error handling to canvas
Phase 1 - Panel Management System:
- Create usePanelStore.ts for centralized panel state management
- Add PanelContainer.tsx for draggable floating panels
- Create FloatingIntrospectionPanel.tsx (persistent, doesn't disappear on node click)
- Create ResultsPanel.tsx for trial result details
- Refactor NodeConfigPanelV2 to use panel store for introspection
- Integrate PanelContainer into CanvasView

Phase 2 - Pre-run Validation:
- Create specValidator.ts with comprehensive validation rules
- Add ValidationPanel (enhanced version with error navigation)
- Add Validate button to SpecRenderer with status indicator
- Block run if validation fails
- Check for: design vars, objectives, extractors, bounds, connections

Phase 3 - Error Handling & Recovery:
- Create ErrorPanel.tsx for displaying optimization errors
- Add error classification (nx_crash, solver_fail, extractor_error, etc.)
- Add recovery suggestions based on error type
- Update status endpoint to return error info
- Add _get_study_error_info helper to check error_status.json and DB
- Integrate error detection into status polling

Documentation:
- Add CANVAS_ROBUSTNESS_PLAN.md with full implementation plan
2026-01-21 21:35:31 -05:00
5c419e2358 fix(canvas): Multiple fixes for drag-drop, undo/redo, and code generation
Drag-drop fixes:
- Fix Objective default data: use nested 'source' object with extractor_id/output_name
- Fix Constraint default data: use 'type' field (not constraint_type), 'threshold' (not limit)

Undo/Redo fixes:
- Remove dependency on isDirty flag (which is always false due to auto-save)
- Record snapshots based on actual spec changes via deep comparison

Code generation improvements:
- Update system prompt to support multiple extractor types:
  * OP2-based extractors for FEA results (stress, displacement, frequency)
  * Expression-based extractors for NX model values (dimensions, volumes)
  * Computed extractors for derived values (no FEA needed)
- Claude will now choose appropriate signature based on user's description
2026-01-20 15:08:49 -05:00
ea437d360e docs: Major documentation overhaul - restructure folders, update tagline, add Getting Started guide
- Restructure docs/ folder (remove numeric prefixes):
  - 04_USER_GUIDES -> guides/
  - 05_API_REFERENCE -> api/
  - 06_PHYSICS -> physics/
  - 07_DEVELOPMENT -> development/
  - 08_ARCHIVE -> archive/
  - 09_DIAGRAMS -> diagrams/

- Replace tagline 'Talk, don't click' with 'LLM-driven optimization framework' in 9 files

- Create comprehensive docs/GETTING_STARTED.md:
  - Prerequisites and quick setup
  - Project structure overview
  - First study tutorial (Claude or manual)
  - Dashboard usage guide
  - Neural acceleration introduction

- Rewrite docs/00_INDEX.md with correct paths and modern structure

- Archive obsolete files:
  - 01_PROTOCOLS.md -> archive/historical/01_PROTOCOLS_legacy.md
  - 03_GETTING_STARTED.md -> archive/historical/
  - ATOMIZER_PODCAST_BRIEFING.md -> archive/marketing/

- Update timestamps to 2026-01-20 across all key files

- Update .gitignore to exclude docs/generated/

- Version bump: ATOMIZER_CONTEXT v1.8 -> v2.0
2026-01-20 10:03:45 -05:00
37f73cc2be docs: Update documentation for AtomizerSpec v2.0 unified configuration
Phase 4 documentation updates:
- CANVAS.md: Add AtomizerSpec v2.0 schema, custom extractors section,
  spec REST API endpoints, and updated file structure
- DASHBOARD.md: Add V3.1 release notes, AtomizerSpec API endpoints,
  and unified configuration features
- CLAUDE.md: Add complete AtomizerSpec v2.0 section with code examples,
  MCP tools reference, API endpoints, and updated directory structure

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-17 10:57:58 -05:00
ac5e9b4054 docs: Comprehensive documentation update for Dashboard V3 and Canvas
## Documentation Updates
- DASHBOARD.md: Updated to V3.0 with Canvas V3 features, file browser, introspection
- DASHBOARD_IMPLEMENTATION_STATUS.md: Marked Canvas V3 features as COMPLETE
- CANVAS.md: New comprehensive guide for Canvas Builder V3 with all features
- CLAUDE.md: Added dashboard quick reference and Canvas V3 features

## Canvas V3 Features Documented
- File Browser: Browse studies directory for model files
- Model Introspection: Auto-discover expressions, solver type, dependencies
- One-Click Add: Add expressions as design variables instantly
- Claude Bug Fixes: WebSocket reconnection, SQL errors resolved
- Health Check: /api/health endpoint for monitoring

## Backend Services
- NX introspection service with expression discovery
- File browser API with type filtering
- Claude session management improvements
- Context builder enhancements

## Frontend Components
- FileBrowser: Modal for file selection with search
- IntrospectionPanel: View discovered model information
- ExpressionSelector: Dropdown for design variable configuration
- Improved chat hooks with reconnection logic

## Plan Documents
- Added RALPH_LOOP_CANVAS_V2/V3 implementation records
- Added ATOMIZER_DASHBOARD_V2_MASTER_PLAN
- Added investigation and sync documentation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-16 20:48:58 -05:00
1c7c7aff05 feat(canvas): Add file browser, introspection, and improve node flow
Phase 1-7 of Canvas V4 Ralph Loop implementation:

Backend:
- Add /api/files routes for browsing model files
- Add /api/nx routes for NX model introspection
- Add NXIntrospector service to discover expressions and extractors
- Add health check with database status

Frontend:
- Add FileBrowser component for selecting .sim/.prt/.fem files
- Add IntrospectionPanel to discover expressions and extractors
- Update NodeConfigPanel with browse and introspect buttons
- Update schema with NODE_HANDLES for proper flow direction
- Update validation for correct DesignVar -> Model -> Solver flow
- Update useCanvasStore.addNode() to accept custom data

Flow correction: Design Variables now connect TO Model (as source),
not FROM Model. This matches the actual data flow in optimization.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-16 14:47:10 -05:00
862d07e309 docs: Mark Canvas integration plan as completed 2026-01-14 22:21:48 -05:00
73a7b9d9f1 feat: Add dashboard chat integration and MCP server
Major changes:
- Dashboard: WebSocket-based chat with session management
- Dashboard: New chat components (ChatPane, ChatInput, ModeToggle)
- Dashboard: Enhanced UI with parallel coordinates chart
- MCP Server: New atomizer-tools server for Claude integration
- Extractors: Enhanced Zernike OPD extractor
- Reports: Improved report generator

New studies (configs and scripts only):
- M1 Mirror: Cost reduction campaign studies
- Simple Beam, Simple Bracket, UAV Arm studies

Note: Large iteration data (2_iterations/, best_design_archive/)
excluded via .gitignore - kept on local Gitea only.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-13 15:53:55 -05:00
69c0d76b50 docs: Mark restructuring plan as complete
All phases successfully implemented:
- OP_08 report generation added
- Protocol numbering fixed (SYS_16-18)
- Code reorganized (surrogates, tests)
- Dependencies cleaned (pyproject.toml)
- M1_Mirror studies archived
- README updated with LLM section

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 09:09:22 -05:00
c061146a77 docs: Final documentation polish and consistency fixes
- Update README.md with LLM assistant section
- Create optimization_memory JSONL structure
- Move implementation plans from skills/modules to docs/plans
- Verify all imports work correctly

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 09:07:44 -05:00
b8a04c62b8 docs: Consolidate documentation and fix protocol numbering (partial)
Phase 2 of restructuring plan:
- Rename SYS_16_STUDY_INSIGHTS -> SYS_17_STUDY_INSIGHTS
- Rename SYS_17_CONTEXT_ENGINEERING -> SYS_18_CONTEXT_ENGINEERING
- Promote Bootstrap V3.0 (Context Engineering) as default
- Archive old Bootstrap V2.0
- Create knowledge_base/playbook.json for ACE framework
- Add OP_08 (Generate Report) to routing tables
- Add SYS_16-18 to protocol tables
- Update docs/protocols/README.md to version 1.1
- Update CLAUDE.md with new protocols
- Create docs/plans/RESTRUCTURING_PLAN.md for continuation

Remaining: Phase 2.8 (Cheatsheet), Phases 3-6

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 08:52:07 -05:00
32caa5d05c feat: Implement Study Interview Mode as default study creation method
Study Interview Mode is now the DEFAULT for all study creation requests.
This intelligent Q&A system guides users through optimization setup with:

- 7-phase interview flow: introspection → objectives → constraints → design_variables → validation → review → complete
- Material-aware validation with 12 materials and fuzzy name matching
- Anti-pattern detection for 12 common mistakes (mass-no-constraint, stress-over-yield, etc.)
- Auto extractor mapping E1-E24 based on goal keywords
- State persistence with JSON serialization and backup rotation
- StudyBlueprint generation with full validation

Triggers: "create a study", "new study", "optimize this", any study creation intent
Skip with: "skip interview", "quick setup", "manual config"

Components:
- StudyInterviewEngine: Main orchestrator
- QuestionEngine: Conditional logic evaluation
- EngineeringValidator: MaterialsDatabase + AntiPatternDetector
- InterviewPresenter: Markdown formatting for Claude
- StudyBlueprint: Validated configuration output
- InterviewState: Persistent state management

All 129 tests passing.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 11:06:07 -05:00
b1ffc64407 feat: Implement SAT v3 achieving WS=205.58 (new campaign record)
Self-Aware Turbo v3 optimization validated on M1 Mirror flat back:
- Best WS: 205.58 (12% better than previous best 218.26)
- 100% feasibility rate, 100% unique designs
- Uses 556 training samples from V5-V8 campaign data

Key innovations in V9:
- Adaptive exploration schedule (15% → 8% → 3%)
- Mass threshold at 118 kg (optimal sweet spot)
- 70% exploitation near best design
- Seeded with best known design from V7
- Ensemble surrogate with R²=0.99

Updated documentation:
- SYS_16: SAT protocol updated to v3.0 VALIDATED
- Cheatsheet: Added SAT v3 as recommended method
- Context: Updated protocol overview

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-31 16:06:33 -05:00
8c7a589547 docs: Add SAT v3 (Self-Aware Turbo) to podcast briefing
- Added new PART 8: Self-Aware Turbo (SAT) - Validated Breakthrough
- Explains ensemble surrogate with epistemic uncertainty
- Documents OOD detection and adaptive exploration schedule
- Includes V9 results: WS=205.58 (best ever)
- Added SAT sound bites for podcast
- Updated document to 12 sections

Generated with Claude Code

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-31 16:05:16 -05:00
5e64cfb211 docs: Update podcast briefing with simulation focus and protocol evolution
Key changes based on feedback:
- Reposition as "optimizer & NX configurator" not "LLM-first"
- Add Part 2: Study Characterization & Performance Learning
- Add Part 3: Protocol Evolution workflow (Research → Review → Approve)
- Add Part 4: MCP-first development approach with documentation hierarchy
- Emphasize simulation optimization over CAD/mesh concerns
- Add LAC knowledge accumulation for parameter-performance relationships
- Add privilege levels for protocol approval (user/power_user/admin)
- Update sound bites and core messaging

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-31 13:29:52 -05:00
f0e594570a docs: Add comprehensive podcast briefing document
- Add ATOMIZER_PODCAST_BRIEFING.md with complete technical overview
- Covers all 12 sections: architecture, optimization, neural acceleration
- Includes impressive statistics and metrics for podcast generation
- Update LAC failure insights from recent sessions
- Add M1_Mirror studies README

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-30 09:36:40 -05:00
773f8ff8af feat: Implement ACE Context Engineering framework (SYS_17)
Complete implementation of Agentic Context Engineering (ACE) framework:

Core modules (optimization_engine/context/):
- playbook.py: AtomizerPlaybook with helpful/harmful scoring
- reflector.py: AtomizerReflector for insight extraction
- session_state.py: Context isolation (exposed/isolated state)
- feedback_loop.py: Automated learning from trial results
- compaction.py: Long-session context management
- cache_monitor.py: KV-cache optimization tracking
- runner_integration.py: OptimizationRunner integration

Dashboard integration:
- context.py: 12 REST API endpoints for playbook management

Tests:
- test_context_engineering.py: 44 unit tests
- test_context_integration.py: 16 integration tests

Documentation:
- CONTEXT_ENGINEERING_REPORT.md: Comprehensive implementation report
- CONTEXT_ENGINEERING_API.md: Complete API reference
- SYS_17_CONTEXT_ENGINEERING.md: System protocol
- Updated cheatsheet with SYS_17 quick reference
- Enhanced bootstrap (00_BOOTSTRAP_V2.md)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-29 20:21:20 -05:00
82f36689b7 feat: Pre-migration checkpoint - updated docs and utilities
Updates before optimization_engine migration:
- Updated migration plan to v2.1 with complete file inventory
- Added OP_07 disk optimization protocol
- Added SYS_16 self-aware turbo protocol
- Added study archiver and cleanup utilities
- Added ensemble surrogate module
- Updated NX solver and session manager
- Updated zernike HTML generator
- Added context engineering plan
- LAC session insights updates

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-29 10:22:45 -05:00
faa7779a43 feat: Add L-BFGS gradient optimizer for surrogate polish phase
Implements gradient-based optimization exploiting MLP surrogate differentiability.
Achieves 100-1000x faster convergence than derivative-free methods (TPE, CMA-ES).

New files:
- optimization_engine/gradient_optimizer.py: GradientOptimizer class with L-BFGS/Adam/SGD
- studies/M1_Mirror/m1_mirror_adaptive_V14/run_lbfgs_polish.py: Per-study runner

Updated docs:
- SYS_14_NEURAL_ACCELERATION.md: Full L-BFGS section (v2.4)
- 01_CHEATSHEET.md: Quick reference for L-BFGS usage
- atomizer_fast_solver_technologies.md: Architecture context

Usage: python -m optimization_engine.gradient_optimizer studies/my_study --n-starts 20

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-28 16:36:18 -05:00
cf454f6e40 feat: Add TrialManager and DashboardDB for unified trial management
- Add TrialManager (trial_manager.py) for consistent trial_NNNN naming
- Add DashboardDB (dashboard_db.py) for Optuna-compatible database schema
- Update CLAUDE.md with trial management documentation
- Update ATOMIZER_CONTEXT.md with v1.8 trial system
- Update cheatsheet v2.2 with new utilities
- Update SYS_14 protocol to v2.3 with TrialManager integration
- Add LAC learnings for trial management patterns
- Add archive/README.md for deprecated code policy

Key principles:
- Trial numbers NEVER reset (monotonic)
- Folders NEVER get overwritten
- Database always synced with filesystem
- Surrogate predictions are NOT trials (only FEA results)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-28 12:20:19 -05:00
f13563d7ab feat: Major update - Physics docs, Zernike OPD, insights, NX journals, tools
Documentation:
- Add docs/06_PHYSICS/ with Zernike fundamentals and OPD method docs
- Add docs/guides/CMA-ES_EXPLAINED.md optimization guide
- Update CLAUDE.md and ATOMIZER_CONTEXT.md with current architecture
- Update OP_01_CREATE_STUDY protocol

Planning:
- Add DYNAMIC_RESPONSE plans for random vibration/PSD support
- Add OPTIMIZATION_ENGINE_MIGRATION_PLAN for code reorganization

Insights System:
- Update design_space, modal_analysis, stress_field, thermal_field insights
- Improve error handling and data validation

NX Journals:
- Add analyze_wfe_zernike.py for Zernike WFE analysis
- Add capture_study_images.py for automated screenshots
- Add extract_expressions.py and introspect_part.py utilities
- Add user_generated_journals/journal_top_view_image_taking.py

Tests & Tools:
- Add comprehensive Zernike OPD test suite
- Add audit_v10 tests for WFE validation
- Add tools for Pareto graphs and mirror data extraction
- Add migrate_studies_to_topics.py utility

Knowledge Base:
- Initialize LAC (Learning Atomizer Core) with failure/success patterns

Dashboard:
- Update Setup.tsx and launch_dashboard.py
- Add restart-dev.bat helper script

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-23 19:47:37 -05:00
d19fc39a2a feat: Add OPD method support to Zernike visualization with Standard/OPD toggle
Major improvements to Zernike WFE visualization:

- Add ZernikeDashboardInsight: Unified dashboard with all orientations (40°, 60°, 90°)
  on one page with light theme and executive summary
- Add OPD method toggle: Switch between Standard (Z-only) and OPD (X,Y,Z) methods
  in ZernikeWFEInsight with interactive buttons
- Add lateral displacement maps: Visualize X,Y displacement for each orientation
- Add displacement component views: Toggle between WFE, ΔX, ΔY, ΔZ in relative views
- Add metrics comparison table showing both methods side-by-side

New extractors:
- extract_zernike_figure.py: ZernikeOPDExtractor using BDF geometry interpolation
- extract_zernike_opd.py: Parabola-based OPD with focal length

Key finding: OPD method gives 8-11% higher WFE values than Standard method
(more conservative/accurate for surfaces with lateral displacement under gravity)

Documentation updates:
- SYS_12: Added E22 ZernikeOPD as recommended method
- SYS_16: Added ZernikeDashboard, updated ZernikeWFE with OPD features
- Cheatsheet: Added Zernike method comparison table

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-22 21:03:19 -05:00
274081d977 refactor: Engine updates and NX hooks improvements
optimization_engine:
- Updated nx_solver.py with improvements
- Enhanced solve_simulation.py
- Updated extractors/__init__.py
- Improved NX CAD hooks (expression_manager, feature_manager,
  geometry_query, model_introspection, part_manager)
- Enhanced NX CAE solver_manager hook

Documentation:
- Updated OP_01_CREATE_STUDY.md protocol
- Updated SYS_12_EXTRACTOR_LIBRARY.md

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-20 13:47:21 -05:00
1612991d0d feat: Add Study Insights module (SYS_16) for physics visualizations
Introduces a new plugin architecture for study-specific physics
visualizations, separating "optimizer perspective" (Analysis) from
"engineer perspective" (Insights).

New module: optimization_engine/insights/
- base.py: StudyInsight base class, InsightConfig, InsightResult, registry
- zernike_wfe.py: Mirror WFE with 3D surface and Zernike decomposition
- stress_field.py: Von Mises stress contours with safety factors
- modal_analysis.py: Natural frequencies and mode shapes
- thermal_field.py: Temperature distribution visualization
- design_space.py: Parameter-objective landscape exploration

Features:
- 5 insight types: zernike_wfe, stress_field, modal, thermal, design_space
- CLI: python -m optimization_engine.insights generate <study>
- Standalone HTML generation with Plotly
- Enhanced Zernike viz: Turbo colorscale, smooth shading, 0.5x AMP
- Dashboard API fix: Added include_coefficients param to extract_relative()

Documentation:
- docs/protocols/system/SYS_16_STUDY_INSIGHTS.md
- Updated ATOMIZER_CONTEXT.md (v1.7)
- Updated 01_CHEATSHEET.md with insights section

Tools:
- tools/zernike_html_generator.py: Standalone WFE HTML generator
- tools/analyze_wfe.bat: Double-click to analyze OP2 files

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-20 13:46:28 -05:00
Antoine
01a7d7d121 docs: Complete M1 mirror optimization campaign V11-V15
## M1 Mirror Campaign Summary
- V11-V15 optimization campaign completed (~1,400 FEA evaluations)
- Best design: V14 Trial #725 with Weighted Sum = 121.72
- V15 NSGA-II confirmed V14 TPE found optimal solution
- Campaign improved from WS=129.33 (V11) to WS=121.72 (V14): -5.9%

## Key Results
- 40° tracking: 5.99 nm (target 4.0 nm)
- 60° tracking: 13.10 nm (target 10.0 nm)
- Manufacturing: 26.28 nm (target 20.0 nm)
- Targets not achievable within current design space

## Documentation Added
- V15 STUDY_REPORT.md: Detailed NSGA-II results analysis
- M1_MIRROR_CAMPAIGN_SUMMARY.md: Full V11-V15 campaign overview
- Updated CLAUDE.md, ATOMIZER_CONTEXT.md with NXSolver patterns
- Updated 01_CHEATSHEET.md with --resume guidance
- Updated OP_01_CREATE_STUDY.md with FEARunner template

## Studies Added
- m1_mirror_adaptive_V13: TPE validation (291 trials)
- m1_mirror_adaptive_V14: TPE intensive (785 trials, BEST)
- m1_mirror_adaptive_V15: NSGA-II exploration (126 new FEA)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-16 14:55:23 -05:00
Antoine
d1261d62fd refactor: Major project cleanup and reorganization
## Removed Duplicate Directories
- Deleted old `dashboard/` (replaced by atomizer-dashboard)
- Deleted old `mcp_server/` Python tools (moved model_discovery to optimization_engine)
- Deleted `tests/mcp_server/` (obsolete tests)
- Deleted `launch_dashboard.bat` (old launcher)

## Consolidated Code
- Moved `mcp_server/tools/model_discovery.py` to `optimization_engine/model_discovery/`
- Updated import in `optimization_config_builder.py`
- Deleted stub `extract_mass.py` (use extract_mass_from_bdf instead)
- Deleted unused `intelligent_setup.py` and `hybrid_study_creator.py`
- Archived `result_extractors/` to `archive/deprecated/`

## Documentation Cleanup
- Deleted deprecated `docs/06_PROTOCOLS_DETAILED/` (14 files)
- Archived dated dev docs to `docs/08_ARCHIVE/sessions/`
- Archived old plans to `docs/08_ARCHIVE/plans/`
- Updated `docs/protocols/README.md` with SYS_15

## Skills Consolidation
- Archived redundant study creation skills to `.claude/skills/archive/`
- Kept `core/study-creation-core.md` as canonical

## Housekeeping
- Updated `.gitignore` to prevent `nul` and `_dat_run*.dat`

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-12 11:24:02 -05:00
Antoine
1bb201e0b7 feat: Add post-optimization tools and mandatory best design archiving
New Tools (tools/):
- analyze_study.py: Generate comprehensive optimization reports
- find_best_iteration.py: Find best iteration folder, optionally copy it
- archive_best_design.py: Archive best design to 3_results/best_design_archive/<timestamp>/

Protocol Updates:
- OP_02_RUN_OPTIMIZATION.md v1.1: Add mandatory archive_best_design step
  in Post-Run Actions. This MUST be done after every optimization run.

V14 Updates:
- run_optimization.py: Auto-archive best design at end of optimization
- optimization_config.json: Expand bounds for V14 continuation
  - lateral_outer_angle: min 13->11 deg (was at 4.7%)
  - lateral_inner_pivot: min 7->5 mm (was at 8.1%)
  - lateral_middle_pivot: max 23->27 mm (was at 99.4%)
  - whiffle_min: max 60->72 mm (was at 96.3%)

Usage:
  python tools/analyze_study.py m1_mirror_adaptive_V14
  python tools/find_best_iteration.py m1_mirror_adaptive_V14
  python tools/archive_best_design.py m1_mirror_adaptive_V14

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-12 10:28:35 -05:00
Antoine
70ac34e3d3 feat: Add E11 Part Mass extractor, document pyNastran mass accuracy issue
New E11 Part Mass Extractor:
- Add nx_journals/extract_part_mass_material.py - NX journal using
  NXOpen.MeasureManager.NewMassProperties() for accurate geometry-based mass
- Add optimization_engine/extractors/extract_part_mass_material.py - Python
  wrapper that reads JSON output from journal
- Add E11 entry to extractors/catalog.json

Documentation Updates:
- SYS_12_EXTRACTOR_LIBRARY.md: Add mass accuracy warning noting pyNastran
  get_mass_breakdown() under-reports ~7% on hex-dominant meshes with
  tet/pyramid fill elements. E11 (geometry .prt) should be preferred over
  E4 (BDF) unless material is overridden at FEM level.
- 01_CHEATSHEET.md: Add mass extraction tip

V14 Config:
- Expand design variable bounds (blank_backface_angle max 4.5°,
  whiffle_triangle_closeness max 80mm, whiffle_min max 60mm)

Testing showed:
- E11 from .prt: 97.66 kg (accurate - matches NX GUI)
- E4 pyNastran get_mass_breakdown(): 90.73 kg (~7% under-reported)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-11 22:15:36 -05:00
Antoine
c1f2634636 docs: Add user guide for proper Atomizer usage and evolution
Comprehensive guide teaching users how to interact with Atomizer so that
the learning system evolves correctly. Covers:

- The right mindset (colleague, not tool)
- Starting sessions with proper context
- Communicating goals, constraints, preferences
- Creating and running optimization studies
- Analyzing and validating results
- Reporting errors effectively
- Contributing to LAC (recording insights, outcomes, workarounds)
- Ending sessions properly to capture learnings

Includes:
- Mermaid diagrams for learning loop and flows
- Good vs bad examples for every interaction type
- Complete example session transcript
- Quick reference card for common patterns
- Golden rules summary

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-11 22:12:28 -05:00
Antoine
f83dc6839f docs: Add comprehensive architecture overview with Mermaid diagrams
Complete visual guide to understanding Atomizer's architecture including:
- Session lifecycle (startup, active, closing)
- Protocol Operating System (4-layer architecture)
- Learning Atomizer Core (LAC) data flow
- Task classification and routing
- AVERVS execution framework
- Optimization flow with extractors
- Knowledge accumulation over time
- File structure reference

Includes 15+ Mermaid diagrams for visual learning.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-11 22:05:09 -05:00
Antoine
fc123326e5 feat: Integrate Learning Atomizer Core (LAC) and master instructions
Add persistent knowledge system that enables Atomizer to learn from every
session and improve over time.

## New Files
- knowledge_base/lac.py: LAC class with optimization memory, session insights,
  and skill evolution tracking
- knowledge_base/__init__.py: Package initialization
- .claude/skills/modules/learning-atomizer-core.md: Full LAC skill documentation
- docs/07_DEVELOPMENT/ATOMIZER_CLAUDE_CODE_INSTRUCTIONS.md: Master instructions

## Updated Files
- CLAUDE.md: Added LAC section, communication style, AVERVS execution framework,
  error classification, and "Atomizer Claude" identity
- 00_BOOTSTRAP.md: Added session startup/closing checklists with LAC integration
- 01_CHEATSHEET.md: Added LAC CLI and Python API quick reference
- 02_CONTEXT_LOADER.md: Added LAC query section and anti-pattern

## LAC Features
- Query similar past optimizations before starting new ones
- Record insights (failures, success patterns, workarounds)
- Record optimization outcomes for future reference
- Suggest protocol improvements based on discoveries
- Simple JSONL storage (no database required)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-11 21:55:01 -05:00
Antoine
96b196de58 feat: Add Zernike GNN surrogate module and M1 mirror V12/V13 studies
This commit introduces the GNN-based surrogate for Zernike mirror optimization
and the M1 mirror study progression from V12 (GNN validation) to V13 (pure NSGA-II).

## GNN Surrogate Module (optimization_engine/gnn/)

New module for Graph Neural Network surrogate prediction of mirror deformations:

- `polar_graph.py`: PolarMirrorGraph - fixed 3000-node polar grid structure
- `zernike_gnn.py`: ZernikeGNN with design-conditioned message passing
- `differentiable_zernike.py`: GPU-accelerated Zernike fitting and objectives
- `train_zernike_gnn.py`: ZernikeGNNTrainer with multi-task loss
- `gnn_optimizer.py`: ZernikeGNNOptimizer for turbo mode (~900k trials/hour)
- `extract_displacement_field.py`: OP2 to HDF5 field extraction
- `backfill_field_data.py`: Extract fields from existing FEA trials

Key innovation: Design-conditioned convolutions that modulate message passing
based on structural design parameters, enabling accurate field prediction.

## M1 Mirror Studies

### V12: GNN Field Prediction + FEA Validation
- Zernike GNN trained on V10/V11 FEA data (238 samples)
- Turbo mode: 5000 GNN predictions → top candidates → FEA validation
- Calibration workflow for GNN-to-FEA error correction
- Scripts: run_gnn_turbo.py, validate_gnn_best.py, compute_full_calibration.py

### V13: Pure NSGA-II FEA (Ground Truth)
- Seeds 217 FEA trials from V11+V12
- Pure multi-objective NSGA-II without any surrogate
- Establishes ground-truth Pareto front for GNN accuracy evaluation
- Narrowed blank_backface_angle range to [4.0, 5.0]

## Documentation Updates

- SYS_14: Added Zernike GNN section with architecture diagrams
- CLAUDE.md: Added GNN module reference and quick start
- V13 README: Study documentation with seeding strategy

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-10 08:44:04 -05:00