feat: Major update - Physics docs, Zernike OPD, insights, NX journals, tools
Documentation: - Add docs/06_PHYSICS/ with Zernike fundamentals and OPD method docs - Add docs/guides/CMA-ES_EXPLAINED.md optimization guide - Update CLAUDE.md and ATOMIZER_CONTEXT.md with current architecture - Update OP_01_CREATE_STUDY protocol Planning: - Add DYNAMIC_RESPONSE plans for random vibration/PSD support - Add OPTIMIZATION_ENGINE_MIGRATION_PLAN for code reorganization Insights System: - Update design_space, modal_analysis, stress_field, thermal_field insights - Improve error handling and data validation NX Journals: - Add analyze_wfe_zernike.py for Zernike WFE analysis - Add capture_study_images.py for automated screenshots - Add extract_expressions.py and introspect_part.py utilities - Add user_generated_journals/journal_top_view_image_taking.py Tests & Tools: - Add comprehensive Zernike OPD test suite - Add audit_v10 tests for WFE validation - Add tools for Pareto graphs and mirror data extraction - Add migrate_studies_to_topics.py utility Knowledge Base: - Initialize LAC (Learning Atomizer Core) with failure/success patterns Dashboard: - Update Setup.tsx and launch_dashboard.py - Add restart-dev.bat helper script 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
@@ -136,9 +136,11 @@ studies/{geometry_type}/{study_name}/
|
||||
| E3 | Von Mises Stress | `extract_solid_stress()` | **Specify element_type!** |
|
||||
| E4 | BDF Mass | `extract_mass_from_bdf()` | kg |
|
||||
| E5 | CAD Mass | `extract_mass_from_expression()` | kg |
|
||||
| E8-10 | Zernike WFE | `extract_zernike_*()` | nm (mirrors) |
|
||||
| E8-10 | Zernike WFE (standard) | `extract_zernike_*()` | nm (mirrors) |
|
||||
| E12-14 | Phase 2 | Principal stress, strain energy, SPC forces |
|
||||
| E15-18 | Phase 3 | Temperature, heat flux, modal mass |
|
||||
| E20 | Zernike Analytic | `extract_zernike_analytic()` | nm (parabola-based) |
|
||||
| E22 | **Zernike OPD** | `extract_zernike_opd()` | nm (**RECOMMENDED**) |
|
||||
|
||||
**Critical**: For stress extraction, specify element type:
|
||||
- Shell (CQUAD4): `element_type='cquad4'`
|
||||
|
||||
1084
.claude/skills/modules/DYNAMIC_RESPONSE_IMPLEMENTATION_PLAN.md
Normal file
1084
.claude/skills/modules/DYNAMIC_RESPONSE_IMPLEMENTATION_PLAN.md
Normal file
File diff suppressed because it is too large
Load Diff
1649
.claude/skills/modules/DYNAMIC_RESPONSE_MASTER_PLAN.md
Normal file
1649
.claude/skills/modules/DYNAMIC_RESPONSE_MASTER_PLAN.md
Normal file
File diff suppressed because it is too large
Load Diff
932
.claude/skills/modules/OPTIMIZATION_ENGINE_MIGRATION_PLAN.md
Normal file
932
.claude/skills/modules/OPTIMIZATION_ENGINE_MIGRATION_PLAN.md
Normal file
@@ -0,0 +1,932 @@
|
||||
# Optimization Engine Reorganization - Migration Plan
|
||||
|
||||
## Comprehensive Guide for Safe Codebase Restructuring
|
||||
|
||||
**Document Version**: 1.0
|
||||
**Created**: 2025-12-23
|
||||
**Status**: PLANNING - Do Not Execute Without Review
|
||||
**Risk Level**: HIGH - Affects 557+ locations across 276 files
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
This document provides a complete migration plan for reorganizing `optimization_engine/` from 50+ loose files to a clean modular structure. This is a **high-impact refactoring** that requires careful execution.
|
||||
|
||||
### Impact Summary
|
||||
|
||||
| Category | Files Affected | Lines to Change |
|
||||
|----------|----------------|-----------------|
|
||||
| Python imports (internal) | 90+ files | ~145 changes |
|
||||
| Python imports (studies) | 30+ folders | ~153 changes |
|
||||
| Python imports (tests) | 30+ files | ~79 changes |
|
||||
| Python imports (dashboard) | 3 files | ~11 changes |
|
||||
| Documentation (protocols, skills) | 119 files | ~200 changes |
|
||||
| JSON configs | 4 files | ~50 changes |
|
||||
| **TOTAL** | **276 files** | **~640 changes** |
|
||||
|
||||
---
|
||||
|
||||
## Part 1: Current State Analysis
|
||||
|
||||
### 1.1 Top-Level Files Requiring Migration
|
||||
|
||||
These 50+ files at `optimization_engine/` root need to move:
|
||||
|
||||
```
|
||||
SURROGATES (6 files) → processors/surrogates/
|
||||
├── neural_surrogate.py
|
||||
├── generic_surrogate.py
|
||||
├── adaptive_surrogate.py
|
||||
├── simple_mlp_surrogate.py
|
||||
├── active_learning_surrogate.py
|
||||
└── surrogate_tuner.py
|
||||
|
||||
OPTIMIZATION CORE (7 files) → core/
|
||||
├── runner.py
|
||||
├── runner_with_neural.py
|
||||
├── base_runner.py
|
||||
├── intelligent_optimizer.py
|
||||
├── method_selector.py
|
||||
├── strategy_selector.py
|
||||
└── strategy_portfolio.py
|
||||
|
||||
NX INTEGRATION (6 files) → nx/
|
||||
├── nx_solver.py
|
||||
├── nx_updater.py
|
||||
├── nx_session_manager.py
|
||||
├── solve_simulation.py
|
||||
├── solve_simulation_simple.py
|
||||
└── model_cleanup.py
|
||||
|
||||
STUDY MANAGEMENT (5 files) → study/
|
||||
├── study_creator.py
|
||||
├── study_wizard.py
|
||||
├── study_state.py
|
||||
├── study_reset.py
|
||||
└── study_continuation.py
|
||||
|
||||
REPORTING (5 files) → reporting/
|
||||
├── generate_report.py
|
||||
├── generate_report_markdown.py
|
||||
├── comprehensive_results_analyzer.py
|
||||
├── visualizer.py
|
||||
└── landscape_analyzer.py
|
||||
|
||||
CONFIG (4 files) → config/
|
||||
├── config_manager.py
|
||||
├── optimization_config_builder.py
|
||||
├── optimization_setup_wizard.py
|
||||
└── capability_matcher.py
|
||||
|
||||
AGENTS/RESEARCH (5 files) → agents/ or future/
|
||||
├── research_agent.py
|
||||
├── pynastran_research_agent.py
|
||||
├── targeted_research_planner.py
|
||||
├── workflow_decomposer.py
|
||||
└── step_classifier.py
|
||||
|
||||
MISC (remaining ~15 files) - evaluate individually
|
||||
├── logger.py → utils/
|
||||
├── op2_extractor.py → extractors/
|
||||
├── extractor_library.py → extractors/
|
||||
├── export_expressions.py → nx/
|
||||
├── import_expressions.py → nx/
|
||||
├── mesh_converter.py → nx/
|
||||
├── simulation_validator.py → validators/
|
||||
├── auto_doc.py → utils/
|
||||
├── auto_trainer.py → processors/surrogates/
|
||||
├── realtime_tracking.py → utils/
|
||||
├── benchmarking_substudy.py → study/
|
||||
├── codebase_analyzer.py → utils/
|
||||
├── training_data_exporter.py → processors/surrogates/
|
||||
├── pruning_logger.py → utils/
|
||||
├── adaptive_characterization.py → processors/
|
||||
└── generate_history_from_trials.py → study/
|
||||
```
|
||||
|
||||
### 1.2 Well-Organized Directories (Keep As-Is)
|
||||
|
||||
These are already properly organized:
|
||||
|
||||
```
|
||||
extractors/ ✓ 20+ extractors, clean __init__.py
|
||||
insights/ ✓ 8 insight types, registry pattern
|
||||
hooks/ ✓ nx_cad/, nx_cae/ subdirs
|
||||
gnn/ ✓ Neural surrogate for Zernike
|
||||
templates/ ✓ Config templates
|
||||
schemas/ ✓ JSON schemas
|
||||
validators/ ✓ Validation code
|
||||
plugins/ ✓ Hook manager system
|
||||
utils/ ✓ Utility functions
|
||||
custom_functions/ ✓ NX material generator
|
||||
model_discovery/ ✓ Model introspection
|
||||
future/ ✓ Experimental code
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Part 2: Target Structure
|
||||
|
||||
### 2.1 Proposed Final Structure
|
||||
|
||||
```
|
||||
optimization_engine/
|
||||
│
|
||||
├── __init__.py # Updated with backwards-compat aliases
|
||||
│
|
||||
├── core/ # NEW - Optimization engine core
|
||||
│ ├── __init__.py
|
||||
│ ├── runner.py
|
||||
│ ├── base_runner.py
|
||||
│ ├── runner_with_neural.py
|
||||
│ ├── intelligent_optimizer.py
|
||||
│ ├── method_selector.py
|
||||
│ ├── strategy_selector.py
|
||||
│ └── strategy_portfolio.py
|
||||
│
|
||||
├── processors/ # NEW - Data processing & algorithms
|
||||
│ ├── __init__.py
|
||||
│ ├── surrogates/
|
||||
│ │ ├── __init__.py
|
||||
│ │ ├── neural_surrogate.py
|
||||
│ │ ├── generic_surrogate.py
|
||||
│ │ ├── adaptive_surrogate.py
|
||||
│ │ ├── simple_mlp_surrogate.py
|
||||
│ │ ├── active_learning_surrogate.py
|
||||
│ │ ├── surrogate_tuner.py
|
||||
│ │ ├── auto_trainer.py
|
||||
│ │ └── training_data_exporter.py
|
||||
│ │
|
||||
│ └── dynamic_response/ # NEW - From master plan
|
||||
│ ├── __init__.py
|
||||
│ ├── modal_database.py
|
||||
│ ├── transfer_functions.py
|
||||
│ ├── random_vibration.py
|
||||
│ ├── psd_profiles.py
|
||||
│ └── utils/
|
||||
│
|
||||
├── nx/ # NEW - NX/Nastran integration
|
||||
│ ├── __init__.py
|
||||
│ ├── solver.py # Was nx_solver.py
|
||||
│ ├── updater.py # Was nx_updater.py
|
||||
│ ├── session_manager.py # Was nx_session_manager.py
|
||||
│ ├── solve_simulation.py
|
||||
│ ├── solve_simulation_simple.py
|
||||
│ ├── model_cleanup.py
|
||||
│ ├── export_expressions.py
|
||||
│ ├── import_expressions.py
|
||||
│ └── mesh_converter.py
|
||||
│
|
||||
├── study/ # NEW - Study management
|
||||
│ ├── __init__.py
|
||||
│ ├── creator.py # Was study_creator.py
|
||||
│ ├── wizard.py # Was study_wizard.py
|
||||
│ ├── state.py # Was study_state.py
|
||||
│ ├── reset.py # Was study_reset.py
|
||||
│ ├── continuation.py # Was study_continuation.py
|
||||
│ ├── benchmarking.py # Was benchmarking_substudy.py
|
||||
│ └── history_generator.py # Was generate_history_from_trials.py
|
||||
│
|
||||
├── reporting/ # NEW - Reports and analysis
|
||||
│ ├── __init__.py
|
||||
│ ├── report_generator.py
|
||||
│ ├── markdown_report.py
|
||||
│ ├── results_analyzer.py
|
||||
│ ├── visualizer.py
|
||||
│ └── landscape_analyzer.py
|
||||
│
|
||||
├── config/ # NEW - Configuration
|
||||
│ ├── __init__.py
|
||||
│ ├── manager.py
|
||||
│ ├── builder.py
|
||||
│ ├── setup_wizard.py
|
||||
│ └── capability_matcher.py
|
||||
│
|
||||
├── extractors/ # EXISTING - Add op2_extractor, extractor_library
|
||||
├── insights/ # EXISTING
|
||||
├── hooks/ # EXISTING
|
||||
├── gnn/ # EXISTING
|
||||
├── templates/ # EXISTING
|
||||
├── schemas/ # EXISTING
|
||||
├── validators/ # EXISTING - Add simulation_validator
|
||||
├── plugins/ # EXISTING
|
||||
├── utils/ # EXISTING - Add logger, auto_doc, etc.
|
||||
├── custom_functions/ # EXISTING
|
||||
├── model_discovery/ # EXISTING
|
||||
└── future/ # EXISTING - Move agents here
|
||||
├── research_agent.py
|
||||
├── pynastran_research_agent.py
|
||||
├── targeted_research_planner.py
|
||||
├── workflow_decomposer.py
|
||||
└── step_classifier.py
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Part 3: Import Mapping Tables
|
||||
|
||||
### 3.1 Old → New Import Mapping
|
||||
|
||||
This is the critical reference for all updates:
|
||||
|
||||
```python
|
||||
# CORE OPTIMIZATION
|
||||
"from optimization_engine.runner" → "from optimization_engine.core.runner"
|
||||
"from optimization_engine.base_runner" → "from optimization_engine.core.base_runner"
|
||||
"from optimization_engine.runner_with_neural" → "from optimization_engine.core.runner_with_neural"
|
||||
"from optimization_engine.intelligent_optimizer" → "from optimization_engine.core.intelligent_optimizer"
|
||||
"from optimization_engine.method_selector" → "from optimization_engine.core.method_selector"
|
||||
"from optimization_engine.strategy_selector" → "from optimization_engine.core.strategy_selector"
|
||||
"from optimization_engine.strategy_portfolio" → "from optimization_engine.core.strategy_portfolio"
|
||||
|
||||
# SURROGATES
|
||||
"from optimization_engine.neural_surrogate" → "from optimization_engine.processors.surrogates.neural_surrogate"
|
||||
"from optimization_engine.generic_surrogate" → "from optimization_engine.processors.surrogates.generic_surrogate"
|
||||
"from optimization_engine.adaptive_surrogate" → "from optimization_engine.processors.surrogates.adaptive_surrogate"
|
||||
"from optimization_engine.simple_mlp_surrogate" → "from optimization_engine.processors.surrogates.simple_mlp_surrogate"
|
||||
"from optimization_engine.active_learning_surrogate" → "from optimization_engine.processors.surrogates.active_learning_surrogate"
|
||||
"from optimization_engine.surrogate_tuner" → "from optimization_engine.processors.surrogates.surrogate_tuner"
|
||||
|
||||
# NX INTEGRATION
|
||||
"from optimization_engine.nx_solver" → "from optimization_engine.nx.solver"
|
||||
"from optimization_engine.nx_updater" → "from optimization_engine.nx.updater"
|
||||
"from optimization_engine.nx_session_manager" → "from optimization_engine.nx.session_manager"
|
||||
"from optimization_engine.solve_simulation" → "from optimization_engine.nx.solve_simulation"
|
||||
"from optimization_engine.model_cleanup" → "from optimization_engine.nx.model_cleanup"
|
||||
"from optimization_engine.export_expressions" → "from optimization_engine.nx.export_expressions"
|
||||
"from optimization_engine.import_expressions" → "from optimization_engine.nx.import_expressions"
|
||||
"from optimization_engine.mesh_converter" → "from optimization_engine.nx.mesh_converter"
|
||||
|
||||
# STUDY MANAGEMENT
|
||||
"from optimization_engine.study_creator" → "from optimization_engine.study.creator"
|
||||
"from optimization_engine.study_wizard" → "from optimization_engine.study.wizard"
|
||||
"from optimization_engine.study_state" → "from optimization_engine.study.state"
|
||||
"from optimization_engine.study_reset" → "from optimization_engine.study.reset"
|
||||
"from optimization_engine.study_continuation" → "from optimization_engine.study.continuation"
|
||||
|
||||
# REPORTING
|
||||
"from optimization_engine.generate_report" → "from optimization_engine.reporting.report_generator"
|
||||
"from optimization_engine.generate_report_markdown" → "from optimization_engine.reporting.markdown_report"
|
||||
"from optimization_engine.comprehensive_results" → "from optimization_engine.reporting.results_analyzer"
|
||||
"from optimization_engine.visualizer" → "from optimization_engine.reporting.visualizer"
|
||||
"from optimization_engine.landscape_analyzer" → "from optimization_engine.reporting.landscape_analyzer"
|
||||
|
||||
# CONFIG
|
||||
"from optimization_engine.config_manager" → "from optimization_engine.config.manager"
|
||||
"from optimization_engine.optimization_config_builder" → "from optimization_engine.config.builder"
|
||||
"from optimization_engine.optimization_setup_wizard" → "from optimization_engine.config.setup_wizard"
|
||||
"from optimization_engine.capability_matcher" → "from optimization_engine.config.capability_matcher"
|
||||
|
||||
# UTILITIES (moving to utils/)
|
||||
"from optimization_engine.logger" → "from optimization_engine.utils.logger"
|
||||
"from optimization_engine.auto_doc" → "from optimization_engine.utils.auto_doc"
|
||||
"from optimization_engine.realtime_tracking" → "from optimization_engine.utils.realtime_tracking"
|
||||
"from optimization_engine.codebase_analyzer" → "from optimization_engine.utils.codebase_analyzer"
|
||||
"from optimization_engine.pruning_logger" → "from optimization_engine.utils.pruning_logger"
|
||||
|
||||
# RESEARCH/AGENTS (moving to future/)
|
||||
"from optimization_engine.research_agent" → "from optimization_engine.future.research_agent"
|
||||
"from optimization_engine.workflow_decomposer" → "from optimization_engine.future.workflow_decomposer"
|
||||
"from optimization_engine.step_classifier" → "from optimization_engine.future.step_classifier"
|
||||
```
|
||||
|
||||
### 3.2 Backwards Compatibility Aliases
|
||||
|
||||
Add to `optimization_engine/__init__.py` for transition period:
|
||||
|
||||
```python
|
||||
# BACKWARDS COMPATIBILITY ALIASES
|
||||
# These allow old imports to work during migration period
|
||||
# Remove after all code is updated
|
||||
|
||||
# Core
|
||||
from optimization_engine.core.runner import *
|
||||
from optimization_engine.core.base_runner import *
|
||||
from optimization_engine.core.intelligent_optimizer import *
|
||||
|
||||
# NX
|
||||
from optimization_engine.nx import solver as nx_solver
|
||||
from optimization_engine.nx import updater as nx_updater
|
||||
from optimization_engine.nx import session_manager as nx_session_manager
|
||||
from optimization_engine.nx import solve_simulation
|
||||
|
||||
# Study
|
||||
from optimization_engine.study import creator as study_creator
|
||||
from optimization_engine.study import wizard as study_wizard
|
||||
from optimization_engine.study import state as study_state
|
||||
|
||||
# Surrogates
|
||||
from optimization_engine.processors.surrogates import neural_surrogate
|
||||
from optimization_engine.processors.surrogates import generic_surrogate
|
||||
|
||||
# Config
|
||||
from optimization_engine.config import manager as config_manager
|
||||
|
||||
# Utils
|
||||
from optimization_engine.utils import logger
|
||||
|
||||
# Deprecation warnings (optional)
|
||||
import warnings
|
||||
def __getattr__(name):
|
||||
deprecated = {
|
||||
'nx_solver': 'optimization_engine.nx.solver',
|
||||
'study_creator': 'optimization_engine.study.creator',
|
||||
# ... more mappings
|
||||
}
|
||||
if name in deprecated:
|
||||
warnings.warn(
|
||||
f"Importing {name} from optimization_engine is deprecated. "
|
||||
f"Use {deprecated[name]} instead.",
|
||||
DeprecationWarning,
|
||||
stacklevel=2
|
||||
)
|
||||
# Return the module anyway for compatibility
|
||||
raise AttributeError(f"module 'optimization_engine' has no attribute '{name}'")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Part 4: Files Requiring Updates
|
||||
|
||||
### 4.1 Internal optimization_engine Files (90+ files)
|
||||
|
||||
**Highest-impact internal files:**
|
||||
|
||||
| File | Imports to Update | Priority |
|
||||
|------|-------------------|----------|
|
||||
| `runner.py` | nx_solver, config_manager, extractors | Critical |
|
||||
| `base_runner.py` | config_manager, validators | Critical |
|
||||
| `intelligent_optimizer.py` | runner, neural_surrogate, method_selector | Critical |
|
||||
| `runner_with_neural.py` | runner, neural_surrogate | High |
|
||||
| `gnn/gnn_optimizer.py` | polar_graph, zernike_gnn, nx_solver | High |
|
||||
| `hooks/nx_cad/*.py` | nx_session_manager | High |
|
||||
| `plugins/hook_manager.py` | validators, config | Medium |
|
||||
|
||||
### 4.2 Study Scripts (30+ folders, 153 imports)
|
||||
|
||||
**Pattern in every study's `run_optimization.py`:**
|
||||
|
||||
```python
|
||||
# BEFORE
|
||||
from optimization_engine.nx_solver import run_nx_simulation
|
||||
from optimization_engine.extractors import ZernikeExtractor
|
||||
from optimization_engine.gnn.gnn_optimizer import ZernikeGNNOptimizer
|
||||
|
||||
# AFTER
|
||||
from optimization_engine.nx.solver import run_nx_simulation
|
||||
from optimization_engine.extractors import ZernikeExtractor # unchanged
|
||||
from optimization_engine.gnn.gnn_optimizer import ZernikeGNNOptimizer # unchanged
|
||||
```
|
||||
|
||||
**Studies requiring updates:**
|
||||
|
||||
```
|
||||
studies/
|
||||
├── M1_Mirror/
|
||||
│ ├── m1_mirror_adaptive_V*/run_optimization.py (12 files)
|
||||
│ └── [other variants]
|
||||
├── Simple_Bracket/
|
||||
│ ├── bracket_*/run_optimization.py (8 files)
|
||||
├── UAV_Arm/
|
||||
│ ├── */run_optimization.py (4 files)
|
||||
├── Drone_Gimbal/
|
||||
├── Simple_Beam/
|
||||
└── [others]
|
||||
```
|
||||
|
||||
### 4.3 Test Files (30+ files, 79 imports)
|
||||
|
||||
Located in `tests/`:
|
||||
|
||||
```
|
||||
tests/
|
||||
├── test_extractors.py
|
||||
├── test_zernike_*.py (5+ files)
|
||||
├── test_neural_surrogate.py
|
||||
├── test_gnn_*.py
|
||||
├── test_nx_solver.py
|
||||
├── test_study_*.py
|
||||
└── [others]
|
||||
```
|
||||
|
||||
### 4.4 Dashboard Backend (3 files, 11 imports)
|
||||
|
||||
```
|
||||
atomizer-dashboard/backend/api/
|
||||
├── main.py # sys.path setup
|
||||
├── routes/optimization.py # study management imports
|
||||
└── routes/insights.py # insight imports
|
||||
```
|
||||
|
||||
### 4.5 Documentation Files (119 files)
|
||||
|
||||
**Protocols requiring updates:**
|
||||
|
||||
| Protocol | References | Changes Needed |
|
||||
|----------|------------|----------------|
|
||||
| SYS_10_IMSO.md | intelligent_optimizer | core.intelligent_optimizer |
|
||||
| SYS_12_EXTRACTOR_LIBRARY.md | extractors/* | None (unchanged) |
|
||||
| SYS_14_NEURAL_ACCELERATION.md | neural_surrogate, gnn/* | processors.surrogates.* |
|
||||
| SYS_15_METHOD_SELECTOR.md | method_selector | core.method_selector |
|
||||
| OP_01_CREATE_STUDY.md | study_creator, study_wizard | study.creator, study.wizard |
|
||||
| OP_02_RUN_OPTIMIZATION.md | runner, nx_solver | core.runner, nx.solver |
|
||||
|
||||
**Skill files requiring updates:**
|
||||
|
||||
| Skill File | Changes |
|
||||
|------------|---------|
|
||||
| 01_CHEATSHEET.md | Path references |
|
||||
| core/study-creation-core.md | Import examples |
|
||||
| modules/extractors-catalog.md | Directory paths |
|
||||
| modules/neural-acceleration.md | Surrogate imports |
|
||||
|
||||
### 4.6 JSON Configuration Files (4 files)
|
||||
|
||||
**feature_registry.json** (878 lines, 50+ path references):
|
||||
|
||||
```json
|
||||
// BEFORE
|
||||
"file_path": "optimization_engine/neural_surrogate.py"
|
||||
|
||||
// AFTER
|
||||
"file_path": "optimization_engine/processors/surrogates/neural_surrogate.py"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Part 5: Migration Execution Plan
|
||||
|
||||
### Phase 0: Pre-Migration (30 min)
|
||||
|
||||
1. **Create full backup**
|
||||
```bash
|
||||
git stash # Save any uncommitted changes
|
||||
git checkout -b refactor/optimization-engine-reorganization
|
||||
cp -r optimization_engine optimization_engine_backup
|
||||
```
|
||||
|
||||
2. **Run baseline tests**
|
||||
```bash
|
||||
python -m pytest tests/ -v --tb=short > baseline_tests.log 2>&1
|
||||
```
|
||||
|
||||
3. **Document current working state**
|
||||
- Run one study end-to-end
|
||||
- Verify dashboard loads
|
||||
- Note any existing failures
|
||||
|
||||
### Phase 1: Create Directory Structure (15 min)
|
||||
|
||||
```bash
|
||||
# Create new directories
|
||||
mkdir -p optimization_engine/core
|
||||
mkdir -p optimization_engine/processors/surrogates
|
||||
mkdir -p optimization_engine/processors/dynamic_response
|
||||
mkdir -p optimization_engine/nx
|
||||
mkdir -p optimization_engine/study
|
||||
mkdir -p optimization_engine/reporting
|
||||
mkdir -p optimization_engine/config
|
||||
|
||||
# Create __init__.py files
|
||||
touch optimization_engine/core/__init__.py
|
||||
touch optimization_engine/processors/__init__.py
|
||||
touch optimization_engine/processors/surrogates/__init__.py
|
||||
touch optimization_engine/processors/dynamic_response/__init__.py
|
||||
touch optimization_engine/nx/__init__.py
|
||||
touch optimization_engine/study/__init__.py
|
||||
touch optimization_engine/reporting/__init__.py
|
||||
touch optimization_engine/config/__init__.py
|
||||
```
|
||||
|
||||
### Phase 2: Move Files (30 min)
|
||||
|
||||
**Execute in this order to minimize circular import issues:**
|
||||
|
||||
```bash
|
||||
# 1. UTILITIES FIRST (no dependencies)
|
||||
mv optimization_engine/logger.py optimization_engine/utils/
|
||||
mv optimization_engine/auto_doc.py optimization_engine/utils/
|
||||
mv optimization_engine/realtime_tracking.py optimization_engine/utils/
|
||||
mv optimization_engine/codebase_analyzer.py optimization_engine/utils/
|
||||
mv optimization_engine/pruning_logger.py optimization_engine/utils/
|
||||
|
||||
# 2. CONFIG (low dependencies)
|
||||
mv optimization_engine/config_manager.py optimization_engine/config/manager.py
|
||||
mv optimization_engine/optimization_config_builder.py optimization_engine/config/builder.py
|
||||
mv optimization_engine/optimization_setup_wizard.py optimization_engine/config/setup_wizard.py
|
||||
mv optimization_engine/capability_matcher.py optimization_engine/config/capability_matcher.py
|
||||
|
||||
# 3. NX INTEGRATION
|
||||
mv optimization_engine/nx_solver.py optimization_engine/nx/solver.py
|
||||
mv optimization_engine/nx_updater.py optimization_engine/nx/updater.py
|
||||
mv optimization_engine/nx_session_manager.py optimization_engine/nx/session_manager.py
|
||||
mv optimization_engine/solve_simulation.py optimization_engine/nx/
|
||||
mv optimization_engine/solve_simulation_simple.py optimization_engine/nx/
|
||||
mv optimization_engine/model_cleanup.py optimization_engine/nx/
|
||||
mv optimization_engine/export_expressions.py optimization_engine/nx/
|
||||
mv optimization_engine/import_expressions.py optimization_engine/nx/
|
||||
mv optimization_engine/mesh_converter.py optimization_engine/nx/
|
||||
|
||||
# 4. SURROGATES
|
||||
mv optimization_engine/neural_surrogate.py optimization_engine/processors/surrogates/
|
||||
mv optimization_engine/generic_surrogate.py optimization_engine/processors/surrogates/
|
||||
mv optimization_engine/adaptive_surrogate.py optimization_engine/processors/surrogates/
|
||||
mv optimization_engine/simple_mlp_surrogate.py optimization_engine/processors/surrogates/
|
||||
mv optimization_engine/active_learning_surrogate.py optimization_engine/processors/surrogates/
|
||||
mv optimization_engine/surrogate_tuner.py optimization_engine/processors/surrogates/
|
||||
mv optimization_engine/auto_trainer.py optimization_engine/processors/surrogates/
|
||||
mv optimization_engine/training_data_exporter.py optimization_engine/processors/surrogates/
|
||||
|
||||
# 5. STUDY MANAGEMENT
|
||||
mv optimization_engine/study_creator.py optimization_engine/study/creator.py
|
||||
mv optimization_engine/study_wizard.py optimization_engine/study/wizard.py
|
||||
mv optimization_engine/study_state.py optimization_engine/study/state.py
|
||||
mv optimization_engine/study_reset.py optimization_engine/study/reset.py
|
||||
mv optimization_engine/study_continuation.py optimization_engine/study/continuation.py
|
||||
mv optimization_engine/benchmarking_substudy.py optimization_engine/study/benchmarking.py
|
||||
mv optimization_engine/generate_history_from_trials.py optimization_engine/study/history_generator.py
|
||||
|
||||
# 6. REPORTING
|
||||
mv optimization_engine/generate_report.py optimization_engine/reporting/report_generator.py
|
||||
mv optimization_engine/generate_report_markdown.py optimization_engine/reporting/markdown_report.py
|
||||
mv optimization_engine/comprehensive_results_analyzer.py optimization_engine/reporting/results_analyzer.py
|
||||
mv optimization_engine/visualizer.py optimization_engine/reporting/
|
||||
mv optimization_engine/landscape_analyzer.py optimization_engine/reporting/
|
||||
|
||||
# 7. CORE (depends on many things, do last)
|
||||
mv optimization_engine/runner.py optimization_engine/core/
|
||||
mv optimization_engine/base_runner.py optimization_engine/core/
|
||||
mv optimization_engine/runner_with_neural.py optimization_engine/core/
|
||||
mv optimization_engine/intelligent_optimizer.py optimization_engine/core/
|
||||
mv optimization_engine/method_selector.py optimization_engine/core/
|
||||
mv optimization_engine/strategy_selector.py optimization_engine/core/
|
||||
mv optimization_engine/strategy_portfolio.py optimization_engine/core/
|
||||
|
||||
# 8. RESEARCH/FUTURE
|
||||
mv optimization_engine/research_agent.py optimization_engine/future/
|
||||
mv optimization_engine/pynastran_research_agent.py optimization_engine/future/
|
||||
mv optimization_engine/targeted_research_planner.py optimization_engine/future/
|
||||
mv optimization_engine/workflow_decomposer.py optimization_engine/future/
|
||||
mv optimization_engine/step_classifier.py optimization_engine/future/
|
||||
|
||||
# 9. REMAINING MISC
|
||||
mv optimization_engine/op2_extractor.py optimization_engine/extractors/
|
||||
mv optimization_engine/extractor_library.py optimization_engine/extractors/
|
||||
mv optimization_engine/simulation_validator.py optimization_engine/validators/
|
||||
mv optimization_engine/adaptive_characterization.py optimization_engine/processors/
|
||||
```
|
||||
|
||||
### Phase 3: Update Internal Imports (1-2 hours)
|
||||
|
||||
**Use sed/grep for bulk updates:**
|
||||
|
||||
```bash
|
||||
# Example sed commands (run from project root)
|
||||
|
||||
# NX imports
|
||||
find . -name "*.py" -exec sed -i 's/from optimization_engine\.nx_solver/from optimization_engine.nx.solver/g' {} +
|
||||
find . -name "*.py" -exec sed -i 's/from optimization_engine\.nx_updater/from optimization_engine.nx.updater/g' {} +
|
||||
find . -name "*.py" -exec sed -i 's/from optimization_engine\.nx_session_manager/from optimization_engine.nx.session_manager/g' {} +
|
||||
find . -name "*.py" -exec sed -i 's/from optimization_engine\.solve_simulation/from optimization_engine.nx.solve_simulation/g' {} +
|
||||
|
||||
# Study imports
|
||||
find . -name "*.py" -exec sed -i 's/from optimization_engine\.study_creator/from optimization_engine.study.creator/g' {} +
|
||||
find . -name "*.py" -exec sed -i 's/from optimization_engine\.study_wizard/from optimization_engine.study.wizard/g' {} +
|
||||
find . -name "*.py" -exec sed -i 's/from optimization_engine\.study_state/from optimization_engine.study.state/g' {} +
|
||||
|
||||
# Config imports
|
||||
find . -name "*.py" -exec sed -i 's/from optimization_engine\.config_manager/from optimization_engine.config.manager/g' {} +
|
||||
|
||||
# Core imports
|
||||
find . -name "*.py" -exec sed -i 's/from optimization_engine\.runner import/from optimization_engine.core.runner import/g' {} +
|
||||
find . -name "*.py" -exec sed -i 's/from optimization_engine\.base_runner/from optimization_engine.core.base_runner/g' {} +
|
||||
find . -name "*.py" -exec sed -i 's/from optimization_engine\.intelligent_optimizer/from optimization_engine.core.intelligent_optimizer/g' {} +
|
||||
|
||||
# Surrogate imports
|
||||
find . -name "*.py" -exec sed -i 's/from optimization_engine\.neural_surrogate/from optimization_engine.processors.surrogates.neural_surrogate/g' {} +
|
||||
find . -name "*.py" -exec sed -i 's/from optimization_engine\.generic_surrogate/from optimization_engine.processors.surrogates.generic_surrogate/g' {} +
|
||||
|
||||
# Logger
|
||||
find . -name "*.py" -exec sed -i 's/from optimization_engine\.logger/from optimization_engine.utils.logger/g' {} +
|
||||
```
|
||||
|
||||
**Handle edge cases manually:**
|
||||
- `import optimization_engine.nx_solver` (without `from`)
|
||||
- Dynamic imports using `importlib`
|
||||
- String references in configs
|
||||
|
||||
### Phase 4: Create __init__.py Files (30 min)
|
||||
|
||||
**Example: `optimization_engine/core/__init__.py`**
|
||||
|
||||
```python
|
||||
"""
|
||||
Optimization Engine Core
|
||||
|
||||
Main optimization runners and algorithm selection.
|
||||
"""
|
||||
|
||||
from .runner import OptimizationRunner
|
||||
from .base_runner import BaseRunner
|
||||
from .intelligent_optimizer import IntelligentOptimizer, IMSO
|
||||
from .method_selector import MethodSelector
|
||||
from .strategy_selector import StrategySelector
|
||||
from .strategy_portfolio import StrategyPortfolio
|
||||
|
||||
__all__ = [
|
||||
'OptimizationRunner',
|
||||
'BaseRunner',
|
||||
'IntelligentOptimizer',
|
||||
'IMSO',
|
||||
'MethodSelector',
|
||||
'StrategySelector',
|
||||
'StrategyPortfolio',
|
||||
]
|
||||
```
|
||||
|
||||
**Create similar for each new directory.**
|
||||
|
||||
### Phase 5: Add Backwards Compatibility (30 min)
|
||||
|
||||
Update `optimization_engine/__init__.py`:
|
||||
|
||||
```python
|
||||
"""
|
||||
Optimization Engine for Atomizer
|
||||
|
||||
Reorganized structure (v2.0):
|
||||
- core/ - Optimization runners
|
||||
- processors/ - Data processing (surrogates, dynamic response)
|
||||
- nx/ - NX/Nastran integration
|
||||
- study/ - Study management
|
||||
- reporting/ - Reports and analysis
|
||||
- config/ - Configuration
|
||||
- extractors/ - Physics extraction
|
||||
- insights/ - Visualizations
|
||||
- gnn/ - Graph neural networks
|
||||
- hooks/ - NX hooks
|
||||
"""
|
||||
|
||||
# Re-export commonly used items at top level for convenience
|
||||
from optimization_engine.core.runner import OptimizationRunner
|
||||
from optimization_engine.core.intelligent_optimizer import IMSO
|
||||
from optimization_engine.nx.solver import run_nx_simulation, NXSolver
|
||||
from optimization_engine.study.creator import create_study
|
||||
from optimization_engine.config.manager import ConfigManager
|
||||
|
||||
# Backwards compatibility aliases (deprecated)
|
||||
# These will be removed in a future version
|
||||
import warnings as _warnings
|
||||
|
||||
def _deprecated_import(old_name, new_location):
|
||||
_warnings.warn(
|
||||
f"Importing '{old_name}' directly from optimization_engine is deprecated. "
|
||||
f"Use '{new_location}' instead.",
|
||||
DeprecationWarning,
|
||||
stacklevel=3
|
||||
)
|
||||
|
||||
# Lazy loading for backwards compatibility
|
||||
def __getattr__(name):
|
||||
# Map old names to new locations
|
||||
_compat_map = {
|
||||
'nx_solver': ('optimization_engine.nx.solver', 'nx_solver'),
|
||||
'nx_updater': ('optimization_engine.nx.updater', 'nx_updater'),
|
||||
'study_creator': ('optimization_engine.study.creator', 'study_creator'),
|
||||
'config_manager': ('optimization_engine.config.manager', 'config_manager'),
|
||||
'runner': ('optimization_engine.core.runner', 'runner'),
|
||||
'neural_surrogate': ('optimization_engine.processors.surrogates.neural_surrogate', 'neural_surrogate'),
|
||||
}
|
||||
|
||||
if name in _compat_map:
|
||||
module_path, attr = _compat_map[name]
|
||||
_deprecated_import(name, module_path)
|
||||
import importlib
|
||||
module = importlib.import_module(module_path)
|
||||
return module
|
||||
|
||||
raise AttributeError(f"module 'optimization_engine' has no attribute '{name}'")
|
||||
```
|
||||
|
||||
### Phase 6: Update Documentation (1 hour)
|
||||
|
||||
**Use sed for bulk updates in markdown:**
|
||||
|
||||
```bash
|
||||
# Update protocol files
|
||||
find docs/protocols -name "*.md" -exec sed -i 's/optimization_engine\/nx_solver/optimization_engine\/nx\/solver/g' {} +
|
||||
find docs/protocols -name "*.md" -exec sed -i 's/optimization_engine\/study_creator/optimization_engine\/study\/creator/g' {} +
|
||||
# ... etc for all mappings
|
||||
|
||||
# Update .claude skills
|
||||
find .claude -name "*.md" -exec sed -i 's/optimization_engine\.nx_solver/optimization_engine.nx.solver/g' {} +
|
||||
# ... etc
|
||||
```
|
||||
|
||||
**Manual review needed for:**
|
||||
- Code examples in markdown
|
||||
- Directory structure diagrams
|
||||
- Command-line examples
|
||||
|
||||
### Phase 7: Update JSON Configs (30 min)
|
||||
|
||||
**feature_registry.json** - Use a Python script:
|
||||
|
||||
```python
|
||||
import json
|
||||
|
||||
with open('optimization_engine/feature_registry.json', 'r') as f:
|
||||
registry = json.load(f)
|
||||
|
||||
# Define path mappings
|
||||
path_map = {
|
||||
'optimization_engine/neural_surrogate.py': 'optimization_engine/processors/surrogates/neural_surrogate.py',
|
||||
'optimization_engine/nx_solver.py': 'optimization_engine/nx/solver.py',
|
||||
# ... all other mappings
|
||||
}
|
||||
|
||||
def update_paths(obj):
|
||||
if isinstance(obj, dict):
|
||||
for key, value in obj.items():
|
||||
if key == 'file_path' and value in path_map:
|
||||
obj[key] = path_map[value]
|
||||
else:
|
||||
update_paths(value)
|
||||
elif isinstance(obj, list):
|
||||
for item in obj:
|
||||
update_paths(item)
|
||||
|
||||
update_paths(registry)
|
||||
|
||||
with open('optimization_engine/feature_registry.json', 'w') as f:
|
||||
json.dump(registry, f, indent=2)
|
||||
```
|
||||
|
||||
### Phase 8: Testing & Validation (1-2 hours)
|
||||
|
||||
```bash
|
||||
# 1. Run Python import tests
|
||||
python -c "from optimization_engine.core.runner import OptimizationRunner"
|
||||
python -c "from optimization_engine.nx.solver import run_nx_simulation"
|
||||
python -c "from optimization_engine.study.creator import create_study"
|
||||
python -c "from optimization_engine.processors.surrogates.neural_surrogate import NeuralSurrogate"
|
||||
|
||||
# 2. Run backwards compatibility tests
|
||||
python -c "from optimization_engine import nx_solver" # Should work with deprecation warning
|
||||
|
||||
# 3. Run test suite
|
||||
python -m pytest tests/ -v --tb=short
|
||||
|
||||
# 4. Test a study end-to-end
|
||||
cd studies/Simple_Bracket/bracket_displacement_maximizing
|
||||
python run_optimization.py --trials 2 --dry-run
|
||||
|
||||
# 5. Test dashboard
|
||||
cd atomizer-dashboard
|
||||
python backend/api/main.py # Should start without import errors
|
||||
```
|
||||
|
||||
### Phase 9: Cleanup (15 min)
|
||||
|
||||
```bash
|
||||
# Remove backup after successful testing
|
||||
rm -rf optimization_engine_backup
|
||||
|
||||
# Remove any .pyc files that might cache old imports
|
||||
find . -name "*.pyc" -delete
|
||||
find . -name "__pycache__" -type d -exec rm -rf {} +
|
||||
|
||||
# Commit
|
||||
git add -A
|
||||
git commit -m "refactor: Reorganize optimization_engine into modular structure
|
||||
|
||||
- Move 50+ top-level files into logical subdirectories
|
||||
- Create core/, processors/, nx/, study/, reporting/, config/
|
||||
- Add backwards compatibility aliases with deprecation warnings
|
||||
- Update all imports across codebase (640+ changes)
|
||||
- Update documentation and protocols
|
||||
|
||||
BREAKING: Direct imports from optimization_engine.* are deprecated.
|
||||
Use new paths like optimization_engine.core.runner instead.
|
||||
|
||||
🤖 Generated with [Claude Code](https://claude.com/claude-code)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Part 6: Rollback Plan
|
||||
|
||||
If migration fails:
|
||||
|
||||
```bash
|
||||
# Option 1: Git reset (if not committed)
|
||||
git checkout -- .
|
||||
|
||||
# Option 2: Restore backup
|
||||
rm -rf optimization_engine
|
||||
mv optimization_engine_backup optimization_engine
|
||||
|
||||
# Option 3: Git revert (if committed)
|
||||
git revert HEAD
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Part 7: Post-Migration Tasks
|
||||
|
||||
### 7.1 Update LAC Knowledge Base
|
||||
|
||||
Record the migration in LAC:
|
||||
|
||||
```python
|
||||
from knowledge_base.lac import get_lac
|
||||
lac = get_lac()
|
||||
lac.record_insight(
|
||||
category="architecture",
|
||||
context="optimization_engine reorganization",
|
||||
insight="Migrated 50+ files into modular structure. New paths: "
|
||||
"core/, processors/, nx/, study/, reporting/, config/. "
|
||||
"Backwards compat aliases provided with deprecation warnings.",
|
||||
confidence=1.0,
|
||||
tags=["refactoring", "architecture", "breaking-change"]
|
||||
)
|
||||
```
|
||||
|
||||
### 7.2 Update CLAUDE.md
|
||||
|
||||
Add section about new structure.
|
||||
|
||||
### 7.3 Schedule Deprecation Removal
|
||||
|
||||
After 2-4 weeks of stable operation:
|
||||
- Remove backwards compatibility aliases
|
||||
- Update remaining old imports
|
||||
- Clean up __init__.py files
|
||||
|
||||
---
|
||||
|
||||
## Part 8: Time Estimate Summary
|
||||
|
||||
| Phase | Task | Time |
|
||||
|-------|------|------|
|
||||
| 0 | Pre-migration (backup, baseline) | 30 min |
|
||||
| 1 | Create directory structure | 15 min |
|
||||
| 2 | Move files | 30 min |
|
||||
| 3 | Update internal imports | 1-2 hours |
|
||||
| 4 | Create __init__.py files | 30 min |
|
||||
| 5 | Add backwards compatibility | 30 min |
|
||||
| 6 | Update documentation | 1 hour |
|
||||
| 7 | Update JSON configs | 30 min |
|
||||
| 8 | Testing & validation | 1-2 hours |
|
||||
| 9 | Cleanup & commit | 15 min |
|
||||
| **TOTAL** | | **6-8 hours** |
|
||||
|
||||
---
|
||||
|
||||
## Part 9: Decision Points
|
||||
|
||||
### Option A: Full Migration Now
|
||||
- **Pros**: Clean structure, enables dynamic_response cleanly
|
||||
- **Cons**: 6-8 hours work, risk of breakage
|
||||
- **When**: When you have a dedicated half-day
|
||||
|
||||
### Option B: Minimal Migration (processors/ only)
|
||||
- **Pros**: Low risk, enables dynamic_response
|
||||
- **Cons**: Leaves technical debt, inconsistent structure
|
||||
- **When**: If you need dynamic_response urgently
|
||||
|
||||
### Option C: Defer Migration
|
||||
- **Pros**: Zero risk now
|
||||
- **Cons**: Dynamic_response goes in awkward location
|
||||
- **When**: If stability is critical
|
||||
|
||||
---
|
||||
|
||||
## Appendix A: Automated Migration Script
|
||||
|
||||
A Python script to automate most of the migration:
|
||||
|
||||
```python
|
||||
#!/usr/bin/env python
|
||||
"""
|
||||
optimization_engine Migration Script
|
||||
|
||||
Run with: python migrate_optimization_engine.py --dry-run
|
||||
Then: python migrate_optimization_engine.py --execute
|
||||
"""
|
||||
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
|
||||
# ... full script would be ~200 lines
|
||||
# Handles: directory creation, file moves, import updates, __init__ generation
|
||||
```
|
||||
|
||||
Would you like me to create this full automation script?
|
||||
|
||||
---
|
||||
|
||||
*This plan ensures a safe, reversible migration with clear validation steps.*
|
||||
15
CLAUDE.md
15
CLAUDE.md
@@ -108,12 +108,23 @@ The Protocol Operating System (POS) provides layered documentation:
|
||||
|
||||
**CRITICAL: Always use the `atomizer` conda environment.**
|
||||
|
||||
### Paths (DO NOT SEARCH - use these directly)
|
||||
```
|
||||
Python: C:\Users\antoi\anaconda3\envs\atomizer\python.exe
|
||||
Conda: C:\Users\antoi\anaconda3\Scripts\conda.exe
|
||||
```
|
||||
|
||||
### Running Python Scripts
|
||||
```bash
|
||||
conda activate atomizer
|
||||
python run_optimization.py
|
||||
# Option 1: PowerShell with conda activate (RECOMMENDED)
|
||||
powershell -Command "conda activate atomizer; python your_script.py"
|
||||
|
||||
# Option 2: Direct path (no activation needed)
|
||||
C:\Users\antoi\anaconda3\envs\atomizer\python.exe your_script.py
|
||||
```
|
||||
|
||||
**DO NOT:**
|
||||
- Search for Python paths (`where python`, etc.) - they're documented above
|
||||
- Install packages with pip/conda (everything is installed)
|
||||
- Create new virtual environments
|
||||
- Use system Python
|
||||
|
||||
@@ -155,9 +155,9 @@ export default function Setup() {
|
||||
})),
|
||||
algorithm: {
|
||||
name: rawConfig.optimizer?.name || rawConfig.algorithm?.name || 'Optuna',
|
||||
sampler: rawConfig.optimization_settings?.sampler || rawConfig.algorithm?.sampler || 'TPESampler',
|
||||
sampler: rawConfig.optimization?.algorithm || rawConfig.optimization_settings?.sampler || rawConfig.algorithm?.sampler || 'TPESampler',
|
||||
pruner: rawConfig.optimization_settings?.pruner || rawConfig.algorithm?.pruner,
|
||||
n_trials: rawConfig.optimization_settings?.n_trials || rawConfig.trials?.n_trials || selectedStudy.progress.total,
|
||||
n_trials: rawConfig.optimization?.n_trials || rawConfig.optimization_settings?.n_trials || rawConfig.trials?.n_trials || selectedStudy.progress.total,
|
||||
timeout: rawConfig.optimization_settings?.timeout
|
||||
},
|
||||
fea_model: rawConfig.fea_model || rawConfig.solver ? {
|
||||
|
||||
35
atomizer-dashboard/restart-dev.bat
Normal file
35
atomizer-dashboard/restart-dev.bat
Normal file
@@ -0,0 +1,35 @@
|
||||
@echo off
|
||||
REM Atomizer Dashboard - Development Restart Script
|
||||
REM Kills existing processes and restarts both backend and frontend
|
||||
|
||||
echo ========================================
|
||||
echo Atomizer Dashboard - Restart
|
||||
echo ========================================
|
||||
|
||||
REM Kill existing processes on ports 8000 and 5173
|
||||
echo Stopping existing processes...
|
||||
for /f "tokens=5" %%a in ('netstat -ano ^| findstr :8000 ^| findstr LISTENING') do (
|
||||
taskkill /F /PID %%a 2>nul
|
||||
)
|
||||
for /f "tokens=5" %%a in ('netstat -ano ^| findstr :5173 ^| findstr LISTENING') do (
|
||||
taskkill /F /PID %%a 2>nul
|
||||
)
|
||||
|
||||
timeout /t 2 /nobreak >nul
|
||||
|
||||
REM Start backend in new window
|
||||
echo Starting backend...
|
||||
start "Atomizer Backend" cmd /k "cd /d %~dp0backend && conda activate atomizer && uvicorn api.main:app --reload --host 0.0.0.0 --port 8000"
|
||||
|
||||
timeout /t 3 /nobreak >nul
|
||||
|
||||
REM Start frontend in new window
|
||||
echo Starting frontend...
|
||||
start "Atomizer Frontend" cmd /k "cd /d %~dp0frontend && npm run dev"
|
||||
|
||||
echo ========================================
|
||||
echo Dashboard restarted!
|
||||
echo Backend: http://localhost:8000
|
||||
echo Frontend: http://localhost:5173
|
||||
echo ========================================
|
||||
pause
|
||||
59
docs/06_PHYSICS/00_INDEX.md
Normal file
59
docs/06_PHYSICS/00_INDEX.md
Normal file
@@ -0,0 +1,59 @@
|
||||
# Physics Documentation Index
|
||||
|
||||
This folder contains detailed physics and domain-specific documentation for Atomizer's analysis capabilities. These documents explain the **engineering and scientific foundations** behind the extractors and insights.
|
||||
|
||||
---
|
||||
|
||||
## Document Catalog
|
||||
|
||||
| Document | Topic | When to Read |
|
||||
|----------|-------|--------------|
|
||||
| [ZERNIKE_FUNDAMENTALS.md](ZERNIKE_FUNDAMENTALS.md) | Zernike polynomial basics, RMS calculation, multi-subcase analysis | Setting up mirror optimization, understanding WFE metrics |
|
||||
| [ZERNIKE_OPD_METHOD.md](ZERNIKE_OPD_METHOD.md) | **Rigorous OPD method** for lateral displacement correction | Lateral support optimization, validating WFE accuracy |
|
||||
|
||||
---
|
||||
|
||||
## Quick Navigation
|
||||
|
||||
### For Mirror/Optics Optimization
|
||||
|
||||
1. **New to Zernike?** Start with [ZERNIKE_FUNDAMENTALS.md](ZERNIKE_FUNDAMENTALS.md)
|
||||
2. **Lateral support optimization?** Read [ZERNIKE_OPD_METHOD.md](ZERNIKE_OPD_METHOD.md) - **critical**
|
||||
3. **Mid-spatial frequency analysis?** See SYS_16 (msf_zernike insight)
|
||||
|
||||
### For Structural Optimization
|
||||
|
||||
- Stress extraction: See `SYS_12_EXTRACTOR_LIBRARY.md` (E3, E12)
|
||||
- Strain energy: See `SYS_12_EXTRACTOR_LIBRARY.md` (E13)
|
||||
|
||||
### For Thermal Analysis
|
||||
|
||||
- Temperature extraction: See `SYS_12_EXTRACTOR_LIBRARY.md` (E15-E17)
|
||||
|
||||
---
|
||||
|
||||
## Related Documentation
|
||||
|
||||
| Location | Content |
|
||||
|----------|---------|
|
||||
| `.claude/skills/modules/extractors-catalog.md` | Quick extractor lookup |
|
||||
| `.claude/skills/modules/insights-catalog.md` | Quick insight lookup |
|
||||
| `docs/protocols/system/SYS_12_EXTRACTOR_LIBRARY.md` | Extractor specifications |
|
||||
| `docs/protocols/system/SYS_16_STUDY_INSIGHTS.md` | Insight specifications |
|
||||
|
||||
---
|
||||
|
||||
## Contributing Physics Documentation
|
||||
|
||||
When adding new physics documentation:
|
||||
|
||||
1. **Naming**: Use `{TOPIC}_{SUBTOPIC}.md` format (e.g., `THERMAL_GRADIENTS.md`)
|
||||
2. **Structure**: Follow the pattern in existing documents:
|
||||
- Executive Summary
|
||||
- Mathematical Formulation
|
||||
- When This Matters
|
||||
- Implementation Details
|
||||
- Usage Guide
|
||||
- Validation
|
||||
3. **Cross-reference**: Update this index and related skill modules
|
||||
4. **Link to code**: Reference the implementing extractors/insights
|
||||
@@ -308,6 +308,26 @@ studies/
|
||||
|
||||
## See Also
|
||||
|
||||
### Related Physics Documentation
|
||||
|
||||
- [ZERNIKE_OPD_METHOD.md](ZERNIKE_OPD_METHOD.md) - **Rigorous OPD method for lateral displacement correction** (critical for lateral support optimization)
|
||||
|
||||
### Protocol Documentation
|
||||
|
||||
- `docs/protocols/system/SYS_12_EXTRACTOR_LIBRARY.md` - Extractor specifications (E8-E10: Standard Zernike, E20-E21: OPD method)
|
||||
- `docs/protocols/system/SYS_16_STUDY_INSIGHTS.md` - Insight specifications (`zernike_wfe`, `zernike_opd_comparison`)
|
||||
|
||||
### Skill Modules (Quick Lookup)
|
||||
|
||||
- `.claude/skills/modules/extractors-catalog.md` - Quick extractor reference
|
||||
- `.claude/skills/modules/insights-catalog.md` - Quick insight reference
|
||||
|
||||
### Code Implementation
|
||||
|
||||
- [optimization_engine/extractors/extract_zernike.py](../../optimization_engine/extractors/extract_zernike.py) - Standard Zernike extractor
|
||||
- [optimization_engine/extractors/extract_zernike_opd.py](../../optimization_engine/extractors/extract_zernike_opd.py) - **OPD-based extractor** (use for lateral supports)
|
||||
- [optimization_engine/extractors/zernike_helpers.py](../../optimization_engine/extractors/zernike_helpers.py) - Helper functions and objective builders
|
||||
|
||||
### Example Configurations
|
||||
|
||||
- [examples/optimization_config_zernike_mirror.json](../examples/optimization_config_zernike_mirror.json) - Full example configuration
|
||||
- [optimization_engine/extractors/extract_zernike.py](../optimization_engine/extractors/extract_zernike.py) - Core implementation
|
||||
- [optimization_engine/extractors/zernike_helpers.py](../optimization_engine/extractors/zernike_helpers.py) - Helper functions
|
||||
579
docs/06_PHYSICS/ZERNIKE_OPD_METHOD.md
Normal file
579
docs/06_PHYSICS/ZERNIKE_OPD_METHOD.md
Normal file
@@ -0,0 +1,579 @@
|
||||
# Rigorous OPD-Based Zernike Analysis for Mirror Optimization
|
||||
|
||||
**Document Version**: 1.0
|
||||
**Created**: 2024-12-22
|
||||
**Author**: Atomizer Framework
|
||||
**Status**: Active
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
This document describes a **rigorous Optical Path Difference (OPD)** method for computing Zernike wavefront error that correctly accounts for **lateral (X, Y) displacements** in addition to axial (Z) displacements.
|
||||
|
||||
**The Problem**: Standard Zernike analysis uses only Z-displacement at the original (x, y) node positions. When supports pinch the mirror or lateral forces cause in-plane deformation, nodes shift in X and Y. The standard method is **blind to this**, potentially leading to:
|
||||
- Optimized designs that appear good but have poor actual optical performance
|
||||
- Optimizer convergence to non-optimal solutions that "cheat" by distorting laterally
|
||||
|
||||
**The Solution**: The OPD method computes the true surface error by accounting for the fact that a laterally-displaced node should be compared against the parabola height **at its new (x+dx, y+dy) position**, not its original position.
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [The Optical Physics Problem](#1-the-optical-physics-problem)
|
||||
2. [Mathematical Formulation](#2-mathematical-formulation)
|
||||
3. [When This Matters](#3-when-this-matters)
|
||||
4. [Implementation Details](#4-implementation-details)
|
||||
5. [Usage Guide](#5-usage-guide)
|
||||
6. [Validation and Testing](#6-validation-and-testing)
|
||||
7. [Migration Guide](#7-migration-guide)
|
||||
|
||||
---
|
||||
|
||||
## 1. The Optical Physics Problem
|
||||
|
||||
### 1.1 What Zernike Analysis Does
|
||||
|
||||
Zernike polynomials decompose a wavefront error surface into orthogonal modes:
|
||||
|
||||
```
|
||||
W(r, θ) = Σ cⱼ Zⱼ(r, θ)
|
||||
```
|
||||
|
||||
Where:
|
||||
- `W` = wavefront error (nm)
|
||||
- `cⱼ` = Zernike coefficient for mode j
|
||||
- `Zⱼ` = Zernike polynomial (Noll indexing)
|
||||
|
||||
For a reflective mirror, the wavefront error is **twice** the surface error:
|
||||
```
|
||||
WFE = 2 × surface_error
|
||||
```
|
||||
|
||||
### 1.2 Standard Method (Z-Only)
|
||||
|
||||
The standard approach:
|
||||
1. Read node original positions `(x₀, y₀, z₀)` from BDF/DAT
|
||||
2. Read displacement vector `(Δx, Δy, Δz)` from OP2
|
||||
3. Compute surface error = `Δz` (Z-displacement only)
|
||||
4. Compute WFE = `2 × Δz × nm_scale`
|
||||
5. Fit Zernike at original coordinates `(x₀, y₀)`
|
||||
|
||||
```python
|
||||
# Standard method (simplified)
|
||||
for nid, (dx, dy, dz) in displacements:
|
||||
x, y, z = original_coords[nid]
|
||||
wfe = dz * 2 * nm_scale # ONLY uses Z-displacement
|
||||
X.append(x) # Original X
|
||||
Y.append(y) # Original Y
|
||||
WFE.append(wfe)
|
||||
|
||||
coeffs = fit_zernike(X, Y, WFE)
|
||||
```
|
||||
|
||||
### 1.3 The Problem: Lateral Displacement is Ignored
|
||||
|
||||
Consider a node on a parabolic mirror:
|
||||
- **Original position**: `(x₀, y₀, z₀)` where `z₀ = -r₀²/(4f)` on the parabola
|
||||
- **Deformed position**: `(x₀+Δx, y₀+Δy, z₀+Δz)`
|
||||
|
||||
**Question**: What is the true surface error?
|
||||
|
||||
**Standard method says**: surface_error = `Δz`
|
||||
|
||||
**But this is wrong!** If the node moved laterally to a new `(x, y)`, the ideal parabola has a **different** Z at that location. The node should be compared against:
|
||||
|
||||
```
|
||||
z_expected = parabola(x₀+Δx, y₀+Δy) = -(x₀+Δx)² + (y₀+Δy)² / (4f)
|
||||
```
|
||||
|
||||
Not against `z₀ = parabola(x₀, y₀)`.
|
||||
|
||||
### 1.4 Visual Example
|
||||
|
||||
```
|
||||
Original parabola
|
||||
___
|
||||
_/ \_
|
||||
/ \
|
||||
/ *A \ A = original node at (x₀, y₀, z₀)
|
||||
/ ↗ ↘ \ B = deformed position (x₀+Δx, y₀+Δy, z₀+Δz)
|
||||
/ B C \ C = where node SHOULD be if staying on parabola
|
||||
/ \
|
||||
/_____________________\
|
||||
|
||||
Standard method: error = z_B - z_A = Δz
|
||||
(compares B to A vertically)
|
||||
|
||||
OPD method: error = z_B - z_C = Δz - Δz_parabola
|
||||
(compares B to where parabola is at B's (x,y))
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. Mathematical Formulation
|
||||
|
||||
### 2.1 Differential OPD Formulation
|
||||
|
||||
For a paraboloid with optical axis along Z:
|
||||
|
||||
```
|
||||
z = -r² / (4f) [concave mirror, vertex at origin]
|
||||
```
|
||||
|
||||
Where:
|
||||
- `r² = x² + y²`
|
||||
- `f` = focal length
|
||||
|
||||
**Key Insight**: We can compute the **change** in parabola Z due to lateral movement:
|
||||
|
||||
```
|
||||
Δz_parabola = z(x₀+Δx, y₀+Δy) - z(x₀, y₀)
|
||||
= -[(x₀+Δx)² + (y₀+Δy)²] / (4f) - [-( x₀² + y₀²) / (4f)]
|
||||
= -[r_def² - r₀²] / (4f)
|
||||
= -Δr² / (4f)
|
||||
```
|
||||
|
||||
Where:
|
||||
```
|
||||
Δr² = r_def² - r₀² = (x₀+Δx)² + (y₀+Δy)² - x₀² - y₀²
|
||||
= 2·x₀·Δx + Δx² + 2·y₀·Δy + Δy²
|
||||
```
|
||||
|
||||
### 2.2 True Surface Error
|
||||
|
||||
The true surface error is:
|
||||
|
||||
```
|
||||
surface_error = Δz - Δz_parabola
|
||||
= Δz - (-Δr² / 4f)
|
||||
= Δz + Δr² / (4f)
|
||||
```
|
||||
|
||||
**Interpretation**:
|
||||
- If a node moves **outward** (larger r), it should also move in **-Z** to stay on the concave parabola
|
||||
- If the FEA says it moved by `Δz`, but staying on the parabola requires `Δz_parabola`, the difference is the true error
|
||||
- This corrects for the "false error" that the standard method counts when nodes shift laterally
|
||||
|
||||
### 2.3 Wavefront Error
|
||||
|
||||
```
|
||||
WFE = 2 × surface_error × nm_scale
|
||||
= 2 × (Δz - Δz_parabola) × nm_scale
|
||||
```
|
||||
|
||||
### 2.4 Zernike Fitting Coordinates
|
||||
|
||||
Another subtlety: the Zernike fit should use the **deformed** coordinates `(x₀+Δx, y₀+Δy)` rather than the original coordinates. This is because the WFE surface represents the error at the positions where the nodes **actually are** after deformation.
|
||||
|
||||
```python
|
||||
# OPD method
|
||||
X_fit = x0 + dx # Deformed X
|
||||
Y_fit = y0 + dy # Deformed Y
|
||||
WFE = surface_error * 2 * nm_scale
|
||||
|
||||
coeffs = fit_zernike(X_fit, Y_fit, WFE)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. When This Matters
|
||||
|
||||
### 3.1 Magnitude Analysis
|
||||
|
||||
The correction term is:
|
||||
|
||||
```
|
||||
Δz_parabola = -Δr² / (4f) ≈ -(2·x₀·Δx + 2·y₀·Δy) / (4f) [ignoring Δx², Δy²]
|
||||
≈ -(x₀·Δx + y₀·Δy) / (2f)
|
||||
```
|
||||
|
||||
For a node at radius `r₀` with tangential displacement `Δ_tangential`:
|
||||
- The correction is approximately: `r₀ · Δ_lateral / (2f)`
|
||||
|
||||
**Example**: Mirror with f = 5000 mm, outer radius = 400 mm
|
||||
- Node at r = 400 mm shifts laterally by Δx = 0.001 mm (1 µm)
|
||||
- Correction: `400 × 0.001 / (2 × 5000) = 0.00004 mm = 40 nm`
|
||||
|
||||
This is **significant** when typical WFE is in the 10-100 nm range!
|
||||
|
||||
### 3.2 Classification by Load Case
|
||||
|
||||
| Load Case | Lateral Disp. | Method Impact |
|
||||
|-----------|--------------|---------------|
|
||||
| **Axial support** (gravity in Z) | Very small | Minimal - both methods similar |
|
||||
| **Lateral support** (gravity in X/Y) | **Large** | **Significant** - OPD method required |
|
||||
| **Clamp/fixture forces** | Can be large locally | May be significant at pinch points |
|
||||
| **Thermal** | Variable | Depends on thermal gradients |
|
||||
| **Mirror cell deflection** | Variable | Check lateral displacement magnitude |
|
||||
|
||||
### 3.3 Diagnostic Thresholds
|
||||
|
||||
The `ZernikeOPDExtractor` provides lateral displacement statistics:
|
||||
|
||||
| Max Lateral Disp. | Recommendation |
|
||||
|-------------------|----------------|
|
||||
| > 10 µm | **CRITICAL**: OPD method required |
|
||||
| 1 - 10 µm | **RECOMMENDED**: OPD method provides meaningful improvement |
|
||||
| 0.1 - 1 µm | **OPTIONAL**: OPD method provides minor improvement |
|
||||
| < 0.1 µm | **EQUIVALENT**: Both methods give essentially identical results |
|
||||
|
||||
---
|
||||
|
||||
## 4. Implementation Details
|
||||
|
||||
### 4.1 Module: `extract_zernike_opd.py`
|
||||
|
||||
Location: `optimization_engine/extractors/extract_zernike_opd.py`
|
||||
|
||||
**Key Functions**:
|
||||
|
||||
```python
|
||||
def compute_true_opd(x0, y0, z0, dx, dy, dz, focal_length, concave=True):
|
||||
"""
|
||||
Compute true surface error accounting for lateral displacement.
|
||||
|
||||
Returns:
|
||||
x_def: Deformed X coordinates
|
||||
y_def: Deformed Y coordinates
|
||||
surface_error: True surface error (not just Δz)
|
||||
lateral_magnitude: |Δx, Δy| for diagnostics
|
||||
"""
|
||||
```
|
||||
|
||||
```python
|
||||
def estimate_focal_length_from_geometry(x, y, z, concave=True):
|
||||
"""
|
||||
Estimate parabola focal length by fitting z = a·r² + b.
|
||||
Focal length = 1 / (4·|a|)
|
||||
"""
|
||||
```
|
||||
|
||||
**Main Class**:
|
||||
|
||||
```python
|
||||
class ZernikeOPDExtractor:
|
||||
"""
|
||||
Rigorous OPD-based Zernike extractor.
|
||||
|
||||
Key differences from ZernikeExtractor:
|
||||
- Uses deformed (x, y) coordinates for fitting
|
||||
- Computes surface error relative to parabola at deformed position
|
||||
- Provides lateral displacement diagnostics
|
||||
"""
|
||||
```
|
||||
|
||||
### 4.2 Algorithm Flow
|
||||
|
||||
```
|
||||
1. Load geometry (BDF) and displacements (OP2)
|
||||
|
||||
2. For each node:
|
||||
a. Get original position: (x₀, y₀, z₀)
|
||||
b. Get displacement: (Δx, Δy, Δz)
|
||||
c. Compute deformed position: (x_def, y_def) = (x₀+Δx, y₀+Δy)
|
||||
d. Compute Δr² = r_def² - r₀²
|
||||
e. Compute Δz_parabola = -Δr² / (4f) [for concave]
|
||||
f. Compute surface_error = Δz - Δz_parabola
|
||||
g. Store lateral_disp = √(Δx² + Δy²)
|
||||
|
||||
3. Convert to WFE: WFE = 2 × surface_error × nm_scale
|
||||
|
||||
4. Fit Zernike coefficients using (x_def, y_def, WFE)
|
||||
|
||||
5. Compute RMS metrics:
|
||||
- Global RMS = √(mean(WFE²))
|
||||
- Filtered RMS = √(mean((WFE - low_order_fit)²))
|
||||
```
|
||||
|
||||
### 4.3 Focal Length Handling
|
||||
|
||||
The extractor can:
|
||||
1. Use a **provided** focal length (most accurate)
|
||||
2. **Auto-estimate** from geometry by fitting `z = a·r² + b`
|
||||
|
||||
Auto-estimation works well for clean parabolic meshes but may need manual override for:
|
||||
- Off-axis parabolas
|
||||
- Aspheric surfaces
|
||||
- Meshes with significant manufacturing errors
|
||||
|
||||
```python
|
||||
# Explicit focal length
|
||||
extractor = ZernikeOPDExtractor(op2_file, focal_length=5000.0)
|
||||
|
||||
# Auto-estimate (default)
|
||||
extractor = ZernikeOPDExtractor(op2_file, auto_estimate_focal=True)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Usage Guide
|
||||
|
||||
### 5.1 Quick Comparison Test
|
||||
|
||||
Run the test script to see how much the methods differ for your data:
|
||||
|
||||
```bash
|
||||
conda activate atomizer
|
||||
python test_zernike_opd_comparison.py
|
||||
```
|
||||
|
||||
Output example:
|
||||
```
|
||||
--- Standard Method (Z-only) ---
|
||||
Global RMS: 171.65 nm
|
||||
Filtered RMS: 28.72 nm
|
||||
|
||||
--- Rigorous OPD Method ---
|
||||
Global RMS: 171.89 nm
|
||||
Filtered RMS: 29.15 nm
|
||||
|
||||
--- Difference (OPD - Standard) ---
|
||||
Filtered RMS: +0.43 nm (+1.5%)
|
||||
|
||||
--- Lateral Displacement ---
|
||||
Max: 0.156 µm
|
||||
RMS: 0.111 µm
|
||||
|
||||
>>> OPTIONAL: Small lateral displacements. OPD method provides minor improvement.
|
||||
```
|
||||
|
||||
### 5.2 Using in Optimization
|
||||
|
||||
**For new studies**, use the OPD extractor:
|
||||
|
||||
```python
|
||||
from optimization_engine.extractors import extract_zernike_opd_filtered_rms
|
||||
|
||||
def objective(trial):
|
||||
# ... parameter suggestion and FEA solve ...
|
||||
|
||||
# Use OPD method instead of standard
|
||||
rms = extract_zernike_opd_filtered_rms(
|
||||
op2_file,
|
||||
subcase='20',
|
||||
focal_length=5000.0 # Optional: specify or let it auto-estimate
|
||||
)
|
||||
return rms
|
||||
```
|
||||
|
||||
**In optimization config** (future enhancement):
|
||||
|
||||
```json
|
||||
{
|
||||
"objectives": [
|
||||
{
|
||||
"name": "filtered_rms",
|
||||
"extractor": "zernike_opd",
|
||||
"extractor_config": {
|
||||
"subcase": "20",
|
||||
"metric": "filtered_rms_nm",
|
||||
"focal_length": 5000.0
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### 5.3 Visualization with Insights
|
||||
|
||||
Generate the comparison insight for a study:
|
||||
|
||||
```bash
|
||||
python -m optimization_engine.insights generate studies/my_study --type zernike_opd_comparison
|
||||
```
|
||||
|
||||
This creates an HTML visualization showing:
|
||||
1. **Lateral displacement map** - Where pinching/lateral deformation occurs
|
||||
2. **WFE surface** - Using the rigorous OPD method
|
||||
3. **Comparison table** - Quantitative difference between methods
|
||||
4. **Recommendation** - Whether OPD method is needed for your study
|
||||
|
||||
### 5.4 API Reference
|
||||
|
||||
```python
|
||||
from optimization_engine.extractors import (
|
||||
# Main extractor class
|
||||
ZernikeOPDExtractor,
|
||||
|
||||
# Convenience functions
|
||||
extract_zernike_opd, # Full metrics dict
|
||||
extract_zernike_opd_filtered_rms, # Just the filtered RMS (float)
|
||||
compare_zernike_methods, # Compare standard vs OPD
|
||||
)
|
||||
|
||||
# Full extraction with all metrics
|
||||
result = extract_zernike_opd(op2_file, subcase='20')
|
||||
# Returns: {
|
||||
# 'filtered_rms_nm': float,
|
||||
# 'global_rms_nm': float,
|
||||
# 'max_lateral_disp_um': float,
|
||||
# 'rms_lateral_disp_um': float,
|
||||
# 'focal_length_used': float,
|
||||
# 'astigmatism_rms_nm': float,
|
||||
# 'coma_rms_nm': float,
|
||||
# ...
|
||||
# }
|
||||
|
||||
# Just the primary metric for optimization
|
||||
rms = extract_zernike_opd_filtered_rms(op2_file, subcase='20')
|
||||
|
||||
# Compare both methods
|
||||
comparison = compare_zernike_methods(op2_file, subcase='20')
|
||||
# Returns: {
|
||||
# 'standard_method': {'filtered_rms_nm': ...},
|
||||
# 'opd_method': {'filtered_rms_nm': ...},
|
||||
# 'delta': {'filtered_rms_nm': ..., 'percent_difference_filtered': ...},
|
||||
# 'lateral_displacement': {'max_um': ..., 'rms_um': ...},
|
||||
# 'recommendation': str
|
||||
# }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Validation and Testing
|
||||
|
||||
### 6.1 Analytical Test Case
|
||||
|
||||
For a simple test: apply a known lateral displacement and verify the correction.
|
||||
|
||||
**Setup**:
|
||||
- Parabola: f = 5000 mm
|
||||
- Node at (x₀, y₀) = (400, 0) mm, so r₀ = 400 mm
|
||||
- Apply uniform X-displacement: Δx = 0.01 mm, Δy = 0, Δz = 0
|
||||
|
||||
**Expected correction**:
|
||||
```
|
||||
Δr² = (400.01)² + 0² - 400² - 0² = 8.0001 mm²
|
||||
Δz_parabola = -8.0001 / (4 × 5000) = -0.0004 mm = -400 nm (surface)
|
||||
WFE_correction = 2 × 400 nm = 800 nm
|
||||
```
|
||||
|
||||
**Standard method**: WFE = 2 × Δz × 1e6 = 0 nm
|
||||
**OPD method**: WFE = 2 × (0 - (-0.0004)) × 1e6 = 800 nm
|
||||
|
||||
The OPD method correctly identifies that a purely lateral displacement **does** affect the wavefront!
|
||||
|
||||
### 6.2 Sanity Checks
|
||||
|
||||
The OPD method should:
|
||||
1. Give **identical** results to standard method when Δx = Δy = 0 everywhere
|
||||
2. Show **larger** WFE when nodes move outward laterally (positive Δr²)
|
||||
3. Show **smaller** WFE when nodes move inward laterally (negative Δr²)
|
||||
4. Scale with 1/f (larger effect for faster mirrors)
|
||||
|
||||
### 6.3 Running the Test
|
||||
|
||||
```bash
|
||||
conda activate atomizer
|
||||
python test_zernike_opd_comparison.py
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. Migration Guide
|
||||
|
||||
### 7.1 For Existing Studies
|
||||
|
||||
1. **Run comparison test** on a few representative iterations
|
||||
2. **Check the difference** - if > 5%, consider re-optimizing
|
||||
3. **For lateral support studies** - strongly recommend re-optimization with OPD method
|
||||
|
||||
### 7.2 For New Studies
|
||||
|
||||
1. **Use OPD method by default** - it's never worse than standard
|
||||
2. **Specify focal length** if known (more accurate than auto-estimate)
|
||||
3. **Monitor lateral displacement** in the insight reports
|
||||
|
||||
### 7.3 Code Changes
|
||||
|
||||
**Before** (standard method):
|
||||
```python
|
||||
from optimization_engine.extractors import extract_zernike_filtered_rms
|
||||
|
||||
rms = extract_zernike_filtered_rms(op2_file, subcase='20')
|
||||
```
|
||||
|
||||
**After** (OPD method):
|
||||
```python
|
||||
from optimization_engine.extractors import extract_zernike_opd_filtered_rms
|
||||
|
||||
rms = extract_zernike_opd_filtered_rms(op2_file, subcase='20', focal_length=5000.0)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Appendix A: Derivation Details
|
||||
|
||||
### A.1 Full Derivation of Δz_parabola
|
||||
|
||||
For a concave paraboloid: `z = -r²/(4f) = -(x² + y²)/(4f)`
|
||||
|
||||
Original position: `z₀ = -(x₀² + y₀²)/(4f)`
|
||||
Deformed position: `z_expected = -((x₀+Δx)² + (y₀+Δy)²)/(4f)`
|
||||
|
||||
```
|
||||
z_expected - z₀ = -[(x₀+Δx)² + (y₀+Δy)² - x₀² - y₀²] / (4f)
|
||||
= -[x₀² + 2x₀Δx + Δx² + y₀² + 2y₀Δy + Δy² - x₀² - y₀²] / (4f)
|
||||
= -[2x₀Δx + Δx² + 2y₀Δy + Δy²] / (4f)
|
||||
= -[r_def² - r₀²] / (4f)
|
||||
= -Δr² / (4f)
|
||||
```
|
||||
|
||||
This is `Δz_parabola` - the Z change required to stay on the ideal parabola.
|
||||
|
||||
### A.2 Sign Convention
|
||||
|
||||
For a **concave** mirror (typical telescope primary):
|
||||
- Surface curves toward -Z (vertex is the highest point)
|
||||
- `z = -r²/(4f)` (negative coefficient)
|
||||
- Moving outward (Δr² > 0) requires moving in -Z direction
|
||||
- `Δz_parabola = -Δr²/(4f)` is negative for outward movement
|
||||
|
||||
For a **convex** mirror:
|
||||
- Surface curves toward +Z
|
||||
- `z = +r²/(4f)` (positive coefficient)
|
||||
- `Δz_parabola = +Δr²/(4f)` is positive for outward movement
|
||||
|
||||
The `concave` parameter in the code handles this sign flip.
|
||||
|
||||
---
|
||||
|
||||
## Appendix B: Files Reference
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `optimization_engine/extractors/extract_zernike_opd.py` | Main OPD extractor implementation |
|
||||
| `optimization_engine/extractors/extract_zernike.py` | Standard (Z-only) extractor |
|
||||
| `optimization_engine/insights/zernike_opd_comparison.py` | Visualization insight |
|
||||
| `test_zernike_opd_comparison.py` | Quick test script |
|
||||
|
||||
### Related Documentation
|
||||
|
||||
| Document | Purpose |
|
||||
|----------|---------|
|
||||
| [ZERNIKE_FUNDAMENTALS.md](ZERNIKE_FUNDAMENTALS.md) | General Zernike usage, RMS calculation, multi-subcase analysis |
|
||||
| [00_INDEX.md](00_INDEX.md) | Physics documentation index |
|
||||
| `.claude/skills/modules/extractors-catalog.md` | Quick extractor lookup |
|
||||
| `.claude/skills/modules/insights-catalog.md` | Quick insight lookup |
|
||||
| `docs/protocols/system/SYS_12_EXTRACTOR_LIBRARY.md` | Extractor specifications (E8-E10, E20-E21) |
|
||||
| `docs/protocols/system/SYS_16_STUDY_INSIGHTS.md` | Insight specifications |
|
||||
|
||||
---
|
||||
|
||||
## Appendix C: Glossary
|
||||
|
||||
| Term | Definition |
|
||||
|------|------------|
|
||||
| **OPD** | Optical Path Difference - the path length difference experienced by light rays |
|
||||
| **WFE** | Wavefront Error - deviation of actual wavefront from ideal (WFE = 2 × surface error for reflection) |
|
||||
| **Zernike polynomials** | Orthogonal basis functions for representing wavefronts over a circular aperture |
|
||||
| **Noll index** | Standard optical indexing scheme for Zernike modes (j=1 is piston, j=4 is defocus, etc.) |
|
||||
| **Filtered RMS** | RMS after removing low-order modes (piston, tip, tilt, defocus) that can be corrected by alignment |
|
||||
| **Lateral displacement** | In-plane (X, Y) movement of nodes, as opposed to axial (Z) movement |
|
||||
| **Focal length** | Distance from vertex to focus for a parabola; f = R/(2) where R is vertex radius of curvature |
|
||||
|
||||
---
|
||||
|
||||
*Document maintained by Atomizer Framework. Last updated: 2024-12-22*
|
||||
212
docs/guides/CMA-ES_EXPLAINED.md
Normal file
212
docs/guides/CMA-ES_EXPLAINED.md
Normal file
@@ -0,0 +1,212 @@
|
||||
# CMA-ES Explained for Engineers
|
||||
|
||||
**CMA-ES** = **Covariance Matrix Adaptation Evolution Strategy**
|
||||
|
||||
A derivative-free optimization algorithm ideal for:
|
||||
- Local refinement around known good solutions
|
||||
- 4-10 dimensional problems
|
||||
- Smooth, continuous objective functions
|
||||
- Problems where gradient information is unavailable (like FEA)
|
||||
|
||||
---
|
||||
|
||||
## The Core Idea
|
||||
|
||||
Imagine searching for the lowest point in a hilly landscape while blindfolded:
|
||||
|
||||
1. **Throw darts** around your current best guess
|
||||
2. **Observe which darts land lower** (better objective)
|
||||
3. **Learn the shape of the valley** from those results
|
||||
4. **Adjust future throws** to follow the valley's direction
|
||||
|
||||
---
|
||||
|
||||
## Key Components
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ CMA-ES Components │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ 1. MEAN (μ) - Current best guess location │
|
||||
│ • Moves toward better solutions each generation │
|
||||
│ │
|
||||
│ 2. STEP SIZE (σ) - How far to throw darts │
|
||||
│ • Adapts: shrinks when close, grows when exploring │
|
||||
│ • sigma0=0.3 means 30% of parameter range initially │
|
||||
│ │
|
||||
│ 3. COVARIANCE MATRIX (C) - Shape of the search cloud │
|
||||
│ • Learns parameter correlations │
|
||||
│ • Stretches search along promising directions │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Visual: How the Search Evolves
|
||||
|
||||
```
|
||||
Generation 1 (Round search): Generation 10 (Learned shape):
|
||||
|
||||
x x x
|
||||
x x x x
|
||||
x ● x ──────► x ● x
|
||||
x x x x
|
||||
x x x
|
||||
|
||||
● = mean (center) Ellipse aligned with
|
||||
x = samples the valley direction
|
||||
```
|
||||
|
||||
CMA-ES learns that certain parameter combinations work well together and stretches its search cloud in that direction.
|
||||
|
||||
---
|
||||
|
||||
## The Algorithm (Simplified)
|
||||
|
||||
```python
|
||||
def cma_es_generation():
|
||||
# 1. SAMPLE: Generate λ candidates around the mean
|
||||
for i in range(population_size):
|
||||
candidates[i] = mean + sigma * sample_from_gaussian(covariance=C)
|
||||
|
||||
# 2. EVALUATE: Run FEA for each candidate
|
||||
for candidate in candidates:
|
||||
fitness[candidate] = run_simulation(candidate)
|
||||
|
||||
# 3. SELECT: Keep the best μ candidates
|
||||
selected = top_k(candidates, by=fitness, k=mu)
|
||||
|
||||
# 4. UPDATE MEAN: Move toward the best solutions
|
||||
new_mean = weighted_average(selected)
|
||||
|
||||
# 5. UPDATE COVARIANCE: Learn parameter correlations
|
||||
C = update_covariance(C, selected, mean, new_mean)
|
||||
|
||||
# 6. UPDATE STEP SIZE: Adapt exploration range
|
||||
sigma = adapt_step_size(sigma, evolution_path)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## The Covariance Matrix Magic
|
||||
|
||||
Consider 4 design variables:
|
||||
|
||||
```
|
||||
Covariance Matrix C (4x4):
|
||||
var1 var2 var3 var4
|
||||
var1 [ 1.0 0.3 -0.5 0.1 ]
|
||||
var2 [ 0.3 1.0 0.2 -0.2 ]
|
||||
var3 [-0.5 0.2 1.0 0.4 ]
|
||||
var4 [ 0.1 -0.2 0.4 1.0 ]
|
||||
```
|
||||
|
||||
**Reading the matrix:**
|
||||
- **Diagonal (1.0)**: Variance in each parameter
|
||||
- **Off-diagonal**: Correlations between parameters
|
||||
- **Positive (0.3)**: When var1 increases, var2 should increase
|
||||
- **Negative (-0.5)**: When var1 increases, var3 should decrease
|
||||
|
||||
CMA-ES **learns these correlations automatically** from simulation results!
|
||||
|
||||
---
|
||||
|
||||
## CMA-ES vs TPE
|
||||
|
||||
| Property | TPE | CMA-ES |
|
||||
|----------|-----|--------|
|
||||
| **Best for** | Global exploration | Local refinement |
|
||||
| **Starting point** | Random | Known baseline |
|
||||
| **Correlation learning** | None (independent) | Automatic |
|
||||
| **Step size** | Fixed ranges | Adaptive |
|
||||
| **Dimensionality** | Good for high-D | Best for 4-10D |
|
||||
| **Sample efficiency** | Good | Excellent (locally) |
|
||||
|
||||
---
|
||||
|
||||
## Optuna Configuration
|
||||
|
||||
```python
|
||||
from optuna.samplers import CmaEsSampler
|
||||
|
||||
# Baseline values (starting point)
|
||||
x0 = {
|
||||
'whiffle_min': 62.75,
|
||||
'whiffle_outer_to_vertical': 75.89,
|
||||
'whiffle_triangle_closeness': 65.65,
|
||||
'blank_backface_angle': 4.43
|
||||
}
|
||||
|
||||
sampler = CmaEsSampler(
|
||||
x0=x0, # Center of initial distribution
|
||||
sigma0=0.3, # Initial step size (30% of range)
|
||||
seed=42, # Reproducibility
|
||||
restart_strategy='ipop' # Increase population on restart
|
||||
)
|
||||
|
||||
study = optuna.create_study(sampler=sampler, direction="minimize")
|
||||
|
||||
# CRITICAL: Enqueue baseline as trial 0!
|
||||
# x0 only sets the CENTER, it doesn't evaluate the baseline
|
||||
study.enqueue_trial(x0)
|
||||
|
||||
study.optimize(objective, n_trials=200)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
### 1. Not Evaluating the Baseline
|
||||
|
||||
**Problem**: CMA-ES samples AROUND x0, but doesn't evaluate x0 itself.
|
||||
|
||||
**Solution**: Always enqueue the baseline:
|
||||
```python
|
||||
if len(study.trials) == 0:
|
||||
study.enqueue_trial(x0)
|
||||
```
|
||||
|
||||
### 2. sigma0 Too Large or Too Small
|
||||
|
||||
| sigma0 | Effect |
|
||||
|--------|--------|
|
||||
| **Too large (>0.5)** | Explores too far, misses local optimum |
|
||||
| **Too small (<0.1)** | Gets stuck, slow convergence |
|
||||
| **Recommended (0.2-0.3)** | Good balance for refinement |
|
||||
|
||||
### 3. Wrong Problem Type
|
||||
|
||||
CMA-ES struggles with:
|
||||
- Discrete/categorical variables
|
||||
- Very high dimensions (>20)
|
||||
- Multi-modal landscapes (use TPE first)
|
||||
- Noisy objectives (add regularization)
|
||||
|
||||
---
|
||||
|
||||
## When to Use CMA-ES in Atomizer
|
||||
|
||||
| Scenario | Use CMA-ES? |
|
||||
|----------|-------------|
|
||||
| First exploration of design space | No, use TPE |
|
||||
| Refining around known good design | **Yes** |
|
||||
| 4-10 continuous variables | **Yes** |
|
||||
| >15 variables | No, use TPE or NSGA-II |
|
||||
| Need to learn variable correlations | **Yes** |
|
||||
| Multi-objective optimization | No, use NSGA-II |
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- Hansen, N. (2016). The CMA Evolution Strategy: A Tutorial
|
||||
- Optuna CmaEsSampler: https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.CmaEsSampler.html
|
||||
- cmaes Python package: https://github.com/CyberAgentAILab/cmaes
|
||||
|
||||
---
|
||||
|
||||
*Created: 2025-12-19*
|
||||
*Atomizer Framework*
|
||||
@@ -71,6 +71,71 @@ When creating a new study:
|
||||
|
||||
---
|
||||
|
||||
## README Hierarchy (Parent-Child Documentation)
|
||||
|
||||
**Two-level documentation system**:
|
||||
|
||||
```
|
||||
studies/{geometry_type}/
|
||||
├── README.md # PARENT: Project-level context
|
||||
│ ├── Project overview # What is this geometry/component?
|
||||
│ ├── Physical system specs # Material, dimensions, constraints
|
||||
│ ├── Optical/mechanical specs # Domain-specific requirements
|
||||
│ ├── Design variables catalog # ALL possible variables with descriptions
|
||||
│ ├── Objectives catalog # ALL possible objectives
|
||||
│ ├── Campaign history # Summary of all sub-studies
|
||||
│ └── Sub-studies index # Links to each sub-study
|
||||
│
|
||||
├── sub_study_V1/
|
||||
│ └── README.md # CHILD: Study-specific details
|
||||
│ ├── Link to parent # "See ../README.md for context"
|
||||
│ ├── Study focus # What THIS study optimizes
|
||||
│ ├── Active variables # Which params enabled
|
||||
│ ├── Algorithm config # Sampler, trials, settings
|
||||
│ ├── Baseline/seeding # Starting point
|
||||
│ └── Results summary # Best trial, learnings
|
||||
│
|
||||
└── sub_study_V2/
|
||||
└── README.md # CHILD: References parent, adds specifics
|
||||
```
|
||||
|
||||
### Parent README Content (Geometry-Level)
|
||||
|
||||
| Section | Content |
|
||||
|---------|---------|
|
||||
| Project Overview | What the component is, purpose, context |
|
||||
| Physical System | Material, mass targets, loading conditions |
|
||||
| Domain Specs | Optical prescription (mirrors), structural limits (brackets) |
|
||||
| Design Variables | Complete catalog with ranges and descriptions |
|
||||
| Objectives | All possible metrics with formulas |
|
||||
| Campaign History | Evolution across sub-studies |
|
||||
| Sub-Studies Index | Table with links, status, best results |
|
||||
| Technical Notes | Domain-specific implementation details |
|
||||
|
||||
### Child README Content (Study-Level)
|
||||
|
||||
| Section | Content |
|
||||
|---------|---------|
|
||||
| Parent Reference | `> See [../README.md](../README.md) for project context` |
|
||||
| Study Focus | What differentiates THIS study |
|
||||
| Active Variables | Which parameters are enabled (subset of parent catalog) |
|
||||
| Algorithm Config | Sampler, n_trials, sigma, seed |
|
||||
| Baseline | Starting point (seeded from prior study or default) |
|
||||
| Results | Best trial, improvement metrics |
|
||||
| Key Learnings | What was discovered |
|
||||
|
||||
### When to Create Parent README
|
||||
|
||||
- **First study** for a geometry type → Create parent README immediately
|
||||
- **Subsequent studies** → Add to parent's sub-studies index
|
||||
- **New geometry type** → Create both parent and child READMEs
|
||||
|
||||
### Example Reference
|
||||
|
||||
See `studies/M1_Mirror/README.md` for a complete parent README example.
|
||||
|
||||
---
|
||||
|
||||
## Detailed Steps
|
||||
|
||||
### Step 1: Gather Requirements
|
||||
|
||||
5
knowledge_base/lac/session_insights/failure.jsonl
Normal file
5
knowledge_base/lac/session_insights/failure.jsonl
Normal file
@@ -0,0 +1,5 @@
|
||||
{"timestamp":"2025-12-17T20:30:00","category":"failure","context":"Killed NX process (ugraf.exe PID 111040) without permission while trying to extract expressions","insight":"CRITICAL RULE VIOLATION: Never kill NX (ugraf.exe) or any user process directly. The NXSessionManager exists specifically to track which NX sessions Atomizer started vs user sessions. Only use manager.close_nx_if_allowed() which checks can_close_nx() before terminating. Direct Stop-Process or taskkill on ugraf.exe is FORBIDDEN unless the session manager confirms we started that PID.","confidence":1.0,"tags":["nx","process-management","safety","critical","session-manager"],"severity":"critical","rule":"NEVER use Stop-Process, taskkill, or any direct process termination on ugraf.exe. Always use NXSessionManager.close_nx_if_allowed() which only closes sessions we started."}
|
||||
{"timestamp":"2025-12-17T20:40:00","category":"failure","context":"Created m1_mirror_cost_reduction_V2 study without README.md despite OP_01 protocol clearly requiring it","insight":"EXECUTION FAILURE: The protocol OP_01_CREATE_STUDY.md already listed README.md as a required output, but I failed to follow my own documentation. This is a process discipline issue, not a knowledge gap. The fix is NOT to add more documentation (it was already there), but to use TodoWrite to track ALL required outputs during study creation and verify completion before declaring done. When creating a study, the todo list MUST include: (1) optimization_config.json, (2) run_optimization.py, (3) README.md, (4) STUDY_REPORT.md - and mark study creation complete ONLY after all 4 are done.","confidence":1.0,"tags":["study-creation","documentation","readme","process-discipline","todowrite"],"severity":"high","rule":"When creating a study, add ALL required files to TodoWrite checklist and verify each is created before marking task complete. The protocol exists - FOLLOW IT."}
|
||||
{"timestamp":"2025-12-19T10:00:00","category":"workaround","context":"NX journal execution via cmd /c with environment variables fails silently or produces garbled output. Multiple attempts with cmd /c SET and && chaining failed to capture run_journal.exe output.","insight":"CRITICAL WORKAROUND: When executing NX journals from Claude Code on Windows, use PowerShell with [Environment]::SetEnvironmentVariable() method instead of cmd /c or $env: syntax. The correct pattern is: powershell -Command \"[Environment]::SetEnvironmentVariable('SPLM_LICENSE_SERVER', '28000@dalidou;28000@100.80.199.40', 'Process'); & 'C:\\Program Files\\Siemens\\DesigncenterNX2512\\NXBIN\\run_journal.exe' 'journal.py' -args 'arg1' 'arg2' 2>&1\". The $env: syntax gets corrupted when passed through bash (colon gets interpreted). The cmd /c SET syntax often fails to capture output. This PowerShell pattern reliably sets license server and captures all output.","confidence":1.0,"tags":["nx","powershell","run_journal","license-server","windows","cmd-workaround"],"severity":"high","rule":"ALWAYS use PowerShell with [Environment]::SetEnvironmentVariable() for NX journal execution. NEVER use cmd /c SET or $env: syntax for setting SPLM_LICENSE_SERVER."}
|
||||
{"timestamp":"2025-12-19T15:30:00","category":"failure","context":"CMA-ES optimization V7 started with random sample instead of baseline. First trial had whiffle_min=45.73 instead of baseline 62.75, resulting in WS=329 instead of expected ~281.","insight":"CMA-ES with Optuna CmaEsSampler does NOT evaluate x0 (baseline) first - it samples AROUND x0 with sigma0 step size. The x0 parameter only sets the CENTER of the initial sampling distribution, not the first trial. To ensure baseline is evaluated first, use study.enqueue_trial(x0) after creating the study. This is critical for refinement studies where you need to compare against a known-good baseline. Pattern: if len(study.trials) == 0: study.enqueue_trial(x0)","confidence":1.0,"tags":["cma-es","optuna","baseline","x0","enqueue","optimization"],"severity":"high","rule":"When using CmaEsSampler with a known baseline, ALWAYS enqueue the baseline as trial 0 using study.enqueue_trial(x0). The x0 parameter alone does NOT guarantee baseline evaluation."}
|
||||
{"timestamp":"2025-12-22T14:00:00","category":"failure","context":"V10 mirror optimization reported impossibly good relative WFE values (40-20=1.99nm instead of ~6nm, 60-20=6.82nm instead of ~13nm). User noticed results were 'too good to be true'.","insight":"CRITICAL BUG IN RELATIVE WFE CALCULATION: The V10 run_optimization.py computed relative WFE as abs(RMS_target - RMS_ref) instead of RMS(WFE_target - WFE_ref). This is mathematically WRONG because |RMS(A) - RMS(B)| ≠ RMS(A - B). The correct approach is to compute the node-by-node WFE difference FIRST, then fit Zernike to the difference field, then compute RMS. The bug gave values 3-4x lower than correct values because the 20° reference had HIGHER absolute WFE than 40°/60°, so the subtraction gave negative values, and abs() hid the problem. The fix is to use extractor.extract_relative() which correctly computes node-by-node differences. Both ZernikeExtractor and ZernikeOPDExtractor now have extract_relative() methods.","confidence":1.0,"tags":["zernike","wfe","relative-wfe","extract_relative","critical-bug","v10"],"severity":"critical","rule":"NEVER compute relative WFE as abs(RMS_target - RMS_ref). ALWAYS use extract_relative() which computes RMS(WFE_target - WFE_ref) by doing node-by-node subtraction first, then Zernike fitting, then RMS."}
|
||||
@@ -0,0 +1,3 @@
|
||||
{"timestamp":"2025-12-22T11:05:00","category":"success_pattern","context":"Organized M1 Mirror documentation with parent-child README hierarchy","insight":"DOCUMENTATION PATTERN: Studies use a two-level README hierarchy. Parent README at studies/{geometry_type}/README.md contains project-wide context (optical specs, design variables catalog, objectives catalog, campaign history, sub-studies index). Child README at studies/{geometry_type}/{study_name}/README.md references parent and contains study-specific details (active variables, algorithm config, results). This eliminates duplication, maintains single source of truth for specs, and makes sub-study docs concise. Pattern documented in OP_01_CREATE_STUDY.md and study-creation-core.md.","confidence":0.95,"tags":["documentation","readme","hierarchy","study-creation","organization"],"rule":"When creating studies for a geometry type: (1) Create parent README with project context if first study, (2) Add reference banner to child README: '> See [../README.md](../README.md) for project overview', (3) Update parent's sub-studies index table when adding new sub-studies."}
|
||||
{"timestamp":"2025-12-22T11:05:00","category":"success_pattern","context":"Created universal mirror optical specs extraction tool","insight":"TOOL PATTERN: Mirror optical specs (focal length, f-number, diameter) can be auto-estimated from FEA mesh geometry by fitting z = a*r² + b to node coordinates. Focal length = 1/(4*|a|). Tool at tools/extract_mirror_optical_specs.py works with any mirror study - just point it at an OP2 file or study directory. Reports fit quality to indicate if explicit focal length should be used instead. Use: python tools/extract_mirror_optical_specs.py path/to/study","confidence":0.9,"tags":["tools","mirror","optical-specs","zernike","opd","extraction"],"rule":"For mirror optimization: (1) Run extract_mirror_optical_specs.py to estimate optical prescription from mesh, (2) Validate against design specs, (3) Document in parent README, (4) Use explicit focal_length in ZernikeOPDExtractor if fit quality is poor."}
|
||||
{"timestamp":"2025-12-22T11:05:00","category":"success_pattern","context":"Implemented OPD-based Zernike method for lateral support optimization","insight":"PHYSICS PATTERN: Standard Zernike WFE analysis uses Z-displacement at original (x,y) coordinates. This is INCORRECT for lateral support optimization where nodes shift in X,Y. The rigorous OPD method computes: surface_error = dz - delta_z_parabola where delta_z_parabola = -delta_r²/(4f) for concave mirrors. This accounts for the fact that laterally displaced nodes should be compared against parabola height at their NEW position. Implemented in extract_zernike_opd.py with ZernikeOPDExtractor class. Use extract_comparison() to see method difference. Threshold: >10µm lateral displacement = CRITICAL to use OPD.","confidence":1.0,"tags":["zernike","opd","lateral-support","mirror","wfe","physics"],"rule":"For mirror optimization with lateral supports or any case where X,Y displacement may be significant: (1) Use ZernikeOPDExtractor instead of ZernikeExtractor, (2) Run zernike_opd_comparison insight to check lateral displacement magnitude, (3) If max lateral >10µm, OPD method is CRITICAL."}
|
||||
@@ -53,7 +53,7 @@ def main():
|
||||
# Start backend - use conda run to ensure atomizer environment
|
||||
print(f"{Colors.YELLOW}Starting backend server (FastAPI on port 8000)...{Colors.END}")
|
||||
backend_proc = subprocess.Popen(
|
||||
["conda", "run", "-n", "atomizer", "python", "-m", "uvicorn", "api.main:app", "--port", "8000"],
|
||||
["conda", "run", "-n", "atomizer", "python", "-m", "uvicorn", "api.main:app", "--port", "8000", "--reload"],
|
||||
cwd=str(backend_dir),
|
||||
shell=True,
|
||||
creationflags=subprocess.CREATE_NEW_PROCESS_GROUP if sys.platform == "win32" else 0
|
||||
|
||||
233
nx_journals/analyze_wfe_zernike.py
Normal file
233
nx_journals/analyze_wfe_zernike.py
Normal file
@@ -0,0 +1,233 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Atomizer Zernike WFE Analyzer
|
||||
=============================
|
||||
|
||||
Analyze Zernike wavefront error from NX Nastran OP2 results.
|
||||
|
||||
IMPORTANT: This script requires numpy/scipy. Run from command line with
|
||||
the atomizer conda environment, NOT from within NX.
|
||||
|
||||
Usage:
|
||||
conda activate atomizer
|
||||
python analyze_wfe_zernike.py "path/to/solution.op2"
|
||||
|
||||
# Or without argument - searches current directory for OP2 files:
|
||||
python analyze_wfe_zernike.py
|
||||
|
||||
Output:
|
||||
- Zernike coefficients for each subcase
|
||||
- Relative WFE metrics (filtered RMS)
|
||||
- Manufacturing workload (J1-J3 filtered)
|
||||
- Weighted sum calculation
|
||||
|
||||
Author: Atomizer
|
||||
Created: 2025-12-18
|
||||
"""
|
||||
|
||||
import sys
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def log(msg):
|
||||
"""Print to console."""
|
||||
print(msg)
|
||||
|
||||
|
||||
def find_op2_file(working_dir=None):
|
||||
"""Find the most recent OP2 file in the working directory."""
|
||||
if working_dir is None:
|
||||
working_dir = Path.cwd()
|
||||
else:
|
||||
working_dir = Path(working_dir)
|
||||
|
||||
# Look for OP2 files
|
||||
op2_files = list(working_dir.glob("*solution*.op2")) + list(working_dir.glob("*.op2"))
|
||||
|
||||
if not op2_files:
|
||||
# Check subdirectories
|
||||
op2_files = list(working_dir.glob("**/*solution*.op2"))
|
||||
|
||||
if not op2_files:
|
||||
return None
|
||||
|
||||
# Return most recently modified
|
||||
return max(op2_files, key=lambda p: p.stat().st_mtime)
|
||||
|
||||
def analyze_zernike(op2_path):
|
||||
"""Run Zernike analysis on OP2 file."""
|
||||
|
||||
# Add Atomizer to path
|
||||
atomizer_root = Path(__file__).parent.parent
|
||||
if str(atomizer_root) not in sys.path:
|
||||
sys.path.insert(0, str(atomizer_root))
|
||||
|
||||
try:
|
||||
from optimization_engine.extractors import ZernikeExtractor
|
||||
except ImportError as e:
|
||||
log(f"ERROR: Could not import ZernikeExtractor: {e}")
|
||||
log(f"Make sure Atomizer is properly installed.")
|
||||
log(f"Atomizer root: {atomizer_root}")
|
||||
return None
|
||||
|
||||
log("=" * 70)
|
||||
log("ZERNIKE WAVEFRONT ERROR ANALYSIS")
|
||||
log("=" * 70)
|
||||
log(f"OP2 File: {op2_path.name}")
|
||||
log(f"Directory: {op2_path.parent}")
|
||||
log("")
|
||||
|
||||
# Create extractor
|
||||
try:
|
||||
extractor = ZernikeExtractor(
|
||||
op2_path,
|
||||
bdf_path=None,
|
||||
displacement_unit='mm',
|
||||
n_modes=50,
|
||||
filter_orders=4
|
||||
)
|
||||
except Exception as e:
|
||||
log(f"ERROR creating extractor: {e}")
|
||||
return None
|
||||
|
||||
# Get available subcases from the extractor's displacement data
|
||||
subcases = list(extractor.displacements.keys())
|
||||
log(f"Available subcases: {subcases}")
|
||||
log("")
|
||||
|
||||
# Standard subcase mapping for M1 mirror
|
||||
subcase_labels = {
|
||||
'1': '90 deg (Manufacturing/Polishing)',
|
||||
'2': '20 deg (Reference)',
|
||||
'3': '40 deg (Operational)',
|
||||
'4': '60 deg (Operational)'
|
||||
}
|
||||
|
||||
# Extract absolute Zernike for each subcase
|
||||
log("-" * 70)
|
||||
log("ABSOLUTE ZERNIKE ANALYSIS (per subcase)")
|
||||
log("-" * 70)
|
||||
|
||||
results = {}
|
||||
for sc in subcases:
|
||||
try:
|
||||
result = extractor.extract_subcase(sc)
|
||||
results[sc] = result
|
||||
label = subcase_labels.get(sc, f'Subcase {sc}')
|
||||
log(f"\n{label}:")
|
||||
log(f" Global RMS: {result['global_rms_nm']:.2f} nm")
|
||||
log(f" Filtered RMS: {result['filtered_rms_nm']:.2f} nm (J4+ only)")
|
||||
except Exception as e:
|
||||
log(f" ERROR extracting subcase {sc}: {e}")
|
||||
|
||||
# Relative analysis (using subcase 2 as reference)
|
||||
ref_subcase = '2'
|
||||
if ref_subcase in subcases:
|
||||
log("")
|
||||
log("-" * 70)
|
||||
log(f"RELATIVE ANALYSIS (vs {subcase_labels.get(ref_subcase, ref_subcase)})")
|
||||
log("-" * 70)
|
||||
|
||||
relative_results = {}
|
||||
for sc in subcases:
|
||||
if sc == ref_subcase:
|
||||
continue
|
||||
try:
|
||||
rel = extractor.extract_relative(sc, ref_subcase)
|
||||
relative_results[sc] = rel
|
||||
label = subcase_labels.get(sc, f'Subcase {sc}')
|
||||
log(f"\n{label} vs Reference:")
|
||||
log(f" Relative Filtered RMS: {rel['relative_filtered_rms_nm']:.2f} nm")
|
||||
if 'relative_rms_filter_j1to3' in rel:
|
||||
log(f" J1-J3 Filtered RMS: {rel['relative_rms_filter_j1to3']:.2f} nm")
|
||||
except Exception as e:
|
||||
log(f" ERROR: {e}")
|
||||
|
||||
# Calculate weighted sum (M1 mirror optimization objectives)
|
||||
log("")
|
||||
log("-" * 70)
|
||||
log("OPTIMIZATION OBJECTIVES")
|
||||
log("-" * 70)
|
||||
|
||||
obj_40_20 = relative_results.get('3', {}).get('relative_filtered_rms_nm', 0)
|
||||
obj_60_20 = relative_results.get('4', {}).get('relative_filtered_rms_nm', 0)
|
||||
obj_mfg = relative_results.get('1', {}).get('relative_rms_filter_j1to3', 0)
|
||||
|
||||
log(f"\n 40-20 Filtered RMS: {obj_40_20:.2f} nm")
|
||||
log(f" 60-20 Filtered RMS: {obj_60_20:.2f} nm")
|
||||
log(f" MFG 90 (J1-J3): {obj_mfg:.2f} nm")
|
||||
|
||||
# Weighted sums for different weight configurations
|
||||
log("")
|
||||
log("Weighted Sum Calculations:")
|
||||
|
||||
# V4 weights: 5*40 + 5*60 + 2*mfg + mass
|
||||
ws_v4 = 5*obj_40_20 + 5*obj_60_20 + 2*obj_mfg
|
||||
log(f" V4 weights (5/5/2): {ws_v4:.2f} (+ mass)")
|
||||
|
||||
# V5 weights: 5*40 + 5*60 + 3*mfg + mass
|
||||
ws_v5 = 5*obj_40_20 + 5*obj_60_20 + 3*obj_mfg
|
||||
log(f" V5 weights (5/5/3): {ws_v5:.2f} (+ mass)")
|
||||
|
||||
return {
|
||||
'absolute': results,
|
||||
'relative': relative_results,
|
||||
'objectives': {
|
||||
'40_20': obj_40_20,
|
||||
'60_20': obj_60_20,
|
||||
'mfg_90': obj_mfg,
|
||||
'ws_v4': ws_v4,
|
||||
'ws_v5': ws_v5
|
||||
}
|
||||
}
|
||||
|
||||
return {'absolute': results}
|
||||
|
||||
def main(args):
|
||||
"""Main entry point."""
|
||||
log("")
|
||||
log("=" * 70)
|
||||
log(" ATOMIZER ZERNIKE WFE ANALYZER")
|
||||
log("=" * 70)
|
||||
log("")
|
||||
|
||||
# Determine OP2 file
|
||||
op2_path = None
|
||||
|
||||
if args and len(args) > 0 and args[0]:
|
||||
# OP2 path provided as argument
|
||||
op2_path = Path(args[0])
|
||||
if not op2_path.exists():
|
||||
log(f"ERROR: OP2 file not found: {op2_path}")
|
||||
return
|
||||
else:
|
||||
# Try to find OP2 in current directory
|
||||
log("No OP2 file specified, searching...")
|
||||
op2_path = find_op2_file()
|
||||
|
||||
if op2_path is None:
|
||||
log("ERROR: No OP2 file found in current directory.")
|
||||
log("Usage: Run after solving, or provide OP2 path as argument.")
|
||||
return
|
||||
|
||||
log(f"Found: {op2_path}")
|
||||
|
||||
# Run analysis
|
||||
results = analyze_zernike(op2_path)
|
||||
|
||||
if results:
|
||||
log("")
|
||||
log("=" * 70)
|
||||
log("ANALYSIS COMPLETE")
|
||||
log("=" * 70)
|
||||
else:
|
||||
log("")
|
||||
log("Analysis failed. Check errors above.")
|
||||
|
||||
if __name__ == '__main__':
|
||||
# Get arguments (works both in NX and command line)
|
||||
if len(sys.argv) > 1:
|
||||
main(sys.argv[1:])
|
||||
else:
|
||||
main([])
|
||||
184
nx_journals/capture_study_images.py
Normal file
184
nx_journals/capture_study_images.py
Normal file
@@ -0,0 +1,184 @@
|
||||
# NX Journal: Capture Study Images for Atomizer Documentation
|
||||
#
|
||||
# Purpose: Capture top view and isometric view images of a part for study documentation
|
||||
# Usage: run_journal.exe capture_study_images.py -args "part_file_path" "output_directory" ["prefix"]
|
||||
#
|
||||
# Arguments:
|
||||
# part_file_path: Full path to the .prt file to capture
|
||||
# output_directory: Directory where images will be saved
|
||||
# prefix (optional): Prefix for image filenames (default: part name)
|
||||
#
|
||||
# Output:
|
||||
# {prefix}_Top.png - Top view image
|
||||
# {prefix}_iso.png - Isometric view image
|
||||
#
|
||||
# Author: Atomizer
|
||||
# Created: 2025-12-18
|
||||
|
||||
import sys
|
||||
import os
|
||||
import math
|
||||
import NXOpen
|
||||
import NXOpen.Gateway
|
||||
|
||||
|
||||
def capture_images(part_path: str, output_dir: str, prefix: str = None):
|
||||
"""
|
||||
Capture top view and isometric view images of a part.
|
||||
|
||||
Args:
|
||||
part_path: Full path to the .prt file
|
||||
output_dir: Directory to save images
|
||||
prefix: Optional prefix for image filenames
|
||||
"""
|
||||
theSession = NXOpen.Session.GetSession()
|
||||
|
||||
# Open the part if not already open
|
||||
try:
|
||||
workPart, loadStatus = theSession.Parts.OpenDisplay(part_path, NXOpen.Part.LoadStatically)
|
||||
loadStatus.Dispose()
|
||||
except:
|
||||
workPart = theSession.Parts.Work
|
||||
|
||||
if workPart is None:
|
||||
print(f"ERROR: Could not open part: {part_path}")
|
||||
return False
|
||||
|
||||
# Determine prefix from part name if not provided
|
||||
if prefix is None:
|
||||
prefix = os.path.splitext(os.path.basename(part_path))[0]
|
||||
|
||||
# Ensure output directory exists
|
||||
if not os.path.exists(output_dir):
|
||||
os.makedirs(output_dir)
|
||||
|
||||
# Hide construction geometry for cleaner images
|
||||
_hide_construction_geometry(theSession, workPart)
|
||||
|
||||
# Capture top view
|
||||
top_image_path = os.path.join(output_dir, f"{prefix}_Top.png")
|
||||
_capture_top_view(theSession, workPart, top_image_path)
|
||||
print(f"Saved: {top_image_path}")
|
||||
|
||||
# Capture isometric view
|
||||
iso_image_path = os.path.join(output_dir, f"{prefix}_iso.png")
|
||||
_capture_isometric_view(theSession, workPart, iso_image_path)
|
||||
print(f"Saved: {iso_image_path}")
|
||||
|
||||
return True
|
||||
|
||||
|
||||
def _hide_construction_geometry(theSession, workPart):
|
||||
"""Hide datums, curves, and sketches for cleaner visualization."""
|
||||
markId = theSession.SetUndoMark(NXOpen.Session.MarkVisibility.Visible, "Hide Construction")
|
||||
|
||||
# Hide datums
|
||||
theSession.DisplayManager.HideByType("SHOW_HIDE_TYPE_DATUMS",
|
||||
NXOpen.DisplayManager.ShowHideScope.AnyInAssembly)
|
||||
|
||||
# Hide curves
|
||||
theSession.DisplayManager.HideByType("SHOW_HIDE_TYPE_CURVES",
|
||||
NXOpen.DisplayManager.ShowHideScope.AnyInAssembly)
|
||||
|
||||
# Hide sketches
|
||||
theSession.DisplayManager.HideByType("SHOW_HIDE_TYPE_SKETCHES",
|
||||
NXOpen.DisplayManager.ShowHideScope.AnyInAssembly)
|
||||
|
||||
theSession.UpdateManager.DoUpdate(markId)
|
||||
workPart.ModelingViews.WorkView.FitAfterShowOrHide(NXOpen.View.ShowOrHideType.HideOnly)
|
||||
theSession.DeleteUndoMark(markId, None)
|
||||
|
||||
|
||||
def _capture_top_view(theSession, workPart, output_path):
|
||||
"""Capture top view (looking down Z-axis)."""
|
||||
# Set top view orientation (looking down -Z)
|
||||
matrix = NXOpen.Matrix3x3()
|
||||
matrix.Xx = 0.0
|
||||
matrix.Xy = -1.0
|
||||
matrix.Xz = 0.0
|
||||
matrix.Yx = -1.0
|
||||
matrix.Yy = 0.0
|
||||
matrix.Yz = 0.0
|
||||
matrix.Zx = 0.0
|
||||
matrix.Zy = 0.0
|
||||
matrix.Zz = -1.0
|
||||
workPart.ModelingViews.WorkView.Orient(matrix)
|
||||
|
||||
# Fit view
|
||||
workPart.ModelingViews.WorkView.Fit()
|
||||
|
||||
# Export image
|
||||
_export_image(workPart, output_path)
|
||||
|
||||
|
||||
def _capture_isometric_view(theSession, workPart, output_path):
|
||||
"""Capture isometric view (standard ISO angle showing backface)."""
|
||||
# Set isometric orientation showing backface structure
|
||||
rotMatrix = NXOpen.Matrix3x3()
|
||||
rotMatrix.Xx = -0.32736574141345925
|
||||
rotMatrix.Xy = -0.94489752125198745
|
||||
rotMatrix.Xz = -0.00058794613984273266
|
||||
rotMatrix.Yx = -0.71924452681462514
|
||||
rotMatrix.Yy = 0.24959027079525001
|
||||
rotMatrix.Yz = -0.64837643955618585
|
||||
rotMatrix.Zx = 0.61279603621108569
|
||||
rotMatrix.Zy = -0.21183335680718612
|
||||
rotMatrix.Zz = -0.76131967460967154
|
||||
|
||||
# Get current scale and set orientation
|
||||
translation = NXOpen.Point3d(0, 0, 0)
|
||||
workPart.ModelingViews.WorkView.SetRotationTranslationScale(rotMatrix, translation, 0.25)
|
||||
|
||||
# Fit view
|
||||
workPart.ModelingViews.WorkView.Fit()
|
||||
|
||||
# Export image
|
||||
_export_image(workPart, output_path)
|
||||
|
||||
|
||||
def _export_image(workPart, output_path, width=1200, height=1000):
|
||||
"""Export current view as PNG image."""
|
||||
imageExportBuilder = workPart.Views.CreateImageExportBuilder()
|
||||
|
||||
try:
|
||||
# Configure export settings
|
||||
imageExportBuilder.RegionMode = False # Use entire view
|
||||
imageExportBuilder.DeviceWidth = width
|
||||
imageExportBuilder.DeviceHeight = height
|
||||
imageExportBuilder.FileFormat = NXOpen.Gateway.ImageExportBuilder.FileFormats.Png
|
||||
imageExportBuilder.FileName = output_path
|
||||
imageExportBuilder.BackgroundOption = NXOpen.Gateway.ImageExportBuilder.BackgroundOptions.Original
|
||||
imageExportBuilder.EnhanceEdges = False
|
||||
|
||||
# Commit export
|
||||
imageExportBuilder.Commit()
|
||||
finally:
|
||||
imageExportBuilder.Destroy()
|
||||
|
||||
|
||||
def main(args):
|
||||
"""Main entry point for journal."""
|
||||
if len(args) < 2:
|
||||
print("Usage: capture_study_images.py -args \"part_path\" \"output_dir\" [\"prefix\"]")
|
||||
print(" part_path: Full path to .prt file")
|
||||
print(" output_dir: Directory for output images")
|
||||
print(" prefix: Optional filename prefix (default: part name)")
|
||||
return
|
||||
|
||||
part_path = args[0]
|
||||
output_dir = args[1]
|
||||
prefix = args[2] if len(args) > 2 else None
|
||||
|
||||
print(f"Capturing images for: {part_path}")
|
||||
print(f"Output directory: {output_dir}")
|
||||
|
||||
success = capture_images(part_path, output_dir, prefix)
|
||||
|
||||
if success:
|
||||
print("Image capture complete!")
|
||||
else:
|
||||
print("Image capture failed!")
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main(sys.argv[1:])
|
||||
111
nx_journals/extract_expressions.py
Normal file
111
nx_journals/extract_expressions.py
Normal file
@@ -0,0 +1,111 @@
|
||||
"""
|
||||
NX Journal Script to Extract All Expressions from a Part
|
||||
|
||||
Usage:
|
||||
run_journal.exe extract_expressions.py <prt_file_path> [output_dir]
|
||||
|
||||
Output:
|
||||
_temp_expressions.json with all expressions from the part
|
||||
"""
|
||||
|
||||
import sys
|
||||
import os
|
||||
import json
|
||||
import NXOpen
|
||||
|
||||
|
||||
def main(args):
|
||||
if len(args) < 1:
|
||||
print("ERROR: No .prt file path provided")
|
||||
return False
|
||||
|
||||
prt_file_path = args[0]
|
||||
output_dir = args[1] if len(args) > 1 else os.path.dirname(prt_file_path)
|
||||
|
||||
print(f"[JOURNAL] Extracting expressions from: {os.path.basename(prt_file_path)}")
|
||||
|
||||
results = {
|
||||
'part_file': os.path.basename(prt_file_path),
|
||||
'part_path': prt_file_path,
|
||||
'expressions': [],
|
||||
'expression_count': 0,
|
||||
'user_expression_count': 0,
|
||||
'success': False,
|
||||
'error': None
|
||||
}
|
||||
|
||||
try:
|
||||
theSession = NXOpen.Session.GetSession()
|
||||
|
||||
# Set load options
|
||||
working_dir = os.path.dirname(prt_file_path)
|
||||
theSession.Parts.LoadOptions.ComponentLoadMethod = NXOpen.LoadOptions.LoadMethod.FromDirectory
|
||||
theSession.Parts.LoadOptions.SetSearchDirectories([working_dir], [True])
|
||||
|
||||
# Open the part file
|
||||
print(f"[JOURNAL] Opening part file...")
|
||||
basePart, partLoadStatus = theSession.Parts.OpenActiveDisplay(
|
||||
prt_file_path,
|
||||
NXOpen.DisplayPartOption.AllowAdditional
|
||||
)
|
||||
partLoadStatus.Dispose()
|
||||
|
||||
workPart = theSession.Parts.Work
|
||||
print(f"[JOURNAL] Loaded part: {workPart.Name}")
|
||||
|
||||
# Extract all expressions
|
||||
print(f"[JOURNAL] Extracting expressions...")
|
||||
for expr in workPart.Expressions:
|
||||
try:
|
||||
expr_data = {
|
||||
'name': expr.Name,
|
||||
'value': expr.Value,
|
||||
'rhs': expr.RightHandSide if hasattr(expr, 'RightHandSide') else None,
|
||||
'units': expr.Units.Name if expr.Units else None,
|
||||
'type': str(expr.Type) if hasattr(expr, 'Type') else 'Unknown',
|
||||
}
|
||||
|
||||
# Check if it's a user expression (not internal p0, p1, etc.)
|
||||
is_internal = expr.Name.startswith('p') and len(expr.Name) > 1 and expr.Name[1:].replace('.', '').replace('_', '').isdigit()
|
||||
expr_data['is_internal'] = is_internal
|
||||
|
||||
results['expressions'].append(expr_data)
|
||||
|
||||
if not is_internal:
|
||||
results['user_expression_count'] += 1
|
||||
|
||||
except Exception as e:
|
||||
print(f"[JOURNAL] Warning: Could not read expression: {e}")
|
||||
|
||||
results['expression_count'] = len(results['expressions'])
|
||||
results['success'] = True
|
||||
|
||||
print(f"[JOURNAL] Found {results['expression_count']} total expressions")
|
||||
print(f"[JOURNAL] Found {results['user_expression_count']} user expressions")
|
||||
|
||||
# Print user expressions
|
||||
print(f"\n[JOURNAL] USER EXPRESSIONS:")
|
||||
print(f"[JOURNAL] " + "=" * 50)
|
||||
for expr in results['expressions']:
|
||||
if not expr['is_internal']:
|
||||
units_str = f" [{expr['units']}]" if expr['units'] else ""
|
||||
print(f"[JOURNAL] {expr['name']}: {expr['value']}{units_str}")
|
||||
|
||||
except Exception as e:
|
||||
results['error'] = str(e)
|
||||
results['success'] = False
|
||||
print(f"[JOURNAL] ERROR: {e}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
|
||||
# Write results
|
||||
output_file = os.path.join(output_dir, "_temp_expressions.json")
|
||||
with open(output_file, 'w') as f:
|
||||
json.dump(results, f, indent=2)
|
||||
print(f"\n[JOURNAL] Results written to: {output_file}")
|
||||
|
||||
return results['success']
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main(sys.argv[1:])
|
||||
96
nx_journals/extract_expressions_standalone.py
Normal file
96
nx_journals/extract_expressions_standalone.py
Normal file
@@ -0,0 +1,96 @@
|
||||
"""
|
||||
Standalone expression extractor - opens part and extracts all expressions
|
||||
Run with: ugraf.exe -run extract_expressions_standalone.py
|
||||
"""
|
||||
import NXOpen
|
||||
import os
|
||||
import json
|
||||
|
||||
def main():
|
||||
session = NXOpen.Session.GetSession()
|
||||
|
||||
part_path = r"C:\Users\antoi\Atomizer\studies\m1_mirror_cost_reduction\1_setup\model\M1_Blank.prt"
|
||||
output_json = r"C:\Users\antoi\Atomizer\_expressions_output.json"
|
||||
output_txt = r"C:\Users\antoi\Atomizer\_expressions_output.txt"
|
||||
|
||||
results = {'expressions': [], 'success': False, 'part': part_path}
|
||||
output_lines = []
|
||||
|
||||
try:
|
||||
# Set load options
|
||||
working_dir = os.path.dirname(part_path)
|
||||
session.Parts.LoadOptions.ComponentLoadMethod = NXOpen.LoadOptions.LoadMethod.FromDirectory
|
||||
session.Parts.LoadOptions.SetSearchDirectories([working_dir], [True])
|
||||
|
||||
# Open the part
|
||||
output_lines.append(f"Opening: {part_path}")
|
||||
basePart, loadStatus = session.Parts.OpenActiveDisplay(
|
||||
part_path,
|
||||
NXOpen.DisplayPartOption.AllowAdditional
|
||||
)
|
||||
loadStatus.Dispose()
|
||||
|
||||
workPart = session.Parts.Work
|
||||
output_lines.append(f"Loaded: {workPart.Name}")
|
||||
output_lines.append("")
|
||||
output_lines.append("=" * 60)
|
||||
output_lines.append("EXPRESSIONS IN M1_Blank.prt")
|
||||
output_lines.append("=" * 60)
|
||||
|
||||
# Extract expressions
|
||||
for expr in workPart.Expressions:
|
||||
try:
|
||||
name = expr.Name
|
||||
|
||||
# Skip internal expressions (p0, p1, p123, etc.)
|
||||
if name.startswith('p') and len(name) > 1:
|
||||
rest = name[1:]
|
||||
# Check if rest is numeric (possibly with dots for decimals)
|
||||
if rest.replace('.', '').replace('_', '').isdigit():
|
||||
continue
|
||||
|
||||
value = expr.Value
|
||||
units = expr.Units.Name if expr.Units else ''
|
||||
rhs = expr.RightHandSide if hasattr(expr, 'RightHandSide') else ''
|
||||
|
||||
results['expressions'].append({
|
||||
'name': name,
|
||||
'value': value,
|
||||
'units': units,
|
||||
'formula': rhs
|
||||
})
|
||||
|
||||
units_str = f" [{units}]" if units else ""
|
||||
output_lines.append(f"{name}: {value}{units_str}")
|
||||
|
||||
except Exception as e:
|
||||
output_lines.append(f"Error reading expression: {e}")
|
||||
|
||||
results['success'] = True
|
||||
results['count'] = len(results['expressions'])
|
||||
output_lines.append("")
|
||||
output_lines.append(f"Total user expressions: {results['count']}")
|
||||
|
||||
except Exception as e:
|
||||
import traceback
|
||||
results['error'] = str(e)
|
||||
results['traceback'] = traceback.format_exc()
|
||||
output_lines.append(f"ERROR: {e}")
|
||||
output_lines.append(traceback.format_exc())
|
||||
|
||||
# Write outputs
|
||||
with open(output_json, 'w') as f:
|
||||
json.dump(results, f, indent=2)
|
||||
|
||||
with open(output_txt, 'w') as f:
|
||||
f.write('\n'.join(output_lines))
|
||||
|
||||
# Exit NX
|
||||
try:
|
||||
session.Parts.Work.Close(NXOpen.BasePart.CloseWholeTree.FalseValue,
|
||||
NXOpen.BasePart.CloseModified.CloseModified, None)
|
||||
except:
|
||||
pass
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
620
nx_journals/introspect_part.py
Normal file
620
nx_journals/introspect_part.py
Normal file
@@ -0,0 +1,620 @@
|
||||
"""
|
||||
NX Journal: Comprehensive Part Introspection Tool
|
||||
===================================================
|
||||
|
||||
This journal performs deep introspection of an NX .prt file and extracts:
|
||||
- All expressions (user and internal, with values, units, formulas)
|
||||
- Mass properties (mass, volume, surface area, center of gravity)
|
||||
- Material properties (name, density, all material attributes)
|
||||
- Body information (solid bodies, sheet bodies, body attributes)
|
||||
- Part attributes (all user-defined attributes)
|
||||
- Groups (all groups and their members)
|
||||
- Features (all features in the part)
|
||||
- References (linked parts, assembly components)
|
||||
- Datum planes, coordinate systems
|
||||
- Units system
|
||||
|
||||
Usage:
|
||||
run_journal.exe introspect_part.py <prt_file_path> [output_dir]
|
||||
|
||||
Output:
|
||||
_temp_introspection.json - Comprehensive JSON with all extracted data
|
||||
|
||||
Author: Atomizer
|
||||
Created: 2025-12-19
|
||||
Version: 1.0
|
||||
"""
|
||||
|
||||
import sys
|
||||
import os
|
||||
import json
|
||||
import NXOpen
|
||||
import NXOpen.UF
|
||||
|
||||
|
||||
def get_expressions(part):
|
||||
"""Extract all expressions from the part."""
|
||||
expressions = {
|
||||
'user': [],
|
||||
'internal': [],
|
||||
'total_count': 0,
|
||||
'user_count': 0
|
||||
}
|
||||
|
||||
try:
|
||||
for expr in part.Expressions:
|
||||
try:
|
||||
expr_data = {
|
||||
'name': expr.Name,
|
||||
'value': expr.Value,
|
||||
'rhs': expr.RightHandSide if hasattr(expr, 'RightHandSide') else None,
|
||||
'units': expr.Units.Name if expr.Units else None,
|
||||
'type': str(expr.Type) if hasattr(expr, 'Type') else 'Unknown',
|
||||
}
|
||||
|
||||
# Determine if internal (p0, p1, p123, etc.)
|
||||
name = expr.Name
|
||||
is_internal = False
|
||||
if name.startswith('p') and len(name) > 1:
|
||||
rest = name[1:].replace('.', '').replace('_', '')
|
||||
if rest.isdigit():
|
||||
is_internal = True
|
||||
|
||||
if is_internal:
|
||||
expressions['internal'].append(expr_data)
|
||||
else:
|
||||
expressions['user'].append(expr_data)
|
||||
|
||||
except Exception as e:
|
||||
pass
|
||||
|
||||
expressions['total_count'] = len(expressions['user']) + len(expressions['internal'])
|
||||
expressions['user_count'] = len(expressions['user'])
|
||||
|
||||
except Exception as e:
|
||||
expressions['error'] = str(e)
|
||||
|
||||
return expressions
|
||||
|
||||
|
||||
def get_all_solid_bodies(part):
|
||||
"""Get all solid bodies from the part."""
|
||||
bodies = []
|
||||
try:
|
||||
for body in part.Bodies:
|
||||
if body.IsSolidBody:
|
||||
bodies.append(body)
|
||||
except Exception as e:
|
||||
pass
|
||||
return bodies
|
||||
|
||||
|
||||
def get_mass_properties(part, bodies):
|
||||
"""Extract mass properties using MeasureManager."""
|
||||
results = {
|
||||
'mass_kg': 0.0,
|
||||
'mass_g': 0.0,
|
||||
'volume_mm3': 0.0,
|
||||
'surface_area_mm2': 0.0,
|
||||
'center_of_gravity_mm': [0.0, 0.0, 0.0],
|
||||
'num_bodies': len(bodies),
|
||||
'success': False
|
||||
}
|
||||
|
||||
if not bodies:
|
||||
return results
|
||||
|
||||
try:
|
||||
measureManager = part.MeasureManager
|
||||
bodyArray = list(bodies)
|
||||
|
||||
# Build mass_units array
|
||||
uc = part.UnitCollection
|
||||
mass_units = [
|
||||
uc.GetBase("Area"),
|
||||
uc.GetBase("Volume"),
|
||||
uc.GetBase("Mass"),
|
||||
uc.GetBase("Length")
|
||||
]
|
||||
|
||||
measureBodies = measureManager.NewMassProperties(mass_units, 0.99, bodyArray)
|
||||
|
||||
if measureBodies:
|
||||
try:
|
||||
results['mass_kg'] = measureBodies.Mass
|
||||
results['mass_g'] = results['mass_kg'] * 1000.0
|
||||
except:
|
||||
pass
|
||||
|
||||
try:
|
||||
results['volume_mm3'] = measureBodies.Volume
|
||||
except:
|
||||
pass
|
||||
|
||||
try:
|
||||
results['surface_area_mm2'] = measureBodies.Area
|
||||
except:
|
||||
pass
|
||||
|
||||
try:
|
||||
cog = measureBodies.Centroid
|
||||
if cog:
|
||||
results['center_of_gravity_mm'] = [cog.X, cog.Y, cog.Z]
|
||||
except:
|
||||
pass
|
||||
|
||||
results['success'] = True
|
||||
|
||||
try:
|
||||
measureBodies.Dispose()
|
||||
except:
|
||||
pass
|
||||
|
||||
except Exception as e:
|
||||
results['error'] = str(e)
|
||||
|
||||
return results
|
||||
|
||||
|
||||
def get_materials(part, bodies):
|
||||
"""Extract all materials from the part."""
|
||||
materials = {
|
||||
'assigned': [],
|
||||
'available': [],
|
||||
'library': []
|
||||
}
|
||||
|
||||
# Get materials assigned to bodies
|
||||
for body in bodies:
|
||||
try:
|
||||
phys_mat = body.GetPhysicalMaterial()
|
||||
if phys_mat:
|
||||
mat_info = {
|
||||
'name': phys_mat.Name,
|
||||
'body': body.Name if hasattr(body, 'Name') else 'Unknown',
|
||||
'properties': {}
|
||||
}
|
||||
|
||||
# Try to get common material properties
|
||||
prop_names = ['Density', 'YoungModulus', 'PoissonRatio',
|
||||
'ThermalExpansionCoefficient', 'ThermalConductivity',
|
||||
'SpecificHeat', 'YieldStrength', 'UltimateStrength']
|
||||
for prop_name in prop_names:
|
||||
try:
|
||||
val = phys_mat.GetPropertyValue(prop_name)
|
||||
if val is not None:
|
||||
mat_info['properties'][prop_name] = float(val)
|
||||
except:
|
||||
pass
|
||||
|
||||
materials['assigned'].append(mat_info)
|
||||
except:
|
||||
pass
|
||||
|
||||
# Get all materials in part via PhysicalMaterialManager
|
||||
try:
|
||||
pmm = part.PhysicalMaterialManager
|
||||
if pmm:
|
||||
all_mats = pmm.GetAllPhysicalMaterials()
|
||||
for mat in all_mats:
|
||||
try:
|
||||
mat_info = {
|
||||
'name': mat.Name,
|
||||
'properties': {}
|
||||
}
|
||||
prop_names = ['Density', 'YoungModulus', 'PoissonRatio']
|
||||
for prop_name in prop_names:
|
||||
try:
|
||||
val = mat.GetPropertyValue(prop_name)
|
||||
if val is not None:
|
||||
mat_info['properties'][prop_name] = float(val)
|
||||
except:
|
||||
pass
|
||||
materials['available'].append(mat_info)
|
||||
except:
|
||||
pass
|
||||
except Exception as e:
|
||||
materials['pmm_error'] = str(e)
|
||||
|
||||
return materials
|
||||
|
||||
|
||||
def get_body_info(part):
|
||||
"""Get detailed body information."""
|
||||
body_info = {
|
||||
'solid_bodies': [],
|
||||
'sheet_bodies': [],
|
||||
'counts': {
|
||||
'solid': 0,
|
||||
'sheet': 0,
|
||||
'total': 0
|
||||
}
|
||||
}
|
||||
|
||||
try:
|
||||
for body in part.Bodies:
|
||||
body_data = {
|
||||
'name': body.Name if hasattr(body, 'Name') else 'Unknown',
|
||||
'is_solid': body.IsSolidBody,
|
||||
'is_sheet': body.IsSheetBody if hasattr(body, 'IsSheetBody') else False,
|
||||
'attributes': []
|
||||
}
|
||||
|
||||
# Get body attributes
|
||||
try:
|
||||
attrs = body.GetUserAttributes()
|
||||
for attr in attrs:
|
||||
try:
|
||||
body_data['attributes'].append({
|
||||
'title': attr.Title,
|
||||
'type': str(attr.Type),
|
||||
'value': attr.StringValue if hasattr(attr, 'StringValue') else str(attr.Value)
|
||||
})
|
||||
except:
|
||||
pass
|
||||
except:
|
||||
pass
|
||||
|
||||
if body.IsSolidBody:
|
||||
body_info['solid_bodies'].append(body_data)
|
||||
body_info['counts']['solid'] += 1
|
||||
else:
|
||||
body_info['sheet_bodies'].append(body_data)
|
||||
body_info['counts']['sheet'] += 1
|
||||
|
||||
body_info['counts']['total'] = body_info['counts']['solid'] + body_info['counts']['sheet']
|
||||
|
||||
except Exception as e:
|
||||
body_info['error'] = str(e)
|
||||
|
||||
return body_info
|
||||
|
||||
|
||||
def get_part_attributes(part):
|
||||
"""Get all part-level attributes."""
|
||||
attributes = []
|
||||
|
||||
try:
|
||||
attrs = part.GetUserAttributes()
|
||||
for attr in attrs:
|
||||
try:
|
||||
attr_data = {
|
||||
'title': attr.Title,
|
||||
'type': str(attr.Type),
|
||||
}
|
||||
|
||||
# Get value based on type
|
||||
try:
|
||||
if hasattr(attr, 'StringValue'):
|
||||
attr_data['value'] = attr.StringValue
|
||||
elif hasattr(attr, 'Value'):
|
||||
attr_data['value'] = attr.Value
|
||||
elif hasattr(attr, 'IntegerValue'):
|
||||
attr_data['value'] = attr.IntegerValue
|
||||
elif hasattr(attr, 'RealValue'):
|
||||
attr_data['value'] = attr.RealValue
|
||||
except:
|
||||
attr_data['value'] = 'Unknown'
|
||||
|
||||
attributes.append(attr_data)
|
||||
except:
|
||||
pass
|
||||
|
||||
except Exception as e:
|
||||
pass
|
||||
|
||||
return attributes
|
||||
|
||||
|
||||
def get_groups(part):
|
||||
"""Get all groups in the part."""
|
||||
groups = []
|
||||
|
||||
try:
|
||||
# NX stores groups in a collection
|
||||
if hasattr(part, 'Groups'):
|
||||
for group in part.Groups:
|
||||
try:
|
||||
group_data = {
|
||||
'name': group.Name if hasattr(group, 'Name') else 'Unknown',
|
||||
'member_count': 0,
|
||||
'members': []
|
||||
}
|
||||
|
||||
# Try to get group members
|
||||
try:
|
||||
members = group.GetMembers()
|
||||
group_data['member_count'] = len(members) if members else 0
|
||||
for member in members[:10]: # Limit to first 10 for readability
|
||||
try:
|
||||
group_data['members'].append(str(type(member).__name__))
|
||||
except:
|
||||
pass
|
||||
except:
|
||||
pass
|
||||
|
||||
groups.append(group_data)
|
||||
except:
|
||||
pass
|
||||
except Exception as e:
|
||||
pass
|
||||
|
||||
return groups
|
||||
|
||||
|
||||
def get_features(part):
|
||||
"""Get summary of features in the part."""
|
||||
features = {
|
||||
'total_count': 0,
|
||||
'by_type': {},
|
||||
'first_10': []
|
||||
}
|
||||
|
||||
try:
|
||||
count = 0
|
||||
for feature in part.Features:
|
||||
try:
|
||||
feat_type = str(type(feature).__name__)
|
||||
|
||||
# Count by type
|
||||
if feat_type in features['by_type']:
|
||||
features['by_type'][feat_type] += 1
|
||||
else:
|
||||
features['by_type'][feat_type] = 1
|
||||
|
||||
# Store first 10 for reference
|
||||
if count < 10:
|
||||
features['first_10'].append({
|
||||
'name': feature.Name if hasattr(feature, 'Name') else 'Unknown',
|
||||
'type': feat_type
|
||||
})
|
||||
|
||||
count += 1
|
||||
except:
|
||||
pass
|
||||
|
||||
features['total_count'] = count
|
||||
|
||||
except Exception as e:
|
||||
features['error'] = str(e)
|
||||
|
||||
return features
|
||||
|
||||
|
||||
def get_datums(part):
|
||||
"""Get datum planes and coordinate systems."""
|
||||
datums = {
|
||||
'planes': [],
|
||||
'csys': [],
|
||||
'axes': []
|
||||
}
|
||||
|
||||
try:
|
||||
# Datum planes
|
||||
if hasattr(part, 'Datums'):
|
||||
for datum in part.Datums:
|
||||
try:
|
||||
datum_type = str(type(datum).__name__)
|
||||
datum_name = datum.Name if hasattr(datum, 'Name') else 'Unknown'
|
||||
|
||||
if 'Plane' in datum_type:
|
||||
datums['planes'].append(datum_name)
|
||||
elif 'Csys' in datum_type or 'Coordinate' in datum_type:
|
||||
datums['csys'].append(datum_name)
|
||||
elif 'Axis' in datum_type:
|
||||
datums['axes'].append(datum_name)
|
||||
except:
|
||||
pass
|
||||
except Exception as e:
|
||||
datums['error'] = str(e)
|
||||
|
||||
return datums
|
||||
|
||||
|
||||
def get_units_info(part):
|
||||
"""Get unit system information."""
|
||||
units_info = {
|
||||
'base_units': {},
|
||||
'system': 'Unknown'
|
||||
}
|
||||
|
||||
try:
|
||||
uc = part.UnitCollection
|
||||
|
||||
# Get common base units
|
||||
unit_types = ['Length', 'Mass', 'Time', 'Temperature', 'Angle',
|
||||
'Area', 'Volume', 'Force', 'Pressure', 'Density']
|
||||
for unit_type in unit_types:
|
||||
try:
|
||||
base_unit = uc.GetBase(unit_type)
|
||||
if base_unit:
|
||||
units_info['base_units'][unit_type] = base_unit.Name
|
||||
except:
|
||||
pass
|
||||
|
||||
# Determine system from length unit
|
||||
if 'Length' in units_info['base_units']:
|
||||
length_unit = units_info['base_units']['Length'].lower()
|
||||
if 'mm' in length_unit or 'millimeter' in length_unit:
|
||||
units_info['system'] = 'Metric (mm)'
|
||||
elif 'meter' in length_unit and 'milli' not in length_unit:
|
||||
units_info['system'] = 'Metric (m)'
|
||||
elif 'inch' in length_unit or 'in' in length_unit:
|
||||
units_info['system'] = 'Imperial (inch)'
|
||||
|
||||
except Exception as e:
|
||||
units_info['error'] = str(e)
|
||||
|
||||
return units_info
|
||||
|
||||
|
||||
def get_linked_parts(theSession, working_dir):
|
||||
"""Get information about linked/associated parts."""
|
||||
linked_parts = {
|
||||
'loaded_parts': [],
|
||||
'fem_parts': [],
|
||||
'sim_parts': [],
|
||||
'idealized_parts': []
|
||||
}
|
||||
|
||||
try:
|
||||
for part in theSession.Parts:
|
||||
try:
|
||||
part_name = part.Name if hasattr(part, 'Name') else str(part)
|
||||
part_path = part.FullPath if hasattr(part, 'FullPath') else 'Unknown'
|
||||
|
||||
part_info = {
|
||||
'name': part_name,
|
||||
'path': part_path,
|
||||
'leaf_name': part.Leaf if hasattr(part, 'Leaf') else part_name
|
||||
}
|
||||
|
||||
name_lower = part_name.lower()
|
||||
if '_sim' in name_lower or name_lower.endswith('.sim'):
|
||||
linked_parts['sim_parts'].append(part_info)
|
||||
elif '_fem' in name_lower or name_lower.endswith('.fem'):
|
||||
if '_i.prt' in name_lower or '_i' in name_lower:
|
||||
linked_parts['idealized_parts'].append(part_info)
|
||||
else:
|
||||
linked_parts['fem_parts'].append(part_info)
|
||||
elif '_i.prt' in name_lower:
|
||||
linked_parts['idealized_parts'].append(part_info)
|
||||
else:
|
||||
linked_parts['loaded_parts'].append(part_info)
|
||||
|
||||
except:
|
||||
pass
|
||||
|
||||
except Exception as e:
|
||||
linked_parts['error'] = str(e)
|
||||
|
||||
return linked_parts
|
||||
|
||||
|
||||
def main(args):
|
||||
"""Main entry point for NX journal."""
|
||||
|
||||
if len(args) < 1:
|
||||
print("ERROR: No .prt file path provided")
|
||||
print("Usage: run_journal.exe introspect_part.py <prt_file> [output_dir]")
|
||||
return False
|
||||
|
||||
prt_file_path = args[0]
|
||||
output_dir = args[1] if len(args) > 1 else os.path.dirname(prt_file_path)
|
||||
prt_filename = os.path.basename(prt_file_path)
|
||||
|
||||
print(f"[INTROSPECT] " + "="*60)
|
||||
print(f"[INTROSPECT] NX COMPREHENSIVE PART INTROSPECTION")
|
||||
print(f"[INTROSPECT] " + "="*60)
|
||||
print(f"[INTROSPECT] Part: {prt_filename}")
|
||||
print(f"[INTROSPECT] Output: {output_dir}")
|
||||
|
||||
results = {
|
||||
'part_file': prt_filename,
|
||||
'part_path': prt_file_path,
|
||||
'success': False,
|
||||
'error': None,
|
||||
'expressions': {},
|
||||
'mass_properties': {},
|
||||
'materials': {},
|
||||
'bodies': {},
|
||||
'attributes': [],
|
||||
'groups': [],
|
||||
'features': {},
|
||||
'datums': {},
|
||||
'units': {},
|
||||
'linked_parts': {}
|
||||
}
|
||||
|
||||
try:
|
||||
theSession = NXOpen.Session.GetSession()
|
||||
|
||||
# Set load options
|
||||
working_dir = os.path.dirname(prt_file_path)
|
||||
theSession.Parts.LoadOptions.ComponentLoadMethod = NXOpen.LoadOptions.LoadMethod.FromDirectory
|
||||
theSession.Parts.LoadOptions.SetSearchDirectories([working_dir], [True])
|
||||
|
||||
# Open the part file
|
||||
print(f"[INTROSPECT] Opening part file...")
|
||||
basePart, partLoadStatus = theSession.Parts.OpenActiveDisplay(
|
||||
prt_file_path,
|
||||
NXOpen.DisplayPartOption.AllowAdditional
|
||||
)
|
||||
partLoadStatus.Dispose()
|
||||
|
||||
workPart = theSession.Parts.Work
|
||||
print(f"[INTROSPECT] Loaded: {workPart.Name}")
|
||||
|
||||
# Extract all data
|
||||
print(f"[INTROSPECT] Extracting expressions...")
|
||||
results['expressions'] = get_expressions(workPart)
|
||||
print(f"[INTROSPECT] Found {results['expressions']['user_count']} user expressions")
|
||||
|
||||
print(f"[INTROSPECT] Extracting body info...")
|
||||
results['bodies'] = get_body_info(workPart)
|
||||
print(f"[INTROSPECT] Found {results['bodies']['counts']['solid']} solid bodies")
|
||||
|
||||
print(f"[INTROSPECT] Extracting mass properties...")
|
||||
bodies = get_all_solid_bodies(workPart)
|
||||
results['mass_properties'] = get_mass_properties(workPart, bodies)
|
||||
print(f"[INTROSPECT] Mass: {results['mass_properties']['mass_kg']:.4f} kg")
|
||||
|
||||
print(f"[INTROSPECT] Extracting materials...")
|
||||
results['materials'] = get_materials(workPart, bodies)
|
||||
print(f"[INTROSPECT] Found {len(results['materials']['assigned'])} assigned materials")
|
||||
|
||||
print(f"[INTROSPECT] Extracting attributes...")
|
||||
results['attributes'] = get_part_attributes(workPart)
|
||||
print(f"[INTROSPECT] Found {len(results['attributes'])} part attributes")
|
||||
|
||||
print(f"[INTROSPECT] Extracting groups...")
|
||||
results['groups'] = get_groups(workPart)
|
||||
print(f"[INTROSPECT] Found {len(results['groups'])} groups")
|
||||
|
||||
print(f"[INTROSPECT] Extracting features...")
|
||||
results['features'] = get_features(workPart)
|
||||
print(f"[INTROSPECT] Found {results['features']['total_count']} features")
|
||||
|
||||
print(f"[INTROSPECT] Extracting datums...")
|
||||
results['datums'] = get_datums(workPart)
|
||||
print(f"[INTROSPECT] Found {len(results['datums']['planes'])} datum planes")
|
||||
|
||||
print(f"[INTROSPECT] Extracting units...")
|
||||
results['units'] = get_units_info(workPart)
|
||||
print(f"[INTROSPECT] System: {results['units']['system']}")
|
||||
|
||||
print(f"[INTROSPECT] Extracting linked parts...")
|
||||
results['linked_parts'] = get_linked_parts(theSession, working_dir)
|
||||
print(f"[INTROSPECT] Found {len(results['linked_parts']['loaded_parts'])} loaded parts")
|
||||
|
||||
results['success'] = True
|
||||
print(f"[INTROSPECT] ")
|
||||
print(f"[INTROSPECT] INTROSPECTION COMPLETE!")
|
||||
print(f"[INTROSPECT] " + "="*60)
|
||||
|
||||
# Summary
|
||||
print(f"[INTROSPECT] SUMMARY:")
|
||||
print(f"[INTROSPECT] Expressions: {results['expressions']['user_count']} user, {len(results['expressions']['internal'])} internal")
|
||||
print(f"[INTROSPECT] Mass: {results['mass_properties']['mass_kg']:.4f} kg ({results['mass_properties']['mass_g']:.2f} g)")
|
||||
print(f"[INTROSPECT] Bodies: {results['bodies']['counts']['solid']} solid, {results['bodies']['counts']['sheet']} sheet")
|
||||
print(f"[INTROSPECT] Features: {results['features']['total_count']}")
|
||||
print(f"[INTROSPECT] Materials: {len(results['materials']['assigned'])} assigned")
|
||||
|
||||
except Exception as e:
|
||||
results['error'] = str(e)
|
||||
results['success'] = False
|
||||
print(f"[INTROSPECT] FATAL ERROR: {e}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
|
||||
# Write results
|
||||
output_file = os.path.join(output_dir, "_temp_introspection.json")
|
||||
with open(output_file, 'w') as f:
|
||||
json.dump(results, f, indent=2)
|
||||
print(f"[INTROSPECT] Results written to: {output_file}")
|
||||
|
||||
return results['success']
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main(sys.argv[1:])
|
||||
55
nx_journals/list_expressions_simple.py
Normal file
55
nx_journals/list_expressions_simple.py
Normal file
@@ -0,0 +1,55 @@
|
||||
"""Simple expression lister - writes to file regardless of print issues"""
|
||||
import NXOpen
|
||||
import os
|
||||
import json
|
||||
|
||||
session = NXOpen.Session.GetSession()
|
||||
output_lines = []
|
||||
results = {'expressions': [], 'success': False}
|
||||
|
||||
try:
|
||||
# Get all open parts and find M1_Blank
|
||||
for part in session.Parts:
|
||||
part_name = part.Name if hasattr(part, 'Name') else str(part)
|
||||
if 'M1_Blank' in part_name and '_fem' not in part_name.lower() and '_i' not in part_name.lower():
|
||||
output_lines.append(f"Found part: {part_name}")
|
||||
|
||||
for expr in part.Expressions:
|
||||
try:
|
||||
name = expr.Name
|
||||
# Skip internal expressions (p0, p1, etc.)
|
||||
if name.startswith('p') and len(name) > 1:
|
||||
rest = name[1:].replace('.', '').replace('_', '')
|
||||
if rest.isdigit():
|
||||
continue
|
||||
|
||||
value = expr.Value
|
||||
units = expr.Units.Name if expr.Units else ''
|
||||
rhs = expr.RightHandSide if hasattr(expr, 'RightHandSide') else ''
|
||||
|
||||
results['expressions'].append({
|
||||
'name': name,
|
||||
'value': value,
|
||||
'units': units,
|
||||
'rhs': rhs
|
||||
})
|
||||
output_lines.append(f"{name}: {value} {units}")
|
||||
except:
|
||||
pass
|
||||
|
||||
results['success'] = True
|
||||
break
|
||||
|
||||
except Exception as e:
|
||||
output_lines.append(f"Error: {str(e)}")
|
||||
results['error'] = str(e)
|
||||
|
||||
# Write to file
|
||||
output_path = r"C:\Users\antoi\Atomizer\_expressions_output.json"
|
||||
with open(output_path, 'w') as f:
|
||||
json.dump(results, f, indent=2)
|
||||
|
||||
# Also write text version
|
||||
text_path = r"C:\Users\antoi\Atomizer\_expressions_output.txt"
|
||||
with open(text_path, 'w') as f:
|
||||
f.write('\n'.join(output_lines))
|
||||
11
nx_journals/test_write.py
Normal file
11
nx_journals/test_write.py
Normal file
@@ -0,0 +1,11 @@
|
||||
"""Simple test - just write a file"""
|
||||
with open(r"C:\Users\antoi\Atomizer\_test_output.txt", 'w') as f:
|
||||
f.write("Journal executed successfully!\n")
|
||||
|
||||
try:
|
||||
import NXOpen
|
||||
f.write("NXOpen imported OK\n")
|
||||
session = NXOpen.Session.GetSession()
|
||||
f.write(f"Session: {session}\n")
|
||||
except Exception as e:
|
||||
f.write(f"NXOpen error: {e}\n")
|
||||
@@ -0,0 +1,229 @@
|
||||
# Designcenter 2512
|
||||
# Journal created by antoi on Thu Dec 18 14:06:36 2025 Eastern Standard Time
|
||||
#
|
||||
import math
|
||||
import NXOpen
|
||||
import NXOpen.Gateway
|
||||
def main(args) :
|
||||
|
||||
theSession = NXOpen.Session.GetSession() #type: NXOpen.Session
|
||||
workPart = theSession.Parts.Work
|
||||
displayPart = theSession.Parts.Display
|
||||
# ----------------------------------------------
|
||||
# Menu: Edit->Show and Hide->Show and Hide...
|
||||
# ----------------------------------------------
|
||||
markId1 = theSession.SetUndoMark(NXOpen.Session.MarkVisibility.Visible, "Start")
|
||||
|
||||
theSession.SetUndoMarkName(markId1, "Show and Hide Dialog")
|
||||
|
||||
markId2 = theSession.SetUndoMark(NXOpen.Session.MarkVisibility.Visible, "Hide Datums")
|
||||
|
||||
numberHidden1 = theSession.DisplayManager.HideByType("SHOW_HIDE_TYPE_DATUMS", NXOpen.DisplayManager.ShowHideScope.AnyInAssembly)
|
||||
|
||||
nErrs1 = theSession.UpdateManager.DoUpdate(markId2)
|
||||
|
||||
workPart.ModelingViews.WorkView.FitAfterShowOrHide(NXOpen.View.ShowOrHideType.HideOnly)
|
||||
|
||||
markId3 = theSession.SetUndoMark(NXOpen.Session.MarkVisibility.Visible, "Hide Curves")
|
||||
|
||||
numberHidden2 = theSession.DisplayManager.HideByType("SHOW_HIDE_TYPE_CURVES", NXOpen.DisplayManager.ShowHideScope.AnyInAssembly)
|
||||
|
||||
nErrs2 = theSession.UpdateManager.DoUpdate(markId3)
|
||||
|
||||
exists1 = theSession.DoesUndoMarkExist(markId3, "Hide Curves")
|
||||
|
||||
theSession.DeleteUndoMark(markId3, "Hide Curves")
|
||||
|
||||
workPart.ModelingViews.WorkView.FitAfterShowOrHide(NXOpen.View.ShowOrHideType.HideOnly)
|
||||
|
||||
markId4 = theSession.SetUndoMark(NXOpen.Session.MarkVisibility.Visible, "Hide Sketches")
|
||||
|
||||
numberHidden3 = theSession.DisplayManager.HideByType("SHOW_HIDE_TYPE_SKETCHES", NXOpen.DisplayManager.ShowHideScope.AnyInAssembly)
|
||||
|
||||
nErrs3 = theSession.UpdateManager.DoUpdate(markId4)
|
||||
|
||||
workPart.ModelingViews.WorkView.FitAfterShowOrHide(NXOpen.View.ShowOrHideType.HideOnly)
|
||||
|
||||
theSession.SetUndoMarkName(markId1, "Show and Hide")
|
||||
|
||||
theSession.DeleteUndoMark(markId1, None)
|
||||
|
||||
matrix1 = NXOpen.Matrix3x3()
|
||||
|
||||
matrix1.Xx = 0.0
|
||||
matrix1.Xy = -1.0
|
||||
matrix1.Xz = 0.0
|
||||
matrix1.Yx = -1.0
|
||||
matrix1.Yy = -0.0
|
||||
matrix1.Yz = -0.0
|
||||
matrix1.Zx = 0.0
|
||||
matrix1.Zy = 0.0
|
||||
matrix1.Zz = -1.0
|
||||
workPart.ModelingViews.WorkView.Orient(matrix1)
|
||||
|
||||
scaleAboutPoint1 = NXOpen.Point3d(-759.81281858578541, -319.30527689743337, 0.0)
|
||||
viewCenter1 = NXOpen.Point3d(759.81281858579484, 319.30527689744417, 0.0)
|
||||
workPart.ModelingViews.WorkView.ZoomAboutPoint(0.80000000000000004, scaleAboutPoint1, viewCenter1)
|
||||
|
||||
scaleAboutPoint2 = NXOpen.Point3d(-949.76602323223278, -399.13159612179305, 0.0)
|
||||
viewCenter2 = NXOpen.Point3d(949.76602323224245, 399.13159612180385, 0.0)
|
||||
workPart.ModelingViews.WorkView.ZoomAboutPoint(0.80000000000000004, scaleAboutPoint2, viewCenter2)
|
||||
|
||||
scaleAboutPoint3 = NXOpen.Point3d(-1394.8708922057567, -214.19365760462478, 0.0)
|
||||
viewCenter3 = NXOpen.Point3d(1394.870892205766, 214.19365760463569, 0.0)
|
||||
workPart.ModelingViews.WorkView.ZoomAboutPoint(1.25, scaleAboutPoint3, viewCenter3)
|
||||
|
||||
scaleAboutPoint4 = NXOpen.Point3d(-1115.8967137646043, -171.35492608369873, 0.0)
|
||||
viewCenter4 = NXOpen.Point3d(1115.8967137646139, 171.35492608370959, 0.0)
|
||||
workPart.ModelingViews.WorkView.ZoomAboutPoint(0.80000000000000004, scaleAboutPoint4, viewCenter4)
|
||||
|
||||
# ----------------------------------------------
|
||||
# Menu: File->Export->Image...
|
||||
# ----------------------------------------------
|
||||
markId5 = theSession.SetUndoMark(NXOpen.Session.MarkVisibility.Visible, "Start")
|
||||
|
||||
imageExportBuilder1 = workPart.Views.CreateImageExportBuilder()
|
||||
|
||||
imageExportBuilder1.RegionMode = True
|
||||
|
||||
regiontopleftpoint1 = [None] * 2
|
||||
regiontopleftpoint1[0] = 95
|
||||
regiontopleftpoint1[1] = 83
|
||||
imageExportBuilder1.SetRegionTopLeftPoint(regiontopleftpoint1)
|
||||
|
||||
imageExportBuilder1.RegionWidth = 1157
|
||||
|
||||
imageExportBuilder1.RegionHeight = 1056
|
||||
|
||||
imageExportBuilder1.DeviceWidth = 2388
|
||||
|
||||
imageExportBuilder1.DeviceHeight = 1172
|
||||
|
||||
imageExportBuilder1.FileFormat = NXOpen.Gateway.ImageExportBuilder.FileFormats.Png
|
||||
|
||||
imageExportBuilder1.FileName = "C:\\Users\\antoi\\Atomizer\\studies\\M1_Mirror\\m1_mirror_cost_reduction_V4\\1_setup\\M1_Blank_Top.png"
|
||||
|
||||
imageExportBuilder1.BackgroundOption = NXOpen.Gateway.ImageExportBuilder.BackgroundOptions.Original
|
||||
|
||||
imageExportBuilder1.EnhanceEdges = False
|
||||
|
||||
nXObject1 = imageExportBuilder1.Commit()
|
||||
|
||||
theSession.DeleteUndoMark(markId5, "Export Image")
|
||||
|
||||
imageExportBuilder1.Destroy()
|
||||
|
||||
markId6 = theSession.SetUndoMark(NXOpen.Session.MarkVisibility.Visible, "Start")
|
||||
|
||||
imageExportBuilder2 = workPart.Views.CreateImageExportBuilder()
|
||||
|
||||
imageExportBuilder2.Destroy()
|
||||
|
||||
theSession.UndoToMark(markId6, None)
|
||||
|
||||
theSession.DeleteUndoMark(markId6, None)
|
||||
|
||||
rotMatrix1 = NXOpen.Matrix3x3()
|
||||
|
||||
rotMatrix1.Xx = -0.34262722569067999
|
||||
rotMatrix1.Xy = -0.93944302509010613
|
||||
rotMatrix1.Xz = 0.0073066288434778118
|
||||
rotMatrix1.Yx = -0.67329035687890959
|
||||
rotMatrix1.Yy = 0.24011894541756998
|
||||
rotMatrix1.Yz = -0.69930178563008338
|
||||
rotMatrix1.Zx = 0.65519972493078527
|
||||
rotMatrix1.Zy = -0.2445193134725811
|
||||
rotMatrix1.Zz = -0.71478921773451431
|
||||
translation1 = NXOpen.Point3d(-691.94814615291523, -16.771832954225655, -903.92900031772103)
|
||||
workPart.ModelingViews.WorkView.SetRotationTranslationScale(rotMatrix1, translation1, 0.20258147300869808)
|
||||
|
||||
scaleAboutPoint5 = NXOpen.Point3d(-1091.8652302284754, -297.78142642594378, 0.0)
|
||||
viewCenter5 = NXOpen.Point3d(1091.8652302284847, 297.78142642595469, 0.0)
|
||||
workPart.ModelingViews.WorkView.ZoomAboutPoint(1.25, scaleAboutPoint5, viewCenter5)
|
||||
|
||||
scaleAboutPoint6 = NXOpen.Point3d(-873.49218418277917, -238.22514114075392, 0.0)
|
||||
viewCenter6 = NXOpen.Point3d(873.49218418278895, 238.2251411407648, 0.0)
|
||||
workPart.ModelingViews.WorkView.ZoomAboutPoint(1.25, scaleAboutPoint6, viewCenter6)
|
||||
|
||||
scaleAboutPoint7 = NXOpen.Point3d(-519.08004438038643, -302.5877231331695, 0.0)
|
||||
viewCenter7 = NXOpen.Point3d(519.08004438039586, 302.58772313318048, 0.0)
|
||||
workPart.ModelingViews.WorkView.ZoomAboutPoint(0.80000000000000004, scaleAboutPoint7, viewCenter7)
|
||||
|
||||
scaleAboutPoint8 = NXOpen.Point3d(-648.85005547548417, -378.23465391646323, 0.0)
|
||||
viewCenter8 = NXOpen.Point3d(648.85005547549372, 378.23465391647414, 0.0)
|
||||
workPart.ModelingViews.WorkView.ZoomAboutPoint(0.80000000000000004, scaleAboutPoint8, viewCenter8)
|
||||
|
||||
scaleAboutPoint9 = NXOpen.Point3d(-726.16874163520447, -271.6602486692816, 0.0)
|
||||
viewCenter9 = NXOpen.Point3d(726.16874163521379, 271.66024866929223, 0.0)
|
||||
workPart.ModelingViews.WorkView.ZoomAboutPoint(1.25, scaleAboutPoint9, viewCenter9)
|
||||
|
||||
rotMatrix2 = NXOpen.Matrix3x3()
|
||||
|
||||
rotMatrix2.Xx = -0.35281096074613638
|
||||
rotMatrix2.Xy = -0.93549939803135751
|
||||
rotMatrix2.Xz = 0.019112882052533756
|
||||
rotMatrix2.Yx = -0.67083068516183819
|
||||
rotMatrix2.Yy = 0.23864906945399289
|
||||
rotMatrix2.Yz = -0.70216295366107118
|
||||
rotMatrix2.Zx = 0.65231174895343103
|
||||
rotMatrix2.Zy = -0.26055229404422597
|
||||
rotMatrix2.Zz = -0.71175970962509794
|
||||
translation2 = NXOpen.Point3d(-445.60899304577225, -25.448049758528374, -903.92478002019129)
|
||||
workPart.ModelingViews.WorkView.SetRotationTranslationScale(rotMatrix2, translation2, 0.25322684126087264)
|
||||
|
||||
rotMatrix3 = NXOpen.Matrix3x3()
|
||||
|
||||
rotMatrix3.Xx = -0.32736574141345925
|
||||
rotMatrix3.Xy = -0.94489752125198745
|
||||
rotMatrix3.Xz = -0.00058794613984273266
|
||||
rotMatrix3.Yx = -0.71924452681462514
|
||||
rotMatrix3.Yy = 0.24959027079525001
|
||||
rotMatrix3.Yz = -0.64837643955618585
|
||||
rotMatrix3.Zx = 0.61279603621108569
|
||||
rotMatrix3.Zy = -0.21183335680718612
|
||||
rotMatrix3.Zz = -0.76131967460967154
|
||||
translation3 = NXOpen.Point3d(-445.6364375527848, -25.373121722553414, -903.99382020435428)
|
||||
workPart.ModelingViews.WorkView.SetRotationTranslationScale(rotMatrix3, translation3, 0.25322684126087264)
|
||||
|
||||
# ----------------------------------------------
|
||||
# Menu: File->Export->Image...
|
||||
# ----------------------------------------------
|
||||
markId7 = theSession.SetUndoMark(NXOpen.Session.MarkVisibility.Visible, "Start")
|
||||
|
||||
imageExportBuilder3 = workPart.Views.CreateImageExportBuilder()
|
||||
|
||||
imageExportBuilder3.RegionMode = True
|
||||
|
||||
regiontopleftpoint2 = [None] * 2
|
||||
regiontopleftpoint2[0] = 129
|
||||
regiontopleftpoint2[1] = 96
|
||||
imageExportBuilder3.SetRegionTopLeftPoint(regiontopleftpoint2)
|
||||
|
||||
imageExportBuilder3.RegionWidth = 1343
|
||||
|
||||
imageExportBuilder3.RegionHeight = 1045
|
||||
|
||||
imageExportBuilder3.DeviceWidth = 2388
|
||||
|
||||
imageExportBuilder3.DeviceHeight = 1172
|
||||
|
||||
imageExportBuilder3.FileFormat = NXOpen.Gateway.ImageExportBuilder.FileFormats.Png
|
||||
|
||||
imageExportBuilder3.FileName = "C:\\Users\\antoi\\Atomizer\\studies\\M1_Mirror\\m1_mirror_cost_reduction_V4\\1_setup\\M1_Blank_iso.png"
|
||||
|
||||
imageExportBuilder3.BackgroundOption = NXOpen.Gateway.ImageExportBuilder.BackgroundOptions.Original
|
||||
|
||||
imageExportBuilder3.EnhanceEdges = False
|
||||
|
||||
nXObject2 = imageExportBuilder3.Commit()
|
||||
|
||||
theSession.DeleteUndoMark(markId7, "Export Image")
|
||||
|
||||
imageExportBuilder3.Destroy()
|
||||
|
||||
# ----------------------------------------------
|
||||
# Menu: Tools->Automation->Journal->Stop Recording
|
||||
# ----------------------------------------------
|
||||
|
||||
if __name__ == '__main__':
|
||||
main(sys.argv[1:])
|
||||
@@ -51,6 +51,7 @@ class DesignSpaceInsight(StudyInsight):
|
||||
insight_type = "design_space"
|
||||
name = "Design Space Explorer"
|
||||
description = "Interactive parameter-objective relationship visualization"
|
||||
category = "design_exploration"
|
||||
applicable_to = ["all"] # Works with any optimization study
|
||||
required_files = [] # Requires study.db, not OP2
|
||||
|
||||
|
||||
@@ -55,6 +55,7 @@ class ModalInsight(StudyInsight):
|
||||
insight_type = "modal"
|
||||
name = "Modal Analysis"
|
||||
description = "Natural frequencies and mode shapes visualization"
|
||||
category = "structural_modal"
|
||||
applicable_to = ["modal", "vibration", "dynamic", "all"]
|
||||
required_files = ["*.op2"]
|
||||
|
||||
|
||||
@@ -56,6 +56,7 @@ class StressFieldInsight(StudyInsight):
|
||||
insight_type = "stress_field"
|
||||
name = "Stress Distribution"
|
||||
description = "3D stress contour plot with Von Mises and principal stresses"
|
||||
category = "structural_static"
|
||||
applicable_to = ["structural", "bracket", "beam", "all"]
|
||||
required_files = ["*.op2"]
|
||||
|
||||
|
||||
@@ -55,6 +55,7 @@ class ThermalInsight(StudyInsight):
|
||||
insight_type = "thermal"
|
||||
name = "Thermal Analysis"
|
||||
description = "Temperature distribution and thermal gradients"
|
||||
category = "thermal"
|
||||
applicable_to = ["thermal", "thermo-structural", "all"]
|
||||
required_files = ["*.op2"]
|
||||
|
||||
|
||||
74
temp_compare.py
Normal file
74
temp_compare.py
Normal file
@@ -0,0 +1,74 @@
|
||||
#!/usr/bin/env python
|
||||
"""Compare V8 and V11 lateral parameter convergence"""
|
||||
import optuna
|
||||
import statistics
|
||||
|
||||
# Load V8 study
|
||||
v8_study = optuna.load_study(
|
||||
study_name='m1_mirror_cost_reduction_V8',
|
||||
storage='sqlite:///studies/M1_Mirror/m1_mirror_cost_reduction_V8/3_results/study.db'
|
||||
)
|
||||
|
||||
# Load V11 study
|
||||
v11_study = optuna.load_study(
|
||||
study_name='m1_mirror_cost_reduction_V11',
|
||||
storage='sqlite:///studies/M1_Mirror/m1_mirror_cost_reduction_V11/3_results/study.db'
|
||||
)
|
||||
|
||||
print("="*70)
|
||||
print("V8 BEST TRIAL (Z-only Zernike)")
|
||||
print("="*70)
|
||||
v8_best = v8_study.best_trial
|
||||
print(f"Trial: {v8_best.number}")
|
||||
print(f"WS: {v8_best.value:.2f}")
|
||||
print("\nLateral Parameters:")
|
||||
for k, v in sorted(v8_best.params.items()):
|
||||
print(f" {k}: {v:.4f}")
|
||||
print("\nObjectives:")
|
||||
for k, v in v8_best.user_attrs.items():
|
||||
if isinstance(v, (int, float)):
|
||||
print(f" {k}: {v:.4f}")
|
||||
|
||||
print("\n" + "="*70)
|
||||
print("V11 BEST TRIAL (ZernikeOPD + extract_relative)")
|
||||
print("="*70)
|
||||
v11_best = v11_study.best_trial
|
||||
print(f"Trial: {v11_best.number}")
|
||||
print(f"WS: {v11_best.value:.2f}")
|
||||
print("\nLateral Parameters:")
|
||||
for k, v in sorted(v11_best.params.items()):
|
||||
print(f" {k}: {v:.4f}")
|
||||
print("\nObjectives:")
|
||||
for k, v in v11_best.user_attrs.items():
|
||||
if isinstance(v, (int, float)):
|
||||
print(f" {k}: {v:.4f}")
|
||||
|
||||
# Compare parameter ranges explored
|
||||
print("\n" + "="*70)
|
||||
print("PARAMETER EXPLORATION COMPARISON")
|
||||
print("="*70)
|
||||
|
||||
params = ['lateral_inner_angle', 'lateral_outer_angle', 'lateral_outer_pivot',
|
||||
'lateral_inner_pivot', 'lateral_middle_pivot', 'lateral_closeness']
|
||||
|
||||
for p in params:
|
||||
v8_vals = [t.params.get(p) for t in v8_study.trials if t.state.name == 'COMPLETE' and p in t.params]
|
||||
v11_vals = [t.params.get(p) for t in v11_study.trials if t.state.name == 'COMPLETE' and p in t.params]
|
||||
|
||||
if v8_vals and v11_vals:
|
||||
print(f"\n{p}:")
|
||||
print(f" V8: mean={statistics.mean(v8_vals):.2f}, std={statistics.stdev(v8_vals) if len(v8_vals) > 1 else 0:.2f}, range=[{min(v8_vals):.2f}, {max(v8_vals):.2f}]")
|
||||
print(f" V11: mean={statistics.mean(v11_vals):.2f}, std={statistics.stdev(v11_vals) if len(v11_vals) > 1 else 0:.2f}, range=[{min(v11_vals):.2f}, {max(v11_vals):.2f}]")
|
||||
print(f" Best V8: {v8_best.params.get(p, 'N/A'):.2f}")
|
||||
print(f" Best V11: {v11_best.params.get(p, 'N/A'):.2f}")
|
||||
|
||||
# Lateral displacement comparison (V11 has this data)
|
||||
print("\n" + "="*70)
|
||||
print("V11 LATERAL DISPLACEMENT DATA (not available in V8)")
|
||||
print("="*70)
|
||||
for t in v11_study.trials:
|
||||
if t.state.name == 'COMPLETE':
|
||||
lat_rms = t.user_attrs.get('lateral_rms_um', None)
|
||||
lat_max = t.user_attrs.get('lateral_max_um', None)
|
||||
if lat_rms is not None:
|
||||
print(f"Trial {t.number}: RMS={lat_rms:.2f} um, Max={lat_max:.2f} um, WS={t.value:.2f}")
|
||||
74
tests/audit_v10_fix.py
Normal file
74
tests/audit_v10_fix.py
Normal file
@@ -0,0 +1,74 @@
|
||||
"""Verify the V10 fix - compare Standard extract_relative vs OPD extract_relative."""
|
||||
import sys
|
||||
from pathlib import Path
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent))
|
||||
|
||||
from optimization_engine.extractors import ZernikeExtractor, ZernikeOPDExtractor
|
||||
|
||||
op2 = Path('studies/M1_Mirror/m1_mirror_cost_reduction_V10/2_iterations/iter1/assy_m1_assyfem1_sim1-solution_1.op2')
|
||||
|
||||
print("="*70)
|
||||
print("VERIFICATION: ZernikeOPDExtractor.extract_relative() vs Standard")
|
||||
print("="*70)
|
||||
print()
|
||||
|
||||
# Standard extractor
|
||||
extractor_std = ZernikeExtractor(op2, n_modes=50, filter_orders=4)
|
||||
|
||||
# OPD extractor (with XY lateral correction)
|
||||
extractor_opd = ZernikeOPDExtractor(op2, n_modes=50, filter_orders=4)
|
||||
|
||||
print("Standard ZernikeExtractor.extract_relative():")
|
||||
rel_40_std = extractor_std.extract_relative('3', '2')
|
||||
rel_60_std = extractor_std.extract_relative('4', '2')
|
||||
rel_90_std = extractor_std.extract_relative('1', '2')
|
||||
print(f" 40-20: {rel_40_std['relative_filtered_rms_nm']:.2f} nm")
|
||||
print(f" 60-20: {rel_60_std['relative_filtered_rms_nm']:.2f} nm")
|
||||
print(f" 90-20 (j1to3): {rel_90_std['relative_rms_filter_j1to3']:.2f} nm")
|
||||
|
||||
print()
|
||||
print("NEW ZernikeOPDExtractor.extract_relative() (with XY lateral correction):")
|
||||
rel_40_opd = extractor_opd.extract_relative('3', '2')
|
||||
rel_60_opd = extractor_opd.extract_relative('4', '2')
|
||||
rel_90_opd = extractor_opd.extract_relative('1', '2')
|
||||
print(f" 40-20: {rel_40_opd['relative_filtered_rms_nm']:.2f} nm")
|
||||
print(f" 60-20: {rel_60_opd['relative_filtered_rms_nm']:.2f} nm")
|
||||
print(f" 90-20 (j1to3): {rel_90_opd['relative_rms_filter_j1to3']:.2f} nm")
|
||||
|
||||
print()
|
||||
print("Lateral displacement diagnostics (OPD method):")
|
||||
print(f" Max lateral: {rel_40_opd['max_lateral_displacement_um']:.3f} um")
|
||||
print(f" RMS lateral: {rel_40_opd['rms_lateral_displacement_um']:.3f} um")
|
||||
|
||||
print()
|
||||
print("="*70)
|
||||
print("COMPARISON")
|
||||
print("="*70)
|
||||
print()
|
||||
print(f"{'Metric':<20} | {'Standard':<12} | {'OPD':<12} | {'Diff %':<10}")
|
||||
print("-"*60)
|
||||
|
||||
def pct_diff(a, b):
|
||||
return 100.0 * (b - a) / a if a > 0 else 0
|
||||
|
||||
print(f"{'40-20 (nm)':<20} | {rel_40_std['relative_filtered_rms_nm']:>12.2f} | {rel_40_opd['relative_filtered_rms_nm']:>12.2f} | {pct_diff(rel_40_std['relative_filtered_rms_nm'], rel_40_opd['relative_filtered_rms_nm']):>+10.1f}%")
|
||||
print(f"{'60-20 (nm)':<20} | {rel_60_std['relative_filtered_rms_nm']:>12.2f} | {rel_60_opd['relative_filtered_rms_nm']:>12.2f} | {pct_diff(rel_60_std['relative_filtered_rms_nm'], rel_60_opd['relative_filtered_rms_nm']):>+10.1f}%")
|
||||
print(f"{'90-20 j1to3 (nm)':<20} | {rel_90_std['relative_rms_filter_j1to3']:>12.2f} | {rel_90_opd['relative_rms_filter_j1to3']:>12.2f} | {pct_diff(rel_90_std['relative_rms_filter_j1to3'], rel_90_opd['relative_rms_filter_j1to3']):>+10.1f}%")
|
||||
|
||||
print()
|
||||
print("="*70)
|
||||
print("WHAT V9 REPORTED (for comparison)")
|
||||
print("="*70)
|
||||
print(" 40-20: 6.10 nm (from DB)")
|
||||
print(" 60-20: 12.76 nm (from DB)")
|
||||
print()
|
||||
print("V10 SHOULD NOW REPORT (using OPD extract_relative):")
|
||||
print(f" 40-20: {rel_40_opd['relative_filtered_rms_nm']:.2f} nm")
|
||||
print(f" 60-20: {rel_60_opd['relative_filtered_rms_nm']:.2f} nm")
|
||||
print(f" 90-20: {rel_90_opd['relative_rms_filter_j1to3']:.2f} nm")
|
||||
print()
|
||||
print("V10 OLD WRONG VALUES WERE:")
|
||||
print(" 40-20: 1.99 nm (WRONG - was computing abs(RMS_target - RMS_ref))")
|
||||
print(" 60-20: 6.82 nm (WRONG)")
|
||||
print()
|
||||
print("FIX VERIFIED: OPD extract_relative() correctly computes RMS of (WFE_target - WFE_ref)")
|
||||
72
tests/audit_v10_method_diff.py
Normal file
72
tests/audit_v10_method_diff.py
Normal file
@@ -0,0 +1,72 @@
|
||||
"""Compare V9 vs V10 calculation methods."""
|
||||
import sys
|
||||
from pathlib import Path
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent))
|
||||
|
||||
from optimization_engine.extractors import ZernikeExtractor
|
||||
|
||||
op2 = Path('studies/M1_Mirror/m1_mirror_cost_reduction_V10/2_iterations/iter1/assy_m1_assyfem1_sim1-solution_1.op2')
|
||||
extractor = ZernikeExtractor(op2, n_modes=50, filter_orders=4)
|
||||
|
||||
print("="*70)
|
||||
print("CRITICAL: V9 vs V10 Calculation Method Comparison")
|
||||
print("="*70)
|
||||
print()
|
||||
|
||||
# This is what V9 does - computes relative WFE THEN fits Zernike
|
||||
rel_40 = extractor.extract_relative('3', '2')
|
||||
rel_60 = extractor.extract_relative('4', '2')
|
||||
rel_90 = extractor.extract_relative('1', '2')
|
||||
|
||||
print('V9 method (ZernikeExtractor.extract_relative):')
|
||||
print(' Computes WFE_diff = WFE_target - WFE_ref node-by-node')
|
||||
print(' Then fits Zernike to WFE_diff')
|
||||
print()
|
||||
print(f' 40-20: {rel_40["relative_filtered_rms_nm"]:.2f} nm')
|
||||
print(f' 60-20: {rel_60["relative_filtered_rms_nm"]:.2f} nm')
|
||||
print(f' 90-20 (j1to3): {rel_90["relative_rms_filter_j1to3"]:.2f} nm')
|
||||
|
||||
# Individual absolute values
|
||||
r20 = extractor.extract_subcase('2')
|
||||
r40 = extractor.extract_subcase('3')
|
||||
r60 = extractor.extract_subcase('4')
|
||||
r90 = extractor.extract_subcase('1')
|
||||
|
||||
print()
|
||||
print('='*70)
|
||||
print('Individual absolute RMS values:')
|
||||
print('='*70)
|
||||
print(f' 20 deg: {r20["filtered_rms_nm"]:.2f} nm')
|
||||
print(f' 40 deg: {r40["filtered_rms_nm"]:.2f} nm')
|
||||
print(f' 60 deg: {r60["filtered_rms_nm"]:.2f} nm')
|
||||
print(f' 90 deg: {r90["filtered_rms_nm"]:.2f} nm')
|
||||
|
||||
print()
|
||||
print('='*70)
|
||||
print('V10 method (WRONG - difference of RMS values):')
|
||||
print(' Computes RMS_target - RMS_ref')
|
||||
print(' This is NOT the same as RMS of the difference!')
|
||||
print('='*70)
|
||||
print()
|
||||
print(f' 40-20: {r40["filtered_rms_nm"] - r20["filtered_rms_nm"]:.2f} nm')
|
||||
print(f' 60-20: {r60["filtered_rms_nm"] - r20["filtered_rms_nm"]:.2f} nm')
|
||||
print(f' After abs(): {abs(r40["filtered_rms_nm"] - r20["filtered_rms_nm"]):.2f} nm')
|
||||
print(f' After abs(): {abs(r60["filtered_rms_nm"] - r20["filtered_rms_nm"]):.2f} nm')
|
||||
|
||||
print()
|
||||
print('='*70)
|
||||
print('CONCLUSION')
|
||||
print('='*70)
|
||||
print()
|
||||
print('V10 BUG: Computes abs(RMS_target - RMS_ref) instead of RMS(WFE_target - WFE_ref)')
|
||||
print()
|
||||
print('The CORRECT relative WFE (from V9 method):')
|
||||
print(f' 40-20: {rel_40["relative_filtered_rms_nm"]:.2f} nm')
|
||||
print(f' 60-20: {rel_60["relative_filtered_rms_nm"]:.2f} nm')
|
||||
print(f' 90-20: {rel_90["relative_rms_filter_j1to3"]:.2f} nm')
|
||||
print()
|
||||
print('The WRONG values V10 reports:')
|
||||
print(f' 40-20: {abs(r40["filtered_rms_nm"] - r20["filtered_rms_nm"]):.2f} nm')
|
||||
print(f' 60-20: {abs(r60["filtered_rms_nm"] - r20["filtered_rms_nm"]):.2f} nm')
|
||||
print()
|
||||
print('V10 values are ~3-4x LOWER than correct values!')
|
||||
143
tests/audit_v10_wfe.py
Normal file
143
tests/audit_v10_wfe.py
Normal file
@@ -0,0 +1,143 @@
|
||||
"""Audit V10 WFE values - independent verification."""
|
||||
import sys
|
||||
from pathlib import Path
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent))
|
||||
|
||||
from optimization_engine.extractors import ZernikeOPDExtractor, ZernikeExtractor
|
||||
|
||||
print('='*70)
|
||||
print('AUDIT: V10 WFE Values - Independent Verification')
|
||||
print('='*70)
|
||||
|
||||
# V10 iter1 (baseline trial)
|
||||
op2_v10 = Path('studies/M1_Mirror/m1_mirror_cost_reduction_V10/2_iterations/iter1/assy_m1_assyfem1_sim1-solution_1.op2')
|
||||
|
||||
if not op2_v10.exists():
|
||||
print('V10 OP2 file not found!')
|
||||
sys.exit(1)
|
||||
|
||||
print(f'OP2 file: {op2_v10}')
|
||||
print(f'Size: {op2_v10.stat().st_size / 1024 / 1024:.1f} MB')
|
||||
|
||||
# Test with ZernikeOPDExtractor (what V10 uses)
|
||||
print()
|
||||
print('='*70)
|
||||
print('Method 1: ZernikeOPDExtractor (what V10 uses)')
|
||||
print('='*70)
|
||||
|
||||
extractor_opd = ZernikeOPDExtractor(op2_v10, n_modes=50, filter_orders=4)
|
||||
|
||||
result_20_opd = extractor_opd.extract_subcase('2') # Reference
|
||||
result_40_opd = extractor_opd.extract_subcase('3') # 40 deg
|
||||
result_60_opd = extractor_opd.extract_subcase('4') # 60 deg
|
||||
result_90_opd = extractor_opd.extract_subcase('1') # 90 deg MFG
|
||||
|
||||
print()
|
||||
print('ABSOLUTE values (ZernikeOPD):')
|
||||
print(f' 20 deg: filtered_rms = {result_20_opd["filtered_rms_nm"]:.2f} nm')
|
||||
print(f' 40 deg: filtered_rms = {result_40_opd["filtered_rms_nm"]:.2f} nm')
|
||||
print(f' 60 deg: filtered_rms = {result_60_opd["filtered_rms_nm"]:.2f} nm')
|
||||
print(f' 90 deg: filtered_rms = {result_90_opd["filtered_rms_nm"]:.2f} nm')
|
||||
|
||||
print()
|
||||
print('RELATIVE values (target - ref) as V10 computes:')
|
||||
rel_40_opd = result_40_opd['filtered_rms_nm'] - result_20_opd['filtered_rms_nm']
|
||||
rel_60_opd = result_60_opd['filtered_rms_nm'] - result_20_opd['filtered_rms_nm']
|
||||
rel_mfg_opd = result_90_opd['rms_filter_j1to3_nm'] - result_20_opd['rms_filter_j1to3_nm']
|
||||
print(f' 40-20: {rel_40_opd:.2f} nm (abs: {abs(rel_40_opd):.2f})')
|
||||
print(f' 60-20: {rel_60_opd:.2f} nm (abs: {abs(rel_60_opd):.2f})')
|
||||
print(f' 90-20 (j1to3): {rel_mfg_opd:.2f} nm (abs: {abs(rel_mfg_opd):.2f})')
|
||||
|
||||
print()
|
||||
print('V10 uses abs() -> stores:')
|
||||
print(f' rel_filtered_rms_40_vs_20: {abs(rel_40_opd):.2f}')
|
||||
print(f' rel_filtered_rms_60_vs_20: {abs(rel_60_opd):.2f}')
|
||||
print(f' mfg_90_optician_workload: {abs(rel_mfg_opd):.2f}')
|
||||
|
||||
# Test with Standard ZernikeExtractor (what V9 uses)
|
||||
print()
|
||||
print('='*70)
|
||||
print('Method 2: Standard ZernikeExtractor (what V9 likely uses)')
|
||||
print('='*70)
|
||||
|
||||
# Find the BDF file
|
||||
bdf_files = list(op2_v10.parent.glob('*.dat'))
|
||||
bdf_path = bdf_files[0] if bdf_files else None
|
||||
print(f'BDF file: {bdf_path}')
|
||||
|
||||
extractor_std = ZernikeExtractor(op2_v10, bdf_path=bdf_path, n_modes=50, filter_orders=4)
|
||||
|
||||
result_20_std = extractor_std.extract_subcase('2')
|
||||
result_40_std = extractor_std.extract_subcase('3')
|
||||
result_60_std = extractor_std.extract_subcase('4')
|
||||
result_90_std = extractor_std.extract_subcase('1')
|
||||
|
||||
print()
|
||||
print('ABSOLUTE values (Standard Z-only):')
|
||||
print(f' 20 deg: filtered_rms = {result_20_std["filtered_rms_nm"]:.2f} nm')
|
||||
print(f' 40 deg: filtered_rms = {result_40_std["filtered_rms_nm"]:.2f} nm')
|
||||
print(f' 60 deg: filtered_rms = {result_60_std["filtered_rms_nm"]:.2f} nm')
|
||||
print(f' 90 deg: filtered_rms = {result_90_std["filtered_rms_nm"]:.2f} nm')
|
||||
|
||||
print()
|
||||
print('RELATIVE values (Standard):')
|
||||
rel_40_std = result_40_std['filtered_rms_nm'] - result_20_std['filtered_rms_nm']
|
||||
rel_60_std = result_60_std['filtered_rms_nm'] - result_20_std['filtered_rms_nm']
|
||||
print(f' 40-20: {rel_40_std:.2f} nm (abs: {abs(rel_40_std):.2f})')
|
||||
print(f' 60-20: {rel_60_std:.2f} nm (abs: {abs(rel_60_std):.2f})')
|
||||
|
||||
# Compare
|
||||
print()
|
||||
print('='*70)
|
||||
print('COMPARISON: OPD vs Standard')
|
||||
print('='*70)
|
||||
print()
|
||||
print(f'40-20: OPD={abs(rel_40_opd):.2f} nm vs Standard={abs(rel_40_std):.2f} nm')
|
||||
print(f'60-20: OPD={abs(rel_60_opd):.2f} nm vs Standard={abs(rel_60_std):.2f} nm')
|
||||
|
||||
print()
|
||||
print('Lateral displacement (OPD method):')
|
||||
print(f' Max: {result_40_opd.get("max_lateral_displacement_um", 0):.3f} um')
|
||||
print(f' RMS: {result_40_opd.get("rms_lateral_displacement_um", 0):.3f} um')
|
||||
|
||||
# Now check what V9 reports
|
||||
print()
|
||||
print('='*70)
|
||||
print('V9 COMPARISON (iter12 from best archive)')
|
||||
print('='*70)
|
||||
|
||||
op2_v9 = Path('studies/M1_Mirror/m1_mirror_cost_reduction_V9/2_iterations/iter12/assy_m1_assyfem1_sim1-solution_1.op2')
|
||||
if op2_v9.exists():
|
||||
extractor_v9_opd = ZernikeOPDExtractor(op2_v9, n_modes=50, filter_orders=4)
|
||||
extractor_v9_std = ZernikeExtractor(op2_v9, n_modes=50, filter_orders=4)
|
||||
|
||||
r20_v9_opd = extractor_v9_opd.extract_subcase('2')
|
||||
r40_v9_opd = extractor_v9_opd.extract_subcase('3')
|
||||
r60_v9_opd = extractor_v9_opd.extract_subcase('4')
|
||||
|
||||
r20_v9_std = extractor_v9_std.extract_subcase('2')
|
||||
r40_v9_std = extractor_v9_std.extract_subcase('3')
|
||||
r60_v9_std = extractor_v9_std.extract_subcase('4')
|
||||
|
||||
rel_40_v9_opd = abs(r40_v9_opd['filtered_rms_nm'] - r20_v9_opd['filtered_rms_nm'])
|
||||
rel_60_v9_opd = abs(r60_v9_opd['filtered_rms_nm'] - r20_v9_opd['filtered_rms_nm'])
|
||||
rel_40_v9_std = abs(r40_v9_std['filtered_rms_nm'] - r20_v9_std['filtered_rms_nm'])
|
||||
rel_60_v9_std = abs(r60_v9_std['filtered_rms_nm'] - r20_v9_std['filtered_rms_nm'])
|
||||
|
||||
print()
|
||||
print('V9 iter12 relative values:')
|
||||
print(f' 40-20: OPD={rel_40_v9_opd:.2f} nm vs Standard={rel_40_v9_std:.2f} nm')
|
||||
print(f' 60-20: OPD={rel_60_v9_opd:.2f} nm vs Standard={rel_60_v9_std:.2f} nm')
|
||||
else:
|
||||
print('V9 OP2 not found')
|
||||
|
||||
print()
|
||||
print('='*70)
|
||||
print('SUMMARY')
|
||||
print('='*70)
|
||||
print()
|
||||
print('V10 reports: 40-20=1.99nm, 60-20=6.82nm (using OPD method)')
|
||||
print('V9 reports: 40-20=6.10nm, 60-20=12.76nm (likely Standard method)')
|
||||
print()
|
||||
print('If both studies have SIMILAR geometry, the OPD method should NOT')
|
||||
print('give such dramatically different values. This needs investigation.')
|
||||
20
tests/check_api_routes.py
Normal file
20
tests/check_api_routes.py
Normal file
@@ -0,0 +1,20 @@
|
||||
"""Check API routes from running backend."""
|
||||
import requests
|
||||
import json
|
||||
|
||||
# Get OpenAPI spec
|
||||
resp = requests.get("http://localhost:8000/openapi.json", timeout=10)
|
||||
spec = resp.json()
|
||||
|
||||
# Find insight routes
|
||||
print("Insight-related routes:")
|
||||
print("=" * 60)
|
||||
for path in sorted(spec.get("paths", {}).keys()):
|
||||
if "insight" in path.lower():
|
||||
print(f" {path}")
|
||||
|
||||
print()
|
||||
print("All routes:")
|
||||
print("-" * 60)
|
||||
for path in sorted(spec.get("paths", {}).keys()):
|
||||
print(f" {path}")
|
||||
83
tests/debug_figure_coords.py
Normal file
83
tests/debug_figure_coords.py
Normal file
@@ -0,0 +1,83 @@
|
||||
"""Debug script to compare figure.dat vs BDF node coordinates."""
|
||||
import sys
|
||||
from pathlib import Path
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent))
|
||||
|
||||
import numpy as np
|
||||
import logging
|
||||
logging.disable(logging.WARNING)
|
||||
|
||||
study_dir = Path(r"c:\Users\antoi\Atomizer\studies\M1_Mirror\m1_mirror_cost_reduction_V9")
|
||||
|
||||
# Load figure.dat
|
||||
from optimization_engine.extractors.extract_zernike_figure import load_figure_geometry
|
||||
fig_geo = load_figure_geometry(study_dir / "1_setup/model/figure.dat")
|
||||
fig_nids = set(fig_geo.keys())
|
||||
|
||||
# Find OP2 and BDF
|
||||
op2_file = list(study_dir.glob("3_results/best_design_archive/**/*.op2"))[0]
|
||||
bdf_file = op2_file.with_suffix(".dat")
|
||||
|
||||
# Load BDF
|
||||
from pyNastran.bdf.bdf import BDF
|
||||
bdf = BDF(log=None, debug=False)
|
||||
bdf.read_bdf(str(bdf_file))
|
||||
bdf_nids = set(bdf.nodes.keys())
|
||||
|
||||
# Load OP2
|
||||
from pyNastran.op2.op2 import OP2
|
||||
op2 = OP2(log=None, debug=False)
|
||||
op2.read_op2(str(op2_file))
|
||||
disps = op2.displacements
|
||||
first_key = list(disps.keys())[0]
|
||||
op2_nids = set(int(n) for n in disps[first_key].node_gridtype[:,0])
|
||||
|
||||
print(f"Figure.dat nodes: {len(fig_nids)}")
|
||||
print(f"BDF nodes: {len(bdf_nids)}")
|
||||
print(f"OP2 nodes: {len(op2_nids)}")
|
||||
|
||||
print()
|
||||
print(f"Figure ^ BDF: {len(fig_nids & bdf_nids)}")
|
||||
print(f"Figure ^ OP2: {len(fig_nids & op2_nids)}")
|
||||
print(f"BDF ^ OP2: {len(bdf_nids & op2_nids)}")
|
||||
|
||||
# Sample coords - use a node in all three
|
||||
common_nids = list(fig_nids & bdf_nids & op2_nids)[:5]
|
||||
print()
|
||||
print("Sample common node coords comparison:")
|
||||
z_diffs = []
|
||||
for nid in common_nids:
|
||||
fig_pos = fig_geo[nid]
|
||||
bdf_pos = bdf.nodes[nid].get_position()
|
||||
diff = np.array(fig_pos) - bdf_pos
|
||||
z_diffs.append(diff[2])
|
||||
print(f" Node {nid}:")
|
||||
print(f" Figure: ({fig_pos[0]:.6f}, {fig_pos[1]:.6f}, {fig_pos[2]:.9f})")
|
||||
print(f" BDF: ({bdf_pos[0]:.6f}, {bdf_pos[1]:.6f}, {bdf_pos[2]:.9f})")
|
||||
print(f" Z diff: {diff[2]*1e6:.3f} nm")
|
||||
|
||||
# Statistics on all matching nodes
|
||||
all_common = fig_nids & bdf_nids
|
||||
all_z_diffs = []
|
||||
all_xy_diffs = []
|
||||
for nid in all_common:
|
||||
fig_pos = np.array(fig_geo[nid])
|
||||
bdf_pos = bdf.nodes[nid].get_position()
|
||||
diff = fig_pos - bdf_pos
|
||||
all_z_diffs.append(diff[2])
|
||||
all_xy_diffs.append(np.sqrt(diff[0]**2 + diff[1]**2))
|
||||
|
||||
all_z_diffs = np.array(all_z_diffs)
|
||||
all_xy_diffs = np.array(all_xy_diffs)
|
||||
|
||||
print()
|
||||
print(f"=== ALL {len(all_common)} COMMON NODES ===")
|
||||
print(f"Z difference (figure - BDF):")
|
||||
print(f" Min: {all_z_diffs.min()*1e6:.3f} nm")
|
||||
print(f" Max: {all_z_diffs.max()*1e6:.3f} nm")
|
||||
print(f" Mean: {all_z_diffs.mean()*1e6:.3f} nm")
|
||||
print(f" RMS: {np.sqrt(np.mean(all_z_diffs**2))*1e6:.3f} nm")
|
||||
print()
|
||||
print(f"XY difference (figure - BDF):")
|
||||
print(f" Max: {all_xy_diffs.max()*1e3:.6f} um")
|
||||
print(f" RMS: {np.sqrt(np.mean(all_xy_diffs**2))*1e3:.6f} um")
|
||||
50
tests/debug_insights.py
Normal file
50
tests/debug_insights.py
Normal file
@@ -0,0 +1,50 @@
|
||||
"""Debug insights availability for a study."""
|
||||
import sys
|
||||
sys.path.insert(0, ".")
|
||||
from pathlib import Path
|
||||
|
||||
# Test study path resolution
|
||||
study_id = 'm1_mirror_cost_reduction_V9'
|
||||
STUDIES_DIR = Path('studies')
|
||||
|
||||
# Check nested path
|
||||
for topic_dir in STUDIES_DIR.iterdir():
|
||||
if topic_dir.is_dir():
|
||||
study_dir = topic_dir / study_id
|
||||
if study_dir.exists():
|
||||
print(f"Found study at: {study_dir}")
|
||||
print(f"Has 1_setup: {(study_dir / '1_setup').exists()}")
|
||||
print(f"Has 2_results: {(study_dir / '2_results').exists()}")
|
||||
|
||||
# Check what insights are available
|
||||
from optimization_engine.insights import list_available_insights, get_configured_insights, recommend_insights_for_study
|
||||
|
||||
print("\n--- Available insights (can_generate=True) ---")
|
||||
available = list_available_insights(study_dir)
|
||||
print(f"Count: {len(available)}")
|
||||
for a in available:
|
||||
print(f" - {a}")
|
||||
|
||||
print("\n--- Configured insights ---")
|
||||
configured = get_configured_insights(study_dir)
|
||||
print(f"Count: {len(configured)}")
|
||||
for c in configured:
|
||||
print(f" - {c.type}: {c.name}")
|
||||
|
||||
print("\n--- Recommendations ---")
|
||||
recs = recommend_insights_for_study(study_dir)
|
||||
print(f"Count: {len(recs)}")
|
||||
for r in recs:
|
||||
print(f" - {r['type']}: {r['name']}")
|
||||
|
||||
# Test individual insight can_generate
|
||||
print("\n--- Testing each insight's can_generate ---")
|
||||
from optimization_engine.insights import get_insight, list_insights
|
||||
|
||||
for info in list_insights():
|
||||
insight = get_insight(info['type'], study_dir)
|
||||
if insight:
|
||||
can = insight.can_generate()
|
||||
print(f" {info['type']:20} can_generate={can}")
|
||||
|
||||
break
|
||||
21
tests/test_insights_import.py
Normal file
21
tests/test_insights_import.py
Normal file
@@ -0,0 +1,21 @@
|
||||
"""Test if insights can be imported from backend context."""
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
# Replicate the path setup from main.py
|
||||
backend_path = Path(__file__).parent.parent / "atomizer-dashboard" / "backend" / "api"
|
||||
sys.path.insert(0, str(backend_path.parent.parent.parent.parent))
|
||||
sys.path.insert(0, str(backend_path.parent))
|
||||
|
||||
print(f"sys.path[0]: {sys.path[0]}")
|
||||
print(f"sys.path[1]: {sys.path[1]}")
|
||||
|
||||
try:
|
||||
from api.routes import insights
|
||||
print(f"insights module imported: {insights}")
|
||||
print(f"insights.router: {insights.router}")
|
||||
print(f"routes: {[r.path for r in insights.router.routes]}")
|
||||
except Exception as e:
|
||||
print(f"ERROR importing insights: {e}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
31
tests/test_zernike_insight.py
Normal file
31
tests/test_zernike_insight.py
Normal file
@@ -0,0 +1,31 @@
|
||||
"""Test script for Zernike WFE insight with OPD method."""
|
||||
import sys
|
||||
from pathlib import Path
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent))
|
||||
|
||||
from optimization_engine.insights.zernike_wfe import ZernikeWFEInsight
|
||||
from optimization_engine.insights.base import InsightConfig
|
||||
|
||||
study = Path('studies/M1_Mirror/m1_mirror_cost_reduction_V9')
|
||||
insight = ZernikeWFEInsight(study)
|
||||
|
||||
if insight.can_generate():
|
||||
print('Insight can be generated!')
|
||||
print(f'OP2: {insight.op2_path}')
|
||||
print(f'Geo: {insight.geo_path}')
|
||||
config = InsightConfig()
|
||||
result = insight.generate(config)
|
||||
if result.success:
|
||||
n_files = len(result.summary.get('html_files', []))
|
||||
print(f'Success! Generated {n_files} files')
|
||||
for f in result.summary.get('html_files', []):
|
||||
print(f' - {Path(f).name}')
|
||||
print()
|
||||
print('Summary:')
|
||||
for k, v in result.summary.items():
|
||||
if k != 'html_files':
|
||||
print(f' {k}: {v}')
|
||||
else:
|
||||
print(f'Failed: {result.error}')
|
||||
else:
|
||||
print('Cannot generate insight')
|
||||
95
tests/test_zernike_opd_comparison.py
Normal file
95
tests/test_zernike_opd_comparison.py
Normal file
@@ -0,0 +1,95 @@
|
||||
#!/usr/bin/env python
|
||||
"""
|
||||
Quick test script to compare Standard vs OPD Zernike methods.
|
||||
|
||||
Usage:
|
||||
conda activate atomizer
|
||||
python test_zernike_opd_comparison.py
|
||||
|
||||
This will analyze a recent OP2 file and show you:
|
||||
1. How much lateral displacement exists
|
||||
2. How different the WFE metrics are between methods
|
||||
3. Whether you need to switch to OPD method for your optimizations
|
||||
"""
|
||||
|
||||
from pathlib import Path
|
||||
import sys
|
||||
|
||||
# Add project root to path
|
||||
sys.path.insert(0, str(Path(__file__).parent))
|
||||
|
||||
|
||||
def main():
|
||||
import numpy as np
|
||||
from optimization_engine.extractors.extract_zernike_opd import (
|
||||
ZernikeOPDExtractor,
|
||||
)
|
||||
|
||||
# Find a recent OP2 file from your studies
|
||||
studies_path = Path("studies/M1_Mirror")
|
||||
|
||||
op2_files = list(studies_path.glob("**/2_iterations/**/*.op2"))
|
||||
if not op2_files:
|
||||
op2_files = list(studies_path.glob("**/*.op2"))
|
||||
|
||||
if not op2_files:
|
||||
print("No OP2 files found in studies/M1_Mirror")
|
||||
return
|
||||
|
||||
# Use the most recent one
|
||||
op2_file = max(op2_files, key=lambda p: p.stat().st_mtime)
|
||||
print(f"Analyzing: {op2_file}")
|
||||
print("=" * 80)
|
||||
|
||||
# Run comparison
|
||||
try:
|
||||
extractor = ZernikeOPDExtractor(op2_file)
|
||||
|
||||
print(f"\nAvailable subcases: {list(extractor.displacements.keys())}")
|
||||
|
||||
# Show geometry info
|
||||
geo = extractor.node_geometry
|
||||
all_pos = np.array(list(geo.values()))
|
||||
print(f"\n--- Geometry Info ---")
|
||||
print(f" Nodes: {len(geo)}")
|
||||
print(f" X range: {all_pos[:,0].min():.1f} to {all_pos[:,0].max():.1f} mm")
|
||||
print(f" Y range: {all_pos[:,1].min():.1f} to {all_pos[:,1].max():.1f} mm")
|
||||
print(f" Z range: {all_pos[:,2].min():.1f} to {all_pos[:,2].max():.1f} mm")
|
||||
|
||||
for label in extractor.displacements.keys():
|
||||
print(f"\n{'=' * 80}")
|
||||
print(f"SUBCASE {label}")
|
||||
print('=' * 80)
|
||||
|
||||
comparison = extractor.extract_comparison(label)
|
||||
|
||||
print(f"\n--- Standard Method (Z-only) ---")
|
||||
print(f" Global RMS: {comparison['standard_method']['global_rms_nm']:.2f} nm")
|
||||
print(f" Filtered RMS: {comparison['standard_method']['filtered_rms_nm']:.2f} nm")
|
||||
|
||||
print(f"\n--- Rigorous OPD Method ---")
|
||||
print(f" Global RMS: {comparison['opd_method']['global_rms_nm']:.2f} nm")
|
||||
print(f" Filtered RMS: {comparison['opd_method']['filtered_rms_nm']:.2f} nm")
|
||||
|
||||
print(f"\n--- Difference (OPD - Standard) ---")
|
||||
delta = comparison['delta']['filtered_rms_nm']
|
||||
pct = comparison['delta']['percent_difference_filtered']
|
||||
sign = "+" if delta > 0 else ""
|
||||
print(f" Filtered RMS: {sign}{delta:.2f} nm ({sign}{pct:.1f}%)")
|
||||
|
||||
print(f"\n--- Lateral Displacement ---")
|
||||
print(f" Max: {comparison['lateral_displacement']['max_um']:.3f} µm")
|
||||
print(f" RMS: {comparison['lateral_displacement']['rms_um']:.3f} µm")
|
||||
print(f" P99: {comparison['lateral_displacement']['p99_um']:.3f} µm")
|
||||
|
||||
print(f"\n>>> {comparison['recommendation']}")
|
||||
|
||||
except Exception as e:
|
||||
print(f"Error: {e}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
return
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
199
tests/test_zernike_opd_with_prescription.py
Normal file
199
tests/test_zernike_opd_with_prescription.py
Normal file
@@ -0,0 +1,199 @@
|
||||
#!/usr/bin/env python
|
||||
"""
|
||||
Test ZernikeOPDExtractor with validated M1 Mirror optical prescription.
|
||||
|
||||
Compares:
|
||||
1. Standard Zernike (Z-displacement only at original x,y)
|
||||
2. OPD Zernike with auto-estimated focal length
|
||||
3. OPD Zernike with correct focal length (1445 mm from prescription)
|
||||
|
||||
M1 Mirror Optical Prescription:
|
||||
- Radius of Curvature: 2890 ± 3 mm
|
||||
- Conic Constant: -0.987 ± 0.001 (near-parabolic)
|
||||
- Clear Aperture: 1202 mm
|
||||
- Central Bore: 271.56 mm
|
||||
- Focal Length: 1445 mm (R/2)
|
||||
"""
|
||||
|
||||
from pathlib import Path
|
||||
import sys
|
||||
|
||||
# Add project root to path
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent))
|
||||
|
||||
import numpy as np
|
||||
|
||||
|
||||
def run_comparison(op2_path: Path):
|
||||
"""Run comparison between standard and OPD Zernike methods."""
|
||||
from optimization_engine.extractors.extract_zernike_opd import ZernikeOPDExtractor
|
||||
from optimization_engine.extractors.extract_zernike_wfe import ZernikeExtractor
|
||||
|
||||
print("=" * 70)
|
||||
print("ZERNIKE METHOD COMPARISON WITH VALIDATED PRESCRIPTION")
|
||||
print("=" * 70)
|
||||
print(f"\nOP2 file: {op2_path.name}")
|
||||
print(f"Optical prescription focal length: 1445 mm")
|
||||
print()
|
||||
|
||||
# 1. Standard Zernike (Z-displacement only)
|
||||
print("1. STANDARD ZERNIKE (Z-displacement at original x,y)")
|
||||
print("-" * 50)
|
||||
try:
|
||||
std_extractor = ZernikeExtractor(op2_path)
|
||||
std_results = std_extractor.extract_all_subcases()
|
||||
|
||||
for sc, data in std_results.items():
|
||||
coeffs = data['coefficients']
|
||||
rms = data['rms_wfe_nm']
|
||||
print(f" Subcase {sc}: RMS WFE = {rms:.2f} nm")
|
||||
except Exception as e:
|
||||
print(f" Error: {e}")
|
||||
std_results = None
|
||||
print()
|
||||
|
||||
# 2. OPD Zernike with auto-estimated focal length
|
||||
print("2. OPD ZERNIKE (auto-estimated focal length)")
|
||||
print("-" * 50)
|
||||
try:
|
||||
opd_auto = ZernikeOPDExtractor(op2_path, concave=True)
|
||||
auto_focal = opd_auto.focal_length
|
||||
print(f" Auto-estimated focal length: {auto_focal:.1f} mm")
|
||||
|
||||
opd_auto_results = opd_auto.extract_all_subcases()
|
||||
for sc, data in opd_auto_results.items():
|
||||
rms = data['rms_wfe_nm']
|
||||
lat = data.get('max_lateral_displacement_um', 0)
|
||||
print(f" Subcase {sc}: RMS WFE = {rms:.2f} nm, Max lateral = {lat:.2f} µm")
|
||||
except Exception as e:
|
||||
print(f" Error: {e}")
|
||||
opd_auto_results = None
|
||||
print()
|
||||
|
||||
# 3. OPD Zernike with correct prescription focal length
|
||||
print("3. OPD ZERNIKE (prescription focal length = 1445 mm)")
|
||||
print("-" * 50)
|
||||
try:
|
||||
opd_correct = ZernikeOPDExtractor(op2_path, focal_length=1445.0, concave=True)
|
||||
print(f" Using focal length: {opd_correct.focal_length:.1f} mm")
|
||||
|
||||
opd_correct_results = opd_correct.extract_all_subcases()
|
||||
for sc, data in opd_correct_results.items():
|
||||
rms = data['rms_wfe_nm']
|
||||
lat = data.get('max_lateral_displacement_um', 0)
|
||||
print(f" Subcase {sc}: RMS WFE = {rms:.2f} nm, Max lateral = {lat:.2f} µm")
|
||||
except Exception as e:
|
||||
print(f" Error: {e}")
|
||||
opd_correct_results = None
|
||||
print()
|
||||
|
||||
# 4. Comparison summary
|
||||
if std_results and opd_correct_results:
|
||||
print("=" * 70)
|
||||
print("COMPARISON SUMMARY")
|
||||
print("=" * 70)
|
||||
print()
|
||||
print(f"{'Subcase':<10} {'Standard':<15} {'OPD (auto)':<15} {'OPD (1445mm)':<15} {'Diff %':<10}")
|
||||
print("-" * 65)
|
||||
|
||||
for sc in std_results.keys():
|
||||
std_rms = std_results[sc]['rms_wfe_nm']
|
||||
auto_rms = opd_auto_results[sc]['rms_wfe_nm'] if opd_auto_results else 0
|
||||
corr_rms = opd_correct_results[sc]['rms_wfe_nm']
|
||||
diff_pct = ((corr_rms - std_rms) / std_rms * 100) if std_rms > 0 else 0
|
||||
|
||||
print(f"{sc:<10} {std_rms:<15.2f} {auto_rms:<15.2f} {corr_rms:<15.2f} {diff_pct:>+8.1f}%")
|
||||
|
||||
print()
|
||||
print("LATERAL DISPLACEMENT ANALYSIS")
|
||||
print("-" * 50)
|
||||
|
||||
for sc, data in opd_correct_results.items():
|
||||
lat = data.get('max_lateral_displacement_um', 0)
|
||||
severity = "CRITICAL - OPD method required" if lat > 10 else "Low - standard OK" if lat < 1 else "Moderate"
|
||||
print(f" Subcase {sc}: Max lateral = {lat:.2f} µm ({severity})")
|
||||
|
||||
print()
|
||||
|
||||
# Tracking WFE comparison (40-20 and 60-20)
|
||||
if 2 in opd_correct_results and 3 in opd_correct_results and 4 in opd_correct_results:
|
||||
print("TRACKING WFE (differential between elevations)")
|
||||
print("-" * 50)
|
||||
|
||||
# Get coefficients for differential analysis
|
||||
z20 = np.array(opd_correct_results[2]['coefficients'])
|
||||
z40 = np.array(opd_correct_results[3]['coefficients'])
|
||||
z60 = np.array(opd_correct_results[4]['coefficients'])
|
||||
|
||||
# Differential (remove J1-J4: piston, tip, tilt, defocus)
|
||||
diff_40_20 = z40 - z20
|
||||
diff_60_20 = z60 - z20
|
||||
|
||||
# RMS of filtered differential (J5+)
|
||||
rms_40_20 = np.sqrt(np.sum(diff_40_20[4:]**2)) # Skip J1-J4
|
||||
rms_60_20 = np.sqrt(np.sum(diff_60_20[4:]**2))
|
||||
|
||||
print(f" 40°-20° tracking WFE: {rms_40_20:.2f} nm RMS (filtered)")
|
||||
print(f" 60°-20° tracking WFE: {rms_60_20:.2f} nm RMS (filtered)")
|
||||
|
||||
print()
|
||||
print(" Standard method comparison:")
|
||||
z20_std = np.array(std_results[2]['coefficients'])
|
||||
z40_std = np.array(std_results[3]['coefficients'])
|
||||
z60_std = np.array(std_results[4]['coefficients'])
|
||||
|
||||
diff_40_20_std = z40_std - z20_std
|
||||
diff_60_20_std = z60_std - z20_std
|
||||
|
||||
rms_40_20_std = np.sqrt(np.sum(diff_40_20_std[4:]**2))
|
||||
rms_60_20_std = np.sqrt(np.sum(diff_60_20_std[4:]**2))
|
||||
|
||||
print(f" 40°-20° tracking WFE (std): {rms_40_20_std:.2f} nm RMS")
|
||||
print(f" 60°-20° tracking WFE (std): {rms_60_20_std:.2f} nm RMS")
|
||||
|
||||
print()
|
||||
print(f" Difference (OPD vs Standard):")
|
||||
print(f" 40°-20°: {rms_40_20 - rms_40_20_std:+.2f} nm ({(rms_40_20/rms_40_20_std - 1)*100:+.1f}%)")
|
||||
print(f" 60°-20°: {rms_60_20 - rms_60_20_std:+.2f} nm ({(rms_60_20/rms_60_20_std - 1)*100:+.1f}%)")
|
||||
|
||||
print()
|
||||
print("=" * 70)
|
||||
|
||||
|
||||
def main():
|
||||
import argparse
|
||||
|
||||
parser = argparse.ArgumentParser(description='Test ZernikeOPD with M1 prescription')
|
||||
parser.add_argument('path', nargs='?', default='.',
|
||||
help='Path to OP2 file or study directory')
|
||||
|
||||
args = parser.parse_args()
|
||||
path = Path(args.path).resolve()
|
||||
|
||||
# Find OP2 file
|
||||
if path.is_file() and path.suffix.lower() == '.op2':
|
||||
op2_path = path
|
||||
elif path.is_dir():
|
||||
# Look for best design or recent iteration
|
||||
patterns = [
|
||||
'3_results/best_design_archive/**/*.op2',
|
||||
'2_iterations/iter1/*.op2',
|
||||
'**/*.op2'
|
||||
]
|
||||
for pattern in patterns:
|
||||
files = list(path.glob(pattern))
|
||||
if files:
|
||||
op2_path = max(files, key=lambda p: p.stat().st_mtime)
|
||||
break
|
||||
else:
|
||||
print(f"No OP2 file found in {path}")
|
||||
sys.exit(1)
|
||||
else:
|
||||
print(f"Invalid path: {path}")
|
||||
sys.exit(1)
|
||||
|
||||
run_comparison(op2_path)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
388
tools/create_pareto_graphs.py
Normal file
388
tools/create_pareto_graphs.py
Normal file
@@ -0,0 +1,388 @@
|
||||
"""
|
||||
Create Pareto front visualizations for M1 Mirror optimization data.
|
||||
Shows relationship between geometric parameters and 60/20 WFE performance.
|
||||
"""
|
||||
|
||||
import pandas as pd
|
||||
import numpy as np
|
||||
import matplotlib.pyplot as plt
|
||||
from matplotlib.colors import LinearSegmentedColormap
|
||||
import matplotlib.patches as mpatches
|
||||
|
||||
# Set style for publication-quality plots
|
||||
plt.rcParams.update({
|
||||
'font.family': 'sans-serif',
|
||||
'font.sans-serif': ['Arial', 'Helvetica', 'DejaVu Sans'],
|
||||
'font.size': 11,
|
||||
'axes.titlesize': 16,
|
||||
'axes.labelsize': 13,
|
||||
'xtick.labelsize': 11,
|
||||
'ytick.labelsize': 11,
|
||||
'legend.fontsize': 10,
|
||||
'figure.dpi': 150,
|
||||
'savefig.dpi': 150,
|
||||
'axes.spines.top': False,
|
||||
'axes.spines.right': False,
|
||||
})
|
||||
|
||||
# Load data
|
||||
df = pd.read_csv(r'c:\Users\antoi\Atomizer\studies\m1_mirror_all_trials_export.csv')
|
||||
|
||||
print("=== Data Overview ===")
|
||||
print(f"Total rows: {len(df)}")
|
||||
print(f"\nColumn data availability:")
|
||||
for col in df.columns:
|
||||
non_null = df[col].notna().sum()
|
||||
if non_null > 0:
|
||||
print(f" {col}: {non_null} ({100*non_null/len(df):.1f}%)")
|
||||
|
||||
print(f"\nStudies: {df['study'].unique()}")
|
||||
|
||||
# Filter for rows with the key parameters
|
||||
thickness_col = 'center_thickness'
|
||||
angle_col = 'blank_backface_angle'
|
||||
wfe_col = 'rel_filtered_rms_60_vs_20'
|
||||
|
||||
print(f"\n=== Key columns ===")
|
||||
print(f"center_thickness non-null: {df[thickness_col].notna().sum()}")
|
||||
print(f"blank_backface_angle non-null: {df[angle_col].notna().sum()}")
|
||||
print(f"rel_filtered_rms_60_vs_20 non-null: {df[wfe_col].notna().sum()}")
|
||||
|
||||
# Create filtered dataset with valid WFE values
|
||||
df_valid = df[df[wfe_col].notna()].copy()
|
||||
print(f"\nRows with valid WFE data (before outlier removal): {len(df_valid)}")
|
||||
|
||||
if len(df_valid) == 0:
|
||||
print("No valid WFE data found!")
|
||||
exit()
|
||||
|
||||
# Remove outliers - WFE values above 1000 are clearly failed simulations
|
||||
WFE_THRESHOLD = 100 # Reasonable upper bound for WFE ratio
|
||||
df_valid = df_valid[df_valid[wfe_col] < WFE_THRESHOLD].copy()
|
||||
print(f"Rows with valid WFE data (after outlier removal, WFE < {WFE_THRESHOLD}): {len(df_valid)}")
|
||||
|
||||
# Show ranges
|
||||
print(f"\n=== Value ranges (clean data) ===")
|
||||
if df_valid[thickness_col].notna().any():
|
||||
print(f"center_thickness: {df_valid[thickness_col].min():.2f} - {df_valid[thickness_col].max():.2f} mm")
|
||||
if df_valid[angle_col].notna().any():
|
||||
print(f"blank_backface_angle: {df_valid[angle_col].min():.2f} - {df_valid[angle_col].max():.2f}°")
|
||||
print(f"rel_filtered_rms_60_vs_20: {df_valid[wfe_col].min():.4f} - {df_valid[wfe_col].max():.4f}")
|
||||
|
||||
# Also check mass
|
||||
if 'mass_kg' in df_valid.columns and df_valid['mass_kg'].notna().any():
|
||||
print(f"mass_kg: {df_valid['mass_kg'].min():.2f} - {df_valid['mass_kg'].max():.2f} kg")
|
||||
|
||||
|
||||
def compute_pareto_front(x, y, minimize_x=True, minimize_y=True):
|
||||
"""
|
||||
Compute Pareto front indices.
|
||||
Returns indices of points on the Pareto front.
|
||||
"""
|
||||
# Create array of points
|
||||
points = np.column_stack([x, y])
|
||||
n_points = len(points)
|
||||
|
||||
# Adjust for minimization/maximization
|
||||
if not minimize_x:
|
||||
points[:, 0] = -points[:, 0]
|
||||
if not minimize_y:
|
||||
points[:, 1] = -points[:, 1]
|
||||
|
||||
# Find Pareto front
|
||||
pareto_mask = np.ones(n_points, dtype=bool)
|
||||
|
||||
for i in range(n_points):
|
||||
if pareto_mask[i]:
|
||||
# Check if any other point dominates point i
|
||||
for j in range(n_points):
|
||||
if i != j and pareto_mask[j]:
|
||||
# j dominates i if j is <= in all objectives and < in at least one
|
||||
if (points[j, 0] <= points[i, 0] and points[j, 1] <= points[i, 1] and
|
||||
(points[j, 0] < points[i, 0] or points[j, 1] < points[i, 1])):
|
||||
pareto_mask[i] = False
|
||||
break
|
||||
|
||||
return np.where(pareto_mask)[0]
|
||||
|
||||
|
||||
def create_pareto_plot(df_plot, x_col, y_col, x_label, y_label, title, filename,
|
||||
minimize_x=True, minimize_y=True, color_by=None, color_label=None):
|
||||
"""Create a publication-quality Pareto front plot."""
|
||||
|
||||
# Filter valid data
|
||||
mask = df_plot[x_col].notna() & df_plot[y_col].notna()
|
||||
df_clean = df_plot[mask].copy()
|
||||
|
||||
if len(df_clean) < 2:
|
||||
print(f"Not enough data for {title}")
|
||||
return
|
||||
|
||||
x = df_clean[x_col].values
|
||||
y = df_clean[y_col].values
|
||||
|
||||
# Compute Pareto front
|
||||
pareto_idx = compute_pareto_front(x, y, minimize_x, minimize_y)
|
||||
|
||||
# Sort Pareto points by x for line drawing
|
||||
pareto_points = np.column_stack([x[pareto_idx], y[pareto_idx]])
|
||||
sort_idx = np.argsort(pareto_points[:, 0])
|
||||
pareto_sorted = pareto_points[sort_idx]
|
||||
|
||||
# Create figure with professional styling
|
||||
fig, ax = plt.subplots(figsize=(12, 8))
|
||||
fig.patch.set_facecolor('white')
|
||||
|
||||
# Professional color palette
|
||||
bg_color = '#f8f9fa'
|
||||
grid_color = '#dee2e6'
|
||||
point_color = '#6c757d'
|
||||
pareto_color = '#dc3545'
|
||||
pareto_fill = '#ffc107'
|
||||
|
||||
ax.set_facecolor(bg_color)
|
||||
|
||||
# Color scheme - use mass as color if available
|
||||
if color_by is not None and color_by in df_clean.columns and df_clean[color_by].notna().sum() > 10:
|
||||
# Only use color if we have enough colored points
|
||||
color_mask = df_clean[color_by].notna()
|
||||
colors = df_clean.loc[color_mask, color_by].values
|
||||
|
||||
# Plot non-colored points in gray
|
||||
ax.scatter(x[~color_mask.values], y[~color_mask.values],
|
||||
c=point_color, alpha=0.3, s=40,
|
||||
edgecolors='white', linewidth=0.3, zorder=2)
|
||||
|
||||
# Plot colored points
|
||||
scatter = ax.scatter(x[color_mask.values], y[color_mask.values],
|
||||
c=colors, cmap='plasma', alpha=0.7, s=60,
|
||||
edgecolors='white', linewidth=0.5, zorder=2)
|
||||
cbar = plt.colorbar(scatter, ax=ax, pad=0.02, shrink=0.8)
|
||||
cbar.set_label(color_label or color_by, fontsize=12, fontweight='bold')
|
||||
cbar.ax.tick_params(labelsize=10)
|
||||
else:
|
||||
ax.scatter(x, y, c=point_color, alpha=0.4, s=50,
|
||||
edgecolors='white', linewidth=0.3, zorder=2, label='Design candidates')
|
||||
|
||||
# Draw Pareto front fill area (visual emphasis)
|
||||
if len(pareto_sorted) > 1:
|
||||
# Smooth interpolation for the Pareto front line
|
||||
from scipy.interpolate import interp1d
|
||||
if len(pareto_sorted) >= 4:
|
||||
# Use cubic interpolation for smooth curve
|
||||
try:
|
||||
f = interp1d(pareto_sorted[:, 0], pareto_sorted[:, 1], kind='cubic')
|
||||
x_smooth = np.linspace(pareto_sorted[:, 0].min(), pareto_sorted[:, 0].max(), 100)
|
||||
y_smooth = f(x_smooth)
|
||||
ax.plot(x_smooth, y_smooth, color=pareto_color, linewidth=3, alpha=0.9, zorder=3)
|
||||
except:
|
||||
ax.plot(pareto_sorted[:, 0], pareto_sorted[:, 1], color=pareto_color,
|
||||
linewidth=3, alpha=0.9, zorder=3)
|
||||
else:
|
||||
ax.plot(pareto_sorted[:, 0], pareto_sorted[:, 1], color=pareto_color,
|
||||
linewidth=3, alpha=0.9, zorder=3)
|
||||
|
||||
# Plot Pareto front points with emphasis
|
||||
ax.scatter(x[pareto_idx], y[pareto_idx], c=pareto_fill, s=180,
|
||||
edgecolors=pareto_color, linewidth=2.5, zorder=5,
|
||||
label=f'Pareto optimal ({len(pareto_idx)} designs)')
|
||||
|
||||
# Styling
|
||||
ax.set_xlabel(x_label, fontsize=14, fontweight='bold', labelpad=12)
|
||||
ax.set_ylabel(y_label, fontsize=14, fontweight='bold', labelpad=12)
|
||||
ax.set_title(title, fontsize=18, fontweight='bold', pad=20, color='#212529')
|
||||
|
||||
# Refined grid
|
||||
ax.grid(True, alpha=0.5, linestyle='-', linewidth=0.5, color=grid_color)
|
||||
ax.set_axisbelow(True)
|
||||
|
||||
# Add minor grid
|
||||
ax.minorticks_on()
|
||||
ax.grid(True, which='minor', alpha=0.2, linestyle=':', linewidth=0.3, color=grid_color)
|
||||
|
||||
# Legend with professional styling
|
||||
legend = ax.legend(loc='upper right', fontsize=11, framealpha=0.95,
|
||||
edgecolor=grid_color, fancybox=True, shadow=True)
|
||||
|
||||
# Add annotation for best point
|
||||
if minimize_y:
|
||||
best_idx = pareto_idx[np.argmin(y[pareto_idx])]
|
||||
else:
|
||||
best_idx = pareto_idx[np.argmax(y[pareto_idx])]
|
||||
|
||||
# Professional annotation box - position dynamically based on data location
|
||||
# Determine best quadrant for annotation
|
||||
x_range = x.max() - x.min()
|
||||
y_range = y.max() - y.min()
|
||||
x_mid = x.min() + x_range / 2
|
||||
y_mid = y.min() + y_range / 2
|
||||
|
||||
# Place annotation away from the best point
|
||||
if x[best_idx] < x_mid:
|
||||
x_offset = 50
|
||||
else:
|
||||
x_offset = -120
|
||||
if y[best_idx] < y_mid:
|
||||
y_offset = 50
|
||||
else:
|
||||
y_offset = -60
|
||||
|
||||
ax.annotate(f'Best WFE: {y[best_idx]:.2f}\n{x_label.split()[0]}: {x[best_idx]:.1f}',
|
||||
xy=(x[best_idx], y[best_idx]),
|
||||
xytext=(x_offset, y_offset), textcoords='offset points',
|
||||
fontsize=11, fontweight='bold',
|
||||
bbox=dict(boxstyle='round,pad=0.6', facecolor='white',
|
||||
edgecolor=pareto_color, linewidth=2, alpha=0.95),
|
||||
arrowprops=dict(arrowstyle='->', connectionstyle='arc3,rad=0.2',
|
||||
color=pareto_color, lw=2))
|
||||
|
||||
# Statistics box in bottom left
|
||||
stats_text = f'Total designs explored: {len(df_clean):,}\nPareto optimal: {len(pareto_idx)}'
|
||||
ax.text(0.02, 0.02, stats_text, transform=ax.transAxes, fontsize=10,
|
||||
verticalalignment='bottom',
|
||||
bbox=dict(boxstyle='round,pad=0.5', facecolor='white',
|
||||
edgecolor=grid_color, alpha=0.9))
|
||||
|
||||
# Adjust spines
|
||||
for spine in ax.spines.values():
|
||||
spine.set_color(grid_color)
|
||||
spine.set_linewidth(1.5)
|
||||
|
||||
plt.tight_layout()
|
||||
plt.savefig(filename, dpi=200, bbox_inches='tight', facecolor='white',
|
||||
edgecolor='none', pad_inches=0.2)
|
||||
plt.close()
|
||||
print(f"Saved: {filename}")
|
||||
|
||||
return pareto_sorted, pareto_idx
|
||||
|
||||
|
||||
# Create plots
|
||||
output_dir = r'c:\Users\antoi\Atomizer\studies'
|
||||
|
||||
# 1. Blank Thickness vs 60/20 WFE
|
||||
print("\n--- Creating Blank Thickness vs WFE plot ---")
|
||||
if df_valid[thickness_col].notna().any():
|
||||
result = create_pareto_plot(
|
||||
df_valid,
|
||||
x_col=thickness_col,
|
||||
y_col=wfe_col,
|
||||
x_label='Blank Thickness (mm)',
|
||||
y_label='60/20 WFE (Relative RMS)',
|
||||
title='M1 Mirror Optimization\nBlank Thickness vs Wavefront Error',
|
||||
filename=f'{output_dir}\\pareto_thickness_vs_wfe.png',
|
||||
minimize_x=False, # Thinner may be desirable
|
||||
minimize_y=True, # Lower WFE is better
|
||||
color_by='mass_kg' if 'mass_kg' in df_valid.columns else None,
|
||||
color_label='Mass (kg)'
|
||||
)
|
||||
else:
|
||||
print("No thickness data available")
|
||||
|
||||
# 2. Blank Backface Angle vs 60/20 WFE
|
||||
print("\n--- Creating Backface Angle vs WFE plot ---")
|
||||
if df_valid[angle_col].notna().any():
|
||||
result = create_pareto_plot(
|
||||
df_valid,
|
||||
x_col=angle_col,
|
||||
y_col=wfe_col,
|
||||
x_label='Blank Backface Angle (degrees)',
|
||||
y_label='60/20 WFE (Relative RMS)',
|
||||
title='M1 Mirror Optimization\nBackface Angle vs Wavefront Error',
|
||||
filename=f'{output_dir}\\pareto_angle_vs_wfe.png',
|
||||
minimize_x=False,
|
||||
minimize_y=True,
|
||||
color_by='mass_kg' if 'mass_kg' in df_valid.columns else None,
|
||||
color_label='Mass (kg)'
|
||||
)
|
||||
else:
|
||||
print("No backface angle data available")
|
||||
|
||||
# 3. Combined 2D Design Space plot
|
||||
print("\n--- Creating Design Space plot ---")
|
||||
if df_valid[thickness_col].notna().any() and df_valid[angle_col].notna().any():
|
||||
mask = df_valid[thickness_col].notna() & df_valid[angle_col].notna()
|
||||
df_both = df_valid[mask].copy()
|
||||
|
||||
if len(df_both) > 0:
|
||||
fig, ax = plt.subplots(figsize=(12, 9))
|
||||
fig.patch.set_facecolor('white')
|
||||
|
||||
bg_color = '#f8f9fa'
|
||||
grid_color = '#dee2e6'
|
||||
ax.set_facecolor(bg_color)
|
||||
|
||||
# Use a perceptually uniform colormap
|
||||
scatter = ax.scatter(
|
||||
df_both[thickness_col],
|
||||
df_both[angle_col],
|
||||
c=df_both[wfe_col],
|
||||
cmap='RdYlGn_r', # Red=bad (high WFE), Green=good (low WFE)
|
||||
s=100,
|
||||
alpha=0.8,
|
||||
edgecolors='white',
|
||||
linewidth=0.5,
|
||||
vmin=df_both[wfe_col].quantile(0.05),
|
||||
vmax=df_both[wfe_col].quantile(0.95)
|
||||
)
|
||||
|
||||
cbar = plt.colorbar(scatter, ax=ax, pad=0.02, shrink=0.85)
|
||||
cbar.set_label('60/20 WFE (Relative RMS)\nLower = Better Performance',
|
||||
fontsize=12, fontweight='bold')
|
||||
cbar.ax.tick_params(labelsize=10)
|
||||
|
||||
ax.set_xlabel('Blank Thickness (mm)', fontsize=14, fontweight='bold', labelpad=12)
|
||||
ax.set_ylabel('Blank Backface Angle (degrees)', fontsize=14, fontweight='bold', labelpad=12)
|
||||
ax.set_title('M1 Mirror Design Space Exploration\nGeometric Parameters vs Optical Performance',
|
||||
fontsize=18, fontweight='bold', pad=20)
|
||||
|
||||
ax.grid(True, alpha=0.5, color=grid_color)
|
||||
ax.minorticks_on()
|
||||
ax.grid(True, which='minor', alpha=0.2, linestyle=':', color=grid_color)
|
||||
|
||||
# Mark best point with star
|
||||
best_idx = df_both[wfe_col].idxmin()
|
||||
best_row = df_both.loc[best_idx]
|
||||
ax.scatter(best_row[thickness_col], best_row[angle_col],
|
||||
c='#ffc107', s=400, marker='*', edgecolors='#dc3545', linewidth=3,
|
||||
zorder=5, label=f'Best Design (WFE={best_row[wfe_col]:.2f})')
|
||||
|
||||
# Add annotation for best point - position in upper left to avoid overlap
|
||||
ax.annotate(f'Best Design\nThickness: {best_row[thickness_col]:.1f}mm\nAngle: {best_row[angle_col]:.2f}°\nWFE: {best_row[wfe_col]:.2f}',
|
||||
xy=(best_row[thickness_col], best_row[angle_col]),
|
||||
xytext=(-100, 60), textcoords='offset points',
|
||||
fontsize=10, fontweight='bold',
|
||||
bbox=dict(boxstyle='round,pad=0.6', facecolor='white',
|
||||
edgecolor='#dc3545', linewidth=2, alpha=0.95),
|
||||
arrowprops=dict(arrowstyle='->', connectionstyle='arc3,rad=0.3',
|
||||
color='#dc3545', lw=2))
|
||||
|
||||
ax.legend(loc='upper right', fontsize=11, framealpha=0.95, fancybox=True, shadow=True)
|
||||
|
||||
# Stats
|
||||
stats_text = f'Designs evaluated: {len(df_both):,}'
|
||||
ax.text(0.02, 0.02, stats_text, transform=ax.transAxes, fontsize=10,
|
||||
verticalalignment='bottom',
|
||||
bbox=dict(boxstyle='round,pad=0.5', facecolor='white',
|
||||
edgecolor=grid_color, alpha=0.9))
|
||||
|
||||
for spine in ax.spines.values():
|
||||
spine.set_color(grid_color)
|
||||
spine.set_linewidth(1.5)
|
||||
|
||||
plt.tight_layout()
|
||||
plt.savefig(f'{output_dir}\\design_space_wfe.png',
|
||||
dpi=200, bbox_inches='tight', facecolor='white', pad_inches=0.2)
|
||||
plt.close()
|
||||
print(f"Saved: design_space_wfe.png")
|
||||
else:
|
||||
print("Not enough data for combined design space plot")
|
||||
|
||||
print("\n" + "="*60)
|
||||
print("PARETO VISUALIZATION COMPLETE")
|
||||
print("="*60)
|
||||
print(f"\nOutput files saved to: {output_dir}")
|
||||
print("\nFiles created:")
|
||||
print(" 1. pareto_thickness_vs_wfe.png - Thickness vs WFE Pareto front")
|
||||
print(" 2. pareto_angle_vs_wfe.png - Backface Angle vs WFE Pareto front")
|
||||
print(" 3. design_space_wfe.png - Combined design space heatmap")
|
||||
192
tools/extract_all_mirror_data.py
Normal file
192
tools/extract_all_mirror_data.py
Normal file
@@ -0,0 +1,192 @@
|
||||
#!/usr/bin/env python
|
||||
"""
|
||||
Extract all M1 mirror optimization trial data from Optuna study databases.
|
||||
Outputs a consolidated CSV file with all parameters and objectives.
|
||||
"""
|
||||
|
||||
import sqlite3
|
||||
import json
|
||||
import csv
|
||||
from pathlib import Path
|
||||
from collections import defaultdict
|
||||
|
||||
# Studies to extract (in order)
|
||||
STUDIES = [
|
||||
"m1_mirror_zernike_optimization",
|
||||
"m1_mirror_adaptive_V11",
|
||||
"m1_mirror_adaptive_V13",
|
||||
"m1_mirror_adaptive_V14",
|
||||
"m1_mirror_adaptive_V15",
|
||||
"m1_mirror_cost_reduction",
|
||||
"m1_mirror_cost_reduction_V2",
|
||||
]
|
||||
|
||||
# All possible design variables (superset across all studies)
|
||||
DESIGN_VARS = [
|
||||
"lateral_inner_angle",
|
||||
"lateral_outer_angle",
|
||||
"lateral_outer_pivot",
|
||||
"lateral_inner_pivot",
|
||||
"lateral_middle_pivot",
|
||||
"lateral_closeness",
|
||||
"whiffle_min",
|
||||
"whiffle_outer_to_vertical",
|
||||
"whiffle_triangle_closeness",
|
||||
"blank_backface_angle",
|
||||
"inner_circular_rib_dia",
|
||||
"center_thickness",
|
||||
]
|
||||
|
||||
# All objectives
|
||||
OBJECTIVES = [
|
||||
"rel_filtered_rms_40_vs_20",
|
||||
"rel_filtered_rms_60_vs_20",
|
||||
"mfg_90_optician_workload",
|
||||
"mass_kg",
|
||||
]
|
||||
|
||||
|
||||
def get_db_path(study_name: str) -> Path:
|
||||
"""Get the database path for a study."""
|
||||
# Check in M1_Mirror topic folder first (new structure)
|
||||
base = Path(__file__).parent / "studies" / "M1_Mirror" / study_name
|
||||
for subdir in ["3_results", "2_results"]:
|
||||
db_path = base / subdir / "study.db"
|
||||
if db_path.exists():
|
||||
return db_path
|
||||
# Fallback to flat structure (backwards compatibility)
|
||||
base = Path(__file__).parent / "studies" / study_name
|
||||
for subdir in ["3_results", "2_results"]:
|
||||
db_path = base / subdir / "study.db"
|
||||
if db_path.exists():
|
||||
return db_path
|
||||
return None
|
||||
|
||||
|
||||
def get_config_path(study_name: str) -> Path:
|
||||
"""Get the config path for a study."""
|
||||
# Check in M1_Mirror topic folder first (new structure)
|
||||
config_path = Path(__file__).parent / "studies" / "M1_Mirror" / study_name / "1_setup" / "optimization_config.json"
|
||||
if config_path.exists():
|
||||
return config_path
|
||||
# Fallback to flat structure
|
||||
return Path(__file__).parent / "studies" / study_name / "1_setup" / "optimization_config.json"
|
||||
|
||||
|
||||
def load_objective_mapping(config_path: Path) -> dict:
|
||||
"""Load objective names from config to map objective_id to name."""
|
||||
with open(config_path) as f:
|
||||
config = json.load(f)
|
||||
|
||||
objectives = config.get("objectives", [])
|
||||
# objective_id 0, 1, 2, ... maps to objectives in order
|
||||
return {i: obj["name"] for i, obj in enumerate(objectives)}
|
||||
|
||||
|
||||
def extract_trials_from_db(db_path: Path, obj_mapping: dict) -> list:
|
||||
"""Extract all completed trials from an Optuna study database."""
|
||||
conn = sqlite3.connect(str(db_path))
|
||||
cursor = conn.cursor()
|
||||
|
||||
# Get all completed trials
|
||||
cursor.execute("""
|
||||
SELECT trial_id FROM trials WHERE state = 'COMPLETE'
|
||||
""")
|
||||
trial_ids = [row[0] for row in cursor.fetchall()]
|
||||
|
||||
trials = []
|
||||
for trial_id in trial_ids:
|
||||
trial_data = {"trial_id": trial_id}
|
||||
|
||||
# Get parameters
|
||||
cursor.execute("""
|
||||
SELECT param_name, param_value FROM trial_params WHERE trial_id = ?
|
||||
""", (trial_id,))
|
||||
for param_name, param_value in cursor.fetchall():
|
||||
trial_data[param_name] = param_value
|
||||
|
||||
# Get individual objective values from user attributes
|
||||
# (Atomizer stores individual objectives here, weighted_sum in trial_values)
|
||||
cursor.execute("""
|
||||
SELECT key, value_json FROM trial_user_attributes WHERE trial_id = ?
|
||||
""", (trial_id,))
|
||||
for key, value in cursor.fetchall():
|
||||
# The value is JSON-encoded (string with quotes for strings, plain for numbers)
|
||||
try:
|
||||
# Try to parse as float first
|
||||
trial_data[key] = float(value)
|
||||
except ValueError:
|
||||
# Keep as string (e.g., source tag)
|
||||
trial_data[key] = value.strip('"')
|
||||
|
||||
trials.append(trial_data)
|
||||
|
||||
conn.close()
|
||||
return trials
|
||||
|
||||
|
||||
def main():
|
||||
studies_dir = Path(__file__).parent / "studies"
|
||||
output_path = studies_dir / "m1_mirror_all_trials_export.csv"
|
||||
|
||||
# CSV header
|
||||
header = ["study", "trial"] + DESIGN_VARS + OBJECTIVES
|
||||
|
||||
all_rows = []
|
||||
stats = {}
|
||||
|
||||
for study_name in STUDIES:
|
||||
db_path = get_db_path(study_name)
|
||||
config_path = get_config_path(study_name)
|
||||
|
||||
if not db_path or not db_path.exists():
|
||||
print(f"[SKIP] {study_name}: No database found")
|
||||
stats[study_name] = 0
|
||||
continue
|
||||
|
||||
if not config_path.exists():
|
||||
print(f"[SKIP] {study_name}: No config found")
|
||||
stats[study_name] = 0
|
||||
continue
|
||||
|
||||
print(f"[LOAD] {study_name}...")
|
||||
|
||||
# Load objective mapping from config
|
||||
obj_mapping = load_objective_mapping(config_path)
|
||||
|
||||
# Extract trials
|
||||
trials = extract_trials_from_db(db_path, obj_mapping)
|
||||
stats[study_name] = len(trials)
|
||||
|
||||
# Convert to rows
|
||||
for trial in trials:
|
||||
row = {
|
||||
"study": study_name,
|
||||
"trial": trial["trial_id"],
|
||||
}
|
||||
# Add design variables
|
||||
for var in DESIGN_VARS:
|
||||
row[var] = trial.get(var, "")
|
||||
# Add objectives
|
||||
for obj in OBJECTIVES:
|
||||
row[obj] = trial.get(obj, "")
|
||||
|
||||
all_rows.append(row)
|
||||
|
||||
# Write CSV
|
||||
with open(output_path, "w", newline="") as f:
|
||||
writer = csv.DictWriter(f, fieldnames=header)
|
||||
writer.writeheader()
|
||||
writer.writerows(all_rows)
|
||||
|
||||
print(f"\n{'='*60}")
|
||||
print(f"EXPORT COMPLETE: {output_path}")
|
||||
print(f"{'='*60}")
|
||||
print(f"\nTotal trials exported: {len(all_rows)}")
|
||||
print(f"\nTrials per study:")
|
||||
for study, count in stats.items():
|
||||
print(f" {study}: {count}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
294
tools/extract_mirror_optical_specs.py
Normal file
294
tools/extract_mirror_optical_specs.py
Normal file
@@ -0,0 +1,294 @@
|
||||
#!/usr/bin/env python
|
||||
"""
|
||||
Extract Mirror Optical Specifications from FEA Mesh Geometry
|
||||
|
||||
This tool analyzes mirror mesh geometry to estimate optical specifications
|
||||
including focal length, aperture diameter, f-number, and radius of curvature.
|
||||
|
||||
Usage:
|
||||
# From study directory containing OP2 files
|
||||
python -m optimization_engine.tools.extract_mirror_optical_specs .
|
||||
|
||||
# From specific OP2 file
|
||||
python -m optimization_engine.tools.extract_mirror_optical_specs path/to/results.op2
|
||||
|
||||
# Save to study README
|
||||
python -m optimization_engine.tools.extract_mirror_optical_specs . --update-readme
|
||||
|
||||
Output:
|
||||
- Console: Optical specifications summary
|
||||
- Optional: Updates parent README.md with validated specs
|
||||
|
||||
Author: Atomizer Framework
|
||||
"""
|
||||
|
||||
from pathlib import Path
|
||||
import argparse
|
||||
import sys
|
||||
|
||||
# Add project root to path (tools/ is at project root, so parent is Atomizer/)
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent))
|
||||
|
||||
import numpy as np
|
||||
|
||||
|
||||
def find_op2_file(path: Path) -> Path:
|
||||
"""Find an OP2 file from path (file or directory)."""
|
||||
path = Path(path)
|
||||
|
||||
if path.is_file() and path.suffix.lower() == '.op2':
|
||||
return path
|
||||
|
||||
if path.is_dir():
|
||||
# Look in common locations
|
||||
search_patterns = [
|
||||
'**/2_iterations/**/*.op2',
|
||||
'**/*.op2',
|
||||
'2_iterations/**/*.op2',
|
||||
'1_setup/model/*.op2',
|
||||
]
|
||||
|
||||
for pattern in search_patterns:
|
||||
op2_files = list(path.glob(pattern))
|
||||
if op2_files:
|
||||
# Return most recent
|
||||
return max(op2_files, key=lambda p: p.stat().st_mtime)
|
||||
|
||||
raise FileNotFoundError(f"No OP2 file found in {path}")
|
||||
|
||||
|
||||
def extract_optical_specs(op2_path: Path, verbose: bool = True) -> dict:
|
||||
"""
|
||||
Extract optical specifications from mirror mesh geometry.
|
||||
|
||||
Args:
|
||||
op2_path: Path to OP2 file
|
||||
verbose: Print detailed output
|
||||
|
||||
Returns:
|
||||
dict with optical specifications
|
||||
"""
|
||||
from optimization_engine.extractors.extract_zernike_opd import (
|
||||
ZernikeOPDExtractor,
|
||||
estimate_focal_length_from_geometry
|
||||
)
|
||||
|
||||
if verbose:
|
||||
print(f"Analyzing: {op2_path}")
|
||||
print("=" * 60)
|
||||
|
||||
extractor = ZernikeOPDExtractor(op2_path)
|
||||
|
||||
# Get geometry
|
||||
geo = extractor.node_geometry
|
||||
all_pos = np.array(list(geo.values()))
|
||||
x, y, z = all_pos[:, 0], all_pos[:, 1], all_pos[:, 2]
|
||||
|
||||
# Compute radius/diameter
|
||||
r = np.sqrt(x**2 + y**2)
|
||||
|
||||
# Estimate focal length
|
||||
focal = estimate_focal_length_from_geometry(x, y, z, concave=True)
|
||||
|
||||
# Derived quantities
|
||||
diameter = 2 * r.max()
|
||||
f_number = focal / diameter
|
||||
RoC = 2 * focal # Radius of curvature
|
||||
sag = r.max()**2 / (4 * focal) # Surface sag at edge
|
||||
central_obs = r.min() if r.min() > 1.0 else 0.0 # Central obscuration
|
||||
|
||||
# Parabola fit quality check
|
||||
r_sq = x**2 + y**2
|
||||
A = np.column_stack([r_sq, np.ones_like(r_sq)])
|
||||
coeffs, _, _, _ = np.linalg.lstsq(A, z, rcond=None)
|
||||
a, b = coeffs
|
||||
z_fit = a * r_sq + b
|
||||
rms_error = np.sqrt(np.mean((z - z_fit)**2))
|
||||
|
||||
# Determine fit quality
|
||||
if rms_error < 0.1:
|
||||
fit_quality = "Excellent"
|
||||
fit_note = "Focal length estimate is reliable"
|
||||
elif rms_error < 1.0:
|
||||
fit_quality = "Good"
|
||||
fit_note = "Focal length estimate is reasonably accurate"
|
||||
else:
|
||||
fit_quality = "Poor"
|
||||
fit_note = "Consider using explicit focal length from optical design"
|
||||
|
||||
specs = {
|
||||
'aperture_diameter_mm': diameter,
|
||||
'aperture_radius_mm': r.max(),
|
||||
'focal_length_mm': focal,
|
||||
'f_number': f_number,
|
||||
'radius_of_curvature_mm': RoC,
|
||||
'surface_sag_mm': sag,
|
||||
'central_obscuration_mm': central_obs,
|
||||
'node_count': len(geo),
|
||||
'x_range_mm': (x.min(), x.max()),
|
||||
'y_range_mm': (y.min(), y.max()),
|
||||
'z_range_mm': (z.min(), z.max()),
|
||||
'parabola_fit_rms_mm': rms_error,
|
||||
'fit_quality': fit_quality,
|
||||
'fit_note': fit_note,
|
||||
'source_file': str(op2_path),
|
||||
}
|
||||
|
||||
if verbose:
|
||||
print()
|
||||
print("MIRROR OPTICAL SPECIFICATIONS (from mesh geometry)")
|
||||
print("=" * 60)
|
||||
print()
|
||||
print(f"Aperture Diameter: {diameter:.1f} mm ({diameter/1000:.3f} m)")
|
||||
print(f"Aperture Radius: {r.max():.1f} mm")
|
||||
if central_obs > 0:
|
||||
print(f"Central Obscuration: {central_obs:.1f} mm")
|
||||
print()
|
||||
print(f"Estimated Focal Length: {focal:.1f} mm ({focal/1000:.3f} m)")
|
||||
print(f"Radius of Curvature: {RoC:.1f} mm ({RoC/1000:.3f} m)")
|
||||
print(f"f-number (f/D): f/{f_number:.2f}")
|
||||
print()
|
||||
print(f"Surface Sag at Edge: {sag:.2f} mm")
|
||||
print()
|
||||
print("--- Mesh Statistics ---")
|
||||
print(f"Node count: {len(geo)}")
|
||||
print(f"X range: {x.min():.1f} to {x.max():.1f} mm")
|
||||
print(f"Y range: {y.min():.1f} to {y.max():.1f} mm")
|
||||
print(f"Z range: {z.min():.2f} to {z.max():.2f} mm")
|
||||
print()
|
||||
print("--- Parabola Fit Quality ---")
|
||||
print(f"RMS fit residual: {rms_error:.4f} mm ({rms_error*1000:.2f} µm)")
|
||||
print(f"Quality: {fit_quality} - {fit_note}")
|
||||
print()
|
||||
print("=" * 60)
|
||||
|
||||
return specs
|
||||
|
||||
|
||||
def generate_readme_section(specs: dict) -> str:
|
||||
"""Generate markdown section for README."""
|
||||
return f"""## 2. Optical Prescription
|
||||
|
||||
> **Source**: Estimated from mesh geometry. Validate against optical design.
|
||||
|
||||
| Parameter | Value | Units | Status |
|
||||
|-----------|-------|-------|--------|
|
||||
| Aperture Diameter | {specs['aperture_diameter_mm']:.1f} | mm | Estimated |
|
||||
| Focal Length | {specs['focal_length_mm']:.1f} | mm | Estimated |
|
||||
| f-number | f/{specs['f_number']:.2f} | - | Computed |
|
||||
| Radius of Curvature | {specs['radius_of_curvature_mm']:.1f} | mm | Computed (2×f) |
|
||||
| Central Obscuration | {specs['central_obscuration_mm']:.1f} | mm | From mesh |
|
||||
| Surface Type | Parabola | - | Assumed |
|
||||
|
||||
**Fit Quality**: {specs['fit_quality']} ({specs['fit_note']})
|
||||
|
||||
### 2.1 Usage in OPD Extractor
|
||||
|
||||
For rigorous WFE analysis, use explicit focal length:
|
||||
|
||||
```python
|
||||
from optimization_engine.extractors import ZernikeOPDExtractor
|
||||
|
||||
extractor = ZernikeOPDExtractor(
|
||||
op2_file,
|
||||
focal_length={specs['focal_length_mm']:.1f}, # mm - validate against design
|
||||
concave=True
|
||||
)
|
||||
```
|
||||
"""
|
||||
|
||||
|
||||
def update_readme(study_dir: Path, specs: dict):
|
||||
"""Update parent README with optical specs."""
|
||||
readme_path = study_dir / 'README.md'
|
||||
|
||||
if not readme_path.exists():
|
||||
print(f"README.md not found at {readme_path}")
|
||||
return False
|
||||
|
||||
content = readme_path.read_text(encoding='utf-8')
|
||||
|
||||
# Find and replace optical prescription section
|
||||
new_section = generate_readme_section(specs)
|
||||
|
||||
# Look for existing section
|
||||
import re
|
||||
pattern = r'## 2\. Optical Prescription.*?(?=## 3\.|$)'
|
||||
|
||||
if re.search(pattern, content, re.DOTALL):
|
||||
content = re.sub(pattern, new_section + '\n---\n\n', content, flags=re.DOTALL)
|
||||
print(f"Updated optical prescription in {readme_path}")
|
||||
else:
|
||||
print(f"Could not find '## 2. Optical Prescription' section in {readme_path}")
|
||||
print("Please add manually or check section numbering.")
|
||||
return False
|
||||
|
||||
readme_path.write_text(content, encoding='utf-8')
|
||||
return True
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(
|
||||
description='Extract mirror optical specifications from FEA mesh',
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog="""
|
||||
Examples:
|
||||
# Analyze current study directory
|
||||
python -m optimization_engine.tools.extract_mirror_optical_specs .
|
||||
|
||||
# Analyze specific OP2 file
|
||||
python -m optimization_engine.tools.extract_mirror_optical_specs results.op2
|
||||
|
||||
# Update parent README with specs
|
||||
python -m optimization_engine.tools.extract_mirror_optical_specs . --update-readme
|
||||
"""
|
||||
)
|
||||
|
||||
parser.add_argument('path', type=str,
|
||||
help='Path to OP2 file or study directory')
|
||||
parser.add_argument('--update-readme', action='store_true',
|
||||
help='Update parent README.md with optical specs')
|
||||
parser.add_argument('--quiet', '-q', action='store_true',
|
||||
help='Suppress detailed output')
|
||||
parser.add_argument('--json', action='store_true',
|
||||
help='Output specs as JSON')
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
try:
|
||||
path = Path(args.path).resolve()
|
||||
op2_path = find_op2_file(path)
|
||||
|
||||
specs = extract_optical_specs(op2_path, verbose=not args.quiet and not args.json)
|
||||
|
||||
if args.json:
|
||||
import json
|
||||
print(json.dumps(specs, indent=2, default=str))
|
||||
|
||||
if args.update_readme:
|
||||
# Find study root (parent of geometry type folder)
|
||||
study_dir = path if path.is_dir() else path.parent
|
||||
# Go up to geometry type level
|
||||
while study_dir.name not in ['studies', ''] and not (study_dir / 'README.md').exists():
|
||||
if (study_dir.parent / 'README.md').exists():
|
||||
study_dir = study_dir.parent
|
||||
break
|
||||
study_dir = study_dir.parent
|
||||
|
||||
if (study_dir / 'README.md').exists():
|
||||
update_readme(study_dir, specs)
|
||||
else:
|
||||
print(f"Could not find parent README.md to update")
|
||||
|
||||
except FileNotFoundError as e:
|
||||
print(f"Error: {e}")
|
||||
sys.exit(1)
|
||||
except Exception as e:
|
||||
print(f"Error: {e}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
155
tools/migrate_studies_to_topics.py
Normal file
155
tools/migrate_studies_to_topics.py
Normal file
@@ -0,0 +1,155 @@
|
||||
#!/usr/bin/env python
|
||||
"""
|
||||
Migration script to reorganize studies into topic-based subfolders.
|
||||
|
||||
Run with --dry-run first to preview changes:
|
||||
python migrate_studies_to_topics.py --dry-run
|
||||
|
||||
Then run without flag to execute:
|
||||
python migrate_studies_to_topics.py
|
||||
"""
|
||||
|
||||
import shutil
|
||||
import argparse
|
||||
from pathlib import Path
|
||||
|
||||
STUDIES_DIR = Path(__file__).parent / "studies"
|
||||
|
||||
# Topic classification based on study name prefixes
|
||||
TOPIC_MAPPING = {
|
||||
'bracket_': 'Simple_Bracket',
|
||||
'drone_gimbal_': 'Drone_Gimbal',
|
||||
'm1_mirror_': 'M1_Mirror',
|
||||
'uav_arm_': 'UAV_Arm',
|
||||
'simple_beam_': 'Simple_Beam',
|
||||
}
|
||||
|
||||
# Files/folders to skip (not studies)
|
||||
SKIP_ITEMS = {
|
||||
'm1_mirror_all_trials_export.csv', # Data export file
|
||||
'.gitkeep',
|
||||
'__pycache__',
|
||||
}
|
||||
|
||||
|
||||
def classify_study(study_name: str) -> str:
|
||||
"""Determine which topic folder a study belongs to."""
|
||||
for prefix, topic in TOPIC_MAPPING.items():
|
||||
if study_name.startswith(prefix):
|
||||
return topic
|
||||
return '_Other'
|
||||
|
||||
|
||||
def get_studies_to_migrate():
|
||||
"""Get list of studies that need migration (not already in topic folders)."""
|
||||
studies = []
|
||||
|
||||
for item in STUDIES_DIR.iterdir():
|
||||
# Skip non-directories and special items
|
||||
if not item.is_dir():
|
||||
continue
|
||||
if item.name in SKIP_ITEMS:
|
||||
continue
|
||||
if item.name.startswith('.'):
|
||||
continue
|
||||
|
||||
# Check if this is already a topic folder (contains study subdirs)
|
||||
# A topic folder would have subdirs with 1_setup folders
|
||||
is_topic_folder = any(
|
||||
(sub / "1_setup").exists()
|
||||
for sub in item.iterdir()
|
||||
if sub.is_dir()
|
||||
)
|
||||
|
||||
if is_topic_folder:
|
||||
print(f"[SKIP] {item.name} - already a topic folder")
|
||||
continue
|
||||
|
||||
# Check if this is a study (has 1_setup or optimization_config.json)
|
||||
is_study = (
|
||||
(item / "1_setup").exists() or
|
||||
(item / "optimization_config.json").exists()
|
||||
)
|
||||
|
||||
if is_study:
|
||||
topic = classify_study(item.name)
|
||||
studies.append({
|
||||
'name': item.name,
|
||||
'source': item,
|
||||
'topic': topic,
|
||||
'target': STUDIES_DIR / topic / item.name
|
||||
})
|
||||
else:
|
||||
print(f"[SKIP] {item.name} - not a study (no 1_setup folder)")
|
||||
|
||||
return studies
|
||||
|
||||
|
||||
def migrate_studies(dry_run: bool = True):
|
||||
"""Migrate studies to topic folders."""
|
||||
studies = get_studies_to_migrate()
|
||||
|
||||
if not studies:
|
||||
print("\nNo studies to migrate. All studies are already organized.")
|
||||
return
|
||||
|
||||
# Group by topic for display
|
||||
by_topic = {}
|
||||
for s in studies:
|
||||
if s['topic'] not in by_topic:
|
||||
by_topic[s['topic']] = []
|
||||
by_topic[s['topic']].append(s)
|
||||
|
||||
print("\n" + "="*60)
|
||||
print("MIGRATION PLAN")
|
||||
print("="*60)
|
||||
|
||||
for topic in sorted(by_topic.keys()):
|
||||
print(f"\n{topic}/")
|
||||
for s in by_topic[topic]:
|
||||
print(f" +-- {s['name']}/")
|
||||
|
||||
print(f"\nTotal: {len(studies)} studies to migrate")
|
||||
|
||||
if dry_run:
|
||||
print("\n[DRY RUN] No changes made. Run without --dry-run to execute.")
|
||||
return
|
||||
|
||||
# Execute migration
|
||||
print("\n" + "="*60)
|
||||
print("EXECUTING MIGRATION")
|
||||
print("="*60)
|
||||
|
||||
# Create topic folders
|
||||
created_topics = set()
|
||||
for s in studies:
|
||||
topic_dir = STUDIES_DIR / s['topic']
|
||||
if s['topic'] not in created_topics:
|
||||
topic_dir.mkdir(exist_ok=True)
|
||||
created_topics.add(s['topic'])
|
||||
print(f"[CREATE] {s['topic']}/")
|
||||
|
||||
# Move studies
|
||||
for s in studies:
|
||||
try:
|
||||
shutil.move(str(s['source']), str(s['target']))
|
||||
print(f"[MOVE] {s['name']} -> {s['topic']}/{s['name']}")
|
||||
except Exception as e:
|
||||
print(f"[ERROR] Failed to move {s['name']}: {e}")
|
||||
|
||||
print("\n" + "="*60)
|
||||
print("MIGRATION COMPLETE")
|
||||
print("="*60)
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="Migrate studies to topic folders")
|
||||
parser.add_argument('--dry-run', action='store_true',
|
||||
help='Preview changes without executing')
|
||||
args = parser.parse_args()
|
||||
|
||||
migrate_studies(dry_run=args.dry_run)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Reference in New Issue
Block a user