chore(hq): daily sync 2026-02-19
This commit is contained in:
@@ -0,0 +1,982 @@
|
||||
# TECHNICAL REVIEW: Atomizer Project Standard Specification v1.0
|
||||
|
||||
**Reviewer:** Technical Lead (Subagent)
|
||||
**Date:** 2026-02-18
|
||||
**Specification Version:** 1.0 (Draft)
|
||||
**Review Scope:** Compatibility, feasibility, implementation complexity
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
The proposed Atomizer Project Standard is **ambitious and well-designed** but requires **significant code changes** to be fully compatible with the existing optimization engine. The core concepts are sound, but several assumptions about auto-extraction and file organization need technical amendments.
|
||||
|
||||
**Overall Assessment:**
|
||||
- ✅ **Philosophy**: Excellent — self-contained, LLM-native, rationale-first
|
||||
- ⚠️ **AtomizerSpec v2.0 Integration**: Requires path refactoring in optimization engine
|
||||
- ⚠️ **Auto-Extraction Claims**: Overstated — some extraction needs manual intervention
|
||||
- ✅ **Study Lifecycle**: Good mapping to existing protocols, but gaps exist
|
||||
- ⚠️ **Schema Alignment**: `.atomizer/project.json` creates redundancy
|
||||
- ⚠️ **Model Organization**: Dual-location pattern is confusing
|
||||
- ✅ **Report Generation**: Feasible with existing tooling
|
||||
- ⚠️ **Playbooks**: Redundant with Protocol Operating System
|
||||
|
||||
**Recommendation:** Adopt with amendments. Priority order: Path compatibility → Auto-extraction → Schema consolidation → Documentation.
|
||||
|
||||
---
|
||||
|
||||
## 1. AtomizerSpec v2.0 Compatibility
|
||||
|
||||
### Current State
|
||||
|
||||
**File Paths in AtomizerSpec v2.0:**
|
||||
```json
|
||||
{
|
||||
"model": {
|
||||
"sim": {
|
||||
"path": "Model_sim1.sim" // Relative to study root
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Optimization Engine Assumptions:**
|
||||
```python
|
||||
# optimization_engine/core/runner.py (line ~30)
|
||||
self.config_path = Path(config_path) # Expects config in study root or 1_setup/
|
||||
|
||||
# optimization_engine/nx/solver.py
|
||||
# Expects model files in study_dir/1_setup/model/
|
||||
```
|
||||
|
||||
**Current Study Structure:**
|
||||
```
|
||||
studies/{geometry_type}/{study_name}/
|
||||
├── 1_setup/
|
||||
│ ├── model/ # NX files here
|
||||
│ └── atomizer_spec.json # Config here (or optimization_config.json)
|
||||
├── 2_iterations/
|
||||
└── 3_results/
|
||||
```
|
||||
|
||||
### Proposed Structure
|
||||
|
||||
```
|
||||
{project-slug}/
|
||||
├── 01-models/
|
||||
│ ├── cad/Model.prt
|
||||
│ ├── fem/Model_fem1.fem
|
||||
│ └── sim/Model_sim1.sim
|
||||
├── 03-studies/
|
||||
│ └── {NN}_{slug}/
|
||||
│ ├── atomizer_spec.json
|
||||
│ └── 1_setup/
|
||||
│ └── model/ # Study-specific model copy (if modified)
|
||||
```
|
||||
|
||||
### Compatibility Issues
|
||||
|
||||
| Issue | Severity | Impact |
|
||||
|-------|----------|--------|
|
||||
| `atomizer_spec.json` path references assume study-relative paths | **HIGH** | Path resolution breaks if models are in `../../01-models/` |
|
||||
| `OptimizationRunner.__init__()` expects config in study dir | **MEDIUM** | Need `--project-root` flag |
|
||||
| Study archival assumes `1_setup/model/` is the source | **MEDIUM** | Backup/restore logic needs project-aware path |
|
||||
| `TrialManager` trial numbering is study-scoped, not project-scoped | **LOW** | Multiple studies in a project can have overlapping trial numbers |
|
||||
|
||||
### Code Changes Required
|
||||
|
||||
#### Change 1: Add Project Root Awareness
|
||||
**File:** `optimization_engine/core/runner.py`
|
||||
**Complexity:** Medium (2-4 hours)
|
||||
|
||||
```python
|
||||
class OptimizationRunner:
|
||||
def __init__(
|
||||
self,
|
||||
config_path: Path,
|
||||
project_root: Optional[Path] = None, # NEW
|
||||
model_updater: Callable,
|
||||
simulation_runner: Callable,
|
||||
result_extractors: Dict[str, Callable]
|
||||
):
|
||||
self.config_path = Path(config_path)
|
||||
self.project_root = Path(project_root) if project_root else self.config_path.parent.parent # NEW
|
||||
|
||||
# Resolve model paths relative to project root
|
||||
if self.project_root:
|
||||
self.model_dir = self.project_root / "01-models"
|
||||
else:
|
||||
self.model_dir = self.config_path.parent / "1_setup" / "model"
|
||||
```
|
||||
|
||||
**Priority:** P0 — Blocking for project-standard adoption
|
||||
|
||||
#### Change 2: Update AtomizerSpec Path Resolution
|
||||
**File:** `optimization_engine/config/spec_models.py`
|
||||
**Complexity:** Low (1-2 hours)
|
||||
|
||||
Add `resolve_paths()` method to `AtomizerSpec`:
|
||||
|
||||
```python
|
||||
class AtomizerSpec(BaseModel):
|
||||
# ... existing fields ...
|
||||
|
||||
def resolve_paths(self, project_root: Path) -> None:
|
||||
"""Resolve all relative paths against project root."""
|
||||
if self.model.sim.path:
|
||||
self.model.sim.path = str(project_root / "01-models" / "sim" / Path(self.model.sim.path).name)
|
||||
if self.model.fem and self.model.fem.path:
|
||||
self.model.fem.path = str(project_root / "01-models" / "fem" / Path(self.model.fem.path).name)
|
||||
# ... etc
|
||||
```
|
||||
|
||||
**Priority:** P0 — Blocking
|
||||
|
||||
#### Change 3: Add `--project-root` CLI Flag
|
||||
**File:** `run_optimization.py` (template)
|
||||
**Complexity:** Trivial (<1 hour)
|
||||
|
||||
```python
|
||||
parser.add_argument(
|
||||
"--project-root",
|
||||
type=Path,
|
||||
default=None,
|
||||
help="Path to project root (for Atomizer Project Standard layout)"
|
||||
)
|
||||
```
|
||||
|
||||
**Priority:** P1 — Nice-to-have
|
||||
|
||||
### Recommendation
|
||||
|
||||
**Option A: Hybrid Mode (Recommended)**
|
||||
|
||||
Support BOTH layouts:
|
||||
- **Legacy mode** (default): `studies/{study}/1_setup/model/`
|
||||
- **Project mode**: `projects/{name}/03-studies/{study}/` + `--project-root` flag
|
||||
|
||||
Detect automatically:
|
||||
```python
|
||||
def detect_layout(study_dir: Path) -> str:
|
||||
if (study_dir.parent.parent / "01-models").exists():
|
||||
return "project-standard"
|
||||
return "legacy"
|
||||
```
|
||||
|
||||
**Option B: Migration Tool**
|
||||
|
||||
Create `migrate_to_project_standard.py` that:
|
||||
1. Creates `01-models/` and copies baseline model
|
||||
2. Moves studies to `03-studies/{NN}_{slug}/`
|
||||
3. Updates all `atomizer_spec.json` paths
|
||||
4. Creates `.atomizer/project.json`
|
||||
|
||||
**Estimated Implementation:** 12-16 hours total
|
||||
|
||||
---
|
||||
|
||||
## 2. Study Lifecycle vs Real Workflow
|
||||
|
||||
### Proposed Lifecycle (Spec §6.1)
|
||||
|
||||
```
|
||||
PLAN → CONFIGURE → VALIDATE → RUN → ANALYZE → REPORT → FEED-BACK
|
||||
```
|
||||
|
||||
### Actual Atomizer Operations (Protocol Operating System)
|
||||
|
||||
| Operation | POS Protocol | Coverage in Spec |
|
||||
|-----------|--------------|------------------|
|
||||
| **OP_01: Create Study** | Bootstrap + study creation | ✅ PLAN + CONFIGURE |
|
||||
| **OP_02: Run Optimization** | Execute trials | ✅ RUN |
|
||||
| **OP_03: Monitor Progress** | Real-time tracking | ❌ Missing in lifecycle |
|
||||
| **OP_04: Analyze Results** | Post-processing | ✅ ANALYZE |
|
||||
| **OP_05: Export Training Data** | Neural acceleration | ❌ Missing |
|
||||
| **OP_06: Troubleshoot** | Debug failures | ❌ Missing |
|
||||
| **OP_07: Disk Optimization** | Free space | ❌ Missing |
|
||||
| **OP_08: Generate Report** | Deliverables | ✅ REPORT |
|
||||
|
||||
### Gaps Identified
|
||||
|
||||
| Gap | Impact | Recommendation |
|
||||
|-----|--------|----------------|
|
||||
| No **MONITOR** stage | Real-time progress tracking is critical | Add between RUN and ANALYZE |
|
||||
| No **EXPORT** stage | Neural surrogate training needs data | Add parallel to FEED-BACK |
|
||||
| No **TROUBLESHOOT** stage | Optimization failures are common | Add decision tree in playbooks |
|
||||
| No **DISK-OPTIMIZE** stage | `.op2` files fill disk quickly | Add to tools/ or playbooks |
|
||||
|
||||
### Recommended Lifecycle (Revised)
|
||||
|
||||
```
|
||||
PLAN → CONFIGURE → VALIDATE → RUN → MONITOR → ANALYZE → REPORT → FEED-BACK
|
||||
↓ ↓
|
||||
EXPORT (optional) TROUBLESHOOT (if needed)
|
||||
```
|
||||
|
||||
### Mapping to Spec Sections
|
||||
|
||||
| Lifecycle Stage | Spec Section | Protocol | Deliverable |
|
||||
|-----------------|--------------|----------|-------------|
|
||||
| PLAN | §6.2 Stage 1 | User-driven | `README.md` hypothesis |
|
||||
| CONFIGURE | §6.2 Stage 2 | OP_01 | `atomizer_spec.json` |
|
||||
| VALIDATE | §6.2 Stage 3 | OP_01 preflight | Validation report |
|
||||
| RUN | §6.2 Stage 4 | OP_02 | `study.db`, `2_iterations/` |
|
||||
| **MONITOR** | *Missing* | OP_03 | Real-time logs |
|
||||
| ANALYZE | §6.2 Stage 5 | OP_04 | Plots, sensitivity |
|
||||
| REPORT | §6.2 Stage 6 | OP_08 | `STUDY_REPORT.md` |
|
||||
| FEED-BACK | §6.2 Stage 7 | Manual | KB updates |
|
||||
| **EXPORT** | *Missing* | OP_05 | `training_data/` |
|
||||
| **TROUBLESHOOT** | *Missing* | OP_06 | Fix + log |
|
||||
|
||||
**Estimated Amendment Effort:** 2-4 hours (documentation)
|
||||
|
||||
---
|
||||
|
||||
## 3. Auto-Extraction Feasibility
|
||||
|
||||
### Spec Claims (§7.2)
|
||||
|
||||
> "The Atomizer intake workflow already extracts:"
|
||||
|
||||
| Data | Claimed Source | Reality Check |
|
||||
|------|---------------|---------------|
|
||||
| All NX expressions | ✅ `introspect_part.py` | **VERIFIED** — works |
|
||||
| Mass properties | ✅ NXOpen MeasureManager | **VERIFIED** — works |
|
||||
| Material assignments | ❓ `.fem` material cards | **PARTIAL** — needs parser |
|
||||
| Mesh statistics | ❓ `.fem` element/node counts | **PARTIAL** — needs parser |
|
||||
| Boundary conditions | ❓ `.sim` constraint definitions | **NO** — not implemented |
|
||||
| Load cases | ❓ `.sim` load definitions | **NO** — not implemented |
|
||||
| Baseline solve results | ✅ `.op2` / `.f06` parsing | **VERIFIED** — pyNastran |
|
||||
| Design variable candidates | ✅ Expression analysis | **VERIFIED** — confidence scoring exists |
|
||||
|
||||
### Actual Extraction Capabilities (E12 Introspector)
|
||||
|
||||
**File:** `optimization_engine/extractors/introspect_part.py`
|
||||
|
||||
**WORKS:**
|
||||
```python
|
||||
result = introspect_part("Model.prt")
|
||||
# Returns:
|
||||
{
|
||||
'expressions': {'user': [...], 'user_count': 47},
|
||||
'mass_properties': {'mass_kg': 1133.01, 'volume_mm3': ...},
|
||||
'materials': {'assigned': [...]}, # Material NAME only
|
||||
'bodies': {'solid_bodies': [...], 'counts': {...}},
|
||||
'features': {'total_count': 152, 'by_type': {...}},
|
||||
}
|
||||
```
|
||||
|
||||
**DOES NOT WORK:**
|
||||
- **Material properties** (E, ν, ρ) — Requires `physicalmateriallibrary.xml` parsing (not implemented)
|
||||
- **Boundary conditions** — `.sim` file is XML, no parser exists
|
||||
- **Load cases** — Same issue
|
||||
- **Mesh statistics** — `.fem` is BDF format, would need pyNastran `read_bdf()`
|
||||
|
||||
### Missing Extractors
|
||||
|
||||
| Missing Capability | File Format | Difficulty | Estimated Effort |
|
||||
|--------------------|-------------|------------|------------------|
|
||||
| Material property extraction | `.xml` | Low | 4-6 hours |
|
||||
| Mesh statistics | `.fem` (BDF) | Low | 2-4 hours (use pyNastran) |
|
||||
| BC extraction | `.sim` (XML) | Medium | 8-12 hours |
|
||||
| Load extraction | `.sim` (XML) | Medium | 8-12 hours |
|
||||
|
||||
### Recommendations
|
||||
|
||||
#### Immediate Actions
|
||||
|
||||
1. **Update spec §7.2** to reflect reality:
|
||||
- ✅ Expression extraction: Fully automatic
|
||||
- ✅ Mass properties: Fully automatic
|
||||
- ⚠️ Material assignments: Name only (properties require manual entry or XML parser)
|
||||
- ⚠️ Mesh statistics: Requires BDF parser (pyNastran — 2 hours to implement)
|
||||
- ❌ Boundary conditions: Not automatic — manual KB entry or 8-12 hour XML parser
|
||||
- ❌ Load cases: Not automatic — manual KB entry
|
||||
|
||||
2. **Create roadmap** for missing extractors (§7.2 becomes aspirational):
|
||||
- Phase 1: Expression + Mass (DONE)
|
||||
- Phase 2: Mesh stats (2-4 hours)
|
||||
- Phase 3: Material properties (4-6 hours)
|
||||
- Phase 4: BCs + Loads (16-24 hours)
|
||||
|
||||
3. **Add to KB population protocol** (§7.3):
|
||||
- Auto-extraction runs what's available TODAY
|
||||
- User fills gaps via structured interview
|
||||
- KB entries flag: `source: auto|manual|mixed`
|
||||
|
||||
**Estimated Total Effort for Full Auto-Extraction:** 30-46 hours
|
||||
|
||||
---
|
||||
|
||||
## 4. Schema Alignment: `.atomizer/project.json` vs `atomizer_spec.json`
|
||||
|
||||
### Redundancy Analysis
|
||||
|
||||
**`.atomizer/project.json` (Spec §9.3):**
|
||||
```json
|
||||
{
|
||||
"project": {
|
||||
"slug": "thermoshield-bracket",
|
||||
"display_name": "ThermoShield Bracket",
|
||||
"created": "2026-02-15T...",
|
||||
"template_version": "1.0",
|
||||
"status": "active",
|
||||
"client": "SpaceCo",
|
||||
"model": {
|
||||
"cad_system": "NX",
|
||||
"solver": "NX_Nastran",
|
||||
"solution_type": "SOL101"
|
||||
},
|
||||
"studies_count": 3,
|
||||
"active_study": "03_cmaes_refinement",
|
||||
"tags": ["thermal", "satellite"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**`atomizer_spec.json` (per study):**
|
||||
```json
|
||||
{
|
||||
"meta": {
|
||||
"version": "2.0",
|
||||
"study_name": "03_cmaes_refinement",
|
||||
"created": "2026-02-16T...",
|
||||
"description": "...",
|
||||
"status": "running",
|
||||
"tags": ["thermal"]
|
||||
},
|
||||
"model": {
|
||||
"sim": {
|
||||
"path": "Model_sim1.sim",
|
||||
"solver": "nastran",
|
||||
"solution_type": "SOL101"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Overlap Map
|
||||
|
||||
| Data | `.atomizer/project.json` | `atomizer_spec.json` | Redundant? |
|
||||
|------|-------------------------|---------------------|------------|
|
||||
| Project slug | ✅ | ❌ | No |
|
||||
| Study name | ❌ | ✅ | No |
|
||||
| Client | ✅ | ❌ | No |
|
||||
| CAD system | ✅ | ✅ | **YES** |
|
||||
| Solver | ✅ | ✅ | **YES** |
|
||||
| Solution type | ✅ | ✅ | **YES** |
|
||||
| Status | ✅ (project) | ✅ (study) | No (different scopes) |
|
||||
| Tags | ✅ (project) | ✅ (study) | Partial (project vs study) |
|
||||
| Created date | ✅ (project) | ✅ (study) | No (different events) |
|
||||
|
||||
### Issues
|
||||
|
||||
1. **Source of Truth Conflict:**
|
||||
- If `project.json` says `"solver": "nastran"` but a study spec says `"solver": "abaqus"`, which wins?
|
||||
- **Resolution:** Study spec ALWAYS wins (more specific). `project.json` is metadata only.
|
||||
|
||||
2. **Synchronization Burden:**
|
||||
- Creating a study requires updating BOTH files
|
||||
- Deleting a study requires updating `studies_count`, `active_study`
|
||||
- **Resolution:** Make `project.json` auto-generated (never hand-edit)
|
||||
|
||||
3. **Discoverability:**
|
||||
- `project.json` enables project-level queries ("show all NX projects")
|
||||
- But `atomizer_spec.json` metadata fields (`meta.tags`) already support this
|
||||
- **Resolution:** Keep project-level aggregation but make it derived
|
||||
|
||||
### Recommendation
|
||||
|
||||
**Option 1: Generate `project.json` on-demand (Recommended)**
|
||||
|
||||
```python
|
||||
# optimization_engine/utils/project_manager.py (NEW)
|
||||
|
||||
def generate_project_json(project_root: Path) -> Dict[str, Any]:
|
||||
"""Generate project.json from study specs + PROJECT.md frontmatter."""
|
||||
studies = list((project_root / "03-studies").glob("*/atomizer_spec.json"))
|
||||
|
||||
# Aggregate from studies
|
||||
solvers = set()
|
||||
tags = set()
|
||||
for study_spec in studies:
|
||||
spec = AtomizerSpec.load(study_spec)
|
||||
solvers.add(spec.model.sim.solver)
|
||||
tags.update(spec.meta.tags or [])
|
||||
|
||||
# Read PROJECT.md frontmatter for client, display_name
|
||||
frontmatter = parse_markdown_frontmatter(project_root / "PROJECT.md")
|
||||
|
||||
return {
|
||||
"project": {
|
||||
"slug": project_root.name,
|
||||
"display_name": frontmatter.get("project", project_root.name),
|
||||
"client": frontmatter.get("client"),
|
||||
"created": frontmatter.get("created"),
|
||||
"template_version": frontmatter.get("template_version", "1.0"),
|
||||
"status": frontmatter.get("status", "active"),
|
||||
"model": {
|
||||
"solvers": list(solvers), # List of solvers used across studies
|
||||
},
|
||||
"studies_count": len(studies),
|
||||
"tags": list(tags)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Call this on:
|
||||
- Project creation (`atomizer project create`)
|
||||
- Study creation/deletion
|
||||
- Dashboard load
|
||||
|
||||
**Option 2: Eliminate `project.json` entirely**
|
||||
|
||||
Use `PROJECT.md` frontmatter as the source of truth for project metadata. Tools parse YAML frontmatter when needed.
|
||||
|
||||
**Pros:** No redundancy
|
||||
**Cons:** Slower parsing (MD vs JSON)
|
||||
|
||||
**Verdict:** Option 1. Keep `project.json` as a **cache**, auto-generated.
|
||||
|
||||
**Estimated Effort:** 4-6 hours
|
||||
|
||||
---
|
||||
|
||||
## 5. Model File Organization
|
||||
|
||||
### Proposed Dual-Location Pattern
|
||||
|
||||
**From Spec §2.1 and §10 (ThermoShield Bracket):**
|
||||
|
||||
```
|
||||
thermoshield-bracket/
|
||||
├── 01-models/ # "Golden copies"
|
||||
│ ├── cad/ThermoShield_Bracket.prt
|
||||
│ ├── fem/ThermoShield_Bracket_fem1.fem
|
||||
│ └── sim/ThermoShield_Bracket_sim1.sim
|
||||
└── 03-studies/
|
||||
└── 03_cmaes_refinement/
|
||||
└── 1_setup/
|
||||
└── model/ # "Study-specific model copy (if modified)"
|
||||
```
|
||||
|
||||
### Confusion Points
|
||||
|
||||
| Question | Current Atomizer | Spec Proposal | Issue |
|
||||
|----------|------------------|---------------|-------|
|
||||
| Where is the source model? | `1_setup/model/` | `01-models/` | Two locations |
|
||||
| Where does the solver look? | `1_setup/model/` | Depends on study | Path confusion |
|
||||
| When do I copy to study? | Always | "if modified" | Unclear trigger |
|
||||
| What if study modifies model? | Study owns it | Study copy diverges | Sync problem |
|
||||
|
||||
### Real-World Example: Hydrotech Beam
|
||||
|
||||
**Current (works):**
|
||||
```
|
||||
projects/hydrotech-beam/
|
||||
└── studies/
|
||||
└── 01_doe_landscape/
|
||||
└── 1_setup/
|
||||
└── model/
|
||||
├── Beam.prt # Study owns this
|
||||
├── Beam_fem1.fem
|
||||
└── Beam_sim1.sim
|
||||
```
|
||||
|
||||
**Proposed (spec):**
|
||||
```
|
||||
projects/hydrotech-beam/
|
||||
├── 01-models/
|
||||
│ ├── cad/Beam.prt # "Golden copy"
|
||||
│ └── ...
|
||||
└── 03-studies/
|
||||
└── 01_doe_landscape/
|
||||
└── 1_setup/
|
||||
└── model/ # Copy if modified? Or symlink?
|
||||
```
|
||||
|
||||
**Problem:** Most optimization studies DO modify the model (that's the point). So "if modified" → "always".
|
||||
|
||||
### Alternative Interpretations
|
||||
|
||||
#### Interpretation A: `01-models/` is baseline, studies always copy
|
||||
|
||||
- `01-models/` = Baseline for comparison
|
||||
- `03-studies/{NN}/1_setup/model/` = Working copy (always present)
|
||||
- Pro: Clear separation
|
||||
- Con: Disk waste (5+ studies × 500 MB models = 2.5 GB)
|
||||
|
||||
#### Interpretation B: `01-models/` is shared, studies symlink
|
||||
|
||||
- `01-models/` = Shared model files
|
||||
- `03-studies/{NN}/1_setup/model/` = Symlinks to `../../../01-models/`
|
||||
- Pro: No duplication
|
||||
- Con: NX doesn't handle symlinks well on Windows
|
||||
|
||||
#### Interpretation C: Single source of truth (current Atomizer)
|
||||
|
||||
- `01-models/` = Documentation/reference only
|
||||
- `03-studies/{NN}/1_setup/model/` = Actual working model
|
||||
- Studies start from a baseline but own their model
|
||||
- Pro: Simple, no sync issues
|
||||
- Con: No "golden copy" for comparison
|
||||
|
||||
### Recommendation
|
||||
|
||||
**Adopt Interpretation C with amendments:**
|
||||
|
||||
1. **`01-models/baseline/`** contains:
|
||||
- Baseline `.prt`, `.fem`, `.sim`
|
||||
- `BASELINE.md` with documented metrics
|
||||
- `results/` with baseline `.op2`, `.f06`
|
||||
|
||||
2. **`03-studies/{NN}/1_setup/model/`** contains:
|
||||
- Working model for THIS study (copied from baseline at creation)
|
||||
- Study owns this and can modify
|
||||
|
||||
3. **Study creation workflow:**
|
||||
```python
|
||||
def create_study(project_root, study_name):
|
||||
baseline = project_root / "01-models" / "baseline"
|
||||
study_model = project_root / "03-studies" / study_name / "1_setup" / "model"
|
||||
|
||||
# Copy baseline to study
|
||||
shutil.copytree(baseline, study_model)
|
||||
|
||||
# Study README notes: "Started from baseline (mass=X, disp=Y)"
|
||||
```
|
||||
|
||||
4. **Update spec §5 (Model File Organization):**
|
||||
- Rename `01-models/` → `01-models/baseline/`
|
||||
- Clarify: "Studies copy from baseline at creation and own their model thereafter"
|
||||
- Add warning: "Do NOT modify `01-models/baseline/` during optimization"
|
||||
|
||||
**Estimated Effort:** 2 hours (documentation update)
|
||||
|
||||
---
|
||||
|
||||
## 6. Report Auto-Generation Feasibility
|
||||
|
||||
### Spec Claims (§8.2)
|
||||
|
||||
> "After a study completes, the following can be auto-generated from `study.db` + `iteration_history.csv` + `atomizer_spec.json`"
|
||||
|
||||
**Template:**
|
||||
```markdown
|
||||
# Study Report: {study_name}
|
||||
## Configuration
|
||||
{Auto-extracted from atomizer_spec.json}
|
||||
## Results Summary
|
||||
| Metric | Best | Mean | Worst | Target |
|
||||
## Convergence
|
||||
{Auto-generated convergence plot reference}
|
||||
## Parameter Analysis
|
||||
{Top parameter importances from Optuna}
|
||||
## Best Trial Details
|
||||
...
|
||||
```
|
||||
|
||||
### Existing Capabilities
|
||||
|
||||
**File:** `optimization_engine/reporting/markdown_report.py`
|
||||
|
||||
```python
|
||||
def generate_study_report(study_dir: Path, output_file: Path):
|
||||
"""Generate markdown study report from Optuna database."""
|
||||
# ✅ Loads study.db
|
||||
# ✅ Extracts best trial, objectives, parameters
|
||||
# ✅ Generates convergence plot (matplotlib → PNG)
|
||||
# ✅ Computes parameter importance (Optuna fANOVA)
|
||||
# ✅ Writes markdown with tables
|
||||
```
|
||||
|
||||
**Example Output (M1 Mirror V14):**
|
||||
```markdown
|
||||
# Study Report: m1_mirror_adaptive_V14
|
||||
|
||||
## Study Information
|
||||
- **Algorithm**: TPE
|
||||
- **Budget**: 785 trials
|
||||
- **Objectives**: wfe_40_20_nm, wfe_60_20_nm, mfg_90_nm, mass_kg
|
||||
|
||||
## Best Trial (#743)
|
||||
| Objective | Value |
|
||||
|-----------|-------|
|
||||
| wfe_40_20_nm | 5.99 |
|
||||
| Weighted Sum | 121.72 |
|
||||
|
||||
## Parameter Importance
|
||||
| Parameter | Importance |
|
||||
|-----------|-----------|
|
||||
| whiffle_min | 0.45 |
|
||||
| lateral_inner_angle | 0.23 |
|
||||
...
|
||||
```
|
||||
|
||||
### Gaps vs Spec Template
|
||||
|
||||
| Spec Feature | Exists? | Gap |
|
||||
|--------------|---------|-----|
|
||||
| Configuration summary | ✅ | Works |
|
||||
| Results summary table | ✅ | Works |
|
||||
| Convergence plot | ✅ | Works |
|
||||
| Parameter importance | ✅ | Works (Optuna fANOVA) |
|
||||
| Feasibility breakdown | ❌ | Missing — need to parse trial states |
|
||||
| Best trial geometry export | ❌ | Missing — need NX journal to export `.step` |
|
||||
| Human annotation section | ✅ | Placeholder exists |
|
||||
|
||||
### Implementation Needs
|
||||
|
||||
#### Feature 1: Feasibility Breakdown
|
||||
|
||||
```python
|
||||
def get_feasibility_stats(study: optuna.Study) -> Dict[str, int]:
|
||||
"""Analyze trial feasibility."""
|
||||
stats = {
|
||||
'total': len(study.trials),
|
||||
'complete': 0,
|
||||
'failed': 0,
|
||||
'pruned': 0,
|
||||
'infeasible_geo': 0, # Geometric constraint violation
|
||||
'infeasible_constraint': 0 # FEA constraint violation
|
||||
}
|
||||
for trial in study.trials:
|
||||
if trial.state == optuna.trial.TrialState.COMPLETE:
|
||||
stats['complete'] += 1
|
||||
# Check user_attrs for infeasibility reason
|
||||
if trial.user_attrs.get('infeasible_reason') == 'geometry':
|
||||
stats['infeasible_geo'] += 1
|
||||
elif trial.user_attrs.get('infeasible'):
|
||||
stats['infeasible_constraint'] += 1
|
||||
elif trial.state == optuna.trial.TrialState.FAIL:
|
||||
stats['failed'] += 1
|
||||
return stats
|
||||
```
|
||||
|
||||
**Effort:** 2-4 hours
|
||||
|
||||
#### Feature 2: Best Trial Geometry Export
|
||||
|
||||
Requires NX journal to:
|
||||
1. Load best trial model from `2_iterations/trial_{N}/`
|
||||
2. Export as `.step` or `.x_t`
|
||||
3. Save to `3_results/best_trial/geometry.step`
|
||||
|
||||
**Effort:** 4-6 hours (NX journal development)
|
||||
|
||||
### Recommendation
|
||||
|
||||
**Phase 1 (Immediate):** Use existing `markdown_report.py`, add:
|
||||
- Feasibility stats (2-4 hours)
|
||||
- Template for human annotation (already exists)
|
||||
|
||||
**Phase 2 (Future):** Add:
|
||||
- Best trial geometry export (4-6 hours)
|
||||
- Automated plot embedding in markdown (2 hours)
|
||||
|
||||
**Update Spec §8.2:** Note that geometry export is manual for now.
|
||||
|
||||
**Total Estimated Effort:** 8-12 hours for full auto-generation
|
||||
|
||||
---
|
||||
|
||||
## 7. Playbooks vs Protocol Operating System
|
||||
|
||||
### Spec Proposal (§2.1, §10)
|
||||
|
||||
```
|
||||
playbooks/
|
||||
├── FIRST_RUN.md # How to run the first trial
|
||||
└── MODAL_ANALYSIS.md # How to extract frequency results
|
||||
```
|
||||
|
||||
### Existing Protocol Operating System
|
||||
|
||||
```
|
||||
docs/protocols/
|
||||
├── operations/
|
||||
│ ├── OP_01_CREATE_STUDY.md
|
||||
│ ├── OP_02_RUN_OPTIMIZATION.md
|
||||
│ ├── OP_03_MONITOR_PROGRESS.md
|
||||
│ ├── OP_04_ANALYZE_RESULTS.md
|
||||
│ └── ...
|
||||
└── system/
|
||||
└── SYS_12_EXTRACTOR_LIBRARY.md
|
||||
```
|
||||
|
||||
### Redundancy Analysis
|
||||
|
||||
| Playbook (Spec) | Protocol (Existing) | Overlap? |
|
||||
|-----------------|---------------------|----------|
|
||||
| `FIRST_RUN.md` | `OP_02_RUN_OPTIMIZATION.md` | **100%** — Same content |
|
||||
| `MODAL_ANALYSIS.md` | `SYS_12_EXTRACTOR_LIBRARY.md` (E2: Frequency) | **90%** — Extractor docs cover this |
|
||||
| `DOE_SETUP.md` (implied) | `OP_01_CREATE_STUDY.md` | **80%** — OP_01 includes sampler selection |
|
||||
|
||||
### Key Difference
|
||||
|
||||
**Protocols (POS):** Framework-level, generic, applies to all projects
|
||||
**Playbooks (Spec):** Project-specific, tailored to this exact component/study
|
||||
|
||||
**Example:**
|
||||
|
||||
**OP_02 (Protocol):**
|
||||
```markdown
|
||||
# OP_02: Run Optimization
|
||||
|
||||
## Quick Start
|
||||
```bash
|
||||
python run_optimization.py --start
|
||||
```
|
||||
|
||||
## Monitoring
|
||||
Check progress: `OP_03`
|
||||
```
|
||||
|
||||
**FIRST_RUN.md (Playbook):**
|
||||
```markdown
|
||||
# First Run: ThermoShield Bracket
|
||||
|
||||
## Specific to THIS project
|
||||
- Solver timeout: 600s (bracket solves fast)
|
||||
- Watch for hole overlap infeasibility
|
||||
- First 10 trials are DOE (LHS)
|
||||
|
||||
## Run
|
||||
```bash
|
||||
cd 03-studies/01_doe_landscape
|
||||
python run_optimization.py --start
|
||||
```
|
||||
|
||||
## Expected behavior
|
||||
- Trial 1-10: ~30 seconds each (DOE)
|
||||
- Trial 11+: TPE kicks in
|
||||
- If mass > 0.8 kg → infeasible (auto-pruned)
|
||||
```
|
||||
|
||||
### Recommendation
|
||||
|
||||
**Keep both, clarify distinction:**
|
||||
|
||||
1. **Playbooks** = Project-specific operational guides
|
||||
- Location: `{project}/playbooks/`
|
||||
- Content: Project quirks, expected behavior, troubleshooting for THIS project
|
||||
- Examples: "On this model, tip displacement is at node 5161"
|
||||
|
||||
2. **Protocols** = Generic framework operations
|
||||
- Location: `Atomizer/docs/protocols/`
|
||||
- Content: How Atomizer works in general
|
||||
- Examples: "To monitor, use OP_03"
|
||||
|
||||
3. **Update spec §2.1:**
|
||||
- Rename `playbooks/` → `playbooks/` (keep name)
|
||||
- Add description: "Project-specific operational notes. See `docs/protocols/` for generic Atomizer operations."
|
||||
|
||||
4. **Create template playbooks:**
|
||||
- `playbooks/FIRST_RUN.md` → Template with placeholders
|
||||
- `playbooks/TROUBLESHOOTING.md` → Project-specific issues log
|
||||
- `playbooks/EXTRACTOR_NOTES.md` → Custom extractor usage for this project
|
||||
|
||||
**Estimated Effort:** 2 hours (documentation)
|
||||
|
||||
---
|
||||
|
||||
## Summary of Recommendations
|
||||
|
||||
### Priority 0: Blocking Issues (Must Fix Before Adoption)
|
||||
|
||||
| Issue | Change | Effort | Owner |
|
||||
|-------|--------|--------|-------|
|
||||
| Path compatibility | Add project root awareness to `OptimizationRunner` | 2-4h | Dev |
|
||||
| Path resolution | Update `AtomizerSpec.resolve_paths()` | 1-2h | Dev |
|
||||
| Auto-extraction claims | Revise §7.2 to reflect reality | 1h | Manager |
|
||||
|
||||
**Total P0 Effort:** 4-7 hours
|
||||
|
||||
### Priority 1: Important Improvements
|
||||
|
||||
| Issue | Change | Effort | Owner |
|
||||
|-------|--------|--------|-------|
|
||||
| Study lifecycle gaps | Add MONITOR, EXPORT, TROUBLESHOOT stages | 2-4h | Manager |
|
||||
| Model organization clarity | Amend §5 to clarify baseline vs working copy | 2h | Manager |
|
||||
| Schema redundancy | Make `.atomizer/project.json` auto-generated | 4-6h | Dev |
|
||||
| Report feasibility stats | Add feasibility breakdown to report generator | 2-4h | Dev |
|
||||
|
||||
**Total P1 Effort:** 10-16 hours
|
||||
|
||||
### Priority 2: Nice-to-Have
|
||||
|
||||
| Issue | Change | Effort | Owner |
|
||||
|-------|--------|--------|-------|
|
||||
| Full auto-extraction | Implement missing extractors (mesh, BC, loads) | 30-46h | Dev |
|
||||
| Best trial geometry export | NX journal for `.step` export | 4-6h | Dev |
|
||||
| Playbook templates | Create template playbooks | 2h | Manager |
|
||||
|
||||
**Total P2 Effort:** 36-54 hours
|
||||
|
||||
---
|
||||
|
||||
## Implementation Roadmap
|
||||
|
||||
### Phase 1: Core Compatibility (Week 1)
|
||||
- [ ] Add project root awareness to optimization engine (4-7h)
|
||||
- [ ] Update spec §7.2 auto-extraction claims (1h)
|
||||
- [ ] Amend spec §6 lifecycle to include missing stages (2h)
|
||||
- [ ] Clarify spec §5 model organization (2h)
|
||||
|
||||
**Deliverable:** Spec v1.1 (compatible with existing Atomizer)
|
||||
|
||||
### Phase 2: Enhanced Integration (Week 2-3)
|
||||
- [ ] Implement `.atomizer/project.json` auto-generation (4-6h)
|
||||
- [ ] Add feasibility stats to report generator (2-4h)
|
||||
- [ ] Create playbook templates (2h)
|
||||
- [ ] Test migration of Hydrotech Beam to project standard (4h)
|
||||
|
||||
**Deliverable:** Working project-standard template + migration tool
|
||||
|
||||
### Phase 3: Advanced Features (Future)
|
||||
- [ ] Implement missing auto-extractors (30-46h)
|
||||
- [ ] Best trial geometry export (4-6h)
|
||||
- [ ] Dashboard integration for project view (TBD)
|
||||
|
||||
**Deliverable:** Full auto-population capability
|
||||
|
||||
---
|
||||
|
||||
## Technical Amendments to Specification
|
||||
|
||||
### Amendment 1: Section 1.2 Decision D3
|
||||
|
||||
**Current:**
|
||||
> "Impact on Atomizer code: The optimization engine needs a `--project-root` flag or config entry pointing to the project folder."
|
||||
|
||||
**Amend to:**
|
||||
> "Impact on Atomizer code:
|
||||
> 1. Add `project_root` parameter to `OptimizationRunner.__init__()`
|
||||
> 2. Update `AtomizerSpec.resolve_paths()` to handle `../../01-models/` references
|
||||
> 3. Add `--project-root` CLI flag to `run_optimization.py` template
|
||||
> 4. Maintain backward compatibility with legacy `studies/{name}/1_setup/model/` layout
|
||||
> **Estimated implementation:** 4-7 hours."
|
||||
|
||||
### Amendment 2: Section 5 (Model File Organization)
|
||||
|
||||
**Add new subsection §5.4: Model Location Decision Tree**
|
||||
|
||||
```markdown
|
||||
### 5.4 Model Location Decision Tree
|
||||
|
||||
**Q: Where should I put the model files?**
|
||||
|
||||
| Scenario | Location | Rationale |
|
||||
|----------|----------|-----------|
|
||||
| Single study, no baseline comparison | `03-studies/01_study/1_setup/model/` | Simple, no duplication |
|
||||
| Multiple studies from same baseline | `01-models/baseline/` + copy to studies | Baseline preserved for comparison |
|
||||
| Studies modify model parametrically | `03-studies/{NN}/1_setup/model/` (copy) | Each study owns its model |
|
||||
| Studies share exact same model | `01-models/` + symlink (advanced) | Disk savings — NX compatibility risk |
|
||||
|
||||
**Default recommendation:** Copy baseline to each study at creation.
|
||||
```
|
||||
|
||||
### Amendment 3: Section 7.2 (Auto-Extraction Feasibility)
|
||||
|
||||
**Replace current table with:**
|
||||
|
||||
| Data | Auto-Extraction Status | Manual Fallback |
|
||||
|------|----------------------|-----------------|
|
||||
| All NX expressions | ✅ **Fully automatic** (`introspect_part.py`) | N/A |
|
||||
| Mass properties | ✅ **Fully automatic** (NXOpen MeasureManager) | N/A |
|
||||
| Material names | ✅ **Fully automatic** (part introspection) | N/A |
|
||||
| Material properties (E, ν, ρ) | ⚠️ **Partial** — requires `physicalmateriallibrary.xml` parser (4-6h dev) | Manual entry in `02-kb/design/materials/{Material}.md` |
|
||||
| Mesh statistics (element/node count) | ⚠️ **Requires implementation** — pyNastran BDF parser (2-4h dev) | Manual entry from NX Pre/Post |
|
||||
| Boundary conditions | ❌ **Not implemented** — requires `.sim` XML parser (8-12h dev) | Manual documentation in `02-kb/analysis/boundary-conditions/` |
|
||||
| Load cases | ❌ **Not implemented** — requires `.sim` XML parser (8-12h dev) | Manual documentation in `02-kb/analysis/loads/` |
|
||||
| Baseline FEA results | ✅ **Fully automatic** (pyNastran `.op2` parsing) | N/A |
|
||||
| Design variable candidates | ✅ **Fully automatic** (expression analysis + confidence scoring) | N/A |
|
||||
|
||||
**Total implementation effort for missing extractors:** 22-34 hours
|
||||
|
||||
### Amendment 4: Section 6.1 (Study Lifecycle)
|
||||
|
||||
**Replace §6.1 lifecycle diagram with:**
|
||||
|
||||
```
|
||||
PLAN → CONFIGURE → VALIDATE → RUN → MONITOR → ANALYZE → REPORT → FEED-BACK
|
||||
│ │ │ │ │ │ │ │
|
||||
│ │ │ │ │ │ │ └─ Update KB (OP_LAC)
|
||||
│ │ │ │ │ │ └─ Generate STUDY_REPORT.md (OP_08)
|
||||
│ │ │ │ │ └─ Post-process, visualize (OP_04)
|
||||
│ │ │ │ └─ Real-time tracking (OP_03)
|
||||
│ │ │ └─ Execute trials (OP_02)
|
||||
│ │ └─ Preflight checks (OP_01)
|
||||
│ └─ Write atomizer_spec.json
|
||||
└─ Formulate hypothesis from previous study
|
||||
|
||||
Optional branches:
|
||||
RUN ──→ EXPORT ──→ Train neural surrogate (OP_05)
|
||||
Any stage ──→ TROUBLESHOOT ──→ Fix + resume (OP_06)
|
||||
```
|
||||
|
||||
### Amendment 5: Section 4.1 (DECISIONS.md vs .atomizer/project.json)
|
||||
|
||||
**Add clarification:**
|
||||
|
||||
> **Note on `.atomizer/project.json`:**
|
||||
> This file is **auto-generated** from study specs and `PROJECT.md` frontmatter. Do NOT hand-edit. If you need to update project metadata, edit `PROJECT.md` frontmatter and regenerate via:
|
||||
> ```bash
|
||||
> atomizer project refresh-metadata
|
||||
> ```
|
||||
|
||||
---
|
||||
|
||||
## Compatibility Assessment Matrix
|
||||
|
||||
| Component | Works As-Is | Needs Code | Needs Doc | Priority |
|
||||
|-----------|------------|-----------|-----------|----------|
|
||||
| Numbered folders (`00-context/`, `01-models/`) | ✅ | ❌ | ❌ | - |
|
||||
| `PROJECT.md` / `AGENT.md` entry points | ✅ | ❌ | ❌ | - |
|
||||
| `DECISIONS.md` format | ✅ | ❌ | ❌ | - |
|
||||
| `02-kb/` structure | ✅ | ❌ | ⚠️ Minor | P1 |
|
||||
| `03-studies/{NN}_{slug}/` naming | ✅ | ❌ | ❌ | - |
|
||||
| `atomizer_spec.json` in studies | ✅ | ⚠️ Path resolution | ❌ | **P0** |
|
||||
| `01-models/` model source | ❌ | ✅ Path refactor | ✅ Clarify | **P0** |
|
||||
| Auto-extraction claims | ❌ | ⚠️ Optional | ✅ Revise | **P0** |
|
||||
| `.atomizer/project.json` | ❌ | ✅ Auto-gen | ✅ Note | P1 |
|
||||
| `playbooks/` vs protocols | ⚠️ Overlap | ❌ | ✅ Clarify | P2 |
|
||||
| Study lifecycle stages | ⚠️ Gaps | ❌ | ✅ Amend | P1 |
|
||||
| `STUDY_REPORT.md` auto-gen | ⚠️ Partial | ⚠️ Feasibility stats | ❌ | P1 |
|
||||
|
||||
**Legend:**
|
||||
- ✅ Good to go
|
||||
- ⚠️ Needs attention
|
||||
- ❌ Blocking issue
|
||||
|
||||
---
|
||||
|
||||
## Final Verdict
|
||||
|
||||
**The Atomizer Project Standard v1.0 is APPROVED with amendments.**
|
||||
|
||||
**Technical Feasibility:** ✅ High — Core concepts are sound
|
||||
**Implementation Complexity:** ⚠️ Medium — Requires 4-7 hours of core changes + 10-16 hours of improvements
|
||||
**Backward Compatibility:** ✅ Maintainable — Can support both legacy and project-standard layouts
|
||||
**Documentation Quality:** ✅ Excellent — Well-researched and thorough
|
||||
**Auto-Extraction Claims:** ⚠️ Overstated — Revise to reflect current capabilities
|
||||
|
||||
**Recommended Action:**
|
||||
1. Apply P0 amendments (path compatibility, auto-extraction reality check) → **Spec v1.1**
|
||||
2. Implement P0 code changes (4-7 hours)
|
||||
3. Test with Hydrotech Beam migration
|
||||
4. Release project-standard template
|
||||
5. Plan P1/P2 features for future sprints
|
||||
|
||||
**Risk Assessment:**
|
||||
- **Low Risk:** Entry point design (`PROJECT.md`, `AGENT.md`), KB structure, naming conventions
|
||||
- **Medium Risk:** Path refactoring (backward compat required), schema redundancy (sync issues)
|
||||
- **High Risk:** Auto-extraction promises (user disappointment if not delivered)
|
||||
|
||||
**Mitigation:** Adopt hybrid mode (support both layouts), phase auto-extraction, clear documentation of what's automatic vs manual.
|
||||
|
||||
---
|
||||
|
||||
*End of Technical Review*
|
||||
|
||||
**Next Steps:** Deliver to Manager for CEO review integration.
|
||||
Reference in New Issue
Block a user