feat: Add TrialManager and DashboardDB for unified trial management
- Add TrialManager (trial_manager.py) for consistent trial_NNNN naming - Add DashboardDB (dashboard_db.py) for Optuna-compatible database schema - Update CLAUDE.md with trial management documentation - Update ATOMIZER_CONTEXT.md with v1.8 trial system - Update cheatsheet v2.2 with new utilities - Update SYS_14 protocol to v2.3 with TrialManager integration - Add LAC learnings for trial management patterns - Add archive/README.md for deprecated code policy Key principles: - Trial numbers NEVER reset (monotonic) - Folders NEVER get overwritten - Database always synced with filesystem - Surrogate predictions are NOT trials (only FEA results) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
@@ -106,17 +106,21 @@ studies/
|
||||
studies/{geometry_type}/{study_name}/
|
||||
├── optimization_config.json # Problem definition
|
||||
├── run_optimization.py # FEA optimization script
|
||||
├── run_nn_optimization.py # Neural acceleration (optional)
|
||||
├── run_turbo_optimization.py # GNN-Turbo acceleration (optional)
|
||||
├── README.md # MANDATORY documentation
|
||||
├── STUDY_REPORT.md # Results template
|
||||
├── 1_setup/
|
||||
│ ├── optimization_config.json # Config copy for reference
|
||||
│ └── model/
|
||||
│ ├── Model.prt # NX part file
|
||||
│ ├── Model_sim1.sim # NX simulation
|
||||
│ └── Model_fem1.fem # FEM definition
|
||||
├── 2_iterations/ # FEA trial folders (iter1, iter2, ...)
|
||||
├── 2_iterations/ # FEA trial folders (trial_NNNN/)
|
||||
│ ├── trial_0001/ # Zero-padded, NEVER reset
|
||||
│ ├── trial_0002/
|
||||
│ └── ...
|
||||
├── 3_results/
|
||||
│ ├── study.db # Optuna database
|
||||
│ ├── study.db # Optuna-compatible database
|
||||
│ ├── optimization.log # Logs
|
||||
│ └── turbo_report.json # NN results (if run)
|
||||
└── 3_insights/ # Study Insights (SYS_16)
|
||||
@@ -435,11 +439,68 @@ python -m optimization_engine.auto_doc templates
|
||||
|
||||
---
|
||||
|
||||
## Trial Management System (v2.3)
|
||||
|
||||
New unified trial management ensures consistency across all optimization methods:
|
||||
|
||||
### Key Components
|
||||
|
||||
| Component | Path | Purpose |
|
||||
|-----------|------|---------|
|
||||
| `TrialManager` | `optimization_engine/utils/trial_manager.py` | Unified trial folder + DB management |
|
||||
| `DashboardDB` | `optimization_engine/utils/dashboard_db.py` | Optuna-compatible database wrapper |
|
||||
|
||||
### Trial Naming Convention
|
||||
|
||||
```
|
||||
2_iterations/
|
||||
├── trial_0001/ # Zero-padded, monotonically increasing
|
||||
├── trial_0002/ # NEVER reset, NEVER overwritten
|
||||
├── trial_0003/
|
||||
└── ...
|
||||
```
|
||||
|
||||
**Key principles**:
|
||||
- Trial numbers **NEVER reset** (monotonically increasing)
|
||||
- Folders **NEVER get overwritten**
|
||||
- Database is always in sync with filesystem
|
||||
- Surrogate predictions (5K) are NOT trials - only FEA results
|
||||
|
||||
### Usage
|
||||
|
||||
```python
|
||||
from optimization_engine.utils.trial_manager import TrialManager
|
||||
|
||||
tm = TrialManager(study_dir)
|
||||
|
||||
# Start new trial
|
||||
trial = tm.new_trial(params={'rib_thickness': 10.5})
|
||||
|
||||
# After FEA completes
|
||||
tm.complete_trial(
|
||||
trial_number=trial['trial_number'],
|
||||
objectives={'wfe_40_20': 5.63, 'mass_kg': 118.67},
|
||||
weighted_sum=42.5,
|
||||
is_feasible=True
|
||||
)
|
||||
```
|
||||
|
||||
### Database Schema (Optuna-Compatible)
|
||||
|
||||
The `DashboardDB` class creates Optuna-compatible schema for dashboard integration:
|
||||
- `trials` - Main trial records with state, datetime, value
|
||||
- `trial_values` - Objective values (supports multiple objectives)
|
||||
- `trial_params` - Design parameter values
|
||||
- `trial_user_attributes` - Metadata (source, solve_time, etc.)
|
||||
- `studies` - Study metadata (directions, name)
|
||||
|
||||
---
|
||||
|
||||
## Version Info
|
||||
|
||||
| Component | Version | Last Updated |
|
||||
|-----------|---------|--------------|
|
||||
| ATOMIZER_CONTEXT | 1.7 | 2025-12-20 |
|
||||
| ATOMIZER_CONTEXT | 1.8 | 2025-12-28 |
|
||||
| BaseOptimizationRunner | 1.0 | 2025-12-07 |
|
||||
| GenericSurrogate | 1.0 | 2025-12-07 |
|
||||
| Study State Detector | 1.0 | 2025-12-07 |
|
||||
@@ -452,6 +513,9 @@ python -m optimization_engine.auto_doc templates
|
||||
| Subagent Commands | 1.0 | 2025-12-07 |
|
||||
| FEARunner Pattern | 1.0 | 2025-12-12 |
|
||||
| Study Insights | 1.0 | 2025-12-20 |
|
||||
| TrialManager | 1.0 | 2025-12-28 |
|
||||
| DashboardDB | 1.0 | 2025-12-28 |
|
||||
| GNN-Turbo System | 2.3 | 2025-12-28 |
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -1,19 +1,21 @@
|
||||
---
|
||||
skill_id: SKILL_001
|
||||
version: 2.1
|
||||
last_updated: 2025-12-22
|
||||
version: 2.2
|
||||
last_updated: 2025-12-28
|
||||
type: reference
|
||||
code_dependencies:
|
||||
- optimization_engine/extractors/__init__.py
|
||||
- optimization_engine/method_selector.py
|
||||
- optimization_engine/utils/trial_manager.py
|
||||
- optimization_engine/utils/dashboard_db.py
|
||||
requires_skills:
|
||||
- SKILL_000
|
||||
---
|
||||
|
||||
# Atomizer Quick Reference Cheatsheet
|
||||
|
||||
**Version**: 2.1
|
||||
**Updated**: 2025-12-22
|
||||
**Version**: 2.2
|
||||
**Updated**: 2025-12-28
|
||||
**Purpose**: Rapid lookup for common operations. "I want X → Use Y"
|
||||
|
||||
---
|
||||
@@ -406,3 +408,75 @@ class FEARunner:
|
||||
**Reference implementations**:
|
||||
- `studies/m1_mirror_adaptive_V14/run_optimization.py`
|
||||
- `studies/m1_mirror_adaptive_V15/run_optimization.py`
|
||||
|
||||
---
|
||||
|
||||
## Trial Management Utilities
|
||||
|
||||
### TrialManager - Unified Trial Folder + DB Management
|
||||
|
||||
```python
|
||||
from optimization_engine.utils.trial_manager import TrialManager
|
||||
|
||||
tm = TrialManager(study_dir)
|
||||
|
||||
# Start new trial (creates folder, saves params)
|
||||
trial = tm.new_trial(
|
||||
params={'rib_thickness': 10.5, 'mirror_face_thickness': 17.0},
|
||||
source="turbo",
|
||||
metadata={'turbo_batch': 1, 'predicted_ws': 42.0}
|
||||
)
|
||||
# Returns: {'trial_id': 47, 'trial_number': 47, 'folder_path': Path(...)}
|
||||
|
||||
# After FEA completes
|
||||
tm.complete_trial(
|
||||
trial_number=trial['trial_number'],
|
||||
objectives={'wfe_40_20': 5.63, 'mass_kg': 118.67},
|
||||
weighted_sum=42.5,
|
||||
is_feasible=True
|
||||
)
|
||||
|
||||
# Mark failed trial
|
||||
tm.fail_trial(trial_number=47, error="NX solver timeout")
|
||||
```
|
||||
|
||||
### DashboardDB - Optuna-Compatible Database
|
||||
|
||||
```python
|
||||
from optimization_engine.utils.dashboard_db import DashboardDB, convert_custom_to_optuna
|
||||
|
||||
# Create new dashboard-compatible database
|
||||
db = DashboardDB(db_path, study_name="my_study")
|
||||
|
||||
# Log a trial
|
||||
trial_id = db.log_trial(
|
||||
params={'rib_thickness': 10.5},
|
||||
objectives={'wfe_40_20': 5.63, 'mass_kg': 118.67},
|
||||
weighted_sum=42.5,
|
||||
is_feasible=True,
|
||||
state="COMPLETE"
|
||||
)
|
||||
|
||||
# Mark best trial
|
||||
db.mark_best(trial_id)
|
||||
|
||||
# Get summary
|
||||
summary = db.get_summary()
|
||||
|
||||
# Convert existing custom database to Optuna format
|
||||
convert_custom_to_optuna(db_path, study_name)
|
||||
```
|
||||
|
||||
### Trial Naming Convention
|
||||
|
||||
```
|
||||
2_iterations/
|
||||
├── trial_0001/ # Zero-padded, monotonically increasing
|
||||
├── trial_0002/ # NEVER reset, NEVER overwritten
|
||||
└── trial_0003/
|
||||
```
|
||||
|
||||
**Key principles**:
|
||||
- Trial numbers **NEVER reset** across study lifetime
|
||||
- Surrogate predictions (5K per batch) are NOT logged as trials
|
||||
- Only FEA-validated results become trials
|
||||
|
||||
Reference in New Issue
Block a user