feat: Major update with validators, skills, dashboard, and docs reorganization

- Add validation framework (config, model, results, study validators) - Add Claude Code skills (create-study, run-optimization, generate-report, troubleshoot, analyze-model) - Add Atomizer Dashboard (React frontend + FastAPI backend) - Reorganize docs into structured directories (00-09) - Add neural surrogate modules and training infrastructure - Add multi-objective optimization support 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-25 19:23:58 -05:00
parent 74a92803b7
commit e3bdb08a22
155 changed files with 52729 additions and 37 deletions
--- a/docs/06_PROTOCOLS_DETAILED/protocol_13_implementation_guide.md
+++ b/docs/06_PROTOCOLS_DETAILED/protocol_13_implementation_guide.md
@@ -0,0 +1,425 @@
+# Implementation Guide: Protocol 13 - Real-Time Tracking
+
+**Date:** 2025-11-21
+**Status:** 🚧 IN PROGRESS
+**Priority:** P0 - CRITICAL
+
+## What's Done ✅
+
+1. **Created [`realtime_tracking.py`](../optimization_engine/realtime_tracking.py)**
+   - `RealtimeTrackingCallback` class
+   - Writes JSON files after EVERY trial (atomic writes)
+   - Files: optimizer_state.json, strategy_history.json, trial_log.json, landscape_snapshot.json, confidence_history.json
+
+2. **Fixed Multi-Objective Strategy (Protocol 12)**
+   - Modified [`strategy_selector.py`](../optimization_engine/strategy_selector.py)
+   - Added `_recommend_multiobjective_strategy()` method
+   - Multi-objective: Random (8 trials) → TPE with multivariate
+
+## What's Needed ⚠️
+
+### Step 1: Integrate Callback into IntelligentOptimizer
+
+**File:** [`optimization_engine/intelligent_optimizer.py`](../optimization_engine/intelligent_optimizer.py)
+
+**Line 48 - Add import:**
+```python
+from optimization_engine.adaptive_characterization import CharacterizationStoppingCriterion
+from optimization_engine.realtime_tracking import create_realtime_callback  # ADD THIS
+```
+
+**Line ~90 in `__init__()` - Create callback:**
+```python
+def __init__(self, study_name: str, study_dir: Path, config: Dict, verbose: bool = True):
+    # ... existing init code ...
+
+    # Create realtime tracking callback (Protocol 13)
+    self.realtime_callback = create_realtime_callback(
+        tracking_dir=self.tracking_dir,
+        optimizer_ref=self,
+        verbose=self.verbose
+    )
+```
+
+**Find ALL `study.optimize()` calls and add callback:**
+
+Search for: `self.study.optimize(`
+
+Replace pattern:
+```python
+# BEFORE:
+self.study.optimize(objective_function, n_trials=check_interval)
+
+# AFTER:
+self.study.optimize(
+    objective_function,
+    n_trials=check_interval,
+    callbacks=[self.realtime_callback]
+)
+```
+
+**Locations to fix (approximate line numbers):**
+- Line ~190: Characterization phase
+- Line ~230: Optimization phase (multiple locations)
+- Line ~260: Refinement phase
+- Line ~380: Fallback optimization
+
+**CRITICAL:** EVERY `study.optimize()` call must include `callbacks=[self.realtime_callback]`
+
+### Step 2: Test Realtime Tracking
+
+```bash
+# Clear old results
+cd studies/bracket_stiffness_optimization_V2
+del /Q 2_results\study.db
+rd /S /Q 2_results\intelligent_optimizer
+
+# Run with new code
+python -B run_optimization.py --trials 10
+
+# Verify files appear IMMEDIATELY after each trial
+dir 2_results\intelligent_optimizer
+# Should see:
+# - optimizer_state.json
+# - strategy_history.json
+# - trial_log.json
+# - landscape_snapshot.json
+# - confidence_history.json
+
+# Check file updates in real-time
+python -c "import json; print(json.load(open('2_results/intelligent_optimizer/trial_log.json'))[-1])"
+```
+
+---
+
+## Dashboard Implementation Plan
+
+### Backend API Endpoints (Python/FastAPI)
+
+**File:** [`atomizer-dashboard/backend/api/routes/optimization.py`](../atomizer-dashboard/backend/api/routes/optimization.py)
+
+**Add new endpoints:**
+
+```python
+@router.get("/studies/{study_id}/metadata")
+async def get_study_metadata(study_id: str):
+    """Read optimization_config.json for objectives, design vars, units."""
+    study_dir = find_study_dir(study_id)
+    config_file = study_dir / "optimization_config.json"
+
+    with open(config_file) as f:
+        config = json.load(f)
+
+    return {
+        "objectives": config["objectives"],
+        "design_variables": config["design_variables"],
+        "constraints": config.get("constraints", []),
+        "study_name": config["study_name"]
+    }
+
+@router.get("/studies/{study_id}/optimizer-state")
+async def get_optimizer_state(study_id: str):
+    """Read realtime optimizer state from intelligent_optimizer/."""
+    study_dir = find_study_dir(study_id)
+    state_file = study_dir / "2_results/intelligent_optimizer/optimizer_state.json"
+
+    if not state_file.exists():
+        return {"available": False}
+
+    with open(state_file) as f:
+        state = json.load(f)
+
+    return {"available": True, **state}
+
+@router.get("/studies/{study_id}/pareto-front")
+async def get_pareto_front(study_id: str):
+    """Get Pareto-optimal solutions for multi-objective studies."""
+    study_dir = find_study_dir(study_id)
+    db_path = study_dir / "2_results/study.db"
+
+    storage = optuna.storages.RDBStorage(f"sqlite:///{db_path}")
+    study = optuna.load_study(study_name=study_id, storage=storage)
+
+    if len(study.directions) == 1:
+        return {"is_multi_objective": False}
+
+    pareto_trials = study.best_trials
+
+    return {
+        "is_multi_objective": True,
+        "pareto_front": [
+            {
+                "trial_number": t.number,
+                "values": t.values,
+                "params": t.params,
+                "user_attrs": dict(t.user_attrs)
+            }
+            for t in pareto_trials
+        ]
+    }
+```
+
+### Frontend Components (React/TypeScript)
+
+**1. Optimizer Panel Component**
+
+**File:** `atomizer-dashboard/frontend/src/components/OptimizerPanel.tsx` (CREATE NEW)
+
+```typescript
+import { useEffect, useState } from 'react';
+import { Card } from './Card';
+
+interface OptimizerState {
+  available: boolean;
+  current_phase?: string;
+  current_strategy?: string;
+  trial_number?: number;
+  total_trials?: number;
+  latest_recommendation?: {
+    strategy: string;
+    confidence: number;
+    reasoning: string;
+  };
+}
+
+export function OptimizerPanel({ studyId }: { studyId: string }) {
+  const [state, setState] = useState<OptimizerState | null>(null);
+
+  useEffect(() => {
+    const fetchState = async () => {
+      const res = await fetch(`/api/optimization/studies/${studyId}/optimizer-state`);
+      const data = await res.json();
+      setState(data);
+    };
+
+    fetchState();
+    const interval = setInterval(fetchState, 1000); // Update every second
+    return () => clearInterval(interval);
+  }, [studyId]);
+
+  if (!state?.available) {
+    return null;
+  }
+
+  return (
+    <Card title="Intelligent Optimizer Status">
+      <div className="space-y-4">
+        {/* Phase */}
+        <div>
+          <div className="text-sm text-dark-300">Phase</div>
+          <div className="text-lg font-semibold text-primary-400">
+            {state.current_phase || 'Unknown'}
+          </div>
+        </div>
+
+        {/* Strategy */}
+        <div>
+          <div className="text-sm text-dark-300">Current Strategy</div>
+          <div className="text-lg font-semibold text-blue-400">
+            {state.current_strategy?.toUpperCase() || 'Unknown'}
+          </div>
+        </div>
+
+        {/* Progress */}
+        <div>
+          <div className="text-sm text-dark-300">Progress</div>
+          <div className="text-lg">
+            {state.trial_number} / {state.total_trials} trials
+          </div>
+          <div className="w-full bg-dark-500 rounded-full h-2 mt-2">
+            <div
+              className="bg-primary-400 h-2 rounded-full transition-all"
+              style={{
+                width: `${((state.trial_number || 0) / (state.total_trials || 1)) * 100}%`
+              }}
+            />
+          </div>
+        </div>
+
+        {/* Confidence */}
+        {state.latest_recommendation && (
+          <div>
+            <div className="text-sm text-dark-300">Confidence</div>
+            <div className="flex items-center gap-2">
+              <div className="flex-1 bg-dark-500 rounded-full h-2">
+                <div
+                  className="bg-green-400 h-2 rounded-full transition-all"
+                  style={{
+                    width: `${state.latest_recommendation.confidence * 100}%`
+                  }}
+                />
+              </div>
+              <span className="text-sm font-mono">
+                {(state.latest_recommendation.confidence * 100).toFixed(0)}%
+              </span>
+            </div>
+          </div>
+        )}
+
+        {/* Reasoning */}
+        {state.latest_recommendation && (
+          <div>
+            <div className="text-sm text-dark-300">Reasoning</div>
+            <div className="text-sm text-dark-100 mt-1">
+              {state.latest_recommendation.reasoning}
+            </div>
+          </div>
+        )}
+      </div>
+    </Card>
+  );
+}
+```
+
+**2. Pareto Front Plot**
+
+**File:** `atomizer-dashboard/frontend/src/components/ParetoPlot.tsx` (CREATE NEW)
+
+```typescript
+import { ScatterChart, Scatter, XAxis, YAxis, CartesianGrid, Tooltip, Cell, ResponsiveContainer } from 'recharts';
+
+interface ParetoData {
+  trial_number: number;
+  values: [number, number];
+  params: Record<string, number>;
+  constraint_satisfied?: boolean;
+}
+
+export function ParetoPlot({ paretoData, objectives }: {
+  paretoData: ParetoData[];
+  objectives: Array<{ name: string; unit?: string }>;
+}) {
+  if (paretoData.length === 0) {
+    return (
+      <div className="h-64 flex items-center justify-center text-dark-300">
+        No Pareto front data yet
+      </div>
+    );
+  }
+
+  const data = paretoData.map(trial => ({
+    x: trial.values[0],
+    y: trial.values[1],
+    trial_number: trial.number,
+    feasible: trial.constraint_satisfied !== false
+  }));
+
+  return (
+    <ResponsiveContainer width="100%" height={400}>
+      <ScatterChart>
+        <CartesianGrid strokeDasharray="3 3" stroke="#334155" />
+        <XAxis
+          type="number"
+          dataKey="x"
+          name={objectives[0]?.name || 'Objective 1'}
+          stroke="#94a3b8"
+          label={{
+            value: `${objectives[0]?.name || 'Objective 1'} ${objectives[0]?.unit || ''}`.trim(),
+            position: 'insideBottom',
+            offset: -5,
+            fill: '#94a3b8'
+          }}
+        />
+        <YAxis
+          type="number"
+          dataKey="y"
+          name={objectives[1]?.name || 'Objective 2'}
+          stroke="#94a3b8"
+          label={{
+            value: `${objectives[1]?.name || 'Objective 2'} ${objectives[1]?.unit || ''}`.trim(),
+            angle: -90,
+            position: 'insideLeft',
+            fill: '#94a3b8'
+          }}
+        />
+        <Tooltip
+          contentStyle={{ backgroundColor: '#1e293b', border: 'none', borderRadius: '8px' }}
+          labelStyle={{ color: '#e2e8f0' }}
+        />
+        <Scatter name="Pareto Front" data={data}>
+          {data.map((entry, index) => (
+            <Cell
+              key={`cell-${index}`}
+              fill={entry.feasible ? '#10b981' : '#ef4444'}
+              r={entry.feasible ? 6 : 4}
+            />
+          ))}
+        </Scatter>
+      </ScatterChart>
+    </ResponsiveContainer>
+  );
+}
+```
+
+**3. Update Dashboard.tsx**
+
+**File:** [`atomizer-dashboard/frontend/src/pages/Dashboard.tsx`](../atomizer-dashboard/frontend/src/pages/Dashboard.tsx)
+
+Add imports at top:
+```typescript
+import { OptimizerPanel } from '../components/OptimizerPanel';
+import { ParetoPlot } from '../components/ParetoPlot';
+```
+
+Add new state:
+```typescript
+const [studyMetadata, setStudyMetadata] = useState(null);
+const [paretoFront, setParetoFront] = useState([]);
+```
+
+Fetch metadata when study selected:
+```typescript
+useEffect(() => {
+  if (selectedStudyId) {
+    fetch(`/api/optimization/studies/${selectedStudyId}/metadata`)
+      .then(res => res.json())
+      .then(setStudyMetadata);
+
+    fetch(`/api/optimization/studies/${selectedStudyId}/pareto-front`)
+      .then(res => res.json())
+      .then(data => {
+        if (data.is_multi_objective) {
+          setParetoFront(data.pareto_front);
+        }
+      });
+  }
+}, [selectedStudyId]);
+```
+
+Add components to layout:
+```typescript
+{/* Add after metrics grid */}
+<div className="grid grid-cols-2 gap-6 mb-6">
+  <OptimizerPanel studyId={selectedStudyId} />
+  {paretoFront.length > 0 && (
+    <Card title="Pareto Front">
+      <ParetoPlot
+        paretoData={paretoFront}
+        objectives={studyMetadata?.objectives || []}
+      />
+    </Card>
+  )}
+</div>
+```
+
+---
+
+## Testing Checklist
+
+- [ ] Realtime callback writes files after EVERY trial
+- [ ] optimizer_state.json updates in real-time
+- [ ] Dashboard shows optimizer panel with live updates
+- [ ] Pareto front appears for multi-objective studies
+- [ ] Units are dynamic (read from config)
+- [ ] Multi-objective strategy switches from random → TPE after 8 trials
+
+---
+
+## Next Steps
+
+1. Integrate callback into IntelligentOptimizer (Steps above)
+2. Implement backend API endpoints
+3. Create frontend components
+4. Test end-to-end with bracket study
+5. Document as Protocol 13
+