Atomizer/docs/plans/RALPH_LOOP_CANVAS_STUDY_SYNC.md

# RALPH LOOP: Canvas-Study Synchronization Overhaul

**Date**: January 16, 2026
**Status**: 🟢 COMPLETED
**Priority**: CRITICAL
**Completion Date**: January 16, 2026

---

## Executive Summary

The Canvas Builder and actual optimization studies are fundamentally disconnected. When a user loads a study, the canvas doesn't reflect the true optimization pipeline. Claude chat has zero awareness of the canvas state, making it useless for canvas-based interactions.

---

## Part 1: Problem Analysis

### 1.1 Core Issues Identified

| # | Issue | Severity | Impact |
|---|-------|----------|--------|
| 1 | **Canvas doesn't match optimization_config.json schema** | CRITICAL | Canvas nodes don't represent actual extractors, objectives, constraints |
| 2 | **Missing data flow representation** | CRITICAL | No visualization of: displacement → Zernike → WFE pipeline |
| 3 | **Claude has no canvas context** | CRITICAL | Can't help modify canvas because it doesn't know current state |
| 4 | **Study loader is incomplete** | HIGH | Loading a study doesn't populate all nodes correctly |
| 5 | **Canvas exporter is broken/missing** | HIGH | Can't generate valid optimization_config.json from canvas |
| 6 | **No extractor node types** | HIGH | E1-E10 extractors not represented as proper nodes |
| 7 | **No output/result node types** | MEDIUM | Missing nodes for WFE, mass, stress outputs |
| 8 | **No validation against real schema** | MEDIUM | Canvas allows invalid configurations |

### 1.2 Real M1 Mirror Optimization Pipeline (What Canvas Should Show)

```
┌─────────────────────────────────────────────────────────────────────────────┐
│                           M1 MIRROR OPTIMIZATION                             │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                              │
│  INPUTS (Design Variables)                                                  │
│  ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐       │
│  │ flatback_    │ │ rib_height   │ │ rib_width    │ │ fillet_      │       │
│  │ thickness    │ │ [25-60mm]    │ │ [4-12mm]     │ │ radius       │       │
│  │ [15-45mm]    │ │              │ │              │ │ [2-10mm]     │       │
│  └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘       │
│         │                │                │                │                │
│         └────────────────┴────────────────┴────────────────┘                │
│                                    │                                         │
│                                    ▼                                         │
│  MODEL + SOLVER                                                             │
│  ┌─────────────────────────────────────────────────────────────┐           │
│  │  m1_mirror_sim1.sim  →  NX Nastran SOL101 (Static)         │           │
│  │  Updates: geometry.prt → idealized_i.prt → fem1.fem        │           │
│  └─────────────────────────────────────────────────────────────┘           │
│                                    │                                         │
│                                    ▼                                         │
│  EXTRACTORS (Post-Processing)                                               │
│  ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐       │
│  │ E5: CAD Mass │ │ E8: Zernike  │ │ E1: Max Disp │ │ E3: Max      │       │
│  │ (expression) │ │ from OP2     │ │ (optional)   │ │ Stress       │       │
│  └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘       │
│         │                │                │                │                │
│         │                ▼                │                │                │
│         │    ┌───────────────────────┐   │                │                │
│         │    │ Zernike Processing    │   │                │                │
│         │    │ - Fit coefficients    │   │                │                │
│         │    │ - Calculate WFE_40_20 │   │                │                │
│         │    │ - Calculate RMS       │   │                │                │
│         │    └───────────┬───────────┘   │                │                │
│         │                │                │                │                │
│         ▼                ▼                ▼                ▼                │
│  OBJECTIVES & CONSTRAINTS                                                   │
│  ┌──────────────┐ ┌──────────────┐ ┌──────────────┐                        │
│  │ MINIMIZE     │ │ MINIMIZE     │ │ CONSTRAINT   │                        │
│  │ mass_kg      │ │ wfe_40_20    │ │ stress < 200 │                        │
│  │ weight: 1.0  │ │ weight: 10.0 │ │ MPa          │                        │
│  └──────────────┘ └──────────────┘ └──────────────┘                        │
│                                                                              │
└─────────────────────────────────────────────────────────────────────────────┘
```

### 1.3 What optimization_config.json Actually Contains

```json
{
  "study_name": "m1_mirror_flatback_lateral",
  "description": "M1 Mirror optimization",

  "design_variables": [
    {"name": "flatback_thickness", "min": 15, "max": 45, "unit": "mm"},
    {"name": "rib_height", "min": 25, "max": 60, "unit": "mm"},
    {"name": "rib_width", "min": 4, "max": 12, "unit": "mm"},
    {"name": "fillet_radius", "min": 2, "max": 10, "unit": "mm"}
  ],

  "model": {
    "sim_file": "1_model/m1_mirror_sim1.sim",
    "prt_file": "1_model/m1_mirror.prt",
    "fem_file": "1_model/m1_mirror_fem1.fem",
    "idealized_file": "1_model/m1_mirror_fem1_i.prt"
  },

  "solver": {
    "type": "SOL101",
    "timeout_minutes": 30
  },

  "extractors": [
    {
      "id": "E5",
      "name": "mass_extractor",
      "type": "cad_mass",
      "expression_name": "total_mass",
      "output_name": "mass_kg"
    },
    {
      "id": "E8",
      "name": "zernike_extractor",
      "type": "zernike_op2",
      "subcase": 1,
      "node_set": "mirror_surface",
      "terms": [40, 20],
      "output_name": "wfe_40_20"
    }
  ],

  "objectives": [
    {"name": "mass_kg", "direction": "minimize", "weight": 1.0},
    {"name": "wfe_40_20", "direction": "minimize", "weight": 10.0}
  ],

  "constraints": [
    {"name": "max_stress", "operator": "<=", "value": 200, "unit": "MPa"}
  ],

  "algorithm": {
    "method": "NSGA-II",
    "max_trials": 200,
    "population_size": 40
  }
}
```

### 1.4 Current Canvas Schema (INCOMPLETE)

The current canvas has these node types:
- `model` - File path only, no solver integration
- `solver` - Just solver type, no connection to model
- `designVar` - Expression name and bounds only
- `algorithm` - Method and max trials
- `extractor` - Just extractor ID, no configuration
- `objective` - Name, direction, weight
- `constraint` - Name, operator, value
- `surrogate` - Neural surrogate toggle

**MISSING:**
- Proper extractor configuration (subcase, node set, terms)
- Data flow connections (extractor output → objective input)
- Model file relationships (sim → fem → prt → idealized)
- Zernike-specific processing nodes
- Output mapping (which extractor feeds which objective)

---

## Part 2: Claude Context Problem

### 2.1 Current State

When user opens canvas and chats with Claude:
1. Claude receives NO information about:
   - Current canvas nodes
   - Current canvas edges (connections)
   - Current study context
   - Current configuration state

2. Claude can only:
   - Answer generic questions
   - Use MCP tools for study queries
   - NOT modify the canvas directly

### 2.2 Required Claude Capabilities

Claude needs to:
1. **See current canvas state** - All nodes, edges, configurations
2. **Modify canvas** - Add/remove/update nodes via API
3. **Validate changes** - Check if configuration is valid
4. **Generate config** - Export canvas to optimization_config.json
5. **Load studies** - Import optimization_config.json to canvas

---

## Part 3: Solution Architecture

### 3.1 Enhanced Canvas Schema

```typescript
// New comprehensive node types
type NodeType =
  | 'model'           // NX model with all file references
  | 'solver'          // Solver configuration
  | 'designVar'       // Design variable (expression)
  | 'extractor'       // Physics extractor with full config
  | 'processor'       // Data processor (Zernike fitting, etc.)
  | 'objective'       // Optimization objective
  | 'constraint'      // Constraint definition
  | 'algorithm'       // Optimization algorithm
  | 'surrogate'       // Neural surrogate
  | 'output'          // Final output metric

// Enhanced extractor node
interface ExtractorNodeData {
  type: 'extractor';
  extractorId: 'E1' | 'E2' | 'E3' | 'E4' | 'E5' | 'E8' | 'E9' | 'E10';
  extractorName: string;

  // E1/E3 specific
  subcase?: number;
  nodeId?: number;
  elementId?: number;

  // E8 Zernike specific
  zernikeTerms?: number[];
  nodeSet?: string;
  referenceRadius?: number;

  // E5 Mass specific
  expressionName?: string;

  // Output mapping
  outputName: string;  // Name of the output variable
}

// Enhanced model node
interface ModelNodeData {
  type: 'model';
  simFile: string;
  prtFile: string;
  femFile: string;
  idealizedFile?: string;

  // Discovered info
  expressions?: Array<{name: string; value: number; unit: string}>;
  meshInfo?: {nodes: number; elements: number};
}

// Data processor node (for Zernike fitting, etc.)
interface ProcessorNodeData {
  type: 'processor';
  processorType: 'zernike_fit' | 'relative_calc' | 'rms_calc' | 'custom';

  // Inputs (connected from extractors)
  inputMapping: Record<string, string>;

  // Processing config
  config: Record<string, any>;

  // Outputs
  outputNames: string[];
}
```

### 3.2 Canvas-Config Synchronization

```typescript
// Canvas State Manager - shared between frontend and Claude
class CanvasStateManager {
  // Current state
  nodes: CanvasNode[];
  edges: CanvasEdge[];
  studyId: string | null;

  // Load from optimization_config.json
  async loadFromConfig(configPath: string): Promise<void>;

  // Export to optimization_config.json
  async exportToConfig(): Promise<OptimizationConfig>;

  // Validation
  validate(): ValidationResult;

  // Claude API
  toClaudeContext(): string;  // Markdown summary for Claude

  // Modification API (for Claude)
  addNode(type: NodeType, data: NodeData): string;
  updateNode(nodeId: string, data: Partial<NodeData>): void;
  removeNode(nodeId: string): void;
  addEdge(source: string, target: string): string;
  removeEdge(edgeId: string): void;
}
```

### 3.3 Claude Context Injection

When Claude receives a message in canvas mode:

```markdown
## Current Canvas State

**Study**: m1_mirror_flatback_lateral
**Status**: Configured (valid)

### Design Variables (4)
| Name | Min | Max | Unit |
|------|-----|-----|------|
| flatback_thickness | 15 | 45 | mm |
| rib_height | 25 | 60 | mm |
| rib_width | 4 | 12 | mm |
| fillet_radius | 2 | 10 | mm |

### Model
- **Sim File**: 1_model/m1_mirror_sim1.sim
- **Solver**: SOL101 (Static)

### Extractors (2)
1. E5: CAD Mass → mass_kg
2. E8: Zernike OP2 → wfe_40_20 (terms: 40, 20)

### Objectives (2)
1. MINIMIZE mass_kg (weight: 1.0)
2. MINIMIZE wfe_40_20 (weight: 10.0)

### Constraints (1)
1. max_stress <= 200 MPa

### Algorithm
- Method: NSGA-II
- Max Trials: 200

---
User can ask to modify any of the above. Use canvas_* tools to make changes.
```

### 3.4 New MCP Tools for Canvas

```typescript
// New tools Claude can use
const canvasTools = {
  // Read current state
  canvas_get_state: () => CanvasState,

  // Modify nodes
  canvas_add_design_var: (name: string, min: number, max: number, unit: string) => NodeId,
  canvas_update_design_var: (nodeId: string, updates: Partial<DesignVarData>) => void,
  canvas_remove_node: (nodeId: string) => void,

  // Add extractors
  canvas_add_extractor: (type: ExtractorType, config: ExtractorConfig) => NodeId,

  // Add objectives/constraints
  canvas_add_objective: (name: string, direction: 'minimize' | 'maximize', weight: number) => NodeId,
  canvas_add_constraint: (name: string, operator: string, value: number) => NodeId,

  // Connections
  canvas_connect: (sourceId: string, targetId: string) => EdgeId,
  canvas_disconnect: (edgeId: string) => void,

  // Validation & Export
  canvas_validate: () => ValidationResult,
  canvas_export_config: () => OptimizationConfig,
  canvas_apply_to_study: (studyPath: string) => void,
};
```

---

## Part 4: Implementation Plan

### Phase 1: Schema & Store Enhancement (Priority: CRITICAL)

**Files to modify:**
- `frontend/src/lib/canvas/schema.ts` - Enhanced node types
- `frontend/src/hooks/useCanvasStore.ts` - State management
- `frontend/src/lib/canvas/configSync.ts` - NEW: Config sync utilities

**Tasks:**
- [ ] Define complete ExtractorNodeData with all E1-E10 configs
- [ ] Define ProcessorNodeData for Zernike processing
- [ ] Define complete ModelNodeData with all file references
- [ ] Add output mapping to all extractors
- [ ] Create validation functions

### Phase 2: Study Loader Enhancement (Priority: CRITICAL)

**Files to modify:**
- `frontend/src/lib/canvas/studyLoader.ts` - NEW: Full study loading
- `backend/api/routes/canvas.py` - NEW: Canvas API endpoints

**Tasks:**
- [ ] Parse optimization_config.json completely
- [ ] Create nodes for ALL config elements
- [ ] Create edges for data flow
- [ ] Handle Zernike-specific extractors
- [ ] Handle nested processor configurations

### Phase 3: Canvas Exporter (Priority: HIGH)

**Files to create:**
- `frontend/src/lib/canvas/configExporter.ts` - Export to config
- `backend/api/routes/canvas.py` - Save config endpoint

**Tasks:**
- [ ] Convert canvas nodes to optimization_config.json
- [ ] Validate exported config
- [ ] Handle edge-to-dependency mapping
- [ ] Write to study folder

### Phase 4: Claude Context Integration (Priority: CRITICAL)

**Files to modify:**
- `backend/api/services/context_builder.py` - Add canvas context
- `backend/api/routes/claude.py` - Include canvas state in prompt
- `frontend/src/components/chat/ChatPanel.tsx` - Send canvas state

**Tasks:**
- [ ] Generate markdown summary of canvas state
- [ ] Include in every Claude message
- [ ] Update context on canvas changes
- [ ] Add canvas modification instructions

### Phase 5: MCP Canvas Tools (Priority: HIGH)

**Files to create:**
- `mcp-server/atomizer-tools/src/canvas-tools.ts` - Canvas modification tools

**Tasks:**
- [ ] Implement canvas_get_state
- [ ] Implement canvas_add_* tools
- [ ] Implement canvas_update_* tools
- [ ] Implement canvas_remove_node
- [ ] Implement canvas_connect/disconnect
- [ ] Implement canvas_validate
- [ ] Implement canvas_export_config

### Phase 6: UI Node Enhancements (Priority: MEDIUM)

**Files to modify:**
- `frontend/src/components/canvas/nodes/ExtractorNode.tsx` - Full config display
- `frontend/src/components/canvas/panels/NodeConfigPanel.tsx` - All config options

**Tasks:**
- [ ] Show extractor configuration in node
- [ ] Show output mapping
- [ ] Show data flow direction
- [ ] Add quick-edit for common settings

---

## Part 5: Immediate Actions

### Action 1: Read actual optimization_config.json from M1 mirror
```bash
Read studies/M1_Mirror/m1_mirror_flatback_lateral/optimization_config.json
```

### Action 2: Read current run_optimization.py to understand pipeline
```bash
Read studies/M1_Mirror/m1_mirror_flatback_lateral/run_optimization.py
```

### Action 3: Compare with canvas schema
```bash
Read atomizer-dashboard/frontend/src/lib/canvas/schema.ts
```

### Action 4: Check current study loading logic
```bash
Find and read study loading code in useCanvasStore
```

### Action 5: Check Claude context builder
```bash
Read atomizer-dashboard/backend/api/services/context_builder.py
```

---

## Part 6: Success Criteria

1. **Loading M1 Mirror study shows ALL nodes**:
   - 4 design variables with correct bounds
   - Model node with all file references
   - Solver node (SOL101)
   - E5 Mass extractor with expression name
   - E8 Zernike extractor with terms [40, 20]
   - 2 objectives (mass, wfe)
   - Proper edge connections

2. **Claude knows canvas state**:
   - User says "add hole_diameter" → Claude immediately adds it
   - No clarifying questions about which study
   - Validates against current config

3. **Canvas exports valid config**:
   - Click "Export" → generates optimization_config.json
   - Config is valid and runnable
   - Matches what run_optimization.py expects

4. **Bidirectional sync**:
   - Edit canvas → config updates
   - Edit config file → canvas updates (on reload)

---

## Tracking

| Phase | Status | Notes |
|-------|--------|-------|
| Phase 1: Schema | 🟢 COMPLETE | Enhanced ExtractorNodeData, DesignVarNodeData, AlgorithmNodeData with ZernikeConfig, baseline, penalty weights |
| Phase 2: Loader | 🟢 COMPLETE | Fixed OptimizationConfig to use min/max, added extraction_method, zernike_settings parsing |
| Phase 3: Exporter | 🟡 Partial | loadFromConfig now creates proper nodes from real configs |
| Phase 4: Claude Context | 🟢 COMPLETE | context_builder.py now has _canvas_context(), useChat passes canvas state |
| Phase 5: MCP Tools | 🟢 COMPLETE | Added canvas_add_node, canvas_update_node, canvas_remove_node, canvas_connect_nodes |
| Phase 6: UI | 🟡 Partial | useCanvasChat syncs canvas state, applyModification handles Claude responses |

---

## Implementation Summary

### Files Modified

**Frontend:**
- `src/lib/canvas/schema.ts` - Enhanced node data types (ZernikeConfig, baseline, penalty weights)
- `src/hooks/useCanvasStore.ts` - Fixed OptimizationConfig interface (min/max), enhanced loadFromConfig
- `src/hooks/useChat.ts` - Added CanvasState, updateCanvasState, passes canvas to backend
- `src/hooks/useCanvasChat.ts` - Syncs canvas state, applyModification for Claude modifications

**Backend:**
- `api/services/context_builder.py` - Added _canvas_context() method with full canvas serialization
- `api/services/session_manager.py` - send_message now accepts canvas_state parameter
- `api/routes/claude.py` - WebSocket handler accepts set_canvas messages, passes to session

**MCP Server:**
- `src/tools/canvas.ts` - Added canvas_add_node, canvas_update_node, canvas_remove_node, canvas_connect_nodes

### Key Changes

1. **Canvas-Config Interface Fixed**:
   - Uses `min/max` instead of `lower/upper`
   - Supports `extraction_method` and `zernike_settings`
   - Handles `baseline`, `units`, `enabled`, `notes` for design vars

2. **Claude Context Injection**:
   - Full canvas state passed with every message
   - Tables for design vars, objectives, extractors
   - Instructions for canvas modification tools

3. **Canvas Modification Tools**:
   - Claude can add design variables with `canvas_add_node`
   - Claude can update weights/bounds with `canvas_update_node`
   - Frontend applies modifications via `applyModification()`

---

*Document completed January 16, 2026*