# Atomizer Unified Configuration Architecture ## Master Design Document **Version**: 1.0 **Date**: January 2026 **Status**: Proposed Architecture **Author**: Atomizer Architecture Review --- ## Executive Summary This document presents a comprehensive analysis of Atomizer's current configuration architecture and proposes a **Unified Configuration System** that establishes a single source of truth for optimization studies. The new architecture enables: 1. **Single Source of Truth**: One canonical JSON schema (`AtomizerSpec`) used everywhere 2. **Bidirectional Sync**: Canvas ↔ Config ↔ Backend with zero data loss 3. **Intelligent Manipulation**: Claude can dynamically modify configs, add custom functions, and validate changes 4. **Real-time Collaboration**: WebSocket-driven updates between Dashboard, Canvas, and Claude Assistant --- ## Table of Contents 1. [Current State Analysis](#1-current-state-analysis) 2. [Problem Statement](#2-problem-statement) 3. [Proposed Architecture](#3-proposed-architecture) 4. [AtomizerSpec Schema](#4-atomizerspec-schema) 5. [Component Architecture](#5-component-architecture) 6. [Intelligent Assistant Integration](#6-intelligent-assistant-integration) 7. [API Design](#7-api-design) 8. [Migration Strategy](#8-migration-strategy) 9. [Implementation Roadmap](#9-implementation-roadmap) --- ## 1. Current State Analysis ### 1.1 Architecture Overview The current system has **four distinct configuration representations** that don't fully align: ``` ┌─────────────────────────────────────────────────────────────────────────────┐ │ CURRENT ARCHITECTURE (FRAGMENTED) │ └─────────────────────────────────────────────────────────────────────────────┘ Canvas UI Backend Optimization ┌────────────────┐ ┌────────────────┐ ┌────────────────┐ │ ReactFlow │ │ FastAPI │ │ Python Engine │ │ Nodes/Edges │ │ Routes │ │ ConfigManager │ │ │ │ │ │ │ │ ┌──────────┐ │ │ ┌──────────┐ │ │ ┌──────────┐ │ │ │ Canvas │──┼──LOSSY──┼─▶│ Intent │──┼──LOSSY──┼─▶│ Config │ │ │ │ State │ │ │ │ JSON │ │ │ │ .json │ │ │ └──────────┘ │ │ └──────────┘ │ │ └──────────┘ │ │ ▲ │ │ │ │ │ │ │ │ │ LOSSY │ │ ▼ │ │ ▼ │ │ │ │ │ ┌──────────┐ │ │ ┌──────────┐ │ │ Load from │◀─LOSSY──┼──│ Claude │ │ │ │ Optuna │ │ │ config │ │ │ Assistant│ │ │ │ Database │ │ └────────────────┘ └────────────────┘ └────────────────┘ ``` ### 1.2 Current Config Formats (4+ Variants) | Variant | Used By | Key Differences | |---------|---------|-----------------| | **Mirror/Zernike** | m1_mirror studies | `extractor_config`, `zernike_settings`, subcases | | **Drone/Structural** | drone_gimbal | `extraction.{action, domain, params}`, `bounds[]` | | **Canvas Intent** | Canvas UI | Simplified, loses metadata | | **Legacy Schema** | optimization_config_schema.json | `parameter`, `bounds[]`, `goal` | ### 1.3 Field Naming Chaos | Concept | Canvas Intent | Mirror Config | Drone Config | Schema | |---------|---------------|---------------|--------------|--------| | DV bounds | `min`, `max` | `min`, `max` | `bounds: [min, max]` | `bounds: [min, max]` | | DV name | `name` | `expression_name` | `parameter` | `parameter` | | Objective dir | `direction` | `direction` | `goal` | `goal` | | Extractor | `extractor: "E5"` | `extractor_config: {}` | `extraction: {}` | `extraction: {}` | | Trials | `max_trials` | `n_trials` | `n_trials` | `n_trials` | ### 1.4 Data Flow Problems ``` PROBLEM 1: Information Loss During Canvas → Config Conversion ═══════════════════════════════════════════════════════════════ Canvas Node: Intent: Config: ┌─────────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ Extractor E8 │ │ extractors: [{ │ │ objectives: [{ │ │ - nModes: 40 │───────▶│ id: "E8", │───────▶│ extractor: │ │ - innerRadius: 58 │ LOST │ name: "..." │ LOST │ "E8" │ │ - subcases: [1,2,3] │───────▶│ }] │───────▶│ }] │ │ - filterLowOrders: 6│ │ │ │ │ └─────────────────────┘ └─────────────────┘ └─────────────────┘ │ │ │ MISSING: │ MISSING: │ - nModes │ - All config │ - innerRadius │ - zernike_settings │ - subcases │ - extraction_method │ - filterLowOrders │ PROBLEM 2: Config → Canvas Loading is Incomplete ═══════════════════════════════════════════════════════════════ Real Config (m1_mirror): Canvas Display: ┌───────────────────────────────┐ ┌─────────────────────────┐ │ design_variables: [ │ │ 11 DesignVar nodes │ ✓ │ { name, min, max, ... } │ │ │ │ ] │ │ │ │ │ │ │ │ objectives: [ │ │ 3 Objective nodes │ ✓ │ { extractor_config: { │ │ (but extractor_config │ │ target_subcase: "3", │ │ NOT displayed) │ ✗ │ metric: "rel_filtered_rms"│ │ │ │ }} │ │ │ │ ] │ │ │ │ │ │ │ │ hard_constraints: [ │ │ Converted to objectives │ ✗ │ { name, limit, penalty } │ │ (loses constraint type) │ │ ] │ │ │ │ │ │ │ │ sat_settings: { │ │ NOT LOADED AT ALL │ ✗ │ n_ensemble_models: 10, │ │ │ │ hidden_dims: [256, 128] │ │ │ │ } │ │ │ └───────────────────────────────┘ └─────────────────────────┘ PROBLEM 3: No Bidirectional Sync ═══════════════════════════════════════════════════════════════ User edits config.json manually │ ▼ Canvas doesn't know about change │ ▼ User refreshes canvas → loses their canvas-only edits │ ▼ Conflict between what config says and what canvas shows ``` ### 1.5 Missing Backend Endpoints | Required Endpoint | Status | Impact | |-------------------|--------|--------| | `POST /studies/create-from-intent` | Missing | Canvas can't create studies | | `POST /studies/{id}/validate-config` | Missing | No schema validation | | `PUT /studies/{id}/config` (atomic) | Partial | No conflict detection | | `POST /canvas/apply-modification` | Missing | Claude can't modify canvas | | `WebSocket /studies/{id}/sync` | Missing | No real-time updates | --- ## 2. Problem Statement ### 2.1 Core Issues 1. **No Single Source of Truth**: Multiple config formats mean the same study looks different depending on where you view it 2. **Lossy Conversions**: Converting Canvas → Intent → Config loses information; converting Config → Canvas is incomplete 3. **No Real-time Sync**: Changes in one place (config file, canvas, Claude) don't propagate to others 4. **Claude Can't Modify Canvas Directly**: MCP tools return JSON but nothing applies changes to the frontend 5. **No Intelligent Extensibility**: User can't ask Claude to "add a custom extraction function" because there's no plugin system ### 2.2 User Pain Points | Scenario | Current Experience | Desired Experience | |----------|-------------------|-------------------| | "Load my existing study into canvas" | Partial load, loses extractor settings | Complete bidirectional load | | "Claude, change the mass objective weight to 5" | Claude returns JSON, user must manually apply | Canvas updates in real-time | | "I need a custom Zernike RMS calculation" | Must edit Python code directly | Claude adds function, canvas shows new node | | "Show me what the config looks like" | Multiple representations exist | One canonical view everywhere | --- ## 3. Proposed Architecture ### 3.1 Unified Architecture Overview ``` ┌─────────────────────────────────────────────────────────────────────────────┐ │ PROPOSED ARCHITECTURE (UNIFIED) │ └─────────────────────────────────────────────────────────────────────────────┘ ┌─────────────────────┐ │ AtomizerSpec │ │ (Single Source │ │ of Truth) │ │ │ │ atomizer_spec.json │ └──────────┬──────────┘ │ ┌──────────────────────────┼──────────────────────────┐ │ │ │ ▼ ▼ ▼ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ Canvas UI │ │ Backend API │ │ Claude Assistant│ │ │ │ │ │ │ │ ReactFlow + │◀─────▶│ FastAPI + │◀─────▶│ MCP Tools + │ │ SpecRenderer │ WS │ SpecManager │ WS │ SpecModifier │ │ │ │ │ │ │ └─────────────────┘ └─────────────────┘ └─────────────────┘ │ │ │ │ WebSocket Real-time Sync Bus │ └──────────────────────────┴──────────────────────────┘ │ ▼ ┌─────────────────────┐ │ Optimization │ │ Engine │ │ │ │ Uses AtomizerSpec │ │ directly │ └─────────────────────┘ ``` ### 3.2 Key Principles 1. **AtomizerSpec is THE config**: One JSON schema, versioned, validated everywhere 2. **No Intermediate Formats**: Canvas renders AtomizerSpec directly; no "Intent" conversion 3. **Changes Flow Through Backend**: All modifications go through SpecManager 4. **WebSocket Sync**: All clients receive real-time updates 5. **Extensible Functions**: Custom extractors/functions defined IN the spec ### 3.3 Data Flow (New) ``` AtomizerSpec (atomizer_spec.json) │ │ Read by all ┌─────────────────────────┼─────────────────────────┐ │ │ │ ▼ ▼ ▼ Canvas Renderer Backend Loader Claude Reader │ │ │ │ User edits node │ │ "Change weight" │ │ │ ▼ ▼ ▼ PATCH /spec/nodes/{id} (validates) MCP: spec_modify │ │ │ └─────────────────────────┼─────────────────────────┘ │ ▼ SpecManager ┌─────────────────────────┐ │ 1. Validate change │ │ 2. Apply to spec │ │ 3. Persist to disk │ │ 4. Broadcast via WS │ └─────────────────────────┘ │ │ WebSocket: spec_updated ┌─────────────────────────┼─────────────────────────┐ │ │ │ ▼ ▼ ▼ Canvas re-renders Other clients Claude sees new state ``` --- ## 4. AtomizerSpec Schema ### 4.1 Schema Design Philosophy - **Self-describing**: Schema version and capabilities in the file - **Complete**: ALL information needed for optimization, canvas, and reporting - **Extensible**: Custom functions, extractors, and plugins declared in-spec - **Validated**: JSON Schema validation at every boundary ### 4.2 AtomizerSpec v2.0 Schema ```json { "$schema": "https://atomizer.io/schemas/atomizer_spec_v2.json", "$id": "atomizer_spec", "meta": { "version": "2.0", "created": "2026-01-17T12:00:00Z", "modified": "2026-01-17T14:30:00Z", "created_by": "canvas", "modified_by": "claude", "study_name": "m1_mirror_optimization_v15", "description": "Multi-objective mirror optimization with Zernike WFE", "tags": ["mirror", "zernike", "multi-objective"] }, "model": { "nx_part": { "path": "C:/Studies/m1_mirror/model/M1_Mirror.prt", "hash": "sha256:abc123...", "idealized_part": "M1_Mirror_fem1_i.prt" }, "fem": { "path": "model/M1_Mirror_fem1.fem", "element_count": 45000, "node_count": 12000 }, "sim": { "path": "model/M1_Mirror_sim1.sim", "solver": "nastran", "solution_type": "SOL101", "subcases": [ { "id": 1, "name": "Gravity 0deg", "type": "static" }, { "id": 2, "name": "Gravity 20deg", "type": "static" }, { "id": 3, "name": "Gravity 40deg", "type": "static" } ] } }, "design_variables": [ { "id": "dv_001", "name": "Lateral Rib Count", "expression_name": "n_lateral_ribs", "type": "integer", "bounds": { "min": 6, "max": 12 }, "baseline": 9, "units": "count", "enabled": true, "description": "Number of lateral support ribs", "canvas_position": { "x": 50, "y": 100 } }, { "id": "dv_002", "name": "Facesheet Thickness", "expression_name": "facesheet_t", "type": "continuous", "bounds": { "min": 2.0, "max": 8.0 }, "baseline": 4.5, "units": "mm", "enabled": true, "step": 0.1, "canvas_position": { "x": 50, "y": 200 } } ], "extractors": [ { "id": "ext_001", "name": "Zernike WFE Extractor", "type": "zernike_opd", "builtin": true, "config": { "inner_radius_mm": 58.0, "outer_radius_mm": 330.0, "n_modes": 40, "filter_low_orders": 6, "displacement_unit": "mm", "reference_subcase": 1 }, "outputs": [ { "name": "wfe_rms_20deg", "subcase": 2, "metric": "filtered_rms_nm" }, { "name": "wfe_rms_40deg", "subcase": 3, "metric": "filtered_rms_nm" }, { "name": "wfe_p2v_40deg", "subcase": 3, "metric": "pv_nm" } ], "canvas_position": { "x": 740, "y": 100 } }, { "id": "ext_002", "name": "Mass Extractor", "type": "mass", "builtin": true, "config": { "source": "expression", "expression_name": "total_mass_kg" }, "outputs": [ { "name": "mass_kg", "metric": "total" } ], "canvas_position": { "x": 740, "y": 300 } }, { "id": "ext_003", "name": "Custom Manufacturability Score", "type": "custom_function", "builtin": false, "function": { "name": "calc_manufacturability", "module": "custom_extractors.manufacturability", "signature": "(design_vars: dict, fem_results: dict) -> float", "source_code": "def calc_manufacturability(design_vars, fem_results):\n # Custom logic here\n rib_count = design_vars['n_lateral_ribs']\n thickness = design_vars['facesheet_t']\n score = 100 - (rib_count * 5) - (10 - thickness) * 3\n return max(0, min(100, score))" }, "outputs": [ { "name": "mfg_score", "metric": "score" } ], "canvas_position": { "x": 740, "y": 500 } } ], "objectives": [ { "id": "obj_001", "name": "Minimize WFE at 40deg", "direction": "minimize", "weight": 5.0, "source": { "extractor_id": "ext_001", "output_name": "wfe_rms_40deg" }, "target": 5.0, "units": "nm", "canvas_position": { "x": 1020, "y": 100 } }, { "id": "obj_002", "name": "Minimize Mass", "direction": "minimize", "weight": 1.0, "source": { "extractor_id": "ext_002", "output_name": "mass_kg" }, "target": 100.0, "units": "kg", "canvas_position": { "x": 1020, "y": 200 } } ], "constraints": [ { "id": "con_001", "name": "Max Mass Limit", "type": "hard", "operator": "<=", "threshold": 120.0, "source": { "extractor_id": "ext_002", "output_name": "mass_kg" }, "penalty_config": { "method": "quadratic", "weight": 1000.0, "margin": 5.0 }, "canvas_position": { "x": 1020, "y": 400 } }, { "id": "con_002", "name": "Minimum Manufacturability", "type": "soft", "operator": ">=", "threshold": 60.0, "source": { "extractor_id": "ext_003", "output_name": "mfg_score" }, "penalty_config": { "method": "linear", "weight": 10.0 }, "canvas_position": { "x": 1020, "y": 500 } } ], "optimization": { "algorithm": { "type": "NSGA-II", "config": { "population_size": 50, "n_generations": 100, "mutation_prob": null, "crossover_prob": 0.9, "seed": 42 } }, "budget": { "max_trials": 500, "max_time_hours": 48, "convergence_patience": 50 }, "surrogate": { "enabled": true, "type": "ensemble", "config": { "n_models": 10, "architecture": [256, 128, 64], "train_every_n_trials": 20, "min_training_samples": 30, "acquisition_candidates": 10000, "fea_validations_per_round": 5 } }, "canvas_position": { "x": 1300, "y": 150 } }, "workflow": { "stages": [ { "id": "stage_exploration", "name": "Design Space Exploration", "algorithm": "RandomSearch", "trials": 30, "purpose": "Build initial training data" }, { "id": "stage_optimization", "name": "Surrogate-Assisted Optimization", "algorithm": "SAT_v3", "trials": 470, "purpose": "Efficient optimization with neural acceleration" } ], "transitions": [ { "from": "stage_exploration", "to": "stage_optimization", "condition": "trial_count >= 30" } ] }, "reporting": { "auto_report": true, "report_triggers": ["new_pareto", "every_50_trials", "convergence"], "insights": [ { "type": "zernike_visualization", "for_trials": "pareto_front", "config": { "include_html": true } }, { "type": "convergence_plot", "config": { "show_pareto_evolution": true } } ] }, "canvas": { "layout_version": "2.0", "viewport": { "x": 0, "y": 0, "zoom": 1.0 }, "edges": [ { "source": "dv_001", "target": "model" }, { "source": "dv_002", "target": "model" }, { "source": "model", "target": "solver" }, { "source": "solver", "target": "ext_001" }, { "source": "solver", "target": "ext_002" }, { "source": "ext_001", "target": "obj_001" }, { "source": "ext_002", "target": "obj_002" }, { "source": "ext_002", "target": "con_001" }, { "source": "obj_001", "target": "optimization" }, { "source": "obj_002", "target": "optimization" }, { "source": "con_001", "target": "optimization" } ], "groups": [ { "id": "grp_inputs", "name": "Design Inputs", "node_ids": ["dv_001", "dv_002"] }, { "id": "grp_physics", "name": "Physics Extraction", "node_ids": ["ext_001", "ext_002"] } ] } } ``` ### 4.3 Schema Versioning | Version | Status | Key Changes | |---------|--------|-------------| | 1.0 | Legacy | Original `optimization_config.json` | | 2.0 | **Proposed** | Unified spec with canvas, custom functions, workflow | | 2.1 | Future | Multi-fidelity, parallel execution | ### 4.4 JSON Schema Definition ```json { "$schema": "http://json-schema.org/draft-07/schema#", "$id": "https://atomizer.io/schemas/atomizer_spec_v2.json", "title": "AtomizerSpec v2.0", "type": "object", "required": ["meta", "model", "design_variables", "extractors", "objectives", "optimization"], "properties": { "meta": { "type": "object", "required": ["version", "study_name"], "properties": { "version": { "type": "string", "pattern": "^2\\.\\d+$" }, "study_name": { "type": "string", "pattern": "^[a-z0-9_]+$" }, "created": { "type": "string", "format": "date-time" }, "modified": { "type": "string", "format": "date-time" }, "created_by": { "enum": ["canvas", "claude", "api", "migration"] }, "modified_by": { "type": "string" }, "description": { "type": "string" }, "tags": { "type": "array", "items": { "type": "string" } } } }, "design_variables": { "type": "array", "minItems": 1, "items": { "$ref": "#/definitions/design_variable" } }, "extractors": { "type": "array", "minItems": 1, "items": { "$ref": "#/definitions/extractor" } }, "objectives": { "type": "array", "minItems": 1, "items": { "$ref": "#/definitions/objective" } } }, "definitions": { "design_variable": { "type": "object", "required": ["id", "name", "expression_name", "type", "bounds"], "properties": { "id": { "type": "string", "pattern": "^dv_\\d{3}$" }, "name": { "type": "string" }, "expression_name": { "type": "string" }, "type": { "enum": ["continuous", "integer", "categorical"] }, "bounds": { "type": "object", "required": ["min", "max"], "properties": { "min": { "type": "number" }, "max": { "type": "number" } } }, "baseline": { "type": "number" }, "units": { "type": "string" }, "enabled": { "type": "boolean", "default": true }, "canvas_position": { "$ref": "#/definitions/position" } } }, "extractor": { "type": "object", "required": ["id", "name", "type", "outputs"], "properties": { "id": { "type": "string", "pattern": "^ext_\\d{3}$" }, "name": { "type": "string" }, "type": { "type": "string" }, "builtin": { "type": "boolean" }, "config": { "type": "object" }, "function": { "type": "object", "properties": { "name": { "type": "string" }, "module": { "type": "string" }, "source_code": { "type": "string" } } }, "outputs": { "type": "array", "items": { "type": "object", "required": ["name"], "properties": { "name": { "type": "string" }, "metric": { "type": "string" }, "subcase": { "type": "integer" } } } } } }, "objective": { "type": "object", "required": ["id", "name", "direction", "source"], "properties": { "id": { "type": "string", "pattern": "^obj_\\d{3}$" }, "name": { "type": "string" }, "direction": { "enum": ["minimize", "maximize"] }, "weight": { "type": "number", "minimum": 0 }, "source": { "type": "object", "required": ["extractor_id", "output_name"], "properties": { "extractor_id": { "type": "string" }, "output_name": { "type": "string" } } }, "target": { "type": "number" }, "units": { "type": "string" } } }, "position": { "type": "object", "properties": { "x": { "type": "number" }, "y": { "type": "number" } } } } } ``` --- ## 5. Component Architecture ### 5.1 Backend: SpecManager Service ```python # atomizer-dashboard/backend/api/services/spec_manager.py from pathlib import Path from datetime import datetime import json import hashlib from typing import Optional, Dict, Any, List from pydantic import BaseModel import jsonschema class SpecManager: """ Central service for managing AtomizerSpec. All modifications go through this service. """ def __init__(self, study_path: Path): self.study_path = study_path self.spec_path = study_path / "atomizer_spec.json" self.schema = self._load_schema() self._subscribers: List[WebSocketConnection] = [] def load(self) -> Dict[str, Any]: """Load and validate the spec.""" with open(self.spec_path) as f: spec = json.load(f) self._validate(spec) return spec def save(self, spec: Dict[str, Any], modified_by: str = "api"): """Save spec with validation and broadcast.""" # Update metadata spec["meta"]["modified"] = datetime.utcnow().isoformat() + "Z" spec["meta"]["modified_by"] = modified_by # Validate self._validate(spec) # Compute hash for conflict detection spec_hash = self._compute_hash(spec) # Atomic write temp_path = self.spec_path.with_suffix(".tmp") with open(temp_path, "w") as f: json.dump(spec, f, indent=2) temp_path.replace(self.spec_path) # Broadcast to all subscribers self._broadcast({ "type": "spec_updated", "hash": spec_hash, "modified_by": modified_by, "timestamp": spec["meta"]["modified"] }) return spec_hash def patch(self, path: str, value: Any, modified_by: str = "api") -> Dict[str, Any]: """ Apply a JSON Patch-style modification. path: JSONPath like "design_variables[0].bounds.max" """ spec = self.load() self._apply_patch(spec, path, value) self.save(spec, modified_by) return spec def add_node(self, node_type: str, data: Dict[str, Any], modified_by: str = "canvas"): """Add a new node (design var, extractor, objective, etc.)""" spec = self.load() # Generate ID node_id = self._generate_id(node_type, spec) data["id"] = node_id # Add canvas position if not provided if "canvas_position" not in data: data["canvas_position"] = self._auto_position(node_type, spec) # Add to appropriate section section = self._get_section_for_type(node_type) spec[section].append(data) self.save(spec, modified_by) return node_id def remove_node(self, node_id: str, modified_by: str = "canvas"): """Remove a node and all edges referencing it.""" spec = self.load() # Find and remove node for section in ["design_variables", "extractors", "objectives", "constraints"]: spec[section] = [n for n in spec.get(section, []) if n.get("id") != node_id] # Remove edges referencing this node if "canvas" in spec and "edges" in spec["canvas"]: spec["canvas"]["edges"] = [ e for e in spec["canvas"]["edges"] if e["source"] != node_id and e["target"] != node_id ] self.save(spec, modified_by) def add_custom_function(self, name: str, code: str, outputs: List[str], modified_by: str = "claude"): """ Add a custom extractor function. Claude can call this to add new physics extraction logic. """ spec = self.load() # Validate Python syntax compile(code, f"", "exec") extractor = { "id": self._generate_id("ext", spec), "name": name, "type": "custom_function", "builtin": False, "function": { "name": name, "module": "custom_extractors.dynamic", "source_code": code }, "outputs": [{"name": o, "metric": "custom"} for o in outputs] } spec["extractors"].append(extractor) self.save(spec, modified_by) return extractor["id"] def validate_and_report(self) -> Dict[str, Any]: """Run full validation and return detailed report.""" spec = self.load() report = { "valid": True, "errors": [], "warnings": [], "summary": {} } # Schema validation try: self._validate(spec) except jsonschema.ValidationError as e: report["valid"] = False report["errors"].append({ "type": "schema", "path": list(e.absolute_path), "message": e.message }) # Semantic validation self._validate_semantic(spec, report) # Summary report["summary"] = { "design_variables": len(spec.get("design_variables", [])), "extractors": len(spec.get("extractors", [])), "objectives": len(spec.get("objectives", [])), "constraints": len(spec.get("constraints", [])), "custom_functions": sum(1 for e in spec.get("extractors", []) if not e.get("builtin", True)) } return report def subscribe(self, ws: WebSocketConnection): """Subscribe to spec changes.""" self._subscribers.append(ws) def unsubscribe(self, ws: WebSocketConnection): """Unsubscribe from spec changes.""" self._subscribers.remove(ws) def _broadcast(self, message: Dict[str, Any]): """Broadcast to all subscribers.""" for ws in self._subscribers: ws.send_json(message) ``` ### 5.2 Frontend: SpecRenderer Component ```typescript // atomizer-dashboard/frontend/src/components/canvas/SpecRenderer.tsx import React, { useCallback, useEffect } from 'react'; import ReactFlow, { Node, Edge, useNodesState, useEdgesState } from 'reactflow'; import { useSpecStore } from '../../hooks/useSpecStore'; import { useWebSocket } from '../../hooks/useWebSocket'; interface AtomizerSpec { meta: SpecMeta; design_variables: DesignVariable[]; extractors: Extractor[]; objectives: Objective[]; constraints: Constraint[]; optimization: OptimizationConfig; canvas: CanvasConfig; } export function SpecRenderer({ studyId }: { studyId: string }) { const { spec, setSpec, updateNode, addNode, removeNode } = useSpecStore(); const { subscribe, send } = useWebSocket(`/api/studies/${studyId}/sync`); // Convert spec to ReactFlow nodes const [nodes, setNodes, onNodesChange] = useNodesState( specToNodes(spec) ); // Convert spec edges to ReactFlow edges const [edges, setEdges, onEdgesChange] = useEdgesState( spec?.canvas?.edges?.map(edgeToReactFlow) || [] ); // Subscribe to real-time updates useEffect(() => { const unsubscribe = subscribe('spec_updated', (data) => { // Reload spec from server fetchSpec(studyId).then(setSpec); }); return unsubscribe; }, [studyId]); // When spec changes, update nodes useEffect(() => { if (spec) { setNodes(specToNodes(spec)); setEdges(spec.canvas?.edges?.map(edgeToReactFlow) || []); } }, [spec]); // Handle node property changes const onNodeDataChange = useCallback((nodeId: string, data: Partial) => { // Send PATCH to backend send({ type: 'patch_node', node_id: nodeId, data }); }, [send]); // Handle node position changes (for canvas layout) const onNodeDragStop = useCallback((event: React.MouseEvent, node: Node) => { send({ type: 'update_position', node_id: node.id, position: node.position }); }, [send]); return ( {/* Toolbars, panels, etc. */} ); } // Convert AtomizerSpec to ReactFlow nodes function specToNodes(spec: AtomizerSpec | null): Node[] { if (!spec) return []; const nodes: Node[] = []; // Model node (synthetic) nodes.push({ id: 'model', type: 'model', position: { x: 280, y: 50 }, data: { label: spec.meta.study_name, filePath: spec.model.sim.path, configured: true } }); // Solver node (synthetic) nodes.push({ id: 'solver', type: 'solver', position: { x: 510, y: 50 }, data: { label: spec.model.sim.solution_type, solverType: spec.model.sim.solution_type, configured: true } }); // Design variables for (const dv of spec.design_variables) { nodes.push({ id: dv.id, type: 'designVar', position: dv.canvas_position || { x: 50, y: nodes.length * 100 }, data: { label: dv.name, expressionName: dv.expression_name, minValue: dv.bounds.min, maxValue: dv.bounds.max, baseline: dv.baseline, unit: dv.units, enabled: dv.enabled, configured: true } }); } // Extractors for (const ext of spec.extractors) { nodes.push({ id: ext.id, type: 'extractor', position: ext.canvas_position || { x: 740, y: nodes.length * 100 }, data: { label: ext.name, extractorId: ext.id, extractorType: ext.type, builtin: ext.builtin, config: ext.config, outputs: ext.outputs, hasCustomCode: !ext.builtin, configured: true } }); } // Objectives for (const obj of spec.objectives) { nodes.push({ id: obj.id, type: 'objective', position: obj.canvas_position || { x: 1020, y: nodes.length * 100 }, data: { label: obj.name, name: obj.name, direction: obj.direction, weight: obj.weight, target: obj.target, units: obj.units, source: obj.source, configured: true } }); } // Constraints for (const con of spec.constraints || []) { nodes.push({ id: con.id, type: 'constraint', position: con.canvas_position || { x: 1020, y: nodes.length * 100 }, data: { label: con.name, name: con.name, operator: con.operator, threshold: con.threshold, constraintType: con.type, source: con.source, configured: true } }); } // Optimization node nodes.push({ id: 'optimization', type: 'algorithm', position: spec.optimization.canvas_position || { x: 1300, y: 150 }, data: { label: spec.optimization.algorithm.type, method: spec.optimization.algorithm.type, maxTrials: spec.optimization.budget.max_trials, config: spec.optimization.algorithm.config, surrogate: spec.optimization.surrogate, configured: true } }); return nodes; } ``` ### 5.3 MCP Tools for Claude ```typescript // mcp-server/atomizer-tools/src/tools/spec_tools.ts export const specTools: AtomizerTool[] = [ { definition: { name: "spec_get", description: "Get the current AtomizerSpec for a study. Returns the complete configuration.", inputSchema: { type: "object", properties: { study_name: { type: "string", description: "Study name" } }, required: ["study_name"] } }, handler: async (args) => { const spec = await specManager.load(args.study_name); return { content: [{ type: "text", text: JSON.stringify(spec, null, 2) }] }; } }, { definition: { name: "spec_modify", description: "Modify part of an AtomizerSpec. Changes are validated and broadcast to all clients.", inputSchema: { type: "object", properties: { study_name: { type: "string" }, modifications: { type: "array", items: { type: "object", properties: { operation: { enum: ["set", "add", "remove"] }, path: { type: "string", description: "JSONPath to the field" }, value: { description: "New value (for set/add)" } } } } }, required: ["study_name", "modifications"] } }, handler: async (args) => { const results = []; for (const mod of args.modifications) { switch (mod.operation) { case "set": await specManager.patch(mod.path, mod.value, "claude"); results.push(`Set ${mod.path}`); break; case "add": const id = await specManager.addNode(mod.path, mod.value, "claude"); results.push(`Added ${id} at ${mod.path}`); break; case "remove": await specManager.removeNode(mod.path, "claude"); results.push(`Removed ${mod.path}`); break; } } return { content: [{ type: "text", text: JSON.stringify({ success: true, applied: results }, null, 2) }] }; } }, { definition: { name: "spec_add_custom_extractor", description: "Add a custom Python function as an extractor. The function will be available in the optimization workflow.", inputSchema: { type: "object", properties: { study_name: { type: "string" }, function_name: { type: "string", description: "Name of the function" }, description: { type: "string" }, code: { type: "string", description: "Python source code" }, outputs: { type: "array", items: { type: "string" }, description: "List of output names this function produces" }, dependencies: { type: "array", items: { type: "string" }, description: "Python packages required (must be installed)" } }, required: ["study_name", "function_name", "code", "outputs"] } }, handler: async (args) => { // Validate code syntax const validation = await validatePythonCode(args.code); if (!validation.valid) { return { content: [{ type: "text", text: JSON.stringify({ error: validation.error }) }], isError: true }; } const extractorId = await specManager.addCustomFunction( args.function_name, args.code, args.outputs, "claude" ); return { content: [{ type: "text", text: JSON.stringify({ success: true, extractor_id: extractorId, message: `Custom extractor "${args.function_name}" added. Canvas will update automatically.`, outputs: args.outputs }, null, 2) }] }; } }, { definition: { name: "spec_validate", description: "Validate an AtomizerSpec and return detailed report.", inputSchema: { type: "object", properties: { study_name: { type: "string" } }, required: ["study_name"] } }, handler: async (args) => { const report = await specManager.validateAndReport(args.study_name); return { content: [{ type: "text", text: JSON.stringify(report, null, 2) }] }; } }, { definition: { name: "spec_create_from_description", description: "Create a new AtomizerSpec from a natural language description. Claude interprets the description and generates a complete spec.", inputSchema: { type: "object", properties: { study_name: { type: "string" }, description: { type: "string", description: "Natural language description of the optimization" }, model_path: { type: "string", description: "Path to the NX model" } }, required: ["study_name", "description", "model_path"] } }, handler: async (args) => { // This would invoke the StudyCreator with Claude interpretation // ... } } ]; ``` --- ## 6. Intelligent Assistant Integration ### 6.1 Claude Capabilities With the unified spec, Claude can: | Capability | Example User Request | Claude Action | |------------|---------------------|---------------| | **Modify Parameters** | "Set mass weight to 3" | `spec_modify` on `objectives[1].weight` | | **Add Custom Logic** | "Add a manufacturing cost function based on rib count" | `spec_add_custom_extractor` with Python code | | **Restructure Workflow** | "Change from single-objective to multi-objective" | Modify `optimization.algorithm.type` and restructure objectives | | **Validate Setup** | "Check if my config is valid" | `spec_validate` and explain issues | | **Explain Canvas** | "What does this optimization do?" | Read spec, generate natural language summary | ### 6.2 Example Interaction ``` User: "Claude, I want to add a manufacturability score that penalizes designs with more than 10 ribs" Claude: I'll add a custom extractor for manufacturability scoring. Let me create the function: [spec_add_custom_extractor] function_name: "manufacturability_score" code: """ def manufacturability_score(design_vars, fem_results): ''' Calculate manufacturability score based on rib count. Higher score = easier to manufacture. ''' rib_count = design_vars.get('n_lateral_ribs', 0) if rib_count <= 8: score = 100 # Easy elif rib_count <= 10: score = 80 - (rib_count - 8) * 10 # Moderate else: score = 60 - (rib_count - 10) * 15 # Difficult, penalized return max(0, score) """ outputs: ["mfg_score"] Done! I've added a manufacturability extractor that: - Returns 100 for 8 or fewer ribs (easy to manufacture) - Returns 60-80 for 9-10 ribs (moderate) - Returns <60 for 11+ ribs (penalized) The canvas has been updated with a new "Manufacturability Score" node. Would you like me to: 1. Add this as a soft constraint (mfg_score >= 60)? 2. Add it as an objective to maximize? ``` ### 6.3 Real-time Canvas Updates ``` ┌─────────────────────────────────────────────────────────────────┐ │ Real-time Update Flow │ └─────────────────────────────────────────────────────────────────┘ User types: "Change mass weight to 5" │ ▼ ┌─────────────────┐ │ Claude receives │ │ message via WS │ └────────┬────────┘ │ ▼ ┌─────────────────┐ │ Claude calls │ │ spec_modify │ │ path: objectives│ │ [1].weight = 5 │ └────────┬────────┘ │ ▼ ┌─────────────────┐ │ SpecManager │ │ - Validates │ │ - Saves spec │ │ - Broadcasts │ └────────┬────────┘ │ ├────────────────────────────────────┐ │ │ ▼ ▼ ┌─────────────────┐ ┌─────────────────┐ │ Canvas receives │ │ Claude confirms │ │ spec_updated │ │ "Weight updated │ │ WebSocket msg │ │ to 5" │ └────────┬────────┘ └─────────────────┘ │ ▼ ┌─────────────────┐ │ Canvas re- │ │ renders node │ │ with new weight │ │ "Mass (w=5)" │ └─────────────────┘ ``` --- ## 7. API Design ### 7.1 REST Endpoints ```yaml # OpenAPI 3.0 specification (partial) paths: /api/studies/{study_id}/spec: get: summary: Get AtomizerSpec responses: 200: content: application/json: schema: $ref: '#/components/schemas/AtomizerSpec' put: summary: Replace entire AtomizerSpec requestBody: content: application/json: schema: $ref: '#/components/schemas/AtomizerSpec' patch: summary: Partial update to spec requestBody: content: application/json: schema: type: object properties: path: type: string description: JSONPath to field value: description: New value modified_by: type: string /api/studies/{study_id}/spec/nodes: post: summary: Add a new node requestBody: content: application/json: schema: type: object properties: type: enum: [designVar, extractor, objective, constraint] data: type: object /api/studies/{study_id}/spec/nodes/{node_id}: patch: summary: Update node properties delete: summary: Remove node /api/studies/{study_id}/spec/validate: post: summary: Validate spec and return report /api/studies/{study_id}/spec/custom-functions: post: summary: Add custom extractor function requestBody: content: application/json: schema: type: object required: [name, code, outputs] properties: name: type: string code: type: string outputs: type: array items: type: string ``` ### 7.2 WebSocket Protocol ```typescript // WebSocket Messages // Client -> Server interface ClientMessage { type: 'subscribe' | 'patch_node' | 'add_node' | 'remove_node' | 'update_position'; study_id: string; // ... type-specific fields } // Server -> Client interface ServerMessage { type: 'spec_updated' | 'validation_error' | 'node_added' | 'node_removed' | 'connection_ack'; timestamp: string; // ... type-specific fields } // Example: spec_updated { "type": "spec_updated", "timestamp": "2026-01-17T14:30:00Z", "modified_by": "claude", "hash": "sha256:abc123...", "changes": [ { "path": "objectives[1].weight", "old": 1, "new": 5 } ] } ``` --- ## 8. Migration Strategy ### 8.1 Backward Compatibility ```python # optimization_engine/config/migrator.py class SpecMigrator: """ Migrate old optimization_config.json to AtomizerSpec v2. """ MIGRATION_MAP = { # Old field -> New field "design_variables.*.parameter": "design_variables.*.expression_name", "design_variables.*.bounds": lambda dv: {"min": dv["bounds"][0], "max": dv["bounds"][1]}, "objectives.*.goal": "objectives.*.direction", "optimization_settings.n_trials": "optimization.budget.max_trials", "optimization_settings.sampler": lambda s: {"type": s.replace("Sampler", "")}, } def migrate(self, old_config: Dict) -> Dict: """Convert old config to AtomizerSpec v2.""" spec = { "meta": { "version": "2.0", "created_by": "migration", "study_name": old_config.get("study_name", "migrated_study") }, "design_variables": [], "extractors": [], "objectives": [], "constraints": [], "optimization": {}, "canvas": {"edges": []} } # Migrate design variables for i, dv in enumerate(old_config.get("design_variables", [])): spec["design_variables"].append(self._migrate_dv(dv, i)) # Migrate objectives for i, obj in enumerate(old_config.get("objectives", [])): extractor_id, extractor = self._infer_extractor(obj) if extractor not in [e["id"] for e in spec["extractors"]]: spec["extractors"].append(extractor) spec["objectives"].append(self._migrate_objective(obj, i, extractor_id)) # ... continue for other sections return spec def _migrate_dv(self, dv: Dict, index: int) -> Dict: """Migrate a design variable.""" bounds = dv.get("bounds", [dv.get("min", 0), dv.get("max", 1)]) return { "id": f"dv_{index:03d}", "name": dv.get("name", dv.get("parameter", f"param_{index}")), "expression_name": dv.get("expression_name", dv.get("parameter")), "type": dv.get("type", "continuous"), "bounds": {"min": bounds[0], "max": bounds[1]}, "baseline": dv.get("baseline"), "units": dv.get("units", dv.get("description", "")), "enabled": dv.get("enabled", True) } ``` ### 8.2 Migration Script ```bash # CLI tool for batch migration python -m optimization_engine.config.migrate \ --input studies/*/optimization_config.json \ --output-suffix "_spec_v2.json" \ --validate \ --dry-run ``` --- ## 9. Implementation Roadmap ### Phase 1: Foundation (Weeks 1-3) | Task | Priority | Effort | |------|----------|--------| | Define AtomizerSpec JSON Schema v2.0 | P0 | 2 days | | Create SpecManager service (Python) | P0 | 3 days | | Implement spec validation | P0 | 2 days | | Add REST endpoints for spec CRUD | P0 | 3 days | | Create migration script | P0 | 3 days | | Update optimization engine to use spec | P1 | 4 days | **Deliverable**: Backend can read/write/validate AtomizerSpec ### Phase 2: Frontend Integration (Weeks 4-6) | Task | Priority | Effort | |------|----------|--------| | Create SpecRenderer component | P0 | 4 days | | Replace useCanvasStore with useSpecStore | P0 | 3 days | | Implement spec ↔ canvas conversion | P0 | 3 days | | Add WebSocket sync | P1 | 3 days | | Update node panels for full spec fields | P1 | 4 days | | Test bidirectional editing | P0 | 3 days | **Deliverable**: Canvas renders and edits AtomizerSpec directly ### Phase 3: Claude Integration (Weeks 7-9) | Task | Priority | Effort | |------|----------|--------| | Implement spec_* MCP tools | P0 | 4 days | | Add custom function support | P1 | 4 days | | Test Claude → Canvas updates | P0 | 3 days | | Add natural language study creation | P2 | 5 days | | Create Claude assistant prompts | P1 | 2 days | **Deliverable**: Claude can read, modify, and extend AtomizerSpec ### Phase 4: Polish & Testing (Weeks 10-12) | Task | Priority | Effort | |------|----------|--------| | Migrate all existing studies | P0 | 3 days | | Integration testing | P0 | 5 days | | Documentation | P1 | 3 days | | Performance optimization | P2 | 3 days | | User acceptance testing | P0 | 4 days | **Deliverable**: Production-ready unified configuration system --- ## Appendix A: Comparison Table | Feature | Current | Proposed | |---------|---------|----------| | Config formats | 4+ variants | 1 (AtomizerSpec) | | Canvas ↔ Config sync | Lossy, manual | Lossless, automatic | | Claude modifications | Returns JSON for manual copy | Direct spec modification | | Custom extractors | Edit Python code | Declare in spec | | Real-time updates | None | WebSocket-driven | | Validation | Partial Python | Full JSON Schema + semantic | | Canvas layout | Lost on reload | Persisted in spec | --- ## Appendix B: Glossary | Term | Definition | |------|------------| | **AtomizerSpec** | The unified JSON configuration format (v2.0) | | **SpecManager** | Backend service for all spec operations | | **SpecRenderer** | Frontend component that renders spec as canvas | | **Custom Extractor** | User-defined Python function declared in spec | | **Spec Patch** | Partial update to a single field in the spec | --- ## Appendix C: File Locations (Proposed) ``` atomizer-dashboard/ ├── backend/api/ │ ├── services/ │ │ └── spec_manager.py # NEW: Core spec management │ ├── routes/ │ │ └── spec.py # NEW: REST endpoints for spec │ └── schemas/ │ └── atomizer_spec_v2.json # NEW: JSON Schema │ ├── frontend/src/ │ ├── hooks/ │ │ └── useSpecStore.ts # NEW: Spec state management │ ├── components/canvas/ │ │ └── SpecRenderer.tsx # NEW: Spec-based canvas │ └── lib/spec/ │ ├── types.ts # NEW: TypeScript types │ └── converter.ts # NEW: Spec ↔ ReactFlow │ mcp-server/atomizer-tools/ └── src/tools/ └── spec_tools.ts # NEW: MCP tools for spec optimization_engine/ ├── config/ │ ├── migrator.py # NEW: v1 → v2 migration │ └── spec_loader.py # NEW: AtomizerSpec loader └── schemas/ └── atomizer_spec_v2.json # Canonical schema ``` --- *This document is the master plan for implementing Atomizer's Unified Configuration Architecture. All implementation work should reference this document.*