Atomizer/docs/plans/UNIFIED_CONFIGURATION_ARCHITECTURE.md

Atomizer Unified Configuration Architecture

Master Design Document

Version: 1.0 Date: January 2026 Status: Proposed Architecture Author: Atomizer Architecture Review

---

Executive Summary

This document presents a comprehensive analysis of Atomizer's current configuration architecture and proposes a Unified Configuration System that establishes a single source of truth for optimization studies. The new architecture enables:

1. Single Source of Truth: One canonical JSON schema (AtomizerSpec) used everywhere
2. Bidirectional Sync: Canvas ↔ Config ↔ Backend with zero data loss
3. Intelligent Manipulation: Claude can dynamically modify configs, add custom functions, and validate changes
4. Real-time Collaboration: WebSocket-driven updates between Dashboard, Canvas, and Claude Assistant

---

Table of Contents

1. Current State Analysis
2. Problem Statement
3. Proposed Architecture
4. AtomizerSpec Schema
5. Component Architecture
6. Intelligent Assistant Integration
7. API Design
8. Migration Strategy
9. Implementation Roadmap

---

1. Current State Analysis

1.1 Architecture Overview

The current system has four distinct configuration representations that don't fully align:

┌─────────────────────────────────────────────────────────────────────────────┐
│                        CURRENT ARCHITECTURE (FRAGMENTED)                     │
└─────────────────────────────────────────────────────────────────────────────┘

       Canvas UI                    Backend                    Optimization
    ┌────────────────┐         ┌────────────────┐         ┌────────────────┐
    │  ReactFlow     │         │   FastAPI      │         │  Python Engine │
    │  Nodes/Edges   │         │   Routes       │         │  ConfigManager │
    │                │         │                │         │                │
    │  ┌──────────┐  │         │  ┌──────────┐  │         │  ┌──────────┐  │
    │  │ Canvas   │──┼──LOSSY──┼─▶│ Intent   │──┼──LOSSY──┼─▶│ Config   │  │
    │  │ State    │  │         │  │ JSON     │  │         │  │ .json    │  │
    │  └──────────┘  │         │  └──────────┘  │         │  └──────────┘  │
    │       ▲        │         │       │        │         │       │        │
    │       │ LOSSY  │         │       ▼        │         │       ▼        │
    │       │        │         │  ┌──────────┐  │         │  ┌──────────┐  │
    │  Load from     │◀─LOSSY──┼──│ Claude   │  │         │  │ Optuna   │  │
    │  config        │         │  │ Assistant│  │         │  │ Database │  │
    └────────────────┘         └────────────────┘         └────────────────┘

1.2 Current Config Formats (4+ Variants)

Variant	Used By	Key Differences
Mirror/Zernike	m1_mirror studies	extractor_config, zernike_settings, subcases
Drone/Structural	drone_gimbal	extraction.{action, domain, params}, bounds[]
Canvas Intent	Canvas UI	Simplified, loses metadata
Legacy Schema	optimization_config_schema.json	parameter, bounds[], goal

1.3 Field Naming Chaos

Concept	Canvas Intent	Mirror Config	Drone Config	Schema
DV bounds	min, max	min, max	bounds: [min, max]	bounds: [min, max]
DV name	name	expression_name	parameter	parameter
Objective dir	direction	direction	goal	goal
Extractor	extractor: "E5"	extractor_config: {}	extraction: {}	extraction: {}
Trials	max_trials	n_trials	n_trials	n_trials

1.4 Data Flow Problems

PROBLEM 1: Information Loss During Canvas → Config Conversion
═══════════════════════════════════════════════════════════════

Canvas Node:                    Intent:                     Config:
┌─────────────────────┐        ┌─────────────────┐        ┌─────────────────┐
│ Extractor E8        │        │ extractors: [{  │        │ objectives: [{  │
│ - nModes: 40        │───────▶│   id: "E8",     │───────▶│   extractor:    │
│ - innerRadius: 58   │  LOST  │   name: "..."   │  LOST  │     "E8"        │
│ - subcases: [1,2,3] │───────▶│ }]              │───────▶│ }]              │
│ - filterLowOrders: 6│        │                 │        │                 │
└─────────────────────┘        └─────────────────┘        └─────────────────┘
                                    │                            │
                                    │   MISSING:                 │   MISSING:
                                    │   - nModes                 │   - All config
                                    │   - innerRadius            │   - zernike_settings
                                    │   - subcases               │   - extraction_method
                                    │   - filterLowOrders        │


PROBLEM 2: Config → Canvas Loading is Incomplete
═══════════════════════════════════════════════════════════════

Real Config (m1_mirror):              Canvas Display:
┌───────────────────────────────┐    ┌─────────────────────────┐
│ design_variables: [           │    │ 11 DesignVar nodes      │ ✓
│   { name, min, max, ... }     │    │                         │
│ ]                             │    │                         │
│                               │    │                         │
│ objectives: [                 │    │ 3 Objective nodes       │ ✓
│   { extractor_config: {       │    │ (but extractor_config   │
│     target_subcase: "3",      │    │  NOT displayed)         │ ✗
│     metric: "rel_filtered_rms"│    │                         │
│   }}                          │    │                         │
│ ]                             │    │                         │
│                               │    │                         │
│ hard_constraints: [           │    │ Converted to objectives │ ✗
│   { name, limit, penalty }    │    │ (loses constraint type) │
│ ]                             │    │                         │
│                               │    │                         │
│ sat_settings: {               │    │ NOT LOADED AT ALL       │ ✗
│   n_ensemble_models: 10,      │    │                         │
│   hidden_dims: [256, 128]     │    │                         │
│ }                             │    │                         │
└───────────────────────────────┘    └─────────────────────────┘


PROBLEM 3: No Bidirectional Sync
═══════════════════════════════════════════════════════════════

User edits config.json manually
         │
         ▼
Canvas doesn't know about change
         │
         ▼
User refreshes canvas → loses their canvas-only edits
         │
         ▼
Conflict between what config says and what canvas shows

1.5 Missing Backend Endpoints

Required Endpoint	Status	Impact
POST /studies/create-from-intent	Missing	Canvas can't create studies
POST /studies/{id}/validate-config	Missing	No schema validation
PUT /studies/{id}/config (atomic)	Partial	No conflict detection
POST /canvas/apply-modification	Missing	Claude can't modify canvas
WebSocket /studies/{id}/sync	Missing	No real-time updates

---

2. Problem Statement

2.1 Core Issues

1. No Single Source of Truth: Multiple config formats mean the same study looks different depending on where you view it

2. Lossy Conversions: Converting Canvas → Intent → Config loses information; converting Config → Canvas is incomplete

3. No Real-time Sync: Changes in one place (config file, canvas, Claude) don't propagate to others

4. Claude Can't Modify Canvas Directly: MCP tools return JSON but nothing applies changes to the frontend

5. No Intelligent Extensibility: User can't ask Claude to "add a custom extraction function" because there's no plugin system

2.2 User Pain Points

Scenario	Current Experience	Desired Experience
"Load my existing study into canvas"	Partial load, loses extractor settings	Complete bidirectional load
"Claude, change the mass objective weight to 5"	Claude returns JSON, user must manually apply	Canvas updates in real-time
"I need a custom Zernike RMS calculation"	Must edit Python code directly	Claude adds function, canvas shows new node
"Show me what the config looks like"	Multiple representations exist	One canonical view everywhere

---

3. Proposed Architecture

3.1 Unified Architecture Overview

┌─────────────────────────────────────────────────────────────────────────────┐
│                     PROPOSED ARCHITECTURE (UNIFIED)                          │
└─────────────────────────────────────────────────────────────────────────────┘

                              ┌─────────────────────┐
                              │    AtomizerSpec     │
                              │   (Single Source    │
                              │    of Truth)        │
                              │                     │
                              │  atomizer_spec.json │
                              └──────────┬──────────┘
                                         │
              ┌──────────────────────────┼──────────────────────────┐
              │                          │                          │
              ▼                          ▼                          ▼
    ┌─────────────────┐       ┌─────────────────┐       ┌─────────────────┐
    │   Canvas UI     │       │  Backend API    │       │ Claude Assistant│
    │                 │       │                 │       │                 │
    │ ReactFlow +     │◀─────▶│ FastAPI +       │◀─────▶│ MCP Tools +     │
    │ SpecRenderer    │  WS   │ SpecManager     │  WS   │ SpecModifier    │
    │                 │       │                 │       │                 │
    └─────────────────┘       └─────────────────┘       └─────────────────┘
              │                          │                          │
              │        WebSocket Real-time Sync Bus                 │
              └──────────────────────────┴──────────────────────────┘
                                         │
                                         ▼
                              ┌─────────────────────┐
                              │  Optimization       │
                              │  Engine             │
                              │                     │
                              │  Uses AtomizerSpec  │
                              │  directly           │
                              └─────────────────────┘

3.2 Key Principles

1. AtomizerSpec is THE config: One JSON schema, versioned, validated everywhere
2. No Intermediate Formats: Canvas renders AtomizerSpec directly; no "Intent" conversion
3. Changes Flow Through Backend: All modifications go through SpecManager
4. WebSocket Sync: All clients receive real-time updates
5. Extensible Functions: Custom extractors/functions defined IN the spec

3.3 Data Flow (New)

                        AtomizerSpec (atomizer_spec.json)
                                    │
                                    │ Read by all
          ┌─────────────────────────┼─────────────────────────┐
          │                         │                         │
          ▼                         ▼                         ▼
    Canvas Renderer          Backend Loader           Claude Reader
          │                         │                         │
          │ User edits node         │                         │ "Change weight"
          │                         │                         │
          ▼                         ▼                         ▼
    PATCH /spec/nodes/{id}    (validates)            MCP: spec_modify
          │                         │                         │
          └─────────────────────────┼─────────────────────────┘
                                    │
                                    ▼
                            SpecManager
                            ┌─────────────────────────┐
                            │ 1. Validate change      │
                            │ 2. Apply to spec        │
                            │ 3. Persist to disk      │
                            │ 4. Broadcast via WS     │
                            └─────────────────────────┘
                                    │
                                    │ WebSocket: spec_updated
          ┌─────────────────────────┼─────────────────────────┐
          │                         │                         │
          ▼                         ▼                         ▼
    Canvas re-renders        Other clients            Claude sees new state

---

4. AtomizerSpec Schema

4.1 Schema Design Philosophy

• Self-describing: Schema version and capabilities in the file
• Complete: ALL information needed for optimization, canvas, and reporting
• Extensible: Custom functions, extractors, and plugins declared in-spec
• Validated: JSON Schema validation at every boundary

4.2 AtomizerSpec v2.0 Schema

{
  "$schema": "https://atomizer.io/schemas/atomizer_spec_v2.json",
  "$id": "atomizer_spec",

  "meta": {
    "version": "2.0",
    "created": "2026-01-17T12:00:00Z",
    "modified": "2026-01-17T14:30:00Z",
    "created_by": "canvas",
    "modified_by": "claude",
    "study_name": "m1_mirror_optimization_v15",
    "description": "Multi-objective mirror optimization with Zernike WFE",
    "tags": ["mirror", "zernike", "multi-objective"]
  },

  "model": {
    "nx_part": {
      "path": "C:/Studies/m1_mirror/model/M1_Mirror.prt",
      "hash": "sha256:abc123...",
      "idealized_part": "M1_Mirror_fem1_i.prt"
    },
    "fem": {
      "path": "model/M1_Mirror_fem1.fem",
      "element_count": 45000,
      "node_count": 12000
    },
    "sim": {
      "path": "model/M1_Mirror_sim1.sim",
      "solver": "nastran",
      "solution_type": "SOL101",
      "subcases": [
        { "id": 1, "name": "Gravity 0deg", "type": "static" },
        { "id": 2, "name": "Gravity 20deg", "type": "static" },
        { "id": 3, "name": "Gravity 40deg", "type": "static" }
      ]
    }
  },

  "design_variables": [
    {
      "id": "dv_001",
      "name": "Lateral Rib Count",
      "expression_name": "n_lateral_ribs",
      "type": "integer",
      "bounds": { "min": 6, "max": 12 },
      "baseline": 9,
      "units": "count",
      "enabled": true,
      "description": "Number of lateral support ribs",
      "canvas_position": { "x": 50, "y": 100 }
    },
    {
      "id": "dv_002",
      "name": "Facesheet Thickness",
      "expression_name": "facesheet_t",
      "type": "continuous",
      "bounds": { "min": 2.0, "max": 8.0 },
      "baseline": 4.5,
      "units": "mm",
      "enabled": true,
      "step": 0.1,
      "canvas_position": { "x": 50, "y": 200 }
    }
  ],

  "extractors": [
    {
      "id": "ext_001",
      "name": "Zernike WFE Extractor",
      "type": "zernike_opd",
      "builtin": true,
      "config": {
        "inner_radius_mm": 58.0,
        "outer_radius_mm": 330.0,
        "n_modes": 40,
        "filter_low_orders": 6,
        "displacement_unit": "mm",
        "reference_subcase": 1
      },
      "outputs": [
        { "name": "wfe_rms_20deg", "subcase": 2, "metric": "filtered_rms_nm" },
        { "name": "wfe_rms_40deg", "subcase": 3, "metric": "filtered_rms_nm" },
        { "name": "wfe_p2v_40deg", "subcase": 3, "metric": "pv_nm" }
      ],
      "canvas_position": { "x": 740, "y": 100 }
    },
    {
      "id": "ext_002",
      "name": "Mass Extractor",
      "type": "mass",
      "builtin": true,
      "config": {
        "source": "expression",
        "expression_name": "total_mass_kg"
      },
      "outputs": [
        { "name": "mass_kg", "metric": "total" }
      ],
      "canvas_position": { "x": 740, "y": 300 }
    },
    {
      "id": "ext_003",
      "name": "Custom Manufacturability Score",
      "type": "custom_function",
      "builtin": false,
      "function": {
        "name": "calc_manufacturability",
        "module": "custom_extractors.manufacturability",
        "signature": "(design_vars: dict, fem_results: dict) -> float",
        "source_code": "def calc_manufacturability(design_vars, fem_results):\n    # Custom logic here\n    rib_count = design_vars['n_lateral_ribs']\n    thickness = design_vars['facesheet_t']\n    score = 100 - (rib_count * 5) - (10 - thickness) * 3\n    return max(0, min(100, score))"
      },
      "outputs": [
        { "name": "mfg_score", "metric": "score" }
      ],
      "canvas_position": { "x": 740, "y": 500 }
    }
  ],

  "objectives": [
    {
      "id": "obj_001",
      "name": "Minimize WFE at 40deg",
      "direction": "minimize",
      "weight": 5.0,
      "source": {
        "extractor_id": "ext_001",
        "output_name": "wfe_rms_40deg"
      },
      "target": 5.0,
      "units": "nm",
      "canvas_position": { "x": 1020, "y": 100 }
    },
    {
      "id": "obj_002",
      "name": "Minimize Mass",
      "direction": "minimize",
      "weight": 1.0,
      "source": {
        "extractor_id": "ext_002",
        "output_name": "mass_kg"
      },
      "target": 100.0,
      "units": "kg",
      "canvas_position": { "x": 1020, "y": 200 }
    }
  ],

  "constraints": [
    {
      "id": "con_001",
      "name": "Max Mass Limit",
      "type": "hard",
      "operator": "<=",
      "threshold": 120.0,
      "source": {
        "extractor_id": "ext_002",
        "output_name": "mass_kg"
      },
      "penalty_config": {
        "method": "quadratic",
        "weight": 1000.0,
        "margin": 5.0
      },
      "canvas_position": { "x": 1020, "y": 400 }
    },
    {
      "id": "con_002",
      "name": "Minimum Manufacturability",
      "type": "soft",
      "operator": ">=",
      "threshold": 60.0,
      "source": {
        "extractor_id": "ext_003",
        "output_name": "mfg_score"
      },
      "penalty_config": {
        "method": "linear",
        "weight": 10.0
      },
      "canvas_position": { "x": 1020, "y": 500 }
    }
  ],

  "optimization": {
    "algorithm": {
      "type": "NSGA-II",
      "config": {
        "population_size": 50,
        "n_generations": 100,
        "mutation_prob": null,
        "crossover_prob": 0.9,
        "seed": 42
      }
    },
    "budget": {
      "max_trials": 500,
      "max_time_hours": 48,
      "convergence_patience": 50
    },
    "surrogate": {
      "enabled": true,
      "type": "ensemble",
      "config": {
        "n_models": 10,
        "architecture": [256, 128, 64],
        "train_every_n_trials": 20,
        "min_training_samples": 30,
        "acquisition_candidates": 10000,
        "fea_validations_per_round": 5
      }
    },
    "canvas_position": { "x": 1300, "y": 150 }
  },

  "workflow": {
    "stages": [
      {
        "id": "stage_exploration",
        "name": "Design Space Exploration",
        "algorithm": "RandomSearch",
        "trials": 30,
        "purpose": "Build initial training data"
      },
      {
        "id": "stage_optimization",
        "name": "Surrogate-Assisted Optimization",
        "algorithm": "SAT_v3",
        "trials": 470,
        "purpose": "Efficient optimization with neural acceleration"
      }
    ],
    "transitions": [
      {
        "from": "stage_exploration",
        "to": "stage_optimization",
        "condition": "trial_count >= 30"
      }
    ]
  },

  "reporting": {
    "auto_report": true,
    "report_triggers": ["new_pareto", "every_50_trials", "convergence"],
    "insights": [
      {
        "type": "zernike_visualization",
        "for_trials": "pareto_front",
        "config": { "include_html": true }
      },
      {
        "type": "convergence_plot",
        "config": { "show_pareto_evolution": true }
      }
    ]
  },

  "canvas": {
    "layout_version": "2.0",
    "viewport": { "x": 0, "y": 0, "zoom": 1.0 },
    "edges": [
      { "source": "dv_001", "target": "model" },
      { "source": "dv_002", "target": "model" },
      { "source": "model", "target": "solver" },
      { "source": "solver", "target": "ext_001" },
      { "source": "solver", "target": "ext_002" },
      { "source": "ext_001", "target": "obj_001" },
      { "source": "ext_002", "target": "obj_002" },
      { "source": "ext_002", "target": "con_001" },
      { "source": "obj_001", "target": "optimization" },
      { "source": "obj_002", "target": "optimization" },
      { "source": "con_001", "target": "optimization" }
    ],
    "groups": [
      { "id": "grp_inputs", "name": "Design Inputs", "node_ids": ["dv_001", "dv_002"] },
      { "id": "grp_physics", "name": "Physics Extraction", "node_ids": ["ext_001", "ext_002"] }
    ]
  }
}

4.3 Schema Versioning

Version	Status	Key Changes
1.0	Legacy	Original optimization_config.json
2.0	Proposed	Unified spec with canvas, custom functions, workflow
2.1	Future	Multi-fidelity, parallel execution

4.4 JSON Schema Definition

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "https://atomizer.io/schemas/atomizer_spec_v2.json",
  "title": "AtomizerSpec v2.0",
  "type": "object",
  "required": ["meta", "model", "design_variables", "extractors", "objectives", "optimization"],

  "properties": {
    "meta": {
      "type": "object",
      "required": ["version", "study_name"],
      "properties": {
        "version": { "type": "string", "pattern": "^2\\.\\d+$" },
        "study_name": { "type": "string", "pattern": "^[a-z0-9_]+$" },
        "created": { "type": "string", "format": "date-time" },
        "modified": { "type": "string", "format": "date-time" },
        "created_by": { "enum": ["canvas", "claude", "api", "migration"] },
        "modified_by": { "type": "string" },
        "description": { "type": "string" },
        "tags": { "type": "array", "items": { "type": "string" } }
      }
    },

    "design_variables": {
      "type": "array",
      "minItems": 1,
      "items": {
        "$ref": "#/definitions/design_variable"
      }
    },

    "extractors": {
      "type": "array",
      "minItems": 1,
      "items": {
        "$ref": "#/definitions/extractor"
      }
    },

    "objectives": {
      "type": "array",
      "minItems": 1,
      "items": {
        "$ref": "#/definitions/objective"
      }
    }
  },

  "definitions": {
    "design_variable": {
      "type": "object",
      "required": ["id", "name", "expression_name", "type", "bounds"],
      "properties": {
        "id": { "type": "string", "pattern": "^dv_\\d{3}$" },
        "name": { "type": "string" },
        "expression_name": { "type": "string" },
        "type": { "enum": ["continuous", "integer", "categorical"] },
        "bounds": {
          "type": "object",
          "required": ["min", "max"],
          "properties": {
            "min": { "type": "number" },
            "max": { "type": "number" }
          }
        },
        "baseline": { "type": "number" },
        "units": { "type": "string" },
        "enabled": { "type": "boolean", "default": true },
        "canvas_position": { "$ref": "#/definitions/position" }
      }
    },

    "extractor": {
      "type": "object",
      "required": ["id", "name", "type", "outputs"],
      "properties": {
        "id": { "type": "string", "pattern": "^ext_\\d{3}$" },
        "name": { "type": "string" },
        "type": { "type": "string" },
        "builtin": { "type": "boolean" },
        "config": { "type": "object" },
        "function": {
          "type": "object",
          "properties": {
            "name": { "type": "string" },
            "module": { "type": "string" },
            "source_code": { "type": "string" }
          }
        },
        "outputs": {
          "type": "array",
          "items": {
            "type": "object",
            "required": ["name"],
            "properties": {
              "name": { "type": "string" },
              "metric": { "type": "string" },
              "subcase": { "type": "integer" }
            }
          }
        }
      }
    },

    "objective": {
      "type": "object",
      "required": ["id", "name", "direction", "source"],
      "properties": {
        "id": { "type": "string", "pattern": "^obj_\\d{3}$" },
        "name": { "type": "string" },
        "direction": { "enum": ["minimize", "maximize"] },
        "weight": { "type": "number", "minimum": 0 },
        "source": {
          "type": "object",
          "required": ["extractor_id", "output_name"],
          "properties": {
            "extractor_id": { "type": "string" },
            "output_name": { "type": "string" }
          }
        },
        "target": { "type": "number" },
        "units": { "type": "string" }
      }
    },

    "position": {
      "type": "object",
      "properties": {
        "x": { "type": "number" },
        "y": { "type": "number" }
      }
    }
  }
}

---

5. Component Architecture

5.1 Backend: SpecManager Service

# atomizer-dashboard/backend/api/services/spec_manager.py

from pathlib import Path
from datetime import datetime
import json
import hashlib
from typing import Optional, Dict, Any, List
from pydantic import BaseModel
import jsonschema

class SpecManager:
    """
    Central service for managing AtomizerSpec.
    All modifications go through this service.
    """

    def __init__(self, study_path: Path):
        self.study_path = study_path
        self.spec_path = study_path / "atomizer_spec.json"
        self.schema = self._load_schema()
        self._subscribers: List[WebSocketConnection] = []

    def load(self) -> Dict[str, Any]:
        """Load and validate the spec."""
        with open(self.spec_path) as f:
            spec = json.load(f)
        self._validate(spec)
        return spec

    def save(self, spec: Dict[str, Any], modified_by: str = "api"):
        """Save spec with validation and broadcast."""
        # Update metadata
        spec["meta"]["modified"] = datetime.utcnow().isoformat() + "Z"
        spec["meta"]["modified_by"] = modified_by

        # Validate
        self._validate(spec)

        # Compute hash for conflict detection
        spec_hash = self._compute_hash(spec)

        # Atomic write
        temp_path = self.spec_path.with_suffix(".tmp")
        with open(temp_path, "w") as f:
            json.dump(spec, f, indent=2)
        temp_path.replace(self.spec_path)

        # Broadcast to all subscribers
        self._broadcast({
            "type": "spec_updated",
            "hash": spec_hash,
            "modified_by": modified_by,
            "timestamp": spec["meta"]["modified"]
        })

        return spec_hash

    def patch(self, path: str, value: Any, modified_by: str = "api") -> Dict[str, Any]:
        """
        Apply a JSON Patch-style modification.
        path: JSONPath like "design_variables[0].bounds.max"
        """
        spec = self.load()
        self._apply_patch(spec, path, value)
        self.save(spec, modified_by)
        return spec

    def add_node(self, node_type: str, data: Dict[str, Any], modified_by: str = "canvas"):
        """Add a new node (design var, extractor, objective, etc.)"""
        spec = self.load()

        # Generate ID
        node_id = self._generate_id(node_type, spec)
        data["id"] = node_id

        # Add canvas position if not provided
        if "canvas_position" not in data:
            data["canvas_position"] = self._auto_position(node_type, spec)

        # Add to appropriate section
        section = self._get_section_for_type(node_type)
        spec[section].append(data)

        self.save(spec, modified_by)
        return node_id

    def remove_node(self, node_id: str, modified_by: str = "canvas"):
        """Remove a node and all edges referencing it."""
        spec = self.load()

        # Find and remove node
        for section in ["design_variables", "extractors", "objectives", "constraints"]:
            spec[section] = [n for n in spec.get(section, []) if n.get("id") != node_id]

        # Remove edges referencing this node
        if "canvas" in spec and "edges" in spec["canvas"]:
            spec["canvas"]["edges"] = [
                e for e in spec["canvas"]["edges"]
                if e["source"] != node_id and e["target"] != node_id
            ]

        self.save(spec, modified_by)

    def add_custom_function(self, name: str, code: str, outputs: List[str], modified_by: str = "claude"):
        """
        Add a custom extractor function.
        Claude can call this to add new physics extraction logic.
        """
        spec = self.load()

        # Validate Python syntax
        compile(code, f"<custom:{name}>", "exec")

        extractor = {
            "id": self._generate_id("ext", spec),
            "name": name,
            "type": "custom_function",
            "builtin": False,
            "function": {
                "name": name,
                "module": "custom_extractors.dynamic",
                "source_code": code
            },
            "outputs": [{"name": o, "metric": "custom"} for o in outputs]
        }

        spec["extractors"].append(extractor)
        self.save(spec, modified_by)

        return extractor["id"]

    def validate_and_report(self) -> Dict[str, Any]:
        """Run full validation and return detailed report."""
        spec = self.load()

        report = {
            "valid": True,
            "errors": [],
            "warnings": [],
            "summary": {}
        }

        # Schema validation
        try:
            self._validate(spec)
        except jsonschema.ValidationError as e:
            report["valid"] = False
            report["errors"].append({
                "type": "schema",
                "path": list(e.absolute_path),
                "message": e.message
            })

        # Semantic validation
        self._validate_semantic(spec, report)

        # Summary
        report["summary"] = {
            "design_variables": len(spec.get("design_variables", [])),
            "extractors": len(spec.get("extractors", [])),
            "objectives": len(spec.get("objectives", [])),
            "constraints": len(spec.get("constraints", [])),
            "custom_functions": sum(1 for e in spec.get("extractors", []) if not e.get("builtin", True))
        }

        return report

    def subscribe(self, ws: WebSocketConnection):
        """Subscribe to spec changes."""
        self._subscribers.append(ws)

    def unsubscribe(self, ws: WebSocketConnection):
        """Unsubscribe from spec changes."""
        self._subscribers.remove(ws)

    def _broadcast(self, message: Dict[str, Any]):
        """Broadcast to all subscribers."""
        for ws in self._subscribers:
            ws.send_json(message)

5.2 Frontend: SpecRenderer Component

// atomizer-dashboard/frontend/src/components/canvas/SpecRenderer.tsx

import React, { useCallback, useEffect } from 'react';
import ReactFlow, { Node, Edge, useNodesState, useEdgesState } from 'reactflow';
import { useSpecStore } from '../../hooks/useSpecStore';
import { useWebSocket } from '../../hooks/useWebSocket';

interface AtomizerSpec {
  meta: SpecMeta;
  design_variables: DesignVariable[];
  extractors: Extractor[];
  objectives: Objective[];
  constraints: Constraint[];
  optimization: OptimizationConfig;
  canvas: CanvasConfig;
}

export function SpecRenderer({ studyId }: { studyId: string }) {
  const { spec, setSpec, updateNode, addNode, removeNode } = useSpecStore();
  const { subscribe, send } = useWebSocket(`/api/studies/${studyId}/sync`);

  // Convert spec to ReactFlow nodes
  const [nodes, setNodes, onNodesChange] = useNodesState(
    specToNodes(spec)
  );

  // Convert spec edges to ReactFlow edges
  const [edges, setEdges, onEdgesChange] = useEdgesState(
    spec?.canvas?.edges?.map(edgeToReactFlow) || []
  );

  // Subscribe to real-time updates
  useEffect(() => {
    const unsubscribe = subscribe('spec_updated', (data) => {
      // Reload spec from server
      fetchSpec(studyId).then(setSpec);
    });

    return unsubscribe;
  }, [studyId]);

  // When spec changes, update nodes
  useEffect(() => {
    if (spec) {
      setNodes(specToNodes(spec));
      setEdges(spec.canvas?.edges?.map(edgeToReactFlow) || []);
    }
  }, [spec]);

  // Handle node property changes
  const onNodeDataChange = useCallback((nodeId: string, data: Partial<NodeData>) => {
    // Send PATCH to backend
    send({
      type: 'patch_node',
      node_id: nodeId,
      data
    });
  }, [send]);

  // Handle node position changes (for canvas layout)
  const onNodeDragStop = useCallback((event: React.MouseEvent, node: Node) => {
    send({
      type: 'update_position',
      node_id: node.id,
      position: node.position
    });
  }, [send]);

  return (
    <ReactFlow
      nodes={nodes}
      edges={edges}
      onNodesChange={onNodesChange}
      onEdgesChange={onEdgesChange}
      onNodeDragStop={onNodeDragStop}
      nodeTypes={customNodeTypes}
      fitView
    >
      {/* Toolbars, panels, etc. */}
    </ReactFlow>
  );
}

// Convert AtomizerSpec to ReactFlow nodes
function specToNodes(spec: AtomizerSpec | null): Node[] {
  if (!spec) return [];

  const nodes: Node[] = [];

  // Model node (synthetic)
  nodes.push({
    id: 'model',
    type: 'model',
    position: { x: 280, y: 50 },
    data: {
      label: spec.meta.study_name,
      filePath: spec.model.sim.path,
      configured: true
    }
  });

  // Solver node (synthetic)
  nodes.push({
    id: 'solver',
    type: 'solver',
    position: { x: 510, y: 50 },
    data: {
      label: spec.model.sim.solution_type,
      solverType: spec.model.sim.solution_type,
      configured: true
    }
  });

  // Design variables
  for (const dv of spec.design_variables) {
    nodes.push({
      id: dv.id,
      type: 'designVar',
      position: dv.canvas_position || { x: 50, y: nodes.length * 100 },
      data: {
        label: dv.name,
        expressionName: dv.expression_name,
        minValue: dv.bounds.min,
        maxValue: dv.bounds.max,
        baseline: dv.baseline,
        unit: dv.units,
        enabled: dv.enabled,
        configured: true
      }
    });
  }

  // Extractors
  for (const ext of spec.extractors) {
    nodes.push({
      id: ext.id,
      type: 'extractor',
      position: ext.canvas_position || { x: 740, y: nodes.length * 100 },
      data: {
        label: ext.name,
        extractorId: ext.id,
        extractorType: ext.type,
        builtin: ext.builtin,
        config: ext.config,
        outputs: ext.outputs,
        hasCustomCode: !ext.builtin,
        configured: true
      }
    });
  }

  // Objectives
  for (const obj of spec.objectives) {
    nodes.push({
      id: obj.id,
      type: 'objective',
      position: obj.canvas_position || { x: 1020, y: nodes.length * 100 },
      data: {
        label: obj.name,
        name: obj.name,
        direction: obj.direction,
        weight: obj.weight,
        target: obj.target,
        units: obj.units,
        source: obj.source,
        configured: true
      }
    });
  }

  // Constraints
  for (const con of spec.constraints || []) {
    nodes.push({
      id: con.id,
      type: 'constraint',
      position: con.canvas_position || { x: 1020, y: nodes.length * 100 },
      data: {
        label: con.name,
        name: con.name,
        operator: con.operator,
        threshold: con.threshold,
        constraintType: con.type,
        source: con.source,
        configured: true
      }
    });
  }

  // Optimization node
  nodes.push({
    id: 'optimization',
    type: 'algorithm',
    position: spec.optimization.canvas_position || { x: 1300, y: 150 },
    data: {
      label: spec.optimization.algorithm.type,
      method: spec.optimization.algorithm.type,
      maxTrials: spec.optimization.budget.max_trials,
      config: spec.optimization.algorithm.config,
      surrogate: spec.optimization.surrogate,
      configured: true
    }
  });

  return nodes;
}

5.3 MCP Tools for Claude

// mcp-server/atomizer-tools/src/tools/spec_tools.ts

export const specTools: AtomizerTool[] = [
  {
    definition: {
      name: "spec_get",
      description: "Get the current AtomizerSpec for a study. Returns the complete configuration.",
      inputSchema: {
        type: "object",
        properties: {
          study_name: { type: "string", description: "Study name" }
        },
        required: ["study_name"]
      }
    },
    handler: async (args) => {
      const spec = await specManager.load(args.study_name);
      return { content: [{ type: "text", text: JSON.stringify(spec, null, 2) }] };
    }
  },

  {
    definition: {
      name: "spec_modify",
      description: "Modify part of an AtomizerSpec. Changes are validated and broadcast to all clients.",
      inputSchema: {
        type: "object",
        properties: {
          study_name: { type: "string" },
          modifications: {
            type: "array",
            items: {
              type: "object",
              properties: {
                operation: { enum: ["set", "add", "remove"] },
                path: { type: "string", description: "JSONPath to the field" },
                value: { description: "New value (for set/add)" }
              }
            }
          }
        },
        required: ["study_name", "modifications"]
      }
    },
    handler: async (args) => {
      const results = [];
      for (const mod of args.modifications) {
        switch (mod.operation) {
          case "set":
            await specManager.patch(mod.path, mod.value, "claude");
            results.push(`Set ${mod.path}`);
            break;
          case "add":
            const id = await specManager.addNode(mod.path, mod.value, "claude");
            results.push(`Added ${id} at ${mod.path}`);
            break;
          case "remove":
            await specManager.removeNode(mod.path, "claude");
            results.push(`Removed ${mod.path}`);
            break;
        }
      }
      return {
        content: [{
          type: "text",
          text: JSON.stringify({ success: true, applied: results }, null, 2)
        }]
      };
    }
  },

  {
    definition: {
      name: "spec_add_custom_extractor",
      description: "Add a custom Python function as an extractor. The function will be available in the optimization workflow.",
      inputSchema: {
        type: "object",
        properties: {
          study_name: { type: "string" },
          function_name: { type: "string", description: "Name of the function" },
          description: { type: "string" },
          code: { type: "string", description: "Python source code" },
          outputs: {
            type: "array",
            items: { type: "string" },
            description: "List of output names this function produces"
          },
          dependencies: {
            type: "array",
            items: { type: "string" },
            description: "Python packages required (must be installed)"
          }
        },
        required: ["study_name", "function_name", "code", "outputs"]
      }
    },
    handler: async (args) => {
      // Validate code syntax
      const validation = await validatePythonCode(args.code);
      if (!validation.valid) {
        return {
          content: [{ type: "text", text: JSON.stringify({ error: validation.error }) }],
          isError: true
        };
      }

      const extractorId = await specManager.addCustomFunction(
        args.function_name,
        args.code,
        args.outputs,
        "claude"
      );

      return {
        content: [{
          type: "text",
          text: JSON.stringify({
            success: true,
            extractor_id: extractorId,
            message: `Custom extractor "${args.function_name}" added. Canvas will update automatically.`,
            outputs: args.outputs
          }, null, 2)
        }]
      };
    }
  },

  {
    definition: {
      name: "spec_validate",
      description: "Validate an AtomizerSpec and return detailed report.",
      inputSchema: {
        type: "object",
        properties: {
          study_name: { type: "string" }
        },
        required: ["study_name"]
      }
    },
    handler: async (args) => {
      const report = await specManager.validateAndReport(args.study_name);
      return { content: [{ type: "text", text: JSON.stringify(report, null, 2) }] };
    }
  },

  {
    definition: {
      name: "spec_create_from_description",
      description: "Create a new AtomizerSpec from a natural language description. Claude interprets the description and generates a complete spec.",
      inputSchema: {
        type: "object",
        properties: {
          study_name: { type: "string" },
          description: { type: "string", description: "Natural language description of the optimization" },
          model_path: { type: "string", description: "Path to the NX model" }
        },
        required: ["study_name", "description", "model_path"]
      }
    },
    handler: async (args) => {
      // This would invoke the StudyCreator with Claude interpretation
      // ...
    }
  }
];

---

6. Intelligent Assistant Integration

6.1 Claude Capabilities

With the unified spec, Claude can:

Capability	Example User Request	Claude Action
Modify Parameters	"Set mass weight to 3"	spec_modify on objectives[1].weight
Add Custom Logic	"Add a manufacturing cost function based on rib count"	spec_add_custom_extractor with Python code
Restructure Workflow	"Change from single-objective to multi-objective"	Modify optimization.algorithm.type and restructure objectives
Validate Setup	"Check if my config is valid"	spec_validate and explain issues
Explain Canvas	"What does this optimization do?"	Read spec, generate natural language summary

6.2 Example Interaction

User: "Claude, I want to add a manufacturability score that penalizes designs with more than 10 ribs"

Claude: I'll add a custom extractor for manufacturability scoring. Let me create the function:

[spec_add_custom_extractor]
function_name: "manufacturability_score"
code: """
def manufacturability_score(design_vars, fem_results):
    '''
    Calculate manufacturability score based on rib count.
    Higher score = easier to manufacture.
    '''
    rib_count = design_vars.get('n_lateral_ribs', 0)

    if rib_count <= 8:
        score = 100  # Easy
    elif rib_count <= 10:
        score = 80 - (rib_count - 8) * 10  # Moderate
    else:
        score = 60 - (rib_count - 10) * 15  # Difficult, penalized

    return max(0, score)
"""
outputs: ["mfg_score"]

Done! I've added a manufacturability extractor that:
- Returns 100 for 8 or fewer ribs (easy to manufacture)
- Returns 60-80 for 9-10 ribs (moderate)
- Returns <60 for 11+ ribs (penalized)

The canvas has been updated with a new "Manufacturability Score" node.

Would you like me to:
1. Add this as a soft constraint (mfg_score >= 60)?
2. Add it as an objective to maximize?

6.3 Real-time Canvas Updates

┌─────────────────────────────────────────────────────────────────┐
│                     Real-time Update Flow                        │
└─────────────────────────────────────────────────────────────────┘

User types: "Change mass weight to 5"
         │
         ▼
┌─────────────────┐
│ Claude receives │
│ message via WS  │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│ Claude calls    │
│ spec_modify     │
│ path: objectives│
│ [1].weight = 5  │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│ SpecManager     │
│ - Validates     │
│ - Saves spec    │
│ - Broadcasts    │
└────────┬────────┘
         │
         ├────────────────────────────────────┐
         │                                    │
         ▼                                    ▼
┌─────────────────┐                 ┌─────────────────┐
│ Canvas receives │                 │ Claude confirms │
│ spec_updated    │                 │ "Weight updated │
│ WebSocket msg   │                 │ to 5"           │
└────────┬────────┘                 └─────────────────┘
         │
         ▼
┌─────────────────┐
│ Canvas re-      │
│ renders node    │
│ with new weight │
│ "Mass (w=5)"    │
└─────────────────┘

---

7. API Design

7.1 REST Endpoints

# OpenAPI 3.0 specification (partial)

paths:
  /api/studies/{study_id}/spec:
    get:
      summary: Get AtomizerSpec
      responses:
        200:
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/AtomizerSpec'

    put:
      summary: Replace entire AtomizerSpec
      requestBody:
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/AtomizerSpec'

    patch:
      summary: Partial update to spec
      requestBody:
        content:
          application/json:
            schema:
              type: object
              properties:
                path:
                  type: string
                  description: JSONPath to field
                value:
                  description: New value
                modified_by:
                  type: string

  /api/studies/{study_id}/spec/nodes:
    post:
      summary: Add a new node
      requestBody:
        content:
          application/json:
            schema:
              type: object
              properties:
                type:
                  enum: [designVar, extractor, objective, constraint]
                data:
                  type: object

  /api/studies/{study_id}/spec/nodes/{node_id}:
    patch:
      summary: Update node properties
    delete:
      summary: Remove node

  /api/studies/{study_id}/spec/validate:
    post:
      summary: Validate spec and return report

  /api/studies/{study_id}/spec/custom-functions:
    post:
      summary: Add custom extractor function
      requestBody:
        content:
          application/json:
            schema:
              type: object
              required: [name, code, outputs]
              properties:
                name:
                  type: string
                code:
                  type: string
                outputs:
                  type: array
                  items:
                    type: string

7.2 WebSocket Protocol

// WebSocket Messages

// Client -> Server
interface ClientMessage {
  type: 'subscribe' | 'patch_node' | 'add_node' | 'remove_node' | 'update_position';
  study_id: string;
  // ... type-specific fields
}

// Server -> Client
interface ServerMessage {
  type: 'spec_updated' | 'validation_error' | 'node_added' | 'node_removed' | 'connection_ack';
  timestamp: string;
  // ... type-specific fields
}

// Example: spec_updated
{
  "type": "spec_updated",
  "timestamp": "2026-01-17T14:30:00Z",
  "modified_by": "claude",
  "hash": "sha256:abc123...",
  "changes": [
    { "path": "objectives[1].weight", "old": 1, "new": 5 }
  ]
}

---

8. Migration Strategy

8.1 Backward Compatibility

# optimization_engine/config/migrator.py

class SpecMigrator:
    """
    Migrate old optimization_config.json to AtomizerSpec v2.
    """

    MIGRATION_MAP = {
        # Old field -> New field
        "design_variables.*.parameter": "design_variables.*.expression_name",
        "design_variables.*.bounds": lambda dv: {"min": dv["bounds"][0], "max": dv["bounds"][1]},
        "objectives.*.goal": "objectives.*.direction",
        "optimization_settings.n_trials": "optimization.budget.max_trials",
        "optimization_settings.sampler": lambda s: {"type": s.replace("Sampler", "")},
    }

    def migrate(self, old_config: Dict) -> Dict:
        """Convert old config to AtomizerSpec v2."""
        spec = {
            "meta": {
                "version": "2.0",
                "created_by": "migration",
                "study_name": old_config.get("study_name", "migrated_study")
            },
            "design_variables": [],
            "extractors": [],
            "objectives": [],
            "constraints": [],
            "optimization": {},
            "canvas": {"edges": []}
        }

        # Migrate design variables
        for i, dv in enumerate(old_config.get("design_variables", [])):
            spec["design_variables"].append(self._migrate_dv(dv, i))

        # Migrate objectives
        for i, obj in enumerate(old_config.get("objectives", [])):
            extractor_id, extractor = self._infer_extractor(obj)
            if extractor not in [e["id"] for e in spec["extractors"]]:
                spec["extractors"].append(extractor)
            spec["objectives"].append(self._migrate_objective(obj, i, extractor_id))

        # ... continue for other sections

        return spec

    def _migrate_dv(self, dv: Dict, index: int) -> Dict:
        """Migrate a design variable."""
        bounds = dv.get("bounds", [dv.get("min", 0), dv.get("max", 1)])
        return {
            "id": f"dv_{index:03d}",
            "name": dv.get("name", dv.get("parameter", f"param_{index}")),
            "expression_name": dv.get("expression_name", dv.get("parameter")),
            "type": dv.get("type", "continuous"),
            "bounds": {"min": bounds[0], "max": bounds[1]},
            "baseline": dv.get("baseline"),
            "units": dv.get("units", dv.get("description", "")),
            "enabled": dv.get("enabled", True)
        }

8.2 Migration Script

# CLI tool for batch migration
python -m optimization_engine.config.migrate \
  --input studies/*/optimization_config.json \
  --output-suffix "_spec_v2.json" \
  --validate \
  --dry-run

---

9. Implementation Roadmap

Phase 1: Foundation (Weeks 1-3)

Task	Priority	Effort
Define AtomizerSpec JSON Schema v2.0	P0	2 days
Create SpecManager service (Python)	P0	3 days
Implement spec validation	P0	2 days
Add REST endpoints for spec CRUD	P0	3 days
Create migration script	P0	3 days
Update optimization engine to use spec	P1	4 days

Deliverable: Backend can read/write/validate AtomizerSpec

Phase 2: Frontend Integration (Weeks 4-6)

Task	Priority	Effort
Create SpecRenderer component	P0	4 days
Replace useCanvasStore with useSpecStore	P0	3 days
Implement spec ↔ canvas conversion	P0	3 days
Add WebSocket sync	P1	3 days
Update node panels for full spec fields	P1	4 days
Test bidirectional editing	P0	3 days

Deliverable: Canvas renders and edits AtomizerSpec directly

Phase 3: Claude Integration (Weeks 7-9)

Task	Priority	Effort
Implement spec_* MCP tools	P0	4 days
Add custom function support	P1	4 days
Test Claude → Canvas updates	P0	3 days
Add natural language study creation	P2	5 days
Create Claude assistant prompts	P1	2 days

Deliverable: Claude can read, modify, and extend AtomizerSpec

Phase 4: Polish & Testing (Weeks 10-12)

Task	Priority	Effort
Migrate all existing studies	P0	3 days
Integration testing	P0	5 days
Documentation	P1	3 days
Performance optimization	P2	3 days
User acceptance testing	P0	4 days

Deliverable: Production-ready unified configuration system

---

Appendix A: Comparison Table

Feature	Current	Proposed
Config formats	4+ variants	1 (AtomizerSpec)
Canvas ↔ Config sync	Lossy, manual	Lossless, automatic
Claude modifications	Returns JSON for manual copy	Direct spec modification
Custom extractors	Edit Python code	Declare in spec
Real-time updates	None	WebSocket-driven
Validation	Partial Python	Full JSON Schema + semantic
Canvas layout	Lost on reload	Persisted in spec

---

Appendix B: Glossary

Term	Definition
AtomizerSpec	The unified JSON configuration format (v2.0)
SpecManager	Backend service for all spec operations
SpecRenderer	Frontend component that renders spec as canvas
Custom Extractor	User-defined Python function declared in spec
Spec Patch	Partial update to a single field in the spec

---

Appendix C: File Locations (Proposed)

atomizer-dashboard/
├── backend/api/
│   ├── services/
│   │   └── spec_manager.py        # NEW: Core spec management
│   ├── routes/
│   │   └── spec.py                # NEW: REST endpoints for spec
│   └── schemas/
│       └── atomizer_spec_v2.json  # NEW: JSON Schema
│
├── frontend/src/
│   ├── hooks/
│   │   └── useSpecStore.ts        # NEW: Spec state management
│   ├── components/canvas/
│   │   └── SpecRenderer.tsx       # NEW: Spec-based canvas
│   └── lib/spec/
│       ├── types.ts               # NEW: TypeScript types
│       └── converter.ts           # NEW: Spec ↔ ReactFlow
│
mcp-server/atomizer-tools/
└── src/tools/
    └── spec_tools.ts              # NEW: MCP tools for spec

optimization_engine/
├── config/
│   ├── migrator.py                # NEW: v1 → v2 migration
│   └── spec_loader.py             # NEW: AtomizerSpec loader
└── schemas/
    └── atomizer_spec_v2.json      # Canonical schema

---

This document is the master plan for implementing Atomizer's Unified Configuration Architecture. All implementation work should reference this document.