Atomizer/docs/07_DEVELOPMENT/DASHBOARD_IMPROVEMENT_PLAN.md

# Atomizer Dashboard Improvement Plan

## Executive Summary

This document outlines a comprehensive plan to enhance the Atomizer dashboard into a self-contained, professional optimization platform with integrated AI assistance through Claude Code.

---

## Current State

### Existing Pages
- **Home** (`/`): Study selection with README preview
- **Dashboard** (`/dashboard`): Real-time monitoring, charts, control panel
- **Results** (`/results`): AI-generated report viewer

### Existing Features
- Study selection with persistence
- README display on study hover
- Convergence plot (Plotly)
- Pareto plot for multi-objective
- Parallel coordinates
- Parameter importance chart
- Console output viewer
- Control panel (start/stop/validate)
- Optuna dashboard launch
- AI report generation

---

## Proposed Improvements

### Phase 1: Core UX Enhancements

#### 1.1 Unified Navigation & Branding
- **Logo & Brand Identity**: Professional Atomizer logo in sidebar
- **Breadcrumb Navigation**: Show current path (e.g., `Atomizer > m1_mirror > Dashboard`)
- **Quick Study Switcher**: Dropdown in header to switch studies without returning to Home
- **Keyboard Shortcuts**: `Ctrl+K` for command palette, `Ctrl+1/2/3` for page navigation

#### 1.2 Study Overview Card (Home Page Enhancement)
When a study is selected, show a summary card with:
- Trial progress ring/chart
- Best objective value with trend indicator
- Last activity timestamp
- Quick action buttons (Start, Validate, Open)
- Thumbnail preview of convergence

#### 1.3 Real-Time Status Indicators
- **Global Status Bar**: Shows running processes, current trial, ETA
- **Live Toast Notifications**: Trial completed, error occurred, validation done
- **Sound Notifications** (optional): Audio cue on trial completion

#### 1.4 Dark/Light Theme Toggle
- Persist theme preference in localStorage
- System theme detection

---

### Phase 2: Advanced Visualization

#### 2.1 Interactive Trial Table
- Sortable/filterable data grid with all trial data
- Column visibility toggles
- Export to CSV/Excel
- Click row to highlight in plots
- Filter by FEA vs Neural trials

#### 2.2 Enhanced Charts
- **Zoomable Convergence**: Brushing to select time ranges
- **3D Parameter Space**: Three.js visualization of design space
- **Heatmap**: Parameter correlation matrix
- **Animation**: Play through optimization history

#### 2.3 Comparison Mode
- Side-by-side comparison of 2-3 trials
- Diff view for parameter values
- Overlay plots

#### 2.4 Design Space Explorer
- Interactive sliders for design variables
- Predict objective using neural surrogate
- "What-if" analysis without running FEA

---

### Phase 3: Claude Code Integration (AI Chat)

#### 3.1 Architecture Overview

```
┌─────────────────────────────────────────────────────────────┐
│                    Atomizer Dashboard                        │
├─────────────────────────────────────────────────────────────┤
│                                                              │
│  ┌─────────────────────────┐  ┌──────────────────────────┐  │
│  │                         │  │                          │  │
│  │    Main Dashboard       │  │   Claude Code Panel      │  │
│  │    (Charts, Controls)   │  │   (Chat Interface)       │  │
│  │                         │  │                          │  │
│  │                         │  │  ┌────────────────────┐  │  │
│  │                         │  │  │ Conversation       │  │  │
│  │                         │  │  │ History            │  │  │
│  │                         │  │  │                    │  │  │
│  │                         │  │  └────────────────────┘  │  │
│  │                         │  │                          │  │
│  │                         │  │  ┌────────────────────┐  │  │
│  │                         │  │  │ Input Box          │  │  │
│  │                         │  │  └────────────────────┘  │  │
│  │                         │  │                          │  │
│  └─────────────────────────┘  └──────────────────────────┘  │
│                                                              │
└─────────────────────────────────────────────────────────────┘
                              │
                              ▼
                    ┌─────────────────┐
                    │   Backend API    │
                    │   /api/claude    │
                    └────────┬────────┘
                             │
                             ▼
                    ┌─────────────────┐
                    │  Claude Agent   │
                    │  SDK Backend    │
                    │  (Python)       │
                    └────────┬────────┘
                             │
                    ┌────────┴────────┐
                    │                 │
               ┌────▼────┐      ┌─────▼─────┐
               │ Atomizer│      │ Anthropic │
               │ Tools   │      │ Claude API│
               └─────────┘      └───────────┘
```

#### 3.2 Backend Implementation

**New API Endpoints:**

```python
# atomizer-dashboard/backend/api/routes/claude.py

@router.post("/chat")
async def chat_with_claude(request: ChatRequest):
    """
    Send a message to Claude with study context

    Request:
        - message: User's message
        - study_id: Current study context
        - conversation_id: For multi-turn conversations

    Returns:
        - response: Claude's response
        - actions: Any tool calls made (file edits, commands)
    """

@router.websocket("/chat/stream")
async def chat_stream(websocket: WebSocket):
    """
    WebSocket for streaming Claude responses
    Real-time token streaming for better UX
    """

@router.get("/conversations")
async def list_conversations():
    """Get conversation history for current study"""

@router.delete("/conversations/{conversation_id}")
async def delete_conversation(conversation_id: str):
    """Delete a conversation"""
```

**Claude Agent SDK Integration:**

```python
# atomizer-dashboard/backend/services/claude_agent.py

from anthropic import Anthropic
import json

class AtomizerClaudeAgent:
    def __init__(self, study_id: str = None):
        self.client = Anthropic()
        self.study_id = study_id
        self.tools = self._load_atomizer_tools()
        self.system_prompt = self._build_system_prompt()

    def _build_system_prompt(self) -> str:
        """Build context-aware system prompt"""
        prompt = """You are Claude Code embedded in the Atomizer optimization dashboard.

You have access to the current optimization study and can help users:
1. Analyze optimization results
2. Modify study configurations
3. Create new studies
4. Explain FEA/Zernike concepts
5. Suggest design improvements

Current Study Context:
{study_context}

Available Tools:
- read_study_config: Read optimization configuration
- modify_config: Update design variables, objectives
- query_trials: Get trial data from database
- create_study: Create new optimization study
- run_analysis: Perform custom analysis
- edit_file: Modify study files
"""
        if self.study_id:
            prompt = prompt.format(study_context=self._get_study_context())
        else:
            prompt = prompt.format(study_context="No study selected")
        return prompt

    def _load_atomizer_tools(self) -> list:
        """Define Atomizer-specific tools for Claude"""
        return [
            {
                "name": "read_study_config",
                "description": "Read the optimization configuration for the current study",
                "input_schema": {
                    "type": "object",
                    "properties": {},
                    "required": []
                }
            },
            {
                "name": "query_trials",
                "description": "Query trial data from the Optuna database",
                "input_schema": {
                    "type": "object",
                    "properties": {
                        "filter": {
                            "type": "string",
                            "description": "SQL-like filter (e.g., 'state=COMPLETE')"
                        },
                        "limit": {
                            "type": "integer",
                            "description": "Max results to return"
                        }
                    }
                }
            },
            {
                "name": "modify_config",
                "description": "Modify the optimization configuration",
                "input_schema": {
                    "type": "object",
                    "properties": {
                        "path": {
                            "type": "string",
                            "description": "JSON path to modify (e.g., 'design_variables[0].max')"
                        },
                        "value": {
                            "type": "any",
                            "description": "New value to set"
                        }
                    },
                    "required": ["path", "value"]
                }
            },
            {
                "name": "create_study",
                "description": "Create a new optimization study",
                "input_schema": {
                    "type": "object",
                    "properties": {
                        "name": {"type": "string"},
                        "description": {"type": "string"},
                        "model_path": {"type": "string"},
                        "design_variables": {"type": "array"},
                        "objectives": {"type": "array"}
                    },
                    "required": ["name"]
                }
            }
        ]

    async def chat(self, message: str, conversation_history: list = None) -> dict:
        """Process a chat message with tool use"""
        messages = conversation_history or []
        messages.append({"role": "user", "content": message})

        response = await self.client.messages.create(
            model="claude-sonnet-4-20250514",
            max_tokens=4096,
            system=self.system_prompt,
            tools=self.tools,
            messages=messages
        )

        # Handle tool calls
        if response.stop_reason == "tool_use":
            tool_results = await self._execute_tools(response.content)
            messages.append({"role": "assistant", "content": response.content})
            messages.append({"role": "user", "content": tool_results})
            return await self.chat("", messages)  # Continue conversation

        return {
            "response": response.content[0].text,
            "conversation": messages + [{"role": "assistant", "content": response.content}]
        }
```

#### 3.3 Frontend Implementation

**Chat Panel Component:**

```tsx
// atomizer-dashboard/frontend/src/components/ClaudeChat.tsx

import React, { useState, useRef, useEffect } from 'react';
import { Send, Bot, User, Sparkles, Loader2 } from 'lucide-react';
import ReactMarkdown from 'react-markdown';
import { useStudy } from '../context/StudyContext';

interface Message {
  role: 'user' | 'assistant';
  content: string;
  timestamp: Date;
  toolCalls?: any[];
}

export const ClaudeChat: React.FC = () => {
  const { selectedStudy } = useStudy();
  const [messages, setMessages] = useState<Message[]>([]);
  const [input, setInput] = useState('');
  const [isLoading, setIsLoading] = useState(false);
  const messagesEndRef = useRef<HTMLDivElement>(null);

  const sendMessage = async () => {
    if (!input.trim() || isLoading) return;

    const userMessage: Message = {
      role: 'user',
      content: input,
      timestamp: new Date()
    };

    setMessages(prev => [...prev, userMessage]);
    setInput('');
    setIsLoading(true);

    try {
      const response = await fetch('/api/claude/chat', {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({
          message: input,
          study_id: selectedStudy?.id,
          conversation_history: messages
        })
      });

      const data = await response.json();

      setMessages(prev => [...prev, {
        role: 'assistant',
        content: data.response,
        timestamp: new Date(),
        toolCalls: data.tool_calls
      }]);
    } catch (error) {
      // Handle error
    } finally {
      setIsLoading(false);
    }
  };

  // Suggested prompts for new conversations
  const suggestions = [
    "Analyze my optimization results",
    "What parameters have the most impact?",
    "Create a new study for my bracket",
    "Explain the Zernike coefficients"
  ];

  return (
    <div className="flex flex-col h-full bg-dark-800 rounded-xl border border-dark-600">
      {/* Header */}
      <div className="px-4 py-3 border-b border-dark-600 flex items-center gap-2">
        <Bot className="w-5 h-5 text-primary-400" />
        <span className="font-medium text-white">Claude Code</span>
        {selectedStudy && (
          <span className="text-xs bg-dark-700 px-2 py-0.5 rounded text-dark-300">
            {selectedStudy.id}
          </span>
        )}
      </div>

      {/* Messages */}
      <div className="flex-1 overflow-y-auto p-4 space-y-4">
        {messages.length === 0 ? (
          <div className="text-center py-8">
            <Sparkles className="w-12 h-12 mx-auto mb-4 text-primary-400 opacity-50" />
            <p className="text-dark-300 mb-4">Ask me anything about your optimization</p>
            <div className="flex flex-wrap gap-2 justify-center">
              {suggestions.map((s, i) => (
                <button
                  key={i}
                  onClick={() => setInput(s)}
                  className="px-3 py-1.5 bg-dark-700 hover:bg-dark-600 rounded-lg
                           text-sm text-dark-300 hover:text-white transition-colors"
                >
                  {s}
                </button>
              ))}
            </div>
          </div>
        ) : (
          messages.map((msg, i) => (
            <div key={i} className={`flex gap-3 ${msg.role === 'user' ? 'justify-end' : ''}`}>
              {msg.role === 'assistant' && (
                <div className="w-8 h-8 rounded-lg bg-primary-600 flex items-center justify-center flex-shrink-0">
                  <Bot className="w-4 h-4 text-white" />
                </div>
              )}
              <div className={`max-w-[80%] rounded-lg p-3 ${
                msg.role === 'user'
                  ? 'bg-primary-600 text-white'
                  : 'bg-dark-700 text-dark-200'
              }`}>
                <ReactMarkdown className="prose prose-sm prose-invert">
                  {msg.content}
                </ReactMarkdown>
              </div>
              {msg.role === 'user' && (
                <div className="w-8 h-8 rounded-lg bg-dark-600 flex items-center justify-center flex-shrink-0">
                  <User className="w-4 h-4 text-dark-300" />
                </div>
              )}
            </div>
          ))
        )}
        {isLoading && (
          <div className="flex gap-3">
            <div className="w-8 h-8 rounded-lg bg-primary-600 flex items-center justify-center">
              <Loader2 className="w-4 h-4 text-white animate-spin" />
            </div>
            <div className="bg-dark-700 rounded-lg p-3 text-dark-400">
              Thinking...
            </div>
          </div>
        )}
        <div ref={messagesEndRef} />
      </div>

      {/* Input */}
      <div className="p-4 border-t border-dark-600">
        <div className="flex gap-2">
          <input
            type="text"
            value={input}
            onChange={(e) => setInput(e.target.value)}
            onKeyPress={(e) => e.key === 'Enter' && sendMessage()}
            placeholder="Ask about your optimization..."
            className="flex-1 px-4 py-2 bg-dark-700 border border-dark-600 rounded-lg
                     text-white placeholder-dark-400 focus:outline-none focus:border-primary-500"
          />
          <button
            onClick={sendMessage}
            disabled={!input.trim() || isLoading}
            className="px-4 py-2 bg-primary-600 hover:bg-primary-500 disabled:opacity-50
                     text-white rounded-lg transition-colors"
          >
            <Send className="w-4 h-4" />
          </button>
        </div>
      </div>
    </div>
  );
};
```

#### 3.4 Claude Code Capabilities

When integrated, Claude Code will be able to:

| Capability | Description | Example Command |
|------------|-------------|-----------------|
| **Analyze Results** | Interpret optimization progress | "Why is my convergence plateauing?" |
| **Explain Physics** | Describe FEA/Zernike concepts | "Explain astigmatism in my mirror" |
| **Modify Config** | Update design variables | "Increase the max bounds for whiffle_min to 60" |
| **Create Studies** | Generate new study from description | "Create a study for my new bracket" |
| **Query Data** | SQL-like data exploration | "Show me the top 5 trials by stress" |
| **Generate Code** | Write custom analysis scripts | "Write a Python script to compare trials" |
| **Debug Issues** | Diagnose optimization problems | "Why did trial 42 fail?" |

---

### Phase 4: Study Creation Wizard

#### 4.1 Guided Study Setup

Multi-step wizard for creating new studies:

1. **Model Selection**
   - Browse NX model files
   - Auto-detect expressions
   - Preview 3D geometry (if possible)

2. **Design Variables**
   - Interactive table to set bounds
   - Baseline detection from model
   - Sensitivity hints from similar studies

3. **Objectives**
   - Template selection (stress, displacement, frequency, Zernike)
   - Direction (minimize/maximize)
   - Target values and weights

4. **Constraints**
   - Add geometric/physical constraints
   - Feasibility preview

5. **Algorithm Settings**
   - Protocol selection (10/11/12)
   - Sampler configuration
   - Neural surrogate options

6. **Review & Create**
   - Summary of all settings
   - Validation checks
   - One-click creation

---

### Phase 5: Self-Contained Packaging

#### 5.1 Electron Desktop App

Package the dashboard as a standalone desktop application:

```
Atomizer.exe
├── Frontend (React bundled)
├── Backend (Python bundled with PyInstaller)
├── NX Integration (optional)
└── Claude API (requires key)
```

Benefits:
- No Node.js/Python installation needed
- Single installer for users
- Offline capability (except AI features)
- Native file dialogs
- System tray integration

#### 5.2 Docker Deployment

```yaml
# docker-compose.yml
version: '3.8'
services:
  frontend:
    build: ./atomizer-dashboard/frontend
    ports:
      - "3000:3000"

  backend:
    build: ./atomizer-dashboard/backend
    ports:
      - "8000:8000"
    volumes:
      - ./studies:/app/studies
    environment:
      - ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}
```

---

## Implementation Priority

| Phase | Feature | Effort | Impact | Priority |
|-------|---------|--------|--------|----------|
| 1.1 | Unified Navigation | Medium | High | P1 |
| 1.2 | Study Overview Card | Low | High | P1 |
| 1.3 | Real-Time Status | Medium | High | P1 |
| 2.1 | Interactive Trial Table | Medium | High | P1 |
| 3.1 | Claude Chat Backend | High | Critical | P1 |
| 3.3 | Claude Chat Frontend | Medium | Critical | P1 |
| 2.2 | Enhanced Charts | Medium | Medium | P2 |
| 2.4 | Design Space Explorer | High | High | P2 |
| 4.1 | Study Creation Wizard | High | High | P2 |
| 5.1 | Electron Packaging | High | Medium | P3 |

---

## Technical Requirements

### Dependencies to Add

**Backend:**
```
anthropic>=0.18.0  # Claude API
websockets>=12.0   # Real-time chat
```

**Frontend:**
```
@radix-ui/react-dialog  # Modals
@radix-ui/react-tabs    # Tab navigation
cmdk                    # Command palette
framer-motion           # Animations
```

### API Keys Required

- `ANTHROPIC_API_KEY`: For Claude Code integration (user provides)

---

## Security Considerations

1. **API Key Storage**: Never store API keys in frontend; use backend proxy
2. **File Access**: Sandbox Claude's file operations to study directories only
3. **Command Execution**: Whitelist allowed commands (no arbitrary shell)
4. **Rate Limiting**: Prevent API abuse through the chat interface

---

## Next Steps

1. Review and approve this plan
2. Prioritize features based on user needs
3. Create GitHub issues for each feature
4. Begin Phase 1 implementation
5. Set up Claude API integration testing

---

*Document Version: 1.0*
*Created: 2024-12-04*
*Author: Claude Code*