Atomizer/docs/04_USER_GUIDES/DASHBOARD.md

# Atomizer Dashboard

**Last Updated**: December 5, 2025

---

## Overview

The Atomizer Dashboard is a real-time web-based interface for monitoring and analyzing multi-objective optimization studies. Built with React, TypeScript, and Tailwind CSS, it provides comprehensive visualization and interaction capabilities for NSGA-II based structural optimization.

---

## Architecture

### Frontend Stack
- **Framework**: React 18 with TypeScript
- **Build Tool**: Vite
- **Styling**: Tailwind CSS with custom dark/light theme support
- **Charts**: Recharts for data visualization
- **State Management**: React hooks (useState, useEffect)
- **WebSocket**: Real-time optimization updates

### Backend Stack
- **Framework**: FastAPI (Python)
- **Database**: Optuna SQLite studies
- **API**: RESTful endpoints with WebSocket support
- **CORS**: Configured for local development

### Ports
- **Frontend**: `http://localhost:3003` (Vite dev server)
- **Backend**: `http://localhost:8000` (FastAPI/Uvicorn)

---

## Key Features

### 1. Multi-Objective Visualization

#### Pareto Front Plot
- 2D scatter plot showing trade-offs between objectives
- Color-coded by constraint satisfaction (green = feasible, red = infeasible)
- Interactive hover tooltips with trial details
- Automatically extracts Pareto-optimal solutions using NSGA-II

#### Parallel Coordinates Plot
**Research-Based Multi-Dimensional Visualization**

Structure: **Design Variables → Objectives → Constraints**

Features:
- **Light Theme**: White background with high-visibility dark text and colors
- **Color-Coded Axes**:
  - Blue background: Design variables
  - Green background: Objectives
  - Yellow background: Constraints
- **Interactive Selection**:
  - Hover over lines to highlight individual trials
  - Click to select/deselect trials
  - Multi-select with visual feedback (orange highlight)
- **Type Badges**: Labels showing DESIGN VAR, OBJECTIVE, or CONSTRAINT
- **Units Display**: Automatic unit labeling (mm, MPa, Hz, g, etc.)
- **Min/Max Labels**: Range values displayed on each axis
- **Feasibility Coloring**:
  - Green: Feasible solutions
  - Red: Infeasible solutions (constraint violations)
  - Blue: Hover highlight
  - Orange: Selected trials

**Implementation**: [ParallelCoordinatesPlot.tsx](atomizer-dashboard/frontend/src/components/ParallelCoordinatesPlot.tsx:1)

**Line colors**:
```typescript
if (isSelected) return '#FF6B00';      // Orange for selected
if (!trial.feasible) return '#DC2626'; // Red for infeasible
if (isHovered) return '#2563EB';       // Blue for hover
return '#10B981';                       // Green for feasible
```

### 2. Optimizer Strategy Panel
Displays algorithm information:
- **Algorithm**: NSGA-II, TPE, or custom
- **Type**: Single-objective or Multi-objective
- **Objectives Count**: Number of optimization objectives
- **Design Variables Count**: Number of design parameters

### 3. Convergence Plot (Enhanced)
**File**: `atomizer-dashboard/frontend/src/components/ConvergencePlot.tsx`

Advanced convergence visualization:
- **Dual-line plot**: Individual trial values + running best trajectory
- **Area fill**: Gradient under trial values curve
- **Statistics panel**: Best value, improvement %, 90% convergence trial
- **Summary footer**: First value, mean, std dev, total trials
- **Step-after interpolation**: Running best shown as step function

### 4. Parameter Importance Chart
**File**: `atomizer-dashboard/frontend/src/components/ParameterImportanceChart.tsx`

Correlation-based parameter analysis:
- **Pearson correlation**: Calculates correlation between each parameter and objective
- **Horizontal bar chart**: Parameters ranked by absolute importance
- **Color coding**: Green (negative correlation - helps minimize), Red (positive - hurts minimize)
- **Tooltip**: Shows percentage importance and raw correlation coefficient (r)
- **Minimum 3 trials**: Required for statistical significance

### 5. Study Report Viewer
**File**: `atomizer-dashboard/frontend/src/components/StudyReportViewer.tsx`

Full-featured markdown report viewer:
- **Modal overlay**: Full-screen report viewing
- **Math equations**: KaTeX support for LaTeX math (`$...$` inline, `$$...$$` block)
- **GitHub-flavored markdown**: Tables, code blocks, task lists
- **Custom styling**: Dark theme with proper typography
- **Syntax highlighting**: Code blocks with language detection
- **Refresh button**: Re-fetch report for live updates
- **External link**: Open in system editor

### 6. Trial History Table
- Comprehensive list of all trials
- Sortable columns
- Status indicators (COMPLETE, PRUNED, FAIL)
- Parameter values and objective values
- User attributes (constraints)

### 7. Pruned Trials Tracking
- **Real-time count**: Fetched directly from Optuna database
- **Pruning diagnostics**: Tracks pruned trial params and causes
- **Database query**: Uses SQLite `state = 'PRUNED'` filter

### 8. Analytics Page (Cross-Study Comparisons)
**File**: `atomizer-dashboard/frontend/src/pages/Analytics.tsx`

Dedicated analytics page for comparing optimization studies:

#### Aggregate Statistics
- **Total Studies**: Count of all studies in the system
- **Running/Paused/Completed**: Status distribution breakdown
- **Total Trials**: Sum of trials across all studies
- **Avg Trials/Study**: Average trial count per study
- **Best Overall**: Best objective value across all studies with study ID

#### Study Comparison Table
- **Sortable columns**: Name, Status, Progress, Best Value
- **Status indicators**: Color-coded badges (running=green, paused=orange, completed=blue)
- **Progress bars**: Visual completion percentage with color coding
- **Quick actions**: Open button to navigate directly to a study's dashboard
- **Selected highlight**: Current study highlighted with "Selected" badge
- **Click-to-expand**: Row expansion for additional details

#### Status Distribution Chart
- Visual breakdown of studies by status
- Horizontal bar chart with percentage fill
- Color-coded: Running (green), Paused (orange), Completed (blue), Not Started (gray)

#### Top Performers Panel
- Ranking of top 5 studies by best objective value (assumes minimization)
- Medal-style numbering (gold, silver, bronze for top 3)
- Clickable rows to navigate to study
- Trial count display

**Usage**: Navigate to `/analytics` when a study is selected. Provides aggregate view across all studies.

### 9. Global Claude Terminal
**Files**:
- `atomizer-dashboard/frontend/src/components/GlobalClaudeTerminal.tsx`
- `atomizer-dashboard/frontend/src/components/ClaudeTerminal.tsx`
- `atomizer-dashboard/frontend/src/context/ClaudeTerminalContext.tsx`

Persistent AI assistant terminal:
- **Global persistence**: Terminal persists across page navigation
- **WebSocket connection**: Real-time communication with Claude Code backend
- **Context awareness**: Automatically includes current study context when available
- **New Study mode**: When no study selected, offers guided study creation wizard
- **Visual indicators**: Connection status shown in sidebar footer
- **Keyboard shortcut**: Open/close terminal from anywhere

**Modes**:
- **With Study Selected**: "Set Context" button loads study-specific context
- **No Study Selected**: "New Study" button starts guided wizard from `.claude/skills/guided-study-wizard.md`

### 10. Shared Markdown Renderer
**File**: `atomizer-dashboard/frontend/src/components/MarkdownRenderer.tsx`

Reusable markdown rendering component:
- **Syntax highlighting**: Prism-based code highlighting with `oneDark` theme
- **GitHub-flavored markdown**: Tables, task lists, strikethrough
- **LaTeX math support**: KaTeX rendering with `remark-math` and `rehype-katex`
- **Custom styling**: Dark theme typography optimized for dashboard
- **Used by**: Home page (README display), Results page (reports)

---

## Pages Structure

### Home Page (`/`)
- Study navigator and selector
- README.md display with full markdown rendering
- New study creation via Claude terminal

### Dashboard Page (`/dashboard`)
- Real-time live tracker for selected study
- Convergence plot, Pareto front, parameter importance
- Trial history table

### Reports Page (`/results`)
- AI-generated optimization report viewer
- Full markdown rendering with syntax highlighting and math
- Copy and download capabilities

### Analytics Page (`/analytics`)
- Cross-study comparison and aggregate statistics
- Study ranking and status distribution
- Quick navigation to individual studies

---

## API Endpoints

### Studies

#### GET `/api/optimization/studies`
List all available optimization studies.

**Response**:
```json
[
  {
    "id": "drone_gimbal_arm_optimization",
    "name": "drone_gimbal_arm_optimization",
    "direction": ["minimize", "maximize"],
    "n_trials": 100,
    "best_value": [3245.67, 165.3],
    "sampler": "NSGAIISampler"
  }
]
```

#### GET `/api/optimization/studies/{study_id}/trials`
Get all trials for a study.

**Response**:
```json
{
  "trials": [
    {
      "number": 0,
      "values": [3456.2, 145.6],
      "params": {
        "beam_half_core_thickness": 7.5,
        "beam_face_thickness": 2.1,
        "holes_diameter": 30.0,
        "hole_count": 11
      },
      "state": "COMPLETE",
      "user_attrs": {
        "max_stress": 95.3,
        "max_displacement": 1.2,
        "frequency": 145.6,
        "mass": 3456.2,
        "constraint_satisfied": true
      }
    }
  ]
}
```

#### GET `/api/optimization/studies/{study_id}/metadata`
Get study metadata including objectives and design variables.

**Response**:
```json
{
  "objectives": [
    {
      "name": "mass",
      "type": "minimize",
      "unit": "g"
    },
    {
      "name": "frequency",
      "type": "maximize",
      "unit": "Hz"
    }
  ],
  "design_variables": [
    {
      "name": "beam_half_core_thickness",
      "unit": "mm",
      "min": 5.0,
      "max": 10.0
    }
  ],
  "sampler": "NSGAIISampler"
}
```

#### GET `/api/optimization/studies/{study_id}/pareto-front`
Get Pareto-optimal solutions for multi-objective studies.

**Response**:
```json
{
  "is_multi_objective": true,
  "pareto_front": [
    {
      "trial_number": 0,
      "values": [3245.67, 165.3],
      "params": {...},
      "user_attrs": {...},
      "constraint_satisfied": true
    }
  ]
}
```

### WebSocket

#### WS `/ws/optimization/{study_id}`
Real-time trial updates during optimization.

**Message Format**:
```json
{
  "type": "trial_complete",
  "trial": {
    "number": 5,
    "values": [3456.2, 145.6],
    "params": {...}
  }
}
```

---

## Running the Dashboard

### Backend

```bash
cd atomizer-dashboard/backend
python -m uvicorn api.main:app --reload --port 8000
```

### Frontend

```bash
cd atomizer-dashboard/frontend
npm run dev
```

Access at: `http://localhost:3003`

---

## Configuration

### Vite Proxy ([vite.config.ts](atomizer-dashboard/frontend/vite.config.ts:1))

```typescript
export default defineConfig({
  plugins: [react()],
  server: {
    host: '0.0.0.0',
    port: 3003,
    proxy: {
      '/api': {
        target: 'http://127.0.0.1:8000',
        changeOrigin: true,
        secure: false,
        ws: true,  // WebSocket support
      }
    }
  }
})
```

### CORS ([backend/api/main.py](atomizer-dashboard/backend/api/main.py:1))

```python
app.add_middleware(
    CORSMiddleware,
    allow_origins=["http://localhost:3003"],
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)
```

---

## Component Structure

```
atomizer-dashboard/
├── frontend/
│   ├── src/
│   │   ├── components/
│   │   │   ├── ParallelCoordinatesPlot.tsx    # Multi-objective visualization
│   │   │   ├── ParetoPlot.tsx                  # Pareto front scatter plot
│   │   │   ├── OptimizerPanel.tsx              # Strategy information
│   │   │   ├── ConvergencePlot.tsx             # Enhanced convergence chart
│   │   │   ├── ParameterImportanceChart.tsx    # Correlation-based importance
│   │   │   ├── StudyReportViewer.tsx           # Markdown report viewer
│   │   │   ├── MarkdownRenderer.tsx            # Shared markdown renderer
│   │   │   ├── ClaudeTerminal.tsx              # Claude AI terminal component
│   │   │   ├── GlobalClaudeTerminal.tsx        # Global terminal wrapper
│   │   │   ├── common/
│   │   │   │   ├── Card.tsx                    # Reusable card component
│   │   │   │   └── Button.tsx                  # Reusable button component
│   │   │   ├── layout/
│   │   │   │   ├── Sidebar.tsx                 # Navigation sidebar
│   │   │   │   └── MainLayout.tsx              # Page layout wrapper
│   │   │   └── dashboard/
│   │   │       ├── MetricCard.tsx              # KPI display
│   │   │       └── StudyCard.tsx               # Study selector
│   │   ├── pages/
│   │   │   ├── Home.tsx                        # Study selection & README
│   │   │   ├── Dashboard.tsx                   # Live optimization tracker
│   │   │   ├── Results.tsx                     # Report viewer
│   │   │   └── Analytics.tsx                   # Cross-study analytics
│   │   ├── context/
│   │   │   ├── StudyContext.tsx                # Global study state
│   │   │   └── ClaudeTerminalContext.tsx       # Terminal state
│   │   ├── hooks/
│   │   │   └── useWebSocket.ts                 # WebSocket connection
│   │   ├── api/
│   │   │   └── client.ts                       # API client
│   │   └── types/
│   │       └── index.ts                        # TypeScript types
│   └── vite.config.ts
└── backend/
    └── api/
        ├── main.py                              # FastAPI app
        └── routes/
            ├── optimization.py                  # Optimization endpoints
            └── terminal.py                      # Claude terminal WebSocket
```

## NPM Dependencies

The frontend uses these key packages:
- `react-markdown` - Markdown rendering
- `remark-gfm` - GitHub-flavored markdown support
- `remark-math` - Math equation parsing
- `rehype-katex` - KaTeX math rendering
- `recharts` - Interactive charts

---

## Data Flow

1. **Optimization Engine** runs trials and stores results in Optuna SQLite database
2. **Backend API** reads from database and exposes REST endpoints
3. **Frontend** fetches data via `/api/optimization/*` endpoints
4. **WebSocket** pushes real-time updates to connected clients
5. **React Components** render visualizations based on fetched data

---

## Troubleshooting

### Dashboard Page Crashes

**Issue**: `TypeError: Cannot read properties of undefined (reading 'split')`

**Fix**: Ensure all data is validated before rendering. ParallelCoordinatesPlot now includes:
```typescript
if (!paretoData || paretoData.length === 0) return <EmptyState />;
if (!objectives || !designVariables) return <EmptyState />;
```

### No Data Showing

1. Check backend is running: `curl http://localhost:8000/api/optimization/studies`
2. Verify study exists in Optuna database
3. Check browser console for API errors
4. Ensure WebSocket connection is established

### CORS Errors

- Backend must allow origin `http://localhost:3003`
- Frontend proxy must target `http://127.0.0.1:8000` (not `localhost`)

---

## Best Practices

### For Multi-Objective Studies

1. **Always use metadata endpoint** to get objective/variable definitions
2. **Extract constraints from user_attrs** for parallel coordinates
3. **Filter Pareto front** using `paretoData.pareto_front` array
4. **Validate constraint_satisfied** field before coloring

### For Real-Time Updates

1. **Use WebSocket** for live trial updates
2. **Debounce state updates** to avoid excessive re-renders
3. **Close WebSocket** connection on component unmount

### For Performance

1. **Limit displayed trials** for large studies (e.g., show last 1000)
2. **Use React.memo** for expensive components
3. **Virtualize large lists** if showing >100 trials in tables

---

## Recent Updates (December 2025)

### Completed
- [x] **Convergence Plot**: Enhanced with running best, statistics, and gradient fill
- [x] **Parameter Importance Chart**: Correlation analysis with color-coded bars
- [x] **Study Report Viewer**: Full markdown rendering with KaTeX math support
- [x] **Pruned Trials**: Real-time count from Optuna database (not JSON file)
- [x] **Chart Data Transformation**: Fixed `values` array mapping for single/multi-objective
- [x] **Analytics Page**: Dedicated cross-study comparison and aggregate statistics view
- [x] **Global Claude Terminal**: Persistent AI terminal with study context awareness
- [x] **Shared Markdown Renderer**: Reusable component with syntax highlighting and math support
- [x] **Study Session Persistence**: localStorage-based study selection that survives page refresh
- [x] **Paused Status Support**: Full support for paused optimization status throughout UI
- [x] **Guided Study Wizard**: Interactive wizard skill for creating new studies via Claude

### Future Enhancements

- [ ] 3D Pareto front visualization for 3+ objectives
- [ ] Advanced filtering and search in trial history
- [ ] Export results to CSV/JSON
- [ ] Custom parallel coordinates brushing/filtering
- [ ] Hypervolume indicator tracking
- [ ] Interactive design variable sliders
- [ ] Constraint importance analysis

---

## References

- **Optuna Documentation**: https://optuna.readthedocs.io/
- **NSGA-II Algorithm**: Deb et al. (2002)
- **Parallel Coordinates**: Inselberg & Dimsdale (1990)
- **React Documentation**: https://react.dev/
- **FastAPI Documentation**: https://fastapi.tiangolo.com/