Atomizer/docs/plans/CANVAS_DEEP_FIX_INVESTIGATION.md

# Canvas Deep Fix Investigation

**Date**: January 16, 2026
**Status**: ✅ IMPLEMENTATION COMPLETE

---

## Executive Summary

Four critical issues have been identified that are blocking Canvas functionality:

| # | Issue | Root Cause | Severity |
|---|-------|------------|----------|
| 1 | Claude Chat Not Working | `asyncio.create_subprocess_exec` fails on Windows | CRITICAL |
| 2 | Expressions Can't Connect to Model | ModelNode has `inputs={0}` - no input handle | CRITICAL |
| 3 | File Browser Only Shows Studies | Web API can't access OS file system | HIGH |
| 4 | Introspection is Fake | Only reads config files, not actual NX models | HIGH |

---

## Issue 1: Claude Chat NotImplementedError

### Root Cause
```python
# session_manager.py line 138
process = await asyncio.create_subprocess_exec(...)
```

On Windows, `asyncio.create_subprocess_exec` raises `NotImplementedError` because Windows doesn't support the ProactorEventLoop subprocess methods the same way Unix does.

### Evidence
```
Traceback:
  File "session_manager.py", line 138, in create_session
    process = await asyncio.create_subprocess_exec(
  File "asyncio\subprocess.py", line 218, in create_subprocess_exec
  File "asyncio\base_events.py", line 498, in _make_subprocess_transport
    raise NotImplementedError
NotImplementedError
```

### Solution
Replace async subprocess with synchronous subprocess + ThreadPoolExecutor:

```python
import subprocess
from concurrent.futures import ThreadPoolExecutor

executor = ThreadPoolExecutor(max_workers=4)

async def create_session(...):
    # Instead of asyncio.create_subprocess_exec
    loop = asyncio.get_event_loop()
    process = await loop.run_in_executor(
        executor,
        lambda: subprocess.Popen(
            ["claude", "--print", ...],
            stdin=subprocess.PIPE,
            stdout=subprocess.PIPE,
            stderr=subprocess.PIPE,
            cwd=str(ATOMIZER_ROOT),
        )
    )
```

**OR** - Simpler approach: Skip session-based subprocess entirely, use HTTP streaming for chat:

The `/api/claude/chat/stream` endpoint already works (it uses `claude_cli_agent.py` which runs Claude one-shot). The WebSocket session approach is over-engineered for the use case.

### Fix Strategy
1. Make `create_session` return a "virtual" session (no subprocess)
2. Route all messages through HTTP streaming endpoint
3. Keep conversation history in `ConversationStore` database
4. WebSocket just wraps the HTTP streaming calls

---

## Issue 2: Expressions Can't Connect to Model

### Root Cause

**ModelNode.tsx:**
```tsx
<BaseNode {...props} icon={...} iconColor="text-blue-400" inputs={0}>
```
`inputs={0}` means Model has **NO input handle** - nothing can connect TO it!

**DesignVarNode.tsx:**
```tsx
<BaseNode {...props} icon={...} iconColor="text-emerald-400">
```
Uses defaults (`inputs=1, outputs=1`) but DesignVar should have:
- `inputs=0` (it's a source node)
- `outputs=1` (connects to Model)

### Visual Problem
```
Current (WRONG):
  DesignVar ←─ Model ──→ Solver
     ↑           ↓
  (has input) (has output only)

Should be:
  DesignVar ──→ Model ──→ Solver
     ↓           ↑↓
  (output)   (input & output)
```

### Fix Required

**ModelNode.tsx** - Add input handle:
```tsx
<BaseNode {...props} icon={...} iconColor="text-blue-400" inputs={1} outputs={1}>
```

**DesignVarNode.tsx** - Remove input handle:
```tsx
<BaseNode {...props} icon={...} iconColor="text-emerald-400" inputs={0} outputs={1}>
```

**SurrogateNode.tsx** - Should be terminal (no output):
```tsx
<BaseNode {...props} icon={...} iconColor="text-pink-400" inputs={1} outputs={0}>
```

---

## Issue 3: File Browser Only Shows Studies Folder

### Root Cause
The current `FileBrowser.tsx` uses fetch to `/api/files/list` which only lists files within the `studies/` directory. The user wants:

1. Native OS file picker to select files from ANYWHERE
2. Import selected files into the study folder
3. Copy all related files (.prt, .sim, .fem, _i.prt, etc.)

### Web Browser Limitation
Browsers can't access the local file system directly for security. Options:

**Option A: File System Access API** (Chrome/Edge only)
```typescript
const handle = await window.showOpenFilePicker({
  types: [{ description: 'NX Files', accept: { '*/*': ['.sim', '.prt', '.fem'] } }]
});
const file = await handle.getFile();
// Upload to backend
```

**Option B: Traditional File Input**
```tsx
<input type="file" accept=".sim,.prt,.fem,.afem" onChange={handleFileUpload} />
```
Then upload to backend which saves to study folder.

**Option C: Backend Path Input + Validation**
User enters full Windows path (e.g., `C:\NX_Models\bracket.prt`), backend validates and copies.

### Recommended Solution
Combine B + C:
1. File input for direct upload (drag & drop)
2. Path input for network drives/existing paths
3. Backend endpoint to copy/import files

---

## Issue 4: Introspection is Fake

### Root Cause
Current `nx_introspection.py` does NOT actually read NX files. It only:
- Reads `optimization_config.json` for existing design variables
- Infers expressions based on folder names ("mirror" → suggest mirror expressions)
- Guesses solver type from file names

### What Real Introspection Needs

**For .prt files (NX Part):**
- Use NX Open API to read expressions
- Get expression names, values, units, formulas
- Requires NX to be installed and licensed

**For .sim files (Simulation):**
- Parse XML-like structure or use NX Open
- Get solver type, boundary conditions, loads
- Identify linked .fem and .prt files

**For .fem files (FEM):**
- Get mesh statistics (nodes, elements)
- Material properties
- Element types used

**For .op2 files (Results):**
- Use PyNastran to read binary results
- Extract displacement, stress, frequency data
- Get node/element IDs for specific extractions

### Implementation Approach

**Phase 1: File Discovery (no NX needed)**
```python
def discover_related_files(sim_path: Path) -> List[Dict]:
    """Find all related files by naming convention"""
    # model_sim1.sim → model.prt, model_fem1.fem, model_fem1_i.prt
```

**Phase 2: Config-based Expression Discovery**
```python
def discover_expressions_from_config(study_dir: Path) -> List[Dict]:
    """Read optimization_config.json for design variables"""
```

**Phase 3: NX Open Integration (requires NX)**
```python
def introspect_with_nx_open(prt_path: Path) -> Dict:
    """Use NX Open API to read actual expressions"""
    # This requires NX to be running
    # Use the existing nx_journals/ infrastructure
```

**Phase 4: OP2 Result Analysis (PyNastran)**
```python
def analyze_op2_results(op2_path: Path) -> Dict:
    """Read OP2 file to discover available result types"""
    from pyNastran.op2.op2 import OP2
    op2 = OP2()
    op2.read_op2(str(op2_path))
    # Return available subcases, result types, etc.
```

---

## Implementation Plan

### Phase 1: Fix Claude Chat (CRITICAL - 30 min)
1. Modify `create_session` to not spawn subprocess
2. Keep session metadata in database only
3. Route all messages through HTTP streaming
4. WebSocket wraps HTTP calls

### Phase 2: Fix Node Handles (CRITICAL - 15 min)
1. Update `ModelNode.tsx`: `inputs={1}`
2. Update `DesignVarNode.tsx`: `inputs={0}, outputs={1}`
3. Update `SurrogateNode.tsx`: `outputs={0}`
4. Test connections work correctly

### Phase 3: Native File Import (HIGH - 45 min)
1. Add file upload input to FileBrowser
2. Create backend `/api/files/upload` endpoint
3. Add path input with validation
4. Create `/api/files/import` for path-based import
5. Copy all related files to study folder

### Phase 4: Real Introspection Service (HIGH - 2 hours)
1. File discovery by naming convention
2. OP2 analysis with PyNastran
3. NX Open integration (optional, requires NX running)
4. Return comprehensive file metadata

### Phase 5: Integration Testing (30 min)
1. Test complete workflow: Select model → Introspect → Add Design Vars → Connect → Execute
2. Fix any remaining issues

---

## Files to Modify

### Backend
- `session_manager.py` - Fix Windows subprocess issue
- `files.py` - Add upload/import endpoints
- `nx_introspection.py` - Real introspection logic

### Frontend
- `ModelNode.tsx` - Add input handle
- `DesignVarNode.tsx` - Remove input, keep output
- `SurrogateNode.tsx` - Remove output
- `FileBrowser.tsx` - Add file upload, path input
- `IntrospectionPanel.tsx` - Display real introspection data

---

## Estimated Total Time: 4-5 hours

---

## Implementation Summary (Completed)

### Phase 1: Claude Chat Windows Fix ✅
**File**: `atomizer-dashboard/backend/api/services/session_manager.py`
- Replaced `asyncio.create_subprocess_exec` with `subprocess.Popen`
- Used `ThreadPoolExecutor` with `run_in_executor()` for async compatibility
- Made sessions stateless (no persistent subprocess)
- Each message handled via one-shot CLI call with 5-minute timeout

### Phase 2: Node Handles ✅
**Files**:
- `ModelNode.tsx`: Changed `inputs={0}` to `inputs={1}` (now accepts connections)
- `DesignVarNode.tsx`: Added `inputs={0} outputs={1}` (source node)

### Phase 3: Native File Import ✅
**Files**:
- `files.py`: Added `/validate-path`, `/import-from-path`, `/upload` endpoints
- `FileBrowser.tsx`: Complete rewrite with 3 tabs:
  - Browse Studies (existing)
  - Import Path (paste Windows path, validate, import related files)
  - Upload Files (drag & drop)

### Phase 4: Real NX Introspection ✅
**File**: `atomizer-dashboard/backend/api/services/nx_introspection.py`
- Added PyNastran OP2 parsing (displacements, eigenvectors, stress)
- BDF/DAT file analysis (mass, grid count, element counts, solver type)
- Study database queries for expression discovery
- Related file discovery by naming convention
- Result file discovery with trial folder detection