docs: Major documentation overhaul - restructure folders, update tagline, add Getting Started guide

- Restructure docs/ folder (remove numeric prefixes): - 04_USER_GUIDES -> guides/ - 05_API_REFERENCE -> api/ - 06_PHYSICS -> physics/ - 07_DEVELOPMENT -> development/ - 08_ARCHIVE -> archive/ - 09_DIAGRAMS -> diagrams/ - Replace tagline 'Talk, don't click' with 'LLM-driven optimization framework' in 9 files - Create comprehensive docs/GETTING_STARTED.md: - Prerequisites and quick setup - Project structure overview - First study tutorial (Claude or manual) - Dashboard usage guide - Neural acceleration introduction - Rewrite docs/00_INDEX.md with correct paths and modern structure - Archive obsolete files: - 01_PROTOCOLS.md -> archive/historical/01_PROTOCOLS_legacy.md - 03_GETTING_STARTED.md -> archive/historical/ - ATOMIZER_PODCAST_BRIEFING.md -> archive/marketing/ - Update timestamps to 2026-01-20 across all key files - Update .gitignore to exclude docs/generated/ - Version bump: ATOMIZER_CONTEXT v1.8 -> v2.0
2026-01-20 10:03:45 -05:00
parent 37f73cc2be
commit ea437d360e
103 changed files with 8980 additions and 327 deletions
--- a/docs/archive/historical/OPTUNA_DASHBOARD.md
+++ b/docs/archive/historical/OPTUNA_DASHBOARD.md
@@ -0,0 +1,227 @@
+# Optuna Dashboard Integration
+
+Atomizer leverages Optuna's built-in dashboard for advanced real-time optimization visualization.
+
+## Quick Start
+
+### 1. Install Optuna Dashboard
+
+```bash
+# Using atomizer environment
+conda activate atomizer
+pip install optuna-dashboard
+```
+
+### 2. Launch Dashboard for a Study
+
+```bash
+# Navigate to your substudy directory
+cd studies/simple_beam_optimization/substudies/full_optimization_50trials
+
+# Launch dashboard pointing to the Optuna study database
+optuna-dashboard sqlite:///optuna_study.db
+```
+
+The dashboard will start at http://localhost:8080
+
+### 3. View During Active Optimization
+
+```bash
+# Start optimization in one terminal
+python studies/simple_beam_optimization/run_optimization.py
+
+# In another terminal, launch dashboard
+cd studies/simple_beam_optimization/substudies/full_optimization_50trials
+optuna-dashboard sqlite:///optuna_study.db
+```
+
+The dashboard updates in real-time as new trials complete!
+
+---
+
+## Dashboard Features
+
+### **1. Optimization History**
+- Interactive plot of objective value vs trial number
+- Hover to see parameter values for each trial
+- Zoom and pan for detailed analysis
+
+### **2. Parallel Coordinate Plot**
+- Multi-dimensional visualization of parameter space
+- Each line = one trial, colored by objective value
+- Instantly see parameter correlations
+
+### **3. Parameter Importances**
+- Identifies which parameters most influence the objective
+- Based on fANOVA (functional ANOVA) analysis
+- Helps focus optimization efforts
+
+### **4. Slice Plot**
+- Shows objective value vs individual parameters
+- One plot per design variable
+- Useful for understanding parameter sensitivity
+
+### **5. Contour Plot**
+- 2D contour plots of objective surface
+- Select any two parameters to visualize
+- Reveals parameter interactions
+
+### **6. Intermediate Values**
+- Track metrics during trial execution (if using pruning)
+- Useful for early stopping of poor trials
+
+---
+
+## Advanced Usage
+
+### Custom Port
+
+```bash
+optuna-dashboard sqlite:///optuna_study.db --port 8888
+```
+
+### Multiple Studies
+
+```bash
+# Compare multiple optimization runs
+optuna-dashboard sqlite:///substudy1/optuna_study.db sqlite:///substudy2/optuna_study.db
+```
+
+### Remote Access
+
+```bash
+# Allow connections from other machines
+optuna-dashboard sqlite:///optuna_study.db --host 0.0.0.0
+```
+
+---
+
+## Integration with Atomizer Workflow
+
+### Study Organization
+
+Each Atomizer substudy has its own Optuna database:
+
+```
+studies/simple_beam_optimization/
+├── substudies/
+│   ├── full_optimization_50trials/
+│   │   ├── optuna_study.db        # ← Optuna database (SQLite)
+│   │   ├── optuna_study.pkl       # ← Optuna study object (pickle)
+│   │   ├── history.json           # ← Atomizer history
+│   │   └── plots/                 # ← Matplotlib plots
+│   └── validation_3trials/
+│       └── optuna_study.db
+```
+
+### Visualization Comparison
+
+**Optuna Dashboard** (Interactive, Web-based):
+- ✅ Real-time updates during optimization
+- ✅ Interactive plots (zoom, hover, filter)
+- ✅ Parameter importance analysis
+- ✅ Multiple study comparison
+- ❌ Requires web browser
+- ❌ Not embeddable in reports
+
+**Atomizer Matplotlib Plots** (Static, High-quality):
+- ✅ Publication-quality PNG/PDF exports
+- ✅ Customizable styling and annotations
+- ✅ Embeddable in reports and papers
+- ✅ Offline viewing
+- ❌ Not interactive
+- ❌ Not real-time
+
+**Recommendation**: Use **both**!
+- Monitor optimization in real-time with Optuna Dashboard
+- Generate final plots with Atomizer visualizer for reports
+
+---
+
+## Troubleshooting
+
+### "No studies found"
+
+Make sure you're pointing to the correct database file:
+
+```bash
+# Check if optuna_study.db exists
+ls studies/*/substudies/*/optuna_study.db
+
+# Use absolute path if needed
+optuna-dashboard sqlite:///C:/Users/antoi/Documents/Atomaste/Atomizer/studies/simple_beam_optimization/substudies/full_optimization_50trials/optuna_study.db
+```
+
+### Database Locked
+
+If optimization is actively writing to the database:
+
+```bash
+# Use read-only mode
+optuna-dashboard sqlite:///optuna_study.db?mode=ro
+```
+
+### Port Already in Use
+
+```bash
+# Use different port
+optuna-dashboard sqlite:///optuna_study.db --port 8888
+```
+
+---
+
+## Example Workflow
+
+```bash
+# 1. Start optimization
+python studies/simple_beam_optimization/run_optimization.py
+
+# 2. In another terminal, launch Optuna dashboard
+cd studies/simple_beam_optimization/substudies/full_optimization_50trials
+optuna-dashboard sqlite:///optuna_study.db
+
+# 3. Open browser to http://localhost:8080 and watch optimization live
+
+# 4. After optimization completes, generate static plots
+python -m optimization_engine.visualizer studies/simple_beam_optimization/substudies/full_optimization_50trials png pdf
+
+# 5. View final plots
+explorer studies/simple_beam_optimization/substudies/full_optimization_50trials/plots
+```
+
+---
+
+## Optuna Dashboard Screenshots
+
+### Optimization History
+![Optuna History](https://optuna.readthedocs.io/en/stable/_images/dashboard_history.png)
+
+### Parallel Coordinate Plot
+![Optuna Parallel Coords](https://optuna.readthedocs.io/en/stable/_images/dashboard_parallel_coordinate.png)
+
+### Parameter Importance
+![Optuna Importance](https://optuna.readthedocs.io/en/stable/_images/dashboard_param_importances.png)
+
+---
+
+## Further Reading
+
+- [Optuna Dashboard Documentation](https://optuna-dashboard.readthedocs.io/)
+- [Optuna Visualization Module](https://optuna.readthedocs.io/en/stable/reference/visualization/index.html)
+- [fANOVA Parameter Importance](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.importance.FanovaImportanceEvaluator.html)
+
+---
+
+## Summary
+
+| Feature | Optuna Dashboard | Atomizer Matplotlib |
+|---------|-----------------|-------------------|
+| Real-time updates | ✅ Yes | ❌ No |
+| Interactive | ✅ Yes | ❌ No |
+| Parameter importance | ✅ Yes | ⚠️  Manual |
+| Publication quality | ⚠️  Web only | ✅ PNG/PDF |
+| Embeddable in docs | ❌ No | ✅ Yes |
+| Offline viewing | ❌ Needs server | ✅ Yes |
+| Multi-study comparison | ✅ Yes | ⚠️  Manual |
+
+**Best Practice**: Use Optuna Dashboard for monitoring and exploration, Atomizer visualizer for final reporting.