Atomizer/docs/03_GETTING_STARTED.md

How to Extend an Optimization Study

Date: November 20, 2025

When you want to run more iterations to get better results, you have three options:

---

Option 1: Continue Existing Study (Recommended)

Best for: When you want to keep all previous trial data and just add more iterations

Advantages:

• Preserves all existing trials
• Continues from current best result
• Uses accumulated knowledge from previous trials
• More efficient (no wasted trials)

Process:

Step 1: Wait for current optimization to finish

Check if the v2.1 test is still running:

# On Windows
tasklist | findstr python

# Check background job status
# Look for the running optimization process

Step 2: Run the continuation script

cd studies/circular_plate_protocol10_v2_1_test
python continue_optimization.py

Step 3: Configure number of additional trials

Edit continue_optimization.py:29:

# CONFIGURE THIS: Number of additional trials to run
ADDITIONAL_TRIALS = 50  # Change to 100 for total of ~150 trials

Example: If you ran 50 trials initially and want 100 total:

• Set ADDITIONAL_TRIALS = 50
• Study will run trials #50-99 (continuing from where it left off)
• All 100 trials will be in the same study database

---

Option 2: Modify Config and Restart

Best for: When you want a completely fresh start with more iterations

Advantages:

• Clean slate optimization
• Good for testing different configurations
• Simpler to understand (one continuous run)

Disadvantages:

• Loses all previous trial data
• Wastes computational budget if previous trials were good

Process:

Step 1: Stop any running optimization

# Kill the running process if needed
# On Windows, find the PID and:
taskkill /PID <process_id> /F

Step 2: Edit optimization config

Edit studies/circular_plate_protocol10_v2_1_test/1_setup/optimization_config.json:

{
  "trials": {
    "n_trials": 100,  // Changed from 50 to 100
    "timeout_per_trial": 3600
  }
}

Step 3: Delete old results

cd studies/circular_plate_protocol10_v2_1_test

# Delete old database and history
del 2_results\study.db
del 2_results\optimization_history_incremental.json
del 2_results\intelligent_optimizer\*.*

Step 4: Rerun optimization

python run_optimization.py

---

Option 3: Wait and Evaluate First

Best for: When you're not sure if more iterations are needed

Process:

Step 1: Wait for current test to finish

The v2.1 test is currently running with 50 trials. Let it complete first.

Step 2: Check results

cd studies/circular_plate_protocol10_v2_1_test

# View optimization report
type 3_reports\OPTIMIZATION_REPORT.md

# Or check test summary
type 2_results\test_summary.json

Step 3: Evaluate performance

Look at:

• Best error: Is it < 0.1 Hz? (target achieved)
• Convergence: Has it plateaued or still improving?
• Pruning rate: < 5% is good

Step 4: Decide next action

• If target achieved: Done! No need for more trials
• If converging: Add 20-30 more trials (Option 1)
• If struggling: May need algorithm adjustment, not more trials

---

Comparison Table

Feature	Option 1: Continue	Option 2: Restart	Option 3: Wait
Preserves data	✅ Yes	❌ No	✅ Yes
Efficient	✅ Very	❌ Wasteful	✅ Most
Easy to set up	✅ Simple	⚠️ Moderate	✅ Simplest
Best use case	Adding more trials	Testing new config	Evaluating first

---

Detailed Example: Extending to 100 Trials

Let's say the v2.1 test (50 trials) finishes with:

• Best error: 0.25 Hz (not at target yet)
• Convergence: Still improving
• Pruning rate: 4% (good)

Recommendation: Continue with 50 more trials (Option 1)

Step-by-step:

1. Check current status:

   import optuna
   storage = "sqlite:///studies/circular_plate_protocol10_v2_1_test/2_results/study.db"
   study = optuna.load_study(study_name="circular_plate_protocol10_v2_1_test", storage=storage)
   
   print(f"Current trials: {len(study.trials)}")
   print(f"Best error: {study.best_value:.4f} Hz")
   

2. Edit continuation script:

   # In continue_optimization.py line 29
   ADDITIONAL_TRIALS = 50  # Will reach ~100 total
   

3. Run continuation:

   cd studies/circular_plate_protocol10_v2_1_test
   python continue_optimization.py
   

4. Monitor progress:

   • Watch console output for trial results
   • Check optimization_history_incremental.json for updates
   • Look for convergence (error decreasing)

5. Verify results:

   # After completion
   study = optuna.load_study(...)
   print(f"Total trials: {len(study.trials)}")  # Should be ~100
   print(f"Final best error: {study.best_value:.4f} Hz")
   

---

Understanding Trial Counts

Important: The "total trials" count includes both successful and pruned trials.

Example breakdown:

Total trials: 50
├── Successful: 47 (94%)
│   └── Used for optimization
└── Pruned: 3 (6%)
    └── Rejected (invalid parameters, simulation failures)

When you add 50 more trials:

Total trials: 100
├── Successful: ~94 (94%)
└── Pruned: ~6 (6%)

The optimization algorithm only learns from successful trials, so:

• 50 successful trials ≈ 53 total trials (with 6% pruning)
• 100 successful trials ≈ 106 total trials (with 6% pruning)

---

Best Practices

When to Add More Trials:

✅ Error still decreasing (not converged yet) ✅ Close to target but need refinement ✅ Exploring new parameter regions

When NOT to Add More Trials:

❌ Error has plateaued for 20+ trials ❌ Already achieved target tolerance ❌ High pruning rate (>10%) - fix validation instead ❌ Wrong algorithm selected - fix strategy selector instead

How Many to Add:

• Close to target (within 2x tolerance): Add 20-30 trials
• Moderate distance (2-5x tolerance): Add 50 trials
• Far from target (>5x tolerance): Investigate root cause first

---

Monitoring Long Runs

For runs with 100+ trials (several hours):

Option A: Run in background (Windows)

# Start minimized
start /MIN python continue_optimization.py

Option B: Use screen/tmux (if available)

# Not standard on Windows, but useful on Linux/Mac
tmux new -s optimization
python continue_optimization.py
# Detach: Ctrl+B, then D
# Reattach: tmux attach -t optimization

Option C: Monitor progress file

# Check progress without interrupting
import json
with open('2_results/optimization_history_incremental.json') as f:
    history = json.load(f)

print(f"Completed trials: {len(history)}")
best = min(history, key=lambda x: x['objective'])
print(f"Current best: {best['objective']:.4f} Hz")

---

Troubleshooting

Issue: "Study not found in database"

Cause: Initial optimization hasn't run yet or database corrupted Fix: Run run_optimization.py first to create the initial study

Issue: Continuation starts from trial #0

Cause: Study database exists but is empty Fix: Delete database and run fresh optimization

Issue: NX session conflicts

Cause: Multiple NX sessions accessing same model Solution: NX Session Manager handles this automatically, but verify:

from optimization_engine.nx_session_manager import NXSessionManager
mgr = NXSessionManager()
print(mgr.get_status_report())

Issue: High pruning rate in continuation

Cause: Optimization exploring extreme parameter regions Fix: Simulation validator should prevent this, but verify rules are active

---

Summary: For your case (wanting 100 iterations), use Option 1 with the continue_optimization.py script. Set ADDITIONAL_TRIALS = 50 and run it after the current test finishes.