Atomizer/docs/FIX_VALIDATOR_PRUNING.md

Validator Pruning Investigation - November 20, 2025

DEPRECATED - This document is retained for historical reference only.

Status: Investigation completed. Aspect ratio validation approach was abandoned.

---

Original Problem

The v2.1 and v2.2 tests showed 18-20% pruning rate. Investigation revealed two separate issues:

Issue 1: Validator Not Enforcing Rules (FIXED, then REMOVED)

The _validate_circular_plate_aspect_ratio() method initially returned only warnings, not rejections.

Fix Applied: Changed to return hard rejections for aspect ratio violations.

Result: All pruned trials in v2.2 still had VALID aspect ratios (5.0-50.0 range).

Conclusion: Aspect ratio violations were NOT the cause of pruning.

Issue 2: pyNastran False Positives (ROOT CAUSE)

All pruned trials failed due to pyNastran FATAL flag sensitivity:

• ✅ Nastran simulations succeeded (F06 files have no errors)
• ⚠️ FATAL flag in OP2 header (benign warning)
• ❌ pyNastran throws exception when reading OP2
• ❌ Valid trials incorrectly marked as failed

Evidence: All 9 pruned trials in v2.2 had:

• is_pynastran_fatal_flag: true
• f06_has_fatal_errors: false
• Valid aspect ratios within bounds

---

Final Solution (Post-v2.3)

Aspect Ratio Validation REMOVED

After deploying v2.3 with aspect ratio validation, user feedback revealed:

User Requirement: "I never asked for this check, where does that come from?"

Issue: Arbitrary aspect ratio limits (5.0-50.0) without:

• User approval
• Physical justification for circular plate modal analysis
• Visibility in optimization_config.json

Fix Applied:

• Removed ALL aspect ratio validation from circular_plate model type
• Validator now returns empty rules {}
• Relies solely on Optuna parameter bounds (50-150mm diameter, 2-10mm thickness)

User Requirements Established:

1. No arbitrary checks - validation rules must be proposed, not automatic
2. Configurable validation - rules should be visible in optimization_config.json
3. Parameter bounds suffice - ranges already define feasibility
4. Physical justification required - any constraint needs clear reasoning

Real Solution: Robust OP2 Extraction

Module: optimization_engine/op2_extractor.py

Multi-strategy extraction that handles pyNastran issues:

1. Standard OP2 read
2. Lenient read (debug=False, skip benign flags)
3. F06 fallback parsing

See PRUNING_DIAGNOSTICS.md for details.

---

Lessons Learned

1. Validator is for simulation failures, not arbitrary physics assumptions

   • Parameter bounds already define feasible ranges
   • Don't add validation rules without user approval

2. 18% pruning was pyNastran false positives, not validation issues

   • All pruned trials had valid parameters
   • Robust extraction eliminates these false positives

3. Transparency is critical

   • Validation rules must be visible in optimization_config.json
   • Arbitrary constraints confuse users and reject valid designs

---

Current State

File: simulation_validator.py

if model_type == 'circular_plate':
    # NOTE: Only use parameter bounds for validation
    # No arbitrary aspect ratio checks - let Optuna explore the full parameter space
    # Modal analysis is robust and doesn't need strict aspect ratio limits
    return {}

Impact: Clean separation of concerns

• Parameter bounds = Feasibility (user-defined ranges)
• Validator = Genuine simulation failures (e.g., mesh errors, solver crashes)

---

References

• SESSION_SUMMARY_NOV20.md - Complete session documentation
• PRUNING_DIAGNOSTICS.md - Robust extraction solution
• optimization_engine/simulation_validator.py - Current validator implementation