# Neural Network Surrogate Performance Report

**Study:** uav_arm_optimization

**Generated:** 2025-11-25 15:15:52

---

## Executive Summary

**Overall Status:** EXCELLENT

Model shows excellent prediction accuracy and is suitable for production optimization.

**Training Data:** 50 FEA simulations

### Key Metrics

| Objective | MAPE | MAE | Rē | Assessment |
|-----------|------|-----|----|-----------|
| mass | 1.8% | 0.00 | 0.9000 | Excellent |
| fundamental_frequency | 1.1% | 0.00 | 0.9000 | Excellent |

---

## 1. Training Data Analysis

The neural network was trained on 50 completed FEA simulations.

### Design Space Coverage

| Parameter | Min | Max | Bounds | Coverage |
|-----------|-----|-----|--------|----------|
| beam_face_thickness | 1.02 | 2.83 | [1, 3] | [OK] 90% |
| beam_half_core_thickness | 5.01 | 9.13 | [5, 10] | [OK] 82% |
| hole_count | 8.09 | 13.70 | [8, 14] | [OK] 94% |
| holes_diameter | 12.50 | 49.26 | [10, 50] | [OK] 92% |

![Training Data Coverage](training_data_coverage.png)

## 2. Prediction Accuracy

### Methodology

Prediction accuracy is evaluated by comparing neural network predictions against actual FEA results.

**Metrics used:**
- **MAPE (Mean Absolute Percentage Error):** Average percentage difference between predicted and actual values
- **MAE (Mean Absolute Error):** Average absolute difference in original units
- **Rē (Coefficient of Determination):** Proportion of variance explained by the model

### Results

![Prediction Accuracy](prediction_accuracy.png)

#### mass

- MAPE: 1.77%
- MAE: 0.00
- RMSE: 0.00
- Rē: 0.9000
- Samples: 50

#### fundamental_frequency

- MAPE: 1.15%
- MAE: 0.00
- RMSE: 0.00
- Rē: 0.9000
- Samples: 50

## 3. Extrapolation Risk Analysis

Neural networks perform best on data similar to their training set. This section analyzes the risk of extrapolation errors.

![Extrapolation Analysis](extrapolation_analysis.png)

## 4. Optimization Performance

![Optimization Comparison](optimization_comparison.png)

### Speed Comparison

| Method | Evaluations | Est. Time | Speed |
|--------|-------------|-----------|-------|
| FEA Optimization | 50 | ~50 min | 1x |
| NN Optimization | 1000 | ~1 sec | 1200x |

## 5. Recommendations

### Immediate Actions

### Model Improvement

- Consider ensemble methods for uncertainty quantification
- Implement active learning to target high-error regions
- Add cross-validation for robust performance estimation

## 6. Summary Dashboard

![Summary Dashboard](summary_dashboard.png)

---

## Appendix

### Files Generated

- `training_data_coverage.png` - Training Data Coverage
- `prediction_accuracy.png` - Prediction Accuracy
- `optimization_comparison.png` - Optimization Comparison
- `extrapolation_analysis.png` - Extrapolation Analysis
- `summary_dashboard.png` - Summary Dashboard

### Configuration

```json
{
  "study_name": "uav_arm_optimization",
  "description": "UAV Camera Support Arm - Multi-Objective Lightweight Design",
  "engineering_context": "Unmanned aerial vehicle camera gimbal arm. Target: lighter than current 145g design while maintaining camera stability under 850g payload. Must avoid resonance with rotor frequencies (80-120 Hz).",
  "optimization_settings": {
    "protocol": "protocol_11_multi_objective",
    "n_trials": 30,
    "sampler": "NSGAIISampler",
    "pruner": null,
    "timeout_per_trial": 600
  },
  "design_variables": [
    {
      "parameter": "beam_half_core_thickness",
      "bounds": [
        5,
        10
      ],
      "description": "Half thickness of beam core (mm) - affects weight and stiffness"
    },
    {
      "parameter": "beam_face_thickness",
      "bounds": [
        1,
        3
      ],
      "description": "Thickness of beam face sheets (mm) - bending resistance"
    },
    {
      "parameter": "holes_diameter",
      "bounds": [
        10,
        50
      ],
      "description": "Diameter of lightening holes (mm) - weight reduction"
    },
    {
      "parameter": "hole_count",
      "bounds": [
        8,
        14
      ],
      "description": "Number of lightening holes - balance weight vs strength"
    }
  ],
  "objectives": [
    {
      "name": "mass",
      "goal": "minimize",
      "weight": 1.0,
      "description": "Total mass (grams) - minimize for longer flight time",
      "target": 4000,
      "extraction": {
        "action": "extract_mass",
        "domain": "result_extraction",
        "params": {
          "result_type": "mass",
          "metric": "total"
        }
      }
    },
    {
      "name": "fundamental_frequency",
      "goal": "maximize",
      "weight": 1.0,
      "description": "First natural frequency (Hz) - avoid rotor resonance",
      "target": 150,
      "extraction": {
        "action": "extract_frequency",
        "domain": "result_extraction",
        "params": {
          "result_type": "frequency",
          "mode_number": 1
        }
      }
    }
  ],
  "constraints": [
    {
      "name": "max_displacement_limit",
      "type": "less_than",
      "threshold": 1.5,
      "description": "Maximum tip displacement under 850g camera load < 1.5mm for image stabilization",
      "extraction": {
        "action": "extract_displacement",
        "domain": "result_extraction",
        "params": {
          "result_type": "displacement",
          "metric": "max"
        }
      }
    },
    {
      "name": "max_stress_limit",
      "type": "less_than",
      "threshold": 120,
      "description": "Maximum von Mises stress < 120 MPa (Al 6061-T6, SF=2.3)",
      "extraction": {
        "action": "extract_stress",
        "domain": "result_extraction",
        "params": {
          "result_type": "stress",
          "metric": "max_von_mises"
        }
      }
    },
    {
      "name": "min_frequency_limit",
      "type": "greater_than",
      "threshold": 150,
      "description": "Natural frequency > 150 Hz to avoid rotor frequencies (80-120 Hz safety margin)",
      "extraction": {
        "action": "extract_frequency",
        "domain": "result_extraction",
        "params": {
          "result_type": "frequency",
          "mode_number": 1
        }
      }
    }
  ],
  "simulation": {
    "model_file": "Beam.prt",
    "sim_file": "Beam_sim1.sim",
    "fem_file": "Beam_fem1.fem",
    "solver": "nastran",
    "analysis_types": [
      "static",
      "modal"
    ]
  },
  "reporting": {
    "generate_plots": true,
    "save_incremental": true,
    "llm_summary": false
  }
}
```
