Assessment of Model Validation, Calibration, and Prediction Approaches in the Presence of Uncertainty

Abstract

Model validation is the process of determining the degree to which a model is an accurate representation of the true value in the real world. The results of a model validation study can be used to either quantify the model form uncertainty or to improve/calibrate the model. The model validation process becomes complex when there is uncertainty in the simulation and/or experimental outcomes. These uncertainties can be in the form of aleatory uncertainties due to randomness or epistemic uncertainties due to lack of knowledge. Five different approaches are used for addressing model validation and predictive capability: (1) the area validation metric (AVM), (2) a modified area validation metric (MAVM) with confidence intervals, (3) the validation uncertainty procedure from ASME V&V 20, (4) a calibration procedure interpreted from ASME V&V 20, and (5) identification of the model discrepancy term using Bayesian estimation. To provide an unambiguous assessment of these different approaches, synthetic experimental data is generated from computational fluid dynamics simulations of an airfoil with a flap. A simplified model is then developed using thin airfoil theory. The accuracy of the simplified model is assessed using the synthetic experimental data. The quantities examined include the two-dimensional lift and moment coefficients for the airfoil with varying angles of attack and flap deflection angles. Each of these approaches is assessed for the ability to tightly encapsulate the true value at conditions both where experimental results are provided and prediction locations where no experimental data are available. Generally, it was seen that the MAVM performed the best in cases where there is a sparse amount of data and/or large extrapolations. Furthermore, it was found that Bayesian estimation outperformed the others where there is an extensive amount of experimental data that covers the application domain.