Factor Effects in Numerical Simulations

Abstract

Numerical simulations allow users to adjust factor settings in experimental runs to understand how changes in those factors affect the output. However, it is not straightforward to analyze these outputs when multiple input factors are changed, especially simultaneously. For the atmospheric sciences, Stein and Alpert introduced a method they termed “factor separation” in order to separate the “pure contribution” of a factor from “pure interactions” of combinations of factors. Although factor separation appears to be used exclusively within the atmospheric sciences, other communities achieve a similar result by computing “main effects” via design of experiments methods. While both methods yield different estimates for the factor effects or contributions, we show that factor separation effects are identical to “simple effects” in the design of experiments literature. We demonstrate how both factor separation effects and design of experiments main effects correspond to multiple linear regression coefficients with different coding methods; thus, effect estimates produced by each method are equivalent through a variable transformation. We illustrate the application of both methods using a shallow-water simulation. This connection between factor separation and the design of experiments discipline extends factor separation to more applications by making available design of experiments methods for decreasing the computational cost and calculating effects for factors with more than two settings, both of which are limitations of factor separation.