Improving Model Performance Using Season-Based Evaluation

Abstract

Computer models have become vital decision-making tools in many areas of science and engineering including water resources. However, models should be properly evaluated before use to improve the likelihood of making sound decisions based on their results. The model evaluation technique practiced today in hydrology assumes that model parameters are season insensitive and attempts to identify “optimal” values that would describe watershed behavior during dry and wet seasons. This assumption could compromise accuracy of model predictions. This study demonstrates performance improvement that would be achieved when a season-based model evaluation approach is pursued. A global sensitivity analysis (SA) model has been used to investigate seasonal sensitivity of streamflow parameters of a watershed simulation model on the headwaters of the Little River Watershed, one of the United States Department of Agriculture’s experimental watersheds. Two separate analyses have been performed: the conventional approach in which model parameters are assumed to be season insensitive; and a season-based evaluation in which the influential parameters may vary for months with a low runoff coefficient and months with a high runoff coefficient. The sensitivity analysis helped to identify dominant model and watershed behaviors for the conventional annual approach and for the wet and dry seasons. The SA results show that the influential parameters exhibited modest seasonal sensitivity for the experimental watershed. Model calibration was then performed by using the dynamically dimensioned search (DDS) algorithm for the conventional and season-based approaches using the principal parameters identified by the global SA model. Performance of the calibration attempts have been verified with the traditional split-sampling technique and also by assessing effectiveness of the model in predicting internal watershed behaviors through comparison of simulated streamflow with observations at multiple internal sites not used for calibration. Several efficiency measures have been used to test goodness of the model simulations. The season-based model evaluation technique showed superior performance compared with the traditional method of assuming constant model parameters across seasons. The watershed simulation model exhibited reasonable accuracy in simulating streamflow at the internal sites and for the verification periods when parameter values are allowed to vary from dry to wet season. The “optimal” parameter values identified by the calibration attempts showed significant seasonal sensitivity.