Complex Quality Assurance of Historical Hourly Surface Airways Meteorological Data

Abstract

A new complex quality assurance (QA) procedure is developed for historical hourly surface airways meteorological data, recently digitized in a U.S. government?sponsored effort that extends the digital period of record back as early as the late 1920s. The procedure builds upon an existing, three-pronged framework and incorporates several checks previously employed for hourly data. New or enhanced checks were also developed, partly with the goal of accommodating the peculiarities of historical observation practices or reporting standards, but also to improve our ability to flag suspicious data. The latter goal is sought by use of more elaborate physics or empirical relationships than are employed in existing checks, or by modifying or replacing them to sharpen their intended focus. Each of the three prongs checks for consistency, whether for internal or temporal consistency or against limits. Six internal consistency and three temporal consistency checks are developed or adapted from recent research and implemented as components in complex QA. These involve temperature, humidity, pressure, wind, present weather, and cloud cover from synoptic reports, as well as station elevation from metadata. In order to determine, from the many component-check flags activated, which element(s) are most likely responsible for the flag(s), a decision tree was developed. Resultant flag levels ?suspect? and ?erroneous? reflect the degree of confidence. Flag rates averaged 9.5% overall but leveled off at about 2% after 1937 (0.3% for ?erroneous? flags), comparing favorably with previously published flag rates. Systematic errors raise many ?suspect? flags, suggesting that hybrid QA?inhomogeneity tests are needed.