Testing, Verification, and Validation of Expert Systems

Abstract

This paper discusses how an expert system can be tested, verified, and validated. A prototype expert system, the Hazardous Location Analyst (HLA), which analyzes high‐accident locations, is used to illustrate the evaluation process and to examine the various aspects of testing, verification, and validation. Identified were several key issues that must be addressed when evaluating an expert system: problem complexity, no “correct answers,” the problem of defining expert‐level performance, and considerations in knowledge‐base refinement. The paper also suggests ways in which to make comparisons between expert‐system and human‐expert inferences so that ease of use, knowledge‐base adequacy, and completeness may be examined. The paper further suggests that although some quantitative performance measures are useful for the evaluation process, a qualitative evaluation is perhaps also appropriate for many aspects of these kinds of systems.