Learning From Accident Reports Using Language Models: Human Errors and Error Mechanisms in Aviation and Railway Accidents

Abstract

Abstract. New paradigms for human–machine interaction are essential to enabling emerging concepts for aviation operations—from remote pilots to multivehicle operations. With the novelty of these arrangements and the resulting high degree of uncertainty, it is critical to consider human error early in the design process, both to reduce the possibility of costly redesign and to prevent accidents. However, existing methods for human error assessment are highly expert-driven and rely on historical knowledge, which in early design and particularly for novel systems is limited. To address this gap, we propose a methodology that uses a language model (i.e., bidirectional encoder representations from transformer (BERT)) to assist with expert-driven identification of human errors, error-producing conditions, and mechanisms from historical incident reports. Moreover, we hypothesize that it is possible to learn across domains to support early design consideration of human elements for novel systems which may not have in-domain data. In particular, we demonstrate the proposed methodology by identifying human errors in aviation and railway domains. The proposed approach yields summarized, tailored reports on human errors from past incidents: nine from railway and 14 from aviation reports. Each human error has at least one error-producing condition, and the majority have more than one error mechanism. Three of the human errors were common to both domains, indicating a degree of knowledge transfer is possible. Additionally, findings indicate that a majority of the errors, error-producing conditions, and mechanisms can be used to inform safe operations across domains—even if they were not found to be common in both datasets—as long as engineering judgment is used to interpret them in context.