A Bayesian Hierarchical Model for Extracting Individuals’ TheoryBased Causal Knowledge

Abstract

Extracting an individual’s scientific knowledge is essential for improving educational assessment and understanding cognitive tasks in engineering activities such as reasoning and decisionmaking. However, knowledge extraction is an almost impossible endeavor if the domain of knowledge and the available observational data are unrestricted. The objective of this paper is to quantify individuals’ theorybased causal knowledge from their responses to given questions. Our approach uses directedacyclic graphs (DAGs) to represent causal knowledge for a given theory and a graphbased logistic model that maps individuals’ questionspecific subgraphs to question responses. We follow a hierarchical Bayesian approach to estimate individuals’ DAGs from observations. The method is illustrated using 205 engineering students’ responses to questions on fatigue analysis in mechanical parts. In our results, we demonstrate how the developed methodology provides estimates of populationlevel DAG and DAGs for individual students. This dual representation is essential for remediation since it allows us to identify parts of a theory that a population or individual struggles with and parts they have already mastered. An addendum of the method is that it enables predictions about individuals’ responses to new questions based on the inferred individualspecific DAGs. The latter has implications for the descriptive modeling of human problemsolving, a critical ingredient in sociotechnical systems modeling.