Assessment of Student Learning Through Reflection on Doing Using the Latent Dirichlet Algorithm

Abstract

Can we provide evidencebased guidance to instructors to improve the delivery of the course based on students’ reflection on doing? Over three years at the University of Oklahoma, Norman, USA, we have collected about 18,000 Takeaways from almost 400 students who participated in an undergraduate design, build, and test course. In this paper, we illustrate the efficacy of using the Latent Dirichlet Algorithm to respond to the question posed above. We describe a method to analyze the Takeaways using a Latent Dirichlet Allocation (LDA) algorithm to extract topics from the Takeaway data and then relate the extracted topics to instructors’ expectations using text similarity. The advantage of the LDA algorithm is anchored in that it provides a means for summarizing large amount of takeaway data into several key topics so that instructors can eliminate the laborintensive evaluation of it. By connecting and comparing what students learned (embodied in Takeaways) and what instructors expected the students to learn (embodied in stated Principles of Engineering Design), we provide evidencebased guidance to instructors on how to improve the delivery of AME4163: Principles of Engineering Design. Our objective in this paper is to introduce a method for quantifying text data to facilitate an instructor to modify the content and delivery of the next version of the course. The proposed method can be extended to other courses patterned after AME4163 to generate similar data sets covering student learning and instructor expectations, and the LDA algorithm can be used for dealing with the large amount of textual data embodied in students’ Takeaways.