Assessing Learning of Computer Programing Skills in the Age of Generative Artificial Intelligence

Abstract

Generative artificial intelligence (AI) tools such as ChatGPT, Bard, and Claude have recently become a concern in the delivery of engineering education. For courses focused on computer coding, such tools are capable for creating working computer code across a range of computer languages and computing platforms. In a course for mechanical engineers focused on C++ coding for the Arduino microcontroller and coding engineering problems in Matlab, a new approach to assessment of programing homework assignments was developed. This assessment moved the focus of assigned points from the correctness of the code to the effort and understanding of the code demonstrated by the student during in-person grading. Students who participated fully in in-person grading did significantly better on a midterm exam. Relative to a previous semester, where grading was focused on correct code, students had a slightly higher average midterm exam score. This approach appears to be effective in supporting computational learning in the face of evolving tools that could be used to circumvent learning. Future work should examine how to also encourage responsible use of generative AI in computational learning.