Predicting Design Performance Utilizing Automated Topic Discovery

Abstract

Increasingly complex engineering design challenges requires the diversification of knowledge required on design teams. In the context of open innovation, positioning key members within these teams or groups based on their estimated abilities leads to more impactful results since mass collaboration is fundamentally a sociotechnical system. Determining how each individual influences the overall design process requires an understanding of the predicted mapping between their technical competency and performance. This work explores this relationship through the use of predictive models composed of various algorithms. With support of a dataset composed of documents related to the design performance of students working on their capstone design project in combination with textual descriptors representing individual technical aptitudes, correlations are explored as a method to predict overall project development performance. Each technical competency and project is represented as a distribution of topic knowledge to produce the performance metrics, which are referred to as topic competencies, since topic representations increase the ability to decompose and identify human-centric performance measures. Three methods of topic identification and five prediction models are compared based on their prediction accuracy. From this analysis, it is found that representing input variables as topics distributions and the resulting performance as a single indicator while using support vector regression provided the most accurate mapping between ability and performance. With these findings, complex open innovation projects will benefit from increased knowledge of individual ability and how that correlates to their predicted performances.