Designing Representative Model Worlds to Study Socio-Technical Phenomena: A Case Study of Communication Patterns in Engineering Systems Design

Abstract

The engineering of complex systems, such as aircraft and spacecraft, involves large number of individuals within multiple organizations spanning multiple years. Since it is challenging to perform empirical studies directly on real organizations at scale, some researchers in systems engineering and design have begun relying on abstracted model worlds that aim to be representative of the reference socio-technical system, but only preserve some aspects of it. However, there is a lack of corresponding knowledge on how to design representative model worlds for socio-technical research. Our objective is to create such knowledge through a reflective case study of the development of a model world. This “inner” study examines how two factors influence interdisciplinary communication during a concurrent design process. The reference real world system is a mission design laboratory (MDL) at NASA, and the model world is a simplified engine design problem in an undergraduate classroom environment. Our analysis focuses on the thought process followed, the key model world design decisions made, and a critical assessment of the extent to which communication phenomena in the model world (engine experiment) are representative of the real world (NASA’s MDL). We find that the engine experiment preserves some but not all of the communication patterns of interest, and we present case-specific lessons learned for achieving and increasing representativeness in this type of study. More generally, we find that representativeness depends not on matching subjects, tasks, and context separately, but rather on the behavior that emerges from the interplay of these three dimensions.