Assessing Student Learning of Systems Thinking Concepts in an Online Education Module

Abstract

Institutions of higher education are vast interconnected networks of departments, programs, majors, and courses, whose complexity is only increased by the rapid growth and availability of technology-based learning in recent years. In today’s data-driven world, it is critical for college students in all academic disciplines to understand the basic concepts of “systems thinking” and how systems thinking strategies can be applied to nearly any problem they encounter in their careers. While making this information available to students online is an easy way to disseminate the content within the complex network of higher education, the decision to do so may be at the expense of students’ understanding of the material. Therefore, this paper aims to assess the effectiveness of an online module in introducing systems thinking concepts to both engineers and non-engineers. To conduct the study, a gap analysis was performed among existing online education platforms, resulting in the selection of Thinkific as the most effective massive open online course (MOOC) platform through which to disseminate our online module content. Thinkific has open access and allows for interactive participation through the Internet. A short online module was developed and validated in Thinkific using human design principles and user testing. Upon completion of the module design, groups of students in the College of Arts and Sciences (CLAS) and in the School of Engineering and Applied Science (SEAS) at the University of Virginia completed a pre-test, the online module, and a post-test. The qualitative and quantitative results of the pre- and post-tests were analyzed to determine the effectiveness of the module and to learn how understanding varies by intended major. Through this analysis, two findings were elicited: online learning increases learning and understanding concerning key systems thinking concepts, and this learning and understanding is not significantly different between CLAS and SEAS participants. These results inform educators about the degree of emphasis that should be placed on continued development and scalability of online learning programs to enhance understanding of systems thinking concepts. More broadly, this study contributes to the growing body of literature which seeks to understand the impact of technology on the spread of information not only within the field of higher education, but within other large systems as well.