A semantic analysis approach for assessing professionalism using free-form text entered online

Abstract

This paper presents the results of an experimental study to measure professionalism for the purpose of assessing a professional development program. Soft skills such as professionalism are increasingly recognized as important, yet measuring and assessing these skills, typically best acquired experientially, has remained challenging. Following established research, we conceptualize professionalism as a construct with the dimensions of autonomy, commitment, belief in public service, self-regulation, and the use of a professional organization as a major referent. We demonstrate how these dimensions can be expressed and the professionalism of free-form text responses measured. These responses are reflections submitted online by participants in a professional development program for undergraduate business majors known as the Management Achievement Program (MAP). Latent semantic analysis is employed to measure the professionalism of these responses and to assess MAP along each of the five dimensions. The method demonstrated in this paper has several advantages over existing methods for assessment, which can be costly, require considerable time and training, and are often tied to subjective interpretation. The method demonstrated here is suitable for replication that leads to continuous improvement by “closing the loop.”