Abstract
Despite their suitability for mitigating survey biases and their potential for enhancing information richness, open and semi-open job satisfaction questions are rarely used in surveys. This is mostly due to the high costs associated with manual coding and difficulties that arise when validating text measures. Recently, advances in computer-aided text analysis have enabled researchers to rely less on manual coding to construct text measures. Yet, little is known about the validity of text measures generated by computer-aided text analysis software and only a handful of studies have attempted to demonstrate their added value. In light of this gap, drawing on a sample of 395 employees, we showed that the responses to a semi-open job satisfaction question can reliably and conveniently be converted into a text measure using two types of computer-aided sentiment analysis: SentimentR, and Linguistic Inquiry and Word Count (LIWC) 2015. Furthermore, the substantial convergence between the LIWC2015 and, in particular, SentimentR measure with a closed question measure of job satisfaction and logical associations with closed question measures of constructs that fall within and outside job satisfaction’s nomological network, suggest that a semi-open question has adequate convergent and discriminant validity. Finally, we illustrated that the responses to our semi-open question can be used to fine-tune the computer-aided sentiment analysis dictionaries and unravel antecedents of job satisfaction.
Highlights
Job satisfaction, “a positive evaluative judgment one makes about one’s job or job situation” (Weiss 2002, p. 175), continues to be closely monitored in corporate surveys (Macey and Schneider 2008) and has a long history of scientific study (Judge et al 2017)
The substantial convergence between the LIWC2015 and, in particular, SentimentR measure with a closed question measure of job satisfaction and logical associations with closed question measures of constructs that fall within and outside job satisfaction’s nomological network, suggest that a semi-open question has adequate convergent and discriminant validity
Concerning discriminant validity, we found that the text measures had logical associations with closed question measures of constructs that fall within and outside job satisfaction’s nomological network
Summary
“a positive (or negative) evaluative judgment one makes about one’s job or job situation” (Weiss 2002, p. 175), continues to be closely monitored in corporate surveys (Macey and Schneider 2008) and has a long history of scientific study (Judge et al 2017). Over the past few decades of scholarly and practitioner attention, a plethora of survey question instruments have been developed to cover conceptual nuances, e.g., affect-oriented vs cognition-oriented scales (Kaplan et al 2009; Organ and Near 1985) and job facet satisfaction vs general job satisfaction (Scarpello and Campbell 1983; Spector 1997; Weiss 2002). The majority of survey instruments share one commonality: they are typically comprised of closed questions or items Closed questions have several advantages over open questions, as they typically pose less burden for respondents than open questions (Krosnick 1999; Vinten 1995; Zehner et al 2016) and they prove to be more straight-forward to code and validate than open questions are (Maxwell and Delaney 1985; Tausczik and Pennebaker 2010)
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
