Scale Development Process Research Articles

The Problem of Scale Contamination and Construct Validity in Mental Health, Social Science and Spirituality-Related Research

This article explores the pervasive issue of scale contamination and its impact on construct validity in mental health, social science, and spirituality-related research. Scale contamination occurs when measurement tools designed to assess one specific domain inadvertently incorporate elements from unrelated constructs, leading to confusion regarding what the scale is actually measuring. The blending of distinct psychological domains, such as depression, anxiety and well-being, undermines measurement precision, leading to skewed research conclusions and misguided clinical practices. The failure to detect or address contamination can perpetuate errors in treatment efficacy, particularly in areas such as mental health and spirituality, where precise measurement is crucial. This article emphasizes the importance of ensuring construct validity, which refers to the accurate measurement of specific concepts such as moral injury or posttraumatic stress disorder, and it highlights the risk of conflating distinct constructs. The widespread use of contaminated scales, often adopted without critical evaluation, poses a significant threat to the integrity of research outcomes. In order to address these challenges, this study advocates for rigorous scale development and validation processes, encouraging researchers, peer reviewers and journal editors to scrutinize the psychometric properties of scales. By ensuring that scales measure only the intended constructs, future research can produce more accurate and meaningful results, leading to improved clinical interventions and policy decisions.

Human-AI Synergy in Survey Development: Implications from Large Language Models in Business and Research

This study examines the novel integration of Large Language Models (LLMs) into the survey development process in business and research through the development and evaluation of the Behavioral Research ASSistant (BRASS) Bot. We first analyzed the traditional scale development process to identify tasks suitable for LLM integration, including both human-in-the-loop and automated LLM data collection methods. Following this analysis, we developed the details of BRASS Bot, incorporating design principles of falsifiability and reproducibility. We then conducted a comprehensive evaluation of the BRASS Bot across a diverse set of LLMs, including GPT, Claude, Gemini, and Llama, to assess its usability, validity, and reliability. We further demonstrated the practical utility of the BRASS Bot by conducting a user study and a predictive validity simulation. Our research presents both theoretical and practical implications. The augmentation approach of the BRASS Bot enriches the theoretical foundations of behavioral constructs by identifying previously overlooked patterns. Additionally, the BRASS Bot offers significant time and resource efficiency gains while enhancing scale validity. Our work lays the foundation for future research on the broader application of LLMs as both assistants and collaborators in survey analysis and behavioral research design and execution, highlighting their potential for a transformative impact on the field.

Vaccine hesitancy as indecision: Creation and evaluation of the Unidimensional Vaccine Hesitancy Scale.

Several authors have argued that vaccine hesitancy should be conceptualized as indecision in the vaccination decision-making process, but no established measure with support for its psychometric properties and validity has been created from this operational definition. To resolve this tension, this article undergoes a four-study scale development process to create the 4-item Unidimensional Vaccine Hesitancy Scale (UVHS). We conduct four survey studies utilizing a total sample size of 884. In Studies 1 (n = 297) and 2 (n = 298), we provide psychometric support for the measure via exploratory and confirmatory factor analysis. In Studies 3 (n = 193) and 4 (n = 106), we support the concurrent and discriminant validity of the measure by assessing its relations with relevant constructs, such as vaccination readiness and acceptance, and we also provide initial indicators of the scale's possible predictive qualities by testing its time-separated effects with vaccination willingness, receipt and word-of-mouth. We leverage these results to provide a number of theoretical insights and suggestions for future practice. Of note, we highlight that different conceptualizations and operationalizations for the same construct can produce notably differing empirical findings, and vaccine hesitancy is no different. Our cumulative efforts indicate that the UVHS is an appropriate measure to assess vaccine hesitancy as indecision.

Understanding emotional well-being of teachers: Development and validation of Teacher Emotions in Teaching Scale in Hong Kong using a mixed-methods approach

In view of lacking in a holistic, indigenous measure to the affective well-being of teachers in Hong Kong, this study utilizes a mixed-methods approach to develop and validate the Teacher Emotions in Teaching Scale (TETS) which assesses positive and negative emotions teachers experience in teaching. The process of scale development and construct validation involving three samples of 726 in-service teachers from Hong Kong was reported. Results of this study supported the five-factor model of the TETS and its convergent, discriminant and criterion validity. This study highlights the use of the TETS to understand and promote emotional well-being of teachers.

Development and validation of the infodemic scale

ABSTRACT Infodemic is closely related to all healthcare professionals. Nurses take the lead in health professions that spend a long time with the society and try to present accurate information due to their roles. Infodemic is the massive spread of false or unreliable information in times of crisis. This study aims to develop a valid and reliable tool that identifies individuals’ approaches to infodemic. This study was methodological. The sample consisted of 504 people (N = 504). Data were collected using the Information Form and the Infodemic Scale. The sample was divided into two for the explanatory factor analysis (EFA) and the confirmatory factor analysis (CFA). The scale was developed using the scale development process in health and social research, which consists of three steps. The EFA revealed 14 items under two factors, which are awareness and literacy. Item factor loadings were found to range between 0.334 and 0.907. The CFA confirmed the two-factor scale structure. The total Cronbach’s alpha of the scale was 0.87, and the composite reliability value was 0.89. The intra-class correlation coefficient (ICC) value was found to be 0.857. According to psychometric evaluations, the Infodemic Scale is a valid and reliable instrument that can make consistent measurements.

Measuring the Social Impact of Social Enterprises–Scale Development and Validation

Social impact measurement has remained a challenge for both social enterprises (SEs) and scholars in the SE field. Despite increased scholarly attention given to SE social impact and its measurement in recent years, scholars continue to call for more research to advance the approaches for measuring social impact in SEs. This study aims to provide a reliable and valid social impact scale for SEs, which can be used as an alternative or a complementary scale to existing quantitative social impact tools. Through a systematic scale development process, it develops a 31-item multidimensional social impact scale based on the triple bottom line sustainability framework and using perception-based or constituency-based feedback. Data analysis confirms the reliability and validity of the scale across two studies involving different SE samples and country contexts. The study also discusses implications for theory and practice and recommendations for future research.

Destination avoidance and scale development: A mixed-methods approach

This research employs an exploratory sequential mixed-methods design in three successive studies to investigate why tourists avoid certain destinations. Initially, a qualitative approach (Study 1) using phenomenology was adopted through semistructured interviews. Preliminary results identify that the destination avoidance phenomenon is pertinent to seven themes: poor tourism product, poor image and safety, lack of authenticity and novelty, high cost, poor accessibility, poor host community, and adverse situations. Subsequently, a quantitative approach (Studies 2 and 3) using a survey method was employed to develop a destination avoidance scale, which quantifies findings from Study 1. After undergoing a rigorous scale development process, a 14-item multidimensional scale to measure destination avoidance is developed. The contribution of this research lies in clarifying rationales behind why tourists reject specific destinations, and in developing a multidimensional scale, thereby enriching the literature on destination avoidance. It also informs destination operators with potential strategies to effectively address issues of destination avoidance.

A scale to assess organizational resilience for the oil and gas industry: testing and future perspectives

PurposeAs part of the scale development process, this paper aims to test a scale to measure organizational resilience for the oil and gas industry. The objective is to provide stakeholders with a set of indicators to evaluate their organizations and prepare them to cope with the negative consequences of disruptions (e.g. Covid-19, shortage of resources, etc.).Design/methodology/approachThe paper conducts exploratory and confirmatory factor analysis to test the suitability, dimensionality and reliability of specific indicators and their items under examination. Therefore, the goal is not to validate hypotheses by testing an organizational resilience scale in the oil and gas industry.FindingsThe study tests and proposes a scale to effectively measure organizational resilience within the oil and gas industry. A comprehensive set of ten indicators and 40 items are identified through this process. The findings of this research provide stakeholders in this sector with a rigorous set of indicators to evaluate the strengths and weaknesses of their organizations and better prepare them to handle disruptions.Originality/valueThis paper fills the gap in existing research by testing and proposing a scale to measure organizational resilience specifically for the oil and gas industry. It highlights the importance of organizational resilience for survival in a sector that is especially susceptible to disruptions.

Development of a scale for measuring psychosocial vulnerability in disaster-prone areas

IntroductionDisaster vulnerability is a complex concept influenced by both natural forces and societal factors, including psychosocial aspects that shape an individual's resilience to natural hazards. Despite technological advancements, assessing psychosocial vulnerability remains challenging, necessitating specialized scales. This study attempted to develop a psychosocial vulnerability assessment scale, recognizing the intricate interplay of physical, psychological, and social factors. MethodsThe study was conducted in two disaster-prone wards in Cuttack, India, susceptible to cyclones, earthquakes, and floods. The scale development process involved two stages: instrument design and judgmental evidence. During the instrument design phase, qualitative data on psychosocial vulnerabilities were gathered through Key Informant Interviews (KIIs) and Focus Group Discussions (FGDs). These insights led to the generation of 44 items, which were subsequently refined to 35 items through expert consultation. The judgmental evidence phase included content validity assessment, internal consistency analysis, and test-retest reliability estimation. Exploratory Factor Analysis (EFA) identified the instrument's factor structure, with items retaining factor loadings exceeding 0.4. Data collection involved 400 individuals selected through multistage sampling, with strict adherence to ethical considerations. ResultData collected in the initial instrument design stage revealed key psychosocial vulnerability themes. These themes informed the creation of a 35-item scale categorized into three domains: Physical, social, and psychological vulnerability. The instrument underwent rigorous validation processes, demonstrating strong content validity agreement among experts. Internal consistency analysis revealed good reliability, as indicated by the Kuder-Richardson coefficient (KR20 = 0.904). Test-retest reliability further established the stability of responses over time, with an ICC of 0.82. EFA confirmed the scale's three-factor structure, explaining 60 % of the variance. After removing three items with low factor loadings, the scale proved reliable and valid, effectively categorizing vulnerabilities into these distinct domains. ConclusionThe scale developed in this study offers a comprehensive tool for understanding the unique challenges people living in disaster-prone areas face. It addresses the often-overlooked psychosocial components of vulnerability, filling a critical gap in existing research and measurement scales. Its application helps in planning psychosocial interventions and improving disaster risk reduction strategies.

Conspicuous compensatory consumption: scale development and initial validation

PurposeThis study aims to develop and validate a measure for conspicuous compensatory consumption. Compensatory consumption phenomenon is gaining increased significance in consumer behavior literature. In a symbolic-rich culture, the use of possessions creates a perfect venue for self-construction and self-repairing to make up for one’s psychological deficiencies and inadequacies.Design/methodology/approachA mixed research design of qualitative and quantitative methodologies is adopted by using elicitation techniques, interviews and survey data. Extensive development and validation procedures are used. A series of studies, encompassing a total sample of 1,782, are reported.FindingsThe current study offers a valid and reliable measure for conspicuous compensatory consumption by chronologically following the stages of the scale development process. Compensatory consumption had a negative influence on subjective happiness and a positive influence on negative affect and satisfaction with life. Respondents with high materialism scores had significantly higher compensatory tendencies than the low materialism group. The low self-compassionate group had significantly higher compensatory tendencies than the high self-compassionate group.Originality/valueThe current study provides theoretical contributions to consumer behavior research by providing a valid and reliable measure for conspicuous compensatory consumption. Contrary to past scales that followed a mood-alleviation perspective where therapeutic shopping is used to regulate negative emotions, this scale is novel in adopting a self-completion approach where products are pursued for a tactical effort to offset threatened self-concepts.

Measuring Social Frailty: A Scoping Review of Available Scales and Tools.

Social frailty is an emerging concept characterized by state of vulnerability due to the lack or absence of social resources that enable health and well-being, particularly among older people. However, there is no consensus on how to accurately measure and assess social frailty, given the broad coverage of social dimensions affecting older individuals. This scoping review aimed to identify the existing tools and scales used to measure social frailty in older people. A scoping review methodology was employed. Articles published between January 2014 and April 2024 were searched in 6 electronic databases: PubMed, PsycINFO, ProQuest, Scopus, SocIndex, and CINAHL. The scoping review followed a 5-stage process by Arksey and O'Malley and adhered to the guidelines provided by Preferred Reporting Items for Systematic Reviews and Meta-Analysis extension for Scoping Reviews. Nine social frailty tools were identified across the 58 papers included in this review. The individual question item commonly used in the scales were classified into main categories based on their conceptual characteristics and intentions. The most common individual questionnaire constructs used to measure social frailty included financial status, social resources, social behaviors and activities, and sense of purpose. The reviewed tools varied in terms of their robustness and the process of scale development. Our review suggested the development of a standardized, psychometrically tested, and accurate screening tool to screen social frailty status. An accurate social frailty assessment can inform the development of useful interventions, which also has implications in preventing the development of physical frailty.

The Dark Triad of brand personality: Scale development and validation

AbstractDespite the considerable magnitude of negative brand personalities in the marketplace, prior research has disproportionately focused on positive personality traits. To address this research gap, the authors present a conceptual and empirical approach that draws from the Dark Triad of psychology and applies it to the branding domain. They conceptualize and validate the Dark Triad of brand personality which comprises brand narcissism, Machiavellianism, and psychopathy dimensions. Through a multiphase scale development process, a reliable and valid 12‐item brief version of the Dark Triad of brand personality is created, enabling its assessment in both research and management contexts. Examining the Dark Triad in branding is crucial as it provides a unique lens to understand the rise of negative, dark brand personalities. It accounts for brand personality aspects not yet captured by existing scales, including manipulation, exploitation, grandiosity, and lack of empathy. This introduction of the Dark Triad brand personality opens new avenues for research into brand transgressions and ethics in brand management. In terms of managerial implications, insights from this research can inform strategic brand management, enabling companies to better manage their brand's image.

Quiet Quitting reason and behavior scale design and development process.

The concept of "Quiet Quitting", which might be considered new in the literature, is used to describe a passive and silent giving up. The aim of this study is to develop a scale to evaluate both the causes and consequences of "quiet quitting" which is believed to be increasing for various reasons in our country as well as in the world. Five-stage scale development method which was presented by Cohen and Swerdik was used in the process of developing the Quiet Quitting Reason and Behavior Scale (QQRBS). First, the conceptual structure of the scale was determined. In the second stage, the scale was structured. Structuring the scale is deciding on the scale type and scaling technique. The third stage is the first application of the scale. After the scale was applied, factor analysis, internal consistency assessment and validity studies were carried out in the item analysis section, which is the fourth stage. The fifth stage is the second application stage of the scale and at this stage, the scale was reviewed by test-retest and an application guideline was prepared. Cronbach's alpha value of draft items is 0.87, quiet quitting behavior sub-dimension Cronbach's Alpha value is 0.76 and reasons for quiet quitting Cronbach's Alpha value is 0.820. It was decided to keep the remaining 35 items after EFA analysis because the corrected correlation numbers of the items were >0.30. This study explains the design and development process of the QQRBS in Turkish society. According to this developed scale, as the scale mean score of the total and sub-dimensions increases, the tendency to quiet quitting increases.

A Scale Development Study on the Perceived Level of Workplace Generational Conflict

Objective: Intergenerational conflict can be defined as disagreements in values, beliefs, perspectives and behaviors of different generations. Considering today’s business environment where different generations work together, its importance emerges once again. Methods: Although intergenerational conflict has been discussed from many perspectives in the literature, studies on this type of conflict, especially in the business environment, are limited and even a scale that directly measures this conflict has not been found. Therefore, a scale was developed through a systematic scale development process based on the literature related to this study. Results: The data was collected from 1005 private sector professionals. Consequently, exploratory and confirmatory factor analysis and reliability analysis were performed and the factor structure of the scale was determined. Conclusions: As a result, the two-dimensional perceived level of workplace generational conflict scale developed and the scaleitems presented in the appendix was obtained. Originality/Value: Intergenerational conflict involves differing values, beliefs, perspectives, and behaviors among generations. In today's multi-generational workplace, its significance is clear. Consequently, a two-dimensional scale to measure workplace generational conflict has been developed, with scale items listed in the appendix.

Medical tourism destination image: Scale development

AbstractIn response to heightened competition among destinations vying for medical tourists, these destinations have prioritized improving their image. Although several scales exist to assess destination images, there is a scarcity of scales specifically tailored to measure the image of medical tourism destinations. To address this gap, a thorough scale development process was employed, resulting in the creation of a second‐order model known as the medical tourism destination image scale. This scale explores four dimensions: quality of doctors and hospitals, ease of travel and safety, variety of treatment, and communication. The findings underscore the significant potential for marketers and policymakers to enhance the attractiveness of medical tourism destinations.

Development and Assessment of a Post-Occupancy Evaluation Scale for Sustainable Office Environments

This study explores the relationship between users and the built environment through a post-occupancy evaluation (POE) conducted at the FNN Sustainability Centre, a noteworthy sustainable building in the region. The study involved a comprehensive approach, encompassing site visits, managerial interviews, and staff surveys. To establish a robust evaluation framework, a scale was developed by analyzing pertinent literature, and indicators were identified to gauge various aspects of the building's performance. Throughout the scale development process, the SPSS data analysis program was used, and expert opinions were solicited to ensure a rigorous and comprehensive methodology. Evaluation categories included lighting, acoustics, climatic comfort and indoor air quality, use and comfort of systems, quality of space and perception, awareness of sustainability and productivity. The building emerged as a physically and psychologically conducive workplace that heightened employee awareness of sustainability. The study identified specific concerns, such as noise disturbance for open-office workers and glare-related issues, which serve as valuable feedback for potential adjustments.

Leadership for quality 4.0: development and validation of a multidimensional measure

Successful implementation of Quality 4.0 (Q4.0) requires leadership, but the understanding of such leadership among scholars and practitioners is insufficient. The purpose of this paper is to explore a leadership scale to provide insights. This study develops the scale via 3 independent samples of quality personnel according to Hinkin's recommended scale development process. Exploratory factor analysis sheds light on the tri-dimensional structure of the scale, which comprises task-oriented, change-oriented, and relations-oriented behaviours. The results are affirmed by confirmatory factor analysis, which also demonstrates satisfactory reliability and construct validity. To further explore the predictive validity of leadership and delve deeper into its mechanics, based on the Job Demands-Resources (JD-R) model, this study identifies connectivity and challenge stressors as keys to the ‘Black Box’ of the relationship between leadership and thriving at work. Structural equation modelling suggests that leadership has a positive influence on thriving at work, connectivity, and challenge stressors. Connectivity and challenge stressors mediate the relationship between leadership and thriving at work. The Q4.0 leadership scale can be used as a guideline for organisations embracing Q4.0 to select and appoint managers, as well as for quality leaders to evaluate and improve themselves.

Firm engagement: Scale development and verification

Stakeholder or actor (e.g., firms, employees, and customers) engagement is critical for value co-creation in the service ecosystem. As a provider of an engagement platform in a service ecosystem, the firm plays a critical role in employee and customer engagement. While customer and employee engagement have received much attention, firm engagement research remains unexplored, nonetheless, is critical for the business development and sustainability. There is no scale that has been developed to measure firm engagement. Development of this scale would provide a holistic picture of engagement from a multi-perspective. Following the conventional scale development process, this study developed and verified a scale to measure firm engagement. First, in-depth interviews with 23 managers of companies in different industries were conducted to establish initial items. Subsequently, item purification (N = 397) and validation (N = 398) were performed. The results show that firm engagement consists of eight dimensions from customer and employee perspectives. Specifically, customer engagement orientation includes empathy, employee participation, customer participation, and resource integration while employee engagement orientation includes management participation, corporate culture, interest alignment, and internal service quality. The scale contributes to the engagement literature and human resource management research and has practical implications for firms to improve employee performance and customer experience.

Pedestrians’ Urban Space Preference Scale: A Study on Reliability and Validity

ABSTRACT This study develops a valid and reliable scale for the spatial preferences of pedestrians in public spaces. The study was conducted in 2023 and includes the development of a model based on survey results. With the help of random sampling method, the research was carried out with 401 random participants who were currently walking in the urban space. The scale development process started with an initial pool of 34 items determined from the literature on the subject. The data obtained from the applied scale were subjected to total item correlation, exploratory and confirmatory factor analyses and reliability analyses. According to the results of the analyses, it was found that the scale consisted of 19 items and 5 factors, the variance rate explaining the whole scale was 62.8%, and the fit index values were acceptable. Cronbach alpha internal consistency reliability coefficient was calculated as 0.813.

Development and psychometric evaluation of the adjuvant endocrine therapy beliefs scale for breast cancer survivors.

Adjuvant endocrine therapy has a vital role in reducing breast cancer mortality. The beliefs in adjuvant endocrine therapy is a very important factor in the medication adherence of breast cancer survivors. Therefore, it is necessary to develop a standardized scale for assessment of adjuvant endocrine therapy. The purpose of this study was to identify the attributes of adjuvant endocrine therapy beliefs, and to evaluate adjuvant endocrine therapy beliefs scale psychometric properties. A hybrid model was applied to identify the concept of adjuvant endocrine therapy beliefs and measurement question were developed by the scale development process. Statistical analysis using validity analysis and Rasch analysis based on item response theory were performed. A total of 228 breast cancer survivors in South Korea participated in the study. The finally developed adjuvant endocrine therapy beliefs scale consisted of 22 items. The items extracted by 4 factors explained 59.72% of the total variance. The model fit showed an acceptable level. The adjuvant endocrine therapy beliefs scale was excellent in convergent and discriminant validity with reliability. This scale is expected to be practical and useful in identifying adjuvant endocrine therapy beliefs and developing intervention strategies to promote adjuvant endocrine therapy adherence. In addition, continuous education and support should be accompanied so that breast cancer survivors can maintain positive beliefs in adjuvant endocrine therapy adherence.