Psychometric Investigation Research Articles

Tale of the tape: psychometric investigation of two popular teacher self-efficacy scales

ABSTRACT This study compares the psychometric properties of two popular teacher self-efficacy scales, namely, Teachers’ Sense of Efficacy Scale (TSES) and Teacher Efficacy Scale (TES). This study was quantitative in nature and distributed the two scales to 287 preservice teachers in the USA and in Australia. Findings indicate that the two scales function well and show linear measurement properties after Rasch optimization. However, the TES’s factor of general teacher efficacy was found to have very low reliability. Regression analyses indicated a significant positive relationship between the Student Engagement subscale of TSES and TES. There is a limited relationship between the two popular scales that have been used for decades. Therefore, we argue for the need for a new scale that will reflect all components of Bandura’s Triadic Reciprocal Determinism.

When Does Episodic Memory Contribute to Performance in Tests of Working Memory?

Both the experimental and the psychometric investigation of the WM capacity limit depend critically on the assumption that performance in our tests of WM reflects that capacity limit to a good approximation. Most tasks to measure WM rely on testing memory after a short time during which participants are asked to maintain information in WM. In these tests, episodic long-term memory is likely to also lay down a trace of the memory set. Therefore, participants can draw on two sources of information when memory is tested, making it difficult to separate the contributions of WM and episodic LTM to the performance on immediate-memory tests. Here we use proactive interference to distinguish between these two sources of remembered information, building on the fact that episodic memory is vulnerable to proactive interference, whereas WM is protected against it. We use a release-from-PI paradigm to determine the extent to which commonly used WM tasks reflect contributions from episodic LTM. We focus on memory for serial order of verbal lists, but also include visual and spatial WM tasks. The results of five experiments demonstrate that although some tasks used to investigate WM are heavily contaminated by episodic LTM, other popular paradigms such as serial and probed recall, and the standard version of the continuous color-reproduction task, are not. Measuring proactive interference can help researchers determine the extent to which WM and episodic LTM contribute to performance in immediate-memory tasks.

How are mental representations of asthma triggers and symptoms related to interpersonal risk perceptions? A psychometric investigation of caregivers of children with asthma

Objective Parents and guardians (hereafter caregivers) make decisions for their children’s medical care. However, many caregivers of children with asthma struggle to understand their child’s illness. We used the psychometric paradigm to investigate how caregivers conceptualize, or mentally represent, asthma triggers and symptoms and how these representations are linked to perceived asthma exacerbation risk. Methods We asked 377 caregivers of children with asthma across the U.S. to rate 20 triggers or 20 symptoms along 15 characteristics. Caregivers also indicated their perceived risk of their child having an asthma exacerbation (hereafter interpersonal risk perceptions). Using principal components analysis, we extracted key dimensions underlying caregivers’ ratings on the characteristics. Then we related the triggers’ and symptoms’ scores on the dimensions to caregivers’ interpersonal risk perceptions. Results Interpersonal risk perceptions were higher for triggers with high ratings for the dimensions severe and relevant, and negative affect—yet manageable, but not chronic—yet unpredictable. Risk perceptions were also higher for symptoms with high ratings for the dimensions severe and unpredictable, and relevant and common, but not self-blame or manageable despite unknown cause. Conclusion By identifying key dimensions underlying caregivers’ mental representations of asthma triggers and symptoms, these findings can inform a new approach to asthma education.

Cross-cultural adaptation and psychometric investigation of the palatable eating motives scale (PEMS) for a sample of Brazilian adults

Cross-cultural adaptation and psychometric investigation of the palatable eating motives scale (PEMS) for a sample of Brazilian adults

Measuring Family Support in Australia, Brazil, Hong Kong, and Turkey: A Psychometric Investigation

This study presents a reduced version of the Perception of Family Support Scale (PFSS)—a measure of family support designed for research and clinical purposes—and examine construct validity, invariance, and reliability evidence for scores obtained from the instrument. Data collection occurred in 2019. Using exploratory and confirmatory factor analyses with college student samples from Australia, Brazil, Hong Kong, and Turkey (N = 1,256), we identified an oblique three-factor model corresponding to the original scale’s internal structure (e.g. affective consistency, family maladaptation, and family autonomy). We also found configural, metric, and scalar invariance, good reliability coefficients, and significant relationships with distress (e.g. depression, anxiety, and stress) across the four studied countries. These results indicate good validity evidence for using the brief version of the PFSS. Applications of the PFSS and implications of the current results are discussed.

A systematic literature review of exploratory factor analyses in management

We review the application and reporting of exploratory factor analysis (EFA) in management. First, we integrate recommendations from relevant reviews and simulation studies to provide modern guidelines regarding EFA for psychometric investigation, and we highlight severe concerns associated with EFA for Harman’s one-factor test. Second, we conduct a systematic literature review including 1,790 articles and 3,396 EFAs from 23 premier journals since the year 2000. Our review reveals that many widespread guidelines for EFA are infrequently applied. We also show that researchers regularly fail to report essential aspects of their EFAs, leaving it unknown whether they are correctly conducting their analyses. Third, our discussion reinforces modern recommendations that are infrequently applied, exposes researchers to emerging developments, and provides new factor loading interpretations based on our systematic review results. To conclude, we provide step-by-step visual guides and a checklist to aid future researchers on their application and reporting of EFA.

Climate for evidence-based mental health care implementation in Germany: psychometric investigation of the Implementation Climate Scale (ICS)

Organizational implementation climate is an important construct in implementation research to describe to what extent implementation is expected, supported, and rewarded. Efforts in bridging the research-practice gap by implementing evidence-based practice (EBP) can benefit from consideration of implementation climate. The Implementation Climate Scale (ICS) is a psychometrically strong measure assessing employees’ perceptions of the implementation climate. The present cross-sectional study aimed at providing a German translation and investigating its psychometric properties. The translation followed standard procedures for adapting psychometric instruments. German psychotherapists (N = 425) recruited online completed the ICS, the Evidence Based Practice Attitudes Scale (EBPAS-36D) and the Intention Scale for Providers (ISP). We conducted standard item and reliability analyses. Factorial validity was assessed by comparing an independent cluster model of Confirmatory Factorial Analysis (ICM-CFA), a Bifactor CFA, a Second-order CFA and an (Bifactor) Exploratory Structural Equation Model (ESEM). Measurement invariance was tested using multiple-group CFA and ESEM, convergent validity with correlation analysis between the ICS and the ISP subjective norms subscale (ISP-D-SN). The mean item difficulty was pi = .47, mean inter-item correlation r = .34, and mean item-total correlation ritc = .55. The total scale (ω = 0.91) and the subscales (ω = .79–.92) showed acceptable to high internal consistencies. The model fit indices were comparable and acceptable (Second-order CFA: RMSEA [90% CI] = .077 [.069; .085], SRMR = .078, CFI = .93). Multiple-group CFA and ESEM indicated scalar measurement invariance across gender and presence of a psychotherapy license. Psychotherapists in training reported higher educational support for EBP than licensed psychotherapists (T = 2.09, p = .037, d = 0.25). The expected high correlation between the ICS and the ISP-D-SN was found (r = .59, p < .001). Results for the German ICS confirm good psychometric properties including validity.

Taking Inventory of the Creative Behavior Inventory: An Item Response Theory Analysis of the CBI

ABSTRACT The original 90-item Creative Behavior Inventory (CBI) was a landmark self-report scale in creativity research, and the 28-item brief form developed nearly 20 years ago continues to be a popular measure of everyday creativity. Relatively little is known, however, about the psychometric properties of this widely used scale. In the current research, we conduct a detailed psychometric investigation into the 28-item CBI by applying methods from item response theory using a sample of 2,082 adults. Our investigation revealed several strengths of the current scale: excellent reliability, suitable dimensionality, appropriate item difficulty, and reasonably good item discrimination. Several areas for improvement were highlighted as well: (1) the four-point response scale should have fewer options; (2) a handful of items showed gender-based differential item functioning, indicating some gender bias; and (3) local dependence statistics revealed clusters of items that are redundant and could be trimmed. These analyses support the continued use of the CBI for assessing engagement in everyday creative behaviors but suggest that the CBI could benefit from thoughtful revision.

Item response theory analysis of the Dysfunctional Beliefs and Attitudes about Sleep-16 (DBAS-16) scale in a university student sample.

Unhelpful beliefs about sleep have been shown to exacerbate distress associated with sleep-related difficulties. University students are particularly vulnerable to experiencing sleep-related problems. The Dysfunctional Beliefs and Attitudes about Sleep-16 (DBAS-16) scale is a widely used instrument that assesses for sleep-disruptive cognitions. Although psychometric support for the DBAS-16 is available, Item Response Theory (IRT) analysis is needed to examine its properties at the item level. Psychometric investigation in non-clinical samples can help identify people who may be at risk for developing sleep problems. We examined the DBAS-16 using IRT on a sample of 759 university students. Our results identified items and subscales that adequately/inadequately differentiated between students who held unhelpful beliefs about sleep and those who did not. The DBAS-16 is a valuable instrument to assess unhelpful beliefs about sleep. We outline recommendations to improve the discriminatory ability of the instrument. Future investigations should establish cross-validation with a clinical sample.

A psychometric investigation of the parent engagement in Evidence-Based Services Questionnaire with a diverse community sample.

A psychometric investigation of the parent engagement in Evidence-Based Services Questionnaire with a diverse community sample.

Psychometric investigation of the affiliate stigma scale in Mexican Parkinson's disease caregivers: Development of a short form.

Parkinson's disease (PD) caregivers, particularly in Latin America, may experience high levels of affiliate stigma due to their association with a person having a disability. The most common measure used of this construct in the literature, the Affiliate Stigma Scale, was validated using non-standard and questionable methods. The purpose of this study was to investigate the factor structure and psychometric properties of the Spanish version of the Affiliate Stigma Scale with PD caregivers in Mexico using more widely accepted psychometric approaches including confirmatory and exploratory factor analyses (CFAs, EFAs). A sample of 148 PD caregivers from Mexico completed this measure, as well as indices of caregiver burden and anxiety. Initial CFAs revealed that the data did not fit either the originally proposed one-factor or three-factor structures. An EFA was then conducted which was unable to discern any factor structure. Upon instituting a stepwise removal alpha-if-item-deleted process, a 5-item Affiliate Stigma Scale Spanish Short Form was retained with an adequate Cronbach's alpha, good convergent validity, and a Short Form CFA generally indicating adequate fit. The new Spanish Affiliate Stigma Scale Short Form holds promise for more appropriately measuring affiliate stigma likely in general but particularly in Spanish and among PD caregivers. The Short Form can assist not only in assessing levels of caregiver affiliate stigma, but in creating novel interventions to help support caregivers and decrease stigma.

Clinical and psychometric study of asthenia in foreign applicants for medical education at different stages of training

This article analyzes the results of the clinical and psychometric study of asthenia in foreign applicants for higher medical education at different stages of training. The study involved 110 applicants of the preparatory department (group 1) and 44 applicants of the I-II course (group 2). They were trained at Dnipro State Medical University. Duration of training was from 1 to 2 months and from 6 months to 2 years, respectively (at the time of participation in the study). The clinical and psychopathological examination was conducted using the semi-structured interview method and the psychometric investigation of the level of asthenia with using the Multidimensional Fatigue Inventory scale (MFI-20). It was found that the investigated group 2 had the significantly higher frequency of detection of both complaints/symptoms and nosological forms (by 1.8 times) than group 1. The analysis of the MFI-20 scale data confirmed the clinically obtained results: the normal (24,5%) and moderate level (23,5%) of asthenia was significantly more often observed in group 1. The high level (32%) of asthenia was in group 2 that proves the maladaptive process deepening. The obtained result indicates the necessity to develop preventive, differential and corrective measures to improve the quality of life of this contingent.

Time for a Measurement Check-Up: Testing the Couple's Satisfaction Index and the Global Measure of Sexual Satisfaction Using Structural Equation Modeling and Item Response Theory.

Relationship and sexual satisfaction are two central outcomes in the study of relationships and are commonly used in both academia and applied practice. However, relationship and sexual satisfaction measures infrequently undergo specific psychometric investigation. Ensuring that measures display strong psychometric performance is an important but under-tested element of replication that has come under more scrutiny lately, and adequate measurement of constructs is an important auxiliary assumption underpinning theory-testing empirical work. A measurement check-up was conducted, including Confirmatory Factor Analysis (CFA) to test factorial validity, measurement invariance to test for group comparability, and Item Response Theory (IRT) to assess the relationship between latent traits and their items/indicators. This format was used to evaluate the psychometric properties of the Couple's Satisfaction Index (CSI) and the Global Measure of Sexual Satisfaction (GMSEX), two commonly used scales of relationship and sexual satisfaction with a sample of 640 midlife (40-59years old) married Canadians who were recruited by Qualtrics Panels. Results of CFA suggested that both models were satisfactory. Invariance testing provided robust support for intercept invariance across all the groupings tested. IRT analysis supported the CSI and GMSEX, however, there was evidence that the GMSEX provided somewhat less information for those high on sexual satisfaction. This measurement check-up found that the CSI and GMSEX were reasonably healthy with some caveats. Implications are discussed in terms of replicability and meaning for scholars and practitioners.

Translation, cross-cultural adaptation, and psychometric investigation of Chinese version of the Orbach & Mikulincer Mental Pain Scale in patients with depressive disorder.

This study aimed to translate, cross-culturally adapt, and preliminarily test the psychometric properties of the Chinese version of the Orbach & Mikulincer Mental Pain Scale (OMMP). Psychometric investigation was performed on 240 depressed patients. The reliability of the Chinese version of the OMMP scale was expressed by internal consistency reliability and test-retest reliability (2-week interval), and confirmatory factor analysis (CFA) on the 8-factor, 31-item OMMP was conducted to examine the construct validity. The CFA showed that the modified model with 31 items had good reliability (Cronbach's α range = 0.691-0.871; ICC = 0.818). Criterion-related validity was also supported by significant and positive correlations between the eight factors and worst-ever suicidal ideation as well as depressive symptoms. The results indicated the usefulness of the OMMP-31 for Chinese depressed patients. It is necessary to estimate psychological pain to improve suicide management in the clinical setting.

Measuring loot box consumption and negative consequences: Psychometric investigation of a Swedish version of the Risky Loot Box Index

Loot boxes are products in videogames that is earned by playing a video game or by buying them. Loot boxes has similar mechanisms as a lottery and there is an ongoing debate if loot boxes are gambling. However, to understand the potential harm of loot boxes valid instruments are needed. An English study psychometrically evaluated an instrument focused on risky loot box use called the Risky Loot Box Index. The study evaluated 12 item scale and based on a factor analysis it was reduced to a five-item scale. The aim of our study is to evaluate a Swedish version of the 12-item instrument from a psychometric perspective. Two samples recruited via an online survey were used. The first sample was recruited from the gambling site Unibet. A mail with an invitation to participate was sent to esports bettors and sport bettors at the gambling site. The second sample was recruited from a Facebook forum focused on e-sports. An invitation was posted on the forum to partake in the study. A total of 195 of respondents (96% men and 4% women) with a mean age of 33.76 (SD = 12.34) answered the Unibet survey and 169 respondents (96% men and 4% women and non-binary) with a mean age of 23.89 (SD = 5.52) answered the Facebook survey. An exploratory factor analysis yielded a two-factor seven item solution. The factors were overconsumption regarding time spent on loot boxes and the other factor was focused on overconsumption of loot boxes in terms of spending money. The confirmatory factor analysis conducted in the Facebook sample validated the result from the exploratory factor analysis. The conclusion of the study is that the Swedish version of the Risky Loot Index has good psychometric properties and can used to measure risky loot box consumption.

Making high-quality measures available in diverse contexts-The psychometric properties of the Revised Child Anxiety and Depression Scale in a Norwegian sample.

Recent initiatives have recommended the Revised Child Anxiety and Depression Scale (RCADS) for use in research and as patient-reported outcome in health care globally. We aimed to investigate, for the first time, whether the psychometric properties of the anxiety and depression youth self-report measures, RCADS-47 and RCADS-25, generalize to a Norwegian setting. We examined gender and age differences in symptomatology among 592 children (mean age 10.7years), and conducted a psychometric investigation of the internal reliability, structural validity, measurement invariance and convergent validity of the RCADS-47 and RCADS-25 youth versions. Girls reported higher levels of anxious and depressive symptoms than boys, but no age differences were observed. Reliability coefficients for the RCADS-47 and RCADS-25 scales indicated good internal consistency. Structural validity for RCADS-47 and RCADS-25 was supported by confirmatory factor analyses results. For both measures, strong gender-based measurement invariance was present. Convergent validity of the RCADS-47 and RCADS-25 with other well-established self-report measures for anxiety (Multidimensional Anxiety Scale for Children) and depression (The Short Mood and Feelings Questionnaire) was supported. The RCADS-47 and RCADS-25 youth versions are valid and reliable instruments for measuring symptoms of anxiety and depression in a Norwegian setting. The results add to the evidence supporting RCADS's cross-cultural validity.

The Brief Course Belonging Scale: Developing a Measure of Postsecondary Students' Course-Level Sense of Belonging Across Online and Face-To-Face Modalities

Sense of belonging is an important topic in higher education. However, few studies have examined this important construct at the course level and in the online learning context; even fewer are quantitative by design. The aim of our study was to develop and evaluate a measure of sense of belonging that could be used across different postsecondary learning contexts. A psychometric investigation was conducted at a large, US southeastern university on data using the newly developed Brief Course Belonging Scale (BCBS). Results provide evidence for the unidimensional treatment of BCBS data across delivery context, convergent validity for BCBS scores as they relate positively to belonging at the university level, connectedness, and academic motivation, and discriminant validity for BCBS scores as they related minimally with loneliness. Differential item functioning was detected on one item, but this did not jeopardize score validity and reliability. Specific psychometric implications regarding the domain-specificity of the course delivery context as well as the administration of the novel instrument to a more broad, and diverse student population are recommended.

MOTHERS’ ANXIETY TO AGGRAVATED ACUTE FEAR OF EARTHQUAKES IN THE CITY OF ZAGREB IN THE COVID 19 PANDEMIC

Here we present the results of a psychometric investigation on current fear of COVID-19 infection aggravated by the earthquake-induced stress in the group of puerperal women. A group of 16 hospitalized puerperal women that gave birth at Clinical Hospital Sveti Duh in Zagreb, Department of Obstetrics and Gynecology were enrolled in retrospective clinical investigation. All the patients delivered during COVID-19 pandemic and experienced devastating earthquake on March 22nd. The women were interviewed on the exact day of the earthquake, 4 hours after the main hit and 6 months after that during second pandemic wave on October of the same year. Women were also questioned about the breastfeeding. The participants provided written informed consent and were interviewed using Generalized Anxiety Disorder 7-item (GAD-7) scale for evaluation of generalized anxiety disorder. Minimal level of anxiety after the earthquake was shown for 10 mothers with 6 of them having the highest level 4. Mild anxiety was proven for 3 patients, 2 exhibited moderate anxiety with one puerperal women with severe anxiety. One could say that 37.5% of enrolled patients exhibited some degree of anxiety after the earthquake (scoring >5). At second time point during COVID-19 pandemic 13 participants had minimal anxiety score, 2 had mild, while 1 participant had moderate score evaluated by GAD-7 scale (18.8%). At the time of the earthquake 13 participants were breastfeeding (81.3%), while 3 were not due to the personal reasons. 6 months after the earthquake only 7 mothers were breastfeeding (43.8%), while 9 of the participants were feeding their children with adapted milk (56%). Main reason for breastfeeding discontinuation was the stop of milk secretion in the 6 months period after the delivery. To our knowledge, our study is the only national study dealing with mental health problems in a population of puerperal and breastfeeding mothers in a challenging time of COVID-19 pandemic aggravated by devastating earthquake.

Construct validity, reliability, and responsiveness of the Wrist Position Sense Test for use in children with hemiplegic cerebral palsy.

We investigate the construct validity, test re-test reliability, and responsiveness of the Wrist Position Sense Test (WPST) for children with hemiplegic cerebral palsy (CP). Twenty-eight children with spastic hemiplegic CP [mean age 10.8 years; SD 2.4 years] and 39 typically developing (TD) children [mean age 11 years; SD 2.9 years] participated in a cross-sectional study to investigate construct validity and association with an upper limb activity measure, the Box and Block Test (BBT). Twenty-two TD children were tested at a second time-point to examine reliability. Test responsiveness was determined by random allocation of 17 children with CP to a treatment (n = 10) or control (n = 7) group with assessments completed at four time-points. Significantly greater differences were observed in mean error of indicated wrist position (p &lt; 0.01) in children with CP at baseline (M = 21.6°, SD = 21.6°) than in TD children (M = 12.8°, SD = 11.0°). Larger WPST errors were associated with poorer performance on the BBT (p &lt; 0.01) indicating a substantial association, and there were no consistent differences between time-points indicating test re-test reliability within a TD population. The WPST demonstrated responsiveness to intervention with a statistically significant reduction in mean error following treatment (p &lt; 0.001), not seen in the control group (p = 0.28). The WPST demonstrated construct validity in this preliminary study. Scores were associated with an upper limb activity measure, and scores changed significantly following somatosensory training. These findings support further research and future psychometric investigation of the WPST in children with CP. This study provides psychometric knowledge about the WPST tool The WPST shows promise as a discriminative measure with preliminary evidence of responsiveness and intra-rater reliability Until further testing, the WPST can be used cautiously in future research studies to measure wrist position sense.

Psychometric Investigation of the Revised Children’s Manifest Anxiety Scale, Second Edition, Short Form among the Korean Youth Population

The Revised Children’s Manifest Anxiety Scale, Second Edition (RCMAS-2) has long been used to measure anxiety levels in youth. Its introduction of a 10-item short form has enriched the efficiency and applicability of the scale; however, more re- search is warranted to elucidate its psychometric properties. This study aims to investigate the factor structure and the mea- surement invariance of the RCMAS-2 short form (RCMAS-2 SF) in Korean youth. In total, 1,525 participants from seven different cities of South Korea were included in the analysis (Mean Age = 12.49, SD = 2.54). After randomly assigning partici- pants into two groups, we consecutively performed exploratory and confirmatory factor analyses. The results of both analyses demonstrated that pediatric anxiety assessed using the RCMAS-2 SF is composed of two distinct latent factors: physiological anxiety and cognitive anxiety. Furthermore, the results supported strong invariance across gender and age. The RCMAS-2 SF score also showed good indices of internal consistency, test-retest reliability, and concurrent validity. These findings highlight that the scale can be widely used as a time-efficient screening tool that enables valid score comparisons between boys and girls of different ages.