Internal Consistency Estimates Research Articles

Stimulus-associated urinary urges in overactive bladder syndrome.

Although anecdotal reports of urinary urgency at one's front door are common in overactive bladder syndrome (OAB), little research has been done on how one's front door and other stimuli are related to urinary symptoms. We hypothesized that individuals with OAB would have higher scores on the Urinary Cues Questionnaire, developed for this study to assess stimulus-associated urinary urges, than those without OAB. Online surveys were administered to 328 women age 18-40 years recruited from a respondent panel maintained by CINT such that one-third of the sample reported a diagnosis of OAB. The survey assessed OAB symptoms and the frequency with which participants associated 42 stimuli with the urge to urinate. Psychometric analyses showed internal consistency estimates of the Urinary Cues Questionnaire of α = 0.97 and a test-retest correlation of 0.91. Women with OAB had significantly higher Urinary Cues Scores than those without OAB, with a t-test showing a large effect size of d = 1.49 (95%CI 1.24, 1.74), P < 0.001. Behavioral treatments aimed at reducing the response to cues may be useful in OAB, but more research is needed on both treatment implications and on the trajectory of symptom development.

A Reliability Generalization on the Children’s Hope Scale

The Children’s Hope Scale is one of the most commonly used self-report measures of a child’s future oriented goal motivation. This study presents a reliability generalization on both the internal consistency and test-retest reliability estimates for the Children’s Hope Scale. While 225 published works were analyzed 4.2% authors did not report reliability estimates for their study and 10.7% induced from a previous study. The average internal consistency score (N = 164) was .81 (95% CI = .79 -- .82) and the test-retest (N = 15) at .71 (95% CI = .64 -- .78) respectively. An analysis of variance showed that non-English language samples produced moderately lower (albeit still acceptable) Cronbach’s Alpha estimates. The results of the reliability generalization suggest the score reliabilities produced by the Children’s Hope Scale are acceptable across samples. The findings of this study paired with the growing number of validation studies suggest researchers can use of the Children’s Hope Scale with increased confidence.

Tell me how you feel and I will tell you who you are: First validation of the affect integration inventory

Affect integration, or the capacity to utilize affects for personal adjustment, is an essential aspect of psychological functioning. Affect integration has been operationalized through the affect consciousness construct as degrees of awareness, tolerance, nonverbal expression, and conceptual expression of nine discrete affects. It is measured by the Affect Consciousness Interview (ACI). The ACI, however, is time-consuming and demands extensive training. It is therefore a priority to develop a self-rated measure of affect integration that is more readily accessible. This study explored the validity of the Affect Integration Inventory (AII), a self-rated operationalization of affect integration, through estimates of internal consistency and internal structure. External validity was addressed by examining relationships between AII scores and emotion regulation, alexithymia, psychiatric symptoms, and interpersonal problems. We found satisfactory reliability, sound internal structure, and associations with external criteria, indicating good convergent and discriminant validity. Such findings included distinct relationships between AII ratings and specific types of interpersonal problems.

Temporal Stability of Implicit and Explicit Measures: A Longitudinal Analysis.

A common assumption about implicit measures is that they reflect early experiences, whereas explicit measures are assumed to reflect recent experiences. This assumption subsumes two distinct hypotheses: (a) Implicit measures are more resistant to situationally induced changes than explicit measures; (b) individual differences on implicit measures are more stable over time than individual differences on explicit measures. Although the first hypothesis has been the subject of numerous studies, the second hypothesis has received relatively little attention. The current research addressed the second hypothesis in two longitudinal studies that compared the temporal stability of individual differences on implicit and explicit measures in three content domains (self-concept, racial attitudes, political attitudes). In both studies, implicit measures showed significantly lower stability over time (weighted average r = .54) than conceptually corresponding explicit measures (weighted average r = .75), despite comparable estimates of internal consistency. Implications for theories of implicit social cognition and interpretations of implicit and explicit measures are discussed.

Factorial structure and long-term stability of the Autonomy Preference Index.

The autonomy preference index scale (API) has been designed to measure patient preference for 2 dimensions of autonomy: Their desire to take part in making medical decisions (decision making, [DM]) and their desire to be informed about their illness and the treatment (information seeking; [IS]). The DM dimension is measured by 6 general items together with 9 items related to 3 clinical vignettes (3 × 3 items). The IS dimension is measured by 8 items. While the API is widely used, a review of literature has identified several inconsistencies in the way it is scored. The first aim of this study was to determine the best scoring structure of the API on the basis of validity and reliability evidence. The second aim was to investigate the long-term stability of API scores. Two-hundred and 85 patients with a diagnosis of psychosis were assessed as they were about to be discharged from involuntary psychiatric hospitalization and they were reassessed after 6 and 12 months. Confirmatory factor analysis (CFA) revealed that a 3-factor solution was most adequate and that 2 distinct DM subscales should be preferred to 1 total DM score. While internal consistency estimates of the 3 subscales were good, the long-term stability of API scores was only modest. Multigroup-CFA revealed scalar invariance indicating API scores kept the same meaning longitudinally. In conclusion, a 3-factor structure seemed to be most adequate for the API scale. Long-term stability estimates suggested that clinicians should regularly assess patients' preferences for autonomy because API scores fluctuate over time. (PsycINFO Database Record

Psychometric Properties of Turkish Adolescent Time Inventory-Time Attitude (ATI-TA) Scores

The Adolescent Time Inventory (ATI; Mello &amp; Worrell, 2007 ) is a relatively new measure developed to assess several aspects of time perspective (e.g., time orientation, time attitudes). Time attitudes, one aspect of time perspective, refer to positive and negative feelings about the past, present, and future. In the current 2-sample paper, we examined the internal consistency, structural validity, and convergent validity of scores on the time attitude subscales (TA) of the Turkish ATI ( Mello, Worrell, Şahin-Baltaci, &amp; Tagay, 2015 ). Results from Sample 1 ( N = 244) indicated that scores on 5 of the 6 Turkish ATI-TA subscales—Past Positive, Past Negative, Present Positive, Present Negative, and Future Positive—were internally consistent, and confirmatory factor analyses provided strong support for the structural validity of a 5-factor model as well as the hypothesized 6-factor model, even though internal consistency estimates for Future Negative scores (the sixth factor) were unacceptably low. Results from Sample 2 ( N = 350) provided additional internal consistency and structural validity evidence for scores on the 5-factor model and also provided evidence of convergent validity with self-esteem, well-being, and optimism. Analyses also indicated that scores on the 2 past and 2 present subscales demonstrated configural, metric, and scalar invariance across samples, whereas the Future Positive scores only demonstrated metric invariance. We concluded that these 5 subscales can be used with Turkish adolescents and that the Future Negative subscale needs to be revised and validated in this national context.

Reliability of surface facial electromyography.

Data from two studies were used to estimate the reliability of facial EMG when used to index facial mimicry (Study 1) or affective reactions to pictorial stimuli (Study 2). Results for individual muscle sites varied between muscles and depending on data treatment. For difference scores, acceptable internal consistencies were found only for corrugator supercilii, and test-retest reliabilities were low. For contrast measures describing patterns of reactions to stimuli, such as high zygomaticus major combined with low corrugator supercilii, acceptable internal consistencies were found for facial reactions to smiling faces and positive affective reactions to affiliative images (Study 2). Facial reactions to negative emotions (Study 1) and facial reactions to power and somewhat less to achievement imagery (Study 2) showed unsatisfactory internal consistencies. For contrast measures, good temporal stability over 24 months (Study 1) and 15 months (Study 2), respectively, was obtained. In Study 1, the effect of method factors such as mode of presentation was more reliable than the emotion effect. Overall, people's facial reactions to affective stimuli seem to be influenced by a variety of factors other than the emotion-eliciting element per se, which resulted in biased internal consistency estimates. However, the influence of these factors in turn seemed to be stable over time.

The Japanese Version of the Adolescent Time Inventory-Time Attitudes (ATI-TA) Scale: Internal Consistency, Structural Validity, and Convergent Validity.

In this study, we translated the Adolescent Time Inventory-Time Attitudes (ATI-TA) scale into Japanese and examined the internal consistency and structural and convergent validity of ATI-TA scores in a sample of 383 Japanese high school students aged 15 to 18 years. The ATI-TA consists of six subscales assessing two valences (positive and negative) for each of three time periods (past, present, and future). Internal consistency estimates for scores on the ATI-TA were over .79. Confirmatory factor analysis indicated that a six-factor model yielded the best fit to scores. Correlations among ATI-TA scores, time attitudes assessed by the semantic differential method, educational career planning, and self-esteem were consistent with our hypotheses. Our results provide evidence that scores on the Japanese version of the ATI-TA are reliable and valid, and thus potentially useful for research in this cultural context.

A measure of facilitators and barriers to rapid response team activation

PurposePoor patient outcomes and increased costs may be associated with underutilization of RRTs. The aim of this study was to develop and test an instrument that identifies specific facilitators and barriers to rapid response team (RRT) activation. MethodsUsing an exploratory design, we surveyed a convenience sample of 250 registered nurses (RNs) employed in five Illinois hospitals. Participants completed the online RRT Facilitators and Barriers Survey (RRT-FBS), a 36 item survey developed by the researchers. The survey contains two sections, facilitators and barriers. Items in the facilitators subscales described nursing unit culture, RRT knowledge, and RRT member characteristics. Items in the barriers subscales described nursing unit culture, RRT education, and RRT member characteristics. Item analyses were conducted through exploratory factor analyses; internal consistency estimates were obtained. Descriptive statistics were conducted on the demographic data to describe sample and setting characteristics. ResultsThe final sample consisted of 202 nurses from four hospitals. We conducted an item analysis and were able to reduce the survey to 30 items with a secondary analysis. The full scale alpha was 0.752. Cronbach's alphas for subscales ranged from 0.770–0.897. ConclusionsFacilitators and barriers may vary across institutions. This scale shows promise for identifying facilitators and barriers to nurses' use of rapid response teams and may provide a foundation for interventional studies promoting RRT utilization. In addition, more frequent education, emphasizing the RRT process, may be an effective method to maintain high rates of RRT activation and increase confidence.

Internal consistency and interrater reliability of the Brazilian version of Martín-Bayarre-Grau (MBG) adherence scale

This paper aims to analyze the measurement equivalence aspects (internal consistency and interrater reliability) of a Brazilian version of Martin-Bayarre-Grau (MBG) adherence questionnaire as part of its cross-cultural adaptation. Item-total correlation and Cronbach's alpha coefficients were used as internal consistency estimates. Stability was evaluated through test and retest comparison and expressed through intraclass correlation coefficient (ICC) and kappa with quadratic weighting. ICC for the overall scale was 0.81, indicating an almost perfect agreement. However, some cases of poor and slight agreements were found while analyzing individual items. The translated version of the MBG questionnaire showed good homogeneity (alpha 0.78), higher than cutoff points suggested in the literature. The scale has proved capable of measuring the level of adherence to treatment in hypertensive and/or diabetic patients in a reliable way.

Screening for Psychosocial and Family Risk in Pediatric Gastrointestinal Disorders.

Screening for psychosocial comorbidity is recommended for pediatric patients presenting at an initial gastroenterology (GI) outpatient consultation. We developed and evaluated the psychometric properties of the GI Screener to address the need for a screening tool specific to pediatric GI patients. 128 patients (8-18 years old, 63% female) and 126 parents completed age-specific versions of the GI Screener and 3 validated psychosocial comparison instruments (The Behavioral Assessment System for Children, The Functional Disability Inventory, The General Functioning scale of the Family Assessment Device) at their initial GI consultation. (30%) of families repeated the measures 2 weeks later. We identified GI Screener content domains and retained items using exploratory factor analysis. We evaluated internal consistency, construct validity, cross-informant reliability, and test-retest reliability of the trimmed measures. Exploratory factor analysis identified 2 factors in both the parent and child scales: Symptom Impact and Emotional Functioning. Internal consistency estimates for the trimmed scales were good (Cronbach's alpha >0.75) for both Child and Parent scales. We found that the GI Screener for both patient and parents had good construct validity. Cross-informant reliability between Parent and Child scales at baseline had an estimated correlation of 0.56, while intra class correlation coefficients between baseline and 2-week scores showed high test-retest reliability (>0.7). The GI Screener is a brief, valid and reliable measure that can aid in identifying families who are at high risk for psychosocial comorbidity facilitating the targeted delivery of psychosocial intervention and efficient use of health care resources.

Psychometric Evidence of SRSS-IE Scores in Middle and High Schools

We report results of an exploratory validation study of the Student Risk Screening Scale–Internalizing and Externalizing (SRSS-IE) applied with the first sample of middle and high school students from nine middle and three high schools from three states. The Student Risk Screening Scale (SRSS) was modified to broaden the scope of this user-friendly systematic screening tool to detect secondary-age students with internalizing behavior patterns. Building on initial evidence establishing reliability and validity of the SRSS-IE at the middle school level, we used classical test theory to explore reliability of SRSS-IE spring screening scores for use with sixth- through 12th-grade students. Specifically, we analyzed item-level data, internal consistency estimates, and factor structure using exploratory factor analysis. Results suggested five of the seven proposed items be retained, yielding the SRSS-IE12. Peer rejection (one of the originally developed items by Drummond) loaded on the internalizing construct, providing preliminary evidence suggesting two factors: SRSS-E7 (the original tool) and SRSS-I6 for use with secondary students, with peer rejection loading on both subscales but only added once when formulating the total score (SRSS-IE12). We present limitations and direction for future inquiry.

Cultural awareness scale: Psychometric properties of the Turkish version

BackgroundCultural competence, a relatively new concept in the world of healthcare professionals, has garnered an increasing global attention over the last 20 years. An accurate assessment of baseline knowledge, skills, attitudes, strengths, and limitations of the students in delivering culturally congruent nursing care appears to be a key component in designing teaching strategies. AimThe current study aimed to analyze the psychometric properties of the Turkish language version of the Cultural Awareness Scale and to determine any possible similarities between the compositions of the Turkish version and the original scale. MethodsThis methodological study included a sample size of 197 undergraduate nursing students. The research data were collected using a sociodemographic form and the Turkish language version of the Cultural Awareness Scale, which contains 36 items designed to address five different factors using a 7-point Likert response format. FindingsThe Turkish language version of the scale comprised 36 items under four different subscales. The scale was found highly reliable, with an internal consistency estimate of reliability at 0.897 for the total scale, which ranged from 0.751 to 0.868 for the subscales. The item-total correlation test showed that there was a significant correlation between each item and the remaining items, with correlation coefficient values ranging from 0.147 to 0.647. ConclusionsThe results of the analyses showed that the Turkish language version of the Cultural Awareness Scale was a highly valid and reliable instrument; therefore, it might prove a valuable asset for use in various healthcare disciplines.

Analyse de la structure factorielle de la version brève du questionnaire de personnalité schizotypique (SPQ-B) – format Likert – en population générale en France

IntroductionThe main objective of the study was to explore the factorial structure of the French version of the Schizotypal Personality Questionnaire-Brief (SPQ-B) in a Likert format, in a representative sample of the general population. In addition, differences in the dimensional scores of schizotypy according to gender and age were analyzed. As the study in the general population of schizotypal traits and its determinants has been recently proposed as a way toward the understanding of aetiology and pathophysiology of schizophrenia, consistent self-report tools are crucial to measure psychometric schizotypy. A shorter version of the widely used Schizotypal Personality Questionnaire (SPQ-Brief) has been extensively investigated in different countries, particularly in samples of students or clinical adolescents, and more recently, a few studies used a Likert-type scale format which allows partial endorsement of items and reduces the risk of defensive answers. MethodA sample of 233 subjects representative of the adult population from an urban area near Paris (Créteil) was recruited using the “itinerary method”. They completed the French version of the SPQ-B with a 5-point Likert-type response format (1=completely disagree; 5=completely agree). We examined the dimensional structure of the French version of the SPQ-B with a Principal Components Analysis (PCA) followed by a promax rotation. Factor selection was based on Eigenvalues over 1.0 (Kaiser's criterion), Cattell's Scree-plot test, and interpretability of the factors. Items with loadings greater than 0.4 were retained for each dimension. The internal consistency estimate of the dimensions was calculated with Cronbach's α. In order to study the influence of age and gender, we carried out a simple linear regression with the subscales as dependent variables. ResultsOur sample was composed of 131 women (mean age=52.5±18.2 years) and 102 men (mean age=53±18.1 years). SPQ-B Likert total scores ranged from 22 to 84 points (mean=43.6±13). Factor analysis resulted in a 3-factor solution that explained 47.7% of the variance. Factor 1 (disorganized; 10 items) included items related to “odd behavior”, “odd speech”, as well as “social anxiety”, one item of “constricted affect” and one item of “ideas of reference”. Factor 2 (interpersonal; 7 items) included items related to “no close friends”, “constricted affect”, and three of the items of “suspiciousness”. Factor 3 (cognitive-perceptual; 5 items) included items related to “ideas of reference”, “magical thinking”, “unusual perceptual experiences” and one item of “suspiciousness”. Coefficient α for the three subscales and total scale were respectively 0.81, 0.81, 0.77 and 0.88. We found no differences in total schizotypy and the three dimensions scores according to age and sex. ConclusionFactor analysis of the French version of the SPQ-B in a Likert format confirmed the three-factor structure of schizotypy. We found a pure cognitive perceptual dimension including the most representative positive features. As expected, “Suspiciousness” subscale is included in both positive and negative dimensions, but mainly in the negative dimension. Surprisingly, “social anxiety” subscale is included in the disorganized dimension in our analysis. The SPQ-B in a Likert format demonstrated good internal reliability for both total and subscales scores. Unlike previous published results, we did not find any influence of age or gender on schizotypal dimensions.

The career adapt-abilities scale-Nigeria form: Psychometric properties and construct validity

This study examined the psychometric properties of Career Adapt-Ability Scale (CAAS) in Nigeria. The CAAS consists of four subscales, each with six items that measure concern, control, curiosity and confidence as psychosocial resources for coping with occupational tasks, work traumas and transitions. The participants in this study included bank (N=143) and hotel (N=128) employees. Internal consistency estimates for the subscale and total scores ranged from good to excellent. The factor structure was similar to that of the CAASInternational Form. While the result revealed that age was significantly related to confidence, gender and education had no significant relationships with the four dimensions of adaptability.

Validation d’une grille d’évaluation des compétences cliniques utilisée par des patients standardisés

BackgroundThis study was designed in the context of a new educational program based on standardized-patients (SPs). The study objectives were (1) to evaluate the reliability of the assessment form used by SPs and (2) to compare global ratings to checklists reproducibility. MethodHistory taking, physical examination and communication skills were assessed by SPs at the end of clinical encounters with year 3 medical students, using generic global rating scales and case-specific checklists. The validation process followed four steps: (1) correlation analysis between global rating and their relative checklist of specific items, (2) internal consistency estimation, (3) validation of the questionnaire dimensions, (4) estimation of the reliability of SPs’ ratings compared to medical teachers’ ratings. ResultsA total of 3322 consultations have been performed by 444 year 3 medical students. Statistical analysis showed a good internal reliability (Cronbach α was greater than 0.7) and an acceptable inter-judge reproducibility except for communication skills. Case-specific checklists did not prove to be more reliable than global ratings. Reproducibility was lower with SPs’ than with medical teachers’ assessment. ConclusionGlobal rating-based assessment should be preferred to checklists because they are faster and easier to use (shorter duration of SPs training). As SPs proved to be acceptable examiners, no third person seems to be required as external observer.

The caring behaviour of primary and middle school teachers in China: features and structure

The present research, comprised of two studies, examines the features and factor structure of the Teacher Caring Questionnaire, a measure of teacher caring developed in the Zhejiang province of China. In the first study, features of teacher caring were derived from open-ended teacher and student descriptions of caring teacher behaviour. A second sample of teachers then rated the necessity of each of the features as an aspect of teacher caring. In the second study, the development of a self-report questionnaire based on these eight features is described, and its factor structure examined through exploratory and confirmatory factor analyses. This process resulted in six initial factors, which were subsequently combined into three higher order factors, which appear to best capture the instrument’s structure. Internal consistency estimates for the three resulting subscales were calculated for five samples, and test–retest reliability calculated for Sample 5. Support for the convergent, discriminant and predictive validity of the measure were examined through correlations with conceptually similar measures, and t-tests comparing subscale scores for participants who indicated willingness to provide help (or not) under different conditions. Our analyses suggest that the Teacher Caring Questionnaire holds promise as a measure of teacher caring in the Zhejiang province. Furthermore, our method might inform similar efforts to develop context-grounded measures, as well as efforts to examine the cultural relevance of existing measures.

Validity of a new assessment rubric for a short-answer test of clinical reasoning.

BackgroundThe validity of high-stakes decisions derived from assessment results is of primary concern to candidates and certifying institutions in the health professions. In the field of orthopaedic manual physical therapy (OMPT), there is a dearth of documented validity evidence to support the certification process particularly for short-answer tests. To address this need, we examined the internal structure of the Case History Assessment Tool (CHAT); this is a new assessment rubric developed to appraise written responses to a short-answer test of clinical reasoning in post-graduate OMPT certification in Canada.MethodsFourteen physical therapy students (novices) and 16 physical therapists (PT) with minimal and substantial OMPT training respectively completed a mock examination. Four pairs of examiners (n = 8) participated in appraising written responses using the CHAT. We conducted separate generalizability studies (G studies) for all participants and also by level of OMPT training. Internal consistency was calculated for test questions with more than 2 assessment items. Decision studies were also conducted to determine optimal application of the CHAT for OMPT certification.ResultsThe overall reliability of CHAT scores was found to be moderate; however, reliability estimates for the novice group suggest that the scale was incapable of accommodating for scores of novices. Internal consistency estimates indicate item redundancies for several test questions which will require further investigation.ConclusionFuture validity studies should consider discriminating the clinical reasoning competence of OMPT trainees strictly at the post-graduate level. Although rater variance was low, the large variance attributed to error sources not incorporated in our G studies warrant further investigations into other threats to validity. Future examination of examiner stringency is also warranted.

A Comparison of Self-Report Measures of Callous-Unemotional Traits Among Incarcerated Youth

The current study compared the psychometric properties and incremental validity of the callous-unemotional (CU) traits dimensions from four self-report measures of youth psychopathic traits, including the Antisocial Process Screening Device Self-Report version (APSD-SR), the Youth Psychopathic Traits Inventory (YPI), Youth Psychopathic Traits Inventory–Short version (YPI-S), and the Inventory of Callous-Unemotional Traits (ICU) among a sample of male youth incarcerated in Portuguese detention centers ( N = 221). Across these measures, estimates of reliability and internal consistency were generally good; however, the correlations between measures ranged from very low to high suggesting moderate convergent validity. With regard to incremental validity, the Callousness dimension of the ICU was the best predictor of aggression, conduct disorder, and crime seriousness, whereas the Uncaring dimension of the ICU was the best predictor of age of crime onset. The current findings have important implications regarding the assessment of CU traits.

Psychometric properties of time attitude scores in young, middle, and older adult samples

In an effort to provide a measure of time perspective that can be used across the life-span, we examined the psychometric properties of the Adolescent Time Inventory-Time Attitude Scale (ATI-TA; Mello & Worrell, 2007) in three independent samples of young (N=388), middle (N=201), and older adults (N=189). Results provided strong psychometric evidence that the ATI-TA can be used appropriately with individuals across adolescence and adulthood. Specifically, internal consistency estimates indicated that scores on the six subscales (Past Positive, Past Negative, Present Positive, Present Negative, Future Positive, & Future Negative) were reliable across the three samples. Confirmatory factor analyses showed that the theorized six-factor structure had acceptable fit and fit the data better than alternate models. Subsequent analyses provided support for invariance across young, middle, and older adults. Overall, these results show that the ATI-TA yields reliable scores and a valid structure across adulthood and can be used to measure time perspective throughout the life-span.