Low Internal Consistency Research Articles

Comparison of Three Evidence‐Based Practice Learning Assessment Methods in Dental Curricula

Incorporating evidence-based practice (EBP) training in dental curricula is now an accreditation requirement for dental schools, but questions remain about the most effective ways to assess learning outcomes. The purpose of this study was to evaluate and compare three assessment methods for EBP training and to assess their relation to students' overall course grades. Participants in the study were dental students from two classes who received training in appraising randomized controlled trials (RCTs) and systematic reviews in 2013 at the University of Dammam, Saudi Arabia. Repeated measures analysis of variance was used to compare students' scores on appraisal assignments, scores on multiple-choice question (MCQ) exams in which EBP concepts were applied to clinical scenarios, and scores for self-reported efficacy in appraisal. Regression analysis was used to assess the relationship among the three assessment methods, gender, program level, and overall grade. The instructors had acceptable reliability in scoring the assignments (overall intraclass correlation coefficient=0.60). The MCQ exams had acceptable discrimination indices although their reliability was less satisfactory (Cronbach's alpha=0.46). Statistically significant differences were observed among the three methods with MCQ exams having the lowest overall scores. Variation in the overall course grades was explained by scores on the appraisal assignment and MCQ exams (partial eta-squared=0.52 and 0.24, respectively), whereas score on the self-efficacy questionnaire was not significantly associated with overall grade. The results suggest that self-reported efficacy is not a valid method to assess dental students' RCT appraisal skills, whereas instructor-graded appraisal assignments explained a greater portion of variation in grade and had inherent validity and acceptable consistency and MCQ exams had good construct validity but low internal consistency.

Incremental Validity of the Subscales of the Emotional Regulation Related to Testing Scale for Predicting Test Anxiety

The Emotional Regulation Related to Testing Scale (ERT Scale) assesses strategies students use to regulate emotion related to academic testing. It has four dimensions: Cognitive Appraising Processes (CAP), Emotion-Focusing Processes (EFP), Task-Focusing Processes (TFP), and Regaining Task-Focusing Processes (RTFP). The study examined the factor structure of each dimension and the incremental validity of the ERT subscales ( N = 213 undergraduates). Confirmatory factor analysis indicated good fit for CAP and EFP dimensions. TFP and RTFP models had poor fit. The TFP dimension appears to involve a two-factor structure, which may account for why it has been observed to have low internal consistency reliability across studies. Subscales of CAP and EFP dimensions resulted in statistically significant increments in R2 after accounting for self-efficacy for learning and metacognitive self-regulation of learning. The TFP dimension and RTFP subscales did not exhibit incremental validity. The importance of assessing self-regulation of emotion and learning is emphasized.

Psychometric properties of the Modified Tonic Immobility Scale. [Propiedades psicométricas de la Escala de Inmovilidad Tónica: algunos problemas

The Tonic Immobility Scale has been only studied in abuse and/or sexual assault survivors, but tonic immobility (TI) also occurs after other types of traumatic events. Consequently, we modified the scale for this purpose and studied its factor structure, internal consistency, and convergent, discriminant and criterion validity in 392 university students, most of whom had suffered one or more traumatic events of various kinds. The Fear subscale of the Modified Tonic Immobility Scale (MTIS) showed very low internal consistency, and must be eliminated or reformulated. Discrimination indices of items 5 and 8 were very low, and these items must be reformulated or removed. After removing the five previous items, the shortened version of the MTIS had a one-factor structure, although model fit was not completely satisfactory. This scale showed acceptable levels of reliability, convergent validity with post-traumatic symptomatology, discriminant validity with measures of depression and anxiety, and criterion validity differentiating between groups that had and had not experienced traumatic events. The psychometric properties of the shortened version of the MTIS should be studied further. It may also be advisable to add new items of TI in order to establish whether this is a construct with more than one dimension.

Translation and validation of patient-practitioner orientation scale in Sri Lanka.

Practice of family medicine and patient centeredness does not get the deserved attention in clinical practice and teaching in Sri Lanka. Non-availability of tools for assessment of patient centeredness deters the process of curricular development and research. The Patient-Practitioner Orientation Scale (PPOS) is a self-administered tool that assesses patient-centeredness in both health care professionals and patients. This study has translated and validated the PPOS to Sinhala language. Translation and cross-cultural adaptation were carried out using forward and backward translation method. The psychometric properties of a pretested new Sinhala version of PPOS (PPOS-Sinhala) was tested in a convenience sample of 1367 patients and health professionals. Temporal stability was tested in a sub-sample of 140 individuals. The comparability of the PPOS scores and association with sex and level of education with those reported for Western populations were examined to establish construct validity. The sample included 543 medical students, 67 doctors, 335 allied health students and 422 patients. Cronbach's alpha for these groups ranged from 0.48 to 0.53 for sharing, 0.42 to 0.53 for caring, and 0.62 to 0.65 for total score. Intraclass correlation coefficients of 0.56, 0.6, and 0.4 were observed for the Total, Sharing, and Caring sub-scales, respectively. PPOS scores did not vary significantly for men and women. Health professionals exhibited higher scores than patients. Age was negatively associated and education level was positively associated with PPOS. PPOS-Sinhala is stable, sufficiently valid and reliable to evaluate patient centeredness among Sinhala speaking health care professionals and patients. Lower internal consistency is found for a few items in the instrument which requires further development. PPOS scores and their correlates for this Sri Lankan population were more similar to that found in other populations in this region than for scores and correlates found in the US.

The Development and Validation of the Social Networking Experiences Questionnaire: A Measure of Adolescent Cyberbullying and Its Impact.

The measurement of cyberbullying has been marked by several inconsistencies that lead to difficulties in cross-study comparisons of the frequency of occurrence and the impact of cyberbullying. Consequently, the first aim of this study was to develop a measure of experience with and impact of cyberbullying victimization in social networking sites in adolescents. The second aim was to investigate the psychometric properties of a purpose-built measure (Social Networking Experiences Questionnaire [SNEQ]). Exploratory factor analysis on 253 adolescent social networking sites users produced a six-factor model of impact. However, one factor was removed because of low internal consistency. Cronbach's alpha was higher than .76 for the victimization and remaining five impact subscales. Furthermore, correlation coefficients for the Victimization scale and related dimensions showed good construct validity. The utility of the SNEQ for victim support personnel, research, and cyberbullying education/prevention programs is discussed.

Detecting Deception by Analyzing Written Statements in Korean

This paper delves into the effect of SCAN and its cross-linguistic applicability by analyzing written statements in Korean. For this research, we conducted an experiment in which truth tellers were asked to write a true statement about a staged event and liars a fabricated one about the same event. We analyzed these two types of written statements using the criteria of SCAN. The results (accuracy rate, 81.6%) indicate that SCAN is effective in detecting deception despite the low internal consistency level among coders (Cronbach’s alpha level, 0.577). It was also shown that the SCAN criteria are not universally applicable across languages as the mode of using pronouns in Korean yields no significant difference between truthful and deceptive statements.

Exploratory factor analysis of the functional movement screen in elite athletes

The functional movement screen is developed to examine individuals’ movement patterns through 7 functional tasks. The purpose of this study was to identify the internal consistency and factor structure of the 7 tasks of the functional movement screen in elite athletes; 290 elite athletes from a variety of Chinese national teams were assessed using the functional movement screen. Cronbach’s alpha was calculated for the scores of the 7 tasks. Exploratory factor analysis was performed to explore the factor structure of the functional movement screen. The mean and standard deviation of the sum score were 15.2 ± 3.0. A low Cronbach’s alpha (0.58) was found for the scores of the 7 tasks. Exploratory factor analysis extracted 2 factors with eigenvalues greater than 1, and these 2 factors explained 47.3% of the total variance. The first factor had a high loading on the rotatory stability (loading = 0.99) and low loadings on the other 6 tasks (loading range: 0.04–0.34). The second factor had high loadings on the deep squat, hurdle step and inline lunge (loading range: 0.46–0.61) and low loadings on the other 3 tasks (loading range: 0.12–0.32). The 7 tasks of the functional movement screen had low internal consistency and were not indicators of a single factor. Evidence for unidimensionality was not found for the functional movement screen in elite athletes. More attention should be paid to the score of each task rather than the sum score when we interpret the functional movement screen scores.

Reliability and Validity of the Spanish Version of the Substance Dependence Severity Scale

Purpose: The purpose of this study was to estimate the reliability and provide evidence for the validity of the Spanish version of the Substance Dependence Severity Scale (SDSS). Methods: A total of 211 substance abusers recruited at a public center that provides treatment for substance-related disorders in an outpatient setting were assessed. Reliability was estimated by Cronbach's α and test-retest. Validity evidence was studied by analyzing the relationships with the European Addiction Severity Index (EuropASI) and Health-Related Quality of Life for Drug Abusers (HRQoLDA). Results: Adequate reliability coefficients were found for the dependence scales of all addressed substances (α = 0.737 to 0.877; test-retest r = 0.796-0.952). Low internal consistency was found for the abuse scales (α = 0.329-0.694), and adequate test-retest coefficients on alcohol, cocaine and heroin (test-retest r = 0.708-0.902). The reliability of the cannabis abuse scales was inadequate. The SDSS scores showed significant relationships with the EuropASI and HRQoLDA dimensions. Conclusions: The psychometric analyses validate the use of the severity of dependence scales. The SDSS abuse scales must be used taking the limitations detected into consideration.

Maintaining factors of posttraumatic stress symptoms following childbirth: A population-based, two-year follow-up study

BackgroundPrevious research has established a number of risk factors that are associated with the onset of PTSD following childbirth. However, little is known about factors that maintain PTSD symptoms. MethodsThis study is part of the Akershus Birth Cohort. Questionnaire data from pregnancy week 17, 8 weeks postpartum and 2 years postpartum were used. 1473 women completed all these three questionnaires and were included in the analyses. Post-traumatic stress symptoms were measured with the Impact of Event Scale. Potential maintaining factors were personality, sleep, support and life events.The factors that were significantly correlated with post-traumatic stress symptoms were entered into regression analyses. Mediation analyses were run to test whether significant predictors would serve as mediator of post-traumatic stress symptoms at 8 weeks postpartum to post-traumatic stress symptoms at 2 years postpartum. ResultsWe found several low to moderate associations between maintaining factors and PTSD symptoms two years postpartum. Adjusting for the starting point – PTSD symptoms 8 weeks postpartum – only insomnia remained significantly associated. Further, insomnia mediated a small portion of the effect of PTSD symptoms 8 weeks postpartum to PTSD symptoms 2 years postpartum. LimitationsLimitations of the study include a relative homogeneous sample, modest effect sizes, low internal consistency of some of the measures and the challenge to distinguish insomnia from PTSD symptoms. ConclusionsTreatment of postpartum PTSD might benefit from addressing insomnia if present. Alleviating insomnia may itself reduce daytime symptoms of PTSD and it may also increase the efficacy of primary PTSD treatments.

Five Factor Model in Iranian Culture: A Psychometrics Analysis of NEO-Five Factor Inventory (NEO-FFI)

This study aimed to investigate the construct validity and factor structure of NEO-Five Factor Inventory (Costa & McCrae, 1992) in Iranian population. Participants were 1639 (780 male, 859 female) Tehran people aged 15-71. The results of explanatory factor analysis showed no notable differences between the factor structures extracted by oblique and orthogonal rotations and didn’t replicate the scoring key. The Openness and Agreeableness had more psychometric problems (low internal consistency and high deleted items). The female’s NEO-FFI factor structure (with 41 items of 60 loaded on intended factors)was clearer than males’ (with 37 items). Confirmatory factor analysis supported the male’s latent modeling of the 31-item but failed to fit the female’s model. The women scored significantly higher in the Neuroticism, Openness, Agreeableness, and Conscientiousness than men who scored significantly higher in the Extraversion. As previous findings, the current results showed the NEO-FFI’s cultural limitations assessing the universality of the Five Factor Model.

Developing an outcome measure for excessive saliva management in MND and an evaluation of saliva burden in Sheffield

There are few studies providing evidence to guide the management of oropharyngeal secretion problems in motor neuron disease (MND). There is a lack of a suitable outcome measure for evaluating management strategies. We applied several potential outcome measures for assessing excessive secretions to patients with MND who attended the Sheffield Care and Research Centre for Motor Neurone Disease between 21 November 2012 and 15 May 2013. These measures were the CSS-MND, a symptom rating scale, and the Drool and Wipe quotient, which were designed to semi-objectively measure patients’ drooling. Of the 143 patients seen in clinic during the study period, 58 had symptoms of excessive secretions, and of whom 50 agreed to participate in the study. Semi-objective measures failed to effectively identify patients complaining of secretion problems. The CSS-MND had a relatively low internal consistency (Cronbach's alpha 0.539; n = 50); however, analysis of the inter-item correlations suggested the appearance of low internal consistency was because the scale was measuring a variety of saliva related symptoms that did not necessarily influence each other. The scale correlated well with patient reported symptom impact (r = 0.673, n = 50). In conclusion, the CSS-MND would be a useful outcome measure in studies assessing the management of oropharyngeal secretion problems.

δ18O in the tropical conifer Agathis robusta records ENSO-related precipitation variations.

Long-lived trees from tropical Australasia are a potential source of information about internal variability of the El Niño-Southern Oscillation (ENSO), because they occur in a region where precipitation variability is closely associated with ENSO activity. We measured tree-ring width and oxygen isotopic composition (O) of -cellulose from Agathis robusta (Queensland Kauri) samples collected in the Atherton Tablelands, Queensland, Australia. Standard ring-width chronologies yielded low internal consistency due to the frequent presence of false ring-like anatomical features. However, in a detailed examination of the most recent 15 years of growth (1995–2010), we found significant correlation between O and local precipitation, the latter associated with ENSO activity. The results are consistent with process-based forward modeling of the oxygen isotopic composition of -cellulose. The O record also enabled us to confirm the presence of a false growth ring in one of the three samples in the composite record, and to determine that it occurred as a consequence of anomalously low rainfall in the middle of the 2004/5 rainy season. The combination of incremental growth and isotopic measures may be a powerful approach to development of long-term (150+ year) ENSO reconstructions from the terrestrial tropics of Australasia.

Development of a 15-item scale to measure parental perceptions of their neighborhood.

Socioecological theory suggests that there are a range of influences that affect the physical activity levels of children, including parents' perceptions of the neighborhood. A questionnaire instrument to quantify parental neighborhood perceptions was developed for the Location of Children's Activity in Their Environment study as a potential predictor of children's physical activity. A literature review revealed a lack of appropriate instruments, many containing highly localized items. Following initial instrument development including pretesting, where items showed face and content validity, a two-phase pilot study was undertaken. Pilot testing investigated test-retest reliability (overall intraclass correlation coefficient = .75) and internal consistency (α = .62 and α = .73 for the two phases, respectively), finding that the instrument was acceptable overall although some changes in wording were made to specific items with low reliability and/or internal consistency. Using data from the first year of the Location of Children's Activity in Their Environment study, the instrument showed acceptable internal consistency (overall α = .73) and predictive validity (associated with time living in residence and walking to school) to be used in future studies. The scale was not found to be associated with child physical activity, which may be due to activity displacement. Proposed subscales for the neighborhood infrastructure and social aspects were marginal for internal consistency and a safety subscale was not acceptable in this regard. Given the impact of parental perceptions of the neighborhood, and its potential modifiable nature, a measure of it should be included in studies looking at the intensity and location of children's physical activity.

A dynamic Thurstonian item response theory of motive expression in the picture story exercise: solving the internal consistency paradox of the PSE.

The measurement of implicit or unconscious motives using the picture story exercise (PSE) has long been a target of debate in the psychological literature. Most debates have centered on the apparent paradox that PSE measures of implicit motives typically show low internal consistency reliability on common indices like Cronbach's alpha but nevertheless predict behavioral outcomes. I describe a dynamic Thurstonian item response theory (IRT) model that builds on dynamic system theories of motivation, theorizing on the PSE response process, and recent advancements in Thurstonian IRT modeling of choice data. To assess the models' capability to explain the internal consistency paradox, I first fitted the model to archival data (Gurin, Veroff, & Feld, 1957) and then simulated data based on bias-corrected model estimates from the real data. Simulation results revealed that the average squared correlation reliability for the motives in the Thurstonian IRT model was .74 and that Cronbach's alpha values were similar to the real data (<.35). These findings suggest that PSE motive measures have long been reliable and increase the scientific value of extant evidence from motivational research using PSE motive measures.

Validation française de la Revised Paranormal Belief Scale

For the last decades, many researchers have focused on paranormal beliefs. Beliefs in the existence of paranormal phenomena would be common and studies conducted in westernized countries have highlighted a high prevalence of individuals believing in the existence of such phenomena. Tobacyk and Milford (1984) developed the Revised Paranormal Belief Scale (RPBS) for assessing beliefs in paranormal phenomena. This 26-item self-reported questionnaire, measuring beliefs in phenomena such as witchcraft or superstition, is one of the most widely used questionnaires to assess such beliefs. While studies focusing on paranormal beliefs tend to develop, there is no French self-report instrument to assess this construct. Researchers have tried to identify specific variables that might be linked to such beliefs, and some have focused on personalities of individuals who believe in the paranormal. Schizotypy has been reported to be significantly and positively correlated with paranormal beliefs. The aim of this study was a) to validate the French version of the RPBS and b) to explore the relationship between Schizotypal Personality Disorder traits and paranormal beliefs. After being recruited using the Internet and social networks (e.g. facebook), a sample of 313 participants (mean [SD] age=31.12 [11.62]; range 18-58years) completed the Schizotypal Personality Questionnaire (SPQ-B), assessing Schizotypal Personality Disorder traits and the Revised Paranormal Belief Scale assessing paranormal beliefs. Confirmatory factor analysis (CFA) was conducted to test the proposed 7-factor structure of the RPB developed by Tobacyk. Several adjustment indices were used to evaluate the model. As the first model did not fit the original one, others models were tested. Our findings indicated that a seven-factor solution, excluding 2 items, best described the item structure: (1) spiritualism, (2) superstition, (3) witchcraft, (4) precognition, (5) traditional religious belief, (6) psi, (7) and extraordinary life forms. Relationships between paranormal beliefs and Schizotypal Personality Disorder traits were also explored. Correlations between some subscales of the SPQ-B and some dimensions of the RPBS were found. The "cognition-perception" subscale was strongly correlated with the "witchcraft", "spirituality", "precognition" and "psi" subscales. Nevertheless, this subscale was lightly correlated with the "traditional religious belief" and "extraordinary life forms" dimensions. No correlation was found between the others dimensions of schizotypy as "disorganized" and "interpersonal" and dimensions of paranormal beliefs. The initial model developed by Tobacyk and Milford did not fit the data from the French population. The low internal consistencies regarding both superstition and extraordinary life forms dimensions highlighted some cultural differences that have to be acknowledged. This result emphasizes that beliefs in some extraordinary life forms as Loch Ness monster are not an important dimension to take into account in a French population. Our findings also indicate that the RPBS is a valid and reliable instrument for assessing paranormal beliefs in a French population. Our findings also highlight that such beliefs are associated with Schizotypal Personality Disorder traits. Developing research on the association between Schizotypal Personality Disorder traits and such beliefs appears of prime importance. Future studies focusing on features associated with paranormal beliefs are also warranted.

The use of the FACT-H&N (v4) in clinical settings within a developing country: A mixed method study

In the last decade there has been an increasing awareness about ‘quality of life’ (QOL) of cancer survivors in developing countries. The study aimed to cross-culturally adapt and validate the FACT-H&N (v4) in Urdu language for Pakistani head and neck cancer patients. In this study the ‘same language adaptation method’ was used. Cognitive debriefing through in-depth interviews of 25 patients to assess semantic, operational and conceptual equivalence was done. The validation phase included 50 patients to evaluate the psychometric properties. The translated FACT-H&N was easily comprehended (100%). Cronbach's alpha for FACT-G subscales ranged from 0.726 – 0.969. The head and neck subscale and Pakistani questions subscale showed low internal consistency (0.426 and 0.541 respectively). Instrument demonstrated known-group validity in differentiating patients of different clinical stages, treatment status and tumor sites (p < 0.05). Most FACT summary scales correlated strongly with each other (r > 0.75) and showed convergent validity (r > 0.90), with little discriminant validity. Factor analysis revealed 6 factors explaining 85.1% of the total variance with very good (>0.8) Kaiser-Meyer-Olkin and highly significant Bartlett's Test of Sphericity (p < 0.001). The cross-culturally adapted FACT-H&N into Urdu language showed adequate reliability and validity to be incorporated in Pakistani clinical settings for head and neck cancer patients.

Psychometric characteristics of the Night Eating Questionnaire in a Middle East population.

To translate and validate the Night Eating Questionnaire (NEQ) into Arabic and to describe the distribution of night eating in an Egyptian sample. The NEQ was translated into Arabic and administered to 420 university students in Cairo, Egypt. Reliability, validity, and clinical cut-scores were generated for the Arabic NEQ. Distribution and demographic information were examined. Cronbach's alpha for the Arabic NEQ was 0.54. A first order confirmatory factor analysis yielded five factors: 1: Nocturnal Ingestions, Factor 2: Evening Hyperphagia, Factor 3: Morning Anorexia, Factor 4: Insomnia, and Factor 5: Mood. A total NEQ score of 26 was determined to be very high after T-score transformation. Of the students, 4.8% scored above this cut score, and no differences for total NEQ score were found for demographic variables. The Arabic NEQ generated a lower internal consistency score as compared to previous translations, but a similar factor structure. A cut score of 26 is similar to that determined significant for screening purposes in the original English version, and the proportion of those scoring above it is also similar to those of several international community samples. More research is needed to characterize night eating syndrome, its symptoms, and clinical impact in the Arabic culture.

Evaluation of the Spoken Knowledge in Low Literacy in Diabetes Scale for Use With Mexican Americans

This article evaluates the Spoken Knowledge in Low Literacy in Diabetes (SKILLD) questionnaire, a measure of essential knowledge for type 2 diabetes self-management, after it was modified for English- and Spanish-speaking Mexican Americans. We collected surveys (SKILLD, demographic, acculturation) and blood for A1C analysis from 72 community-recruited participants to analyze the SKILLD's internal consistency, interrater reliability, item analysis, and construct validity. Clinical experts evaluated content validity. The SKILLD demonstrated low internal consistency but high interrater reliability and content and construct validity. There were significant correlations in expected directions between SKILLD scores and acculturation, education, and A1C and significant differences in SKILLD scores between and within groups after an educational intervention and between high- and low-acculturated participants. The SKILLD generates useful information about Mexican Americans' diabetes knowledge. Lower SKILLD scores suggest less diabetes knowledge, lower health literacy, and participants' difficulties understanding items. Further modifications should improve use with low-acculturated Mexican Americans.

Interpersonal stress, not depression or hopelessness, predicts suicidality in university students in Macao

Joiner's Interpersonal Theory of Suicide, which states that an individual will desire death when they perceive themselves to be a burden to others, and experience social isolation, has received much support, but has not been directly assessed in non-Eurocentric individuals. Joiner's Interpersonal Needs Questionnaire (measuring both perceived burdensomeness and thwarted belongingness) and measures of depression, hopelessness, gender, relationship status and region of residence were evaluated as risk factors for suicidality after controlling for response biases. Participants were 273 undergraduate university students of Chinese ethnicity between the ages of 17 and 23 years in Macao. The predictors in sum distinguished between the presence and absence of suicidality (χ(2) = 62.759, p < .01). Perceived burdensomeness (p < .01) and relationship status (p < .01) made significant contributions to the presence of suicidality. Low internal consistency prevented thwarted belongingness from being assessed as a risk factor. Interpersonal variables may account for the seemingly disparate risk factors for suicide.

Measuring Hope in the Philippines: Validating the Short Version of the Locus-of-Hope Scale in Filipino

A short version of the locus-of-hope Scale was developed and translated into the Filipino language for the purpose of assessing hope in Filipinos who have lower levels of education and who may not have adequate levels of proficiency in English. The 20-item scale had four locus-of-hope subscales (internal, external-family, external-peers, and external-spiritual) each with four items (another four were filler items); all items described thoughts associated with some capacity and strategy to attain important goals in life. A sample (N = 362) of adults from communities in Metro Manila were asked to answer the scale, as well as measures of optimism, self-esteem, and life satisfaction. Confirmatory factor analysis indicated an adequate fit between the hypothesized four-factor structure and the data, but the internal locus-of-hope scale was found to have low internal consistency. Regression analysis showed positive associations between the internal and external-family locus-of-hope subscales and the three measures of wellbeing. But external-peers locus-of-hope was negatively associated with optimism and self-esteem, and external-spiritual locus-of-hope was positively related with optimism.