Substantial Internal Consistency Research Articles

The Development and Validation of the Emotionally Focused Therapy Skills Rubric

This article outlines the initial psychometric evidence for the emotionally focused therapy skills rubric (EFTSR). The EFTSR is designed to assess clinician's emotionally focused therapy (EFT) skill proficiency, identify how EFT skills are used within the basic movements of EFT, and evaluate a clinician's ability to use the EFT attachment injury resolution model . The EFTSR content validity was confirmed with two overall content validity indexes of .86 and .87. There was substantial item-level internal consistency reliability, with α scores ranging from .70 to .99 for both videos. Item-total correlations were higher than .20 for most items demonstrating substantial interrater reliability. Thus, results indicate initial support for using the EFTSR as theoretically based instrument designed to assess counselors’ ability to execute the skills of the EFT model.

Translation, adaptation, and psychometric properties of the Brazilian-Portuguese version of the Quality of Life in Children with Vernal Keratoconjunctivitis questionnaire.

The prevalence of ocular allergy varies according to the population and location of the study. Severe forms of ocular allergy are associated with compromised quality of life. In this study, we aimed to evaluate the application of the Brazilian-Portuguese version of the Quality of Life in Children with Keratoconjunctivitis questionnaire to children and adolescents with different subtypes of allergic conjunctivitis. A total of 48 patients (aged 5-12 years) with allergic conjunctivitis were included in this study. They were enrolled and monitored at a specialized center. After the clinical appointment, the children responded to the questionnaire on two occasions at an interval of 30 days. Individual scores (ranging from 0 to 3) of the 16 items were added. The Brazilian-Portuguese version of the Quality of Life in Children with Keratoconjunctivitis questionnaire demonstrated good translation, adaptation, and intellectual properties, with substantial internal consistency (Cronbach's α coefficient = 0.702). There was no significant difference between the responses of the two interviews, revealing good reproducibility. The moderate/severe forms of allergic conjunctivitis had significantly higher quality of life scores (indicating a poorer quality of life) than the mild forms. The Brazilian-Portuguese version of the Quality of Life in Children with Keratoconjunctivitis proved to be quick, reliable, and reproducible for assessing the quality of life in children with allergic conjunctivitis. However, its ability to detect changes resulting from symptom aggravation or treatment needs to be further evaluated.

Translation and cross-cultural adaptation of the Noise Exposure Questionnaire (NEQ) to Brazilian Portuguese.

To translate and cross-culturally adapt the Noise Exposure Questionnaire (NEQ) and 1-Minute Noise Screen (NEQ-S) instruments to Brazilian Portuguese. Procedures widely known in health research were used in the translation and cross-cultural adaptation process, comprising the following steps: initial translation, synthesis of translations, back-translation, expert committee, pretest, and content and layout validation. Altogether, 60 workers participated in the pretest by answering the questionnaires and then evaluating them in terms of understandability, layout, clarity, and writing. Reliability was verified with Cohen's kappa test, and the internal consistency was analyzed with Cronbach's alpha coefficient. The translated and adapted versions of NEQ and NEQ-S were similar in terms of general and referential meanings. However, some modifications and adaptations were made to adapt them to the Brazilian reality. The kappa test indicated moderate agreement and Cronbach's alpha coefficient, substantial internal consistency. The translation and cross-cultural adaptation were carried out according to the methodology recommended in the national and international literature, performing the necessary equivalences to maintain the face and content validity with the original instrument. The availability of NEQ and NEQ-S in Brazilian Portuguese opens new fields of research to quantify yearly noise exposure more in-depth.

Reliability and validity of the international standards for neurological classification of spinal cord injury in patients with non-traumatic spinal cord lesions.

Prospective, observational study. The International Standards for Neurological Classification of Spinal Cord Injury (ISNCSCI) represent the gold standard for the assessment of patients with spinal cord injury (SCI) and their measurement properties have been evaluated in patients with traumatic lesions. Albeit the ISNCSCI are widely used also for the assessment and prognosis of patients with non-traumatic SCI, a validation of this grading system in this sample has never been performed. Therefore, the aim of this study is to evaluate the measurement properties of the ISNCSCI in a population of persons with non-traumatic SCI. Three Italian rehabilitation hospitals. The sample included 140 patients with non-traumatic SCI of different etiology, level and grade, for a total of 169 evaluations performed by two examiners. Cronbach's Alpha was used to evaluate the internal consistency of the ISNCSCI various components. The agreement between two examiners of each center in the definition of different components was used to assess the inter-rater reliability. The construct validity was evaluated through the correlation of the ISNCSCI with the Spinal Cord Independence Measure (SCIM). The ISNCSCI showed substantial internal consistency, and substantial inter-rater agreement for AIS grade, cumulative motor and sensory scores. The motor scores for upper and lower extremity showed fair to moderate correlation with SCIM self-care and motility subscores, respectively. The ISNCSCI total motor score correlated with the total SCIM score. Our study demonstrates that the ISNCSCI are a valid and reliable tool for the assessment of patients with non-traumatic SCI.

Psychedelic Communitas: Intersubjective Experience During Psychedelic Group Sessions Predicts Enduring Changes in Psychological Wellbeing and Social Connectedness.

Background: Recent years have seen a resurgence of research on the potential of psychedelic substances to treat addictive and mood disorders. Historically and contemporarily, psychedelic studies have emphasized the importance of contextual elements ('set and setting') in modulating acute drug effects, and ultimately, influencing long-term outcomes. Nevertheless, current small-scale clinical and laboratory studies have tended to bypass a ubiquitous contextual feature of naturalistic psychedelic use: its social dimension. This study introduces and psychometrically validates an adapted Communitas Scale, assessing acute relational experiences of perceived togetherness and shared humanity, in order to investigate psychosocial mechanisms pertinent to psychedelic ceremonies and retreats. Methods: In this observational, web-based survey study, participants (N = 886) were measured across five successive time-points: 2 weeks before, hours before, and the day after a psychedelic ceremony; as well as the day after, and 4 weeks after leaving the ceremony location. Demographics, psychological traits and state variables were assessed pre-ceremony, in addition to changes in psychological wellbeing and social connectedness from before to after the retreat, as primary outcomes. Using correlational and multiple regression (path) analyses, predictive relationships between psychosocial 'set and setting' variables, communitas, and long-term outcomes were explored. Results: The adapted Communitas Scale demonstrated substantial internal consistency (Cronbach's alpha = 0.92) and construct validity in comparison with validated measures of intra-subjective (visual, mystical, challenging experiences questionnaires) and inter-subjective (perceived emotional synchrony, identity fusion) experiences. Furthermore, communitas during ceremony was significantly correlated with increases in psychological wellbeing (r = 0.22), social connectedness (r = 0.25), and other salient mental health outcomes. Path analyses revealed that the effect of ceremony-communitas on long-term outcomes was fully mediated by communitas experienced in reference to the retreat overall, and that the extent of personal sharing or ‘self-disclosure’ contributed to this process. A positive relationship between participants and facilitators, and the perceived impact of emotional support, facilitated the emergence of communitas. Conclusion: Highlighting the importance of intersubjective experience, rapport, and emotional support for long-term outcomes of psychedelic use, this first quantitative examination of psychosocial factors in guided psychedelic settings is a significant step toward evidence-based benefit-maximization guidelines for collective psychedelic use.

Development and validation of the nursing clinical assessment tool (NCAT): a psychometric research study.

The aim of this pilot psychometric study was to develop and assess the reliability, validity, and applicability of a nursing clinical assessment tool (NCAT) to assess students' clinical performance in the clinical training. Total of 325 forms were completed by 36 clinical raters for undergraduate nursing students who were selected using convenience sampling. International quality and safety education for nurses (QSEN) and Benson and Clark's (Benson, J., & Clark, F. (1982). A guide for instrument development and validation. The American Journal of Occupational Therapy, 36(12), 789-800) model guided the construction of the NCAT. Following literature review, a cross-cultural adaptation and psychometric testing were used. This newly developed tool showed substantial overall internal consistency reliability (Cronbach's alpha = 0.74). Three factors (i.e. Care Provider, Leadership & Team Work and Professionalism and Ethical Considerations) were identified for construct validity using exploratory factor analysis (EFA). This study suggests that the NCAT is a reliable, valid and applicable tool. Itcan be used in clinical and educational settings. Further testing of the tool in similar and different cultures is recommended.

Influence of recurrent vulvovaginal candidiasis on quality of life issues.

More than 370 million women will experience recurrent vulvovaginal candidiasis (RVVC) during their lifetime. However, RVVC is often trivialized as clinically insignificant and not worthy of research funding. We evaluated the influence of RVVC on the quality of life in affected women. The validated World Health Organization Quality of Life Abbreviated Assessment (WHOQOL-Bref) questionnaire was administered to 100 women with RVVC and to 101 epidemiologically matched women with no history of vulvovaginal candidiasis. RVVC was defined as at least four episodes of clinical and culture-positive vaginal candidiasis within a 1year period. Data were analyzed by Chi square, Student t test and analysis of variance. Internal consistency of responses to questions was evaluated by Cronbach alpha. The Cronbach alpha coefficient was > 0.80 for responses to generalized questions and > 0.65 for answers to more specific questions, indicating substantial internal consistency. Perception of quality of life and satisfaction with their health was greatly reduced in the RVVC group (p < 0.001). Diminished responses to physical and psychological well-being were also reported by women with RVVC (p < 0.001). Various aspects of social relations including sexual activity were similarly reduced (p < 0.001) as were satisfaction with issues such as home environment, financial resources and employment (p < 0.001). RVVC affects multiple aspects of a woman's well-being. Women with this condition deserve serious attention from clinicians and research into susceptibility, prevention and treatment of this infection deserves much greater emphasis.

Reliability and validity of the Turkish version of Pain Assessment Checklist for Seniors with Limited Ability to Communicate (PACSLAC-T)

This study aimed to assess validity and reliability of the Turkish version of Pain Assessment Checklist for Seniors with Limited Ability to Communicate (PACSLAC-T). The individuals who met inclusion criteria of the study were in patients of a hospital and a long-term care facility. Mini Mental Status Exam (MMSE), Cornell Dementia Depression Scale (CDDS), Global Deterioration Scale (GDS), visual analogue scale (VAS), and PACSLAC-T were administered to all subjects. The scales were repeated with an interval of two weeks for test–retest reliability. A total of 112 patients with dementia were included in the study. The intraclass correlation coefficient ICC for test–retest reliability of the PACSLAC-T was 0.713 with a 95% confidence interval of 0.486–0.843. The Cronbach’s α coefficient for total PACSLAC-T was 0.842 for test and 0.888 for retest, which indicated substantial internal consistency. In convergent validity, there were significant correlations between PACSLAC-T total score VAS (r = 0.684, P < 0.001), while no correlation was found between PACSLAC-T total score and CDDS (r = 0.127, P = 0.094), and GDS (r = 0.096, P = 0.167). Also, significant correlations were found between PACSLAC-T total score and MMSE (r = –0.468, P = 0.016). This study showed that PACSLAC-T could be a promising tool for the management of pain in older adults with limited communication skills.

The Development and Psychometric Evaluation of the Group Schema Therapy Rating Scale - Revised.

Recent research has supported the efficacy of schema therapy as a treatment for personality disorders. A group format has been developed (group schema therapy; GST), which has been suggested to improve both the clinical and cost-effectiveness of the treatment. Efficacy studies of GST need to assess treatment fidelity. The aims of the present study were to improve, describe and evaluate a fidelity measure for GST, the Group Schema Therapy Rating Scale - Revised (GSTRS-R). Following a pilot study on an initial version of the scale (GSTRS), items were revised and guidelines were modified in order to improve the reliability of the scale. Students highly experienced with the scale rated recorded GST therapy sessions using the GSTRS-R in addition to a group cohesion measure, the Harvard Community Health Plan Group Cohesiveness Scale - II (GCS-II). The scores were used to assess internal consistency and inter-rater reliability. Discriminant validity was assessed by comparing the scores on the GSTRS-R with the GCS-II. The GSTRS-R displayed substantial internal consistency and inter-rater reliability, and adequate discriminate validity, evidenced by a weak positive correlation with the GCS-II. Overall, the GSTRS-R is a reliable tool that may be useful for evaluating therapist fidelity to GST model, and assisting GST training and supervision. Initial validity was supported by a weak association with GCS-II, indicating that although associated with cohesiveness, the instrument also assesses factors specific to GST. Limitations are discussed.

Reliability and validity of severity dimensions of psychopathology assessed using the Structured Clinical Interview for DSM-5 (SCID).

This study examined whether the Structured Clinical Interview for DSM (SCID), a widely used semistructured interview designed to assess psychopathology categorically, can be adapted to identify reliable and valid severity dimensions of psychopathology. The present study also examined whether these severity dimensions have better psychometric properties (internal consistency, test-retest reliability, and concurrent and predictive validity) than categorical diagnoses. Participants (N=234) were recruited from the community and clinics. Retest reliability and prospective predictive validity (symptoms and functioning 1year later) were examined in subsamples of participants. Dimensional severity scales were created from an adapted version of the SCID for both current and lifetime major depression, alcohol, substance, post-traumatic stress disorder, panic, agoraphobia, social anxiety, specific phobia, obsessive-compulsive disorder, and generalized anxiety disorder. The SCID's severity scales demonstrated substantial internal consistency (all Cronbach's αs >.80), test-retest reliability, and concurrent and predictive validity. Symptom severity scales demonstrated significant incremental validity over and above categorical diagnoses for both current and prospective outcomes. The psychometric properties of SCID-identified symptom scales were far superior to the psychometrics of categorical diagnoses for both current and lifetime psychopathology. These results highlight the feasibility and utility of the SCID to assess reliable and valid symptom severity dimensions of both current and lifetime psychopathology.

DEVELOPMENT AND VALIDATION OF THE TAIWAN CHILDREN'S ENVIRONMENTAL ACTION INDEX

In this study, we use Smith-Sebasto & Fortner’s (1994) Environmental Action Internal Control Index (EAICI) as a framework to develop, and validate a useful instrument for assessing environmental attitudes and behavior among elementary and middle school children within the Taiwanese context. We dub the new instrument the Taiwan Children’s Environmental Action Index (TCEAI). Our findings suggest that the TCEAI displays substantial internal consistency (Cronbach's alpha=.92), moderately positive correlations with self-report measures of environmentally responsible behavior (R = .35 to .46, p < .01), and few threats to validity by age or gender. The results suggest that the TCEAI may be used to elicit important dimensions of environmental attitudes and to predict environmentally responsible behavior for elementary and middle school children in Taiwan. Practical implications are discussed.

Finnish version of the Tampa Scale of Kinesiophobia: Reference values in the Finnish general population and associations with leisure-time physical activity.

To create reference values for the general Finnish population using the Tampa Scale of Kinesiophobia (TSK-FIN), to study gender differences in the TSK-FIN, to assess the internal consistency of the TSK-FIN, to estimate the prevalence of high levels of kinesiophobia in Finnish men and women, and to examine the association between kinesiophobia and leisure-time physical activity and the impact of co-morbidities on kinesiophobia. The study population comprised 455 men and 579 women. Participants completed a self-administered questionnaire about their socio-demographic factors, leisure-time physical activity, co-morbidities and kinesiophobia. The mean TSK-FIN score was significantly higher for men (mean 34.2, standard deviation (SD) 6.9) compared with women (mean 32.9, SD 6.5), with an age-adjusted p = 0.004 for the difference between men and women. Cronbach's alpha was 0.72, indicating substantial internal consistency. Men over 55 years of age and women over 65 years of age had a higher (p < 0.001) TSK score compared with younger people. There was a significant (p < 0.001) inverse association between kinesiophobia and leisure-time physical activity among both sexes. The presence of cardiovascular disease, musculoskeletal disease or a mental disorder was associated with a higher TSK-FIN score compared with the absence of the aforementioned disorders. We present here the reference values for the TSK-FIN. The reference values and prevalence among the general population may help clinicians to define the level of kinesiophobia among patients. Disorders other than musculoskeletal diseases were associated with kinesiophobia, which should be noted in daily practice.

Validation of the children's leisure activities study survey questionnaire for 12-year old South African children

This study evaluated the reliability and construct validity of a modified Children’s Leisure Activities Study Survey (CLASS) questionnaire for 12-year old South African children. Test-retest reliability of the questionnaire was examined in 108 (n=45 boys; n=63 girls) school children aged 12 years at 3 weeks interval. Construct validity was assessed by comparing data on the reported minutes spent in physical activities (PAs) obtained from the CLASS questionnaire responses with ten physical fitness (PF) parameters, assessed using the Eurofit test battery. Cronbach alpha’s test of internal consistency, intra-class correlation coefficients (ICC) and the paired t-test were used to determine reliability. Validity was evaluated with the Spearman correlation coefficients. Substantial internal consistency and significant Intra-class correlations estimates for all intensities of PA and sedentary time (ST) were found (Cronbach’s alpha ranged from 0.71 to 0.84, p < 0.05; ICC ranged from 0.73 to 0.95, p < 0.05). Non-significant differences were found in the means of test and retest measurements. BMI had moderate negative correlations with both moderate and total PA (r = -0.34, r = -0.36, respectively, p < 0.05). Aerobic fitness was significantly correlated with all intensities of PA and ST (r = 0.63, 0.57, 0.43 and-0.34, respectively; p < 0.05). ST was significantly and negatively associated with sit-ups (r = -0.38, p<0.01) and shuttle run (r = -0.29, p<0.05). Similarly, the results of the shuttle run (10x5 meter run), sit-ups, bent arm hang, and standing broad jump tests, had significant associations with vigorous PA. The CLASS questionnaire is a valid and reliable measure of PA in 12-year old South African children. Keywords : CLASS questionnaire, validation, physical activity questionnaire, physical fitness, children.

Validation of the children's leisure activities study survey questionnaire for 12-year old South African children

This study evaluated the reliability and construct validity of a modified Children’s Leisure Activities Study Survey (CLASS) questionnaire for 12-year old South African children. Test-retest reliability of the questionnaire was examined in 108 (n=45 boys; n=63 girls) school children aged 12 years at 3 weeks interval. Construct validity was assessed by comparing data on the reported minutes spent in physical activities (PAs) obtained from the CLASS questionnaire responses with ten physical fitness (PF) parameters, assessed using the Eurofit test battery. Cronbach alpha’s test of internal consistency, intra-class correlation coefficients (ICC) and the paired t-test were used to determine reliability. Validity was evaluated with the Spearman correlation coefficients. Substantial internal consistency and significant Intra-class correlations estimates for all intensities of PA and sedentary time (ST) were found (Cronbach’s alpha ranged from 0.71 to 0.84, p < 0.05; ICC ranged from 0.73 to 0.95, p < 0.05). Non-significant differences were found in the means of test and retest measurements. BMI had moderate negative correlations with both moderate and total PA (r = -0.34, r = -0.36, respectively, p < 0.05). Aerobic fitness was significantly correlated with all intensities of PA and ST (r = 0.63, 0.57, 0.43 and-0.34, respectively; p < 0.05). ST was significantly and negatively associated with sit-ups (r = -0.38, p<0.01) and shuttle run (r = -0.29, p<0.05). Similarly, the results of the shuttle run (10x5 meter run), sit-ups, bent arm hang, and standing broad jump tests, had significant associations with vigorous PA. The CLASS questionnaire is a valid and reliable measure of PA in 12-year old South African children. Keywords : CLASS questionnaire, validation, physical activity questionnaire, physical fitness, children.

The reliability, validity, and applicability of an English language version of the Mini-ICF-APP

This study aimed at establishing the validity and reliability of an English language version of the Mini-ICF-APP. One hundred and five patients under the care of secondary mental health care services were assessed using the Mini-ICF-APP and several well-established measures of functioning and symptom severity. 47 (45%) patients were interviewed on two occasions to ascertain test-retest reliability and 50 (48%) were interviewed by two researchers simultaneously to determine the instrument's inter-rater reliability. Occupational and sick leave status were also recorded to assess construct validity. The Mini-ICF-APP was found to have substantial internal consistency (Chronbach's α 0.869-0.912) and all 13 items correlated highly with the total score. Analysis also showed that the Mini-ICF-APP had good test-retest (ICC 0.832) and inter-rater (ICC 0.886) reliability. No statistically significant association with length of sick leave was found, but the unemployed scored higher on the Mini ICF-APP than those in employment (mean 18.4, SD 9.1 vs. 9.4, SD 6.4, p<0.001). The Mini-ICF-APP correlated highly with the other measures of illness severity and functioning considered in the study. The English version of the Mini-ICF-APP is a reliable and valid measure of disorders of capacity as defined by the International Classification of Functioning. Further work is necessary to establish whether the scale could be divided into sub scales which would allow the instrument to more sensitively measure an individual's specific impairments.

Quality of care as a determinant of health-related quality of life in ill-hospitalized adolescents at a tertiary care hospital in North India

To evaluate if quality of care (QoC) provided by hospital is a determinant of ill-hospitalized adolescent's health-related quality of life (HRQoL) from parent's perspective. Prospective cross-sectional study conducted at a tertiary care hospital of Northern India after institutional ethical approval. Hospital in pediatric department of a tertiary care, teaching medical University in Lucknow, northern India. Sick adolescents aged between 10 and 19 years and hospitalized for four categories of illnesses, namely, acute infective; chronic infective, non-hemopoetic; hemopoetic disorders and miscellaneous. QOC assessment was done using 'Pyramid instrument' and HRQoL by culturally modified WHOQOL-BREF (World Health Organization Quality of Life-BREF). The Pyramid instrument comprises 43 questions to collect information about awareness of eight indices: namely illness, routines, accessibility, medical treatment, care processes, staff attitude, participation and staff work environment and scored on 1-4 Likert scale. WHOQOL-BREF has four domains: physical, psychological, social relations and environment and scored on 1-5 Likert scale. From January 2008 to December 2008, 300 adolescents with a mean age of 12.5 ± 2.6 years and 61.3% males were included. The pyramid instrument showed a substantial internal consistency (α = 0.88, P-value < 0.0001). The mean QoC was highest for medical treatment (0.76 ± 0.13) and lowest for participation (0.54 ± 0.16). The mean parent's report of child's HRQoL was highest for physical (42.8 ± 7.4) and lowest for environment domain (37.2 ± 7.1). Four QoC indices namely, medical treatment, care processes, staff attitude and participation had significant associations with the mean HRQoL. In a hierarchical linear regression, staff attitude was the only significant determinant of HRQoL (β coefficient: 23.16, 95% confidence interval: 15.8-30.5, P-value < 0.0001). The Pyramid instrument is a reliable instrument for assessing parent's perception of QoC provided to hospitalized adolescents in Indian context. QoC was positively associated with HRQoL, thus, an increased focus on QoC especially staff attitude is likely to enhance adolescent's overall HRQoL.

Measuring social desirability amongst men with intellectual disabilities: The psychometric properties of the Self- and Other-Deception Questionnaire—Intellectual Disabilities

BackgroundSocial desirability has been construed as either inaccurately attributing positive characteristics to oneself (self-deception), or inaccurately denying that one possesses undesirable characteristics to others (other-deception or impression management). These conceptualisations of social desirability have not been considered in relation to people with intellectual disabilities (IDs), but they are important constructs to consider when undertaking a psychological assessment of an individual, especially within forensic contexts. Therefore, we revised two existing measures of self- and other-deception and considered their psychometric properties. MethodsThirty-two men with mild IDs and 28 men without IDs completed the Self- and Other-Deception Questionnaires—Intellectual Disabilities (SDQ-ID and ODQ-ID) on two occasions, two weeks apart. ResultsMen with IDs scored significantly higher on the SDQ-ID and the ODQ-ID than men without IDs. However, these differences disappeared when Full Scale IQ, Verbal IQ and Performance IQ were controlled in relation to the SDQ-ID, and partially disappeared in relation to the ODQ-ID. The SDQ-ID and the ODQ-ID had substantial internal consistency in relation to men with IDs (k=0.82 and 0.84 respectively). The test-retest reliability of the SDQ-ID was good (ri=0.68), while the test-retest reliability of the ODQ-ID was moderate (ri=0.56), for men with IDs. The SDQ-ID had moderate (k=0.60) and the ODQ-ID had substantial (k=0.70) internal consistency in relation to men without IDs, while the test-retest reliability of the SDQ-ID was excellent (ri=0.87) as was the case for the ODQ-ID (ri=0.85). ConclusionsThe SDQ-ID and the ODQ-ID have satisfactory psychometric properties in relation to men with and without IDs. Future research using these instruments is proposed.

Psychometric Properties of the Spanish Version of the Therapeutic Collaboration Scale (TCS)

The development of appropriately normed Spanish-language assessments is a necessity as a result of (1) a growing Latino population in need of behavioral health services and (2) a limited number of linguistically and culturally sensitive instruments that are currently available. As part of a broader assessment on the needs and satisfaction with services received, 103 Latina women completed the Therapeutic Collaboration Scale (TCS). This measure attained a substantial internal consistency (Cronbach’s alpha =.94) and good convergent validity with another instrument that also assessed the therapeutic relationship named the Therapeutic Alliance with Clinician (r = .71, p &lt; .05). A principal component analysis evidenced one component underlying the TCS, which further confirmed the unidimensionality of this instrument. This measure achieved a good correlation with an instrument assessing the participants’ satisfaction with services (SWS, r = .44, p &lt; .05). Conclusions and implications of this study are presented in this article.

Development of the Attitudes Toward Osteopathic Principles and Practice Scale (ATOPPS): preliminary results.

Little empirical work has been done to examine how osteopathic medical students' attitudes toward osteopathic principles and practice (OPP) develop and evolve over the course of their medical education. A major obstacle to conducting this research is the absence of reliable and sensitive instruments to measure students' attitudes toward OPP. The purpose of this project is to develop a sensitive and reliable instrument to measure students' attitudes toward OPP. Face-to-face and telephone interviews were conducted with osteopathic medical students, osteopathic manipulative medicine (OMM) residents, OMM undergraduate fellows, and three board-certified OMM specialists. These interviews were summarized in a 39-item instrument administered to 127 students at the completion of their core OMM rotation at the University of North Texas Health Science Center at Fort Worth-Texas College of Osteopathic Medicine. Factor analysis of student responses to the 39 candidate items yielded two interpretable factors. Factor 1 contained 24 items and accounted for 33% of the item response variance, and factor 2 contained four items and accounted for 5.6% of the item response variance. Based on these results, 14 of the original 39 statements were eliminated and the smaller second factor was dropped. Factor 1 contained items reflecting both positive and negative attitudes toward the application of OPP in patient care, the importance of OPP in medical education, and professional distinctiveness. One of the original 39 candidate items was returned to factor 1 because it was judged by the investigators to be consistent with the underlying construct of the scale and helped balance the number of forward-scored and reversed-scored items in the final instrument. Two internal consistency estimates of reliability were computed for the revised 25-item Attitudes Toward Osteopathic Principles and Practice Scale (ATOPPS): the Spearman-Brown unequal-length corrected coefficient alpha and the split-half reliability coefficients. Estimates for the split-half coefficients were .89 for part 1 and .87 for part 2. The Spearman-Brown coefficient alpha was .93, indicating substantial internal consistency. The 25-item ATOPPS seems to reflect a continuum of positive and negative attitudes toward OPP. This preliminary report documents reliability for the 25-item ATOPPS. With continued support for its construct validity, ATOPPS provides investigators with a reliable tool to assess the development of attitudes toward OPP.

Consistency, inter-rater reliability, and validity of 441 consecutive mock oral examinations in anesthesiology: implications for use as a tool for assessment of residents.

Oral practice examinations (OPEs) are used extensively in many anesthesiology programs for various reasons, including assessment of clinical judgment. Yet oral examinations have been criticized for their subjectivity. The authors studied the reliability, consistency, and validity of their OPE program to determine if it was a useful assessment tool. From 1989 through 1993, we prospectively studied 441 OPEs given to 190 residents. The examination format closely approximated that used by the American Board of Anesthesiology. Pass-fail grade and an overall numerical score were the OPE results of interest. Internal consistency and inter-rater reliability were determined using agreement measures. To assess their validity in describing competence, OPE results were correlated with in-training examination results and faculty evaluations. Furthermore, we analyzed the relationship of OPE with implicit indicators of resident preparation such as length of training. The internal consistency coefficient for the overall numerical score was 0.82, indicating good correlation among component scores. The interexaminer agreement was 0.68, indicating moderate or good agreement beyond that expected by chance. The actual agreement among examiners on pass-fail was 84%. Correlation of overall numerical score with in-training examination scores and faculty evaluations was moderate (r = 0.47 and 0.41, respectively; P < 0.01). OPE results were significantly (P < 0.01) associated with training duration, previous OPE experience, trainee preparedness, and trainee anxiety. Our results show the substantial internal consistency and reliability of OPE results at a single institution. The positive correlation of OPE scores with in-training examination scores, faculty evaluations, and other indicators of preparation suggest that OPEs are a reasonably valid tool for assessment of resident performance.