Item-level Analyses Research Articles

The Needs and Provision Complexity Scale: a multicentre prospective cohort analysis of met and unmet needs and their cost implications for patients with complex neurological disability

ObjectiveTo provide a brief overview of the Needs and Provision Complexity Scale (NPCS) and report its first application to describe the level of ‘met’ and ‘unmet’ health/social care needs, and...

Scale- and Item-Level Factor Analyses of the VIA Inventory of Strengths

The VIA Inventory of Strengths (VIA-IS) has emerged as the primary instrument for gauging individual strengths and virtues. Prior studies have generated inconsistent results concerning the latent structure of the VIA-IS. The present study attempted to address some of these inconsistencies. VIA-IS results from a large sample (N = 458,998) of U.S. adults who completed the inventory online were subjected to a series of principal components and factor analyses. The sample was 66.46% female with a mean age of 34.36 years (SD = 14.13 years) and consistent with the general U.S. population in terms of geographic distribution. Information on ethnicity was not available. The size of the sample permitted both scale- and item-level analyses. The scale-level analyses produced findings similar to those of previous studies, but raised concerns about multidimensionality in the scales. Item-level analyses suggested an alternate set of 24 scales, 20 of which overlapped substantially with existing VIA-IS scales. A second-order analysis suggested five factors, including a new one labeled Future Orientation, versus the original six virtues proposed in the development of the VIA-IS. The results were used to speculate about elements of a second-generation model of strengths.

Outcomes of discharged females versus those waiting for discharge from Vlore Psychiatric Hospital (Albania)

This study examines the psychosocial outcomes of women discharged from the Vlore Psychiatric Hospital in Albania. The study was designed as a controlled, not randomized, follow-up study. It included 16 women diagnosed with psychosis who were discharged from a psychiatric hospital to live in group homes in the community. The control group included 20 women diagnosed with psychosis who lived at the psychiatric hospital while awaiting discharge. All subjects were assessed twice using the HoNOS-Rome tool, at the start of the study (T0) and 12 months later (T1). Both groups showed an improvement in the HoNOS total score between T0 and T1 (p < .001). This improvement was significantly larger in the Home group compared to the Hospital group (p = .014). An item-level analysis indicated a significantly greater improvement in the Home group on items 11 (autonomy), 12 (work), 16 (family) and 18 (goals). Projects for social inclusion of people suffering from psychosis must design living spaces that offer viable alternatives to psychiatric hospitals. This study also documents the efforts in the Vlore region of Albania to follow European standards of mental health care.

Methodologies for Investigating Item- and Test-Level Measurement Equivalence in International Large-Scale Assessments

In this study, the Canadian English and French versions of the Problem-Solving Measure of the Programme for International Student Assessment 2003 were examined to investigate their degree of measurement comparability at the item- and test-levels. Three methods of differential item functioning (DIF) were compared: parametric and nonparametric item response theory and ordinal logistic regression. Corresponding derivations of these three DIF methods were investigated at the test-level to examine both differential test functioning (DTF) and the correspondence between findings at the item-level with those at the test-level. Item-level findings suggested consistency in DIF detection across methods; however, differences in effect sizes of DIF were found by each method. Test-level results revealed a high degree of consistency across DTF methods. Discrepancies were found between item- and test-level comparability analyses. Item-level analyses suggested moderate to low degrees of comparability, whereas test-level findings suggested a higher degree of comparability. Findings also indicated the direction of DIF was mixed as some DIF items favored English-speaking students and others favored French-speaking students, suggesting that DIF cancellation may explain why item-level incomparability was not detected at the test-level.

Is non-suicidal self-injury an “addiction”? A comparison of craving in substance use and non-suicidal self-injury

There is debate among researchers regarding the most appropriate conceptual model of non-suicidal self-injury (NSSI). Some argue that NSSI is best viewed within an addictions framework. Because craving of substances is a key concept in the addictions literature, we sought to compare the nature of craving in NSSI and substance use. Measures of NSSI, substance use, and craving were administered to a sample of adolescents (n=58) receiving psychiatric treatment. It was found that total craving scores were significantly lower for NSSI than for substances. Item-level analyses suggested that substances are craved in a variety of contexts, whereas NSSI is typically craved in the context of negative emotions. The pattern of results remained the same when analyses were limited to patients who engaged in both NSSI and substance use. Thus, findings appear to be due to differences in the nature of the behaviors themselves rather than to individual differences between those who engage in NSSI or use substances. We conclude that, while both behaviors have powerful reinforcement contingencies, NSSI appears to be almost exclusively maintained by negative reinforcement (e.g., the reduction of aversive emotions). Findings are more consistent with emotion regulation than addiction models of NSSI.

Short Forms of the Social Interaction Anxiety Scale and the Social Phobia Scale

Mattick and Clarke's (1998) Social Interaction Anxiety Scale (SIAS) and Social Phobia Scale (SPS) are commonly used self-report measures that assess 2 dimensions of social anxiety. Given the need for short, readable measures, this research proposes short forms of both scales. Item-level analyses of readability characteristics of the SIAS and SPS items led to the selection of 6 items from each scale for use in the short forms. The SIAS and SPS short forms had reading levels at approximately the 6th and 5th grade level, respectively. Results using nonclinical (Study 1: N = 469) and clinical (Study 2: N = 145) samples identified these short forms as being factorially sound, possessing adequate internal consistency, and having strong convergence with their full-length counterparts. Moreover, these short forms showed convergence with other measures of social anxiety, showed divergence from measures assessing related constructs, and predicted concurrent interpersonal functioning. Recommendations for the use of these short forms are discussed.

Quality of life impairment in a community sample of women with eating disorders

Studies of quality of life among individuals with eating disorders have relied almost exclusively on clinical samples. We examined impairment in quality of life in a community sample of women with eating disorders recruited as part of an epidemiological study. Measures of health-related quality of life (Medical Outcomes Study 12-item Short-Form Physical and Mental Component Summary scales) and subjective well-being (WHOQOL-BREF Psychological Functioning and Social Relationships subscales) were completed by women with eating disorders (n = 159), primarily variants of bulimia nervosa and binge eating disorder, and a comparison group of healthy women (n = 232). When compared with healthy women, women with eating disorders reported substantial impairment in aspects of quality of life relating to mental health, although item-level analysis indicated considerable variation in the extent to which specific aspects of emotional well-being were affected. Impairment in social relationship and in physical health was less pronounced and due, at least in part, to between-group differences in age, body weight and demographic characteristics. Impairment in certain aspects of perceived physical health was, however, apparent among women with eating disorders, even after controlling for between-group differences in body weight. Community cases of women with eating disorders experience marked impairment in quality of life as this relates to mental health functioning and at least some impairment in physical health functioning. Personality characteristics and ego-syntonic aspects of eating-disordered behaviour may complicate the interpretation of findings relating to impairment in specific aspects of quality of life.

P-1240 - The possibility of quite early detection of schizophrenic children

Aim The subclinical characteristics of schizophrenic children have not been sufficiently investigated to present specific evidences. To elucidate the picture of them, and to find out indicators which predict later development of schizophrenia, childhood behaviors of the adult schizophrenia subjects were investigated in a questionnaire-based retrospective study. Method Schizophrenia outpatients (n = 30) in his/her twenties and normal healthy subjects (n = 200) were investigated. All patients are diagnosed according to DSM-IV-TR as schizophrenia, and who present now mainly negative symptoms after passing an acute stage. By modified use of the CBCL (Child Behavior Checklist) as a retrospective assessment questionnaire, the parents of the patients and of control subjects rated their childhood behavior. Results The hit-rate when classifying the schizophrenic and normal subjects by discriminant function using the syndrome scales of CBCL totaled 89,0%. Notable in an item-level analysis was an extremely attenuated aggression in personal relations in the schizophrenia subjects. Among eight subscales of the CBCL, those of Withdrawal, Social Problems, Attention Problems and Aggressive Behavior contributed most to the accuracy of the prediction of group membership. Conclusion The results suggest that subclinical behavioral and psychological characteristics of schizophrenia already exist in the patients' childhood. The quite early detection of schizophrenic children seems possible.

Barriers to Meditation by Gender and Age Among Cancer Family Caregivers

Despite solid basic science research supporting meditation's physiologic benefits, meditation remains a marginalized practice for many Westerners; observational and descriptive studies indicate a spectrum of barriers to meditation practice. The aim of this study was to determine differences in barriers to meditation by gender and age. A cross-sectional survey study of 150 family caregivers to adults with cancer visiting an outpatient chemotherapy center in Connecticut was conducted. The primary outcome was the Determinants of Meditation Practice Inventory. Explanatory variables included demographic characteristics, Center for Epidemiologic Studies-Depression Scale, Big Five Inventory, and Caregiver Reaction Assessment. Participants included 98 women and 52 men. Age range was 18-84 years (M = 52.3 years). The highest frequency of barriers for both genders related to misconceptions about meditation. The total number of barriers to meditation did not significantly vary by gender (p = .10) nor age (p = .27). After adjusting for personality trait, reactions to caregiving, and emotional distress, gender (adjusted β = 0.81, SE = 1.70, p = .63) and age (adjusted β = 0.02, SE = 0.05, p = .67) still did not predict the number of barriers to meditation. Backward elimination in model building showed that personality trait and reactions to caregiving account for 32% of the variability in barriers. The total number of barriers to meditation was examined, and a difference was not found by age or gender. It is possible that differences by age and gender exist at the item level of evaluation but were not evident when evaluating total scores. Further study is needed with samples large enough to have statistical power for item-level analysis.

The Five Factor Model in personnel selection: Measurement equivalence between applicant and non-applicant groups

This study investigates the measurement equivalence of a Five Factor measure of personality between applicant and non-applicant samples. The Big Five Questionnaire-2 was administered in two samples: A group of volunteers ( n = 903), who completed the test for research purposes, and a group applying for jobs, who completed the test during hiring procedures ( n = 401). Multiple-group confirmatory factor analysis on item composites was conducted to test for the equivalence of factor covariance and mean structures of twenty facets of the Five Factors. Item-level analyses were carried out through analysis of variance to further examine the issue of measurement invariance. Findings suggested that personality facets have the same measurement unit across applicants and non-applicants, while a lack of equivalence was found in the origin of the scales. Similar results were found at the item-level. Implications for personality assessment are advanced and discussed.

EXAMINING THE FACTOR STRUCTURE OF A STATE STANDARDS‐BASED SCIENCE ASSESSMENT FOR STUDENTS WITH LEARNING DISABILITIES

ABSTRACTThis study examined the scores on a state standards‐based Grade 5 Science assessment obtained by a group of students without learning disabilities who took the standard form of the test and by three groups of students with learning disabilities: one taking the standard form of the test without accommodations or modifications, a second taking the test with accommodations, and a third group taking the test with modifications. The groups received accommodations or modifications that were specified in their 504 or IEP plans. Unlimited time was granted to complete the test. A series of item‐level, then parcel‐level, exploratory and confirmatory factor analyses investigated whether or not the assessment demonstrated factorial invariance for the four groups of students studied. The results of this study help substantiate the validity of test scores for students with disabilities who take the test with the particular set of accommodations or modifications that were used in this study. In addition, the results lend support to modifying this state's policy in order to aggregate scores obtained by students without learning disabilities and by students with learning disabilities who have taken the assessment with accommodations or modifications required by their 504 plans or IEPs for AYP purposes.

Validation of the research capacity and culture (RCC) tool: measuring RCC at individual, team and organisation levels

Research capacity building (RCB) in Australia has recently focussed on strategies that take a whole of system approach to developing research culture at individual, team and organisation levels. Although a theoretical framework exists, no tool has been published that quantitatively measures the effectiveness of RCB interventions aimed at these three levels. A sample of 134 allied health workers was used to validate the research capacity and culture (RCC) tool. Item level analysis was undertaken using Cronbach's α and exploratory factor analysis, and test-retest reliability was examined using intra-class correlations (ICC). The tool had one factor emerge for each domain, with excellent internal consistency for organisation, team and individual domains (α=0.95, 0.96 and 0.96 respectively; and factor loadings ranges of 0.58-0.89, 0.65-0.89 and 0.59-0.93 respectively). The overall mean score (total) for each domain was: 5.4 (inter-quartile range 3.9-7.7), 4.4 (IQR 2.6-6.1) and 3.9 (IQR 2.9-6) for the organisation, team and individual domains respectively. Test-retest reliability was strong for each domain: organisation ICC=0.77, team ICC=0.83 and individual ICC=0.82. The RCC tool has three domains measuring research capacity and culture at organisation, team and individual levels. It demonstrates excellent internal consistency and strong test-retest reliability.

The Effect of Simulation on Clinical Performance

Patient simulation has been used to augment the traditional clinical model, but its value is unclear. The aim of this study was to evaluate the effects of a theory-driven pediatric simulation curriculum on nursing students' clinical performance. The convenience sample included 116 junior nursing students enrolled in a pediatric course. Using a staggered timing model, students attended simulation instead of clinical for 2 weeks (25%) of an 8-week semester. The students spent the same amount of time in simulation as in clinical (12 hours per week). Student clinical performance was assessed using a Likert-style tool at 2-week intervals by the clinical faculty. Scores of students who attended simulation in the first 2 weeks were compared with students who had not yet attended simulation. Data were analyzed using repeated measure analysis with the mixed model, and covariate effects were considered. A Compound Symmetry covariance model was used to control the correlation between weeks within each subject. Statistical significance was determined at the 5% level. Faculty rated students with patient simulation experience higher than those who had not yet attended simulation (mean ± standard error: 1.74 ± 0.75, P = 0.02). On item-level analysis, therapeutic skills were positively impacted by simulation (P = 0.02). Time in simulation enhanced clinical performance, as simulation students achieved higher scores more quickly than those without simulation and maintained high performance levels. These findings suggest patient simulation is a valuable addition to augment the apprenticeship model.

Beyond the Shadow of a Trait: Understanding Discounting Through Item-Level Analysis of Personality Scales

Temporal discounting, the loss in perceived value associated with delayed outcomes, correlates with a number of personality measures, suggesting that an item-level analysis of trait measures might provide a more detailed understanding of discounting. The current report details two studies that investigate the utility of such an item-level analysis. Study 1 demonstrates that discount rates correlate with the overall score as well as with 11 individual items on the Total SelfControl Scale (TSCS). Study 2 demonstrates a correlation between discount rates and the Consideration of Future Consequences (CFC) scale, but not with a measure of general self-efficacy. An item-level analysis of the CFC found that the CFC-Immediate subscale and four individual items correlated with discount rates. Overall, these two studies sketch an initial picture of how item-level analyses of existing trait measures can better inform our understanding of discounting.

Do Different Approaches to Examining Construct Comparability in Multilanguage Assessments Lead to Similar Conclusions?

In this study, we examine the degree of construct comparability and possible sources of incomparability of the English and French versions of the Programme for International Student Assessment (PISA) 2003 problem-solving measure administered in Canada. Several approaches were used to examine construct comparability at the test- (examination of test data structure, reliability comparisons and test characteristic curves) and item-levels (differential item functioning, item parameter correlations, and linguistic comparisons). Results from the test-level analyses indicate that the two language versions of PISA are highly similar as shown by similarity of internal consistency coefficients, test data structure (same number of factors and item factor loadings) and test characteristic curves for the two language versions of the tests. However, results of item-level analyses reveal several differences between the two language versions as shown by large proportions of items displaying differential item functioning, differences in item parameter correlations (discrimination parameters) and number of items found to contain linguistic differences.

Development and psychometric validation of the Clinical Assessment Interview for Negative Symptoms (CAINS)

Progress in the development of new pharmacological and psychosocial treatments for the negative symptoms of schizophrenia is impeded by limitations of available assessment instruments. The multi-site Collaboration to Advance Negative Symptom Assessment in Schizophrenia (CANSAS) was established to develop and validate a new clinical rating scale using a transparent, iterative, and data-driven process. The Clinical Assessment Interview for Negative Symptoms (CAINS) was designed to address limitations of existing measures and assess consensus-based sub-domains, including asociality, avolition, anhedonia, affective blunting, and alogia. The structure and psychometric properties of the CAINS were evaluated in a sample of 281 schizophrenia and schizoaffective outpatients at four sites. Converging structural analyses indicated that the scale was comprised of two moderately correlated factors — one reflecting experiential impairments (diminished motivation and enjoyment of social, vocational, and recreational activities) and one reflecting expressive impairments (diminished non-verbal and verbal communication). Item-level analyses revealed generally good distributional properties, inter-rater agreement, discriminating anchor points, and preliminary convergent and discriminant validity. Results indicate that the CAINS is a promising new measure for quantifying negative symptoms in clinical neuroscience and treatment studies. Results guided item modification or deletion, and the reliability and validity of the revised, shorter version of the CAINS is in the final phase of development within the CANSAS project.

Culture and the Anxiety Disorders: Recommendations for DSM-V

Background: The anxiety disorders specified in the fourth edition, text revision, of The Diagnostic and Statistical Manual (DSM-IV-TR) are identified universally in human societies, and also show substantial cultural particularities in prevalence and symptomatology. Possible explanations for the observed epidemiological variability include lack of measurement equivalence, true differences in prevalence, and limited validity or precision of diagnostic criteria. One central question is whether, through inadvertent “over-specification” of disorders, the post-DSM-III nosology has missed related but somewhat different presentations of the same disorder because they do not exactly fit specified criteria sets. This review canvases the mental health literature for evidence of cross-cultural limitations in DSM-IV-TR anxiety disorder criteria. Methods: Searches were conducted of the mental health literature, particularly since 1994, regarding cultural or race/ethnicity-related factors that might limit the universal...

Characterizing and Quantifying the Symptomatic Features of COPD Exacerbations

There is a need for a standardized, valid, and reliable instrument for quantifying exacerbations of COPD. The objective of this study was to identify symptom items that characterize COPD exacerbations to form a new patient diary for evaluating exacerbation frequency, severity, and duration. Twenty-three symptom items identified from patient interviews were administered to 410 patients with COPD aged (mean ± SD) 65 ± 10 years with stable FEV(1) of 51% predicted ± 20% predicted and 1.8 ± 1.8 exacerbations in the preceding 12 months. A total of 222 patients had a physician-diagnosed exacerbation; 188 were stable. Item-level analyses (floor and ceiling effects, criterion keying, item-total correlation) were used in the first stage of item reduction. Further reduction was conducted using Rasch model and descriptive item analyses. Exploratory factor analysis was performed on the items that survived the exclusion process. No item behaved differently between stable and exacerbation conditions. One item was removed after item-level analysis, and eight were removed following Rasch analysis. Together, the surviving 14 items met the criteria for a unidimensional measure of exacerbation severity. Internal consistency (person separation index) was excellent at 0.92. Post hoc exploratory factor analysis revealed one dominant factor, with three domains (breathlessness, cough and sputum, and chest symptoms) that accounted for 68% of the variance. An exacerbation appears to be a quantitative rather than qualitative change from the stable state. This analysis identified a range of symptoms that form a unidimensional construct of overall exacerbation severity. The 14 items identified form the Exacerbations of Chronic Pulmonary Disease Tool (EXACT), a daily diary for detecting and quantifying exacerbation severity in COPD.

Age-group differences in facets of positive and negative affect

Objectives: The higher order structure of Positive Affect (PA) and Negative Affect (NA) is comparable in self-report affect data from younger and older adults. The current study advances this work by comparing the factor structure of facets of PA and NA in older and younger adults using exploratory and confirmatory factor analyses. Method: Older (N = 203; M age = 73.5 years, range 65–92) and younger (N = 349; M age = 19.1 years, range 18–30) adults completed the Positive and Negative Affect Schedule–Expanded Form (PANAS-X) (Watson, D., & Clark, L.A. (1999). Manual for the Positive and Negative Affect Schedule -- Expanded Form. Iowa City, IA: The University of Iowa), which measures General PA and NA as well as three facets of PA (Joviality, Self-Assurance, and Attentiveness) and four facets of NA (Fear, Sadness, Guilt, and Hostility). Results: Item-level exploratory factor analyses of the facet scales revealed structures that were similar in older and younger adults; however, older adult solutions were more diffuse and diverged more from the PANAS-X scale structure. The facet of Sadness exhibited the largest age-group difference, relating more to guilt and anxiety in older than younger adults. Conclusion: Older adults may discriminate less amongst specific affect terms or may experience greater affective heterogeneity. Further, Sadness may manifest in age-specific ways. The construct variance of Sadness, and how this issue might be related to the assessment of depression in older adults, is discussed.

Doctoral Student Satisfaction: An Examination of Disciplinary, Enrollment, and Institutional Differences

Previous research suggests that it is the department, not the graduate school that bears the greatest responsibility for doctoral students’ progress and success (Ehrenberg et al., Doctoral education and the faculty of the future (pp. 15–34). Ithaca, NY: Cornell University Press, 2009) dictating the need to examine and understand how doctoral students experience their education at the department level. In the present study, we analyzed the NAGPS’ 2000 National Doctoral Program Survey (NDPS) data in an effort to understand the differences in the satisfaction levels of doctoral students (current, recent graduates, and former) across various academic disciplines (e.g. social sciences, humanities, engineering) and different institutional types (e.g. research extensive and research intensive). Employing both traditional (ANOVA) and item-level (Rasch Rating Scale Model) analyses we found that although overall satisfaction with doctoral experiences appears to be equivalent/similar across multiple disciplines, student satisfaction within disciplines varied significantly and consistently with respect to specific academic experiences.