Four-week Test-retest Reliability Research Articles

Evaluating the reliability and validity of the Questionnaire on Well-Being: a validation study for a clinically informed measurement of subjective well-being

ABSTRACT Researchers and clinicians are becoming increasingly aware of the importance of assessing positive functioning to inform clinical outcomes. This paper evaluates the Questionnaire on Well-Being (QWB, available for free https://doi.org/10.17605/OSF.IO/GSC3R), a clinically informed instrument that assesses subjective well-being, across two studies. Study One, consisting of treatment-seeking individuals in an assertiveness training sample (n = 495), explored the factorial structure of the QWB, assessed the four-week test-retest reliability, criterion-related validity, and identified a preliminary cutoff point for the QWB with clinical significance. Study Two, including participants from the general public (n = 1561), confirmed the factorial structure of the QWB and further evaluated criterion-related validity. The results provided support for a unidimensional structure for the QWB. Furthermore, the QWB exhibited excellent internal reliability (Cronbach’s alpha = 0.93 and 0.94 in Study One and Two, respectively), high test-retest reliability (ICC3 = .50 at a four-week follow-up in Study One), and appropriate criterion-related validity demonstrating positive correlations with positive affect and negative correlations with psychopathology. Finally, a cutoff point on the QWB below 50 was associated with marked psychopathology. These findings provide preliminary support for the usage of the QWB in clinical and non-clinical settings, establishing the QWB as a reliable indicator of subjective well-being.

Imagine chocolate: The craving experience questionnaire in the food domain

Measurement of food craving has gained relevance in the current obesity epidemic. The Craving Experience Questionnaire (CEQ) considers not only craving intensity but also cognitive intrusiveness and imagery vividness as separate craving factors and could thus refine food craving assessment. It is available in two versions with ten items each. The CEQ-F assesses craving frequency across specific time periods and the CEQ-S time-point specific craving strength. Across three independent studies, N = 533 participants completed the German chocolate CEQ-F referenced at the past year to operationalise trait-like craving. Among them, N = 402 also completed the German chocolate CEQ-S referenced at the current moment to operationalise state-like craving. Four-week test-retest reliability was measured. For external validity, we assessed self-reported chocolate consumption, body-mass-index, trait approach motivation, general imagery vividness, and the most widely used food craving questionnaire, namely the Food Cravings Questionnaires in a trait (FCQ-T-r) and state version (FCQ-S), as well as behavioural approach bias (reaction time-based measurement). The three-factor structure was replicated with excellent internal consistency for both CEQ-F and CEQ-S. Test-retest reliability was moderate for both CEQ versions. CEQ-F scores were related to higher levels of chocolate consumption, approach motivation, and FCQ-T-r scores, but not to body-mass-index, imagery vividness, or approach bias. CEQ-S scores were associated with FCQ-S scores and partly with approach bias, but not with approach motivation and imagery vividness. The current results support the factor structure, validity and reliability of the German chocolate CEQ-S and CEQ-F with questions remaining regarding the ability of the CEQ-S to measure state craving. Thus, CEQ-F and CEQ-S usefully contribute to food craving assessment.

The Developmental Crisis Questionnaire (DCQ-12): Psychometric Development and Validation

Developmental crisis is a construct that is central to many theories of psychosocial adult development, yet there is currently no validated psychometric measure of adult developmental crisis that can be used across adult age groups. To address this gap in the literature, we developed and validated an age-independent measure of adult developmental crisis for research and applied purposes, entitled the Developmental Crisis Questionnaire (DCQ-12). Exploratory and confirmatory factor analyses were conducted separately on different samples. A three-factor structure emerged as the best fit with the data: (1) Disconnection and Distress; (2) Lack of Clarity and Control and (3) Transition and Turning Point. The DCQ-12 showed predictive validity with measures of self-esteem, locus of control, authentic living, optimism, presence of and search for meaning, turning points and a related crisis measure. Four-week test–retest reliability ranged from 0.78 to 0.89 across subscales. As well as research uses, the DCQ-12 measure has potential application in practice, given that assessment of developmental crisis has relevance to professionals working in clinical and non-clinical roles to support and coach adults through periods of transition.

Psychometric properties of the 7-item version of the Concise Health Risk Tracking Self-Report Scale for rural adults enrolled in an Internet-delivered self-help program for depression.

Monitoring suicide risk in clinical practice requires valid and reliable assessment instruments. This study evaluated the psychometric properties of the 7-item version of the Concise Health Risk Tracking Self-Report, CHRT-SR7 in a primarily rural population. The sample comprised 788 participants (81.7% female) of an effectiveness trial of an internet-based self-help intervention for depression. Participants completed self-report questionnaires, including the CHRT-SR7 , Patient Health Questionnaire-9, Generalized Anxiety Disorder-7, Work and Social Adjustment Scale, Connor-Davidson Resilience Scale-10, and Barriers to Seeking Mental Health Care. Four-week test-retest reliability was calculated for a subsample of 147 participants randomized to a waitlist control group. The CHRT-SR7 internal consistency was α = 0.80 (total sample), α = 0.80 (women), and α = 0.83 (men). The 4-week test-retest reliability was strong for women (r = 0.78) and moderate for men (r = 0.66). Confirmatory factor analysis supported the original 3-factor solution: Hopelessness (2 items), Perceived Lack of Social Support (2 items), and Current Suicidal Thoughts and Plans (3 items), which was invariant across gender and rural status. Convergent and divergent validity was supported as reflected in significant correlations of the CHRT-SR7 and its subscales with measures of depression, anxiety, adjustment, and resilience. Limitations include the limited demographic diversity (mostly non-Hispanic White women) and reliance on self-report data. Our findings complement those reported in prior studies of patients with severe depression and support the use of the CHRT-SR7 for measuring suicide risk in rural adults; future studies should further test the instrument's psychometric properties in racial or ethnic minority rural residents.

The Developmental Crisis Questionnaire (DCQ-12): Psychometric Development and Validation

Developmental crisis is a construct that is central to many theories of psychosocial adult development, yet there is currently no validated psychometric measure of adult developmental crisis that can be used across adult age groups. To address this gap in the literature, we developed and validated an age-independent measure of adult developmental crisis for research and applied purposes, entitled the Developmental Crisis Questionnaire (DCQ-12). Exploratory and confirmatory factor analyses were conducted separately on different samples. A three-factor structure emerged as the best fit with the data; (1) Disconnection and Distress; (2) Lack of Clarity and Control; (3) Transition and Turning Point. The DCQ-12 showed predictive validity with measures of self-esteem, locus of control, authentic living, optimism, presence of and search for meaning, turning points and a related crisis measure. Four-week test-retest reliability ranged from 0.78 to 0.89 across subscales,. As well as research uses, the DCQ-12 measure has potential application in practice, given that assessment of developmental crisis has relevance to professionals working in clinical and non-clinical roles to support and coach adults through periods of transition.

Feasibility of the Schizophrenia Hope Scale-9: A Psychometric Study.

Hope is essential in rehabilitating persons with schizophrenia, though scales to measure hope are not appropriate for this population. The purpose of this cross-sectional study was to identify the psychometric properties of the Schizophrenia Hope Scale-9 (SHS-9) using data from 83 people with schizophrenia in four mental health centers and 762 healthy persons from two universities in South Korea. The total SHS-9 score is calculated by adding all items’ scores and ranges from 0 to 18. The mean (standard deviation) SHS-9 score of the participants with schizophrenia and healthy participants was 11.53 (SD = 4.78) and 14.78 (SD = 3.19), respectively. Lower scores indicate a lower level of hope. The Cronbach’s alpha coefficient was 0.92 with a four-week test-retest reliability of 0.89. Criterion-related construct validity was established by examining the correlation between the SHS-9 and the State-Trait Hope Inventory scores. Divergent validity was identified through a negative relationship of SHS-9 with the Beck Hopelessness Scale. In persons with schizophrenia and healthy college students, Bartlett’s test of sphericity yielded χ2 = 465.03 (p < 0.001) and χ2 = 2679.24 (p < 0.001) respectively. The values of the Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy were 0.89 and 0.90, respectively. The construct validity of the SHS-9 was confirmed through principal component analysis with extraction methods, which resulted in a one-factor solution, accounting for 61.83% of the total item variance. This study provides evidence for the validity and reliability of the SHS-9; therefore, it could be used to study the relationships between hope and other variables (e.g., depression and recovery) in persons with schizophrenia and measure the effect of psychosocial interventions on their hope.

What Do People With Schizophrenia Do All Day? Ecological Momentary Assessment of Real-World Functioning in Schizophrenia

Schizophrenia is a major cause of disability worldwide. As new treatments for functioning are tested, the need grows to demonstrate real-world functioning gains. Ecological momentary assessment (EMA) may provide a more ecologically valid measure of functioning. In this study, smartphone-based EMA was used to signal participants with schizophrenia (N = 100) and controls (N = 71) 7 times a day for 7 days to respond to brief questionnaires about social interactions and functioning behaviors. Excellent adherence was found, with both groups completing an average of 85% of surveys and only 3% of participants with schizophrenia excluded for poor adherence. Four-week test-retest reliability was high (r = .83 for total productive behaviors). Relative to controls, participants with schizophrenia reported significantly less total productive activity (d = 1.2), fewer social interactions (d = 0.3), more nonproductive behaviors (d = 1.0; watching TV, resting), and more time at home (d = 0.8). Within the schizophrenia group, participants living independently showed better functioning on EMA relative to participants in supported housing (d = 0.8) and participants engaged in vocational activities showed better functioning than individuals not engaged in vocational activities (d = 0.55). Modest correlations were found between EMA and an in-lab self-report measure of functioning activities performed in the community, but not between EMA and measures of functional capacity or potential. This study demonstrated the feasibility, sensitivity reliability, and validity of EMA methods to assess functioning in schizophrenia. EMA provides a much-needed measure of what individuals with schizophrenia are actually doing in real-world contexts. These results also suggest that there may be important disjunctions between indices of abilities and actual real-world functioning.

Preliminary Validation of a Free-to-Use, Brief Assessment of Adult Intelligence for Research Purposes: The Matrix Matching Test.

The concept of intelligence as a measurable trait of intellectual function continues to be an important issue in psychology. Traditionally, a core field of differential psychology and widely employed in applied settings, it is also important in various research fields. Here, I describe development of a new assessment of general intelligence of adults that has no language component and can be administered in about 10 minutes. A total sample of 176 adult participants, from various settings, was assessed with a set of matrix tasks that involved either visuospatial (fluid) or semantic (crystallized) reasoning. The internal consistency was acceptable (α = .748), and there was good four-week test-retest reliability ( r = .931). Concurrent validity was demonstrated by a high correlation between the new test and the (seven-subtest version) Wechsler Adult Intelligence Scale-IV (WAIS-IV) scores ( r = .889). A principal component analysis also suggested that the new test measures the same latent construct as the WAIS-IV-thought to be general intelligence. Predictive validity was shown in a subsample of 60 undergraduates by a medium-sized correlation between test scores and grade point average data ( r = .396). These preliminary results suggest that the Matrix Matching Test may be a useful research tool.

Validity, reliability and ease of use of the disabilities of arm, shoulder and hand questionnaire in adults following stroke

Purpose: The Disabilities of Arm, Shoulder and Hand (DASH) questionnaire is a patient reported outcome measure for evaluating upper limb function in people with musculoskeletal conditions. While the DASH has good psychometric properties when used with people with musculoskeletal conditions, it has not been tested with adults after stroke.Methods: Data for n = 61 adults following stroke (aged 32–93 years, 44% male) were analyzed to test validity and reliability of the DASH for use with a stroke population. Data included demographic and clinical attributes, DASH scores (baseline and four weeks later) and Patient Rated Wrist Evaluation (PRWE) measures.Results: Internal consistency was good (Cronbach alpha 0.92, SEM 6.65). Factor analysis and Rasch modeling suggested that the questionnaire comprised three subscales: pain, impact and function. Concurrent validity between the DASH and PRWE (Spearman’s Rho rs = 0.41) was moderate. The scale was perceived by clinicians to be useful, quick and simple to administer. The DASH had low four-week test-retest reliability (ICC 0.56 [95% Cl 0.05–0.79]).Conclusions: The DASH is considered to have acceptable validity when used with adults following stroke. Test–retest reliability was low but further research is needed to establish whether this is a result of condition-related change or the stability of the measure.Implications for RehabilitationThe DASH questionnaire examines upper limb function in task performance and appears to be a useful tool, which is simple to administer in the clinical setting with adults following stroke.Upper limb function post stroke can be meaningfully assessed using the DASH as it has good internal consistency and moderate concurrent validity.Rasch analysis and factor analysis suggests that the tool appears to consist of three subscales: pain, impact and function. The total score of the DASH may be less meaningful than the totals of these subscales.The test–retest reliability of the DASH requires further research; over a four-week period DASH stability was poor in a group of people with moderate to severe upper limb impairment.

Reliability and Validity of the Dissociative Experiences Scale Among South Korean Patients With Schizophrenia

The Dissociative Experiences Scale (DES) is the most widely used self-report tool for use with the general population and various clinical entities, including those with schizophrenia. This preliminary study tested the reliability of the DES, especially temporal stability and validity, in patients diagnosed with schizophrenia receiving inpatient or outpatient treatment. The DES was administered at a university-affiliated training hospital in South Korea to a total of 68 patients (39 inpatients and 29 outpatients) whose diagnosis of schizophrenia was confirmed using the Structured Clinical Interview for DSM–IV Axis Disorders. Internal consistency, 4-week test–retest reliability, and validity were calculated. Participants also completed the Childhood Trauma Questionnaire, the Beck Depression Inventory, and the Symptom Checklist–Posttraumatic Stress Disorder Scale. Four-week test–retest reliability was moderate (r = .69) and internal consistency was good (Cronbach’s α = .95). Total score on the DES was correlated with childhood trauma (r = .69, p < .001), posttraumatic symptoms (r = .50, p < .001), and depression (r = .52, p < .001), demonstrating the convergent validity of the scale. Our study confirms the psychometric soundness of the DES among patients with schizophrenia after the acute stage of illness (e.g., 3 weeks after admission), supporting use of the scale for screening and evaluating dissociative symptoms within this population.

Resilience to affective disorders: A comparative validation of two resilience scales

BackgroundResilience to affective disorders in rehabilitating patients or in individuals with a severe disability is of special research interest. However, there is no gold standard for measuring resilience. We aimed to test the accuracy of the Dutch translation of the Brief Resilience Scale (BRSnl) and of the Resilience Scale (RSnl) in recognizing rehabilitating patients without anxiety and depression, and to determine the reliability and construct validity of both scales. MethodsA within-subjects longitudinal study with six assessments, each one week apart. Forty residents of a nursing home rehabilitating unit were interviewed to assess resilience (BRSnl and RSnl), optimism and pessimism (LOT-R), depression and anxiety (HADS), positive and negative affect (PANAS), and pain (VAS). ResultsReceiver operating characteristic analyses for recognizing the absence of depression and anxiety (HADS-score≤7) revealed better accuracy (P=0.038) for the BRSnl (AUC=0.84; p<0.0001) than for the RSnl (AUC=0.68; P=0.017). The scales correlated moderately at baseline (rs=0.35; p=0.026), and at four-week follow-up (rs=0.50; p=0.004). The RSnl was positively associated with positive outcomes (optimism and positive affect), and the BRSnl positively with positive outcomes, and negatively with negative outcomes (pessimism, anxiety and negative affect). The RSnl showed a better four-week test–retest reliability (ICC, 0.94; 95% CI, 0.87 to 0.97) than the BRSnl (0.66; 95% CI, 0.29 to.83). LimitationsShort study duration, a relatively small sample. ConclusionThe BRSnl showed better performance in detecting people without depression and anxiety than the RSnl, and performed better on construct validity.

The Anxiety Severity Interview for Children and Adolescents: An Individualized Repeated Measure of Anxiety Severity

The Anxiety Severity Interview for Children and Adolescents (ASICA) was developed for the repeated assessment of the impact of anxiety and control over anxiety symptoms. The ASICA incorporates three main components of anxiety: physical response, avoidant behaviour and anxious thoughts. The objective of this study was to evaluate the psychometric properties of the ASICA in children with anxiety disorder (n = 139, age 8-18 years) and a non-anxious control group (n = 40, age 8-18 years). A confirmatory factor analysis confirmed the intended factor structure. Internal reliability was moderate to good; inter-rater reliability was excellent. Four-week test-retest reliability was good. The ASICA discriminated between anxious and non-anxious children and appeared sensitive to treatment change. A cut-off score of 13 was determined. Convergent validity with anxiety symptoms was moderate; discriminant validity with depressive symptoms was less strong. The results suggest that the ASICA is a reliable instrument that could be used in clinical practice to repeatedly monitor anxiety severity.

Psychometric properties of the Childhood Trauma Questionnaire-Short Form (CTQ-SF) in Korean patients with schizophrenia

ObjectiveDespite increasing interest in the relationship between childhood trauma and psychosis, measures used to assess early trauma have not had their psychometric properties extensively tested among individuals with serious mental illness. This study investigated the reliability and validity of one of the most widely-used self-reports of early adversity, the Childhood Trauma Questionnaire, Short Form (CTQ), among patients with schizophrenia. MethodsThe CTQ was administered to 100 patients (52 inpatients and 48 outpatients) diagnosed with schizophrenia in three training hospitals. Internal consistency, four-week test–retest reliability and validity were calculated. Participants also completed the Trauma Antecedents Questionnaire (TAQ), the Impact of Events Scale-Revised (IES-R), and the Dissociative Experiences Scale-Taxon (DES-T). ResultsOur analysis indicated high test–retest reliability (Spearman ρ=0.75) and internal consistency (Cronbach α=0.89). Concurrent validity was confirmed as each type of childhood trauma was significantly correlated with the corresponding subscales of the TAQ. In addition, the CTQ was positively related to post-traumatic stress symptoms and pathological dissociation, demonstrating the convergent validity of the scale. ConclusionThe CTQ is a reliable and valid self-report measure for assessing childhood trauma in both inpatients and outpatients with schizophrenia.

Psychometric Properties and Construct Validity of the Weight‐Related Eating Questionnaire in a Diverse Population

This study evaluates the 16-item, four-factor Weight-Related Eating Questionnaire (WREQ), which assesses theory-based aspects of eating behavior, across diverse, nonclinical subgroups. A total of 621 men and women aged 18-81 years (34.3 ± 16.4) with a mean BMI of 25.7 ± 6.1kg/m(2) (range 15.5-74.1kg/m(2)) were recruited from general education classes at the University of Hawai'i, Manoa and an online survey panel of Hawai'i residents to complete a web-based survey. Participants were predominantly white (23%), Asian/Asian-mix (42%), or Native Hawaiian/Pacific Islander (18%). The WREQ's factor structure was successfully replicated by confirmatory factor analysis (CFA) for the entire sample and by weight status, gender, age, and race with strong internal consistency. Four-week test-retest reliability (n = 31) for the subscales was excellent with interclass correlations of 0.849-0.932. Tests of population invariance confirmed the generalizability of the WREQ across all subgroups having provided no evidence that the factor structure, factor loadings, or indicator intercepts varied significantly between the groups. Multivariate regression analyses showed that emotional eating was independently associated with BMI (β = 0.272, P < 0.001) as well as moderate- and long-term weight change rates (weight gain) in young adults (β = 0.152, P = 0.042) and adults (β = 0.217, P = 0.001). Compensatory restraint was negatively associated with weight gain in adults (β = -0.133, P = 0.039). Routine restraint and emotional eating were highest among dieters. All associations remained significant after accounting for gender, age, and race. The hypothesized WREQ measurement model demonstrated very good construct validity, confirming the unbiased generalizability of the WREQ measure across sex, age, race, and BMI subgroups, and excellent criterion-related validity with respect to current BMI, weight change, and weight control status.

Reliability and Validity of the Korean Version of the Childhood Trauma Questionnaire-Short Form for Psychiatric Outpatients

ObjectiveThe Childhood Trauma Questionnaire (CTQ) is perhaps the most widely used and well-studied retrospective measure of childhood abuse or neglect. This study tested the initial reliability and validity of a Korean translation of the Childhood Trauma Questionnaire (CTQ-K) among non-psychotic psychiatric outpatients.MethodsThe CTQ-K was administered to a total of 163 non-psychotic psychiatric outpatients at a university-affiliated training hospital. Internal consistency, four-week test-retest reliability, and validity were calculated. A portion of the participants (n=65) also completed the Trauma Assessment Questionnaire (TAQ), the Impact of Events Scale-Revised, and the Dissociative Experiences Scale-Taxon.ResultsFour-week test-retest reliability was high (r=0.87) and internal consistency was good (Cronbach's α=0.88). Each type of childhood trauma was significantly correlated with the corresponding subscale of the TAQ, thus confirming its concurrent validity. In addition, the CTQ-K total score was positively related to post-traumatic symptoms and pathological dissociation, demonstrating the convergent validity of the scale. The CTQ-K was also negatively correlated with the competence and safety subscale of the TAQ, confirming discriminant validity. Additionally, we confirmed the factorial validity by identifying a five-factor structure that explained 64% of the total variance.ConclusionOur study indicates that the CTQ-K is a measure of psychometric soundness that can be used to assess childhood abuse or neglect in Korean patients. It also supports the cross-cultural equivalence of the scale.

Test‐retest reliability of voluntary emotion regulation

Despite growing interest in emotion regulation, the degree to which psychophysiological measures of emotion regulation are stable over time remains unknown. We examined four-week test-retest reliability of corrugator electromyographic and eyeblink startle measures of negative emotion and its regulation. Both measures demonstrated similar sensitivity to the emotion manipulation, but only individual differences in corrugator modulation and regulation showed adequate reliability. Startle demonstrated diminished sensitivity to the regulation instructions across assessments and poor reliability. This suggests that corrugator represents a trait-like measure of voluntary emotion regulation, whereas startle should be used with caution for assessing individual differences. The data also suggest that corrugator and startle might index partially dissociable constructs and underscore the need to collect multiple measures of emotion.

Development of the Inventario de Comportamiento Escolar (IDCE) for Puerto Rican Children

The Inventario de Comportamiento Escolar (IDCE) is a teacher rating inventory for the assessment of Puerto Rican children, particularly those with a behavior pattern indicative of attention deficit disorder, learning disabilities, or academic underachievement. Factor analysis of classroom behavior ratings of a representative sample of 961 Puerto Rican students (kindergarten to sixth grade) yielded five scales: Inattention, Hyperactivity, Inappropriate Social Behavior, Irritability-Hostility, and Anxiety. Factor analysis of school performance ratings yielded five scales: Reading-Spelling, Expressive Language, Memory, Directionality-Laterality, and Motor Skills. Alpha coefficients (mean: .90; range: .81 to .97) and item total correlations (mean: .67; range: .52 to .78) provided evidence of the internal consistency of the scales. Four-week test-retest reliability ranged from .70 to .90.