Summated Rating Scale Research Articles

Measuring Implicit Traits in Organizational Research: Development of an Indirect Measure of Employee Implicit Self-Concept

Values, attitudes, and goals are often automatically activated, yet organizational research has relied predominantly on techniques that measure these phenomena at explicit or conscious levels. In this study, the authors validated an indirect measure designed to assess employee self-concept at implicit levels. Because self-concept is believed to operate primarily at implicit levels, it was hypothesized that an indirect measure would be an effective predictor of work criteria. The criteria examined were task performance, citizenship and counterproductive behavior, and the quality of supervisor—subordinate relations. Consistent with predictions, the indirect measure—a word fragment completion task—contributed more to the prediction of criteria than the direct measure— self-report survey items with summated rating scales. The authors discuss the implications of these findings for the use of indirect measures in applied settings.

Psychometric validation of the Rhinitis Control Assessment Test: a brief patient-completed instrument for evaluating rhinitis symptom control

Allergic rhinitis is common, but a validated tool for comprehensive assessment of disease control is not available. To develop a simple patient-completed instrument (the Rhinitis Control Assessment Test [RCAT]) to help detect problems with control of rhinitis symptoms. During a visit to an allergy specialist, 410 patients with allergic rhinitis completed a Total Nasal Symptom Score (TNSS) assessment and the 26-item developmental RCAT. Physicians also completed a global assessment of rhinitis symptom control for each patient. Stepwise regression methods identified 6 items from the developmental RCAT (frequency of nasal congestion, sneezing, and watery eyes; sleep interference; activity avoidance; and self-assessed control) that were most predictive of the allergist's global rating of rhinitis symptom control. A summated rating scale from these 6 items showed good convergent validity (r > 0.70) with scale scores from the TNSS. The discriminant validity of the 6-item scale was demonstrated as mean RCAT scale scores differed significantly across groups of patients differing in physician-rated disease severity (F = 54.4), TNSS severity (F = 193.8), and physician-recommended change in therapy (F = 50.6) (P < .001 for all). The RCAT, a 6-item patient-completed instrument, has satisfactory psychometric properties and seems to be a valid tool for assessing control of allergic rhinitis. Further validation studies will provide confirmation.

Subject-centered scalability: The sine qua non of summated ratings.

A unifying theory of subject-centered scalability is offered that is grounded in structural true score modeling, is conceptually distinct from internal consistency and homogeneity as determined by item correlations, and is empirically confirmable. Scalability holds when item true scores are perfectly correlated but differ in their individual scale metric. The condition of scalability imposes constraints that allow individual item reliability to be estimated independently of scalability. Scalability is shown to imply unit rank and to be testable by a single-factor confirmatory factor analysis reinterpreted as a test of unit rank. High item correlations are shown, contrary to intuition, to be an insufficient condition for scalability. Conversely, low item correlations do not necessarily imply lack of scalability. A stepped decision-oriented procedure is offered as a guideline in summated rating scale construction.

Measuring change in dry‐mouth symptoms over time using the Xerostomia Inventory

To examine the responsiveness to change of the Xerostomia Inventory (XI). The XI is an 11-item summated rating scale which results in a single continuous scale score representing the severity of chronic xerostomia. While the XI has been used as an outcome measure in clinical research, the magnitude of a clinically meaningful change score has yet to be determined. This study comprises a secondary analysis of data from a longitudinal study of changes in xerostomia symptoms in two groups whose symptom trajectories were likely to differ substantially: the normal group was a convenience sample of asymptomatic middle-aged and older individuals with otherwise stable perceptions of mouth dryness; and the onset group comprised patients who were about to undergo radiotherapy for head/neck cancer (and would therefore be expected to develop more severe xerostomia after the baseline measurements). Statistical analyses examined cross-sectional construct validity and internal consistency, test-retest reliability and the measure's responsiveness and longitudinal construct validity. The mean change scores of those for whom 'a little' improvement was reported were used to determine the minimally important difference for the XI. Over two-thirds of the onset group members reported dry mouth 'frequently' or 'always' at follow-up (2 months) and there was a concomitant increase in their mean XI score. Test-retest reliability was acceptable. Examination of within-individual change among those who changed and those for whom stability was observed, showed that only those who worsened had significantly greater XI scores at follow-up. The minimally important difference to reflect deterioration in xerostomia symptoms was determined to be 6 scale points. Between baseline and 2 months, the XI scores of 32 participants (33.7%) deteriorated by the minimally important difference (7.1% and 54.7% respectively among the normal and onset groups; p < 0.0001). The validity and responsiveness of the XI appear to be acceptable, and a change in XI score of 6 or more points is clinically meaningful.

A COMPARISON OF STUDENT TEACHERS’ PERCEPTIONS OF IMPORTANT ELEMENTS OF THE STUDENT TEACHING EXPERIENCE BEFORE AND AFTER A 12-WEEK FIELD EXPERIENCE

This study describes student teachers’ perceptions of important elements of the student teaching experience. Selected characteristics of student teachers and their cooperating centers are also identified. The purposive sample (N = 25) included all student teachers who completed a 12- week field experience during the spring 2004 semester. Student teachers rated 34 elements of student teaching before and after the experience using a summated rating scale: “5” = “High Importance”, “1” = “No Importance.” Questionnaire items also identified selected characteristics of participants. Cronbach’s coefficient alpha reliability estimates for five core areas of the experience ranged from .60 to .84; the overall importance scale yielded an estimate of .91. The overall pretest and posttest means were 4.30 and 4.58, respectively, or about midway between “much” and “high importance.” Results from both observations revealed student teachers perceived that the relationship with their cooperating teacher was the most important aspect of the student teaching experience. Perceptions related to students’ SAEs held the lowest importance before and after student teaching. Findings provide some support for what other researchers posit about the role of concrete experience and modification of one’s perceptions, especially viewpoints that may have been abstracted originally.

Television News Avoidance: Exploratory Results From a One-Year Follow-Up Study

Drawing on uses and gratifications research on the one hand and structural factors explanations on the other hand, this study looked at 4 types of news viewing: intentional news exposure, unintentional news exposure, unintentional news avoidance, and intentional news avoidance. News exposure and news avoidance were analyzed using both a single question and a summated rating scale. Flemish secondary school children (N = 1,648) answered these questions in February 2003 and February 2004. Self-reported news viewing changed over time. Respondents' self-reported news-viewer type predicted 43% of self-reported news-viewing volume. A one-year follow-up design only added 3% explained variance to the model. A cross-sectional approach to explaining news exposure and news avoidance appears to be both valid and parsimonious.

Teachers’ knowledge about epilepsy and attitudes toward students with epilepsy: Results of a national survey

The attitudes and epilepsy-related knowledge of teachers are an important component of the educational experiences of children with epilepsy. Unfortunately however, the exploration of teacher attitudes and knowledge has been extremely limited in the United States. This article describes a survey-based research study of the attitudes and epilepsy-related knowledge of a randomly selected national sample of 512 elementary and middle school teachers in the United States. The questionnaire included the Scale of Attitudes Toward Persons with Epilepsy (ATPE), a summated rating scale that measures both attitudes toward persons with epilepsy and knowledge about epilepsy, as well as a demographic and teaching experience survey and several additional attitude and knowledge items developed by the researchers. The results suggest that although teachers’ attitudes about epilepsy were generally positive, there were significant deficits in terms of general knowledge about epilepsy, its impact in educational settings, and the appropriate management of epilepsy and seizures in the classroom. Critical areas in which to focus remedial education and outreach efforts are identified.

Construction, Implementation, and Analysis of Summated Rating Attitude Scales

Surveys are perhaps the most used, and sometimes misused, methodological tool among academic researchers. Self-administered survey questionnaires (and guided interviews) allow us to elicit behaviors, attributes, beliefs, and attitudes among populations. Data gathered from poorly designed instrumentation is suspect and investigators are disappointed when research involving the use of attitude scales is viewed unfavorably by peer reviewers. Summated rating scales may be commonly used to elicit feelings or attitudes from students, faculty members, administrators, and others. Investigators must be conscious of the referent object and the research objectives when constructing items comprising a summated ratings scale. Items should be gathered from careful review of the literature and consultation with experts. Pilot testing a survey instrument and its component attitude scale are critical to success. Adherence to a few basic tenets will help to ensure that responses to the scale are valid and reliable. There a...

Factors shaping biblical literalism: a study among Anglican laity

Biblical literalism was assessed among 404 adult Anglicans from a variety of church traditions using a summated rating scale based on 10 items referring to events in the Bible. The literalism scale showed high internal reliability (α = .92) and scores were highest (i.e. most literal) in Evangelical churches, intermediate in Broad churches and lowest in Anglo‐Catholic churches. Decisions about whether or not an event happened appeared to be based on a combination of general doctrinal belief about the Bible, the plausibility of the event and the doctrinal weight associated with it. A number of different factors predicted the degree of literalism, including general educational experience, experience of theological education at certificate level or higher, charismatic experience and the frequency of reading the Bible. Both general and specifically theological higher education seemed to reduce literalism in Anglo‐Catholic and Broad churches, but this was not so among Evangelicals, who maintained high levels of literalism whatever their educational experience.

Evaluating Treatment Outcomes for African American and White Clients Receiving Treatment at a Community Mental Health Agency in the Rural South

The present study examines the relationship between race and treatment outcomes among clients receiving services at a community mental health agency in the rural southeastern United States. A nonprobabilistic sample of clients completed the BASIS-32, a self-report summated rating scale, at an initial assessment and a series of three follow-up assessments conducted at 3-month intervals. The data were analyzed using a hierarchical linear model (HLM) consisting of an individual growth model and a between-client model examining possible differences in growth trajectories attributable to race, diagnosis, socioeconomic status, and gender. Clients’ symptomatology remained stable or decreased during the study period. Of the demographic variables examined, only diagnosis was significantly related to variation in treatment outcomes. No racial differences in treatment outcomes emerged. Although the findings must be considered preliminary, they do suggest that community mental health agencies can effectively respond to various ethnic, cultural, and racial groups by providing appropriate and individualized services.

The responsiveness of headache impact scales scored using 'classical' and 'modern' psychometric methods: a re-analysis of three clinical trials.

While item response theory (IRT) offers many theoretical advantages over classical test theory in the construction and scoring of patient based measures of health few studies compare scales constructed from both methodologies head to head. Compare the responsiveness to treatment of migraine specific scales scored using summated rating scale methods vs. IRT methods. The data came from three clinical studies of migraine treatment that used the Migraine Specific Quality of Life Questionnaire (MSQ). Five methods of quantifying responsiveness were used to evaluate and compare changes from pre- to post-treatment in MSQ scales scored using Likert and IRT scaling methods. Changes in all MSQ scale scores from pre- to post-treatment were highly significant in all three studies. A single index scored from the MSQ using IRT methods was determined to be more responsive than any one of the MSQ subscales across the five methods used to quantify responsiveness. Across 13 of the 15 tests (5 responsiveness methods * 3 studies) conducted, the single index scored from the MSQ using IRT methods was the most responsive measure. IRT methods increased the responsiveness of the MSQ to the treatment of migraine. The results agree with the psychometric evidence that suggest that it is feasible to score a single index from the MSQ using IRT methods. This approach warrants further testing with other measures of migraine impact.

Validation of indices for functional assessment of dentures.

To validate an assessment of qualities of complete dentures that may relate to functional success. A consecutive sample of 131 complete denture wearers took part in the study. a Greek department of prosthetic dentistry. All participants and their dentures underwent clinical examination. The Functional Assessment of Dentures (FAD) 10-item criteria' describing clinical factors of denture quality was used. The rationale for selecting these particular criteria was to include a range of clinical factors to produce an overall assessment of complete dentures. The 10-item scale was examined for Internal Consistency and Construct Validity providing a summated rating scale whose total score could be used as an index for the quality of dentures. Both intra- and inter-examiner agreement for all 10 dichotomous scale criteria were all good or very good; the intra-examiner Kappa values were 0.72 to 0.91 and the inter- examiner values were 0.72 to 0.99. Statistical analysis revealed that a 7-item summated rating scale could be constructed which has marginally better internal consistency (reliability) and Construct Validity with fewer questions. The FAD criteria' can be used to give good repeatability. The adjustments made to the original summation scale, if confirmed by new studies, would further increase the practical utility of a FAD index for general dentists. In due course with data from future studies, discrimination on weighting of items will be appropriate to relate the impact of denture quality on quality of life.

Constructing Mail Survey Questionnaires to Maximize Rates of Return and Assure the Validity and Reliability of Responses

OBJECTIVES: Because the results generated by Shane P. Desselle, R.Ph., Ph.D. from mail survey questionnaires can be a rich source of information for practitioners, administrators, marketers, and policy-makers, the researcher must have confidence in the validity and reliability of the estimates obtained. Following simple tenets of survey questionnaire construction can help prevent spurious results. After completing this continuing education program, participants will be able to: 1) Identify methods of increasing the likelihood of a higher rate of return for mail surveys. 2) Discuss the importance of a properly written cover letter. 3) Describe the components of a properly written cover letter. 4) Design survey questions to collect demographic, personal, and practice- related data. 5) Identify some potential pitfalls when constructing attitude scales. 6) Construct items comprising a summated ratings scale.

Implementing Mail Survey Questionnaires

OBJECTIVES: An increasingly greater number of pharmacists in managerial positions are required to gather data to facilitate the decision-making process. Mail survey questionnaires are powerful tools when utilized properly. After completing this continuing education program, participants will be able to: 1) Identify situations for which surveys may be utilized to collect data 2) Identify important research design and sampling considerations in survey research 3) Design an appropriate cover letter to accompany a mail survey that maximizes its rate of return 4) Select an appropriate scaling procedure 5) Describe methods used to assess the validity and reliability of a summated ratings scale 6) Describe procedures used to determine the potential for non-response bias in mail surveys

PPN6: VALIDATION OF THE 24 HOUR HEADACHE DISABILITY QUESTIONNAIRE (DISQ-24) IN A SAMPLE OF HEADACHE SUFFERERS

OBJECTIVES: The Disq-24 was developed to measure headache disability on a 24-hour basis and was first utilized in a sample of migraine subjects who experienced chronic daily headaches. The purpose of this study was to evaluate this instrument's psychometric properties. METHODS: The Disq-24 is a 14-item questionnaire with a 6-point Likert-type scale that measures disability with regards to three hypothesized dimensions: family/social (4-item), work activities (5-item), and emotions/feelings (5-item). This instrument was administered in a prospective, multicenter, randomized, double-blind, placebo-controlled, parallel group trial of 170 transformed migraine sufferers. Four scaling assumptions of summated rating scales (equal variances, equal weights, item-internal consistency and discriminant validity) and scale-level reliability were examined using the baseline data. RESULTS: The means and standard deviations of items within each scale were similar. However, item evaluation demonstrated a floor effect in all items ranging from 22% to 66%. Item internal consistency was generally high and ranged from 0.62 to 0.90. Cronbach's alpha ranged from 0.93 to 0.95 for the three dimensions and was 0.97 for the total score, exceeding the generally acceptable criteria of 0.70. Although each item was highly correlated with the hypothesized scale, the item-scale correlations did not discriminate significantly with other scales. CONCLUSIONS: Although the Disq-24 satisfied the equal variance, equal weights, and internal consistency assumptions of summated rating scales, item discriminant validity was not supported in the above analysis. This lack of support could be due to the small sample size and/or the existence of a unidimensional structure. The validity of the instrument warrants further testing in a headache population which is not restricted to chronic daily headache patients.

Women's confidence to consume adequate servings of milk products: validation of a summated rating scale.

To design and validate a scale to assess women's confidence to consume adequate servings of milk products. Using the electoral rolls as a sampling frame, two large random population surveys of women aged 25-70 years were conducted. The structure of the self-efficacy scale was explored in the first survey of 500 women, and the component structure was refined and confirmed in an independent sample of 1200 women. The final single component self-efficacy scale consisted of nine items to assess confidence to consume milk products in various situations. Confidence was higher in women who were already consuming adequate servings of milk products than in women who were not yet ready to consume adequate servings of milk products, providing construct validity for the scale. The data provide preliminary support for the validity of the self-efficacy scale. The potential usefulness of the scale in interventions designed to improve women's milk product consumption is discussed.

Evaluating Farmers’ Knowledge and Awareness of Integrated Pest Management (IPM): Assessment of the IPM Collaborative Research Support Project in Uganda

The IPM CRSP (Integrated Pest Management Collaborative Research Support Program) has been applying a farmer participatory IPM strategy at on-farm research sites in Eastern Uganda since 1995. Comparison groups composed of project participants and nonparticipants were used to evaluate the impact of project activities on IPM knowledge and awareness change among two hundred small scale farmers. The evaluation instrument used a summated ratings scale consisting of four attributes to measure farmers’ knowledge of IPM, and crop specific indices to measure pest management. The results demonstrate that more active participation increases knowledge of IPM pest management knowledge, providing preliminary support for the project’s participatory research and extension approach. However, project beneficiaries were relatively few and were slightly more socioeconomically advantaged. Recommendations for increasing the number of farmer participants and improving the evaluation process are made.

Measuring quality of sedation in adult mechanically ventilated critically ill patients: the Vancouver Interaction and Calmness Scale

There are no reliable, valid, and responsive scales to measure the quality of sedation in adult critically ill patients. Our objective was to develop a summated rating scale with these properties and to define the minimal clinically important difference (MCID). We developed and tested the scale in an 18-bed medical–surgical intensive care unit (ICU) (12-bed acute and 6-bed subacute unit). Following identification of relevant domains and item derivation, 116 observations were made on 38 patients; psychometric properties and interrater reliability were assessed to allow item reduction. The final scale consisted of two five-item subscales quantifying calmness and interaction along a continuum from 5 to 30 points. Interrater reliability was 0.89 and 0.90; internal consistency was 0.95 for both subscales. To test construct validity, MCID, and responsiveness 302 observations were made on 54 patients. Construct validity: calmness score vs. need for further intervention to make the patient calm ( R = −0.82, P < 0.001); interaction score discriminated between acute vs. subacute units, mean scores 15.28 ± 8.26 vs. 23.54 ± 7.42, mean difference 8.27 (95% CI − 10.32 to −6.22); MCID - 2.2 and 2.5 for the calmness and interaction subscales; Guyatt's responsiveness statistics - 1.4 and 2.3. The Vancouver Interaction and Calmness Scale (VICS) is reliable, valid, and responsive.

Validity and reliability of the migraine-specific quality of life questionnaire (MSQ Version 2.1).

To assess the scaling properties, reliability, and validity of the revised Migraine-Specific Quality of Life Questionnaire (MSQ) (Version 2.1) The MSQ is a disease-specific, quality-of-life instrument with three hypothesized scales that has been developed, tested, and revised. The study used a multicenter, nondrug, prospective, parallel group, quasi-experimental design. Patients with migraine were recruited at outpatient headache specialty practices and were administered the MSQ, the Medical Outcomes Study 36-Item Short Form Health Survey (SF-36), and migraine symptom questionnaires at baseline, 4 weeks, and 12 weeks. Internal consistency (Cronbach alpha) and 4-week test-retest reproducibility (intraclass correlation coefficients) were estimated to assess reliability. Construct validity was assessed using an adaptation of the Campbell and Fiske multitrait-multimethod approach and by correlating MSQ scores with symptom measures. A total of 267 subjects enrolled in the trial. The criteria for summated rating scales were all met. The internal consistency coefficients ranged from 0.86 to 0.96, and the intraclass correlation coefficients ranged from 0.57 to 0.63 across the three dimensions. As anticipated, the MSQ dimensions had low-to-modest correlations with the two component scores of the SF-36 and were modestly to moderately correlated with migraine symptoms. The MSQ is a reliable instrument in the assessment of quality of life for patients with migraine with items that can be summed without weights. The MSQ has demonstrated evidence of construct validity.

Background sound pressure fluctuations (5 DB) from overhead ventilation systems increase subjective fatigue of university students during three-hour lectures.

During each of four successive sessions (once per week), 21 university students attended 3-hr. lectures. During alternative weeks the fans of the room's ventilation system were either on or off. When operating, they generated an average sound pressure that varied continuously between 60 and 65 dB. The dominant frequency of this 5-dB amplitude modulation of sound pressure was within the electroencephalographic range (5 Hz to 25 Hz). At the end of each hour of the lecture for each session each student estimated on 7-point summated rating scales fatigue (none to maximum) and concentration (poor to excellent). As a group, the students reported more fatigue during lectures when the fans were operating relative to lectures when the fans were not operating. This environmental effect explained about 30% of the variance in fatigue ratings and may be sufficient to affect adversely the attention of students within these settings.