Percentage Exact Agreement Research Articles

Long-term test-retest reliability of personality disorder diagnoses in opiate dependent patients.

This investigation reports the two year test-retest reliability of DSM-III-R personality disorder (PD) diagnoses in a sample of 219 patients with opiate dependence admitted to methadone treatment. Different MA/PhD interviewers at each assessment used a semistructured diagnostic interview for PD, the Structured Interview for DSM-III-R Personality Disorders (SIDP-R), to make their diagnoses. The reliability of any PD diagnosis versus no PD was fair (kappa = .51). The reliability for any specific PD (weighted kappa = .31) was poor. Antisocial (kappa = .45) and sadistic (kappa = .42), were the only specific PDs for which at least fair reliability was achieved. At the cluster level, only Cluster B had fair reliability (kappa = .47). The intraclass correlation coefficients between number of criteria for the specific PDs at the two evaluation points were consistently higher (range .22 to .62.) than were the corresponding kappas for categorical diagnoses. In that the base rates for most of the PDs were low and agreement for the specific PDs typically exceeded 90%. Increasing the base rate by lowering the diagnostic threshold, or examining more severe cases by raising the diagnostic threshold, did not consistently effect reliability. Reasons for the low kappa coefficients and the implications for PD research are discussed.

Reliability of Pain and Stiffness Assessments in Clinical Manual Lumbar Spine Examination

The purpose of this study was to determine the intertherapist reliability of judgments of stiffness and pain at L-1 to L-5 made using posteroanterior (PA) central pressure testing. Three pairs of manipulative physical therapists with a minimum of 5 years of experience were asked to rate pain and stiffness in a total of 90 patients with low back pain. Each pair of therapists assessed 30 patients within their own clinic, using their preferred technique to perform an examination using the PA central pressure test at the five lumbar levels. Each pair of therapists recorded their ratings of pain and stiffness. Reliability of judgments was evaluated by intraclass correlation coefficients (ICC) and percentage of exact agreement scores. The ICC values for pain judgments for the group as a whole ranged from .67 to .72, with agreement scores ranging from 31% to 43%. The ICC values for stiffness judgments ranged from .03 to .37, with agreement scores ranging from 21% to 29%. Judgments of stiffness made by experienced manipulative physical therapists examining patients in their own clinics were found to have poor reliability, whereas pain judgments had good reliability. Further investigation of this test is required in order to develop a more reliable method of assessing PA stiffness.

Department of Veterans Affairs Cooperative Studies Project number 147: level of examiner reliability over seven years.

In 1980 the Veterans Administration Cooperative Studies Program No. 147 initiated a multicenter, multidisciplinary clinical study to evaluate the use of alternative alloys for porcelain-fused-to-metal restorations. An investigation of interexaminer agreement was conducted after seven years utilizing three measures of reliability: percentage of exact agreement using a five-point scale, percentage of agreement using a dichotomous (satisfactory-unsatisfactory) scale, and [kappa], a statistic which is intended to measure agreement beyond what would be expected to occur by chance. Exact agreement on the five-point scale ranged from 77 to 99 percent. Percent agreement on the dichotomous scale was above 99 percent for all of the criteria. Kappa ranged from 0.56 to 0.91, indicating moderate to near perfect agreement among examiners. Reliability of the study evaluations reinforced the validity and significance of the findings.

A scale of primary classroom practices (SPCP)

The reliability and validity of a Scale of Primary Classroom Practices (SPCP) were tested using 12 independent observers who rated two first grade classrooms chosen for evaluation on the basis of their contrasting classroom practices. Observers were blind to the nature of these classrooms. Classroom A was identified as demonstrating a high degree of developmentally appropriate practices, while Classroom B was identified as demonstrating less developmentally appropriate practices. The mean percentages of exact agreement between the ratings of observers on the total SPCP was moderately high (85.30%) for Classroom A, but moderately low (50.50%) for Classroom B. However, the mean percentages of general agreement between the ratings of observers on the total SPCP was very high (98.10%) for Classroom A, and moderately high (76.00%) for Classroom B. Estimated alpha coefficients were very high, ranging for .95 to .99 for the total SPCP and its subscales. In addition, the total SPCP, including its subscales, signi...

Incidence of Associated Reaction in Young Boys and Reliability of Observations

This study was conducted to examine the incidence of associated reactions in boys and to determine the reliability of observing these reactions. The subjects were 89 healthy boys: five, seven, and nine years old. Each boy was asked to perform a set of three upper extremity tasks with both his preferred and non-preferred hand under two conditions: first with his eyes closed, and then with the eyes open, attempting to prevent associated movements. All trials were videotaped and later scored by two raters. Intra-rater reliability of detecting extraneous movements was found to be 88 and 89 percent exact agreement. Inter-rater reliability was 90 percent exact agreement. The incidence of ipsilateral associated movements was found to exceed the incidence of contralateral associated movements in all tasks. Age differences were found in the incidence of contralateral associated movements in some tasks, but not in the incidence of ipsilateral movements. No age differences were found in the ability to prevent associ...

Sleep-wake states in infant rabbits: Profiles from motility monitoring

Behavioral states in infant rabbits were determined from a single-channel analog recording of motility, including body movements and respiration. Behavioral states were first recorded by direct observation. Two observers simultaneously made 1-hour observations of bunnies at 7, 8, 9, 10, and 19 days of age, recording state every 10 seconds. Inter-observer reliability was assessed from epoch-by-epoch agreement on each observation. The mean percent exact agreement was 85.4. Then the animals were observed for one hour by one observer who recorded from direct observation while motility was recorded from a sensor-platform. Subsequently another person judged state from the analog recordings of motility patterns only. Validity assessment for these observations were also based on exact epoch agreement and the mean percent exact agreement was 84.0. The states recorded from motility patterns included: Waking, Sleep-Wake Transition, Active Sleep, and Quiet Sleep. Although state-related behaviors vary over this age range, the patterns of motility produced by body movements and respiration during waking and sleep states are consistently distinctive. These results suggest that the patterning of motility including motor movements and respiration is a total organismic expression of behavioral state. The procedure of motility recording permits prolonged monitoring periods and makes it possible to study the ontogeny of state organization in freely moving animals.