Between-observer Repeatability Research Articles

Clinical Agreement between Nurses in the Ultrasound Measurement of Abdominal Aortic Diameter within a National Screening Programme

Accurate measurement is central to abdominal aortic aneurysm screening, but information concerning differences between observers using modern ultrasound devices is lacking. Our aim was to assess clinical agreement among nurses within a national screening programme. Between-observer repeatability was assessed among 2 pairs of nurses (A & B and C & D) screening a consecutive series of men at a single-community clinic in Grampian, Scotland. All 4 nurses used the same ultrasound device (GE-LOGIQe 1.5-4.6MHz curvilinear probe) to measure maximal infrarenal inner-to-inner (ITI) anteroposterior diameter in longitudinal and transverse planes. Nurses alternated in their measurements and were blinded to their partners' measurements. Participants remained supine while "double-scanned." Clinical agreement was assessed as twice the standard deviation (2 SD) of mean differences. Analysis was undertaken using IBM-SPSS-Statistics (version 22) using the Bland-Altman "limits of agreement" (95% LoA) approach (mean difference ±2 SD) and related plots. A total of 63 consecutive men underwent assessment (30 men by nurses A & B; 33 men by nurses C & D). Mean age 65.5years, brachial blood pressure 145/88mm Hg, current smokers 14%, never smoked 41%, diabetic 18%, arterial disease 11%, daily aspirin 16%, antihypertensives 35%, and statin therapy 44%. Mean aortic diameter (ITI) was 1.81centimeters (cm; range 1.28-2.45; SD 0.18). Pooled mean differences between nurses was 0.05cm (95% confidence interval 0.02-0.08); 2 SD±0.23 with 95% LoA -0.18 to 0.28cm. Repeatability was similar in both planes and for both pairs of nurses. Nurses can achieve a high level of agreement in the measurement of aortic diameter in a routine clinical setting.

Evaluation of udder firmness by palpation and a dynamometer

Swelling of the mammary gland is an important health status sign for clinical exploration and palpation is a routine diagnostic tool for mastitis detection in dairy cows. Data on repeatability or validity of specific methods of udder palpation are rare. The overall objective was to study the validity of estimates of udder firmness generated by palpation and by using a validated dynamometer. Specifically, we set out to determine within-observer repeatability and between-observer repeatability in 2 specific experiments. Additionally, we compared a 4-point palpation scoring system with estimates obtained with a dynamometer in this study. In a pilot trial, we determined the range of udder firmness of 25 cows and developed an in vitro model for udder firmness. This model enabled training of the observers and allowed investigating a 4-point palpation scoring system. In vivo, udder firmness was determined before and after milking by palpation and by using a dynamometer. Within-observer repeatability based on estimates of udder firmness of 25 cows obtained by 3 observers on a single day by palpation was 0.968. Within-observer repeatability of estimates of udder firmness of 25 cows obtained with the dynamometer by a single observer was 0.997. The coefficient of variation of the same measures was 9.1%. To determine between-observer repeatability (palpation: 0.932; dynamometer: 0.898), udder firmness of 100 cows was measured on 4 different days by 9 observers in experiment 2. Udder firmness in dairy cows could be measured repeatably with the dynamometer and by palpation, especially when performed by a single observer. Estimates of udder firmness generated by palpation and with the dynamometer were moderately related (correlation coefficient=0.54). Training of observers through the pilot trial or practical experience in the 4 d of the study in experiment 2 did not improve the correlation. Further research is warranted to understand how udder firmness develops in infected udders.

Reproducibility of fetal renal pelvis volume measurement using three‐dimensional ultrasound

To assess the reproducibility of fetal renal pelvis volume measurement in hydronephrotic kidneys using transabdominal three-dimensional (3D) ultrasound. The fetal renal pelvis volume was measured using 3D ultrasound in one kidney in each of 15 fetuses with hydronephrosis in the second or third trimester of pregnancy. Hydronephrosis was diagnosed when the fetus had an anteroposterior renal pelvis diameter > or = 5 mm. After volume acquisition by one of the observers, the repeatability of volume calculation with manual delineation of the fetal renal pelvis was assessed by six different observers using the Virtual Organ Computer-aided AnaLysis (VOCAL(trade mark)) imaging program. The intraclass correlation coefficients (ICC), coefficient of variation (CV) and within- and between-observer repeatability coefficient (r) were calculated and Bland-Altman plots were constructed. Both intra- and interobserver reliability of the fetal renal pelvis volume measurements were considered to be very good. For intraobserver reliability, the ICC was 0.996 and the CV was 10.8%. For the overall interobserver reliability, the ICC was 0.998 and the CV was 15.7%; the interobserver reliability between pairs of observers had ICCs between 0.994 and 0.999, and CVs between 19.5% and 7.6% for inexperienced and experienced observers, respectively. With 3D ultrasound using the VOCAL imaging program, it is technically feasible to reproduce fetal renal pelvis volume measurements. Further research to establish the clinical applications of this technology is warranted.

Indices of cardiovascular function derived from peripheral pulse wave analysis using radial applanation tonometry: a measurement repeatability study

Pulse wave analysis (PWA) using applanation tonometry is a non-invasive technique for assessing cardiovascular function. It produces three important indices: ejection duration index (ED%), augmentation index adjusted for heart rate (AIX@75), and subendocardial viability ratio (SEVR%). The aim of this study was to assess within- and between-observer repeatability of these measurements. After resting supine for 15 minutes, 20 ambulant patients (16 male) in sinus rhythm underwent four PWA measurements on a single occasion. Two nurses (A & B) independently and alternately undertook PWA measurements using the same equipment (Omron HEM-757; SphygmoCor with Millar hand-held tonometer) blind to the other nurse's PWA measurements. Within- and between-observer differences were analysed using the Bland-Altman ;limits of agreement' approach (mean difference +/- 2 standard deviations, 2SD). Mean age was 56 (blood pressure, BP 136/79; pulse rate 64). BP/PWA measurements remained stable during assessment. Based on the average of two PWA measurements the mean +/- 2SD between-observer difference in ED% was 0.3 +/- 2.0; AIX@75 1.0 +/- 3.9; and SEVR% 1.7 +/- 14.2. Based on a single PWA measurement the between-observer difference was ED% 0.3 +/- 3.3; AIX@75 1.7 +/- 6.9; and SEVR% 0.6 +/- 22.6. Within-observer differences for nurse-A were ED% 0.0 +/- 5.4; AIX@75 1.5 +/- 7.0; and SEVR% 1.7 +/- 39.0 (nurse-B: 0.1 +/- 3.8; 0.1 +/- 8.0; and 0.6 +/- 23.3, respectively). PWA demonstrates high levels of repeatability even when used by relatively inexperienced staff and has the potential to be included in the routine cardiovascular assessment of ambulant patients.

Echocardiographic assessment of blood flow volume in the superior vena cava and descending aorta in the newborn infant

Background:Clinical methods of assessing adequacy of the circulation are poor predictors of volume of blood flow in the newborn preterm. Doppler echocardiography can be used to assess perfusion at various...

Repeatability of non-invasive measurement of intracerebral pulse wave velocity using transcranial Doppler

In the present study, the repeatability of three techniques for measuring peripheral PWV (pulse wave velocity) has been studied. A transcranial Doppler provided a wave reading from the middle cerebral artery. Using the transit time between the R-wave of an ECG and the 'foot' of this wave we were able to calculate a PWV (PWV-brain). An ear clip transducer provided a pressure wave reading (PWV-ear). A third pressure reading came from a Finapres transducer on the left middle finger (PWV-finger). The PWV was calculated as distance between two points/transit time of the pulse wave. Eleven volunteers had three sets of readings averaged for each technique taken in two separate sessions. There was good agreement between observers for the mean PWV values, and good agreement for mean results in different sessions. The RC%s (repeatability coefficient percentages) for between-observer repeatability in each session were good and approximately equivalent for PWV-finger (5-7%) and PWV-brain (5-7%). The repeatability of the PWV-ear measurement was less satisfactory (8-18%). The RC% for the same observer between sessions was less good, being 11% for the PWV-finger, 16-17% for PWV-brain and 11-19% for PWV-ear. The RC%s for the inter-session inter-observer measurements were between 10.7-12.1% for the PWV-finger, 14.7-19.5% for PWV-brain and 8.3-15% for PWV-ear. The transit time RC%s were lower in most measurements. The between-observer repeatability of all measures was satisfactory. Owing to the less good repeatability on different occasions, the use of PWV-brain and PWV-ear will depend on the magnitude of differences to be expected.

Repeatability of central corneal thickness measurements measured with the Topcon SP2000P specular microscope

The non-contact specular microscope has become the method of choice for a quick, accurate and non-invasive assessment of central corneal thickness (CCT), which is an important variable to monitor before and after refractive surgery. The consistency of the results produced by such widely used methods/equipment must be assessed to determine their reliability. The purpose of this study was to assess within- and between-observer repeatability of, and to determine if a systematic bias exists in the measurements made by, the Topcon SP2000P specular microscope. The CCT of the right eyes of 70 adult subjects, divided equally between men and women, was assessed on two separate occasions (4-7 days apart) by each of two examiners using the low-intensity auto mode of the SP2000P specular microscope. The average CCT values for men and women, measured by one observer, were 0.52+/-0.03 mm (mean +/- SD) and 0.52+/-0.04 mm, respectively. The average for the entire sample was 0.52+/-0.04 mm. Within- and between-observer repeatability were assessed by plotting the mean difference (for each subject) between two readings made by the same observer or one each by both observers against the combined average CCT reading of both sessions; the mean difference between two sets of observations was not significantly different from zero (P<0.05). For the first observer, the 95% limits of repeatability were between -0.015 and 0.017 mm. For the second observer, the 95% limits of repeatability were between -0.018 and 0.018 mm. For the between-observer repeatability, the 95% limits of agreement were between -0.016 and 0.016 mm. For both within- and between-observer repeatability, we found no systematic bias of the mean difference with the average CCT reading. The within- and between-observer limits of agreement we found were similar to those previously reported for the Topcon SP2000P specular microscope; the range of the 95% limits of repeatability were within +/-1 SD of the average CCT reading for both sessions. We suggest that a technique be considered reliable if: (1) the mean difference between two measurements does not vary significantly from zero, (2) there is no systematic bias of the mean difference with the magnitude of the measured quantity and (3) the error inherent in a measurement technique is within +/-1 SD of the average measurement of the two sessions.

Criteria for assessing pain and nonarticular soft-tissue rheumatic disorders of the neck and upper limb

Objective: To critically review the criteria used to diagnose nonarticular soft-tissue rheumatic disorders of the neck and upper limb. Methods: An extensive search of the literature, including a search of Medline and EMBASE, authoritative recent reviews, and relevant textbooks, was completed. The diagnostic criteria used in epidemiologic studies were compared and the reliability and validity of these criteria were assessed. Results: Altogether, the search identified 117 relevant research articles, among which 69 included a physical examination component, but few specified diagnostic criteria. Evidence supported respectable levels of between-observer repeatability regarding: symptom questionnaires (κ, 0.52 to 0.79); measurement of shoulder range of motion with a goniometer (intraclass coefficients > 0.70); tests for carpal tunnel syndrome (Tinel’s and Phalen’s κ, 0.53 to 0.80); and demonstration of neck tenderness (κ = 0.43). The Katz hand diagram, and combinations of physical signs of carpal tunnel syndrome, show reasonable sensitivity and specificity for that diagnosis but only among patients referred to specialists with that putative diagnosis; no such validity has been shown among the general population. Only 1 diagnostic examination schedule has published data on both the reliability and the validity of its criteria and diagnoses. For the remaining soft-tissue upper-limb disorders, diagnostic criteria rely apparently on face and content validity and reliability data have not been published. Conclusion: Classification of specific disorders of the neck and upper limb requires a back to basics approach. At present, the diagnosis of most of these conditions relies heavily on the clinical opinions of investigators and there are insufficient data to indicate that these criteria are repeatable, sensitive, or specific. Recent European initiatives offer scope to follow a more disciplined approach, but more work is urgently required.

DEFINING OSTEOARTHRITIS OF THE HIP FOR EPIDEMIOLOGIC STUDIES

The authors compared seven radiologic indices of hip osteoarthritis to establish which provided the best definition of the disease for epidemiologic purposes. Hip joints were assessed from intravenous urograms taken in a British hospital between 1982 and 1987 in 1,315 men aged 60-75 years. The indices examined were an overall qualitative grading of osteoarthritis, four measures of joint space, the maximum thickness of subchondral sclerosis, and the size of the largest osteophyte. Minimal joint space (i.e., the shortest distance between the femoral head margin and the acetabulum) was the index most strongly associated with other radiologic features of osteoarthritis. Among a subset of 759 men who answered a questionnaire about symptoms, the overall qualitative grading, minimal joint space, and thickness of subchondral sclerosis were the radiologic indices most predictive of hip pain. Within- and between-observer repeatability were tested in a subset of 50 subjects. Measures of joint space were more reproducible than other indices. These data suggest that, at least in men, minimal joint space is the best radiologic criterion of hip osteoarthritis for use in epidemiologic studies.

Evaluation of PAS‐diastase and carcinoembryonic antigen staining in the differential diagnosis of malignant mesothelioma and papillary serous carcinoma of the ovary

A series of malignant mesotheliomas and papillary serous carcinomas of the ovary were stained using the periodic acid Schiff reaction with diastase digestion (PAS-D) and an immunoperoxidase technique for carcinoembryonic antigen (CEA). The methods were evaluated in terms of repeatability and validity. Staining reactions were classed as positive or negative by three 'blinded' observers. Within-observer repeatability was greater for PAS-D than for CEA but there was no statistically significant difference for between-observer repeatability. The validity of the methods clearly differed, however, since the sensitivity and specificity of CEA were consistently less than that of PAS-D. Our most consistent estimates of sensitivity and specificity were compared with those of other studies and the reasons for this difference were analysed. Positive PAS-D and CEA staining were uncorrelated but a combination of the two did not appear significantly superior to PAS-D alone. We believe that PAS-D remains the method of choice for differentiating peritoneal mesotheliomas and disseminated papillary serous carcinoma of the ovary in the absence of reliable evidence of an ovarian primary because the significance of CEA positivity in occasional mesothelial tumours remains to be determined.