Proportion Of Agreement Research Articles

A Symptom Self-report Tool for Children With Inflammatory Bowel Disease (IBDnow).

Clinical symptom evaluation for children with inflammatory bowel disease (IBD) is typically done using composite tools: the Pediatric Crohn's Disease Activity Index (PCDAI) and Pediatric Ulcerative Colitis Activity Index (PUCAI). Both rely on clinician interpretation of child or parent symptom recall. No universal self-report tool has yet been developed for children with IBD to assess and report their symptoms. The research objective was to develop a self-report tool that produced information congruent with that obtained by clinicians using the PUCAI or PCDAI. A children's symptom self-report tool (IBDnow) was developed with picture and text Likert symptom scales. The clinician and child completed their reports during the same outpatient consultation. Agreement levels were calculated at the individual level (identical child and clinician answers), category level (symptom severity), and aggregate level (cohort scores). Internal consistency was measured with Cronbach alpha. One hundred children from Christchurch (New Zealand) (n = 65), and Sydney (Australia) (n = 35) completed the study (Crohn's Disease (CD):88, ulcerative colitis (UC):12), mean age 13.9 years (±3.6). Mean individual agreement was 0.76 (±0.19). Category severity had very good or good inter-rater reliability for 5 of the 7 symptom scales and overall severity agreement of 76%. Aggregate mean scores were significantly different between clinicians (14.9, ±18.8), and participants (21.6, ±19.4), (P <0.005, confidence interval -9.0, -4.4), but 60 pairs had scores within a 10% margin. Cronbach alpha was 0.74. This self-report tool had good proportionate agreement between raters, and good crude agreement for symptom categories. Assigning PUCAI or PCDAI scores caused inter-rater discrepancies to be misleadingly magnified. Pediatric gastroenterologists may consider utilizing IBDnow to elicit symptom self-reports from children with IBD to enable them to communicate meaningful information on their ongoing symptom burden. This would be a positive step in helping children feel included in clinical encounters and promoting self-management, at the same time producing valid, subjective symptom recall.

Validity and efficiency of a smartphone-based electronic data collection tool for operative data in rotator cuff repair

This study tested validity and efficiency of Orthopaedic Minimal Data Set (OrthoMiDaS) Episode of Care (OME). We analyzed 100 isolated rotator cuff repair cases in the OME database. Surgeons completed a traditional operative note and OME report. A blinded reviewer extracted data from operative notes and implant logs in electronic medical records by manual chart review. OME and electronic medical record data were compared with data counts and agreement between 40 variables of rotator cuff disease and repair procedures. Data counts were assessed using raw percentages and McNemartest (with continuity correction). Agreement of categorical variables was analyzed using Cohenκ (unweighted) and of numerical variables using the concordance correlation coefficient (CCC). Efficiency was assessed by median time to complete. OME database had significantly higher data counts for 25% (10/40) of variables. A high level of proportional and statistical agreement was demonstrated between the data. Among 35 categorical variables, proportional agreement was perfect for 17%, almost perfect (0.81 ≤ κ ≤ 1.00) for 37%, substantial (0.61 ≤ κ ≤ 0.80) for 20%, moderate (0.41 ≤ κ ≤ 0.60) for 14%, fair (0.21 ≤ κ ≤ 0.40) for 6%, and slight (0.0 ≤ κ ≤ 0.20) for 6%. Of 5 numerical variables, agreement was almost perfect (CCC > 0.99) for 20%and poor (CCC < 0.90) for 80%. Median OME completion time was 161.5 seconds (interquartile range, 116-224.5). OME is an efficient, valid tool for collecting comprehensive, standardized data on rotator cuff repair.

Content validity analysis of first semester formative test on biology subject for senior high school

This study aims to analyze the content validity of first-semester formative test instrument of biology subjects on senior high school in the early stages of developing the good test instrument. The test was multiple-choice format with a reason which uses for diagnosis of biology misconceptions. The content validity was determined by quantitative analysis of expert judgments and qualitative expert reviews. Two quantitative approaches to content validity estimations (Lawshe’s CVR and Aiken’s V) were compared in the analysis of a newly developed instrument which consists of 35 items. The data obtained from a panel of five expert judges. A Content Validity Ratio (Lawshe’s CVR) initially determined that only one item lacked inter-rater proportion agreement about its essentiality to the test instrument (CVR = - (0,2). The result of further content validity analysis shows three items had low content validity coefficient (Aiken’s V), that is indicating poor item relevance to the test. The qualitative reviews suggest to give attention to questions stem language to nine items include three items earlier. The findings supported the revision of nine items.

An evaluation of diagnostic agreement rates between advanced practice physiotherapists and paediatric orthopaedic consultants for children with musculoskeletal complaints.

The purpose of the present study was to establish diagnostic agreement rates between orthopaedic consultants and advanced practice physiotherapists (APPs) for paediatric orthopaedic patients, examine the appropriateness of referrals by APPs to consultants and report on the surgical conversion rate (SCR). A retrospective review of all patients referred from an APP clinic to orthopaedic consultants was conducted for 2013, with a 3-year longitudinal follow-up through to the end of 2016. Study participants were two APPs and four orthopaedic consultants at a single site. Descriptive statistics report the proportion of appropriate onward referrals and SCRs. Raw proportion agreement and kappa coefficients were used to evaluate diagnostic agreement rates. The mean raw agreement was 82% (95% confidence interval = 73 to 87), with 12 of the 15 individual codes demonstrating agreement rates ≥90% (range 57-100). Good to excellent agreement was demonstrated in almost all categories based on the kappa coefficient. Eighty-seven per cent of referrals from APPs were deemed to be appropriate by the orthopaedic consultants. The SCR was 23%. This was the first study to evaluate diagnostic agreement rates between APPs and orthopaedic consultants, the appropriateness of onward referral by APPs and the SCR for paediatric musculoskeletal patients referred from an APP clinic. Good to excellent diagnostic agreement was observed for routine elective paediatric orthopaedic patients, in tandem with agreement levels reported in the adult literature. The study demonstrates that APPs are clinically effective in the diagnosis and onward referral of paediatric orthopaedic patients in a triage setting.

Validation of a Novel Surgical Data Capturing System Following Total Hip Arthroplasty

BackgroundThe OrthoMiDaS (Orthopedic Minimal Data Set) Episode of Care (OME) database was developed in an effort to advance orthopedic outcome measurements on a national scale. This study was designed to evaluate if the OME data capture system would increase the quality of data collected in the context of primary and revision total hip arthroplasty (THA) compared to conventional operative notes. MethodsThis study includes data from the first 100 primary THAs and 100 revision THAs performed by 15 surgeons at a single institution from January through April 2016. Surgeons prospectively entered procedural details into OME following surgery. The OME database and operative notes were compared to evaluate completion rates and agreement. Completion rates were compared using McNemar’s test (with continuity correction), while agreement was analyzed using Cohen’s kappa (κ) and concordance correlation coefficient. ResultsThe OME database had significantly higher completion rates for 41% (39/96) of the variables. Proportion of data points that matched between the operative notes and OME data revealed that 54% (52/96) had a proportion agreement >0.90, and 79% (76/96) had a proportion agreement >0.80. In regard to measured agreement, 25% (24/96) of variables had almost perfect agreement, 29% (28/96) had substantial agreement, and 14% (13/96) had moderate agreement. Only 4% (4/96) had fair agreement, 8% (8/96) had slight agreement, and 6% (6/96) had poor agreement. ConclusionThe OME data capture system is an efficient tool to document procedural details following THA. The system is user-friendly, comprehensive, and accurate. It has the potential to be a valuable tool for future orthopedic research.

Development and validation of bioelectrical impedance analysis equations for prediction total body water and fat-free mass using D2O technique in Moroccan children aged between 8 and 11 years old.

Estimating body composition using bioelectric impedance analysis (BIA) requires specific prediction equations. The purpose of our study was to examine the validity of published BIA equations for assessing total body water (TBW) and fat-free mass (FFM) using deuterium oxide dilution (D2O) as a reference method and to develop new FFM and TBW BIA equations if needed for Moroccan pre-pubertal children. Data were collected from 247 schoolchildren aged 8-11 years old. Children were sorted by gender and age group and assigned in development and validation groups. D2O dilution was used as reference method for estimating TBW and FFM. Bland and Altman test, effect size, pure error, and proportional bias were used to assess the reliability of previous published equations. Cross-validation was performed by Bland and Altman test and BIA new equations were developed by linear regression. Previously published equations were tested and showed significant bias values indicating that if used they would provide biased values of TBW and FFM. The new prediction equations developed were: [Formula: see text] (l) = 0.269 + 0.292 Ht2/R (cm2/Ω) + 0.221 weight (kg) + 0.824 sex (boys = 1, girls = 0) + 0.291 age (years) (R2 = 0.91, RMSE = 1.54 kg); [Formula: see text] (kg) = -0.450 + 0.380 Ht2/R (cm2/Ω) + 0.291 weight (kg) + 1.294 sex (boys = 1, girls = 0) + 0.446 age (years) (R2 = 0.91, RMSE = 1.97 kg). These equations provide better values of proportional bias, agreement, and pure error than the other tested equations. The new BIA prediction equations seem to be the most accurate for Moroccan pre-pubertal children.

Effect of clinical information and previous exam execution on observer agreement and reliability in the analysis of hysteroscopic video-recordings.

Inter-observer agreement and reliability in hysteroscopic image assessment remain uncertain and the type of factors that may influence it has only been studied in relation to the experience of hysteroscopists. We aim to assess the effect of clinical information and previous exam execution on observer agreement and reliability in the analysis of hysteroscopic video-recordings. Ninety hysteroscopies were video-recorded and randomized into a group without (Group 1) and with clinical information (Group 2). The videos were independently analyzed by three hysteroscopists, regarding lesion location, dimension, and type, as well as decision to perform a biopsy. One of the hysteroscopists had executed all the exams before. Proportions of agreement (PA) and kappa statistics (κ) with 95% confidence intervals (95% CI) were used. In Group 2, there was a higher proportion of a normal diagnosis (p<0.001) and a lower proportion of biopsies recommended (p=0.027). Observer agreement and reliability were better in Group 2, with the PA and κ ranging, respectively, from 0.73 (95% CI 0.62, 0.83) and 0.44 (95% CI 0.26, 0.63), for image quality, to 0.94 (95% CI 0.88, 0.99) and 0.85 (95% CI 0.65, 0.95), for the decision to perform a biopsy. Execution of the exams before the analysis of the video-recordings did not significantly affect the results. With clinical information, agreement and reliability in the overall analysis of hysteroscopic video-recordings may reach almost perfect results and this was not significantly affected by the execution of the exams before the analysis. However, there is still uncertainty in the analysis of specific endometrial cavity abnormalities.

Word intuition agreement among Chinese speakers: a Mechanical Turk-based study

Word intuition is speakers’ intuitive knowledge on wordhood. Collective word intuition is the word intuition of the whole language community. Given this definition, the optimal word segmentation result in Chinese NLP should reflect collective word intuition. It is also believed that an ideal definition of Chinese word should accord with the collective word intuition of Chinese speakers. To test the validity and feasibility of modeling collective word intuition, it is important to know to what extent Chinese speakers agree with each other on what is a word. In this study, we measured word intuition agreement using Mechanical Turk-based Chinese word segmentation experiment. Three metrics were used: proportionate agreement, Cohen’s kappa, and Fleiss’ kappa. The results show that Chinese speakers agree with each other almost perfectly on what is a word. And we found no evidence to support an effect of semantic transparency on word intuition agreement. Such high word intuition agreement among Chinese speakers supports the psychological reality of Chinese word and also suggests that that it is quite feasible to formulate a definition of Chinese word by modeling the collective word intuition of Chinese speakers.

DSCImageCalc – Software for Determining Similarity Coefficients for the Analysis of Image Segmentations

DSCImageCalc is free software for calculating similarity coefficients on segmented images. It is written in Visual BASIC .NET and is available at: https://doi.org/10.6084/m9.figshare.894428 . It presently calculates Sorensen-Dice Coefficient, Jaccard Coefficient, Proportional Agreement, Cohen’s Kappa, Goodman & Kruskal’s Gamma, and Rogot-Goldberg Agreement. It also calculates segment centroids and the distance between them. Whilst it was originally designed for calculating Dice coefficients on segmentations of ultrasound images, it has potential for re-use in any field where similarity of image segments is to be analysed; predominantly medical radiology and ecology.

Data-driven mapping of hypoxia-related tumor heterogeneity using DCE-MRI and OE-MRI.

PurposePrevious work has shown that combining dynamic contrast‐enhanced (DCE)‐MRI and oxygen‐enhanced (OE)‐MRI binary enhancement maps can identify tumor hypoxia. The current work proposes a novel, data‐driven method for mapping tissue oxygenation and perfusion heterogeneity, based on clustering DCE/OE‐MRI data.MethodsDCE‐MRI and OE‐MRI were performed on nine U87 (glioblastoma) and seven Calu6 (non‐small cell lung cancer) murine xenograft tumors. Area under the curve and principal component analysis features were calculated and clustered separately using Gaussian mixture modelling. Evaluation metrics were calculated to determine the optimum feature set and cluster number. Outputs were quantitatively compared with a previous non data‐driven approach.ResultsThe optimum method located six robustly identifiable clusters in the data, yielding tumor region maps with spatially contiguous regions in a rim‐core structure, suggesting a biological basis. Mean within‐cluster enhancement curves showed physiologically distinct, intuitive kinetics of enhancement. Regions of DCE/OE‐MRI enhancement mismatch were located, and voxel categorization agreed well with the previous non data‐driven approach (Cohen's kappa = 0.61, proportional agreement = 0.75).ConclusionThe proposed method locates similar regions to the previous published method of binarization of DCE/OE‐MRI enhancement, but renders a finer segmentation of intra‐tumoral oxygenation and perfusion. This could aid in understanding the tumor microenvironment and its heterogeneity. Magn Reson Med 79:2236–2245, 2018. © 2017 The Authors Magnetic Resonance in Medicine published by Wiley Periodicals, Inc. on behalf of International Society for Magnetic Resonance in Medicine. This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

Test–retest reliability of the Brunel Lifestyle Physical Activity Questionnaire

ObjectivesTo establish the test–retest reliability of planned physical activity (PPA) and unplanned physical activity (UPA) components of the Brunel Lifestyle Physical Activity Questionnaire (BLPAQ). To provide evidence of the BLPAQ's stability using the proportion of agreement (PoA) method over a 5-week period. DesignTest–retest over a 5-week period using three diverse samples of adults. MethodsThe 277 participants were subdivided into three adult samples: gymnasium users (n = 80), undergraduate students (n = 111), and university staff members (n = 86). They were asked to complete the test–retest measure in their places of exercise, study, or work respectively. ResultsCorrelation coefficients between test–retest administrations were calculated for each participant group and intraclass correlations were calculated for each item. Pearson's product-moment correlations ranged from r = 0.95 to r = 0.96 for the PPA subscale and r = 0.93 to r = 0.98 for the UPA subscale. Intraclass correlations ranged from R = 0.52 to R = 0.99 for PPA and R = 0.87 to R = 0.99 for UPA. Fisher's z tests indicated that the test–retest correlation coefficients for the BLPAQ subscales were, on the whole, significantly stronger than those of older, comparable subscales from lifestyle physical activity questionnaires. The PoA analysis for each item revealed that the test–retest administrations were in high agreement (>95%). ConclusionsOverall, the PPA and UPA factors of the BLPAQ demonstrated high reliability and stability. The present study also illustrates the utility of PoA analysis in establishing the stability of physical activity measures.

Pain Assessment Recommendations for Women, Made by Women: A Mixed Methods Study.

To quantitatively describe women's priorities for pain assessment and qualitatively explain unique features of women's pain experiences. Mixed-methods study that included a three-round Delphi study followed by in-depth interviews. Clinical research study. Twenty-three women with chronic pain recruited from three women's pain treatment facilities and one interdisciplinary chronic pain clinic. Phase 1 (Delphi) involved completion of a questionnaire that rated agreement with the importance of 32 commonly used pain assessment measures. Answers were compiled, and controlled feedback was provided after each round. This iterative process continued until acceptable stability was reached. Stability was defined as proportion agreement for each response that reached the a priori cutoff score of 75%. Phase 2 (qualitative) involved one-to-one telephone interviews that followed a semistructured interview guide partially informed from phase 1 findings. A descriptive approach summarized and described participants' perspectives while avoiding abstractions. Textual data were analyzed using content analysis. Phase 1 identified 15 pain assessments as important. Some commonly used pain assessment measures such as the numeric pain intensity rating scale did not reach agreement as important. However, no pain assessments reached agreement as unimportant. Ten additional women completed face-to-face interviews, and an overall theme of stigmatization emerged that highlighted the importance of soliciting the pain narrative and why some aspects of psychosocial pain assessment did not reach agreement. Priorities identified by women for the assessment of pain were largely consistent with expert recommendations; however, important differences were raised that merit consideration for clinicians to reduce stigma.

Comparison of three methods for evaluation of work postures in a truck assembly plant

This study compared the results of three risk assessment tools (self-reported questionnaire, observational tool, direct measurement method) for the upper limbs and back in a truck assembly plant at two cycle times (11 and 8 min). The weighted Kappa factor showed fair agreement between the observational and direct measurement method for the arm (0.39) and back (0.47). The weighted Kappa factor for these methods was poor for the neck (0) and wrist (0) but the observed proportional agreement (Po) was 0.78 for the neck and 0.83 for the wrist. The weighted Kappa factor between questionnaire and direct measurement showed poor or slight agreement (0) for different body segments in both cycle times. The results revealed moderate agreement between the observational tool and the direct measurement method, and poor agreement between the self-reported questionnaire and direct measurement.Practitioner Summary: This study provides risk exposure measurement by different common ergonomic methods in the field. The results help to develop valid measurements and improve exposure evaluation. Hence, the ergonomist/practitioners should apply the methods with caution, or at least knowing what the issues/errors are.

Real time RULA assessment using Kinect v2 sensor

The evaluation of the exposure to risk factors in workplaces and their subsequent redesign represent one of the practices to lessen the frequency of work-related musculoskeletal disorders. In this paper we present K2RULA, a semi-automatic RULA evaluation software based on the Microsoft Kinect v2 depth camera, aimed at detecting awkward postures in real time, but also in off-line analysis. We validated our tool with two experiments. In the first one, we compared the K2RULA grand-scores with those obtained with a reference optical motion capture system and we found a statistical perfect match according to the Landis and Koch scale (proportion agreement index = 0.97, k = 0.87). In the second experiment, we evaluated the agreement of the grand-scores returned by the proposed application with those obtained by a RULA expert rater, finding again a statistical perfect match (proportion agreement index = 0.96, k = 0.84), whereas a commercial software based on Kinect v1 sensor showed a lower agreement (proportion agreement index = 0.82, k = 0.34).

Automatic Coronary Artery Calcium Scoring on Radiotherapy Planning CT Scans of Breast Cancer Patients: Reproducibility and Association with Traditional Cardiovascular Risk Factors.

ObjectivesCoronary artery calcium (CAC) is a strong and independent predictor of cardiovascular disease (CVD) risk. This study assesses reproducibility of automatic CAC scoring on radiotherapy planning computed tomography (CT) scans of breast cancer patients, and examines its association with traditional cardiovascular risk factors.MethodsThis study included 561 breast cancer patients undergoing radiotherapy between 2013 and 2015. CAC was automatically scored with an algorithm using supervised pattern recognition, expressed as Agatston scores and categorized into five categories (0, 1–10, 11–100, 101–400, >400). Reproducibility between automatic and manual expert scoring was assessed in 79 patients with automatically determined CAC above zero and 84 randomly selected patients without automatically determined CAC. Interscan reproducibility of automatic scoring was assessed in 294 patients having received two scans (82% on the same day). Association between CAC and CVD risk factors was assessed in 36 patients with CAC scores >100, 72 randomly selected patients with scores 1–100, and 72 randomly selected patients without CAC. Reliability was assessed with linearly weighted kappa and agreement with proportional agreement.Results134 out of 561 (24%) patients had a CAC score above zero. Reliability of CVD risk categorization between automatic and manual scoring was 0.80 (95% Confidence Interval (CI): 0.74–0.87), and slightly higher for scans with breath-hold. Agreement was 0.79 (95% CI: 0.72–0.85). Interscan reliability was 0.61 (95% CI: 0.50–0.72) with an agreement of 0.84 (95% CI: 0.80–0.89). Ten out of 36 (27.8%) patients with CAC scores above 100 did not have other cardiovascular risk factors.ConclusionsAutomatic CAC scoring on radiotherapy planning CT scans is a reliable method to assess CVD risk based on Agatston scores. One in four breast cancer patients planned for radiotherapy have elevated CAC score. One in three patients with high CAC scores don't have other CVD risk factors and wouldn't have been identified as high risk.

Validity of subjective questionnaire in evaluating dwelling characteristics, home dampness, and indoor odors in Shanghai, China: Cross-sectional survey and on-site inspection

Numerous questionnaire-based studies were conducted to investigate associations of home environment with childhood health, whereas few studies confirmed validity of parents-reported questionnaire regarding dwelling characteristics and dampness-related indicators. According to data obtained by standard questions regarding these items via on-site inspection in 454 residences with young children during 2013–2014, and data collected for these items from their parents-reported questionnaires in cross-sectional survey during 2011–2012 in Shanghai, China, this paper validated consistencies of information for these items between the two phases, thus to confirm validity of parents-reported questionnaire regarding these items. According to Kappa (k) values, parents and inspectors had moderate/substantial consistencies regarding dwelling characteristics (k=0.212–0.894), but had slight consistencies regarding dampness-related indicators and indoor odors (absolute k=0.019–0.144). However, according to the observed, positive, and negative proportional agreement (Po, Ppos, and Pneg), parents and inspectors had substantial/prefect consistencies in most of the studied items (Po=0.45–0.96). These trends of consistencies for different items had no notable differences, regardless of father or mother as the questionnaire-reporter, and among the residences inspected in different seasons. This study suggests parents-reported questionnaire could be a reliable and valid method to indicate the overall status of dwelling characteristics and home dampness-related exposures.

57-06: Prevalence of Early Repolarization pattern in 12 lead electrocardiogram: a population based study

Background: Early Repolarization pattern (ERP) has been considered a benign finding. However, this has been recently challenged due to its association with idiopathic ventricular fibrillation. The exact prevalence and characterization in the Southern Cone of Latin America are unknown. Purpose: The main objective of the study was to determine the prevalence of ERP and its characteristics in Southern Cone of Latin America according to the last consensus (Macfarlane et al.). Methods: The study population consisted in subjects of the CESCAS I study. CESCAS I study is an observational prospective cohort study with a multistage probabilistic sample of 8000 participants aged 35–74 years from 4 cities representing the Southern Cone of Latin America designed to estimate the prevalence and distribution of and secular trends in major cardiovascular disease events and risk factors. In addition to undergoing standard resting 12-lead electrocardiography, the subjects completed a questionnaire regarding their health habits, known diseases, and medications. Biochemical and physical parameters were measured by a trained nurse. For the present analysis we use 5940 ECGs from 3 cities, we excluded 300 subjects for whom ECGs were missing or incomplete. We also excluded 242 subjects with bundle branch block, Wolf–Parkinson–White pattern, atrial fibrillation, and cardiac pacing. Two trained cardiologists, blinded to clinical data, independently assessed all ECGs and recorded the presence of ERP. A third cardiologist judged the disagreements. Interobserver reliability was evaluated. Results: A total of 5398 ECGs were analysed with a male proportion of 41.3% and a mean age of 54.3 ± 10.7 years. The inferior location was found in 67.94% of cases (267/5398). The most common type was the “slurring” appearance without ST elevation represented 76.34% of cases. There was an important strength of agreement between the two initial interpreters (k= 0.89, proportion agreement 86.46%). In the multivariable regression model independent predictors of ERP were body mass index (OR 0.97 95% CI 0.95; 0.99), male sex (OR 1.47 95% CI 1.19; 1.81) and Sokolov Index (OR 1.04 95% CI 1.02; 1.06), p vale < 0.01. Conclusions: Consistent with published estimates ranging from 1% to 13% we found an overall prevalence of ERP of 8.1%. Male sex, lower body mass index and lower Sokolov Index were independently associated with the presence of ERP. Figure 1. Weighted prevalence of ERP by age and gender.

Abstract P3-12-22: Reproducibility of automated coronary artery calcification scoring on radiotherapy treatment planning computed tomography scans of breast cancer patients

Abstract Introduction: Presence of coronary artery calcifications (CAC) is a major independent risk factor for cardiovascular (CV) disease. CAC can be visualized on CT scans. Most breast cancer patients planned for radiotherapy (RT) receive planning CT scans. These scans may provide a reliable estimate of a patients' CV risk. This study evaluated the feasibility and reproducibility of an automated algorithm for CAC scoring on RT planning CT scans of breast cancer patients. Methods: This study was conducted within the Utrecht cohort for Multiple BReast cancer intErvention studies and Long-term evaLuAtion (UMBRELLA), and includes 562 breast cancer patients undergoing RT at University Medical Center Utrecht. Planning CT scans were performed using a 16-slice scanner (16 x 0.75 mm collimation, 3 mm thickness, 120 kVp, with or without breath hold (BH), without ECG synchronization). CAC were automatically scored using an algorithm developed with chest CT scans that considers lesions &amp;gt;130 Hounsfield units as CAC. CAC were identified using a supervised pattern recognition based on texture, size, and spatial features. To test validity of automated CAC scoring, manually scoring by an expert was performed in all scans with CAC (n = 80) and a random sample of scans without CAC (n = 83). Interscan reproducibility of automated CAC scoring was assessed in patients having two scans (n = 295). All scans with CAC score ≥ 1000 were manually checked and corrected if appropriate. Agatston calcification scores were analyzed continuously and categorically (0, 1-10, 11-100, 101-400, &amp;gt;400). Agreement and reliability for categories were determined with proportional agreement (%) and linearly weighted kappa. Reliability of Agatston scores were assessed with Intraclass correlation coefficients (ICC). Results: Of 562 patients, 129 (23%) patients had CAC scores &amp;gt; 0 with a mean of 93 (standard deviation: 166). Four patients had CAC scores ≥ 1000, which were erroneous and corrected. Of the 163 CT scans scored manually and automatically, 58 (36%) were performed with BH. Proportion of agreement was 79% (95% Confidence Interval (CI): 0.72-0.85) for all 163 scans: 88% (0.76-0.95) for 58 scans with BH and 74% (0.65-0.82) for 105 scans without. Proportion of agreement beyond chance was 0.80 (95% CI: 0.74-0.87) for all scans: 0.86 (0.77-0.96) with BH and 0.77 (0.684-0.853) without. Agatston score ICC was 0.86 (95% CI: 0.82-0.90) for all scans: 0.95 (0.91-0.97) with BH and 0.67 (0.55-0.76) without. For the interscan reproducibility (n = 295), the majority of patients (81%) had one scan with BH and one scan without. Proportion of agreement was 84% (95% CI: 0.79-0.88) and reliability was 0.61 (95% CI: 0.50-0.72). Agatston score ICC was 0.75 (95% CI: 0.69-0.80). Conclusion: Automated CAC scoring on RT planning CT scans of breast cancer patients is feasible. Agreement with manually scored scans is high and higher in CT scans performed with BH. Interscan reproducibility is fair. Automated CAC scores ≥ 1000 should to be manually checked and corrected if necessary. Automated CAC scoring on RT planning CT scans of breast cancer patients is available for all patients undergoing RT, and can provide information on CV risk at no additional cost. Citation Format: Gernaat SAM, de Vos BD, Isgum I, Rijnberg N, Bijlsma RM, Takx RAP, Pignol JP, Leiner T, Grobbee DE, van der Graaf Y, van den Bongard DHJG, Verkooijen HM. Reproducibility of automated coronary artery calcification scoring on radiotherapy treatment planning computed tomography scans of breast cancer patients. [abstract]. In: Proceedings of the Thirty-Eighth Annual CTRC-AACR San Antonio Breast Cancer Symposium: 2015 Dec 8-12; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2016;76(4 Suppl):Abstract nr P3-12-22.

Reliability and criterion validity of an observation protocol for working technique assessments in cash register work

We evaluated the intra- and inter-observer reliability and criterion validity of an observation protocol, developed in an iterative process involving practicing ergonomists, for assessment of working technique during cash register work for the purpose of preventing upper extremity symptoms. Two ergonomists independently assessed 17 15-min videos of cash register work on two occasions each, as a basis for examining reliability. Criterion validity was assessed by comparing these assessments with meticulous video-based analyses by researchers. Intra-observer reliability was acceptable (i.e. proportional agreement >0.7 and kappa >0.4) for 10/10 questions. Inter-observer reliability was acceptable for only 3/10 questions. An acceptable inter-observer reliability combined with an acceptable criterion validity was obtained only for one working technique aspect, ‘Quality of movements’. Thus, major elements of the cashiers’ working technique could not be assessed with an acceptable accuracy from short periods of observations by one observer, such as often desired by practitioners.Practitioner Summary: We examined an observation protocol for assessing working technique in cash register work. It was feasible in use, but inter-observer reliability and criterion validity were generally not acceptable when working technique aspects were assessed from short periods of work. We recommend the protocol to be used for educational purposes only.

Inter-rater agreement of an alternative guideline for the assessment of the activities of daily living in home nursing

Method A quantitative comparative field study in a stratified random sample of 612 patients receiving home nursing for ADL deficiencies. A proportional stratification according to mutuality membership, region (Flanders, Brussels and the Walloon region) and the patient’s level of ADLdependency in 5 categories: minor dependency (T2 en T7), level A, level B, level C (which is the highest level) was performed. Home care nurses were invited by the ‘National College of Advising Physicians (NCAPh) and the Belgian sickness funds, to participate in the study and to perform ADL assessments according to two guidelines: the current guideline of the Belgian Evaluation Scale (BES) and an alternative guideline which was written by a workgroup of the Agreement Commission for home care nursing of the National Institute for Health and Disability Insurance. The weighted kappa statistic (κ) (and its 95% confidence interval) and the proportion agreement were the main outcome measures.