Validation Substudy Research Articles

Demographic-specific Validity of the Cancer Prevention Study-3 Sedentary Time Survey.

This study examined the 1-yr test-retest reliability and criterion validity of sedentary time survey items in a subset of participants from a large, nationwide prospective cohort. Participants included 423 women and 290 men age 31 to 72 yr in the Cancer Prevention Study-3. Reliability was assessed by computing Spearman correlation coefficients between responses from prestudy and poststudy surveys. Validity was assessed by comparing survey-estimated sedentary time with a latent variable representing true sedentary time estimated from the 7-d diaries, accelerometry, and surveys through the method of triads. Sensitivity analyses were restricted to 566 participants with an average of 14+ h of diary and accelerometer data per day for 7 d per quarter. Reliability estimates for total sitting time were moderate or strong across all demographic strata (Spearman ρ ≥ 0.6), with significant differences by race (P = 0.01). Reliability estimates were strongest for the TV-related sedentary time item (Spearman ρ, 0.74; 95% confidence interval, 0.70-0.77). The overall validity coefficient (VC) for survey-assessed total sedentary time was 0.62 (95% confidence interval, 0.55-0.69), although VC varied by age group and activity level (P < 0.05). However, VC were similar across groups (P < 0.05) when restricting to highly compliant participants in a sensitivity analysis. The Cancer Prevention Study-3 sedentary behavior questionnaire has acceptable reliability and validity for ranking or categorizing participants according to sedentary time. Acceptable reliability and validity estimates persist across various demographic subgroups.

Cohort Profile: the Predictors of Breast Cancer Recurrence (ProBe CaRE) Premenopausal Breast Cancer Cohort Study in Denmark

PurposeThe Predictors of Breast Cancer Recurrence (ProBe CaRe) study was established to evaluate modification of tamoxifen (TAM) effectiveness in premenopausal women through reduced activity of TAM-metabolising enzymes. It comprehensively evaluates...

The Clinical Validation of the Athlete Sleep Screening Questionnaire: an Instrument to Identify Athletes that Need Further Sleep Assessment

BackgroundPrevious research has established that general sleep screening questionnaires are not valid and reliable in an athlete population. The Athlete Sleep Screening Questionnaire (ASSQ) was developed to address this need. While the initial validation of the ASSQ has been established, the clinical validity of the ASSQ has yet to be determined. The main objective of the current study was to evaluate the clinical validity of the ASSQ.MethodsCanadian National Team athletes (N = 199; mean age 24.0 ± 4.2 years, 62% females; from 23 sports) completed the ASSQ. A subset of athletes (N = 46) were randomized to the clinical validation sub-study which required subjects to complete an ASSQ at times 2 and 3 and to have a clinical sleep interview by a sleep medicine physician (SMP) who rated each subjects’ category of clinical sleep problem and provided recommendations to improve sleep. To assess clinical validity, the SMP category of clinical sleep problem was compared to the ASSQ.ResultsThe internal consistency (Cronbach’s alpha = 0.74) and test-retest reliability (r = 0.86) of the ASSQ were acceptable. The ASSQ demonstrated good agreement with the SMP (Cohen’s kappa = 0.84) which yielded a diagnostic sensitivity of 81%, specificity of 93%, positive predictive value of 87%, and negative predictive value of 90%. There were 25.1% of athletes identified to have clinically relevant sleep disturbances that required further clinical sleep assessment. Sleep improved from time 1 at baseline to after the recommendations at time 3.ConclusionsSleep screening athletes with the ASSQ provides a method of accurately determining which athletes would benefit from preventative measures and which athletes suffer from clinically significant sleep problems. The process of sleep screening athletes and providing recommendations improves sleep and offers a clinical intervention output that is simple and efficient for teams and athletes to implement.

The epidemiology and transmission of methicillin-resistant Staphylococcus aureus in the community in Singapore: study protocol for a longitudinal household study

Background/aimMethicillin-resistant Staphylococcus aureus (MRSA) is one of the most common multidrug-resistant organisms in healthcare settings worldwide, but little is known about MRSA transmission outside of acute healthcare settings especially in Asia. We describe the methods for a prospective longitudinal study of MRSA prevalence and transmission.MethodsMRSA-colonized individuals were identified from MRSA admission screening at two tertiary hospitals and recruited together with their household contacts. Participants submitted self-collected nasal, axilla and groin (NAG) swabs by mail for MRSA culture at baseline and monthly thereafter for 6 months. A comparison group of households of MRSA-negative patients provided swab samples at one time point. In a validation sub-study, separate swabs from each site were collected from randomly selected individuals, to compare MRSA detection rates between swab sites, and between samples collected by participants versus those collected by trained research staff. Information on each participant’s demographic information, medical status and medical history, past healthcare facilities usage and contacts, and personal interactions with others were collected using a self-administered questionnaire.Discussion/conclusionUnderstanding the dynamics of MRSA persistence and transmission in the community is crucial to devising and evaluating successful MRSA control strategies. Close contact with MRSA colonized patients may to be important for MRSA persistence in the community; evidence from this study on the extent of community MRSA could inform the development of household- or community-based interventions to reduce MRSA colonization of close contacts and subsequent re-introduction of MRSA into healthcare settings. Analysis of longitudinal data using whole-genome sequencing will yield further information regarding MRSA transmission within households, with significant implications for MRSA infection control outside acute hospital settings.

Prevaccination epidemiology of herpes zoster in Denmark: Quantification of occurrence and risk factors.

Herpes zoster (HZ) is a vaccine-preventable disease caused by reactivation of the varicella-zoster virus. Unfortunately, formulation of recommendations on routine immunization is hampered by a lack of data on disease burden, since most countries do not record cases of HZ in the general population. We developed and validated an algorithm to identify HZ based on routinely collected registry data and used it to quantify HZ occurrence and risk factors in Denmark prior to marketing of the HZ vaccine. We included patients aged ≥40years with a first-time systemic Acyclovir, Valacyclovir, or Famciclovir prescription or a hospital-based HZ diagnosis in the Danish nationwide health registries during 1997-2013. In a validation substudy (n=176), we computed the proportion of persons with HZ among patients who redeemed antiviral prescriptions. In a cohort study, we computed age-specific rates of HZ (45,297,258 person-years). In a case-control study, we then computed odds ratios (ORs) for common chronic diseases and immunosuppressive factors among HZ cases (n=189,025) vs. matched population controls (n=945,111). Medical record review confirmed HZ in 87% (95% confidence interval: 79-93%) of persons ≥40years who dispensed antivirals at doses recommended for HZ. HZ rates increased from 2.15/1000 person-years in 40-year-olds to 9.45/1000 person-years in 95-year-olds. Rates were highest in women. HZ was diagnosed during hospitalization among 3.5%. As expected, persons with severe immunosuppressive conditions had the highest ORs of HZ (between 1.82 and 4.12), but various autoimmune diseases, asthma, chronic kidney disease, and inhaled glucocorticoids were also associated with increased ORs (between 1.06 and 1.64). This algorithm is a valid tool for identifying HZ in routine healthcare data. It shows that HZ is common in Denmark, especially in patients with certain chronic conditions. Prioritized vaccination of such high-risk patients might be an option in countries considering alternatives to universal vaccination.

A pooled analysis of exenatide use and risk of acute pancreatitis

Objective:To estimate the association between exenatide BID use and acute pancreatitis across two claims-based studies.Research design and methods:We pooled two cohort studies within separate commercial health insurance databases. We included initiators of exenatide BID and all other antihyperglycemic drugs without prior pancreatitis from 2005−2008. Poisson regression models provided rate ratios (RRs) and 95% confidence intervals (CIs) of the association of exenatide BID with acute pancreatitis adjusted for quintiles of propensity scores.Main outcome measures:Primary inpatient diagnoses of acute pancreatitis with correction for misclassification via a validation sub-study.Results:There were 49,956 initiators of exenatide BID and 692,333 initiators of other antihyperglycemic drugs. Patients in the two studies were similar on many demographic and clinical characteristics. Exenatide BID initiators had a higher prevalence of diagnoses consistent with diabetes complications (e.g. peripheral neuropathy) and cardiovascular risk factors (e.g. hypertension). In both studies, current exenatide BID use was not associated with uncorrected outcomes of acute pancreatitis (pooled RR 1.0; CI 0.8–1.3). PPV correction resulted in a slightly higher point estimate for current use (pooled RR 1.3; CI 1.0–1.7) and past use (pooled RR 1.6; 95% CI 1.2−2.1).Conclusions:These data are consistent with little or no higher risk of acute pancreatitis associated with current exenatide BID use relative to nonuse. Although previous work identified non-causal mechanisms, an increased incidence of acute pancreatitis following cessation of treatment remains a possibility. Bias due to residual confounding or outcome misclassification may remain, and should be considered a potential explanation for these findings.

Cotinine Levels Among Betel Quid Users and Cigarette Smokers in Cambodia

Smokeless tobacco use in the form of the betel quid is common in the Western Pacific Region, and yet few studies have determined the nicotine delivery of this habit. During a validation substudy, we randomly sampled 201 adults from a rural province of Cambodia and determined nonparametric (bootstrapped) confidence intervals (CIs) for salivary cotinine levels in tobacco users. We found that cotinine levels for daily betel quid use among women (95% CI = 218.6-350.0 ng/mL) were (1) similar to the levels for daily cigarette smoking in men (95% CI = 240.2-317.1 ng/mL) and (2) significantly higher than the levels for daily cigarette smoking in women (95% CI = 71.8-202.7 ng/mL). The 95% confidence range for these habits exceeded the threshold for addiction. Our findings from rural Cambodia indicate that the typical betel quid habit among women supports the same level of nicotine addiction as the typical cigarette habit in men.

Hypertension and risk of uterine leiomyomata in US black women

Previous studies have found a positive association between hypertension and risk of hysterectomy-confirmed uterine leiomyomata (UL). The association of hypertension with UL confirmed by ultrasound or other surgery is less clear. The present study evaluated the association of hypertension with UL incidence according to confirmation method (hysterectomy, other surgery or ultrasound) in the Black Women's Health Study, 1997-2007. We collected prospective data every 2 years on physician-diagnosed hypertension and UL in 22 530 premenopausal women. Validation sub-studies confirmed 99 and 96% of hypertension and UL self-reported diagnoses, respectively. Cox regression was used to estimate incidence rate ratios (IRRs) and 95% confidence intervals (CIs) for the association of hypertension and UL, adjusting for potential confounders. During 172 162 person-years of follow-up, there were 6447 incident cases of UL confirmed by ultrasound (n = 5111), hysterectomy (n = 670) or other surgery (n = 666). Treated hypertension was associated with UL confirmed by hysterectomy (IRR = 1.32, 95% CI: 1.06, 1.63), but it was not associated with UL confirmed by ultrasound (IRR = 1.05, 95% CI: 0.96, 1.16) or other surgery (IRR = 1.13, 95% CI: 0.88, 1.46). Treated hypertension was associated with UL confirmed by hysterectomy, but not UL confirmed by other methods (other surgery or ultrasound). These data suggest it is premature to conclude that hypertension is related to an increased risk of UL. Additional studies are needed to assess whether the association with hysterectomy-confirmed UL can be explained by other sources of bias, such as patient or physician preferences for specific types of medical care.

Alternative approaches to assessing intervention effectiveness in randomized trials: application in a colorectal cancer screening study

Previous analysis of a randomized community-based trial of a multi-component intervention to increase colorectal cancer (CRC) screening among Filipino Americans (n = 548) found significantly higher screening rates in the two intervention groups compared to the control group, when using intent-to-treat analysis and self-reported screening as the outcome. This report describes more nuanced findings obtained from alternative approaches to assessing intervention effectiveness to inform future intervention implementation. The effect of the intervention on CRC screening receipt during follow-up was estimated using methods that adjusted for biases due to missing data and self-report and for different combinations of intervention components. Adjustment for self-report used data from a validation substudy. Effectiveness within demographic subgroups was also examined. Analyses accounting for self-report bias and missing data supported the effectiveness of the intervention. The intervention was also broadly effective across the demographic characteristics of the sample. Estimates of the intervention effect were highest among participants whose providers received a letter as part of the intervention. The findings increase confidence that the intervention could be broadly effective at increasing CRC screening in this population. Subgroup analyses and attempts to deconstruct multi-component interventions can provide important information for future intervention development, implementation, and dissemination.

Maximum likelihood estimation of odds ratios in misclassified binary data with a validation substudy

We consider misclassified binary data with a validation substudy. For such data various methods have been developed for estimating the odds ratio. It is well-known that the maximum likelihood estimator (MLE) of the odds ratio is efficient but requires iterative algorithms to compute. In this article, we derive a closed-form formula for the MLE and its asymptotic standard error. We compute the closed-form MLE on a data set that has been analyzed by other methods, and the results are compared.

Likelihood-based confidence intervals for the risk difference of two-sample binary data with a fallible classifier and a gold standard

We develop likelihood-based confidence intervals for risk difference in two-sample misclassified binary data. Such data consist of two studies. The first study is the main study where individuals are classified by an inexpensive fallible classifier which may misclassify. The second study is a validation substudy where individuals are classified by using both the fallible classifier and an expensive gold standard which classifies perfectly. We propose and examine three likelihood-based confidence interval methods and conclude that the modified Wald method applied to small-number adjusted new data performs well and has nominal coverage probabilities.

Bayesian Inference of Odds Ratios in Misclassified Binary Data with a Validation Substudy

We propose a fully Bayesian model with a non-informative prior for analyzing misclassified binary data with a validation substudy. In addition, we derive a closed-form algorithm for drawing all parameters from the posterior distribution and making statistical inference on odds ratios. Our algorithm draws each parameter from a beta distribution, avoids the specification of initial values, and does not have convergence issues. We apply the algorithm to a data set and compare the results with those obtained by other methods. Finally, the performance of our algorithm is assessed using simulation studies.

Regression Calibration for Dichotomized Mismeasured Predictors

Epidemiologic research focuses on estimating exposure-disease associations. In some applications the exposure may be dichotomized, for instance when threshold levels of the exposure are of primary public health interest (e.g., consuming 5 or more fruits and vegetables per day may reduce cancer risk). Errors in exposure variables are known to yield biased regression coefficients in exposure-disease models. Methods for bias-correction with continuous mismeasured exposures have been extensively discussed, and are often based on validation substudies, where the "true" and imprecise exposures are observed on a small subsample. In this paper, we focus on biases associated with dichotomization of a mismeasured continuous exposure. The amount of bias, in relation to measurement error in the imprecise continuous predictor, and choice of dichotomization cut point are discussed. Measurement error correction via regression calibration is developed for this scenario, and compared to naïvely using the dichotomized mismeasured predictor in linear exposure-disease models. Properties of the measurement error correction method (i.e., bias, mean-squared error) are assessed via simulations.

Colorectal Cancer Screening: Prevalence Among Low-Income Groups With Health Insurance

We examined the prevalence of colorectal cancer (CRC) screening in a low-income, racial/ethnic minority sample, among whom 97 percent had health insurance that covered CRC screening. This is a model for examining the impact of health insurance on racial/ethnic disparities in screening. Screening rates (67 percent self-reported; 52 percent adjusted based on a validation substudy) were higher than among similar population-based samples who have lower levels of insurance coverage. There were no differences by race/ethnicity. This study suggests that insurance coverage for CRC screening should be considered as part of a comprehensive approach to address CRC disparities.

Maximum likelihood, multiple imputation and regression calibration for measurement error adjustment

In epidemiologic studies of exposure-disease association, often only a surrogate measure of exposure is available for the majority of the sample. A validation sub-study may be conducted to estimate the relation between the surrogate measure and true exposure levels. In this article, we discuss three methods of estimation for such a main study/validation study design: (i) maximum likelihood (ML), (ii) multiple imputation (MI) and (iii) regression calibration (RC). For logistic regression, we show how each method depends on a different numerical approximation to the likelihood, and we adapt standard software to compute both MI and ML estimates. We use simulation to compare the performance of the estimators for both realistic and extreme settings, and for both internal and external validation designs. Our results indicate that with large measurement error or large enough sample sizes, ML performs as well as or better than MI and RC. However, for smaller measurement error and small sample sizes, either ML or RC may have the advantage. Interestingly, in most cases the relative advantage of RC versus ML was determined by the relative variance rather than the bias of the estimators. Software code for all three methods in SAS is provided.

Twelve-year trends and correlates of dietary salt intakes for the general adult population of Geneva, Switzerland

Investigate dietary salt intake trends by gender, and their associations with risk factors for cardiovascular diseases in Geneva, Switzerland. Continuous surveillance of the Geneva general adult (35-74 years) population for 12 years (1993-2004) using a validated, semi-quantitative food frequency questionnaire (FFQ) in random, cross-sectional, representative samples (6688 men, 6647 women). Dietary salt intake assessment by FFQ excluded discretionary salt, but was calibrated on total salt intake using an independent validation substudy of 100 volunteers who additionally provided 24-h urine collections. Quartiles (mean) of calibrated dietary salt intake (g per day) were 9.9, 10.5, 11.2 (10.6) in men, and 7.0, 7.8, 8.9 (8.1) in women and were above current recommendations. Quartiles (mean) of salt density (g MJ(-1)) were 0.99, 1.16, 1.39 (1.23) in men, and 0.98, 1.12, 1.30 (1.17) in women. Both measures were stable during the 12-year surveillance period, regardless of hypertension treatment. Salt-density differences between cardiovascular disease risk factor subgroups were moderate. Salt density increased with age and body mass index. The main dietary non-discretionary salt food sources (men/women: 47/48%) were breads (17/17%), cheeses (11/10%), meat and meat products (8/7%), soups (6/9%) and ready-to-eat foods (5/5%). Salt intakes from all sources for the Geneva, and perhaps the Swiss adult population are above current recommendations. The quantitative and qualitative data provided in this paper could be used to develop and implement strategies for salt-intake reduction in Switzerland.

Nutritional status and lifestyles of adolescents from a public health perspective. The HELENA Project—Healthy Lifestyle in Europe by Nutrition in Adolescence

The HELENA Project—Healthy Lifestyle in Europe by Nutrition in Adolescence—is a European, collaborative research project financed by the EU Sixth Framework Programme in the area of nutrition-related adolescent health. The basic objective of the HELENA project is to obtain reliable and comparable data from a random sample of European adolescents (boys and girls aged 13–16 years) on a broad battery of relevant nutrition and health-related parameters: dietary intake, food choices and preferences, anthropometry, serum indicators of lipid metabolism and glucose metabolism, vitamin and mineral status, immunological markers, physical activity, fitness and genetic markers. The HELENA project is conceived as a scientific construction with four complementary sub-studies that are elaborated through 14 well-defined work packages. Sub-studies are focused, respectively, on “a cross-sectional description of lifestyles and indicators of nutritional status (HELENA-CSS)”, “a lifestyle education intervention programme (HELENA-LSEI), “a metabolic study with cross-over design (HELENA-COMS)” and a “study on behaviour, food preferences and food development” (HELENA-BEFO). The project unites 20 research centres from 10 European countries. In addition, the consortium comprises five SMEs (small and medium-sized enterprises) that are actively involved in the research activities. The core of the HELENA project study material is an overall European cohort of 3,000 adolescents, equally recruited in ten cities from nine countries. Standardization of methods among partners is a key issue in the project and is obtained through the development of standard protocols, training sessions, validation sub-studies and pilot projects. Health-related problems have a tendency to evolve in cycles, with ever new problems emerging in ever new contexts that call for appropriate and tailored actions. The HELENA project is expected to offer essential elements for use in the overall machinery of required public health nutrition cycles. It is of the greatest importance for its results to prove useful that it can communicate with other initiatives on the level of science and society.

Multiple-imputation for measurement-error correction

There are many methods for measurement-error correction. These methods remain rarely used despite the ubiquity of measurement error. Treating measurement error as a missing-data problem, the authors show how multiple-imputation for measurement-error (MIME) correction can be done using SAS software and evaluate the approach with a simulation experiment. Based on hypothetical data from a planned cohort study of 600 children with chronic kidney disease, the estimated hazard ratio for end-stage renal disease from the complete data was 2.0 [95% confidence limits (95% CL) 1.4, 2.8] and was reduced to 1.5 (95% CL 1.1, 2.1) using a misclassified exposure of low glomerular filtration rate at study entry (sensitivity of 0.9 and specificity of 0.7). The MIME correction hazard ratio was 2.0 (95% CL 1.2, 3.3), the regression calibration (RC) hazard ratio was 2.0 (95% CL 1.1, 3.7), and restriction to a 25% validation substudy yielded a hazard ratio of 2.0 (95% CL 1.0, 3.7). Based on Monte Carlo simulations across eight scenarios, MIME was approximately unbiased, had approximately correct coverage, and was sometimes more powerful than misclassified or RC analyses. Using root mean squared error as a criterion, the MIME bias correction is sometimes outweighed by added imprecision. The choice between MIME and RC depends on performance, ease, and objectives. The usefulness of MIME correction in specific applications will depend upon the sample size or the proportion validated. MIME correction may be valuable in interpreting imperfectly measured epidemiological data.

Validation of self-reported fall events in intervention studies

Objective: To determine if self-reported data recalling at least one fall over a sixmonth recall period could be used as a reliable measure of falls by comparing retrospective self-report with a ‘gold standard' method of reporting these events using a prospective calendar-reported method. Design: Prospective study. Setting: Community-dwelling older people living in New South Wales (NSW) and Queensland in Australia. Subjects: A stratified sample of 264 randomly selected participants from a concurrent randomized controlled trial (RCT) of preventive health assessment for people over the age of 70 years. Interventions: Intervention and control group participants in the RCT kept a calendar of fall events for a six-month validation substudy. At the end of six months, participants were asked to retrospectively report their falls over the six-month period. Main results: Results indicated that percentage agreement between retrospective self-report and the calendar report of falls was 84%, however, sensitivity was 56% (95% confidence interval (CI) 44.1−67.5). Ten participants (4%) gave false positive self-reports of falls, and 33 (13%) participants gave false negative self-reports of falls. Therefore, retrospective self-reported fall rates were likely to involve under-reporting of falls. Significant differences were found in the sensitivity of retrospectively selfreported falls reported by the intervention group participants (sensitivity=71%; 95% CI 56.6−85.5) compared with the control group (sensitivity=40.5%; 95% CI 24.7−56.4, P =0.008). Validity of self-reported injuries as a result of a fall was less accurate (71% agreement; sensitivity 24% (95% CI 16.0−33.6). Conclusions: As recall of falls was more accurate in the intervention group than in the control group, it could appear that the intervention had the effect of increasing falls compared to the control group. Using these results, a method was devised to weight self-reported retrospective data to compensate for the poorer recall demonstrated in the control group, so that a more accurate estimate of falls could be derived from the self-reported data.

RESPONSE LETTER TO DR. PATEL

To the Editor: Patel raises a number of concerns about the design of our study. Using administrative and patient-care databases, we identified subjects with atrial fibrillation (AF), identified risk factors (comorbidities) for stroke, and found that advancing age was a deterrent to warfarin administration for stroke prevention. The case definition of AF required both a healthcare encounter with a diagnostic code (International Classification of Diseases, Ninth Revision (ICD-9)) for AF as well as a recorded electrocardiogram (EKG) tracing for this arrhythmia. A medical record–based validation substudy demonstrated 97% accuracy of the diagnosis in the cohort. Patel noted that 747 potential subjects were excluded because ICD-9 diagnostic data did not confirm the EKG finding and suggest that inclusion of these individuals might have “dramatically” changed the results. Because a validation substudy of these excluded subjects confirmed AF in only 26% of the sample (unpublished data), we felt it prudent to exclude them from further analyses. Patel makes note of potential weaknesses in our method of ascertaining comorbidities and further point out that the “use of any prescription of warfarin as a marker of anticoagulation leaves the question of intensity and adequacy of the anticoagulation unanswered.” We agree with these concerns and note only that the available data limit the methods used and the questions that can be addressed in a nonexperimental (database) study. Such studies are rarely comprehensive or definitive but rather inform in an exploratory or confirmatory capacity. Despite this, we feel that our study clearly demonstrates that, in clinical practice, age is a deterrent to anticoagulant use for stroke prevention in AF. Financial Disclosure: All authors state that they have no financial support of any kind related to this paper. Author Contributions: Mary T. Brophy and Louis D. Fiore: study concept and design, analysis and interpretation of data, preparation of manuscript. Kerri E. Snyder: acquisition of data, analysis and interpretation of data, preparation of manuscript. Stephen Gaehde, Charlene Ives, and David Gagnon: analysis and interpretation of data. Sponsor's Role: This study was not funded nor supported by any pharmaceutical company.