Balanced Repeated Replication Research Articles

Scalar-on-function regression: Estimation and inference under complex survey designs.

Increasingly, large, nationally representative health and behavioral surveys conducted under a multistage stratified sampling scheme collect high dimensional data with correlation structured along some domain (eg, wearable sensor data measured continuously and correlated over time, imaging data with spatiotemporal correlation) with the goal of associating these data with health outcomes. Analysis of this sort requires novel methodologic work at the intersection of survey statistics and functional data analysis. Here, we address this crucial gap in the literature by proposing an estimation and inferential framework for generalizable scalar-on-function regression models for data collected under a complex survey design. We propose to: (1) estimate functional regression coefficients using weighted score equations; and (2) perform inference using novel functional balanced repeated replication and survey-weighted bootstrap for multistage survey designs. This is the first frequentist study to discuss the estimation of scalar-on-function regression models in the context of complex survey studies and to assess the validity of various inferential techniques based on re-sampling methods via a comprehensive simulation study. We implement our methods to predict mortality using diurnal activity profiles measured via wearable accelerometers using the National Health and Nutrition Examination Survey 2003-2006 data. The proposed computationally efficient methods are implemented in R software package surveySoFR.

Retention of Community Health Workers in the Public Health Workforce: Public Health Workforce Interests and Needs Survey, 2017 and 2021.

Objectives. To investigate the organizational factors contributing to the intent of community health workers (CHWs) to quit their jobs in local and state health departments in the United States. Methods. We used the 2017 (n = 844) and 2021 (n = 1014) Public Health Workforce Interests and Needs Survey data sets to predict CHWs' intent to leave with Stata 17 balanced repeated replication survey estimations. Results. CHWs dissatisfied with organizational support, pay, or job security had high probabilities of reporting an intent to leave (50%, P < .01; 39%, P < .01; and 42%, P < .01, respectively) relative to satisfied or neutral workers (24%, P < .01; 21%, P < .01; and 26%, P < .01, respectively). Conclusions. Improving organizational support, pay satisfaction, and job security satisfaction in public health agencies can significantly improve CHW retention, potentially lowering overall organizational costs, enhancing organizational morale, and promoting community health. Public Health Implications. Our findings shed light on actionable ways to improve CHW retention, including assessing training needs; prioritizing diversity, equity, and inclusion; and improving communication between management and workers. (Am J Public Health. 2024;114(1):44-47. https://doi.org/10.2105/AJPH.2023.307462).

Abstract 15711: Association Between State Tobacco Surcharge Policy and Health Insurance Enrollment, Health Care Utilization, and Smoking Cessation Behaviors

Introduction: The Patient Protection and Affordable Care Act allows insurers to charge up to 50% higher premiums to people who use tobacco. Several states have outlawed tobacco surcharges or capped the surcharges to lower levels. Evidence on the impact of state policies limiting tobacco surcharges is limited. This study aims to examine the association between state tobacco surcharge policies and individuals’ health insurance enrollment, health care utilization, and smoking cessation behaviors. Methods: This study used data from the Population Assessment of Tobacco and Health (PATH) Study Wave 6 (Mar-Nov 2021). The study sample was adults 18-64 years old who reported smoking cigarettes 20 or more days in the past 30 days (N=4,235). Exposure was living in states limiting tobacco surcharges (outlawed or capped). Outcome variables were currently covered by health insurance, seen a health professional in the past 12 months, intention to quit within 12 months, and any quit attempt in the past 12 months. Single-wave sample weights and balanced repeated replication method were applied to generate representative estimates and variances. Multivariate logistic regressions were used to estimate the associations between exposure and outcomes, controlling for socio-demographic variables, other tobacco use status, and state Medicaid expansion and tobacco control policies. Results: Among study sample, 19.8% (95% CI: 16.0%-24.3%) lived in states limiting tobacco surcharges. Bivariate associations showed that people living in states limiting surcharges were more likely to be covered by health insurance (84.5%, 95% CI: 81.2%-87.3% vs. 75.0%, 95% CI: 72.9%-77.1%, p &lt; 0.001). After adjusting for covariates, the association between state policies and health insurance coverage became statistically insignificant (adjusted odds ratio, aOR=1.09, 95% CI: 0.80-1.47). Participants living in states limiting surcharges were less likely to have intention to quit within 12 months (aOR=0.78, 95% CI: 0.65-0.94). Other associations were not statistically significant. Conclusions: This study indicated that people living in states limiting tobacco surcharges had lower intention to quit smoking. More studies are needed to evaluate the impact of tobacco surcharge policies.

Young videogamers and their approach to science inquiry

BackgroundWritten instructions seldom need to be read when playing videogames. Instead, gaming often involves early information foraging and expansive exploration behaviors. We use data from the Programme for International Student Assessment (PISA) to explore whether students who regularly play videogames (daily gamers) adopt behaviors that are typical of gaming while they complete a computer-based assessment of science and if such behaviors matter for performance in procedural science knowledge.MethodsWe use item-level data from PISA 2018 from countries that administered the optional ICT questionnaire. Using a sample item and the full set of interactive science items, we develop regression models to estimate differences between daily gamers and other students in information harvesting, time to first action, and procedural science knowledge. We report average associations across countries, country-specific estimates, as well as differences between boys and girls. We report raw differences as well as differences adjusting for background characteristics. To account for the multilevel nature of the data and for the two-stage sampling design, we use replicate weights supplied with the PISA database to estimate robust standard errors using balanced repeated replication (BRR).ResultsIn 2018, 33% 15-year-olds reported playing videogames every day or almost every day. Among boys, that proportion was 49%. Daily gamers do not differ from other students in science content knowledge and in reading fluency. Nevertheless, daily gamers spend marginally less time reading instructions and display more active exploration behaviors in the assessment on items that include simulation tools.ConclusionsScience teachers and assessment developers may find inspiration in games to develop scenarios in which students can practice effective strategies for information harvesting. The extensive exploration of a problem space in order to obtain data in support of future decisions often corresponds to a positive behaviour with multiple advantages in authentic problem situations. By contrast, fast transitions into action may, in the particular situation of an assessment, be an inadequate response. Assessment developers can ensure that instructions are carefully read and understood by test takers and teachers and can guide their students to read instructions adequately.

34 Normative beliefs about tobacco products differ by age: Implications for smoking cessation and harm reduction

OBJECTIVES/GOALS: The prevalence of combusted cigarette (CC) smoking among older adults is stagnant, with zero declines attributable to e-cigarette (EC) use. Normative beliefs predict quitting and switching to ECs (a behavior likely to yield health benefits for those unable to quit), so this study seeks to characterize the role of age in norms about CC and EC use. METHODS/STUDY POPULATION: Data come from Wave 5 (2018-2019) of the adult Population Assessment of Tobacco Use and Health (PATH) study, a nationally-representative, U.S. longitudinal cohort. Analyses were restricted to people with established CC use (smoked CCs in the past year, currently smoke regularly, and smoked ≥100 lifetime CCs; n=8,590). Cross-sectional weighted estimates of the prevalence of normative beliefs about CCs and ECs were calculated by age using the Balanced Repeated Replication (BRR) method with Fay=0.3. We used chi-square tests to examine the association of age group (18-24, 25-34, 35-44, 45-54, 55-64, or ≥65) with the prevalence of 2 descriptive and 4 personal social norms. RESULTS/ANTICIPATED RESULTS: The prevalence of the normative belief that most people disapprove of CCs (p DISCUSSION/SIGNIFICANCE: Older adults are more likely than younger adults to endorse anti-tobacco norms, which prior work shows is associated with quitting smoking. These beliefs could be leveraged to create targeted communications towards older adults encouraging smoking cessation. More research is needed to assess age-related tobacco beliefs and switching from CC to EC.

COVID-19 Vaccine Intentions Within the Medicare Population

COVID-19 vaccine intentions by older adults reflect individual care seeking behavior and medical system trust and broader systemic cultural shifts related to vaccine hesitancy. The purpose of this paper is to examine the October wave of the rapid response panel survey fielded by the Centers for Medicare and Medicaid Services (CMS) to track and monitor the effects of the pandemic within the Medicare population. With a sample size of 9686 Medicare beneficiaries, the calculated statistics use replicate weights to adjust for the complex survey sample design and balanced repeated replication using Fay’s adjustment of 0.3 for variance estimation. When asked about the likelihood of getting the COVID-19 vaccine if one were available, 58 percent of the Medicare population definitely or probably intended to get the vaccine, 16 percent expressed they would probably or definitely not, and 26 percent were not sure. Black or Hispanic Medicare beneficiaries were significantly more likely to express they would probably not or definitely not get the vaccine than White, non-Hispanic Medicare beneficiaries. Distrust of what government says about the vaccine and concern about the safety or side effects were the most common reasons for not intending to get the vaccine. Those expressing intentions to not get the COVID-19 vaccine in the October 2020 survey wave were more likely to lack access to the internet, which is a potential systematic barrier if they changed their intentions following the FDA approvals of the COVID-19 vaccines and more information became available in the winter and spring of 2021.

Abstract 411: New estimators for mediation effects in complex sampling schemes: A comparison with applications

Abstract Mediation analysis examines the indirect effects via a third variable on the pathway between exposures and outcomes. For the finite-population surveys, that often use stratified cluster sampling, the conventional methods assuming simple random sampling are not appropriate. The point estimation of the effect can be corrected using sampling weights, but the standard error needs further improvement. Recent studies have integrated mediation analysis with balanced repeated replication for standard error estimation. Alternative estimators include Jackknife repeated replication (JRR), bootstrap, and Taylor series linear approximation (TSE). This study proposes new standard error estimators by incorporating the classic linear mediation analysis with JRR, bootstrap, and TSE for complex surveys and compares them using Monte Carlo simulations. The preliminary results show that replication methods have lower Type-I error rates and higher confidence-interval coverage rates than conventional methods. The proposed methods are applied to National Health Interview Surveys to explore the mediation effects on the pathway between cancer survivors' health status and employment status. Citation Format: Yujiao Mai, Deo Kumar Srivastava. New estimators for mediation effects in complex sampling schemes: A comparison with applications [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2021; 2021 Apr 10-15 and May 17-21. Philadelphia (PA): AACR; Cancer Res 2021;81(13_Suppl):Abstract nr 411.

Distributed leadership practices and student science performance through the four-path model: examining failure in underprivileged schools

PurposeAs a failure analysis emphasizing school leadership in underprivileged schools serving socioeconomically disadvantaged and minority students, the study is interested in determining whether and to what extent variations in distributed leadership (DL) practices are related to student performance through the mediating effects of four-path variables.Design/methodology/approachThis research conducted secondary data analysis using the 2015 PISA American data. The study employed factor analysis and structural equation models (SEMs) to investigate multidimensional associations among a set of variables, including school socioeconomic status (SES), student composition, DL practices, school four-path factors and student performance. The research used a design-based resampling approach with balanced repeated replication (BRR) weights to analyze the complex survey data.FindingsThe results indicate that, within a DL framework, teacher leadership in instructional management is positively and directly related to student performance. Governing board leadership in school administration is indirectly related to student performance through four-path variables' mediating effects. Importantly, though the two leadership sources help improve student performance, they are less prevalent in underprivileged schools with disproportional minority and socioeconomically disadvantaged students.Originality/valueThis research is a failure analysis through the lens of DL for underprivileged schools. The study used rigorous quantitative approaches and examined multidimensional associations among school socioeconomic status (SES), DL, school factors that school leaders could maneuver and student performance. The evidence sheds light on remedial actions in failed schools to focus on improving teacher leadership and organizational capacity.

Focusing on the Practice of Distributed Leadership: The International Evidence From the 2013 TALIS

Purpose: This study is primarily interested in the operational evidence of distributed leadership using large-scale and multicountry data. Specifically, this study investigates (a) how three position-based human units—the principal, the management team, and teachers—could lead nine school leadership functions together and (b) how the country and school contextual factors are related to the leadership distribution mechanism, which examines the associations among variables across national boundaries including 32 countries. Research Design: This study conducted a secondary data analysis using the 2013 Teaching and Learning International Survey. The researcher first constructed the latent variables to quantify the extent to which the three groups led the nine leadership functions. Then a resampling approach with balanced repeated replication weights was used to disclose the extent to which each group led each of the 17 leadership tasks. Finally, a structural equation model helped reveal the existing associations among contextual factors and distributed leadership operation. Findings: First, distributed leadership was reported by principals to exist in 25 countries, and there were seven distributed leadership patterns discovered among 32 countries included in the sample, though two patterns were revealed to have little leadership distribution. Second, this study found that the principal, the management team, and teachers led varied leadership functions. Finally, both country and school contextual factors were associated with how leadership was distributed. Conclusions: This study, using large-scale and international data, has added new evidence with a particular interest in the effects of contextual influence on distributed leadership practices.

The Effect of Instructional Leadership and Distributed Leadership on Teacher Self-efficacy and Job Satisfaction: Mediating Roles of Supportive School Culture and Teacher Collaboration

Research evidence regarding the relative effects of instructional leadership and distributed leadership on teacher job satisfaction and self-efficacy is limited; it is even less evidential when the indirect effects of mediation variables between school leadership and teacher outcomes, including supportive school culture and teacher collaboration, are added to the total effects. In this study, the six aforementioned variables are added to one model focusing on both the direct effects instructional and distributed leadership have on teacher job satisfaction and self-efficacy, and the indirect effects through the mediation variables of supportive school culture and teacher collaboration. Using the 2013 Teaching and Learning International Survey data, this research applied a rigorous structural equation model (SEM)with the design-based approach using the balanced repeated replication (BRR) weights. The results suggest that distributed leadership and instructional leadership are both positively and directly associated with teacher job satisfaction and teacher self-efficacy, respectively. Meanwhile, distributed leadership is positively and indirectly associated with both teacher job satisfaction and self-efficacy, while instructional leadership is indirectly associated with teacher job satisfaction through the mediation effects of supportive school culture and teacher collaboration.

Adult E-Cigarettes Use Associated with a Self-Reported Diagnosis of COPD.

The use of electronic cigarettes (e-cigarettes) has increased in the US, but little is known about the effects of these products on lung health. The main purpose of this study was to examine the association between e-cigarette use and a participant’s report of being diagnosed with chronic obstructive pulmonary disease (COPD) in a nationally representative sample of adults. Methods: The first wave of the Population Assessment of Tobacco and Health (PATH) survey adult data was used (N = 32,320). Potential confounders between e-cigarette users and non-users were balanced using propensity score matching. Odds ratios (OR) were calculated to examine the association between e-cigarette use and COPD in the propensity-matched sample, the entire sample, different age groups, and in nonsmokers. Replicate weights and balanced repeated replication methods were utilized to account for the complex survey design. Results: Of the 3642 participants who met the criteria for e-cigarette use, 2727 were propensity matched with 2727 non e-cigarette users. In the propensity-matched sample, e-cigarette users were more likely to report being diagnosed with COPD (OR 1.43, 95% confidence interval [CI] 1.12–1.85) than non-e-cigarette users after adjusting for confounders. The result was similar in the entire sample and in the different age subgroups. Among nonsmokers, the odds of reporting a COPD diagnosis were even greater among e-cigarette users (OR 2.94, 95% CI 1.73–4.99) compared to non-e-cigarette users. Conclusion: Our findings demonstrate that e-cigarette use was associated with a reported diagnosis of COPD among adults in the US. Further research is necessary to characterize the nature of this association and on the long-term effects of using e-cigarettes.

Estimation of Small Area Characteristics Using Multivariate Rao-Yu Model

Abstract The growing demand for high-quality statistical data for small areas coming from both the public and private sector makes it necessary to develop appropriate estimation methods. The techniques based on small area models that combine time series and cross-sectional data allow for efficient "borrowing strength" from the entire population and they can also take into account changes over time. In this context, the EBLUP estimation based on multivariate Rao-Yu model, involving both autocorrelated random effects between areas and sampling errors, can be useful. The efficiency of this approach involves the degree of correlation between dependent variables considered in the model. In the paper we take up the subject of the estimation of incomes and expenditure in Poland by means of the multivariate Rao-Yu model based on the sample data coming from the Polish Household Budget Survey and administrative registers. In particular, the advantages and limitations of bivariate models have been discussed. The calculations were performed using the sae and sae2 packages for R-project environment. Direct estimates were performed using the WesVAR software, and the precision of the direct estimates was determined using a balanced repeated replication (BRR) method.

Association of Added Sugars Intake with Physiologic Parameters in Adults: NHANES 2001–2012

The association of added sugars intake with liver enzymes, cardiovascular risk factors, and markers of red cell counts was determined using NHANES 2001–2012 data from adults (19+ years; n=26,402). Dietary intake was determined using 24‐hour dietary recall interviews using an Automated Multiple‐Pass Method. The usual intake of added sugars as a percent of energy was estimated using the Markov Chain Monte Carlo (MCMC) ratio method as implemented in v1.0 of the SAS programs published by the National Cancer Institute (NCI). Balanced repeated replication (BRR) was used for variance estimation. Subjects were separated into six groups: 0&lt;5, 5 to &lt;10, 10 to&lt;15, 15 to &lt;20, 20 to &lt;25, and ≥25% of energy as added sugars. Group trends and linear trends for were determined using linear regression (p&lt;0.01) for: alkaline phosphatase, alanine aminotransferase, aspartate aminotransferase, gamma‐glutamyl transferase, lactate dehydrogenase, blood pressure, high‐ and low‐density lipoprotein cholesterol, triglycerides, glucose, c‐reactive protein, waist circumference, ferritin, serum and red blood cell folate, hemoglobin, and hematocrit. None of the liver enzyme levels showed any significant association with added sugars intake. Mean plasma glucose levels (mg/dL) showed an inverse group (beta = −0.71; p=0.0003) and linear trend (beta = −0.13); p=0.0002) association with increasing percent energy of added sugars. No other cardiovascular risk factors, including metabolic syndrome, showed any association with added sugars intake. Only hemoglobin (g/dL) (beta = −0.03, p=0.0007; beta = 0.005, p=0.0013) and hematocrit (%) (beta = −0.09, p=0.0001; beta = −0.02, p=0.0002) showed a group or linear trend, respectively. No other statistical associations were found. These results suggest that the association of usual added sugars intake, as determined by sophisticated models, up to &gt;25% of energy, had limited effect on physiologic parameters in adults.Support or Funding InformationUSDA Hatch Project LAB 94209 and USDA/ARS through specific cooperative agreement 58‐6250‐6‐003 was received. Partial support was also received from the Corn Refiners Association.

Personality, Motivation, and Math Achievement Among Turkish Students

Using the Turkish portion of the Programme for International Student Assessment dataset ( N = 4,848; 51% boys, 49% girls; age, M = 15.81 years, SD = 0.28), this study investigated factors associated with mathematics achievement among Turkish students. Three different models were estimated using the method of balanced repeated replication with Fay's method and taking into account the presence of five plausible values of the dependent variable. Results showed that male students and older students had better mathematics proficiency. Socio-economic status and school resources also played a significant role in explaining student achievement in mathematics. Finally, students who were more open to problem solving, who attributed their failure to external factors, and who were intrinsically motivated to learn mathematics achieved higher scores. Policy implications are provided.

Estimation of Mean Income for Small Areas in Poland Using Rao-Yu Model

Modelling and estimating relationships that combine time series and crosssectional data is often discussed in the statistical literature but in these considerations sampling errors are seldom taken into account. In the paper the application of the Rao-Yu model involving both- autocorrelated random effects between areas and sampling errors-has been presented. On the basis of this model the empirical best linear unbiased predictor (EBLUP) with time correlation has been obtained. As an example the application of several income-related variables for the Polish voivodships (regions) and the years 2003–2011 was used on the basis of the Polish Household Budget Survey and selected explanatory variables obtained from Polish Local Data Bank. The computations were performed using sae2 and sae packages for R-project environment and WesVAR software. The precision of the direct estimates was obtained using Balanced Repeated Replication (BRR) technique.For most investigated cases, the proposed methods based on the Rao-Yu model yielded the significant improvement of small area estimates due to substantial reduction of their relative estimation errors as compared to the ordinary EBLUP technique. For some income variables examined within the study very high values of time-related autocorrelation coefficient were observed. These values were in some cases higher than 0.9, what can be – in our opinion – a good illustration of income growth tendency observed in Poland in the period under consideration.

Successive Difference Replication Variance Estimation in Two-Phase Sampling

Successive difference replication (SDR) is a method of variance estimation under systematic sampling. The method has been used at the U.S. Census Bureau for important studies, including the Current Population Survey and the American Community Survey. The statistical properties of SDR are evaluated in the context of other variance estimators applicable to single-phase systematic sampling. A simulation study is conducted to compare the behavior of SDR and balanced repeated replication (BRR) under various choices of their respective tuning constants. Further, SDR has been used to estimate variances in a two-phase setting, such as the National Survey of College Graduates (NSCG), a subsample of the American Community Survey (ACS). We study properties of a two-phase SDR variance estimator using a phase-one sample of ACS data and repeated phase-two samples from the ACS following the NSCG design. The estimator accounts for systematic sampling in phase one and stratified sampling in phase two. The estimator is conservative for some NSCG variables because the NSCG uses not only explicit stratification on some ACS variables but also systematic sampling within strata after sorting on additional ACS variables. The simulation results suggest that the two-phase SDR variance estimator, with an appropriate adjustment to account for the systematic sampling in phase two, performs well in this context of subsampling the NSCG from the ACS sample.

Complex Samples and Regression‐Based Inference: Considerations for Consumer Researchers

This article demonstrates that researchers who treat data collected via complex sampling procedures as if they were collected via simple random sample (SRS) may draw improper inferences when estimating regression models. Using complex sample data from the 2004 panel of the Survey of Income and Program Participation (SIPP) two models—one ordinary least squares (OLS) regression and one logistic regression—were estimated using three methods: SRS with and without population weights, Taylor series linearization, and Fay's Balanced Repeated Replication (BRR). The results of the alternative models demonstrate that depending on the variables of interest, authors who fail to incorporate sample design information or fail to consider the effects of weighting may draw improper inferences from their regression models. Reasons why researchers continue to neglect complex sample‐based variance are proposed and discussed, and example SAS and Stata code is offered to encourage adoption by the consumer research community.

Assessing usual dietary intake in complex sample design surveys: the National Dietary Survey

The National Cancer Institute (NCI) method allows the distributions of usual intake of nutrients and foods to be estimated. This method can be used in complex surveys. However, the user must perform additional calculations, such as balanced repeated replication (BRR), in order to obtain standard errors and confidence intervals for the percentiles and mean from the distribution of usual intake. The objective is to highlight adaptations of the NCI method using data from the National Dietary Survey. The application of the NCI method was exemplified analyzing the total energy (kcal) and fruit (g) intake, comparing estimations of mean and standard deviation that were based on the complex design of the Brazilian survey with those assuming simple random sample. Although means point estimates were similar, estimates of standard error using the complex design increased by up to 60% compared to simple random sample. Thus, for valid estimates of food and energy intake for the population, all of the sampling characteristics of the surveys should be taken into account because when these characteristics are neglected, statistical analysis may produce underestimated standard errors that would compromise the results and the conclusions of the survey.

Assessing usual dietary intake in complex sample design surveys: the National Dietary Survey

The National Cancer Institute (NCI) method allows the distributions of usual intake of nutrients and foods to be estimated. This method can be used in complex surveys. However, the user must perform additional calculations, such as balanced repeated replication (BRR), in order to obtain standard errors and confidence intervals for the percentiles and mean from the distribution of usual intake. The objective is to highlight adaptations of the NCI method using data from the National Dietary Survey. The application of the NCI method was exemplified analyzing the total energy (kcal) and fruit (g) intake, comparing estimations of mean and standard deviation that were based on the complex design of the Brazilian survey with those assuming simple random sample. Although means point estimates were similar, estimates of standard error using the complex design increased by up to 60% compared to simple random sample. Thus, for valid estimates of food and energy intake for the population, all of the sampling characteristics of the surveys should be taken into account because when these characteristics are neglected, statistical analysis may produce underestimated standard errors that would compromise the results and the conclusions of the survey.

Web Appendices to 'Progressivity and Potential Income: Measuring the Effect of Changing Work Patterns on Income Tax Progressivity'

This document contains four appendices to Sanchirico, Chris William, Progressivity and Potential Income: Measuring the Effect of Changing Work Patterns on Income Tax Progressivity. Columbia Law Review, Vol. 108, Pg. 1551, 2008. Available at SSRN: http://ssrn.com/abstract=961310These are: 1. The Stata do file creating variables from PSID data; 2. The Stata program that runs the estimation procedure and was submitted to balanced repeated replication; 3. And appendix explaining actual income tax figures and graphs in terms of tax code provisions; 4. The more technically detailed working paper version of the full paper.