Random Assumption Research Articles

Positive Unlabeled Learning Selected Not At Random (PULSNAR): class proportion estimation without the selected completely at random assumption

Positive and unlabeled (PU) learning is a type of semi-supervised binary classification where the machine learning algorithm differentiates between a set of positive instances (labeled) and a set of both positive and negative instances (unlabeled). PU learning has broad applications in settings where confirmed negatives are unavailable or difficult to obtain, and there is value in discovering positives among the unlabeled (e.g., viable drugs among untested compounds). Most PU learning algorithms make the selected completely at random (SCAR) assumption, namely that positives are selected independently of their features. However, in many real-world applications, such as healthcare, positives are not SCAR (e.g., severe cases are more likely to be diagnosed), leading to a poor estimate of the proportion, α, of positives among unlabeled examples and poor model calibration, resulting in an uncertain decision threshold for selecting positives. PU learning algorithms vary; some estimate only the proportion, α, of positives in the unlabeled set, while others calculate the probability that each specific unlabeled instance is positive, and some can do both. We propose two PU learning algorithms to estimate α, calculate calibrated probabilities for PU instances, and improve classification metrics: i) PULSCAR (positive unlabeled learning selected completely at random), and ii) PULSNAR (positive unlabeled learning selected not at random). PULSNAR employs a divide-and-conquer approach to cluster SNAR positives into subtypes and estimates α for each subtype by applying PULSCAR to positives from each cluster and all unlabeled. In our experiments, PULSNAR outperformed state-of-the-art approaches on both synthetic and real-world benchmark datasets.

EFFICIENCY IN ESTIMATION UNDER MONOTONIC ATTRITION

Attrition is monotonic when agents leaving multi-period studies do not return. Under a general missing at random (MAR) assumption, we study efficiency in estimation of parameters defined by moment restrictions on the distributions of the counterfactuals that were unrealized due to monotonic attrition. We discuss novel issues related to overidentification, usability of sample units, and the information content of various MAR assumptions for estimation of such parameters. We propose a standard doubly robust estimator for these parameters by equating to zero the sample analog of their respective efficient influence functions. Our proposed estimator performs well and vastly outperforms other estimators in our simulation experiment and empirical illustration.

Design and implementation of a Type-2 hybrid, prospective randomized trial of opioid agonist therapies integration into primary care clinics in Ukraine

IntroductionUkraine has high HIV prevalence, concentrated among people who inject drugs (PWID), mostly of opioids. Maintenance on opioid agonist therapies (OAT) is the most effective evidence-based treatment for opioid use disorder. As PWID experience high morbidity and mortality from preventable and treatable non-communicable diseases, international agencies recommend integrating OAT into primary care centers (PCC). MethodsA randomized, type-2 hybrid implementation trial was carried out to compare outcomes of OAT integration in PCC to OAT delivery at specialty treatment centers (STC) – standard-of-care. Tele-education supporting PCC providers in managing OAT, HIV, tuberculosis and non-communicable diseases along with pay-for-performance incentives were used to facilitate implementation. Consenting patients underwent 1:2 randomization to either STC or PCC. Quality health indicators (QHIs), a composite percentage of recommended primary and specialty services accessed by patients (blood/urine tests, cancer screenings, etc.), were defined as efficacy outcomes and were assessed by participant self-report at baseline and every 6 months over 24 months and electronic chart reviews after the completion of the follow-up. The primary outcome is defined as the difference in composite QHI scores at 24 months, in which a repeated measures likelihood-based mixed model with missing at random assumptions will be used. Providers at PCC completed surveys at baseline, 12 and 24 months to assess implementation outcomes including changes in stigma and attitudes towards OAT and PWID. Preliminary resultsAmong the 1459 participants allocated to STC (N = 509) or PCC (N = 950), there were no differences in clinical and demographic characteristics. Self-reported prevalences were available for HIV (42 %), HCV (57 %), and prior tuberculosis (17 %). Study retention at 6, 12, 18, and 24 months was as 91 %, 85 %, 80 %, and 74 %, respectively. ConclusionPWID have a high prevalence of medical comorbidities and integrating OAT into primary care settings has the potential to improve the health of PWID. Findings from this study can help guide implementation of integrated care in Ukraine and throughout similar low-resource, high-burden countries in the Eastern European and Central Asian region.

Software application profile: tpc and micd-R packages for causal discovery with incomplete cohort data.

The Peter Clark (PC) algorithm is a popular causal discovery method to learn causal graphs in a data-driven way. Until recently, existing PC algorithm implementations in R had important limitations regarding missing values, temporal structure or mixed measurement scales (categorical/continuous), which are all common features of cohort data. The new R packages presented here, micd and tpc, fill these gaps. micd and tpc packages are R packages. The micd package provides add-on functionality for dealing with missing values to the existing pcalg R package, including methods for multiple imputations relying on the Missing At Random assumption. Also, micd allows for mixed measurement scales assuming conditional Gaussianity. The tpc package efficiently exploits temporal information in a way that results in a more informative output that is less prone to statistical errors. The tpc and micd packages are freely available on the Comprehensive R Archive Network (CRAN). Their source code is also available on GitHub (https://github.com/bips-hb/micd; https://github.com/bips-hb/tpc).

Factor-Analytic Variance-Covariance Structures for Prediction Into a Target Population of Environments.

Finlay-Wilkinson regression is a popular method for modeling genotype-environment interaction in plant breeding and crop variety testing. When environment is a random factor, this model may be cast as a factor-analytic variance-covariance structure, implying a regression on random latent environmental variables. This paper reviews such models with a focus on their use in the analysis of multi-environment trials for the purpose of making predictions in a target population of environments. We investigate the implication of random versus fixed effects assumptions, starting from basic analysis-of-variance models, then moving on to factor-analytic models and considering the transition to models involving observable environmental covariates, which promise to provide more accurate and targeted predictions than models with latent environmental variables.

Solving the missing at random problem in semi‐supervised learning: An inverse probability weighting method

We propose an estimator for the population mean under the semi‐supervised learning setting with the Missing at Random (MAR) assumption. This setting assumes that the probability of observing , denoted by , depends on the total sample size and satisfies . To efficiently estimate , we introduce an adaptive estimator based on inverse probability weighting and cross‐fitting. Theoretical analysis reveals that our proposed estimator is consistent and efficient, with a convergence rate of , slower than the typical rate, due to the diminishing proportion of labelled data as the sample size increases in the semi‐supervised setting. We also prove the consistency of inverse probability weighting (IPW)–Nadaraya–Watson density function estimators. Extensive simulations and an application to the Los Angeles homeless data validate the effectiveness of our approach.

Addressing missing outcome data in randomised controlled trials: A methodological scoping review

BackgroundMissing outcome data is common in trials, and robust methods to address this are needed. Most trial reports currently use methods applicable under a missing completely at random assumption (MCAR), although this strong assumption can often be inappropriate. ObjectiveTo identify and summarise current literature on the analytical methods for handling missing outcome data in randomised controlled trials (RCTs), emphasising methods appropriate for data missing at random (MAR) or missing not at random (MNAR). Study design and settingWe conducted a methodological scoping review and identified papers through searching four databases (MEDLINE, Embase, CENTRAL, and CINAHL) from January 2015 to March 2023. We also performed forward and backward citation searching. Eligible papers discussed methods or frameworks for handling missing outcome data in RCTs or simulation studies with an RCT design. ResultsFrom 1878 records screened, our search identified 101 eligible papers. 90 (89%) papers described specific methods for addressing missing outcome data and 11 (11%) described frameworks for overall methodological approach. Of the 90 methods papers, 30 (33%) described methods under the MAR assumption, 48 (53%) explored methods under the MNAR assumption and 11 (12%) discussed methods under a hybrid of MAR and MNAR assumptions. Control-based methods under the MNAR assumption were the most common method explored, followed by multiple imputation under the MAR assumption. ConclusionThis review provides guidance on available analytic approaches for handling missing outcome data, particularly under the MNAR assumption. These findings may support trialists in using appropriate methods to address missing outcome data.

A kinematic real-time PPP approach with estimating signal in space range errors

Since 2013, the International GNSS Service (IGS) real-time service (RTS) has been providing precise orbit and clock corrections for ten years, which enables real-time positioning at the decimeter level. High precision positioning relies heavily on accurate orbit and clock. However, constrained by the relatively short observation duration, the accuracy of real-time products are often worse than the final products. Previous studies indicate that compensating for errors in orbit and clock can improve positioning accuracy. Based on this theory, we proposed an improved positioning model, which incorporates the signal-in-space range error (SISRE) into the ionosphere-free combination observation equation and treats it as a parameter in the Kalman filter. Through the analysis of SISRE, we find that the time-varying characteristic of the parameter follows a random walk assumption. To find the optimal parameter settings of the SISRE parameters, a two-dimensional sensitivity analysis is employed, and a set of recommended values are provided. In the simulated kinematic positioning experiments, the proposed method achieves positioning accuracies of 17.3 cm, 19.6 cm, and 18.0 cm for GPS, BDS-3, and Galileo, respectively, representing a 10–20 % improvement over traditional methods. In real-time PPP experiment, the accuracy reaches 21.7 cm, indicating a 12.9 % improvement compared to traditional methods.

An Ad Hoc Procedure for Testing Serial Correlation in Spatial Fixed-Effects Panels

We consider testing for error persistence in spatial panels with (potentially correlated) individual heterogeneity. We propose two variants of an ad hoc testing procedure based on first transforming out the individual effects, either by time-demeaning or by taking forward orthogonal deviations, then estimating an encompassing spatio-temporal model. The procedure can also be employed under the random effects assumption, in which case, although suboptimal, it can be computationally cheaper and safer than existing tests. We report Monte Carlo simulations demonstrating satisfactory empirical properties.

Circulating Metabolite Abundances Associated With Risks of Bipolar Disorder, Schizophrenia, and Depression: A Mendelian Randomization Study

BackgroundPreventive measures and treatments for psychiatric disorders are limited. Circulating metabolites are potential candidates for biomarker and therapeutic target identification, given their measurability and essential roles in biological processes. MethodsLeveraging large-scale genome-wide association studies, we conducted Mendelian randomization analyses to assess the associations between circulating metabolite abundances and the risks of bipolar disorder, schizophrenia, and depression. Genetic instruments were selected for 94 metabolites measured in the Canadian Longitudinal Study on Aging cohort (N = 8299). We repeated Mendelian randomization analyses based on the UK Biobank, INTERVAL, and EPIC (European Prospective Investigation into Cancer)–Norfolk studies. ResultsAfter validating Mendelian randomization assumptions and colocalization evidence, we found that a 1 SD increase in genetically predicted circulating abundances of eicosapentaenoate and docosapentaenoate was associated with odds ratios of 0.72 (95% CI, 0.65–0.79) and 0.63 (95% CI, 0.55–0.72), respectively, for bipolar disorder. Genetically increased Ω-3 unsaturated fatty acids abundance and Ω-3-to-total fatty acids ratio, as well as genetically decreased Ω-6-to-Ω-3 ratio, were negatively associated with the risk of bipolar disorder in the UK Biobank. Genetically increased circulating abundances of 3 N-acetyl-amino acids were associated with an increased risk of schizophrenia with a maximum odds ratio of 1.31 (95% CI, 1.18–1.44) per 1 SD increase. Furthermore, a 1 SD increase in genetically predicted circulating abundance of hypotaurine was associated with an odds ratio of 0.85 (95% CI, 0.78–0.93) for depression. ConclusionsThe biological mechanisms that underlie Ω-3 unsaturated fatty acids, NAT8-catalyzed N-acetyl-amino acids, and hypotaurine warrant exploration to identify new biomarkers and potential therapeutic targets.

Efficient estimation of crystal filterability using the discrete element method and the Kozeny-Carman equation

A new method is proposed for efficiently predicting filter cake resistance as a function of crystal size distribution. The method is useful for the preliminary design of processes where an economic tradeoff is encountered between a crystallization step and a filtration step. In that case it is necessary to estimate the filter cake resistance repeatedly as a function of crystal size distribution for the process design and optimization. The method is faster than CFD-DEM and more precise than the random packing assumption. The proposed method is validated using published experimental data. To demonstrate the proposed method, an integrated crystallization-filtration process of ammonium alum is simulated and analyzed. The results show that by increasing the residence time of ammonium alum crystals in the crystallizer from 15 min to 60 min, the crystal cake resistance can be reduced by 42% and the filter area can be reduced by 23%.

Comparing the Robustness of Simple Network Scale-Up Method Estimators.

The network scale-up method (NSUM) is a cost-effective approach to estimating the size or prevalence of a group of people that is hard to reach through a standard survey. The basic NSUM involves two steps: estimating respondents’ degrees and estimating the prevalence of the hard-to-reach population of interest using respondents’ estimated degrees and the number of people they report knowing in the hard-to-reach group. Each of these two steps involves taking either an average of ratios or a ratio of averages. Using the ratio of averages for each step has so far been the most common approach. However, the authors present theoretical arguments that using the average of ratios at the second, prevalence-estimation step often has lower mean squared error when the random mixing assumption is violated, which seems likely in practice; this estimator was proposed early in NSUM development but has largely been unexplored and unused. Simulation results using an example network data set also support these findings. On the basis of this theoretical and empirical evidence, the authors suggest that future surveys that use a simple estimator may want to use this mixed estimator, and estimation methods based on this estimator may produce new improvements.

A data-driven approach to understanding non-response and restoring sample representativeness in the UK Next Steps cohort

Non-response is common in longitudinal surveys, reducing efficiency and introducing the potential for bias. Principled methods, such as multiple imputation, are generally required to obtain unbiased estimates in surveys subject to missingness which is not completely at random. The inclusion of predictors of non-response in such methods, for example as auxiliary variables in multiple imputation, can help improve the plausibility of the missing at random assumption underlying these methods and hence reduce bias. We present a systematic data-driven approach used to identify predictors of non-response at Wave 8 (age 25–26) of Next Steps, a UK national cohort study that follows a sample of 15,770 young people from age 13–14 years. The identified predictors of non-response were across a number of broad categories, including personal characteristics, schooling and behaviour in school, activities and behaviour outside of school, mental health and well-being, socio-economic status, and practicalities around contact and survey completion. We found that including these predictors of non-response as auxiliary variables in multiple imputation analyses allowed us to restore sample representativeness in several different settings, though we acknowledge that this is unlikely to universally be the case. We propose that these variables are considered for inclusion in future analyses using principled methods to explore and attempt to reduce bias due to non-response in Next Steps. Our data-driven approach to this issue could also be used as a model for investigations in other longitudinal studies.

Localization of overdamped bosonic modes and transport in strange metals

The strange metal phase of correlated electrons materials was described in a recent theory by a model of a Fermi surface coupled a two-dimensional quantum critical bosonic field with a spatially random Yukawa coupling. With the assumption of self-averaging randomness, similar to that in the Sachdev-Ye-Kitaev model, numerous observed properties of a strange metal were obtained for a wide range of intermediate temperatures, including the linear in temperature resistivity. The Harris criterion implies that spatial fluctuations in the local position of the critical point must dominate at lower temperatures. For an [Formula: see text]-component boson with [Formula: see text], we use multiple graphics processing units (GPUs) to compute the real frequency spectrum of the boson propagator in a self-consistent mean-field treatment of the boson self-interactions, but an exact treatment of multiple realizations of the spatial randomness from the random boson mass. We find that Landau damping from the fermions leads to the emergence of the physics of the random transverse-field Ising model at low temperatures, as has been proposed by Hoyos, Kotabage, and Vojta. This regime is controlled by localized overdamped eigenmodes of the bosonic scalar field, also has a resistivity which is nearly linear-in-temperature, and extends into a "quantum critical phase" away from the quantum critical point, as observed in several cuprates. For the [Formula: see text] Ising scalar, the mean-field treatment is not applicable, and so we use Hybrid Monte Carlo simulations running on multiple GPUs; we find a rounded transition and localization physics, with strange metal behavior in an extended region around the transition.

Covariate adjustment in randomized experiments with missing outcomes and covariates

Summary Covariate adjustment can improve precision in analysing randomized experiments. With fully observed data, regression adjustment and propensity score weighting are asymptotically equivalent in improving efficiency over unadjusted analysis. When some outcomes are missing, we consider combining these two adjustment methods with inverse probability of observation weighting for handling missing outcomes, and show that the equivalence between the two methods breaks down. Regression adjustment no longer ensures efficiency gain over unadjusted analysis unless the true outcome model is linear in covariates or the outcomes are missing completely at random. Propensity score weighting, in contrast, still guarantees efficiency over unadjusted analysis, and including more covariates in adjustment never harms asymptotic efficiency. Moreover, we establish the value of using partially observed covariates to secure additional efficiency by the missingness indicator method, which imputes all missing covariates by zero and uses the union of the completed covariates and corresponding missingness indicators as the new, fully observed covariates. Based on these findings, we recommend using regression adjustment in combination with the missingness indicator method if the linear outcome model or missing complete at random assumption is plausible and using propensity score weighting with the missingness indicator method otherwise.

Forecasting weekly dengue incidence in Sri Lanka: Modified Autoregressive Integrated Moving Average modeling approach.

Dengue poses a significant and multifaceted public health challenge in Sri Lanka, encompassing both preventive and curative aspects. Accurate dengue incidence forecasting is pivotal for effective surveillance and disease control. To address this, we developed an Autoregressive Integrated Moving Average (ARIMA) model tailored for predicting weekly dengue cases in the Colombo district. The modeling process drew on comprehensive weekly dengue fever data from the Weekly Epidemiological Reports (WER), spanning January 2015 to August 2020. Following rigorous model selection, the ARIMA (2,1,0) model, augmented with an autoregressive component (AR) of order 16, emerged as the best-fitted model. It underwent initial calibration and fine-tuning using data from January 2015 to August 2020, and was validated against independent 2000 data. Selection criteria included parameter significance, the Akaike Information Criterion (AIC), and Schwarz Bayesian Information Criterion (SBIC). Importantly, the residuals of the ARIMA model conformed to the assumptions of randomness, constant variance, and normality affirming its suitability. The forecasts closely matched observed dengue incidence, offering a valuable tool for public health decision-makers. However, an increased percentage error was noted in late 2020, likely attributed to factors including potential underreporting due to COVID-19-related disruptions amid rising dengue cases. This research contributes to the critical task of managing dengue outbreaks and underscores the dynamic challenges posed by external influences on disease surveillance.

Doubly robust calibration of prediction sets under covariate shift.

Conformal prediction has received tremendous attention in recent years and has offered new solutions to problems in missing data and causal inference; yet these advances have not leveraged modern semi-parametric efficiency theory for more efficient uncertainty quantification. We consider the problem of obtaining well-calibrated prediction regions that can data adaptively account for a shift in the distribution of covariates between training and test data. Under a covariate shift assumption analogous to the standard missing at random assumption, we propose a general framework based on efficient influence functions to construct well-calibrated prediction regions for the unobserved outcome in the test sample without compromising coverage.

Nonrandom foraging and resource distributions affect the relationships between host density, contact rates and parasite transmission.

Nonrandom foraging can cause animals to aggregate in resource dense areas, increasing host density, contact rates and pathogen transmission, but when should nonrandom foraging and resource distributions also have density-independent effects? Here, we used a factorial experiment with constant resource and host densities to quantify host contact rates across seven resource distributions. We also used an agent-based model to compare pathogen transmission when host movement was based on random foraging, optimal foraging or something between those states. Nonrandom foraging strongly depressed contact rates and transmission relative to the classic random movement assumptions used in most epidemiological models. Given nonrandom foraging in the agent-based model and experiment, contact rates and transmission increased with resource aggregation and average distance to resource patches due to increased host movement in search of resources. Overall, we describe three density-independent mechanisms by which host behaviour and resource distributions alter contact rate functions and pathogen transmission.

Copula-based pairwise estimator for quantile regression with hierarchical missing data

Quantile regression can be a helpful technique for analysing clustered (such as longitudinal) data. It can characterize the change in response over time without making distributional assumptions and is robust to outliers in the response. A quantile regression model using a copula-based multivariate asymmetric Laplace distribution for addressing correlation due to clustering is introduced. Furthermore, we propose a pairwise estimator for the parameters of the model. Since it is based on pseudo-likelihood, it needs to be modified to avoid bias in presence of missingness. Therefore, we enhance the model with inverse probability weighting. In this way, our proposal is unbiased under the missing at random assumption. Based on simulations, the estimator is efficient and computationally fast. Finally, the methodology is illustrated using a study in ophthalmology.

Comparison between inverse-probability weighting and multiple imputation in Cox model with missing failure subtype.

Identifying and distinguishing risk factors for heterogeneous disease subtypes has been of great interest. However, missingness in disease subtypes is a common problem in those data analyses. Several methods have been proposed to deal with the missing data, including complete-case analysis, inverse-probability weighting, and multiple imputation. Although extant literature has compared these methods in missing problems, none has focused on the competing risk setting. In this paper, we discuss the assumptions required when complete-case analysis, inverse-probability weighting, and multiple imputation are used to deal with the missing failure subtype problem, focusing on how to implement these methods under various realistic scenarios in competing risk settings. Besides, we compare these three methods regarding their biases, efficiency, and robustness to model misspecifications using simulation studies. Our results show that complete-case analysis can be seriously biased when the missing completely at random assumption does not hold. Inverse-probability weighting and multiple imputation estimators are valid when we correctly specify the corresponding models for missingness and for imputation, and multiple imputation typically shows higher efficiency than inverse-probability weighting. However, in real-world studies, building imputation models for the missing subtypes can be more challenging than building missingness models. In that case, inverse-probability weighting could be preferred for its easy usage. We also propose two automated model selection procedures and demonstrate their usage in a study of the association between smoking and colorectal cancer subtypes in the Nurses' Health Study and Health Professional Follow-Up Study.