Bias-variance Research Articles

Bias and Variance of Post-processing in Differential Privacy

Post-processing immunity is a fundamental property of differential privacy: it enables the application of arbitrary data-independent transformations to the results of differentially private outputs without affecting their privacy guarantees. When query outputs must satisfy domain constraints, post-processing can be used to project them back onto the feasibility region. Moreover, when the feasible region is convex, a widely adopted class of post-processing steps is also guaranteed to improve accuracy. Post-processing has been applied successfully in many applications including census data, energy systems, and mobility. However, its effects on the noise distribution is poorly understood: It is often argued that post-processing may introduce bias and increase variance. This paper takes a first step towards understanding the properties of post-processing. It considers the release of census data and examines, both empirically and theoretically, the behavior of a widely adopted class of post-processing functions.

Self-Sacrifice Is Not the Only Way to Practice Filial Piety for Chinese Adolescents in Conflict With Their Parents

We applied the theoretical perspective of the dual filial piety model to consider the diversity of parent–child conflict resolution strategies in order to determine whether Chinese adolescents use strategies other than self-sacrifice to practice filial piety when in conflict with their parents. Study 1 utilized a cross-sectional design with 247 valid responses. The structural equation modeling analysis indicated that Taiwanese adolescents’ authoritarian filial piety (AFP) beliefs are positively related to use of a self-sacrifice strategy, and reciprocal filial piety (RFP) beliefs are positively related to use of compatibility and compromise strategies. Adolescents’ AFP and RFP beliefs are negatively related to use of utility and escape strategies. Study 2 applied a temporal separation procedure with a 1-year lag to remedy common method variance bias. Analysis of 1,063 valid responses replicated the findings of Study 1 and indicated that adolescents’ function-oriented appraisal of conflict can play a mediating role between RFP and the use of the compatibility and compromise strategies. These findings broaden the understanding of filial piety in modern Chinese societies and have implications for adolescents’ well-being and family life.

Propensity score weighting for causal subgroup analysis.

A common goal in comparative effectiveness research is to estimate treatment effects on prespecified subpopulations of patients. Though widely used in medical research, causal inference methods for such subgroup analysis (SGA) remain underdeveloped, particularly in observational studies. In this article, we develop a suite of analytical methods and visualization tools for causal SGA. First, we introduce the estimand of subgroup weighted average treatment effect and provide the corresponding propensity score weighting estimator. We show that balancing covariates within a subgroup bounds the bias of the estimator of subgroup causal effects. Second, we propose to use the overlap weighting (OW) method to achieve exact balance within subgroups. We further propose a method that combines OW and LASSO, to balance the bias‐variance tradeoff in SGA. Finally, we design a new diagnostic graph—the Connect‐S plot—for visualizing the subgroup covariate balance. Extensive simulation studies are presented to compare the proposed method with several existing methods. We apply the proposed methods to the patient‐centered results for uterine fibroids (COMPARE‐UF) registry data to evaluate alternative management options for uterine fibroids for relief of symptoms and quality of life.

Dynamic PET reconstruction using the kernel method with non-local means denoising

Non-local means with a spatiotemporal search window (NLM-ST) has been developed to denoise dynamic positron emission tomography (PET) images. The improved image quality, however, may not be good enough to generate reliable parametric images. In this work, we propose an iterative reconstruction algorithm which aims to improve the quality of dynamic PET images by incorporating NLM-ST denoising directly within the kernelized expectation-maximization (KEM) reconstruction algorithm. Since the NLM-ST denoising was employed after each KEM update, the proposed algorithm was called NLM-ST-AU-KEM. Computer simulations were conducted to evaluate the performance of the proposed reconstruction algorithm, and the results were compared to KEM with a post-reconstruction NLM-ST denoising filter (KEM + NLM-ST). The root mean squared errors (RMSE) of the dynamic PET images reconstructed using the KEM algorithm were increased after 40 iterations. Both the NLM-ST-AU-KEM and the KEM + NLM-ST methods could achieve stable RMSE values after 50 iterations, but the former had lower RMSE values. Compared to the proposed NLM-ST-AU-KEM method, the KEM + NLM-ST method tended to over-smooth dynamic PET images and parametric images. For K1 and Ki, the proposed NLM-ST-AU-KEM method had lower bias but higher variance than the KEM + NLM-ST method. For k2 and k3, the proposed NLM-ST-AU-KEM method had higher variance than the KEM + NLM-ST method, but the higher variance could be reduced by applying a kernel-based post-filtering method to the NLM-ST-AU-KEM-generated parametric images. NLM-ST denoising during image reconstruction seems to be a better strategy than that after image reconstruction.

Differential impacts of hemolysis on coagulation parameters of blood samples: A STROBE-compliant article.

This study aimed at investigating the impact of hemolysis on different coagulation parameters.A total of 216 venous blood samples without visible hemolysis were collected from adult patients at a tertiary referral center over six months. The plasma obtained was quantified for six coagulation parameters including prothrombin time, activated partial thromboplastin time, fibrinogen, D-dimer, antithrombin III, and protein C. The rest of the plasma from each blood sample was aliquoted into three tubes, each containing 1 mL of plasma with three different volumes of cell-free hemoglobin (i.e., 2, 4, 8 μL) from lysed RBCs to create simulated hemolyzed blood samples with hemoglobin concentration of approximately 0.1, 0.2, and 0.4 g/dL to mimic mild (1+), moderate (2+), and severe (3+) hemolysis, respectively, before repeating the coagulation tests to determine possible correlation between the simulated degree of hemolysis and the changes in test results of the coagulation parameters.Spearman correlation analysis showed significant decreases in the values of activated partial thromboplastin time, fibrinogen, D-dimer, and protein C values with an increasing degree of simulated hemolysis (all P < .01). Comparison of the percentage bias of biological variance showed significant positive associations of cell-free hemoglobin concentrations with the percentage bias of D-dimer and protein C. However, only the former was still within the range of biological variance under condition of simulated hemolysis. Besides, the presence of cell-free hemoglobin regardless of concentration had a notable impact on the percentage bias of activated partial thromboplastin time, whereas the influence was non-significant for prothrombin time, fibrinogen, and antithrombin III.The results showed different impacts of simulated hemolysis on six coagulation parameters, highlighting the dependence of clinical reliability on the coagulation parameter to be investigated in hemolytic blood samples.

Matched filtering with non-Gaussian noise for planet transit detections

ABSTRACT We develop a method for planet detection in transit data, which is based on the matched filter technique, combined with the Gaussianization of the noise outliers. The method is based on Fourier transforms and is as fast as the existing methods for planet searches. The Gaussianized matched filter (GMF) method significantly outperforms the standard baseline methods in terms of the false positive rate, enabling planet detections at up to 30 per cent lower transit amplitudes. Moreover, the method extracts all the main planet transit parameters, amplitude, period, phase, and duration. By comparison to the state-of-the-art Gaussian process methods on both simulations and real data, we show that all the transit parameters are determined with an optimal accuracy (no bias and minimum variance), meaning that the GMF method can be used for both the initial planet detection and the follow-up planet parameter analysis.

Reliable Model Selection without Reference Values by Utilizing Model Diversity with Prediction Similarity.

Predictive modeling (calibration or training) with various data formats, such as near-infrared (NIR) spectra and quantitative structure-activity relationship (QSAR) data, provides essential information if a proper model is selected. Similarly, with a general model selection approach, spectral model maintenance (updating) from original modeling conditions to new conditions can be performed for dynamic modeling. Fundamental modeling (partial least-squares (PLS) and others) and maintenance processes (domain adaptation or transfer learning and others) require selection of tuning parameter(s) values to isolate models that can accurately predict new samples or molecules, e.g., number of PLS latent variables to predict analyte concentration. Regardless of the modeling task, model selection is complex and without a reliable protocol. Tuning parameter selection typically depends on only one model quality measure assessing model bias using prediction accuracy. Developed in this paper is a generic model selection process using concepts from consensus modeling and QSAR activity landscapes. It is a consensus filtering approach that prioritizes model diversity (MD) while conserving prediction similarity (PS) fused with a common bias-variance trade-off measure. A significant feature of MDPS is that a cross-validation scheme is not needed because models are selected relative to predicting new samples or molecules, i.e., model selection uses unlabeled samples (without reference values) for active predictions. The versatility and reliability of MDPS model selection is shown using four NIR data sets and a QSAR data set. The study also substantiates the Rashomon effect where there is not one best model tuning parameter value that provides accurate predictions.

Analyzing Medical Data by Using Statistical Learning Models

In this work, we investigated the prognosis of three medical data specifically, breast cancer, heart disease, and prostate cancer by using 10 machine learning models. We applied all 10 models to each dataset to identify patterns in them. Furthermore, we use the models to diagnose risk factors that increases the chance of these diseases. All the statistical learning techniques discussed were grouped into linear and nonlinear models based on their similarities and learning styles. The models performances were significantly improved by selecting models while taking into account the bias-variance tradeoffs and using cross-validation for selecting the tuning parameter. Our results suggests that no particular class of models or learning style dominated the prognosis and diagnosis for all three medical datasets. However nonlinear models gave the best predictive performance for breast cancer data. Linear models on the other hand gave the best predictive performance for heart disease data and a combination of linear and nonlinear models for the prostate cancer dataset.

Bayesian inverse modeling of large-scale spatial fields on iteratively corrected principal components

Inverse modeling of large-scale spatially variable parameters fields at fine resolution can be reduced to estimating the projections on dominant principal components of the underlying parameter fields based on principal component analysis of the spatial covariance. For unknown or biased prior structural parameters of spatial covariance models, an iterative procedure consisting of two successive steps is usually implemented, i.e., estimation of spatial covariance followed by estimation of the spatially variable parameter fields conditional on the spatial covariance and observations. In this study, we develop an iterative, computationally efficient method to update dominant principal components for nonlinear inverse problems of large-scale spatial fields that adaptively corrects the bias from the initially defined prior spatial covariance. Our algorithm involves two-layer iterations: the inner iteration is to obtain the best estimates of projections on given retained principal components, and the outer iteration implements an efficient rank-one updating to correct the retained principal components using the posterior covariance associated with the best estimates of the projections. Numerical experiments show that inversion results can be significantly improved for large-scale inverse problems with biased structural parameters for spatial covariance. The experiment results show that the iterative correction is essentially to match the distribution patterns of the spatially correlated parameter field with its most dominant principal components. We also investigate the performance of the developed method under different biased covariance model initialization, including model type bias, variance bias and correlation length bias. The correction cannot fundamentally change the smoothness defined by the covariance model type, but can still describe major distribution patterns including anisotropy. Biased variance can be corrected and yields similar best estimates and variance maps, and biased correlation length can be corrected within an applicable range.

A Penalized Likelihood Estimator for Variance Components in Repeated Measurements Model

In this paper, we consider the repeated measurement model. We estimate the parameters of the model by using the maximum likelihood method and compared these estimators with the penalized likelihood estimators by using mean square error. We also determine the corresponding moments of each estimators. Finally, we find bias equations and variance for each estimator.

Analysis of Conditions for Reliable Predictions by Moodle Machine Learning Models

In this paper the issue of bias-variance trade-off in building and operating Moodle Machine Learning (ML) models are discussed to avoid traps of get-ting unreliable predictions. Moodle is one of the world’s most popular open source Learning Management System (LMS) with millions of users. Although since Moodle 3.4 release it is possible to create ML models within the LMS system very few studies have been published so far about the conditions of its proper application. Using these models as black boxes hold serious risks to get unreliable predictions and false alarms. From a comprehensive study of differently built machine learning models elaborated at the University of Dunaújváros in Hungary, one specific issue is addressed here, namely the in-fluence of the size and the row-column ratio of the predictor matrix on the goodness of the predictions. In the so-called Time Splitting Method in Moo-dle Learning Analytics the effect of varying numbers of time splits and of predictors has also been studied to see their influence on the bias and the variance of the models. An Applied Statistics course is used to demonstrate the consequences of the different model set up.

Estimating logit models with small samples

AbstractIn small samples, maximum likelihood (ML) estimates of logit model coefficients have substantial bias away from zero. As a solution, we remind political scientists of Firth's (1993,Biometrika,80, 27–38) penalized maximum likelihood (PML) estimator. Prior research has described and used PML, especially in the context of separation, but its small sample properties remain under-appreciated. The PML estimator eliminates most of the bias and, perhaps more importantly, greatly reduces the variance of the usual ML estimator. Thus, researchers do not face a bias-variance tradeoff when choosing between the ML and PML estimators—the PML estimator has a smaller biasanda smaller variance. We use Monte Carlo simulations and a re-analysis of George and Epstein (1992,American Political Science Review,86, 323–337) to show that the PML estimator offers a substantial improvement in small samples (e.g., 50 observations) and noticeable improvement even in larger samples (e.g., 1000 observations).

A Comparative Study on the Performance of Gene Expression Programming and Machine Learning Methods

Usually, evolutionary algorithms are used to provide strong approximations to problems that are difficult to solve with other methods. Gene expression programming (GEP) is a type of evolutionary algorithm used in computer programming to generate computer programs or models. These computer programs are complex tree structures that, like a living organism, learn and adapt by modifying their sizes, shapes, and composition. In the present work, a comparison study was made among GEP and the standard prediction techniques to find the best predicting model on the BOSTON HOUSING dataset. Three approaches viz. GEP, ANN and polynomial regression were implemented on the dataset. The study showed how the three methods solve the problem of high bias and high variance and which one outperforms the other. The research work, however, gave a glimpse of the actual limitations and advantages of the methods on one another indicating the dependency of method on the type of data used. The results conclude the comparison of different methods on different performance metrics. The GEP model however reduced the problem of high bias and high variance by giving a slight difference between the train and test accuracy but was not able to outperform ANN and polynomial regression in terms of performance metrics.

Varying coefficient frailty models with applications in single molecular experiments.

Motivated by an analysis of single molecular experiments in the study of T-cell signaling, a new model called varying coefficient frailty model with local linear estimation is proposed. Frailty models have been extensively studied, but extensions to nonconstant coefficients are limited to spline-based methods that tend to produce estimation bias near the boundary. To address this problem, we introduce a local polynomial kernel smoothing technique with a modified expectation-maximization algorithm to estimate the unknown parameters. Theoretical properties of the estimators, including their unbiased property near the boundary, are derived along with discussions on the asymptotic bias-variance trade-off. The finite sample performance is examined by simulation studies, and comparisons with existing spline-based approaches are conducted to show the potential advantages of the proposed approach. The proposed method is implemented for the analysis of T-cell signaling. The fitted varying coefficient model provides a rigorous quantification of an early and rapid impact on T-cell signaling from the accumulation of bond lifetime, which can shed new light on the fundamental understanding of how T cells initiate immuneresponses.

Evaluating Misclassification Effects on Single Sequential Treatment in Sequential Multiple Assignment Randomized Trial (SMART) Designs

Sequential multiple assignment randomized trial designs tailor individual treatment by rerandomizing participants to subsequent therapies based on their response to initial treatment. Misclassification of participant responses to initial treatment can lead to inappropriate treatment assignment and thus impact the final outcome. The aim of this study is to derive a series of formulas for quantifying potential misclassification effects on the mean, variance, and statistical inference of a single sequential treatment (SST) effect with continuous outcome. Relative bias is expressed as a function of sensitivity, specificity, and the probability of being true responders. Results show that misclassification can introduce bias to the estimated treatment effect. Though the magnitude of bias varies, there are a few general conclusions: (1) for any fixed sensitivity (or specificity) the relative bias of the mean of responders (or nonresponders) always approaches 0 in a monotonic nonlinear pattern as specificity (or sensitivity) increases; (2) the relative bias of SST variance always has nonmonotone nonlinear relationship with sensitivity or specificity; (3) the SST variance under misclassification is always over-estimated. Furthermore, the results show that misclassification can affect statistical inference, with power exhibiting either monotonic or nonmonotonic patterns and resulting in either under- or over-estimation.

An Enhanced Deep Learning Model to Network Attack Detection, by using Parameter Tuning, Hidden Markov Model and Neural Network

In recent years, Deep Learning has become a critical success factor for Machine Learning. In the present study, we introduced a Deep Learning model to network attack detection, by using Hidden Markov Model and Artificial Neural Networks. We used a model aggregation technique to find a single consolidated Deep Learning model for better data fitting. The model selection technique is applied to optimize the bias-variance trade-off of the expected prediction. We demonstrate its ability to reduce the convergence, reach the optimal solution and obtain more cluttered decision boundaries. Experimental studies conducted on attack detection indicate that our proposed model outperformed existing Deep Learning models and gives an enhanced generalization.

The Influence of Affective State on Subjective-Report Measurements: Evidence From Experimental Manipulations of Mood.

A substantial portion of the knowledge base of psychology is based on subjective reports with a risk of information bias. The objective of the present study was to elucidate one contextual source of variance and potential bias in subjective reports: the influence of affective state at the time of responding to questionnaires. Employees (N = 67, abstaining from stimulants and activities that may influence emotional and physiological state) were subjected to mood-induction procedures in the laboratory. Neutral, positive, and negative moods were induced by combinations of pictures from the international affective picture set (IAPS) and music. The subjects responded to questions on visual analog scales (VAS) in order to optimize sensitivity and attenuate short-term memory effects. Most subjects exhibited significant affective-state inductions with no change in arousal. The analyses took affective response to the manipulation into account. Only four of 20 questions were somewhat influenced by induced affective state: job overload, social support from co-workers, satisfaction with getting to develop personally, and an item measuring agreeableness. In general, responding to questions of work that were phrased for valence was little or insignificantly influenced by induced affective state.

New quantile based ridge M-estimator for linear regression models with multicollinearity and outliers

The ordinary least squares and ridge regression estimators in a multiple linear regression model with multicollinearity and y-direction outliers lead to unfavorable results. In order to mitigate such situation, the available literature provides few ridge M-estimators to get precise estimates. The ridge parameter, k, plays a vital role in a bias-variance tradeoff for these estimators. However, for high signal-to-noise ratio and multicollinearity with y-direction outliers, the available methods may not perform well in terms of their mean squared error. In this article, we propose a new quantile based ridge M-estimator. The new estimator gives an automated choice of quantile probability of ridge parameter according to the level of noise and multicollinearity. Based on a simulation study, the new estimator outperforms the ordinary least square estimator, ridge estimator, and other considered ridge M-estimators especially for high multicollinearity, significant error variance, and y-direction outliers. Besides normal distribution, new estimator also performs well for heavy-tailed error distribution. Finally, two real-life examples are used to illustrate the application of the proposed estimator.

Neutron-gamma discrimination method based on blind source separation and machine learning

The discrimination of neutrons from gamma rays in a mixed radiation field is crucial in neutron detection tasks. Several approaches have been proposed to enhance the performance and accuracy of neutron-gamma discrimination. However, their performances are often associated with certain factors, such as experimental requirements and resulting mixed signals. The main purpose of this study is to achieve fast and accurate neutron-gamma discrimination without a priori information on the signal to be analyzed, as well as the experimental setup. Here, a novel method is proposed based on two concepts. The first method exploits the power of nonnegative tensor factorization (NTF) as a blind source separation method to extract the original components from the mixture signals recorded at the output of the stilbene scintillator detector. The second one is based on the principles of support vector machine (SVM) to identify and discriminate these components. In addition to these two main methods, we adopted the Mexican-hat function as a continuous wavelet transform to characterize the components extracted using the NTF model. The resulting scalograms are processed as colored images, which are segmented into two distinct classes using the Otsu thresholding method to extract the features of interest of the neutrons and gamma-ray components from the background noise. We subsequently used principal component analysis to select the most significant of these features wich are used in the training and testing datasets for SVM. Bias-variance analysis is used to optimize the SVM model by finding the optimal level of model complexity with the highest possible generalization performance. In this framework, the obtained results have verified a suitable bias–variance trade-off value. We achieved an operational SVM prediction model for neutron-gamma classification with a high true-positive rate. The accuracy and performance of the SVM based on the NTF was evaluated and validated by comparing it to the charge comparison method via figure of merit. The results indicate that the proposed approach has a superior discrimination quality (figure of merit of 2.20).

Assessment of Ocean-Atmosphere Interactions for the Boreal Summer Intraseasonal Oscillations in CMIP5 Models over the Indian Monsoon Region

The boreal summer intraseasonal oscillations (BSISO) are the prominent features of South Asian summer monsoon and mainly governed by the internal atmospheric dynamics and air-sea interactions. The present study aims to understand and evaluate the relationship between the convection and the associated air-sea interactions during the BSISO over the Indian monsoon region. To accomplish this, the present study utilizes observations and the 22 general circulation model (GCM) simulations from the Coupled Model Intercomparison Project Phase 5 (CMIP5). Representation of Indian monsoon season rainfall, sea surface temperature (SST) and latent heat fluxes in CMIP5 models such as climatological and intraseasonal are assessed using Global Precipitation Climatology Project rainfall by Taylor diagram metric. Results suggest that the majority of CMIP5 models simulated the northward propagation of precipitation and zonal wind at 850 hPa. However, models bias of BSISO variance shows significant spatial heterogeneity over the regions of the Arabian Sea (AS), Sub-Continent of India (SCI) and Bay of Bengal (BoB). The CMIP5 model which shows large biases in the mean state is degrading the northward propagation of BSISO. The phase relationship of ocean (land) and atmospheric interactions are diagnosed with lead-lag regression analysis. On ISO timescales over north Indian Ocean (NIO) convection leads the turbulent fluxes and westerly winds by a week. However, the majority of the models shows large uncertainty to represent this prominent feature over AS and SCI. Further, improper representation of the lead-lag relationship of SST and precipitation on ISO scales over the AS, BoB, and NIO in the CMIP5 models are attributing for significant bias variances. The present study advocates that BSISO propagation in CMIP5 models is mainly attributing from the internal atmospheric dynamics and air-sea interactions. However, for the realistic amplitude simulation of BSISO, proper representation of air-sea feedback mechanisms is crucial in CMIP5 models. The present study further suggests that the oceanic feedback processes of the CMIP5 models need to be improved for the accurate prediction of the intraseasonal variations.