Dropout Effects Research Articles

PRIME: a probabilistic imputation method to reduce dropout effects in single-cell RNA sequencing.

Single-cell RNA sequencing technology provides a novel means to analyze the transcriptomic profiles of individual cells. The technique is vulnerable, however, to a type of noise called dropout effects, which lead to zero-inflated distributions in the transcriptome profile and reduce the reliability of the results. Single-cell RNA sequencing data, therefore, need to be carefully processed before in-depth analysis. Here, we describe a novel imputation method that reduces dropout effects in single-cell sequencing. We construct a cell correspondence network and adjust gene expression estimates based on transcriptome profiles for the local subnetwork of cells of the same type. We comprehensively evaluated this method, called PRIME (PRobabilistic IMputation to reduce dropout effects in Expression profiles of single-cell sequencing), on synthetic and eight real single-cell sequencing datasets and verified that it improves the quality of visualization and accuracy of clustering analysis and can discover gene expression patterns hidden by noise. The source code for the proposed method is freely available at https://github.com/hyundoo/PRIME. Supplementary data are available at Bioinformatics online.

Schools and neighborhood crime: The effects of dropouts and high-performing schools on juvenile crime

ABSTRACT Most scholars have focused on the simple presence of schools when examining their influence on neighborhood crime. This paper instead examines two school characteristics that might affect local juvenile crime. More specifically, this study uses negative binomial and logistic regression models to estimate the effects of high school dropouts and high-performing schools on juvenile violent crime in Orange County, CA. An integrated theoretical approach based on social bond and routine activity theories is utilized to guide this research. Findings from this study suggest that dropouts are associated with increases in aggravated assault and robbery incidents, but high-performing schools do not significantly affect these crime types. Finally, the models predicting juvenile crime are compared with models predicting all crime (both juveniles and adults), supporting our argument that juvenile crime is the theoretically appropriate crime measure.

The dropout effects of career pathways: Evidence from California

Contemporary Career and Technical Education (CTE) models have shifted from isolated courses to sequences of study that integrate academics and skills in high-demand sectors. Providing career pathways to high school students may reduce asymmetries about the available careers and strategies for attaining them but they may also catalyze students’ intrinsic motivation by shifting their understanding of their social role and capacity for success. In this study, I estimate the effects of an ambitious $500 million effort to encourage the formation of career pathways in California. Funding supported the formation of tripartite partnerships between K-12 school districts, employers and community colleges to develop career pathway curricula (i.e., articulated course sequences) in high-demand occupations and sectors. I provide causal estimates of implementing this multifaceted intervention by leveraging a natural experiment that occurs at the margin of grant receipt. Using Regression Discontinuity (RD) designs, I provide evidence on the most proximate mechanism, increased CTE spending. Per pupil CTE expenditures increased by 21.7 percent for grant recipients at the assignment threshold relative to the CTE spending of unsuccessful applicants. Furthermore, dropout rates declined by 23 percent in treatment districts but were more pronounced for females than males.

Underwater navigation methodology based on intelligent velocity model for standard AUV

This paper addresses a state-of-the-art navigation system aided by an intelligent velocity model for autonomous underwater vehicle (AUV). It's critical to obtain a precise navigation for the safety and effectiveness of AUV missions. The reliability of AUV navigation-related sensors becomes particularly important, while the accuracy of sensors, especially the Doppler velocity log (DVL), which is easy to be interfered, affects navigation accuracy directly. However, standard AUV does not have redundant velocity sensors. Once DVL is invalid caused by fish movement, sound scattering, or DVL overrange, it will have a catastrophic effect on AUV missions. To improve navigation robustness and avoid the effects of DVL dropouts, an intelligent velocity model is proposed to assist navigation by using optimally pruned extreme learning machine. The performance of model-aided navigation system is evaluated on data from our research platform Sailfish 210 AUV during sea trials on the high seas of yellow sea, which achieve 0.5% navigation accuracy. In addition, the intelligent velocity model without modifying the model parameters is more flexible in model acquisition than the traditional vehicle dynamic model when AUV sensor distribution and AUV mass change. Results show that it is possible to improve accuracy and robustness of navigation as a redundant velocity information for standard AUV.

Understanding dropout as an optimization trick

As one of standard approaches to train deep neural networks, dropout has been applied to regularize large models to avoid overfitting, and the improvement in performance by dropout has been explained as avoiding co-adaptation between nodes. However, when correlations between nodes are compared after training the networks with or without dropout, one question arises if co-adaptation avoidance explains the dropout effect completely. In this paper, we propose an additional explanation of why dropout works and propose a new technique to design better activation functions. First, we show that dropout can be explained as an optimization technique to push the input towards the saturation area of nonlinear activation function by accelerating gradient information flowing even in the saturation area in backpropagation. Based on this explanation, we propose a new technique for activation functions, gradient acceleration in activation function (GAAF), that accelerates gradients to flow even in the saturation area. Then, input to the activation function can climb onto the saturation area which makes the network more robust because the model converges on a flat region. Experiment results support our explanation of dropout and confirm that the proposed GAAF technique improves image classification performance with expected properties.

Dropout vs. batch normalization: an empirical study of their impact to deep learning

Overfitting and long training time are two fundamental challenges in multilayered neural network learning and deep learning in particular. Dropout and batch normalization are two well-recognized approaches to tackle these challenges. While both approaches share overlapping design principles, numerous research results have shown that they have unique strengths to improve deep learning. Many tools simplify these two approaches as a simple function call, allowing flexible stacking to form deep learning architectures. Although their usage guidelines are available, unfortunately no well-defined set of rules or comprehensive studies to investigate them concerning data input, network configurations, learning efficiency, and accuracy. It is not clear when users should consider using dropout and/or batch normalization, and how they should be combined (or used alternatively) to achieve optimized deep learning outcomes. In this paper we conduct an empirical study to investigate the effect of dropout and batch normalization on training deep learning models. We use multilayered dense neural networks and convolutional neural networks (CNN) as the deep learning models, and mix dropout and batch normalization to design different architectures and subsequently observe their performance in terms of training and test CPU time, number of parameters in the model (as a proxy for model size), and classification accuracy. The interplay between network structures, dropout, and batch normalization, allow us to conclude when and how dropout and batch normalization should be considered in deep learning. The empirical study quantified the increase in training time when dropout and batch normalization are used, as well as the increase in prediction time (important for constrained environments, such as smartphones and low-powered IoT devices). It showed that a non-adaptive optimizer (e.g. SGD) can outperform adaptive optimizers, but only at the cost of a significant amount of training times to perform hyperparameter tuning, while an adaptive optimizer (e.g. RMSProp) performs well without much tuning. Finally, it showed that dropout and batch normalization should be used in CNNs only with caution and experimentation (when in doubt and short on time to experiment, use only batch normalization).

Finite-horizon Anisotropy-based Estimation with Packet Dropouts

In this paper, the estimation problem is studied for a class of linear discrete time-varying system with packet dropout in the framework of anisotropy-based theory. The extended vector of fragment of the disturbance sequence is from the set of random vectors with bounded anisotropy. The packet dropout effect is considered to be random and described by a binary switching sequence with Bernoulli distribution. The input-to-error dynamics is obtained for multiplicative noise system with mutually independent noises and input disturbance. By using anisotropy-based approach, the estimation problem is reduced to optimization one with convex constraints. The developed method provides the (sub)optimal estimator ensuring the boundedness of anisotropic norm for input-to-output error system. Numerical example is provided to demonstrate efficiency of proposed approach.

Encoding-Decoding Mechanism-Based Finite-Level Quantized Iterative Learning Control With Random Data Dropouts

Learning control is investigated to solve the tracking problem for linear systems via unreliable networks with random data dropouts. By using an encoding-decoding mechanism-based finite-level uniform quantizer, the communication burden is remarkably reduced while retaining a precise tracking performance. An intermittent update principle is adopted in both the encoding-decoding mechanism and the learning control algorithm to handle the effects of data dropouts. A special case, in which no consecutive dropouts exist along the iteration axis, is provided first and then extended to a general case, in which bounded consecutive data dropouts are allowed. A precise analysis of the maximum range of the finite-level quantizer and the associated asymptotical convergence is conducted. Illustrative simulations demonstrate the validity of the proposed framework.

Quantile regression for challenging cases of eQTL mapping.

Mapping of expression quantitative trait loci (eQTLs) facilitates interpretation of the regulatory path from genetic variants to their associated disease or traits. High-throughput sequencing of RNA (RNA-seq) has expedited the exploration of these regulatory variants. However, eQTL mapping is usually confronted with the analysis challenges caused by overdispersion and excessive dropouts in RNA-seq. The heavy-tailed distribution of gene expression violates the assumption of Gaussian distributed errors in linear regression for eQTL detection, which results in increased Type I or Type II errors. Applying rank-based inverse normal transformation (INT) can make the expression values more normally distributed. However, INT causes information loss and leads to uninterpretable effect size estimation. After comprehensive examination of the impact from overdispersion and excessive dropouts, we propose to apply a robust model, quantile regression, to map eQTLs for genes with high degree of overdispersion or large number of dropouts. Simulation studies show that quantile regression has the desired robustness to outliers and dropouts, and it significantly improves eQTL mapping. From a real data analysis, the most significant eQTL discoveries differ between quantile regression and the conventional linear model. Such discrepancy becomes more prominent when the dropout effect or the overdispersion effect is large. All the results suggest that quantile regression provides more reliable and accurate eQTL mapping than conventional linear models. It deserves more attention for the large-scale eQTL mapping.

Predictive Outcome Validity of General Health Questionnaire (GHQ-28) in Substance Abuse Patients Treated in Therapeutic Communities

Objective: The use of psychoactive substances has been one of the most important global public health problems over the last few decades. Among the problems associated with substance use, dual diagnosis is one of the most relevant. This study aims to investigate the predictive validity of the GHQ-28 (General Health Questionnaire-28; “probable psychiatric cases”) in relation to poor treatment outcome measured by (a) “early treatment dropout” and (b) “nonclinically relevant improvement at discharge.” Methods: A longitudinal prospective design was used. A sample of 219 substance use disorder patients, who received treatment in a therapeutic community, was selected. Patients were assessed using the GHQ-28 and the outcome variables were registered. A hierarchical logistic regression model was performed to identify factors independently associated with the outcome measure (“early treatment dropout” and “nonclinically relevant improvement at discharge”). Results: Of the total sample, 79 subjects (36%) were considered “early treatment dropouts” and 56.6% (102) presented a “nonclinically relevant improvement at discharge.” The two hierarchical logistic regression results show that being classified as a “probable psychiatric case” was significantly and directly associated with “early treatment dropout” and “nonclinically relevant improvement at discharge,” as a poor in-treatment outcome indicator. Conclusions: The results of this study support the notion that the probable psychiatric cases identified by the GHQ-28 scale have a greater probability of “early treatment dropout” and have a greater probability of “nonclinically relevant improvement in discharge” of the therapeutic community. These data indicate that GHQ-28 is a suitable clinical instrument for predicting dropout and treatment effect in residential substance use disorder treatment.

De novo identification of essential protein domains from CRISPR-Cas9 tiling-sgRNA knockout screens

High-throughput CRISPR-Cas9 knockout screens using a tiling-sgRNA design permit in situ evaluation of protein domain function. Here, to facilitate de novo identification of essential protein domains from such screens, we propose ProTiler, a computational method for the robust mapping of CRISPR knockout hyper-sensitive (CKHS) regions, which refer to the protein regions associated with a strong sgRNA dropout effect in the screens. Applied to a published CRISPR tiling screen dataset, ProTiler identifies 175 CKHS regions in 83 proteins. Of these CKHS regions, more than 80% overlap with annotated Pfam domains, including all of the 15 known drug targets in the dataset. ProTiler also reveals unannotated essential domains, including the N-terminus of the SWI/SNF subunit SMARCB1, which is validated experimentally. Surprisingly, the CKHS regions are negatively correlated with phosphorylation and acetylation sites, suggesting that protein domains and post-translational modification sites have distinct sensitivities to CRISPR-Cas9 mediated amino acids loss.

The Phenomenon of Treatment Dropout, Reasons and Moderators in Acceptance and Commitment Therapy and Other Active Treatments: A Meta-Analytic Review

Treatment dropout is one of the most crucial issues that a therapist has to face on a daily basis. The negative effects of premature termination impact the client who is usually found to demonstrate poorer treatment outcomes. This meta-analysis reviewed and systematically examined dropout effects of Acceptance and Commitment Therapy (ACT) as compared to other active treatments. The goals of this study were to compare treatment dropout rates and dropout reasons, examine the influence of demographic variables and identify possible therapy moderators associated with dropout.The current meta-analysis reviewed 76 studies of ACT reporting dropout rates for various psychological and health-related conditions.Across reviewed studies (N = 76), the overall weighted mean dropout rate was 17.95% (ACT = 17.35% vs. comparison conditions = 18.62%). Type of disorder, recruitment setting and therapists’ experience level were significant moderators of dropout. The most frequently reported reasons for dropout from ACT were lost contact, personal and transportation difficulties, whereas for comparative treatments they were lost contact, therapy factors and time demands.Given that most moderators of influence are not amenable to direct changes by clinicians, mediation variables should also be explored. Overall, results suggest that ACT appears to present some benefits in dropout rates for specific disorders, settings and therapists.There was no difference in dropout rate between ACT and control conditions (17.35% vs.18.62%). Significant moderators were client disorder, therapists’ experience level and recruitment and setting.Comparison condition frequently reported therapy related dropout factors, suggesting that ACT may be a more acceptable option.There was no difference in dropout rate between ACT and control conditions (17.35% vs.18.62%). Significant moderators were client disorder, therapists’ experience level and recruitment and setting.Comparison condition frequently reported therapy related dropout factors, suggesting that ACT may be a more acceptable option.

Quantized control for nonhomogeneous Markovian jump T-S fuzzy systems with missing measurements

In this paper, in terms of the T-S fuzzy technique, the quantization control designs are resolved for a class of nonhomogeneous Markov jump systems (MJSs) with partially unknown transition probabilities. Different from the previous research, the transition probabilities are time-variant and not known exactly in the MJSs. Particularly in a network environment, it is considered that the effects of data packet dropouts and the occurrence of signal quantization simultaneously emerge in the closed-loop circuit. Furthermore, based on a fuzzy Lyapunov function and a set of linear matrix inequalities, one can achieve the desired H∞ performance and the sufficient conditions such that the corresponding closed-loop system is stochastically stable. By the cone complementarity linearisation (CCL) procedure, a sequential minimization problem is tackled efficiently to gain the solutions of the dynamic output feedback controller (DOFC). Finally, the validity of the suggested technique is showed via a simulation example.

Patient-reported outcomes in schizophrenia patients treated with once-monthly extended-release risperidone in a long-term clinical study.

PurposeRBP-7000 (PERSERIS™) is a once-monthly subcutaneous extended-release risperidone formulation approved by the United States Food and Drug Administration for the treatment of schizophrenia in adults. The objective of this study was to describe the long-term impact of RBP-7000 on health-related quality of life (HRQoL), subjective well-being, treatment satisfaction and medication preference in patients with schizophrenia.Patients and methodsHRQoL was derived from a 52-week multicentre Phase III single-arm open-label outpatient study that assessed the safety and efficacy of RBP-7000 (120 mg) in patients with schizophrenia. HRQoL was measured using the EuroQol EQ-5D-5L and Short-Form Survey SF-36 version 2; well-being using the Subjective Well-being Under Neuroleptic Treatment – Short Version (SWN-S); satisfaction using the Medication Satisfaction Questionnaire and medication preference using the Preference of Medication questionnaire.ResultsOf 482 participants at baseline, 234 remained through the end of study (EOS; week 52). Mean HRQoL and well-being scores remained stable between baseline (EQ-5D-5L index: 0.83; SF-36v2 Physical Component Score: 50; SF-36v2 Mental Component Score: 46; total SWN-S score: 89) and EOS (EQ-5D-5L index: 0.86; SF-36v2 Physical Component Score: 49; SF-36v2 Mental Component Score: 47; total SWN-S score: 90). The proportion of participants reporting satisfaction increased between week 4 (66%) and EOS (81%), with a similar trend for the preference of RBP-7000 over previous treatment (week 4: 66%; EOS: 72%). Sensitivity analyses suggested a minor effect of dropout on characterization of change over time in patient-reported outcomes (PRO) measures.ConclusionStudy participants attained mean HRQoL scores near that of the general US population. Over two-thirds reported high satisfaction with and preference for RBP-7000 across the study period. Additional research is needed to confirm whether these PRO translate into improved outcomes such as adherence and ultimately fewer relapses in patients with schizophrenia.

Why Your Neural Network is Still Singular and What You Can Do About It

We investigate the effects of neural network regularization techniques. First, we reason formally through the effect of dropout and training stochasticity on gradient descent. Then, we conduct classification experiments on the ImageNet data set, as well as regression experiments in the OneNow Reinforcement Learning data set. A network layer's weight matrix is quantified via Singular Value Decomposition and Conditioning ratios. Our regression network appeared to be well conditioned. However, we find that learning for large-scale classification applications is likely to be capped by poor conditioning. We propose approaches that may prove breakthroughs in learning, providing early evidence. We introduce a gradient perturbation layer, a method to maximize generalization experimentally. Our numerical analysis showcases the opportunity to introduce network circuitry compression, relying on the principal components of a layer's weights, when conditioning peaks. Generally, we propose conditioning as an objective function constraint.

Distributed H∞-Consensus Filtering for Attitude Tracking Using Ground-Based Radars.

This paper is concerned with the distributed H ∞ -consensus filtering problem on attitude tracking over a radar filter network subject to switching topology and random packet dropouts occurring in the data transmission from both the Sun sensor and the filters. Since ground-based radars cannot directly measure the satellite attitude, a Sun sensor is deployed at the satellite side and its measurements are transmitted to radar filters through different network communication channels while suffering from random packet dropouts with different probabilities. In the radar filter network, each radar filter receives data not only from the Sun sensor but also from its local neighboring radar filters in accordance with a switching network topology. A delicate distributed H ∞ -consensus filtering algorithm, which incorporates the effects of switching network topology and random packet dropouts, is adopted to estimate attitude and attitude-rate. The algorithm guarantees H ∞ -consensus attenuation performance for the estimation deviations among radar filters, and the robustness against the switching network topology and packet dropouts for the radar filter network. The illustrative examples are given to verify the effectiveness of the proposed distributed H ∞ -consensus filtering algorithm.

Locality Sensitive Imputation for Single Cell RNA-Seq Data.

One of the most notable challenges in single cell RNA-Seq data analysis is the so called drop-out effect, where only a fraction of the transcriptome of each cell is captured. The random nature of dropouts, however, makes it possible to consider imputation methods as means of correcting for dropouts. In this article, we study some existing single cell RNA sequencing (scRNA-Seq) imputation methods and propose a novel iterative imputation approach based on efficiently computing highly similar cells. We then present the results of a comprehensive assessment of existing and proposed methods on real scRNA-Seq data sets with varying per cell sequencing depth.

Networked predictive control for linear systems with quantizers by an event-driven strategy

In this paper, networked predictive control is investigated for a networked control system with quantizers by an event-driven strategy. An event generator is designed according to a deviation of state estimation between the current time and last trigger time. A predictive strategy is proposed to compensate effect of network-induced delays and packet dropouts. The quantizers are used to deal with signals by converting real-valued signals into effective ones in both feedback and forward channels. Based on a “zoom” strategy, sufficient conditions are given to ensure stabilization of the networked control system by solving linear matrix inequalities. A simulation example is proposed to exhibit advantages and availability of the developed techniques.

Improving visual question answering using dropout and enhanced question encoder

Using dropout in Visual Question Answering (VQA) is a common practice to prevent overfitting. However, the current way to use dropout in multi-path networks may cause two problems: the co-adaptations of neurons and the explosion of output variance. In this paper, we propose coherent dropout and siamese dropout mechanism to solve the two problems, respectively. Specifically, in coherent dropout, the relevant dropout layers in multiple paths are forced to work coherently to maximize the ability of preventing neuron co-adaptations. We show that the coherent dropout is simple in implementation but very effective to overcome overfitting. As for the explosion of output variance, we develop a siamese dropout mechanism to explicitly minimize the difference between the two output vectors produced from the same input data during training phase. Such mechanism can reduce the gap between training and inference phases and make the VQA model more robust. With the help of the two techniques, we further design an enhanced question encoder called Multi-path Stacked Residual RNNs which is deeper and wider and more powerful than current shallow question encoder. Extensive experiments are conducted to verify the effectiveness of coherent dropout, siamese dropout and the enhanced question encoder. And the results show that our methods can bring clear improvements to the state-of-the-art VQA models on VQA-v1 and VQA-v2 datasets.

Comparative Research on Influencing Factors of LSTM Deep Neural Network in Stock Market Time Series Prediction

<p><em>During training process of LSTM, the prediction accuracy is affected by a variation of factors, including the selection of training samples, the network structure, the optimization algorithm, and the stock market status. This paper tries to conduct a systematic research on several influencing factors of LSTM training in context of time series prediction. The experiment uses Shanghai and Shenzhen 300 constituent stocks from 2006 to 2017 as samples. The influencing factors of the study include indicator sampling, sample length, network structure, optimization method, and data of the bull and bear market, and this experiment compared the effects of PCA, dropout, and L2 regularization on predict accuracy and efficiency. Indice sampling, number of samples, network structure, optimization techniques, and PCA are found to be have their scope of application. Further, dropout and L2 regularization are found positive to improve the accuracy. The experiments cover most of the factors, however have to be compared by data overseas. This paper is of significance for feature and parameter selection in LSTM training process.</em></p>