Design Of Learning Algorithms Research Articles

MaxiMin Active Learning in Overparameterized Model Classes

Generating labeled training datasets has become a major bottleneck in Machine Learning (ML) pipelines. Active ML aims to address this issue by designing learning algorithms that automatically and adaptively select the most informative examples for labeling so that human time is not wasted labeling irrelevant, redundant, or trivial examples. This paper proposes a new approach to active ML with nonparametric or overparameterized models such as kernel methods and neural networks. In the context of binary classification, the new approach is shown to possess a variety of desirable properties that allow active learning algorithms to automatically and efficiently identify decision boundaries and data clusters.

Evolution of Post‐natal Mouse Respiratory Physiology Revealed by Machine Learning‐based Data Analytics

Current methods to analyze respiratory behavior utilize plethysmography to acquire standing values of frequency, volume and flow at specific time points in murine maturation. However, these factors have been independently analyzed as a function of time with no further analysis of the interplay these variables have amongst each other in the context of maturation. We have developed a R‐based analytical workflow to examine how variation and relationships of frequency (f) and tidal volume (TV) with respect to inspiratory and expiratory parameters shift with age in mice. Our analysis evidences that deeper physiological relationships and variation in breathing behavior can be seen with age. Moreover, models with respect to age are predictive‐‐suggesting a statistical method to evidence changes in respiratory breathing interplay with age. To further this, we designed learning algorithms in R to test the ability to predict age based on these parameters. This data indicates that respiratory pattern control changes and strengthens with age and provides new insight to model paradigms of respiratory maturation in developmental research.Support or Funding InformationThis work was supported by NIH/NHLBI (R01HL132355 for CMC, JJO).

Generalizable sgRNA design for improved CRISPR/Cas9 editing efficiency.

The development of clustered regularly interspaced short palindromic repeat (CRISPR)/CRISPR-associated protein 9 (Cas9) technology has provided a simple yet powerful system for targeted genome editing. In recent years, this system has been widely used for various gene editing applications. The CRISPR editing efficacy is mainly dependent on the single guide RNA (sgRNA), which guides Cas9 for genome cleavage. While there have been multiple attempts at improving sgRNA design, there is a pressing need for greater sgRNA potency and generalizability across various experimental conditions. We employed a unique plasmid library expressed in human cells to quantify the potency of thousands of CRISPR/Cas9 sgRNAs. Differential sequence and structural features among the most and least potent sgRNAs were then used to train a machine learning algorithm for assay design. Comparative analysis indicates that our new algorithm outperforms existing CRISPR/Cas9 sgRNA design tools. The new sgRNA design tool is freely accessible as a web application, http://crispr.wustl.edu. Supplementary data are available at Bioinformatics online.

A Learning-Based Incentive Mechanism for Federated Learning

Internet of Things (IoT) generates large amounts of data at the network edge. Machine learning models are often built on these data, to enable the detection, classification, and prediction of the future events. Due to network bandwidth, storage, and especially privacy concerns, it is often impossible to send all the IoT data to the data center for centralized model training. To address these issues, federated learning has been proposed to let nodes use the local data to train models, which are then aggregated to synthesize a global model. Most of the existing work has focused on designing learning algorithms with provable convergence time, but other issues, such as incentive mechanism, are unexplored. Although incentive mechanisms have been extensively studied in network and computation resource allocation, yet they cannot be applied to federated learning directly due to the unique challenges of information unsharing and difficulties of contribution evaluation. In this article, we study the incentive mechanism for federated learning to motivate edge nodes to contribute model training. Specifically, a deep reinforcement learning-based (DRL) incentive mechanism has been designed to determine the optimal pricing strategy for the parameter server and the optimal training strategies for edge nodes. Finally, numerical experiments have been implemented to evaluate the efficiency of the proposed DRL-based incentive mechanism.

Tuning heuristics and convergence analysis of reinforcement learning algorithm for online data-based optimal control design

A heuristic for tuning and convergence analysis of the reinforcement learning algorithm for control with output feedback with only input / output data generated by a model is presented. To promote convergence analysis, it is necessary to perform the parameter adjustment in the algorithms used for data generation, and iteratively solve the control problem. A heuristic is proposed to adjust the data generator parameters creating surfaces to assist in the convergence and robustness analysis process of the optimal online control methodology. The algorithm tested is the discrete linear quadratic regulator (DLQR) with output feedback, based on reinforcement learning algorithms through temporal difference learning in the policy iteration scheme to determine the optimal policy using input / output data only. In the policy iteration algorithm, recursive least squares (RLS) is used to estimate online parameters associated with output feedback DLQR. After applying the proposed tuning heuristics, the influence of the parameters could be clearly seen, and the convergence analysis facilitated.

Recent Advances in Single-Atom Electrocatalysts for Oxygen Reduction Reaction.

Oxygen reduction reaction (ORR) plays significant roles in electrochemical energy storage and conversion systems as well as clean synthesis of fine chemicals. However, the ORR process shows sluggish kinetics and requires platinum-group noble metal catalysts to accelerate the reaction. The high cost, rare reservation, and unsatisfied durability significantly impede large-scale commercialization of platinum-based catalysts. Single-atom electrocatalysts (SAECs) featuring with well-defined structure, high intrinsic activity, and maximum atom efficiency have emerged as a novel field in electrocatalytic science since it is promising to substitute expensive platinum-group noble metal catalysts. However, finely fabricating SAECs with uniform and highly dense active sites, fully maximizing the utilization efficiency of active sites, and maintaining the atomically isolated sites as single-atom centers under harsh electrocatalytic conditions remain urgent challenges. In this review, we summarized recent advances of SAECs in synthesis, characterization, oxygen reduction reaction (ORR) performance, and applications in ORR-related H2O2 production, metal-air batteries, and low-temperature fuel cells. Relevant progress on tailoring the coordination structure of isolated metal centers by doping other metals or ligands, enriching the concentration of single-atom sites by increasing metal loadings, and engineering the porosity and electronic structure of the support by optimizing the mass and electron transport are also reviewed. Moreover, general strategies to synthesize SAECs with high metal loadings on practical scale are highlighted, the deep learning algorithm for rational design of SAECs is introduced, and theoretical understanding of active-site structures of SAECs is discussed as well. Perspectives on future directions and remaining challenges of SAECs are presented.

Learning Discrete-Time Markov Chains Under Concept Drift.

Learning under concept drift is a novel and promising research area aiming at designing learning algorithms able to deal with nonstationary data-generating processes. In this research field, most of the literature focuses on learning nonstationary probabilistic frameworks, while some extensions about learning graphs and signals under concept drift exist. For the first time in the literature, this paper addresses the problem of learning discrete-time Markov chains (DTMCs) under concept drift. More specifically, following a hybrid active/passive approach, this paper introduces both a family of change-detection mechanisms (CDMs), differing in the required assumptions and performance, for detecting changes in DTMCs and an adaptive learning algorithm able to deal with DTMCs under concept drift. The effectiveness of both the proposed CDMs and the adaptive learning algorithm has been extensively tested on synthetically generated experiments and real data sets.

Horn-ICE learning for synthesizing invariants and contracts

We design learning algorithms for synthesizing invariants using Horn implication counterexamples (Horn-ICE), extending the ICE-learning model. In particular, we describe a decision-tree learning algorithm that learns from nonlinear Horn-ICE samples, works in polynomial time, and uses statistical heuristics to learn small trees that satisfy the samples. Since most verification proofs can be modeled using nonlinear Horn clauses, Horn-ICE learning is a more robust technique to learn inductive annotations that prove programs correct. Our experiments show that an implementation of our algorithm is able to learn adequate inductive invariants and contracts efficiently for a variety of sequential and concurrent programs.

Dense Associative Memory Is Robust to Adversarial Inputs.

Deep neural networks (DNNs) trained in a supervised way suffer from two known problems. First, the minima of the objective function used in learning correspond to data points (also known as rubbish examples or fooling images) that lack semantic similarity with the training data. Second, a clean input can be changed by a small, and often imperceptible for human vision, perturbation so that the resulting deformed input is misclassified by the network. These findings emphasize the differences between the ways DNNs and humans classify patterns and raise a question of designing learning algorithms that more accurately mimic human perception compared to the existing methods. Our article examines these questions within the framework of dense associative memory (DAM) models. These models are defined by the energy function, with higher-order (higher than quadratic) interactions between the neurons. We show that in the limit when the power of the interaction vertex in the energy function is sufficiently large, these models have the following three properties. First, the minima of the objective function are free from rubbish images, so that each minimum is a semantically meaningful pattern. Second, artificial patterns poised precisely at the decision boundary look ambiguous to human subjects and share aspects of both classes that are separated by that decision boundary. Third, adversarial images constructed by models with small power of the interaction vertex, which are equivalent to DNN with rectified linear units, fail to transfer to and fool the models with higher-order interactions. This opens up the possibility of using higher-order models for detecting and stopping malicious adversarial attacks. The results we present suggest that DAMs with higher-order energy functions are more robust to adversarial and rubbish inputs than DNNs with rectified linear units.

Pushing the Limits of Deep CNNs for Pedestrian Detection

Compared with other applications in computer vision, convolutional neural networks (CNNs) have underperformed on pedestrian detection. A breakthrough was made very recently using sophisticated deep CNN (DCNN) models, with a number of handcrafted features or explicit occlusion handling mechanism. In this paper, we show that by reusing the convolutional feature maps of a DCNN model as image features to train an ensemble of boosted decision models, we are able to achieve the best reported accuracy without using specially designed learning algorithms. We empirically identify and disclose important implementation details. We also show that pixel labeling may be simply combined with a detector to boost the detection performance. By adding complementary handcrafted features such as optical flow, the DCNN-based detector can be further improved. We advance the state-of-the-art results by lowering the log-average miss rate from 11.7% to 8.9% on the Caltech data set and from 11.2% to 8.6% on the Inria data set. We also achieve a comparable result to state-of-the-art approaches on the KITTI data set.

Generalising better: Applying deep learning to integrate deleteriousness prediction scores for whole-exome SNV studies.

Many automatic classifiers were introduced to aid inference of phenotypical effects of uncategorised nsSNVs (nonsynonymous Single Nucleotide Variations) in theoretical and medical applications. Lately, several meta-estimators have been proposed that combine different predictors, such as PolyPhen and SIFT, to integrate more information in a single score. Although many advances have been made in feature design and machine learning algorithms used, the shortage of high-quality reference data along with the bias towards intensively studied in vitro models call for improved generalisation ability in order to further increase classification accuracy and handle records with insufficient data. Since a meta-estimator basically combines different scoring systems with highly complicated nonlinear relationships, we investigated how deep learning (supervised and unsupervised), which is particularly efficient at discovering hierarchies of features, can improve classification performance. While it is believed that one should only use deep learning for high-dimensional input spaces and other models (logistic regression, support vector machines, Bayesian classifiers, etc) for simpler inputs, we still believe that the ability of neural networks to discover intricate structure in highly heterogenous datasets can aid a meta-estimator. We compare the performance with various popular predictors, many of which are recommended by the American College of Medical Genetics and Genomics (ACMG), as well as available deep learning-based predictors. Thanks to hardware acceleration we were able to use a computationally expensive genetic algorithm to stochastically optimise hyper-parameters over many generations. Overfitting was hindered by noise injection and dropout, limiting coadaptation of hidden units. Although we stress that this work was not conceived as a tool comparison, but rather an exploration of the possibilities of deep learning application in ensemble scores, our results show that even relatively simple modern neural networks can significantly improve both prediction accuracy and coverage. We provide open-access to our finest model via the web-site: http://score.generesearch.ru/services/badmut/.

Evaluation for Sortie Generation Capacity of the Carrier Aircraft Based on the Variable Structure RBF Neural Network with the Fast Learning Rate

The neural network has the advantages of self‐learning, self‐adaptation, and fault tolerance. It can establish a qualitative and quantitative evaluation model which is closer to human thought patterns. However, the structure and the convergence rate of the radial basis function (RBF) neural network need to be improved. This paper proposes a new variable structure radial basis function (VS‐RBF) with a fast learning rate, in order to solve the problem of structural optimization design and parameter learning algorithm for the radial basis function neural network. The number of neurons in the hidden layer is adjusted by calculating the output information of neurons in the hidden layer and the multi‐information between neurons in the hidden layer and output layer. This method effectively solves the problem that the RBF neural network structure is too large or too small. The convergence rate of the RBF neural network is improved by using the robust regression algorithm and the fast learning rate algorithm. At the same time, the convergence analysis of the VS‐RBF neural network is given to ensure the stability of the RBF neural network. Compared with other self‐organizing RBF neural networks (self‐organizing RBF (SORBF) and rough RBF neural networks (RS‐RBF)), VS‐RBF has a more compact structure, faster dynamic response speed, and better generalization ability. The simulations of approximating a typical nonlinear function, identifying UCI datasets, and evaluating sortie generation capacity of an carrier aircraft show the effectiveness of VS‐RBF.

Dynamic Inventory Control with Stockout Substitution and Demand Learning

We consider an inventory control problem with multiple products and stockout substitution. The firm knows neither the primary demand distribution for each product nor the customers' substitution probabilities between products a priori, and needs to learn such information from sales data on the fly. A main challenge in this problem is that the firm cannot distinguish between primary demand and substitution (overflow) demand from the sales data of any product, and due to data censoring, lost customers from either demand sources cannot be observed. To circumvent these difficulties, we construct learning stages with each stage including a cyclic testing scheme and a benchmark exploration interval. The benchmark interval allows us to isolate the primary demand information from the sales data, that is used against the sales data from a cyclic exploration interval to estimate substitution probabilities. Since raising inventory level helps obtain primary demand information but hinders substitution demand information, inventory decisions have to be carefully balanced to learn both of them. We design learning algorithms for both stationary and dynamically changing environments, and show that their regret rates (almost) match the theoretical lower bounds. Numerical experiments reveal that the proposed algorithms perform very well.

Learning Methods for Dynamic Topic Modeling in Automated Behavior Analysis.

Semisupervised and unsupervised systems provide operators with invaluable support and can tremendously reduce the operators' load. In the light of the necessity to process large volumes of video data and provide autonomous decisions, this paper proposes new learning algorithms for activity analysis in video. The activities and behaviors are described by a dynamic topic model. Two novel learning algorithms based on the expectation maximization approach and variational Bayes inference are proposed. Theoretical derivations of the posterior estimates of model parameters are given. The designed learning algorithms are compared with the Gibbs sampling inference scheme introduced earlier in the literature. A detailed comparison of the learning algorithms is presented on real video data. We also propose an anomaly localization procedure, elegantly embedded in the topic modeling framework. It is shown that the developed learning algorithms can achieve 95% success rate. The proposed framework can be applied to a number of areas, including transportation systems, security, and surveillance.

Optimal output-feedback tracking of SISO stochastic nonlinear systems using multi-dimensional Taylor network

The randomness and nonlinearity of stochastic nonlinear systems increase computational complexity and impede their tracking performance. However, randomness and nonlinearity are inevitable in practical applications, and the existing methods can hardly achieve the desirable control effect, especially in real-time control. For this end, a new network control strategy based on multi-dimensional Taylor network (MTN), whose design depends only on the system output, is put forward to solve the optimal output-feedback tracking problem of the SISO stochastic nonlinear systems. The network structure of the MTN is given first, and its approximation properties are proven. Based on the quadratic cost function design learning algorithm, the tracking error is minimized to update the controller parameters, and the desired tracking performance is obtained. Using the Lyapunov stability theorem, it is proved that the corresponding closed-loop system is bounded in the sense of probability and it can be ensured that the output tracking error converges to a small residual set around the origin in the sense of probability. An example is provided to illustrate the effectiveness of the proposed design approach. Comparative simulation study reveals that the proposed solution promises desirable real-time dynamic performance.

Label Efficient Learning by Exploiting Multi-Class Output Codes

We present a new perspective on the popular multi-class algorithmic techniques of one-vs-all and error correcting output codes. Rather than studying the behavior of these techniques for supervised learning, we establish a connection between the success of these methods and the existence of label-efficient learning procedures. We show that in both the realizable and agnostic cases, if output codes are successful at learning from labeled data, they implicitly assume structure on how the classes are related. By making that structure explicit, we design learning algorithms to recover the classes with low label complexity. We provide results for the commonly studied cases of one-vs-all learning and when the codewords of the classes are well separated. We additionally consider the more challenging case where the codewords are not well separated, but satisfy a boundary features condition that captures the natural intuition that every bit of the codewords should be significant.

Multiagent Cooperative Reinforcement Learning by Expert Agents (MCRLEA)

The paper gives novel approach Multiagent Cooperative Reinforcement Learning by Expert Agents (MCRLEA) for dynamic decision making in the retail application. Furthermore, it put up different cooperation schemes for multiagent cooperative reinforcement learning i.e. EQ learning, EGroup, EDynamic, EGoal driven and Expert agents scheme. Implementation outcome includes a demonstration of recommended cooperation schemes that are competent enough to speedup the collection of agents that achieve excellent action policies. Accordingly this approach presents three retailer stores in the retail market place. Retailers can help to each other and can obtain profit from cooperation knowledge through learning their own strategies that just stand for their aims and benefit. The vendors are the knowledgeable agents in the hypothesis to employ cooperative learning to train in the circumstances. Assuming significant hypothesis on the vendor’s stock policy, restock period, arrival process of the consumers, the approach is formed as Markov decision process model that makes it possible to design learning algorithms. The proposed algorithms noticeably learn dynamic consumer performance. Moreover, the paper illustrates results of Cooperative Reinforcement Learning Algorithms of three shop agents for the period of one year sale duration and then demonstrated the results using proposed approach for three shop agents for the period of one year sale duration. The results obtained by the proposed expert agent based cooperation approach show that such methods can put into a quick convergence of agents in the dynamic environment.

The Quality Prediction in Small-batch Producing Based on Weighted Least Squares Support Vector Regression

A novel quality prediction method with mobile time window is proposed for small-batch producing process based on weighted least squares support vector regression (LS-SVR). The design steps and learning algorithm are also addressed. In the method, weighted LS-SVR is taken as the intelligent kernel, with which the small-batch learning is solved well and the nearer sample is set a larger weight, while the farther is set the smaller weight in the history data. A typical machining process of cutting bearing outer race is carried out and the real measured data are used to contrast experiment. The experimental results demonstrate that the prediction accuracy of the weighted LSSVR based model is only 20%-30% that of the standard LS-SVR based one in the same condition. It provides a better candidate for quality prediction of small-batch producing process.

Self-organization hybrid evolution learning algorithm for recurrent wavelet-based neuro-fuzzy identifier design

In this paper, a recurrent wavelet-based neuro-fuzzy identifier RWNFI with a self-organization hybrid evolution learning algorithm SOHELA is proposed for solving various identification problems. In the proposed SOHELA, the group-based symbiotic evolution GSE is adopted such that each group in the GSE represents a collection of only one fuzzy rule. The proposed SOHELA consists of structure learning and parameter learning. In structure learning, the proposed SOHELA uses the self-organization algorithm SOA to determine a suitable rule number in the RWNFI. In parameter learning, the proposed SOHELA uses the data mining-based selection method DMSM and the data mining-based crossover method DMCM to determine groups and parent groups using the data mining method called the frequent pattern growth FP-Growth method. Based on identification simulations, the excellent performance of the proposed SOHELA compares with other various existing models.

Architecture of Artificial Neural Network in Identification of Internal Dynamics and Prediction of Dynamic System Rainfall Data Time Series

The objective of this study is to expand and evaluate the back-propagation artificial neural network (BPANN) and to apply in the identification of internal dynamics of very high dynamic system such long-range total rainfall data time series. This objective is considered via comprehensive review of literature (1978-2011). It is found that, detail of discussion concerning the architecture of ANN for the same is rarely visible in the literature; however various applications of ANN are available. The detail architecture of BPANN with its parameters, i.e., learning rate, number of hidden layers, number of neurons in hidden layers, number of input vectors in input layer, initial and optimized weights etc., designed learning algorithm, observations of local and global minima, and results have been discussed. It is observed that obtaining global minima is almost complicated and always a temporal nervousness. However, achievement of global minima for the period of the training has been discussed. It is found that, the application of the BPANN on identification for internal dynamics and prediction for the long-range total annual rainfall has produced good results. The results are explained through the strong association between rainfall predictors i.e., climate parameter (independent parameter) and total annual rainfall (dependent parameter) are presented in this paper as well.