Current Machine Learning Algorithms Research Articles

Optimal neural inference of stimulus intensities

In natural data, the class and intensity of stimuli are correlated. Current machine learning algorithms ignore this ubiquitous statistical property of stimuli, usually by requiring normalized inputs. From a biological perspective, it remains unclear how neural circuits may account for these dependencies in inference and learning. Here, we use a probabilistic framework to model class-specific intensity variations, and we derive approximate inference and online learning rules which reflect common hallmarks of neural computation. Concretely, we show that a neural circuit equipped with specific forms of synaptic and intrinsic plasticity (IP) can learn the class-specific features and intensities of stimuli simultaneously. Our model provides a normative interpretation of IP as a critical part of sensory learning and predicts that neurons can represent nontrivial input statistics in their excitabilities. Computationally, our approach yields improved statistical representations for realistic datasets in the visual and auditory domains. In particular, we demonstrate the utility of the model in estimating the contrastive stress of speech.

CLUSTERING OF DATA WITH MISSING ENTRIES.

The analysis of large datasets is often complicated by the presence of missing entries, mainly because most of the current machine learning algorithms are designed to work with full data. The main focus of this work is to introduce a clustering algorithm, that will provide good clustering even in the presence of missing data. The proposed technique solves an ℓ 0 fusion penalty based optimization problem to recover the clusters. We theoretically analyze the conditions needed for the successful recovery of the clusters. We also propose an algorithm to solve a relaxation of this problem using saturating non-convex fusion penalties. The method is demonstrated on simulated and real datasets, and is observed to perform well in the presence of large fractions of missing entries.

Artificial Intelligence versus Doctors' Intelligence: A Glance on Machine Learning Benefaction in Electrocardiography.

Computational machine learning, especially self-enhancing algorithms, prove remarkable effectiveness in applications, including cardiovascular medicine. This review summarizes and cross-compares the current machine learning algorithms applied to electrocardiogram interpretation. In practice, continuous real-time monitoring of electrocardiograms is still difficult to realize. Furthermore, automated ECG interpretation by implementing specific artificial intelligence algorithms is even more challenging. By collecting large datasets from one individual, computational approaches can assure an efficient personalized treatment strategy, such as a correct prediction on patient-specific disease progression, therapeutic success rate and limitations of certain interventions, thus reducing the hospitalization costs and physicians’ workload. Clearly such aims can be achieved by a perfect symbiosis of a multidisciplinary team involving clinicians, researchers and computer scientists. Summarizing, continuous cross-examination between machine intelligence and human intelligence is a combination of precision, rationale and high-throughput scientific engine integrated into a challenging framework of big data science.

Online Extreme Learning Machine with Hybrid Sampling Strategy for Sequential Imbalanced Data

In real applications of cognitive computation, data with imbalanced classes are used to be collected sequentially. In this situation, some of current machine learning algorithms, e.g., support vector machine, will obtain weak classification performance, especially on minority class. To solve this problem, a new hybrid sampling online extreme learning machine (ELM) on sequential imbalanced data is proposed in this paper. The key idea is keeping the majority and minority classes balanced with similar sequential distribution characteristic of the original data. This method includes two stages. At the offline stage, we introduce the principal curve to build confidence regions of minority and majority classes respectively. Based on these two confidence zones, over-sampling of minority class and under-sampling of majority class are both conducted to generate new synthetic samples, and then, the initial ELM model is established. At the online stage, we first choose the most valuable ones from the synthetic samples of majority class in terms of sample importance. Afterwards, a new online fast leave-one-out cross validation (LOO CV) algorithm utilizing Cholesky decomposition is proposed to determine whether to update the ELM network weight at online stage or not. We also prove theoretically that the proposed method has upper bound of information loss. Experimental results on seven UCI datasets and one real-world air pollutant forecasting dataset show that, compared with ELM, OS-ELM, meta-cognitive OS-ELM, and OSELM with SMOTE strategy, the proposed method can simultaneously improve the classification performance of minority and majority classes in terms of accuracy, G-mean value, and ROC curve. As a conclusion, the proposed hybrid sampling online extreme learning machine can be effectively applied to the sequential data imbalance problem with better generalization performance and numerical stability.

Clustering-based undersampling in class-imbalanced data

Class imbalance is often a problem in various real-world data sets, where one class (i.e. the minority class) contains a small number of data points and the other (i.e. the majority class) contains a large number of data points. It is notably difficult to develop an effective model using current data mining and machine learning algorithms without considering data preprocessing to balance the imbalanced data sets. Random undersampling and oversampling have been used in numerous studies to ensure that the different classes contain the same number of data points. A classifier ensemble (i.e. a structure containing several classifiers) can be trained on several different balanced data sets for later classification purposes. In this paper, we introduce two undersampling strategies in which a clustering technique is used during the data preprocessing step. Specifically, the number of clusters in the majority class is set to be equal to the number of data points in the minority class. The first strategy uses the cluster centers to represent the majority class, whereas the second strategy uses the nearest neighbors of the cluster centers. A further study was conducted to examine the effect on performance of the addition or deletion of 5 to 10 cluster centers in the majority class. The experimental results obtained using 44 small-scale and 2 large-scale data sets revealed that the clustering-based undersampling approach with the second strategy outperformed five state-of-the-art approaches. Specifically, this approach combined with a single multilayer perceptron classifier and C4.5 decision tree classifier ensembles delivered optimal performance over both small- and large-scale data sets.

Future Directions in Machine Learning

Current machine learning algorithms identify statistical regularities in complex data sets and are regularly used across a range of application domains, but they lack the robustness and generalizability associated with human learning. If machine learning techniques could enable computers to learn from fewer examples, transfer knowledge between tasks, and adapt to changing contexts and environments, the results would have very broad scientific and societal impacts. Increased processing and memory resources have enabled larger, more capable learning models, but there is growing recognition that even greater computing resources would not be sufficient to yield algorithms capable of learning from a few examples and generalizing beyond initial training sets. This paper presents perspectives on feature selection, representation schemes and interpretability, transfer learning, continuous learning, and learning and adaptation in time-varying contexts and environments, five key areas that are essential for advancing machine learning capabilities. Appropriate learning tasks that require these capabilities can demonstrate the strengths of novel machine learning approaches that could address these challenges.

양자 기계학습 기술의 현황 및 전망

본고에서는 양자역학 기반의 기계학습인 양자 기계학습의 현황과 전망을 조망하고자 한다. 양자역학 기반의 양자컴퓨팅이 보여준 혁신적인 계산속도 개선에 힘입어 기계학습 분야에 양자컴퓨팅 알고리즘을 적용하는 연구는 빅데이터 시대의 도래에 따라 최근 집중적인 관심을 받고 있다. 고전적인 기계학습 알고리즘들에 양자컴퓨팅을 접목하여 획기적인 속도개선을 가능하게 하는 알고리즘 연구들과 최초의 상용 양자컴퓨터로 화제가 되고 있는 양자 담금질 알고리즘 등을 중심으로 양자 기계학습의 최신동향과 가능성을 살펴보고자 한다.

Identifying oncogenes as features for clinical cancer prognosis by Bayesian nonparametric variable selection algorithm

In clinical research, DNA microarrays are widely applied in the identification of the oncogenes, which are differentially expressed between two clinical states and considered as predictors for the cancer prognosis. Due to the heterogeneity of clinical samples, the differentially expressed genes (DEGs) discovered by current statistical methods or machine learning algorithms involve a number of genes unrelated to the phenotypic differences between the compared samples and, consequently, will impact on the reliability of the predictive models in the cancer prognosis. In our study, we proposed Bayesian nonparametric variable selection algorithm, a stochastic random and hierarchical search method, to separate out the cancer-related genes from the DEG lists. The importance of the genes in the DEG lists can be inferred from the posterior distribution of the predicted clinical endpoints, which can be simulated by the Markov Chain Monte Carlo (MCMC) algorithm. The cancer-related genes were identified according to their importance and used to construct models for the prediction of three clinical endpoints, namely the estrogen receptor status (ER status) of the breast cancer patient, the preoperative treatment response of breast cancer and the overall survival milestone outcome of acute myeloma leukemia (OS of AML). The prediction accuracies of preoperative treatment response, ER status and OS of AML were 86%, 89% and 58%, and the Mathew’s correlation coefficients were 0.42, 0.77 and 0.33, which were higher than those reported in previous studies. Furthermore, most of the genes identified by our method were reported as oncogenes in previous literatures. Our results demonstrated that the Bayesian nonparametric variable selection algorithm proposed in current study can efficiently identify the oncogenes for cancer prognosis and enhance the performance of the predictive models.

Generic Feature Learning in Computer Vision

Current Machine learning algorithms are highly dependent on manually designing features and the Performance of such algo- rithms predominantly depend on how good our representations are. Manually we might never be able to produce best and diverse set of features that closely describe all the variations that occur in our data. Understanding this, vision community is moving towards learning the optimum features itself instead of learning from the features. Traditional hand engineered features lack in generalizing well to other domains/Problems, are time consuming, expensive, requires expert knowledge on the problem domain and doesn’t facilitate learning from previous learnings/Representations(Transfer learning). All these issues are resolved in learning deep representations. Since 2006 a wide range of representation learning algorithms has been proposed but by the recent success and breakthroughs of few deep learning models, the representation learning algorithms have gained the spotlight. This paper aims to give short overview of deep learning approaches available for vision tasks. We also discuss their applicability (With respect to their properties) in vision field.

Big-data analytics: a critical review and some future directions

The aim of the paper is to present a critical review of analytics and visualisation technology for big data, and propose future directions to overcome the shortcomings of the current technologies. The current machine learning and data-mining algorithms are operating mostly on predefined scales of aggregation, while in the vast amounts of data the problem arises at the level of aggregation which cannot be defined ahead of time. We therefore identify a novel and extended architecture to operate on flexible multi-resolution hypothesis space. With such architecture framework the goal is to open a space of possibly discovered models towards classes of data, which are by today's approaches discovered only for special cases. Furthermore, the multi-resolution approach to big-data analytics could allow scenarios like semi-supervised and unsupervised anomaly detection, detecting complex relationships from the heterogeneous data sources, and providing ground for visualisation of complex processes.

A Cascade-Classifier Approach for Fall Detection

The current machine learning algorithms in fall detection, especially those that use a sliding window, have a high computational cost because they need to compute the features from almost all samples. This computation causes energy drain and means that the associated wearable devices re- quire frequent recharging, making them less usable. This study proposes a cascade approach that reduces the computational cost of the fall detection classifier. To examine this approach, accelerometer data from 48 subjects performing a combination of falls and ordinary behaviour is used to train 3 types of classifier (J48 Decision Tree, Logistic Regression, and Multilayer Perceptron). The results show that the cascade approach significantly reduces the computational cost both for learning the classifier and executing it once learnt. Furthermore, the Multilayer Perceptron achieves the highest performance with precision of 93.5%, recall of 94.2%, and f-measure of 93.5%.

A survey on learning from data streams: current and future trends

Nowadays, there are applications in which the data are modeled best not as persistent tables, but rather as transient data streams. In this article, we discuss the limitations of current machine learning and data mining algorithms. We discuss the fundamental issues in learning in dynamic environments like continuously maintain learning models that evolve over time, learning and forgetting, concept drift and change detection. Data streams produce a huge amount of data that introduce new constraints in the design of learning algorithms: limited computational resources in terms of memory, cpu power, and communication bandwidth. We present some illustrative algorithms, designed to taking these constrains into account, for decision-tree learning, hierarchical clustering and frequent pattern mining. We identify the main issues and current challenges that emerge in learning from data streams that open research lines for further developments.

MHC-I prediction using a combination of T cell epitopes and MHC-I binding peptides

We propose a novel learning method that combines multiple experimental modalities to improve the MHC Class-I binding prediction. Multiple experimental modalities are often accessible in the context of a binding problem. Such modalities can provide different labels of data, such as binary classifications, affinity measurements, or direct estimations of the binding profile. Current machine learning algorithms usually focus on a given label type. We here present a novel Multi-Label Vector Optimization (MLVO) formalism to produce classifiers based on the simultaneous optimization of multiple labels. Within this methodology, all label types are combined into a single constrained quadratic dual optimization problem. We apply the MLVO to MHC class-I epitope prediction. We combine affinity measurements (IC50/EC50), binary classifications of epitopes as T cell activators and existing algorithms. The multi-label vector optimization algorithms produce classifiers significantly better than the ones resulting from any of its components. These matrix based classifiers are better or equivalent to the existing state of the art MHC-I epitope prediction tools in the studied alleles.

Active Learning: Learning a Motor Skill Without a Coach

When we learn a new skill (e.g., golf) without a coach, we are "active learners": we have to choose the specific components of the task on which to train (e.g., iron, driver, putter, etc.). What guides our selection of the training sequence? How do choices that people make compare with choices made by machine learning algorithms that attempt to optimize performance? We asked subjects to learn the novel dynamics of a robotic tool while moving it in four directions. They were instructed to choose their practice directions to maximize their performance in subsequent tests. We found that their choices were strongly influenced by motor errors: subjects tended to immediately repeat an action if that action had produced a large error. This strategy was correlated with better performance on test trials. However, even when participants performed perfectly on a movement, they did not avoid repeating that movement. The probability of repeating an action did not drop below chance even when no errors were observed. This behavior led to suboptimal performance. It also violated a strong prediction of current machine learning algorithms, which solve the active learning problem by choosing a training sequence that will maximally reduce the learner's uncertainty about the task. While we show that these algorithms do not provide an adequate description of human behavior, our results suggest ways to improve human motor learning by helping people choose an optimal training sequence.

A Tabu Clustering algorithm for Intrusion Detection

Traditional methods of intrusion detection lack the extensibility in face of changing network configurations and the adaptability in face of unknown intrusion types. Meanwhile, current machine-learning algorithms for intrusion detection need labeled data to be trained, so they are expensive in computation and sometimes misled by artificial data. In order to solve these problems, a new detection algorithm is proposed in this paper, the Intrusion Detection Based on Tabu Clustering (IDBTC) algorithm. It can automatically set up clusters and detect intrusions by labeling normal and abnormal groups. Computer simulations show that this algorithm is effective for intrusion detection.

A genetic clustering method for intrusion detection

Traditional intrusion detection methods lack extensibility in face of changing network configurations as well as adaptability in face of unknown attack types. Meanwhile, current machine-learning algorithms need labeled data for training first, so they are computational expensive and sometimes misled by artificial data. In this paper, a new detection algorithm, the Intrusion Detection Based on Genetic Clustering (IDBGC) algorithm, is proposed. It can automatically establish clusters and detect intruders by labeling normal and abnormal groups. Computer simulations show that this algorithm is effective for intrusion detection.

Overcoming the Myopia of Inductive Learning Algorithms with RELIEFF

Current inductive machine learning algorithms typically use greedy search with limited lookahead. This prevents them to detect significant conditional dependencies between the attributes that descr...