Minimal Learning Machine Research Articles

Mechanical properties prediction in rebar using kernel-based regression models

ABSTRACT A successful application of neural networks to the prediction of four important mechanical properties of steel rebar used in civil construction has been reported recently. In the current work, we advanced further in this issue by evaluating the performances of three kernel-based regression models, namely, the minimal learning machine (MLM), the support vector regression (SVR), and the least-squares SVR (LSSVR) in the estimation of the yield strength (YS), ultimate tensile strength (UTS), UTS/YS ratio, and percent elongation (PE) from chemical composition and parameters used during hot rolling and heat treatment. The achieved results indicate that the LSSVR model consistently outperforms the SVR and MLM models for all four properties studied.

UPDATING STRATEGIES FOR DISTANCE BASED CLASSIFICATION MODEL WITH RECURSIVE LEAST SQUARES

Abstract. The idea is to create a self-learning Minimal Learning Machine (MLM) model that is computationally efficient, easy to implement and performs with high accuracy. The study has two hypotheses. Experiment A examines the possibilities of introducing new classes with Recursive Least Squares (RLS) updates for the pre-trained self learning-MLM model. The idea of experiment B is to simulate the push broom spectral imagers working principles, update and test the model based on a stream of pixel spectrum lines on a continuous scanning process. Experiment B aims to train the model with a significantly small amount of labelled reference points and update it continuously with (RLS) to reach maximum classification accuracy quickly.The results show that the new self-learning MLM method can classify new classes with RLS update but with a cost of decreasing accuracy. With a larger amount of reference points, one class can be introduced with reasonable accuracy. The results of experiment B indicate that self-learning MLM can be trained with a few reference points, and the self-learning model quickly reaches accuracy results comparable with nearest-neighbour NN-MLM. It seems that the self-learning MLM could be a comparable machine learning method for the application of hyperspectral imaging and remote sensing.

PIECEWISE ANOMALY DETECTION USING MINIMAL LEARNING MACHINE FOR HYPERSPECTRAL IMAGES

Abstract. Hyperspectral imaging, with its applications, offers promising tools for remote sensing and Earth observation. Recent development has increased the quality of the sensors. At the same time, the prices of the sensors are lowering. Anomaly detection is one of the popular remote sensing applications, which benefits from real-time solutions. A real-time solution has its limitations, for example, due to a large amount of hyperspectral data, platform’s (drones or a cube satellite) constraints on payload and processing capability. Other examples are the limitations of available energy and the complexity of the machine learning models. When anomalies are detected in real-time from the hyperspectral images, one crucial factor is to utilise a computationally efficient method. The Minimal Learning Machine is a distance-based classification algorithm, which can be modified for anomaly detection. Earlier studies confirms that the Minimal learning Machine (MLM) is capable of detecting efficiently global anomalies from the hyperspectral images with a false alarm rate of zero. In this study, we will show that by using a carefully selected lower threshold besides the higher threshold of the variance, it is possible to detect local and global anomalies with the MLM. The downside is that the improved method is highly sensitive with the respect to the noise. Thus, the second aim of this study is to improve the MLM’s robustness with respect to noise by introducing a novel approach, the piecewise MLM. With the new approach, the piecewise MLM can detect global and local anomalies, and the method is significantly more robust with respect to noise than the MLM. As a result, we have an interesting, easy to implement and computationally light method which is suitable for remote sensing applications.

Do Randomized Algorithms Improve the Efficiency of Minimal Learning Machine?

Minimal Learning Machine (MLM) is a recently popularized supervised learning method, which is composed of distance-regression and multilateration steps. The computational complexity of MLM is dominated by the solution of an ordinary least-squares problem. Several different solvers can be applied to the resulting linear problem. In this paper, a thorough comparison of possible and recently proposed, especially randomized, algorithms is carried out for this problem with a representative set of regression datasets. In addition, we compare MLM with shallow and deep feedforward neural network models and study the effects of the number of observations and the number of features with a special dataset. To our knowledge, this is the first time that both scalability and accuracy of such a distance-regression model are being compared to this extent. We expect our results to be useful on shedding light on the capabilities of MLM and in assessing what solution algorithms can improve the efficiency of MLM. We conclude that (i) randomized solvers are an attractive option when the computing time or resources are limited and (ii) MLM can be used as an out-of-the-box tool especially for high-dimensional problems.

MINIMAL LEARNING MACHINE IN ANOMALY DETECTION FROM HYPERSPECTRAL IMAGES

Abstract. Anomaly detection from hyperspectral data needs computationally efficient methods to process the data when the data gathering platform is a drone or a cube satellite. In this study, we introduce a minimal learning machine for hyperspectral anomaly detection. Minimal learning machine is a novel distance-based classification algorithm, which is now modified to detect anomalies. Besides being computationally efficient, minimal learning machine is also easy to implement. Based on the results, we show that minimal learning machine is efficient in detecting global anomalies from the hyperspectral data with low false alarm rate.

Parsimonious Minimal Learning Machine via Multiresponse Sparse Regression.

The training procedure of the minimal learning machine (MLM) requires the selection of two sets of patterns from the training dataset. These sets are called input reference points (IRP) and output reference points (ORP), which are used to build a mapping between the input geometric configurations and their corresponding outputs. In the original MLM, the number of input reference points is the hyper-parameter and the patterns are chosen at random. Therefore, the conventional proposal does not consider which patterns will belong to each reference point group, since the model does not implement an appropriate way of selecting the most suitable patterns as reference points. Such an approach can impact on the decision function in terms of smoothness, resulting in high complexity models. This paper introduces a new approach to select IRP for MLM applied to classification tasks. The optimally selected minimal learning machine (OS-MLM) relies on the multiresponse sparse regression (MRSR) ranking method and the leave-one-out (LOO) criterion to sort the patterns in terms of relevance and select an appropriate number of input reference points, respectively. The experimental assessment conducted on UCI datasets reports the proposal was able to produce sparser models and achieve competitive performance when compared to the regular strategy of selecting MLM input RPs.

A new perspective for Minimal Learning Machines: A lightweight approach

This paper introduces a new procedure to train Minimal Learning Machines (MLM) for regression tasks. Besides that, we propose a new prediction process in MLM. A well-known drawback concerning the (original) MLM model formulation is the lack of sparseness.The most recent efforts on this problem strongly rely on the selection of reference points before training and prediction steps in MLM, all based on some supposition regarding the data. In the opposite direction, here, we explore another formulation of MLM in which we do not rely on any assumption regarding the data for prior selection. Instead, our proposal, named Lightweight Minimal Learning Machine (LW-MLM), builds a regularized system that imposes sparseness. We thrive in such a sparse criterion, not by selection but instead using a piece of weighted information into the model. We validate the contributions of this paper through four types of experiments to evaluate different aspects of our proposal: the prediction error performance, the goodness-of-fit of estimated vs. measured values, the norm values which are related to the sparsity, and finally, the prediction error in high dimensional settings. Based on the results, we show that LW-MLM is a valid alternative since achieved similar or higher accuracy rates against other variants being all seen as statistically equivalent.

Extreme minimal learning machine: Ridge regression with distance-based basis

The extreme learning machine (ELM) and the minimal learning machine (MLM) are nonlinear and scalable machine learning techniques with a randomly generated basis. Both techniques start with a step in which a matrix of weights for the linear combination of the basis is recovered. In the MLM, the feature mapping in this step corresponds to distance calculations between the training data and a set of reference points, whereas in the ELM, a transformation using a radial or sigmoidal activation function is commonly used. Computation of the model output, for prediction or classification purposes, is straightforward with the ELM after the first step. In the original MLM, one needs to solve an additional multilateration problem for the estimation of the distance-regression based output. A natural combination of these two techniques is proposed and experimented here: to use the distance-based basis characteristic in the MLM in the learning framework of the regularized ELM. In other words, we conduct ridge regression using a distance-based basis. The experimental results characterize the basic features of the proposed technique and surprisingly, indicate that overlearning with the distance-based basis is in practice avoided in classification problems. This makes the model selection for the proposed method trivial, at the expense of computational costs.

Automatic classification of pulmonary diseases using a structural co-occurrence matrix

The World Health Organization (WHO) estimates that 300 million people have asthma and that this disease causes about 250 thousand deaths per year worldwide. The number of deaths from chronic obstructive pulmonary disease increased by 30% in 2015, and this disease will, according to WHO, be the third major cause of death worldwide in 2030. The identification of diseases using medical image processing techniques is in high demand to assist medical doctors to make more accurate diagnoses. However, although these techniques contribute in making medical diagnoses, most of them still need to have some parameters set and this can be a difficult and tedious process. In this paper, a new automatic approach to identify and classify lung diseases from a structural co-occurrence matrix (SCM) in chest computed tomography images is proposed. The most important novelty of this approach is that only the image is used as the input data and extract the structural information of the disease which, in this case, is related to the lower frequencies. In order to demonstrate the efficiency of the proposed technique, it was compared with other well-known state-of-art feature extractors. In addition, the SCM was evaluated with four filters (Gaussian, Fourier, Laplace and Sobel) using linear discriminant analysis, multi-layer perceptron, support vector machines and minimal learning machine classifiers. The results showed that the SCM, when using low frequencies, is able to adapt to different images and extract the most significant structural data, without the need of any additional parameters, yet maintaining the diagnostic precision.

Neurologist-level classification of stroke using a Structural Co-Occurrence Matrix based on the frequency domain

This paper presents a new approach to automatically classify hemorrhagic and ischemic strokes using the Structural Co-Occurrence Matrix (SCM) extracted from the main frequencies of computed tomography images of the brain. The main advantage of this approach is that it only uses the image as a parameter. Specificity, sensitivity, positive predictive value, F-Score, harmonic mean, and accuracy were used as metrics to evaluate the efficiency of the SCM. The SCM was compared with local binary patterns, gray-level co-occurrence matrices, invariant moments of Hu and with the feature extraction method based on human tissue density patterns named Analysis of Human Tissue Densities. Multiple machine learning classifiers were used including support vector machine, multilayer perceptron, minimal learning machine and the linear discriminant analysis. The results show that the SCM in the frequency domain can extract the most discriminant structural information of strokes automatically, obtaining good results without the needed of additional parameters.

Fast Co-MLM: An Efficient Semi-supervised Co-training Method Based on the Minimal Learning Machine

Co-training is a framework for semi-supervised learning that has attracted much attention due to its good performance and easy adaptation for various learning algorithms. In a recent work, Caldas et al. proposed a co-training-based method using the recently proposed supervised learning method named minimal learning machine (MLM). Although the proposed method, referred to as Co-MLM, presented results that are comparable to other semi-supervised algorithms, using MLM as a base learner resulted in a formulation with heavy computational cost. Aiming to mitigate this problem, in this paper, we propose an improved variant of Co-MLM with reduced computational cost on both training and testing phases. The proposed method is compared to Co-MLM and other Co-training-based semi-supervised methods, presenting comparable performances.

Euclidean distance estimation in incomplete datasets

This paper proposes a method to estimate the expected value of the Euclidean distance between two possibly incomplete feature vectors. Under the Missing at Random assumption, we show that the Euclidean distance can be modeled by a Nakagami distribution, for which the parameters we express as a function of the moments of the unknown data distribution. In our formulation the data distribution is modeled using a mixture of Gaussians. The proposed method, named Expected Euclidean Distance (EED), is validated through a series of experiments using synthetic and real-world data. Additionally, we show the application of EED to the Minimal Learning Machine (MLM), a distance-based supervised learning method. Experimental results show that EED outperforms existing methods that estimate Euclidean distances in an indirect manner. We also observe that the application of EED to the MLM provides promising results.

Ensemble of Efficient Minimal Learning Machines for Classification and Regression

Minimal Learning Machine (MLM) is a recently proposed supervised learning algorithm with performance comparable to most state-of-the-art machine learning methods. In this work, we propose ensemble methods for classification and regression using MLMs. The goal of ensemble strategies is to produce more robust and accurate models when compared to a single classifier or regression model. Despite its successful application, MLM employs a computationally intensive optimization problem as part of its test procedure (out-of-sample data estimation). This becomes even more noticeable in the context of ensemble learning, where multiple models are used. Aiming to provide fast alternatives to the standard MLM, we also propose the Nearest Neighbor Minimal Learning Machine and the Cubic Equation Minimal Learning Machine to cope with classification and single-output regression problems, respectively. The experimental assessment conducted on real-world datasets reports that ensemble of fast MLMs perform comparably or superiorly to reference machine learning algorithms.

A novel mobile robot localization approach based on topological maps using classification with reject option in omnidirectional images

Mobile robot localization, which allows a robot to identify its position, is one of main challenges in the field of Robotics. In this work, we provide an evaluation of consolidated feature extractions and machine learning techniques from omnidirectional images focusing on topological map and localization tasks. The main contributions of this work are a novel method for localization via classification with reject option using omnidirectional images, as well as two novel omnidirectional image data sets. The localization system was analyzed in both virtual and real environments. Based on the experiments performed, the Minimal Learning Machine with Nearest Neighbors classifier and Local Binary Patterns feature extraction proved to be the best combination for mobile robot localization with accuracy of 96.7% and an Fscore of 96.6%.

Minimal Learning Machine: A novel supervised distance-based approach for regression and classification

In this work, a novel supervised learning method, the Minimal Learning Machine (MLM), is proposed. Learning in MLM consists in building a linear mapping between input and output distance matrices. In the generalization phase, the learned distance map is used to provide an estimate of the distance from K output reference points to the unknown target output value. Then, the output estimation is formulated as multilateration problem based on the predicted output distance and the locations of the reference points. Given its general formulation, the Minimal Learning Machine is inherently capable of operating on nonlinear regression problems as well as on multidimensional response spaces. In addition, an intuitive extension of the MLM is proposed to deal with classification problems. A comprehensive set of computer experiments illustrates that the proposed method achieves accuracies that are comparable to more traditional machine learning methods for regression and classification thus offering a computationally valid alternative to such approaches.