Effective Machine Learning Algorithm Research Articles

FEMa: a finite element machine for fast learning

Machine learning has played an essential role in the past decades and has been in lockstep with the main advances in computer technology. Given the massive amount of data generated daily, there is a need for even faster and more effective machine learning algorithms that can provide updated models for real-time applications and on-demand tools. This paper presents FEMa—a finite element machine classifier—for supervised learning problems, where each training sample is the center of a basis function, and the whole training set is modeled as a probabilistic manifold for classification purposes. FEMa has its theoretical basis in the finite element method, which is widely used for numeral analysis in engineering problems. It is shown FEMa is parameterless and has a quadratic complexity for both training and classification phases when basis functions are used that satisfy certain properties. The proposed classifier yields very competitive results when compared to some state-of-the-art supervised pattern recognition techniques.

Multi-temporal RapidEye Tasselled Cap data for land cover classification

ABSTRACT Land cover mapping can be seen as a key element to understand the spatial distribution of habitats and thus to sustainable management of natural resources. Multi-temporal remote sensing data are a valuable data source for land cover mapping. However, the increased amount of data requires effective machine learning algorithms and data compression approaches. In this study, the Random Forest and C 5.0 classification algorithms were applied to (1) a multi-temporal Tasselled-Cap-transformed, (2) top of atmosphere and (3) surface reflectance RapidEye time-series. The overall accuracies ranged from 91.44% to 91.80%, with only minor differences between algorithms and datasets. The McNemar test showed, however, significant differences between the Tasselled-Cap-transformed and untransformed mapping results in most cases. The temporal profiles for the Tasselled-Cap-transformed RapidEye data indicated a good separability between considered classes. The phenological profiles of vegetated surfaces followed a typical green-up curve for the Greenness Tasselled-Cap-index. A permutation-based variable importance measure indicated that late autumn should be considered as most important phenological phase contributing to the classification model performance. The results suggested that the RapidEye Tasselled Cap Transformation, which was designed for agricultural applications, can be an effective data compression tool, suitable to map heterogeneous landscapes with no measurable negative impact on classification accuracy.

Automatic Detection of Gender Identity: The Phenomenon of Russian Women's Prose

The article deals with the method of automatic detection of authors ' gender identity on the material of fiction prose of 1980-2000. During this period, there is a special construct, called women's prose, which is characterized by a special genre and stylistic originality. We set ourselves the task to find out whether the concept of “women's prose” refers only to the non-text reality or is clearly reflected at the level of language. We have collected corpus of texts 1980-2000 and conducted that identified the most effective machine learning algorithms for the classification of male and female prose.

Break-out Detection for High-speed Small Hole Drilling EDM Based on Machine Learning

High-speed small hole drilling EDM is of the advantages of no cutting force, high machining accuracy and efficiency, therefore, it is very suitable for drilling multiple film cooling holes with variable tilting angles on the turbine blades which is made of high-temperature alloy. When machining through holes, however, the break-outs need to be accurately and timely detected in order to adjust the machining strategy to assure the quality of the drilled holes and prevent the back strike which may result in machining failure, hence the online break-out detection becomes very essential for the process itself, especially in a fully automated operation.This paper proposes a new method for break-out detection, which involves support vector machine (SVM) method, one of the effective machine learning algorithms. The inputs of the model include discharge pulse duration, pulse interval, peak current, effective discharge frequency and actual electrode feed rate, etc., and the output is whether the event of break-out is happening or not. The algorithm is seamlessly integrated in a newly developed computer numerical control (CNC) system, which is dedicated for High-speed small hole drilling EDM, with a sampling circuit collecting real-time current signals of electrical discharges.A series of experiments were carried out using the proposed method. Raw data were first acquired by machining through holes, then preprocessed offline to recognize effective discharge pulses from the pulse trains. The results along with the discharge parameters were used as the training data to build the SVM model for the subsequent online detection. Finally, the effectiveness of the proposed method was verified by drilling holes on workpieces with different thicknesses. The results show that the proposed method can detect the break-outs very effectively, with the correctness nearly 100% among 200 tests and the decision cycle time less than 100ms in current experimental conditions.

Adaptive forgetting factor OS-ELM and bootstrap for time series prediction

Online sequential extreme learning machine (OS-ELM) for single-hidden layer feedforward networks (SLFNs) is an effective machine learning algorithm. But OS-ELM has some underlying weaknesses of neglecting time series timeliness and being incapable to provide quantitative uncertainty for prediction. To overcome these shortcomings, a time series prediction method is presented based on the combination of OS-ELM with adaptive forgetting factor (AFF-OS-ELM) and bootstrap (B-AFF-OS-ELM). Firstly, adaptive forgetting factor is added into OS-ELM for adjusting the effective window length of training data during OS-ELM sequential learning phase. Secondly, the current bootstrap is developed to fit time series prediction online. Then associated with improved bootstrap, the proposed method can compute prediction interval as uncertainty information, meanwhile the improved bootstrap enhances prediction accuracy and stability of AFF-OS-ELM. Performances of B-AFF-OS-ELM are benchmarked with other traditional and improved OS-ELM on simulation and practical time series data. Results indicate the significant performances achieved by B-AFF-OS-ELM.

A novel meta-cognitive fuzzy-neural model with backstepping strategy for adaptive control of uncertain nonlinear systems

A significant increase of system complexity and state changes requires an effective data-driven system identification and machine learning algorithm to deal with the control of nonlinear systems. Using streams of data collected from the system, the data-driven controller aims to stabilize the unknown nonlinear systems with modeling uncertainties and external disturbances. The paper proposes a novel data-driven adaptive control approach with the backstepping strategy for online control of unknown nonlinear systems with no human intervention. A new meta-cognitive fuzzy-neural model is first introduced to construct the unknown system dynamics and utilize the self-adaptive tracking error as the learning parameters to determine the deletion of the state data, adapt the structure and parameters of the controller using the information extracted from nonstationary data streams. Subsequently, the control law is constructed based on the meta-cognitive fuzzy-neural model rather than the actual systems and the backstepping control strategy. Then, the stability analysis of the closed-loop system is presented from the Lyapunov function and shows that the tracking errors converge to zero. In the proposed control scheme, the bound of the control input is considered and ensured via the stable projection-type adaptation laws of the parameters. Moreover, in order to further save online computation time, only the parameters of the rule nearest to the current state are updated while those of other rules maintain unchanged. This is different from the existing studies where the parameters of all rules are updated. Finally, various simulation results from an inverted pendulum system and a thrust active magnetic bearing system demonstrate the superior performance of the proposed meta-cognitive fuzzy-neural control approach.

Combatting nonlinear phase noise in coherent optical systems with an optimized decision processor based on machine learning

An effective machine learning algorithm, the support vector machine (SVM), is presented in the context of a coherent optical transmission system. As a classifier, the SVM can create nonlinear decision boundaries to mitigate the distortions caused by nonlinear phase noise (NLPN). Without any prior information or heuristic assumptions, the SVM can learn and capture the link properties from only a few training data. Compared with the maximum likelihood estimation (MLE) algorithm, a lower bit-error rate (BER) is achieved by the SVM for a given launch power; moreover, the launch power dynamic range (LPDR) is increased by 3.3dBm for 8 phase-shift keying (8PSK), 1.2dBm for QPSK, and 0.3dBm for BPSK. The maximum transmission distance corresponding to a BER of 1×10−3 is increased by 480km for the case of 8PSK. The larger launch power range and longer transmission distance improve the tolerance to amplitude and phase noise, which demonstrates the feasibility of the SVM in digital signal processing for M-PSK formats. Meanwhile, in order to apply the SVM method to 16 quadratic amplitude modulation (16QAM) detection, we propose a parameter optimization scheme. By utilizing a cross-validation and grid-search techniques, the optimal parameters of SVM can be selected, thus leading to the LPDR improvement by 2.8dBm. Additionally, we demonstrate that the SVM is also effective in combating the laser phase noise combined with the inphase and quadrature (I/Q) modulator imperfections, but the improvement is insignificant for the linear noise and separate I/Q imbalance. The computational complexity of SVM is also discussed. The relatively low complexity makes it possible for SVM to implement the real-time processing.

Newsvendor Problems with Demand Shocks and Unknown Demand Distributions

ABSTRACTIn today's competitive market, demand volume and even the underlying demand distribution can change quickly for a newsvendor seller. We refer to sudden changes in demand distribution as demand shocks. When a newsvendor seller has limited demand distribution information and also experiences underlying demand shocks, the majority of existing methods for newsvendor problems may not work well since they either require demand distribution information or assume stationary demand distribution. We present a new, robust, and effective machine learning algorithm for newsvendor problems with demand shocks but without any demand distribution information. The algorithm needs only an approximate estimate of the lower and upper bounds of demand range; no other knowledge such as demand mean, variance, or distribution type is necessary. We establish the theoretical bounds that determine this machine learning algorithm's performance in handling demand shocks. Computational experiments show that this algorithm outperforms the traditional approaches in a variety of situations including large and frequent shocks of the demand mean. The method can also be used as a meta‐algorithm by incorporating other traditional approaches as experts. Working together, the original algorithm and the extended meta‐algorithm can help manufacturers and retailers better adapt their production and inventory control decisions in dynamic environments where demand information is limited and demand shocks are frequent

Asymmetric author-topic model for knowledge discovering of big data in toxicogenomics.

The advancement of high-throughput screening technologies facilitates the generation of massive amount of biological data, a big data phenomena in biomedical science. Yet, researchers still heavily rely on keyword search and/or literature review to navigate the databases and analyses are often done in rather small-scale. As a result, the rich information of a database has not been fully utilized, particularly for the information embedded in the interactive nature between data points that are largely ignored and buried. For the past 10 years, probabilistic topic modeling has been recognized as an effective machine learning algorithm to annotate the hidden thematic structure of massive collection of documents. The analogy between text corpus and large-scale genomic data enables the application of text mining tools, like probabilistic topic models, to explore hidden patterns of genomic data and to the extension of altered biological functions. In this paper, we developed a generalized probabilistic topic model to analyze a toxicogenomics dataset that consists of a large number of gene expression data from the rat livers treated with drugs in multiple dose and time-points. We discovered the hidden patterns in gene expression associated with the effect of doses and time-points of treatment. Finally, we illustrated the ability of our model to identify the evidence of potential reduction of animal use.

Monthly streamflow forecasting using Gaussian Process Regression

Streamflow forecasting plays a critical role in nearly all aspects of water resources planning and management. In this work, Gaussian Process Regression (GPR), an effective kernel-based machine learning algorithm, is applied to probabilistic streamflow forecasting. GPR is built on Gaussian process, which is a stochastic process that generalizes multivariate Gaussian distribution to infinite-dimensional space such that distributions over function values can be defined. The GPR algorithm provides a tractable and flexible hierarchical Bayesian framework for inferring the posterior distribution of streamflows. The prediction skill of the algorithm is tested for one-month-ahead prediction using the MOPEX database, which includes long-term hydrometeorological time series collected from 438 basins across the U.S. from 1948 to 2003. Comparisons with linear regression and artificial neural network models indicate that GPR outperforms both regression methods in most cases. The GPR prediction of MOPEX basins is further examined using the Budyko framework, which helps to reveal the close relationships among water-energy partitions, hydrologic similarity, and predictability. Flow regime modification and the resulting loss of predictability have been a major concern in recent years because of climate change and anthropogenic activities. The persistence of streamflow predictability is thus examined by extending the original MOPEX data records to 2012. Results indicate relatively strong persistence of streamflow predictability in the extended period, although the low-predictability basins tend to show more variations. Because many low-predictability basins are located in regions experiencing fast growth of human activities, the significance of sustainable development and water resources management can be even greater for those regions.

Medical Image Segmentation using an Extended Active Shape Model

In this paper, an approach for improving active shape segmentation of medical images using machine learning techniques which can achieve high segmentation accuracy is described. A statistical shape model is created from a training dataset which is used to search for an object of interest in an image. Active shape model has shown over time to be a reliable image segmentation methodology, but its segmentation accuracy is hindered especially by poor initialization which can‟t be guaranteed to always be perfect. In our methodology, we extract features for each landmark using haarfilters[9]. We train a classifier with these features and use the classifier to classify points around the final points of an Active shape model search. The aim of this approach is to better place points that might have been wrongly placed from the ASM search. We have used the simple, yet effective K-Nearest Neighbour machine learning algorithm, and have demonstrated the ability of this method to improve segmentation accuracy by segmenting 2d images of the lateral ventricles of the brain. General Terms

Active multiple kernel learning for interactive 3D object retrieval systems

An effective relevance feedback solution plays a key role in interactive intelligent 3D object retrieval systems. In this work, we investigate the relevance feedback problem for interactive intelligent 3D object retrieval, with the focus on studying effective machine learning algorithms for improving the user's interaction in the retrieval task. One of the key challenges is to learn appropriate kernel similarity measure between 3D objects through the relevance feedback interaction with users. We address this challenge by presenting a novel framework of Active multiple kernel learning (AMKL), which exploits multiple kernel learning techniques for relevance feedback in interactive 3D object retrieval. The proposed framework aims to efficiently identify an optimal combination of multiple kernels by asking the users to label the most informative 3D images. We evaluate the proposed techniques on a dataset of over 10,000 3D models collected from the World Wide Web. Our experimental results show that the proposed AMKL technique is significantly more effective for 3D object retrieval than the regular relevance feedback techniques widely used in interactive content-based image retrieval, and thus is promising for enhancing user's interaction in such interactive intelligent retrieval systems.

Classification study of novel piperazines as antagonists for the melanocortin-4 receptor based on least-squares support vector machines

The least-squares support vector machine (LS-SVM), as an effective machine learning algorithm, was used to develop a nonlinear binary classification model of novel piperazines-bis- piperazines as antagonists for the melanocortin-4 (MC4) receptor based on their activity. Each compound was represented by calculated structural descriptors that encode constitutional, topological, geometrical, electrostatic, quantum-chemical features. Five descriptors selected by forward stepwise linear discriminant analysis (LDA) were used as inputs of the LS-SVM model. The nonlinear model developed from LS-SVM algorithm (with prediction accuracy of 95% on the test set) outperformed LDA (test accuracy of 90%). The proposed method is very useful for chemists to screen antagonists for the MC4 receptor.