Ensemble Learning System Research Articles

Relevance vector machine based infinite decision agent ensemble learning for credit risk analysis

In this paper, a relevance vector machine based infinite decision agent ensemble learning (RVM Ideal ) system is proposed for the robust credit risk analysis. In the first level of our model, we adopt soft margin boosting to overcome overfitting. In the second level, the RVM algorithm is revised for boosting so that different RVM agents can be generated from the updated instance space of the data. In the third level, the perceptron Kernel is employed in RVM to simulate infinite subagents. Our system RVM Ideal also shares some good properties, such as good generalization performance, immunity to overfitting and predicting the distance to default. According to the experimental results, our proposed system can achieve better performance in term of sensitivity, specificity and overall accuracy.

Building an ensemble system for diagnosing masses in mammograms

Classification of a suspicious mass (region of interest, ROI) in a mammogram as malignant or benign may be achieved using mass shape features. An ensemble system was built for this purpose and tested. Multiple contours were generated from a single ROI using various parameter settings of the image enhancement functions for the segmentation. For each segmented contour, the mass shape features were computed. For classification, the dataset was partitioned into four subsets based on the patient age (young/old) and the ROI size (large/small). We built an ensemble learning system consisting of four single classifiers, where each classifier is a specialist, trained specifically for one of the subsets. Those specialist classifiers are also an optimal classifier for the subset, selected from several candidate classifiers through preliminary experiment. In this scheme, the final diagnosis (malignant or benign) of an instance is the classification produced by the classifier trained for the subset to which the instance belongs. The Digital Database for Screening Mammography (DDSM) from the University of South Florida was used to test the ensemble system for classification of masses, which achieved a 72% overall accuracy. This ensemble of specialist classifiers achieved better performance than single classification (56%). An ensemble classifier for mammography-detected masses may provide superior performance to any single classifier in distinguishing benign from malignant cases.

RAMOBoost: Ranked Minority Oversampling in Boosting

In recent years, learning from imbalanced data has attracted growing attention from both academia and industry due to the explosive growth of applications that use and produce imbalanced data. However, because of the complex characteristics of imbalanced data, many real-world solutions struggle to provide robust efficiency in learning-based applications. In an effort to address this problem, this paper presents Ranked Minority Oversampling in Boosting (RAMOBoost), which is a RAMO technique based on the idea of adaptive synthetic data generation in an ensemble learning system. Briefly, RAMOBoost adaptively ranks minority class instances at each learning iteration according to a sampling probability distribution that is based on the underlying data distribution, and can adaptively shift the decision boundary toward difficult-to-learn minority and majority class instances by using a hypothesis assessment procedure. Simulation analysis on 19 real-world datasets assessed over various metrics-including overall accuracy, precision, recall, F-measure, G-mean, and receiver operation characteristic analysis-is used to illustrate the effectiveness of this method.

Financial information processing and development of emerging financial markets

With the rapid development and globalization of financial markets (especially emerging financial markets), financial information processing has become a hot research area due to its immense practical applications. Such applications include stock market analysis, foreign exchange rate forecasting, option pricing, bank failure prediction, financial risk management, credit rating and scoring, bank loan management, customer relationship management, and antimoney laundering. Accordingly, there has been an increasing demand in using financial information processing techniques for many core financial tasks. Nevertheless, as a new cross-disciplinary field, the existing financial information processing methods are far from practical for scenarios in the global financial market; it is currently not clear how the information processing techniques, which are rapidly emerging, can be used to improve the quality of financial information processing. Therefore, it is necessary to conduct a thorough investigation of the financial information processing problems and understand its fundamental theoretical difficulties. At the same time, investigating the formulation mechanism and generic operations law for all kinds of financial markets is necessary for deep information processing. For this purpose, this special section presents new progress in financial information processing that is expected to trigger thought and deepen future research. In this section, a two-round peer review process was performed. In the first-round of the review process, 12 papers were selected from 16 invited talks and 46 submissions related to financial information processing from different countries and regions. After making necessary revisions in terms of reviewer’s recommendations, the second-round review process selected nine papers for final publication. The nine papers can be divided into three categories. The first category focuses on financial risk analysis and modeling. The first article applies a hidden Markov model to handle the default risk; it is titled “Modeling default risk via a hidden Markov model of multiple sequences” by Ching et al. The second paper emphasizes customer credit risk classification using support vector machine (SVM) ensemble learning technique, while the third paper proposes an interest force accumulated function model with Gauss process and Poisson process as the basis for the life insurance reserve model and meantime the risk caused by drawing reserve is analyzed in terms of a stochastic interest rates environment. They are “Developing an SVM-based ensemble learning system for customer risk identification collaborating with customer relationship management” by Yu et al., and “A class of life insurance reserve model and risk analysis in a stochastic interest rates environment” by Jia et al., respectively. In the second category, three papers are selected for covering strategic games, financial distress diagnosis and fund performance evaluation. These are named now in respective order. The fourth paper, titled “N-person

Developing an SVM-based ensemble learning system for customer risk identification collaborating with customer relationship management

In this study, we propose a support vector machine (SVM)-based ensemble learning system for customer relationship management (CRM) to help enterprise managers effectively manage customer risks from the risk aversion perspective. This system differs from the classical CRM for retaining and targeting profitable customers; the main focus of the proposed SVM-based ensemble learning system is to identify high-risk customers in CRM for avoiding possible loss. To build an effective SVM-based ensemble learning system, the effects of ensemble members’ diversity, ensemble member selection and different ensemble strategies on the performance of the proposed SVM-based ensemble learning system are each investigated in a practical CRM case. Through experimental analysis, we find that the Bayesian-based SVM ensemble learning system with diverse components and choose from space selection strategy show the best performance over various testing samples.

Support vector machine based multiagent ensemble learning for credit risk evaluation

In this paper, a four-stage support vector machine (SVM) based multiagent ensemble learning approach is proposed for credit risk evaluation. In the first stage, the initial dataset is divided into two independent subsets: training subset (in-sample data) and testing subset (out-of-sample data) for training and verification purposes. In the second stage, different SVM learning paradigms with much dissimilarity are constructed as intelligent agents for credit risk evaluation. In the third stage, multiple individual SVM agents are trained using training subsets and the corresponding evaluation results are also obtained. In the final stage, all individual results produced by multiple SVM agents in the previous stage are aggregated into an ensemble result. In particular, the impact of the diversity of individual intelligent agents on the generalization performance of the SVM-based multiagent ensemble learning system is examined and analyzed. For illustration, one corporate credit card application approval dataset is used to verify the effectiveness of the SVM-based multiagent ensemble learning system.

Ensemble component selection for improving ICA based microarray data prediction models

Independent component analysis (ICA) has been widely used to tackle the microarray dataset classification problem, but there still exists an unsolved problem that the independent component (IC) sets may not be reproducible after different ICA transformations. Inspired by the idea of ensemble feature selection, we design an ICA based ensemble learning system to fully utilize the difference among different IC sets. In this system, some IC sets are generated by different ICA transformations firstly. A multi-objective genetic algorithm (MOGA) is designed to select different biologically significant IC subsets from these IC sets, which are then applied to build base classifiers. Three schemes are used to fuse these base classifiers. The first fusion scheme is to combine all individuals in the final generation of the MOGA. In addition, in the evolution, we design a global-recording technique to record the best IC subsets of each IC set in a global-recording list. Then the IC subsets in the list are deployed to build base classifier so as to implement the second fusion scheme. Furthermore, by pruning about half of less accurate base classifiers obtained by the second scheme, a compact and more accurate ensemble system is built, which is regarded as the third fusion scheme. Three microarray datasets are used to test the ensemble systems, and the corresponding results demonstrate that these ensemble schemes can further improve the performance of the ICA based classification model, and the third fusion scheme leads to the most accurate ensemble system with the smallest ensemble size.

Ensemble Learning for Chemical Sensor Arrays

Electrochemical sensors, like ion-selective field transistors (ISFET), are electronic devices that merge solid-state electronic technology with chemical sensors so as to be sensitive to the concentration of a particular ion in a solution. However, as it has been previously reported, their response does not only depend on a single ion but also is affected by several interfering ions found in the solution to be measured. These interfering ions can be considered as noise and consequently, a post-processing stage that increases the SNR is obligatory. Our work shows how ensemble learning methods could be used in an array of chemical sensors in order to deal with this problem. In particular, we introduce a novel neural learning architecture for ISFET arrays, which employ ISFET models as prior knowledge. The proposed ensemble learning systems are RBF-like solutions based on bagging and optimal linear combination. Several experimental results are included, which demonstrate the interest and viability of the proposed solution.

Simultaneous training of negatively correlated neural networks in an ensemble

This paper presents a new cooperative ensemble learning system (CELS) for designing neural network ensembles. The idea behind CELS is to encourage different individual networks in an ensemble to learn different parts or aspects of a training data so that the ensemble can learn the whole training data better. In CELS, the individual networks are trained simultaneously rather than independently or sequentially. This provides an opportunity for the individual networks to interact with each other and to specialize. CELS can create negatively correlated neural networks using a correlation penalty term in the error function to encourage such specialization. This paper analyzes CELS in terms of bias-variance-covariance tradeoff. CELS has also been tested on the Mackey-Glass time series prediction problem and the Australian credit card assessment problem. The experimental results show that CELS can produce neural network ensembles with good generalization ability.