AdaBoost Strategy Research Articles

AGNN: Alternating Graph-Regularized Neural Networks to Alleviate Over-Smoothing.

Graph convolutional network (GCN) with the powerful capacity to explore graph-structural data has gained noticeable success in recent years. Nonetheless, most of the existing GCN-based models suffer from the notorious over-smoothing issue, owing to which shallow networks are extensively adopted. This may be problematic for complex graph datasets because a deeper GCN should be beneficial to propagating information across remote neighbors. Recent works have devoted effort to addressing over-smoothing problems, including establishing residual connection structure or fusing predictions from multilayer models. Because of the indistinguishable embeddings from deep layers, it is reasonable to generate more reliable predictions before conducting the combination of outputs from various layers. In light of this, we propose an alternating graph-regularized neural network (AGNN) composed of graph convolutional layer (GCL) and graph embedding layer (GEL). GEL is derived from the graph-regularized optimization containing Laplacian embedding term, which can alleviate the over-smoothing problem by periodic projection from the low-order feature space onto the high-order space. With more distinguishable features of distinct layers, an improved Adaboost strategy is utilized to aggregate outputs from each layer, which explores integrated embeddings of multi-hop neighbors. The proposed model is evaluated via a large number of experiments including performance comparison with some multilayer or multi-order graph neural networks, which reveals the superior performance improvement of AGNN compared with the state-of-the-art models.

Predicting the deformation behaviour of concrete face rockfill dams by combining support vector machine and AdaBoost ensemble algorithm

The accurate evaluation of deformation behaviour is very important to the optimal design and safety evaluation of concrete face rockfill dams (CFRDs). Deformation prediction is a key problem in the CFRD construction. Based on statistical review of typical deformation behaviour of 94 CFRD cases, the relationship amongst three typical deformation indexes and six influencing factors of CFRDs is analysed using multiple linear regression method. The main influencing factors of the dam behaviour are discussed. An improved support vector machine (SVM) dam deformation prediction model is proposed based on the construction of an adaptive hybrid kernel function. The particle swarm intelligent optimisation algorithm is used to determine the main parameters. To solve the problem of nonlinear mutation and discreteness of case data, the SVM-AdaBoost prediction model of typical deformation indexes is established by iterative optimisation of the basic learner composed of the improved SVM model through the AdaBoost strategy. The comparative analysis of the proposed model and existing models shows that the proposed model has good prediction accuracy and can be used to predict the typical deformation behaviour of CFRDs accurately.

Modeling UF fouling and backwash in seawater RO feedwater treatment using neural networks with evolutionary algorithm and Bayesian binary classification

A machine learning approach to describing the dynamics of ultrafiltration performance in pretreatment of seawater reverse osmosis (RO) feedwater was explored via ensemble back propagation neural network (BPNN) model. The BPNN model was developed with Alopex evolutionary algorithm (AEA) optimization and AdaBoost strategy. The progression of ultrafiltration (UF) membrane resistance during both filtration and backwash, along with backwash efficiency were modeled via the ensemble BPNN-AEA approach relying on 422 days of operational data for an integrated SWRO UF-RO system. Model performance, for UF membrane resistance and backwash efficiency, evaluated over a wide range of operating conditions and coagulant dosing strategies, revealed excellent performance with a forecasting capability even for cases of temporally variable water quality. The performance level attained with the current machine learning modeling approach, which is particularly suited for handling the dynamics of UF operation, should prove useful for (a) determining UF performance deviation from intended baseline performance, (b) forecasting expected UF performance due to anticipated changes in water quality, and (c) providing a basis for model-based control of UF operation.

SMOTE-Boost-based sparse Bayesian model for flood prediction

With a significant development of big data analysis and cloud-fog-edge computing, human-centered computing (HCC) has been a hot research topic worldwide. Essentially, HCC is a cross-disciplinary research domain, in which the core idea is to build an efficient interaction among persons, cyber space, and real world. Inspired by the improvement of HCC on big data analysis, we intend to involve related core and technologies to help solve one of the most important issues in the real world, i.e., flood prediction. To minimize the negative impacts brought by floods, researchers pay special attention to improve the accuracy of flood forecasting with quantity of technologies including HCC. However, historical flood data is essentially imbalanced. Imbalanced data causes machine learning classifiers to be more biased towards patterns with majority samples, resulting in poor classification of pattern with minority samples. In this paper, we propose a novel Synthetic Minority Over-sampling Technique (SMOTE)-Boost-based sparse Bayesian model to perform flood prediction with both high accuracy and robustness. The proposed model consists of three modules, namely, SMOTE-based data enhancement, AdaBoost training strategy, and sparse Bayes model construction. In SMOTE-based data enhancement, we adopt a SMOTE algorithm to effectively cover diverse data modes and generate more samples for prediction pattern with minority samples, which greatly alleviates the problem of imbalanced data by involving experts’ analysis and users’ intentions. During AdaBoost training strategy, we propose a specifically designed AdaBoost training strategy for sparse Bayesian model, which not only adaptively and inclemently increases prediction ability of Bayesian model, but also prevents its over-fitting performance. Essentially, the design of AdaBoost strategy helps keep balance between prediction ability and model complexity, which offers different but effective models over diverse rivers and users. Finally, we construct a sparse Bayesian model based on AdaBoost training strategy, which could offer flood prediction results with high rationality and robustness. We demonstrate the accuracy and effectiveness of the proposed model for flood prediction by conducting experiments on a collected dataset with several comparative methods.

Robust labeling of human motion markers in the presence of occlusions

Human motion capture by optical sensors produces snapshots of the motion of a cloud of points that need to be labeled in order to carry out ensuing motion analysis for medical or other purposes. We generate the labeling of instantaneous captures of the cloud of points, discarding temporal correlations, in the presence of occlusions. Our approach proposes an ensemble of weak classifiers defined over geometrical features extracted from small subsets of the cloud of points. We apply an Adaboost strategy to select a minimal ensemble of weak classifiers achieving a target correct labeling detection accuracy. Furthermore, we use these features to generate the labeling of the points in the cloud even in the presence of occlusions.To deal with the occlusions of markers we search for ensembles of partial labeling solvers which can provide partial consistent labelings which cover the unoccluded markers. We test two greedy search approaches and a genetic algorithm in the search for the optimal ensemble of partial solvers We demonstrate the approach on a real dataset obtained from the measurement of gait motion of persons, with available ground truth labeling. Results are encouraging, achieving high accuracy label generation at a reduced computational cost.

A robust temperature prediction model of shuttle kiln based on ensemble random vector functional link network

In the sintering process for shuttle kiln, the sintering temperature is the most important thermal parameter, which plays a critical role in stable and efficient operation of the production process. However, due to the complex structure of shuttle kiln, this poses a great challenge for accurate and reliable online prediction of sintering temperature. Therefore, in this paper, a novel prediction model based on ensemble random vector functional link network (AB-RRVFLN) is proposed to predict the temperature. First, the mutual information method is proposed to select the optimal variables for prediction model. Then, considering the outlier influence of industrial data, a robust random vector functional link network (RRVFLN) based on the iteratively reweighted least squares is developed to build prediction model. Moreover, AdaBoost strategy is integrated with RRVFLN model to enhance prediction performance. Finally, the model output is further corrected by offset compensation technique. Based on industrial data from shuttle kiln, the application results on sintering temperature prediction demonstrate that the proposed ensemble method achieves better accuracy and shows stronger robustness than other methods.

Compressive Sequential Learning for Action Similarity Labeling.

Human action recognition in videos has been extensively studied in recent years due to its wide range of applications. Instead of classifying video sequences into a number of action categories, in this paper, we focus on a particular problem of action similarity labeling (ASLAN), which aims at verifying whether a pair of videos contain the same type of action or not. To address this challenge, a novel approach called compressive sequential learning (CSL) is proposed by leveraging the compressive sensing theory and sequential learning. We first project data points to a low-dimensional space by effectively exploring an important property in compressive sensing: the restricted isometry property. In particular, a very sparse measurement matrix is adopted to reduce the dimensionality efficiently. We then learn an ensemble classifier for measuring similarities between pairwise videos by iteratively minimizing its empirical risk with the AdaBoost strategy on the training set. Unlike conventional AdaBoost, the weak learner for each iteration is not explicitly defined and its parameters are learned through greedy optimization. Furthermore, an alternative of CSL named compressive sequential encoding is developed as an encoding technique and followed by a linear classifier to address the similarity-labeling problem. Our method has been systematically evaluated on four action data sets: ASLAN, KTH, HMDB51, and Hollywood2, and the results show the effectiveness and superiority of our method for ASLAN.

Sequential Compact Code Learning for Unsupervised Image Hashing.

Effective hashing for large-scale image databases is a popular research area, attracting much attention in computer vision and visual information retrieval. Several recent methods attempt to learn either graph embedding or semantic coding for fast and accurate applications. In this paper, a novel unsupervised framework, termed evolutionary compact embedding (ECE), is introduced to automatically learn the task-specific binary hash codes. It can be regarded as an optimization algorithm that combines the genetic programming (GP) and a boosting trick. In our architecture, each bit of ECE is iteratively computed using a weak binary classification function, which is generated through GP evolving by jointly minimizing its empirical risk with the AdaBoost strategy on a training set. We address this as greedy optimization by embedding high-dimensional data points into a similarity-preserved Hamming space with a low dimension. We systematically evaluate ECE on two data sets, SIFT 1M and GIST 1M, showing the effectiveness and the accuracy of our method for a large-scale similarity search.

Hierarchical abstract semantic model for image classification

Semantic gap limits the performance of bag-of-visual-words. To deal with this problem, a hierarchical abstract semantics method that builds abstract semantic layers, generates semantic visual vocabularies, measures semantic gap, and constructs classifiers using the Adaboost strategy is proposed. First, abstract semantic layers are proposed to narrow the semantic gap between visual features and their interpretation. Then semantic visual words are extracted as features to train semantic classifiers. One popular form of measurement is used to quantify the semantic gap. The Adaboost training strategy is used to combine weak classifiers into strong ones to further improve performance. For a testing image, the category is estimated layer-by-layer. Corresponding abstract hierarchical structures for popular datasets, including Caltech-101 and MSRC, are proposed for evaluation. The experimental results show that the proposed method is capable of narrowing semantic gaps effectively and performs better than other categorization methods.

Sample-screening MKL method via boosting strategy for hyperspectral image classification

The problem of limited training samples is always a major concern in hyperspectral remote sensing image classification. In this paper, a sample-screening multiple kernel learning (S2MKL) method is proposed for hyperspectral image classification with limited training samples. The core idea of the proposed method is to employ boosting strategy for screening the limited training samples under MKL framework. Different from existing methods, the proposed MKL method exploits the boosting trick to try different combinations of the limited training samples and adaptively determine the optimal weights of base kernels in the linear combination. Morphological profiles are firstly extracted as the both spatial and spectral features for classification instead of the original spectra. With the morphological profiles, AdaBoost strategy is then introduced to guide the construction of multiple kernel learning machine. By means of boosting strategy, the limited samples are effectively screened and used for classification. Meanwhile, the weights of base kernels in the linear combination are automatically determined in the process of screening samples. Three real hyperspectral data sets are used to evaluate the proposed method. The experimental results show that the proposed boosting-based multiple kernel learning method is superior to state-of-the-art methods in terms of classification performance while limited samples are used.

A PSO-AB classifier for solving sequence classification problems

Recently, considerable attention has focused on compound sequence classification methods which integrate multiple data mining techniques. Among these methods, sequential pattern mining (SPM) based sequence classifiers are considered to be efficient for solving complex sequence classification problems. Although previous studies have demonstrated the strength of SPM-based sequence classification methods, the challenges of pattern redundancy, inappropriate sequence similarity measures, and hard-to-classify sequences remain unsolved. This paper proposes an efficient two-stage SPM-based sequence classification method to address these three problems. In the first stage, during the sequential pattern mining process, redundant sequential patterns are identified if the pattern is a sub-sequence of other sequential patterns. A list of compact sequential patterns is generated excluding redundant patterns and used as representative features for the second stage. In the second stage, a sequence similarity measurement is used to evaluate partial similarity between sequences and patterns. Finally, a particle swarm optimization-AdaBoost (PSO-AB) sequence classifier is developed to improve sequence classification accuracy. In the PSO-AB sequence classifier, the PSO algorithm is used to optimize the weights in the individual sequence classifier, while the AdaBoost strategy is used to adaptively change the distribution of patterns that are hard to classify. The experiments show that the proposed two-stage SPM-based sequence classification method is efficient and superior to other approaches.

Evolutionary compact embedding for large-scale image classification

Effective dimensionality reduction is a classical research area for many large-scale analysis tasks in computer vision. Several recent methods attempt to learn either graph embedding or binary hashing for fast and accurate applications. In this paper, we propose a novel framework to automatically learn the task-specific compact coding, called evolutionary compact embedding (ECE), which can be regarded as an optimization algorithm combining genetic programming (GP) and a boosting trick. As an evolutionary computation methodology, GP can solve problems inspired by natural evolution without any prior knowledge of the solutions. In our evolutionary architecture, each bit of ECE is iteratively computed using a binary classification function, which is generated through GP evolving by jointly minimizing its empirical risk with the AdaBoost strategy on a training set. We address this as greedy optimization leading to small Hamming distances for similar samples and large distances for dissimilar samples. We then evaluate ECE on four image datasets: USPS digital hand-writing, CMU PIE face, CIFAR-10 tiny image and SUN397 scene, showing the accurate and robust performance of our method for large-scale image classification.