Random Subspace Learning Research Articles

Deep Random Subspace Learning: A Spatial-Temporal Modeling Approach for Air Quality Prediction

Decrease in air quality is one of the most crucial threats to human health. There is an imperative and necessary need for more accurate air quality prediction. To meet this need, we propose a novel long short-term memory-based deep random subspace learning (LSTM-DRSL) framework for air quality forecasting. Specifically, we incorporate real-time pollutant emission data into the model input. We also design a spatial-temporal analysis approach to make good use of these data. The prediction model is developed by combining random subspace learning with a deep learning algorithm in order to improve the prediction accuracy. Empirical analyses based on multiple datasets over China from January 2015 to September 2017 are performed to demonstrate the efficacy of the proposed framework for hourly pollutant concentration prediction at an urban-agglomeration scale. The empirical results indicate that our framework is a viable method for air quality prediction. With consideration of the regional scale, the LSTM-DRSL framework performs better at a relatively large regional scale (around 200–300 km). In addition, the quality of predictions is higher in industrial areas. From a temporal point of view, the LSTM-DRSL framework is more suitable for hourly predictions.

Using Ensemble Technique to Improve Multiclass Classification

Many real world applications inevitably contain datasets that have multiclass structure characterized by imbalance classes, redundant and irrelevant features that degrade performance of classifiers. Minority classes in the datasets are treated as outliers’ classes. The research aimed at establishing the role of ensemble technique in improving performance of multiclass classification. Multiclass datasets were transformed to binary and the datasets resampled using Synthetic minority oversampling technique (SMOTE) algorithm. Relevant features of the datasets were selected by use of an ensemble filter method developed using Correlation, Information Gain, Gain-Ratio and ReliefF filter selection methods. Adaboost and Random subspace learning algorithms were combined using Voting methodology utilizing random forest as the base classifier. The classifiers were evaluated using 10 fold stratified cross validation. The model showed better performance in terms of outlier detection and classification prediction for multiclass problem. The model outperformed other well-known existing classification and outlier detection algorithms such as Naive bayes, KNN, Bagging, JRipper, Decision trees, RandomTree and Random forest. The study findings established that ensemble technique, resampling datasets and decomposing multiclass results in an improved classification performance as well as enhanced detection of minority outlier (rare) classes. Keywords: Multiclass, Classification, Outliers, Ensemble, Learning Algorithm DOI : 10.7176/JIEA/9-5-04 Publication date : August 31 st 2019

Blockchain and Random Subspace Learning-Based IDS for SDN-Enabled Industrial IoT Security.

The industrial control systems are facing an increasing number of sophisticated cyber attacks that can have very dangerous consequences on humans and their environments. In order to deal with these issues, novel technologies and approaches should be adopted. In this paper, we focus on the security of commands in industrial IoT against forged commands and misrouting of commands. To this end, we propose a security architecture that integrates the Blockchain and the Software-defined network (SDN) technologies. The proposed security architecture is composed of: (a) an intrusion detection system, namely RSL-KNN, which combines the Random Subspace Learning (RSL) and K-Nearest Neighbor (KNN) to defend against the forged commands, which target the industrial control process, and (b) a Blockchain-based Integrity Checking System (BICS), which can prevent the misrouting attack, which tampers with the OpenFlow rules of the SDN-enabled industrial IoT systems. We test the proposed security solution on an Industrial Control System Cyber attack Dataset and on an experimental platform combining software-defined networking and blockchain technologies. The evaluation results demonstrate the effectiveness and efficiency of the proposed security solution.

Exploring spatial–temporal relations via deep convolutional neural networks for traffic flow prediction with incomplete data

Traffic flow prediction is a fundamental component in intelligent transportation systems. Various computational methods have been applied in this field, among which machine learning based methods are believed to be promising and scalable for big data. In general, most of machine learning based methods encounter three fundamental issues: feature representation of traffic patterns, learning from single location or network, and data quality. In order to address these three issues, in this work we present a deep architecture for traffic flow prediction that learns deep hierarchical feature representation with spatio-temporal relations over the traffic network. Furthermore, we design an ensemble learning strategy via random subspace learning to make the model be able to tolerate incomplete data. Accordingly the contributions of this work are summarized as the three points. First, we transform the time series analysis problem into the task of image-like analysis. Benefitting from the image-like data form, we can jointly explore spatio-temporal relations simultaneously by the two-dimension convolution operator. In addition, the proposed model can tolerate the incomplete data, which is very common in traffic application field. Finally, we propose an improved random search based on uniform design in order to optimize hyper-parameters for deep Convolutional Neural Networks (deep CNN). A large range of experiments with various traffic conditions have been performed on the traffic data originated from the California Freeway Performance Measurement System (PeMS). The experimental results corroborate the effectiveness of the proposed approach compared with the state of the art.

Random Subspace Learning (RASSEL) with data driven weighting schemes

Random Subspace Learning (RASSEL) with data driven weighting schemes

Prediction Error Reduction Function as a Variable Importance Score

This paper introduces and develops a novel variable importance score function in the context of ensemble learning, and demonstrates its appeal empirically. Our proposed score function is simple and more straightforward than its counterpart proposed in the context of random forest, and by avoiding permutations, it is by design computationally more efficient than the random forest variable importance function. Just like the random forest variable importance function, our score handles both regression and classification seamlessly. One of the distinct advantage of our proposed score is the fact that it offers a natural cut off at zero, with all the positive scores indicating importance and significance, while the negative scores are deemed indications of insignificance. An extra advantage of our proposed score lies in the fact it works very well beyond ensemble of trees and can seamlessly be used with any base learners in the random subspace learning context. Our examples, both simulated and real, demonstrate that our proposed score does compete mostly favorably with the random forest score.

Random Subspace Learning Approach to High-Dimensional Outliers Detection

We introduce and develop a novel approach to outlier detection based on adaptation of random subspace learning. Our proposed method handles both high-dimension low-sample size and traditional low-dimensional high-sample size datasets. Essentially, we avoid the computational bottleneck of techniques like Minimum Covariance Determinant (MCD) by computing the needed determinants and associated measures in much lower dimensional subspaces. Both theoretical and computational development of our approach reveal that it is computationally more efficient than the regularized methods in high-dimensional low-sample size, and often competes favorably with existing methods as far as the percentage of correct outlier detection are concerned.

Robust view-invariant multiscale gait recognition

The paper proposes a two-phase view-invariant multiscale gait recognition method (VI-MGR) which is robust to variation in clothing and presence of a carried item. In phase 1, VI-MGR uses the entropy of the limb region of a gait energy image (GEI) to determine the matching gallery view of the probe using 2-dimensional principal component analysis and Euclidean distance classifier. In phase 2, the probe subject is compared with the matching view of the gallery subjects using multiscale shape analysis. In this phase, VI-MGR applies Gaussian filter to a GEI to generate a multiscale gait image for gradually highlighting the subject׳s inner shape characteristics to achieve insensitiveness to boundary shape alterations due to carrying conditions and clothing variation. A weighted random subspace learning based classification is used to exploit the high dimensionality of the feature space for improved identification by avoiding overlearning. Experimental analyses on public datasets demonstrate the efficacy of VI-MGR.