Stochastic Sensitivity Measure Research Articles

Radial Basis Function Neural Network with Localized Stochastic-Sensitive Autoencoder for Home-Based Activity Recognition.

Over the past few years, the Internet of Things (IoT) has been greatly developed with one instance being smart home devices gradually entering into people’s lives. To maximize the impact of such deployments, home-based activity recognition is required to initially recognize behaviors within smart home environments and to use this information to provide better health and social care services. Activity recognition has the ability to recognize people’s activities from the information about their interaction with the environment collected by sensors embedded within the home. In this paper, binary data collected by anonymous binary sensors such as pressure sensors, contact sensors, passive infrared sensors etc. are used to recognize activities. A radial basis function neural network (RBFNN) with localized stochastic-sensitive autoencoder (LiSSA) method is proposed for the purposes of home-based activity recognition. An autoencoder (AE) is introduced to extract useful features from the binary sensor data by converting binary inputs into continuous inputs to extract increased levels of hidden information. The generalization capability of the proposed method is enhanced by minimizing both the training error and the stochastic sensitivity measure in an attempt to improve the ability of the classifier to tolerate uncertainties in the sensor data. Four binary home-based activity recognition datasets including OrdonezA, OrdonezB, Ulster, and activities of daily living data from van Kasteren (vanKasterenADL) are used to evaluate the effectiveness of the proposed method. Compared with well-known benchmarking approaches including support vector machine (SVM), multilayer perceptron neural network (MLPNN), random forest and an RBFNN-based method, the proposed method yielded the best performance with 98.35%, 86.26%, 96.31%, 92.31% accuracy on four datasets, respectively.

Robust Neural Networks Learning via a Minimization of Stochastic Output Sensitivity

this article, we propose Sensitivity Minimization Learning (SML) to overcome the performance degradation problem caused by features corruption at the testing phase by using the stochastic sensitivity measure (STSM) as a regularizer. The STSM measures output deviations between each training sample and its noisy versions generated by a feature perturbation strategy. The feature perturbation strategy is user defined to simulate noises that our model target to defense at the testing phase. The SML is computational efficiency in training and testing phase and minimizes the generalization error on both training samples and its noise version set with feature perturbations. Models regularized by the STSM can be trained by the stochastic gradient descent algorithm efficiently and applied to very large scale applications. Experiment results on eight grayscale image, one color image, and two face databases show that the SML significantly outperforms several regularization techniques and yields much lower classification error when testing sets are contaminated with noises.

Evaluation of radial basis function neural network minimizing L-GEM for sensor-based activity recognition

Sensor-based activity recognition involves the automatic recognition of a user’s activity in a smart environment using computational methods. The use of wearable devices and video-based approaches have attracted considerable interest in ubiquitous computing. Nevertheless, these methods have limitations such as issues with privacy invasion, ethics, comfort and obtrusiveness. Environmental sensors are an increasingly promising consideration in the ubiquitous computing domain for long-term monitoring, as these devices are non-invasive to inhabitants, yet certain challenges remain with activity recognition in sensorised environments, for example, addressing the challenge of intraclass variation between activities and reasoning from low-level uncertain information. In an effort to address these challenges, this paper proposes and evaluates the performance of a Radial Basis Function Neural Network approach for activity recognition with environmental sensors. The model is trained using the Localized Generalization Error and focuses on the generalization ability by considering both the training error and stochastic sensitivity measure. This measures the network output fluctuation with respect to the minor perturbation of input, to address the tolerance of the low-level uncertain sensor data. This approach is compared with three benchmark Neural Network approaches, including a popular deep learning approach using an Autoencoder, and it is evaluated with a simulated dataset as well as a number of publicly available datasets. The proposed method has shown advantages over the other models for all four evaluated datasets. This paper provides insights into the importance of model generalization abilities and an initial analysis of the limitation of deep Neural Networks with respect to sensor-based activity recognition.

MLPNN Training via a Multiobjective Optimization of Training Error and Stochastic Sensitivity.

The training of a multilayer perceptron neural network (MLPNN) concerns the selection of its architecture and the connection weights via the minimization of both the training error and a penalty term. Different penalty terms have been proposed to control the smoothness of the MLPNN for better generalization capability. However, controlling its smoothness using, for instance, the norm of weights or the Vapnik-Chervonenkis dimension cannot distinguish individual MLPNNs with the same number of free parameters or the same norm. In this paper, to enhance generalization capabilities, we propose a stochastic sensitivity measure (ST-SM) to realize a new penalty term for MLPNN training. The ST-SM determines the expectation of the squared output differences between the training samples and the unseen samples located within their Q -neighborhoods for a given MLPNN. It provides a direct measurement of the MLPNNs output fluctuations, i.e., smoothness. We adopt a two-phase Pareto-based multiobjective training algorithm for minimizing both the training error and the ST-SM as biobjective functions. Experiments on 20 UCI data sets show that the MLPNNs trained by the proposed algorithm yield better accuracies on testing data than several recent and classical MLPNN training methods.

Diversified Sensitivity-Based Undersampling for Imbalance Classification Problems.

Undersampling is a widely adopted method to deal with imbalance pattern classification problems. Current methods mainly depend on either random resampling on the majority class or resampling at the decision boundary. Random-based undersampling fails to take into consideration informative samples in the data while resampling at the decision boundary is sensitive to class overlapping. Both techniques ignore the distribution information of the training dataset. In this paper, we propose a diversified sensitivity-based undersampling method. Samples of the majority class are clustered to capture the distribution information and enhance the diversity of the resampling. A stochastic sensitivity measure is applied to select samples from both clusters of the majority class and the minority class. By iteratively clustering and sampling, a balanced set of samples yielding high classifier sensitivity is selected. The proposed method yields a good generalization capability for 14 UCI datasets.

Architecture selection for networks trained with extreme learning machine using localized generalization error model

The initial localized generalization error model (LGEM) aims to find an upper bound of error between a target function and a radial basis function neural network (RBFNN) within a neighborhood of the training samples. The contribution of LGEM can be briefly described as that the generalization error is less than or equal to the summation of three terms: training error, stochastic sensitivity measure (SSM), and a constant. This paper extends the initial LGEM to a new LGEM model for single-hidden layer feed-forward neural networks (SLFNs) trained with extreme learning machine (ELM) which is a type of new training algorithms without iterations. The development of this extended LGEM can provide some useful guidelines for improving the generalization ability of SLFNs trained with ELM. An algorithm for architecture selection of the SLFNs is also proposed based on the extended LGEM. Experimental results on a number of benchmark data sets show that an approximately optimal architecture in terms of number of neurons of a SLFN can be found using our method. Furthermore, the experimental results on eleven UCI data sets show that the proposed method is effective and efficient.

THE LOCALIZED GENERALIZATION ERROR MODEL FOR SINGLE LAYER PERCEPTRON NEURAL NETWORK AND SIGMOID SUPPORT VECTOR MACHINE

We had developed the localized generalization error model for supervised learning with minimization of Mean Square Error. In this work, we extend the error model to Single Layer Perceptron Neural Network (SLPNN) and Support Vector Machine (SVM) with sigmoid kernel function. For a trained SLPNN or SVM and a given training dataset, the proposed error model bounds above the error for unseen samples which are similar to the training samples. As the major component of the localized generalization error model, the stochastic sensitivity measure formula for perceptron neural network derived in this work has relaxed the assumptions of same distribution for all inputs and each sample perturbed only once in previous works. These make the sensitivity measure applicable to pattern classification problems. The stochastic sensitivity measure of SVM with Sigmoid kernel is also derived in this work as a component of the localized generalization error model. At the end of this paper, we discuss the advantages of the proposed error bound over existing error bound.

Classifier learning with a new locality regularization method

It is well known that the generalization capability is one of the most important criterions to develop and evaluate a classifier for a given pattern classification problem. The localized generalization error model ( R SM ) recently proposed by Ng et al. [Localized generalization error and its application to RBFNN training, in: Proceedings of the International Conference on Machine Learning and Cybernetics, China, 2005; Image classification with the use of radial basis function neural networks and the minimization of the localized generalization error, Pattern Recognition 40(1) (2007) 4–18] provides a more intuitive look at the generalization error. Although R SM gives a brand-new method to promote the generalization performance, it is in nature equivalent to another type of regularization. In this paper, we first prove the essential relationship between R SM and regularization, and demonstrate that the stochastic sensitivity measure in R SM exactly corresponds to a regularizing term. Then, we develop a new generalization error bound from the regularization viewpoint, which is inspired by the proved relationship between R SM and regularization. Moreover, we derive a new regularization method, called as locality regularization (LR), from the bound. Different from the existing regularization methods which artificially and externally append the regularizing term in order to smooth the solution, LR is naturally and internally deduced from the defined expected risk functional and calculated by employing locality information. Through combining with spectral graph theory, LR introduces the local structure information of the samples into the regularizing term and further improves the generalization capability. In contrast with R SM , which is relatively sensitive to the different sampling of the samples, LR uses the discrete k-neighborhood rather than the common continuous Q-neighborhood in R SM to differentiate the relative position of different training samples automatically and avoid the complex computation of Q for various classifiers. Furthermore, LR uses the regularization parameter to control the trade-off between the training accuracy and the classifier stability. Experimental results on artificial and real world problems show that LR yields better generalization capability than both R SM and some traditional regularization methods.

Localized Generalization Error Model and Its Application to Architecture Selection for Radial Basis Function Neural Network

The generalization error bounds found by current error models using the number of effective parameters of a classifier and the number of training samples are usually very loose. These bounds are intended for the entire input space. However, support vector machine (SVM), radial basis function neural network (RBFNN), and multilayer perceptron neural network (MLPNN) are local learning machines for solving problems and treat unseen samples near the training samples to be more important. In this paper, we propose a localized generalization error model which bounds from above the generalization error within a neighborhood of the training samples using stochastic sensitivity measure. It is then used to develop an architecture selection technique for a classifier with maximal coverage of unseen samples by specifying a generalization error threshold. Experiments using 17 University of California at Irvine (UCI) data sets show that, in comparison with cross validation (CV), sequential learning, and two other ad hoc methods, our technique consistently yields the best testing classification accuracy with fewer hidden neurons and less training time.

Localized Generalization Error of Gaussian-based Classifiers and Visualization of Decision Boundaries

In pattern classification problem, one trains a classifier to recognize future unseen samples using a training dataset. Practically, one should not expect the trained classifier could correctly recognize samples dissimilar to the training dataset. Therefore, finding the generalization capability of a classifier for those unseen samples may not help in improving the classifiers accuracy. The localized generalization error model was proposed to bound above the generalization mean square error for those unseen samples similar to the training dataset only. This error model is derived based on the stochastic sensitivity measure(ST-SM)of the classifiers. We present the ST-SMS for various Gaussian based classifiers: radial basis function neural networks and support vector machine in this paper. At the end of this work, we compare the decision boundaries visualization using the training samples yielding the largest sensitivity measures and the one using support vectors in the input space.

Selection of weight quantisation accuracy for radial basis function neural network using stochastic sensitivity measure

Minimising the number of bits per connection weight in hardware realisation of a radial basis function neural network (RBFNN) will result in high-speed and low-cost implementation, with possible increase in output error. A weight quantisation accuracy selection method is proposed, to find an appropriate number of bits for a given stochastic sensitivity measure, which quantifies the relationship between the variance of the output error and first- and second-order statistics of input, weight and their perturbations.

Localization of aeroelastic modes in mistuned high-energy turbines

The effects of blade mistuning on the aeroelastic vibration characteristics of high-energy turbines are investigated, using the first stage of the oxidizer turbopump in the Space Shuttle main rocket engine as an example. A modal aeroelastic analysis procedure is used in concert with a linearized unsteady aerodynamic theory that accounts for the effects of blade thickness, camber, and steady loading. High sensitivity of the dynamic characteristics of mistuned rotors is demonstrated. In particular, the aeroelastic free vibration modes become localized to a few blades, possibly leading to rogue blade failure, and the locus of the aeroelastic eigenvalues loses its regular structure when small mistuning (of the order usually present in actual rotors) is introduced. Perturbation analyses that yield physical insights into these phenomena are presented. A powerful but easily calculated stochastic sensitivity measure that allows the global prediction of mistuning effects is developed.