Original Input Space Research Articles

Fault detection and diagnosis strategy based on a weighted and combined index in the residual subspace associated with PCA

AbstractProcess monitoring and diagnosis are crucial for efficient and optimal operation of a chemical plant. Most multivariate statistical process monitoring strategies, such as principal component analysis, kernel principal component analysis, and dynamic principal component analysis, take advantage of the squared prediction error statistic to monitor the state of samples in a residual subspace (RS). Squared prediction error is defined as the square of the 2‐norm of a residual vector, and it is calculated as the squared norm of the residual components. When the distributions of variables in an RS are quite different from one another, the detection ability of squared prediction error visibly declines. To accurately monitor the faults occurring in the RS, a new fault detection index based on a weighted combination of Hotelling's T2 and squared Euclidean distance is developed in this paper. Principal component analysis is first introduced for dividing the original input space into a principal component subspace and an RS. Next, a weighted and combined index is implemented to monitor the variability of samples in the RS. In addition, a corresponding fault diagnosis strategy based on the contribution plot is also developed in this paper. The proposed method is tested on a numerical example and the Tennessee Eastman process. Simulation results show that the new index is effective in both fault detection and diagnosis.

Developing a Dual-Track Modeling Approach for Increased Understanding of Sensors and their Forecasting Capabilities

Event Abstract Back to Event Developing a Dual-Track Modeling Approach for Increased Understanding of Sensors and their Forecasting Capabilities Raquel C. Galvan-Garza1*, Peter B. Bryan1*, Amanda E. Kraft1, Alison M. Perez1, Matthew J. Pava1, William D. Casebeer1 and Matthias D. Ziegler1 1 Lockheed Martin (United States), United States Forecasting human behavior from sensor-derived biometrics poses a modeling challenge encountered across statistical domains: balancing interpretability against model power. On one hand, polynomial models derived from literature review and a priori assumptions of feature interactions are transparent (i.e., explained variance can be meaningfully interpreted in the original input space) but fail to capture higher-order interactions that were not explicitly predicted. On the other, black-box methods from machine learning are equipped to successfully capture this variance but sacrifice explainability. Here, we evaluate a dual-track approach for modeling of human state. The first track (theory-driven) consists of white-box modeling methodologies while the second track (data-driven) consists of a black-box model, namely a recurrent neural network. The data sets we use in this presentation focus on physiological, interpersonal, and environmental factors; however, we are beginning to use the same methodology with multi-sensor studies that include neuroimaging techniques such as fNIRS and EEG. We will discuss details on how we 1) create theory-driven models for estimating variables such as subjective stress, anxiety, affect, personality, and job performance, 2) create data-driven models using complex neural networks in parallel, and 3) adapt the theory-driven model to include new connections discovered by the data-driven model using a systematic automated approach. Using methods including local sensitivity analysis, we attempt to extract variance-capturing, low-dimensional feature combinations from the data-driven track to augment theory-driven models. We conclude with a discussion of the respective advantages and disadvantages of each track, a summary of challenges intrinsic to this method, and an evaluation of the successes and failures of our complementary modeling approach. The research is based upon work supported by the Office of the Director of National Intelligence (ODNI), Intelligence Advanced Research Projects Activity (IARPA), via 2017-17042800004. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of the ODNI, IARPA, or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright annotation thereon. Keywords: biometrics, forecasting human behavior, Model selection, theory-driven modeling, Data-driven Modeling, machine learning applied to neuroscience Conference: 2nd International Neuroergonomics Conference, Philadelphia, PA, United States, 27 Jun - 29 Jun, 2018. Presentation Type: Oral Presentation Topic: Neuroergonomics Citation: Galvan-Garza RC, Bryan PB, Kraft AE, Perez AM, Pava MJ, Casebeer WD and Ziegler MD (2019). Developing a Dual-Track Modeling Approach for Increased Understanding of Sensors and their Forecasting Capabilities. Conference Abstract: 2nd International Neuroergonomics Conference. doi: 10.3389/conf.fnhum.2018.227.00029 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 30 Mar 2018; Published Online: 27 Sep 2019. * Correspondence: PhD. Raquel C Galvan-Garza, Lockheed Martin (United States), Bethesda, United States, raquel.c.galvan@lmco.com Mr. Peter B Bryan, Lockheed Martin (United States), Bethesda, United States, peterbbryan@gmail.com Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Raquel C Galvan-Garza Peter B Bryan Amanda E Kraft Alison M Perez Matthew J Pava William D Casebeer Matthias D Ziegler Google Raquel C Galvan-Garza Peter B Bryan Amanda E Kraft Alison M Perez Matthew J Pava William D Casebeer Matthias D Ziegler Google Scholar Raquel C Galvan-Garza Peter B Bryan Amanda E Kraft Alison M Perez Matthew J Pava William D Casebeer Matthias D Ziegler PubMed Raquel C Galvan-Garza Peter B Bryan Amanda E Kraft Alison M Perez Matthew J Pava William D Casebeer Matthias D Ziegler Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.

Prediction for Big Data Through Kriging: Small Sequential and One-Shot Designs

Kriging or Gaussian process (GP) modeling is an interpolation method that assumes the outputs (responses) are more correlated, the closer the inputs (explanatory or independent variables) are. A GP has unknown (hyper)parameters that must be estimated; the standard estimation method uses the maximum likelihood criterion. However, big data make it hard to compute the estimates of these GP parameters, and the resulting Kriging predictor and the variance of this predictor. To solve this problem, some authors select a relatively small subset from the big set of previously observed old data; their method is sequential and depends on the variance of the Kriging predictor. The resulting designs turn out to be local; i.e., most design points are concentrated around the point to be predicted. We develop three alternative one-shot methods that do not depend on GP parameters: (i) select a small subset such that this subset still covers the original input space albeit coarser; (ii) select a subset with relatively many but not all combinations close to the new combination that is to be predicted, and (iii) select a subset with the nearest neighbors (NNs) of this new combination. To evaluate these designs, we compare their squared prediction errors in several numerical (Monte Carlo) experiments. These experiments show that our NN design is a viable alternative for the more sophisticated sequential designs.

A synthetic neighborhood generation based ensemble learning for the imbalanced data classification

Constructing effective classifiers from imbalanced datasets has emerged as one of the main challenges in the data mining community, due to its increased prevalence in various real-world domains. Ensemble solutions are quite often applied in this field for their ability to provide better classification ability than single classifier. However, most existing methods adopt data sampling to train the base classifier on balanced datasets, but not to directly enhance the diversity. Thus, the performance of the final classifier can be limited. This paper suggests a new ensemble learning that can address the class imbalance problem and promote diversity simultaneously. Inspired by the localized generalization error model, this paper generates some synthetic samples located within some local area of the training samples, and trains the base classifiers with the union of original training samples and synthetic neighborhoods samples. By controlling the number of generated samples, the base classifiers can be trained with balanced datasets. Meanwhile, as the generated samples can extend different parts of the original input space and can be quite different from the original training samples, the obtained base classifiers are guaranteed to be accurate and diverse. A thorough experimental study on 36 benchmark datasets was performed, and the experimental results demonstrated that our proposed method can deliver significant better performance than the state-of-the-art ensemble solutions for the imbalanced problems.

Stacked-Structure-Based Hierarchical Takagi-Sugeno-Kang Fuzzy Classification Through Feature Augmentation

In this paper, a new stacked-structure-based hierarchical Takagi–Sugeno–Kang (TSK) fuzzy classifier called SHFA-TSK-FC with both promising performance and high interpretability is proposed to tackle with the shortcoming of the existing hierarchical fuzzy classifiers in interpreting the outputs and fuzzy rules of intermediate layers. In order to achieve the enhanced classification performance, each component unit, which is a zero-order TSK fuzzy classifier, in SHFA-TSK-FC is organized in a stacked way such that all the input features of the original training samples plus the interpretable augmented features, corresponding to the interpretable output of each previous component unit, are fed as the input features of the current component unit. These augmented features can essentially open the manifold structure of the original input space such that the enhanced classification performance can be expected. In designing each component unit, its analytical solution to the consequent parts of fuzzy rules therein is obtained quickly by using the least learning machine such that SHFA-TSK-FC becomes scalable for large datasets. Its high interpretability is guaranteed by randomly selecting the input features and randomly choosing the fixed five Gaussian membership functions for the selected input features in the premise of each fuzzy rule. Experimental results on real-life datasets and an application case demonstrate the enhanced or at least comparable classification performance and high interpretability of SHFA-TSK-FC.

Deep neural mapping support vector machines

The choice of kernel has an important effect on the performance of a support vector machine (SVM). The effect could be reduced by NEUROSVM, an architecture using multilayer perceptron for feature extraction and SVM for classification. In binary classification, a general linear kernel NEUROSVM can be theoretically simplified as an input layer, many hidden layers, and an SVM output layer. As a feature extractor, the sub-network composed of the input and hidden layers is first trained together with a virtual ordinary output layer by backpropagation, then with the output of its last hidden layer taken as input of the SVM classifier for further training separately. By taking the sub-network as a kernel mapping from the original input space into a feature space, we present a novel model, called deep neural mapping support vector machine (DNMSVM), from the viewpoint of deep learning. This model is also a new and general kernel learning method, where the kernel mapping is indeed an explicit function expressed as a sub-network, different from an implicit function induced by a kernel function traditionally. Moreover, we exploit a two-stage procedure of contrastive divergence learning and gradient descent for DNMSVM to jointly training an adaptive kernel mapping instead of a kernel function, without requirement of kernel tricks. As a whole of the sub-network and the SVM classifier, the joint training of DNMSVM is done by using gradient descent to optimize the objective function with the sub-network layer-wise pre-trained via contrastive divergence learning of restricted Boltzmann machines. Compared to the separate training of NEUROSVM, the joint training is a new algorithm for DNMSVM to have advantages over NEUROSVM. Experimental results show that DNMSVM can outperform NEUROSVM and RBFSVM (i.e., SVM with the kernel of radial basis function), demonstrating its effectiveness.

Projective label propagation by label embedding: A deep label prediction framework for representation and classification

Virtually all existing label propagation (LP) approaches estimate the unknown labels of points from the original input space directly, but the transductive results are “shallow” and usually contain unfavorable mixed signs that may decrease the performance of both transductive models and out-of-sample extensions. To solve this issue, we propose a Projective Label Propagation (ProjLP) framework by label embedding, which can deliver more discriminating “deep” labels of samples to enhance representation and classification. ProjLP is originally proposed to improve the transductive prediction results by simultaneously learning a robust projection to remove the unfavorable mixed signs and convert the shallow labels into the discriminating deep ones. By including a regressive reconstruction loss term to correlate the extracted discriminative features with the deep labels, the out-of-sample extension of ProjLP is also presented by delivering a linear neighborhood preserving projection classifier, where the neighborhood preserving power is enabled by sharing the same graph weights in the original embedded deep label space and the approximated label space. Thus, the deep label of each new data can be obtained by embedding it directly onto the classifier. We also show the inclusion method by reconstructing the deep label of new data from the deep labels of its neighbors. To show the deep property of embedded “deep” labels, a multilayer network architecture of our model is illustrated. Simulations on artificial and real-world datasets demonstrate the validity of our techniques for representation and classification, compared with other state-of-the-arts.

Dual autoencoders features for imbalance classification problem

Many classification problems encountered in real-world applications exhibit a profile of imbalanced data. Current methods depend on data resampling. In fact, if the feature set provides a clear decision boundary, resampling may not be needed to solve the imbalanced classification problem. Therefore, this work proposes a feature learning method based on the autoencoder to learn a set of features with better classification capabilities of the minority and the majority classes to address the imbalanced classification problems. Two sets of features are learned by two stacked autoencoders with different activation functions to capture different characteristics of the data and they are combined to form the Dual Autoencoding Features. Samples are then classified in the new feature space learned in this manner instead of the original input space. Experimental results show that the proposed method outperforms current resampling-based methods with statistical significance for imbalanced pattern classification problems.

Neural network training and rule extraction with augmented discretized input

The classification and prediction accuracy of neural networks can be improved when they are trained with discretized continuous attributes as additional inputs. Such input augmentation makes it easier for the network weights to form more accurate decision boundaries when the data samples of different classes in the data set are contained in distinct hyper-rectangular subregions in the original input space. In this paper, we present first how a neural network can be trained with augmented discretized inputs. The additional inputs are obtained by dividing the original interval of each continuous attribute into subintervals of equal length. The network is then pruned to remove most of the discretized inputs as well as the original continuous attributes as long as the network still achieves a minimum preset accuracy requirement. We then discuss how comprehensible classification rules can be extracted from the pruned network by analyzing the activations of the network hidden units and the weights of the network connections that remain in the pruned network. Our experiments on artificial data sets show that the rules extracted from the neural networks can perfectly replicate the class membership rules used to create the data perfectly. On real-life benchmark data sets, neural networks trained with augmented discretized inputs are shown to achieve better accuracy than neural networks trained with the original data.

ACCELERATING DYNAMIC MAGNETIC RESONANCE IMAGING BY NONLINEAR SPARSE CODING.

Although being high-dimensional, dynamic magnetic resonance images usually lie on low-dimensional manifolds. Nonlinear models have been shown to capture well that latent low-dimensional nature of data, and can thus lead to improvements in the quality of constrained recovery algorithms. This paper advocates a novel reconstruction algorithm for dynamic magnetic resonance imaging (dMRI) based on nonlinear dictionary learned from low-spatial but high-temporal resolution images. The nonlinear dictionary is initially learned using kernel dictionary learning, and the proposed algorithm subsequently alternates between sparsity enforcement in the feature space and the data-consistency constraint in the original input space. Extensive numerical tests demonstrate that the proposed scheme is superior to popular methods that use linear dictionaries learned from the same set of training data.

Hadoop based uncertain possibilistic kernelized c-means algorithms for image segmentation and a comparative analysis

Over the years data clustering algorithms have been used for image segmentation. Due to the presence of uncertainty in real life datasets, several uncertainty based data clustering algorithms have been developed. The c-means clustering algorithms form one such family of algorithms. Starting with the fuzzy c-means (FCM) a subfamily of this family comprises of rough c-means (RCM), intuitionistic fuzzy c-means (IFCM) and their hybrids like rough fuzzy c-means (RFCM) and rough intuitionistic fuzzy c-means (RIFCM). In the basic subfamily of this family of algorithms, the Euclidean distance was being used to measure the similarity of data. However, the sub family of algorithms obtained replacing the Euclidean distance by kernel based similarities produced better results. Especially, these algorithms were useful in handling viably cluster data points which are linearly inseparable in original input space. During this period it was inferred by Krishnapuram and Keller that the membership constraints in some rudimentary uncertainty based clustering techniques like fuzzy c-means imparts them a probabilistic nature, hence they suggested its possibilistic version. In fact all the other member algorithms from basic subfamily have been extended to incorporate this new notion. Currently, the use of image data is growing vigorously and constantly, accounting to huge figures leading to big data. Moreover, since image segmentation happens to be one of the most time consuming processes, industries are in the need of algorithms which can solve this problem at a rapid pace and with high accuracy. In this paper, we propose to combine the notions of kernel and possibilistic approach together in a distributed environment provided by Apache™ Hadoop. We integrate this combined notion with map-reduce paradigm of Hadoop and put forth three novel algorithms; Hadoop based possibilistic kernelized rough c-means (HPKRCM), Hadoop based possibilistic kernelized rough fuzzy c-means (HPKRFCM) and Hadoop based possibilistic kernelized rough intuitionistic fuzzy c-means (HPKRIFCM) and study their efficiency in image segmentation. We compare their running times and analyze their efficiencies with the corresponding algorithms from the other three sub families on four different types of images, three different kernels and six different efficiency measures; the Davis Bouldin index (DB), Dunn index (D), alpha index (α), rho index (ρ), alpha star index (α*) and gamma index (γ). Our analysis shows that the hyper-tangent kernel with Hadoop based possibilistic kernelized rough intuitionistic fuzzy c-means is the best one for image segmentation among all these clustering algorithms. Also, the times taken to render segmented images by the proposed algorithms are drastically low in comparison to the other algorithms. The implementations of the algorithms have been carried out in Java and for the proposed algorithms we have used Hadoop framework installed on CentOS. For statistical plotting we have used matplotlib (python library).

Integration of spectral and spatial information via local covariance matrices for segmentation and classification of hyperspectral images

In this work, a novel approach is presented for the feature extraction step in hyperspectral image processing to form more discriminative features between different pixel regions. The proposed method combines both spatial and spectral information, which is very important for segmentation and classification of hyperspectral images. For comparison, five different feature sets are formed using eigen decomposition of local covariance matrices of subcubes located around a pixel of interest in the scene. Subcubes of neighbor pixels are obtained by a windowed structure to expose pattern similarities. As a novel approach, local covariance matrices are computed in eigenspace and proposed feature sets are created after this stage. Before the formation of feature sets in eigenspace, the original input space is transferred to eigenspace by linear and nonlinear manner by principal component analysis (PCA) and its kernelized version (KPCA) and they are used in the experiments comparatively. In the simulations, one hyperspectral scene with ground-truth and one without ground-truth are used for the segmentation and classification tasks. Results of experiments are evaluated with four different unsupervised learning algorithms for data without ground-truth and three different supervised learning algorithms for data with ground-truth comparatively.

Robust Kernel Low-Rank Representation.

Recently, low-rank representation (LRR) has shown promising performance in many real-world applications such as face clustering. However, LRR may not achieve satisfactory results when dealing with the data from nonlinear subspaces, since it is originally designed to handle the data from linear subspaces in the input space. Meanwhile, the kernel-based methods deal with the nonlinear data by mapping it from the original input space to a new feature space through a kernel-induced mapping. To effectively cope with the nonlinear data, we first propose the kernelized version of LRR in the clean data case. We also present a closed-form solution for the resultant optimization problem. Moreover, to handle corrupted data, we propose the robust kernel LRR (RKLRR) approach, and develop an efficient optimization algorithm to solve it based on the alternating direction method. In particular, we show that both the subproblems in our optimization algorithm can be efficiently and exactly solved, and it is guaranteed to obtain a globally optimal solution. Besides, our proposed algorithm can also solve the original LRR problem, which is a special case of our RKLRR when using the linear kernel. In addition, based on our new optimization technique, the kernelization of some variants of LRR can be similarly achieved. Comprehensive experiments on synthetic data sets and real-world data sets clearly demonstrate the efficiency of our algorithm, as well as the effectiveness of RKLRR and the kernelization of two variants of LRR.

Using tensor products to detect unconditional label dependence in multilabel classifications

Multilabel (ML) classification tasks consist of assigning a set of labels to each input. It is well known that detecting label dependencies is crucial in order to improve the performance in ML problems. In this paper, we study a new kernel approach to take into account unconditional label dependence between labels. The aim is to improve the performance measured by a micro-averaged loss function. The core idea is to transform a ML task into a binary classification problem whose inputs are drawn from a tensor space of the original input space and a representation of the labels. In this joint feature space we define a kernel to explicitly involve both labels and object descriptions. In addition to the theoretical contributions, the experimental results of this study provide an interesting conclusion: the performance in terms of Hamming Loss can be improved when unconditional label dependence is considered, as our method does. We report a thoroughly experimentation carried out with real world domains and several synthetic datasets devised to analyze the effect of exploiting label dependence in scenarios with different degrees of dependency.

Bayesian Classifier for Sparsity-Promoting Feature Selection

A Bayesian classifier for sparsity-promoting feature selection is developed in this paper, where a set of nonlinear mappings for the original data is performed as a pre-processing step. The linear classification model with such mappings from the original input space to a nonlinear transformation space can not only construct the nonlinear classification boundary, but also realize the feature selection for the original data. A zero-mean Gaussian prior with Gamma precision and a finite approximation of Beta process prior are used to promote sparsity in the utilization of features and nonlinear mappings in our model, respectively. We derive the Variational Bayesian (VB) inference algorithm for the proposed linear classifier. Experimental results based on the synthetic data set, measured radar data set, high-dimensional gene expression data set, and several benchmark data sets demonstrate the aggressive and robust feature selection capability and comparable classification accuracy of our method comparing with some other existing classifiers.

Nonlinear Metric Learning with Kernel Density Estimation

Metric learning, the task of learning a good distance metric, is a key problem in machine learning with ample applications. This paper introduces a novel framework for nonlinear metric learning, called kernel density metric learning (KDML), which is easy to use and provides nonlinear, probability-based distance measures. KDML constructs a direct nonlinear mapping from the original input space into a feature space based on kernel density estimation. The nonlinear mapping in KDML embodies established distance measures between probability density functions, and leads to accurate classification on datasets for which existing linear metric learning methods would fail. It addresses the severe challenge to distance-based classifiers when features are from heterogeneous domains and, as a result, the Euclidean or Mahalanobis distance between original feature vectors is not meaningful. We also propose two ways to determine the kernel bandwidths, including an adaptive local scaling approach and an integrated optimization algorithm that learns the Mahalanobis matrix and kernel bandwidths together. KDML is a general framework that can be combined with any existing metric learning algorithm. As concrete examples, we combine KDML with two leading metric learning algorithms, large margin nearest neighbors (LMNN) and neighborhood component analysis (NCA). KDML can naturally handle not only numerical features, but also categorical ones, which is rarely found in previous metric learning algorithms. Extensive experimental results on various datasets show that KDML significantly improves existing metric learning algorithms in terms of classification accuracy.

The benefits of modeling slack variables in SVMs.

In this letter, we explore the idea of modeling slack variables in support vector machine (SVM) approaches. The study is motivated by SVM+, which models the slacks through a smooth correcting function that is determined by additional (privileged) information about the training examples not available in the test phase. We take a closer look at the meaning and consequences of smooth modeling of slacks, as opposed to determining them in an unconstrained manner through the SVM optimization program. To better understand this difference we only allow the determination and modeling of slack values on the same information--that is, using the same training input in the original input space. We also explore whether it is possible to improve classification performance by combining (in a convex combination) the original SVM slacks with the modeled ones. We show experimentally that this approach not only leads to improved generalization performance but also yields more compact, lower-complexity models. Finally, we extend this idea to the context of ordinal regression, where a natural order among the classes exists. The experimental results confirm principal findings from the binary case.

SPMoE: a novel subspace-projected mixture of experts model for multi-target regression problems

In this paper, we focus on modeling multi-target regression problems with high-dimensional feature spaces and a small number of instances that are common in many real-life problems of predictive modeling. With the aim of designing an accurate prediction tool, we present a novel mixture of experts (MoE) model called subspace-projected MoE (SPMoE). Training the experts of the SPMoE is done using a boosting-like manner by a combination of ideas from subspace projection method and the negative correlation learning algorithm (NCL). Instead of using whole original input space for training the experts, we develop a new cluster-based subspace projection method to obtain projected subspaces focused on the difficult instances at each step of the boosting approach for training the diverse experts. The experts of the SPMoE are trained on the obtained subspaces using a new NCL algorithm called sequential NCL. The SPMoE is compared with the other ensemble models using three real cases of high-dimensional multi-target regression problems; the electrical discharge machining, energy efficiency and an important problem in the field of operations strategy called the practice---performance problem. The experimental results show that the prediction accuracy of the SPMoE is significantly better than the other ensemble and single models and can be considered to be a promising alternative for modeling the high-dimensional multi-target regression problems.

Bayesian history matching of complex infectious disease models using emulation: a tutorial and a case study on HIV in Uganda.

Advances in scientific computing have allowed the development of complex models that are being routinely applied to problems in disease epidemiology, public health and decision making. The utility of these models depends in part on how well they can reproduce empirical data. However, fitting such models to real world data is greatly hindered both by large numbers of input and output parameters, and by long run times, such that many modelling studies lack a formal calibration methodology. We present a novel method that has the potential to improve the calibration of complex infectious disease models (hereafter called simulators). We present this in the form of a tutorial and a case study where we history match a dynamic, event-driven, individual-based stochastic HIV simulator, using extensive demographic, behavioural and epidemiological data available from Uganda. The tutorial describes history matching and emulation. History matching is an iterative procedure that reduces the simulator's input space by identifying and discarding areas that are unlikely to provide a good match to the empirical data. History matching relies on the computational efficiency of a Bayesian representation of the simulator, known as an emulator. Emulators mimic the simulator's behaviour, but are often several orders of magnitude faster to evaluate. In the case study, we use a 22 input simulator, fitting its 18 outputs simultaneously. After 9 iterations of history matching, a non-implausible region of the simulator input space was identified that was times smaller than the original input space. Simulator evaluations made within this region were found to have a 65% probability of fitting all 18 outputs. History matching and emulation are useful additions to the toolbox of infectious disease modellers. Further research is required to explicitly address the stochastic nature of the simulator as well as to account for correlations between outputs.

Feature selection for noisy variation patterns using kernel principal component analysis

Kernel Principal Component Analysis (KPCA) is a technique widely used to understand and visualize non-linear variation patterns by inverse mapping the projected data from a high-dimensional feature space back to the original input space. Variation patterns often occur in a small number of relevant features out of the overall set of features that are recorded in the data. It is, therefore, crucial to discern this set of relevant features that define the pattern. Here we propose a feature selection procedure that augments KPCA to obtain importance estimates of the features given the noisy training data. Our feature selection strategy involves projecting the data points onto sparse random vectors for calculating the kernel matrix. We then match pairs of such projections, and determine the preimages of the data with and without a feature, thereby trying to identify the importance of that feature. Thus, preimages’ differences within pairs are used to identify the relevant features. An advantage of our method is it can be used with any suitable KPCA algorithm. Moreover, the computations can be parallelized easily leading to significant speedup. We demonstrate our method on several simulated and real data sets, and compare the results to alternative approaches in the literature.