Model Input Space Research Articles

Bayesian optimization approach to quantify the effect of input parameter uncertainty on predictions of numerical physics simulations

An understanding of how input parameter uncertainty in the numerical simulation of physical models leads to simulation output uncertainty is a challenging task. Common methods for quantifying output uncertainty, such as performing a grid or random search over the model input space, are computationally intractable for a large number of input parameters represented by a high-dimensional input space. It is, therefore, generally unclear as to whether a numerical simulation can reproduce a particular outcome (e.g., a set of experimental results) with a plausible set of model input parameters. Here, we present a method for efficiently searching the input space using Bayesian optimization to minimize the difference between the simulation output and a set of experimental results. Our method allows explicit evaluation of the probability that the simulation can reproduce the measured experimental results in the region of input space defined by the uncertainty in each input parameter. We apply this method to the simulation of charge-carrier dynamics in the perovskite semiconductor methyl-ammonium lead iodide (MAPbI3), which has attracted attention as a light harvesting material in solar cells. From our analysis, we conclude that the formation of large polarons, quasiparticles created by the coupling of excess electrons or holes with ionic vibrations, cannot explain the experimentally observed temperature dependence of electron mobility.

Long Short-term Memory-Based Prediction of the Spread of Influenza-Like Illness Leveraging Surveillance, Weather, and Twitter Data: Model Development and Validation.

The potential to harness the plurality of available data in real time along with advanced data analytics for the accurate prediction of influenza-like illness (ILI) outbreaks has gained significant scientific interest. Different methodologies based on the use of machine learning techniques and traditional and alternative data sources, such as ILI surveillance reports, weather reports, search engine queries, and social media, have been explored with the ultimate goal of being used in the development of electronic surveillance systems that could complement existing monitoring resources. The scope of this study was to investigate for the first time the combined use of ILI surveillance data, weather data, and Twitter data along with deep learning techniques toward the development of prediction models able to nowcast and forecast weekly ILI cases. By assessing the predictive power of both traditional and alternative data sources on the use case of ILI, this study aimed to provide a novel approach for corroborating evidence and enhancing accuracy and reliability in the surveillance of infectious diseases. The model's input space consisted of information related to weekly ILI surveillance, web-based social (eg, Twitter) behavior, and weather conditions. For the design and development of the model, relevant data corresponding to the period of 2010 to 2019 and focusing on the Greek population and weather were collected. Long short-term memory (LSTM) neural networks were leveraged to efficiently handle the sequential and nonlinear nature of the multitude of collected data. The 3 data categories were first used separately for training 3 LSTM-based primary models. Subsequently, different transfer learning (TL) approaches were explored with the aim of creating various feature spaces combining the features extracted from the corresponding primary models' LSTM layers for the latter to feed a dense layer. The primary model that learned from weather data yielded better forecast accuracy (root mean square error [RMSE]=0.144; Pearson correlation coefficient [PCC]=0.801) than the model trained with ILI historical data (RMSE=0.159; PCC=0.794). The best performance was achieved by the TL-based model leveraging the combination of the 3 data categories (RMSE=0.128; PCC=0.822). The superiority of the TL-based model, which considers Twitter data, weather data, and ILI surveillance data, reflects the potential of alternative public sources to enhance accurate and reliable prediction of ILI spread. Despite its focus on the use case of Greece, the proposed approach can be generalized to other locations, populations, and social media platforms to support the surveillance of infectious diseases with the ultimate goal of reinforcing preparedness for future epidemics.

Extending the Input and Transformation Space of Different TP Models: An LMI-based Feasibility Analysis

Extending the Input and Transformation Space of Different TP Models: An LMI-based Feasibility Analysis

Design of Computational Models for Hydroturbine Units Based on a Nonparametric Regression Approach with Adaptation by Evolutionary Algorithms

This article deals with the problem of designing regression models for evaluating the parameters of the operation of complex technological equipment—hydroturbine units. A promising approach to the construction of regression models based on nonparametric Nadaraya–Watson kernel estimates is considered. A known problem in applying this approach is to determine the effective values of kernel-smoothing coefficients. Kernel-smoothing factors significantly impact the accuracy of the regression model, especially under conditions of variability of noise and parameters of samples in the input space of models. This fully corresponds to the characteristics of the problem of estimating the parameters of hydraulic turbines. We propose to use the evolutionary genetic algorithm with an addition in the form of a local-search stage to adjust the smoothing coefficients. This ensures the local convergence of the tuning procedure, which is important given the high sensitivity of the quality criterion of the nonparametric model. On a set of test problems, the results were obtained showing a reduction in the modeling error by 20% and 28% for the methods of adjusting the coefficients by the standard and hybrid genetic algorithms, respectively, in comparison with the case of an arbitrary choice of the values of such coefficients. For the task of estimating the parameters of the operation of a hydroturbine unit, a number of promising approaches to constructing regression models based on artificial neural networks, multidimensional adaptive splines, and an evolutionary method of genetic programming were included in the research. The proposed nonparametric approach with a hybrid smoothing coefficient tuning scheme was found to be most effective with a reduction in modeling error of about 5% compared with the best of the alternative approaches considered in the study, which, according to the results of numerical experiments, was the method of multivariate adaptive regression splines.

CrowdTeacher: Robust Co-teaching with Noisy Answers and Sample-Specific Perturbations for Tabular Data.

Samples with ground truth labels may not always be available in numerous domains. While learning from crowdsourcing labels has been explored, existing models can still fail in the presence of sparse, unreliable, or differing annotations. Co-teaching methods have shown promising improvements for computer vision problems with noisy labels by employing two classifiers trained on each others' confident samples in each batch. Inspired by the idea of separating confident and uncertain samples during the training process, we extend it for the crowdsourcing problem. Our model, CrowdTeacher, uses the idea that perturbation in the input space model can improve the robustness of the classifier for noisy labels. Treating crowdsourcing annotations as a source of noisy labeling, we perturb samples based on the certainty from the aggregated annotations. The perturbed samples are fed to a Co-teaching algorithm tuned to also accommodate smaller tabular data. We showcase the boost in predictive power attained using CrowdTeacher for both synthetic and real datasets across various label density settings. Our experiments reveal that our proposed approach beats baselines modeling individual annotations and then combining them, methods simultaneously learning a classifier and inferring truth labels, and the Co-teaching algorithm with aggregated labels through common truth inference methods.

Urban Crime Risk Prediction Using Point of Interest Data

Geographical information systems have found successful applications to prediction and decision-making in several areas of vital importance to contemporary society. This article demonstrates how they can be combined with machine learning algorithms to create crime prediction models for urban areas. Selected point of interest (POI) layers from OpenStreetMap are used to derive attributes describing micro-areas, which are assigned crime risk classes based on police crime records. POI attributes then serve as input attributes for learning crime risk prediction models with classification learning algorithms. The experimental results obtained for four UK urban areas suggest that POI attributes have high predictive utility. Classification models using these attributes, without any form of location identification, exhibit good predictive performance when applied to new, previously unseen micro-areas. This makes them capable of crime risk prediction for newly developed or dynamically changing neighborhoods. The high dimensionality of the model input space can be considerably reduced without predictive performance loss by attribute selection or principal component analysis. Models trained on data from one area achieve a good level of prediction quality when applied to another area, which makes it possible to transfer or combine crime risk prediction models across different urban areas.

How does combinatorial testing perform in the real world: an empirical study

Studies have shown that combinatorial testing (CT) can be effective for detecting faults in software systems. By focusing on the interactions between different factors of a system, CT shows its potential for detecting faults, especially those that can be revealed only by the specific combinations of values of multiple factors (multi-factor faults). However, is CT practical enough to be applied in the industry? Can it be more effective than other industry-favored techniques? Are there any challenges when applying CT in practice? These research questions remain in the context of industrial settings. In this paper, we present an empirical study of CT on five industrial systems with real faults. The details of the input space model (ISM) construction, such as factor identification and value assignment, are included. We compared the faults detected by CT with those detected by the in-house testing teams using other methods, and the results suggest that despite some challenges, CT is an effective technique to detect real faults, especially multi-factor faults, of software systems in industrial settings. Observations and lessons learned are provided to further improve the fault detection effectiveness and overcome various challenges.

Sensitivity analysis based dimension reduction of multiscale models

In this paper, the sensitivity analysis of a single scale model is employed in order to reduce the input dimensionality of the related multiscale model, in this way, improving the efficiency of its uncertainty estimation. The approach is illustrated with two examples: a reaction model and the standard Ornstein–Uhlenbeck process. Additionally, a counterexample shows that an uncertain input should not be excluded from uncertainty quantification without estimating the response sensitivity to this parameter. In particular, an analysis of the function defining the relation between single scale components is required to understand whether single scale sensitivity analysis can be used to reduce the dimensionality of the overall multiscale model input space.

Chained ensemble classifier for image annotation

Image annotation is the task of assigning keywords or identifiers to images, holistically or in specific regions. These keywords serve as descriptors of high-level semantics to facilitate retrieval and organization of visual information. It plays an important role in content-based image understanding, as well as in areas such as object recognition in robotics, content-based image searching and knowledge extraction. Automatic image annotation is usually approached by means of supervised classification, where a set of previously annotated images is required to train a learning algorithm that later predicts the labels for new images. This paper proposes a novel ensemble classifier for the supervised image annotation task inspired in chain classifiers. In the proposed approach a chain of individual classifiers is build, where each classifier is trained by using a different modality. In addition, the input space of models in the chain is augmented with the output of the preceding model in the sequence. Each model in the chain deals with the same classification problem, making the proposed method an ensemble model build from multimodal data. To the best of our knowledge, chain classifiers have not been used in this particular setting. Experimental results in a challenging image collection show that the proposed method is able to obtain an f −value superior to 0.5, outperforming related work.

Validating Dynamic Engineering Models Under Uncertainty

Validating dynamic engineering models is critically important in practical applications by assessing the agreement between simulation results and experimental observations. Though significant progresses have been made, the existing metrics lack the capability of managing uncertainty in both simulations and experiments. In addition, it is challenging to validate a dynamic model aggregately over both the time domain and a model input space with data at multiple validation sites. To overcome these difficulties, this paper presents an area-based metric to systemically handle uncertainty and validate computational models for dynamic systems over an input space by simultaneously integrating the information from multiple validation sites. To manage the complexity associated with a high-dimensional data space, eigenanalysis is performed for the time series data from simulations at each validation site to extract the important features. A truncated Karhunen–Loève (KL) expansion is then constructed to represent the responses of dynamic systems, resulting in a set of uncorrelated random coefficients with unit variance. With the development of a hierarchical data-fusion strategy, probability integral transform (PIT) is then employed to pool all the resulting random coefficients from multiple validation sites across the input space into a single aggregated metric. The dynamic model is thus validated by calculating the cumulative area difference of the cumulative density functions. The proposed model validation metric for dynamic systems is illustrated with a mathematical example, a supported beam problem with stochastic loads, and real data from the vehicle occupant-restraint system.

Improving scenario discovery by bagging random boxes

Scenario discovery is a model-based approach to scenario development under deep uncertainty. Scenario discovery relies on the use of statistical machine learning algorithms. The most frequently used algorithm is the Patient Rule Induction Method (PRIM). This algorithm identifies regions in an uncertain model input space that are highly predictive of model outcomes that are of interest. To identify these regions, PRIM uses a hill-climbing optimization procedure. This suggests that PRIM can suffer from the usual defects of hill climbing optimization algorithms, including local optima, plateaus, and ridges and valleys. In case of PRIM, these problems are even more pronounced when dealing with heterogeneously typed data. Drawing inspiration from machine learning research on random forests, we present an improved version of PRIM. This improved version is based on the idea of performing multiple PRIM analyses based on randomly selected features and combining these results using a bagging technique. The efficacy of the approach is demonstrated using three cases. Each of the cases has been published before and used PRIM. We compare the results found using PRIM with the results found using the improved version of PRIM. We find that the improved version is more robust to new data, can better cope with heterogeneously typed data, and is less prone to overfitting.

Improving scenario discovery for handling heterogeneous uncertainties and multinomial classified outcomes

Scenario discovery is a novel model-based approach to scenario development in the presence of deep uncertainty. Scenario discovery frequently relies on the Patient Rule Induction Method (PRIM). PRIM identifies regions in the model input space that are highly predictive of producing model outcomes that are of interest. To identify these, PRIM uses a lenient hill climbing optimization procedure. PRIM struggles when confronted with cases where the uncertain factors are a mix of data types, and can be used only for binary classifications. We compare two more lenient objective functions which both address the first problem, and an alternative objective function using Gini impurity which addresses the second problem. We assess the efficacy of the modification using previously published cases. Both modifications are effective. The more lenient objective functions produce better descriptions of the data, while the Gini impurity objective function allows PRIM to be used when handling multinomial classified data.

Discovering an active subspace in a single‐diode solar cell model

AbstractPredictions from science and engineering models depend on the values of the model's input parameters. As the number of parameters increases, algorithmic parameter studies such as optimization and uncertainty quantification require many more model evaluations. One way to combat this curse of dimensionality is to seek an alternative parameterization with fewer variables that produces comparable predictions. The active subspace is a low‐dimensional linear subspace defined by important directions in the model's input space; input perturbations along these directions change the model's prediction more, on average, than perturbations orthogonal to the important directions. We describe a method for checking if a model admits an exploitable active subspace and apply this method to a single‐diode solar cell model with five input parameters. We find that the maximum power of the solar cell has a dominant one‐dimensional active subspace, which enables us to perform thorough parameter studies in one dimension instead of five.

Advantages and applications of transforming empirical model input space with dimensional models

Model-based calibration of steady-state engine operation is commonly performed with highly parameterized empirical models that are accurate but not very robust, particularly when predicting highly nonlinear responses such as diesel smoke emissions. To address this problem, and to boost the accuracy of more robust non-parametric methods to the same level, GT-Power was used to transform the empirical model input space into multiple input spaces that simplified the input-output relationship and improved the accuracy and robustness of smoke predictions made by three commonly used empirical modeling methods: Multivariate Regression, Neural Networks and the k-Nearest Neighbor method. The availability of multiple input spaces allowed the development of two committee techniques: a ‘Simple Committee’ technique that used averaged predictions from a set of 10 pre-selected input spaces chosen by the training data and the “Minimum Variance Committee” technique where the input spaces for each prediction were chosen on the basis of disagreement between the three modeling methods. This latter technique equalized the performance of the three modeling methods. The successively increasing improvements resulting from the use of a single best transformed input space (Best Combination Technique), Simple Committee Technique and Minimum Variance Committee Technique were verified with hypothesis testing. The transformed input spaces were also shown to improve outlier detection and to improve k-Nearest Neighbor performance when predicting dynamic emissions with steady-state training data. An unexpected finding was that the benefits of input space transformation were unaffected by changes in the hardware or the calibration of the underlying GT-Power model.

Analysis and prediction of transient opacity spikes using dimensional modeling

Dimensional modeling, GT-Power in particular, has been used for two related purposes—to quantify and understand the inaccuracies of transient engine flow estimates that cause transient smoke spikes and to improve empirical models of opacity or particulate matter used for engine calibration. It has been proposed by dimensional modeling that exhaust gas recirculation flow rate was significantly underestimated and volumetric efficiency was overestimated by the electronic control module during the turbocharger lag period of an electronically controlled heavy duty diesel engine. Factoring in cylinder-to-cylinder variation, it has been shown that the electronic control module estimated fuel–Oxygen ratio was lower than actual by up to 35% during the turbocharger lag period but within 2% of actual elsewhere, thus hindering fuel–Oxygen ratio limit–based smoke control. The dimensional modeling of transient flow was enabled with a new method of simulating transient data in which the manifold pressures and exhaust gas recirculation system flow resistance, characterized as a function of exhaust gas recirculation valve position at each measured transient data point, were replicated by quasi-static or transient simulation to predict engine flows. Dimensional modeling was also used to transform the engine operating parameter model input space to a more fundamental lower dimensional space so that a nearest neighbor approach could be used to predict smoke emissions. This new approach, intended for engine calibration and control modeling, was termed the “nonparametric reduced dimensionality” approach. It was used to predict federal test procedure cumulative particulate matter within 7% of measured value, based solely on steady-state training data. Very little correlation between the model inputs in the transformed space was observed as compared to the engine operating parameter space. This more uniform, smaller, shrunken model input space might explain how the nonparametric reduced dimensionality approach model could successfully predict federal test procedure emissions when roughly 40% of all transient points were classified as outliers as per the steady-state training data.

Texture Analysis Based Damage Detection of Ageing Infrastructural Elements

Abstract: To make visual data a part of quantitative assessment for infrastructure maintenance management, it is important to develop computer‐aided methods that demonstrate efficient performance in the presence of variability in damage forms, lighting conditions, viewing angles, and image resolutions taking into account the luminous and chromatic complexities of visual data. This article presents a semi‐automatic, enhanced texture segmentation approach to detect and classify surface damage on infrastructure elements and successfully applies them to a range of images of surface damage. The approach involves statistical analysis of spatially neighboring pixels in various color spaces by defining a feature vector that includes measures related to pixel intensity values over a specified color range and statistics derived from the Grey Level Co‐occurrence Matrix calculated on a quantized grey‐level scale. Parameter optimized non‐linear Support Vector Machines are used to classify the feature vector. A Custom‐Weighted Iterative model and a 4‐Dimensional Input Space model are introduced. Receiver Operating Characteristics are employed to assess and enhance the detection efficiency under various damage conditions.

Improving the efficiency of mean sound level calculations through importance sampling.

Outdoor sound levels vary in space and time due to randomness in acoustic source emissions and environmental effects on wave propagation. Sound levels may also appropriately be viewed as random due to uncertainties in the modeling process. To assess the impact of the randomness on sound level predictions, Monte Carlo sampling of the model input space can be used; that is, equally likely sets of model inputs are generated, to which a sound propagation model is applied repeatedly. To improve the efficiency of the calculations, it is highly desirable to sample primarily the acoustical frequency bands and environmental conditions (such as downwind propagation and hard ground surfaces) contributing most strongly to the sound level. Importance sampling techniques, which adaptively focus sampling effort on the most important parts of the input parameter space, can be used for this purpose. This paper explores application of importance sampling to prediction of sound level statistics. The technique is found to be...

Bayesian Statistic Based Multivariate Gaussian Process Approach for Offline/Online Fatigue Crack Growth Prediction

Offline and online fatigue crack growth prediction of Aluminum 2024 compact-tension (CT) specimens under variable loading has been modeled, using multivariate Gaussian Process (GP) technique. The GP model is a Bayesian statistic stochastic model that projects the input space to an output space by probabilistically inferring the underlying nonlinear function. For the offline prediction, the input space of the model is trained with parameters that affect fatigue crack growth, such as the number of fatigue cycles, minimum load, maximum load, and load ratio. For the online prediction, the model input space is trained using piezoelectric sensor signal features rather than training the input space with loading parameters, which are difficult to measure in a real time scenario. Principal Component Analysis (PCA) is used to extract the principal features from sensor signals. In both the offline and online case, the output space is trained with known associated crack lengths or crack growth rates. Once the GP model is trained, a new output space for which the corresponding crack length or crack growth rate is not known, is predicted using the trained GP model. The models are validated through several numerical examples.

Inference in ensemble experiments

We consider inference based on ensembles of climate model evaluations, and contrast the Monte Carlo approach, in which the evaluations are selected at random from the model-input space, with a more overtly statistical approach using emulators and experimental design.

A robust algorithm for automatic development of neural-network models for microwave applications

For the first time, we propose a robust algorithm for automating the neural-network-based RF/microwave model development process. Starting with zero amount of training data and then proceeding with neural-network training in a stage-wise manner, the algorithm can automatically produce a neural model that meets the user-desired accuracy. In each stage, the algorithm utilizes neural-network error criteria to determine additional training/validation samples required and their location in model input space. The algorithm dynamically generates these new data samples during training, by automatic driving of simulation tools (e.g., OSA90, Ansoft-HFSS, Agilent-ADS). Initially, fewer hidden neurons are used, and the algorithm adjusts the neural-network size whenever it detects under-learning. Our technique integrates all the subtasks involved in neural modeling, thereby facilitating a more efficient and automated model development framework. It significantly reduces the intensive human effort demanded by the conventional step-by-step neural modeling approach. The algorithm inherently distinguishes nonlinear and smooth regions of model behavior and uses relatively fewer samples in smooth subregions. It automatically deals with large data errors that can occur during dynamic sampling by using a Huber quasi-Newton technique. The algorithm is demonstrated through practical microwave device and circuit examples.