Accuracy Of Individual Models Research Articles

ENSEMBLE OF ADAPTIVE PREDICTORS FOR MULTIVARIATE NONSTATIONARY SEQUENCES AND ITS ONLINE LEARNING

Context. In this research, we explore an ensemble of metamodels that utilizes multivariate signals to generate forecasts. The ensemble includes various traditional forecasting models such as multivariate regression, exponential smoothing, ARIMAX, as well as nonlinear structures based on artificial neural networks, ranging from simple feedforward networks to deep architectures like LSTM and transformers.
 Objective. A goal of this research is to develop an effective method for combining forecasts from multiple models forming metamodels to create a unified forecast that surpasses the accuracy of individual models. We are aimed to investigate the effectiveness of the proposed ensemble in the context of forecasting tasks with nonstationary signals.
 Method. The proposed ensemble of metamodels employs the method of Lagrange multipliers to estimate the parameters of the metamodel. The Kuhn-Tucker system of equations is solved to obtain unbiased estimates using the least squares method. Additionally, we introduce a recurrent form of the least squares algorithm for adaptive processing of nonstationary signals.
 Results. The evaluation of the proposed ensemble method is conducted on a dataset of time series. Metamodels formed by combining various individual models demonstrate improved forecast accuracy compared to individual models. The approach shows effectiveness in capturing nonstationary patterns and enhancing overall forecasting accuracy.
 Conclusions. The ensemble of metamodels, which utilizes multivariate signals for forecast generation, offers a promising approach to achieve better forecasting accuracy. By combining diverse models, the ensemble exhibits robustness to nonstationarity and improves the reliability of forecasts.

Space-dependent turbulence model aggregation using machine learning

Computational models of fluid flows based on the Reynolds-Averaged Navier–Stokes (RANS) equations supplemented with a turbulence model are the gold standard in engineering applications. A plethora of turbulence models and related variants exist, none of which are fully reliable outside the range of flow configurations for which they were calibrated. Thus, the choice of a suitable turbulence closure largely relies on subjective expert judgement and engineering know-how. In this article, we propose a data-driven methodology for combining the solutions of a set of competing turbulence models. The individual model predictions are linearly combined for providing an ensemble solution accompanied by estimates of predictive uncertainty due to the turbulence model choice. First, for a set of training flow configurations we assign high weights to component models in the regions where they perform best, and vice versa, based on a measure of distance between high-fidelity data and individual model predictions. The model weights are then mapped into a space of features, representative of the local flow physics, and regressed using a Random Forests (RF) algorithm. Finally, the RF regressor is used to infer spatial distributions of the model weights for unseen configurations. Predictions of new cases are constructed as a convex linear combination of the underlying models solutions, while the between-model variance provides information about regions of high model uncertainty. The method is demonstrated for a class of flows through the compressor cascade NACA65 V103 at Re≃3×105. The results show that the aggregated solution outperforms the accuracy of individual models for the quantity used to inform the RF regressor, and performs well for other quantities well-correlated with the previous one. The estimated uncertainty intervals are generally consistent with the target high-fidelity data. The present approach then represents a viable methodology for a more objective selection and combination of alternative turbulence models in configurations of interest for engineering practice.

Application of Artificial Intelligence Methods for Predicting the Compressive Strength of Self-Compacting Concrete with Class F Fly Ash.

Replacing a specified quantity of cement with Class F fly ash contributes to sustainable development and reducing the greenhouse effect. In order to use Class F fly ash in self-compacting concrete (SCC), a prediction model that will give a satisfactory accuracy value for the compressive strength of such concrete is required. This paper considers a number of machine learning models created on a dataset of 327 experimentally tested samples in order to create an optimal predictive model. The set of input variables for all models consists of seven input variables, among which six are constituent components of SCC, and the seventh model variable represents the age of the sample. Models based on regression trees (RTs), Gaussian process regression (GPR), support vector regression (SVR) and artificial neural networks (ANNs) are considered. The accuracy of individual models and ensemble models are analyzed. The research shows that the model with the highest accuracy is an ensemble of ANNs. This accuracy expressed through the mean absolute error (MAE) and correlation coefficient (R) criteria is 4.37 MPa and 0.96, respectively. This paper also compares the accuracy of individual prediction models and determines their accuracy. Compared to theindividual ANN model, the more transparent multi-gene genetic programming (MGPP) model and the individual regression tree (RT) model have comparable or better prediction accuracy. The accuracy of the MGGP and RT models expressed through the MAE and R criteria is 5.70 MPa and 0.93, and 6.64 MPa and 0.89, respectively.

Inclusive Multiple Models (IMM) for predicting groundwater levels and treating heterogeneity

An explicit model management framework is introduced for predictive Groundwater Levels (GWL), particularly suitable to Observation Wells (OWs) with sparse and possibly heterogeneous data. The framework implements Multiple Models (MM) under the architecture of organising them at levels, as follows: (i) Level 0: treat heterogeneity in the data, e.g. Self-Organised Mapping (SOM) to classify the OWs; and decide on model structure, e.g. formulate a grey box model to predict GWLs. (ii) Level 1: construct MMs, e.g. two Fuzzy Logic (FL) and one Neurofuzzy (NF) models. (iii) Level 2: formulate strategies to combine the MM at Level 1, for which the paper uses Artificial Neural Networks (Strategy 1) and simple averaging (Strategy 2). Whilst the above model management strategy is novel, a critical view is presented, according to which modelling practices are: Inclusive Multiple Modelling (IMM) practices contrasted with existing practices, branded by the paper as Exclusionary Multiple Modelling (EMM). Scientific thinking over IMMs is captured as a framework with four dimensions: Model Reuse (MR), Hierarchical Recursion (HR), Elastic Learning Environment (ELE) and Goal Orientation (GO) and these together make the acronym of RHEO. Therefore, IMM-RHEO is piloted in the aquifer of Tabriz Plain with sparse and possibly heterogeneous data. The results provide some evidence that (i) IMM at two levels improves on the accuracy of individual models; and (ii) model combinations in IMM practices bring ‘model-learning’ into fashion for learning with the goal to explain baseline conditions and impacts of subsequent management changes.

Ensembles of ecosystem service models can improve accuracy and indicate uncertainty

Many ecosystem services (ES) models exist to support sustainable development decisions. However, most ES studies use only a single modelling framework and, because of a lack of validation data, rarely assess model accuracy for the study area. In line with other research themes which have high model uncertainty, such as climate change, ensembles of ES models may better serve decision-makers by providing more robust and accurate estimates, as well as provide indications of uncertainty when validation data are not available. To illustrate the benefits of an ensemble approach, we highlight the variation between alternative models, demonstrating that there are large geographic regions where decisions based on individual models are not robust. We test if ensembles are more accurate by comparing the ensemble accuracy of multiple models for six ES against validation data across sub-Saharan Africa with the accuracy of individual models. We find that ensembles are better predictors of ES, being 5.0–6.1% more accurate than individual models. We also find that the uncertainty (i.e. variation among constituent models) of the model ensemble is negatively correlated with accuracy and so can be used as a proxy for accuracy when validation is not possible (e.g. in data-deficient areas or when developing scenarios). Since ensembles are more robust, accurate and convey uncertainty, we recommend that ensemble modelling should be more widely implemented within ES science to better support policy choices and implementation.

Consensus Ensemble System for Traffic Flow Prediction

Traffic flow prediction is a key component of an intelligent transportation system. Accurate traffic flow prediction provides a foundation for other tasks, such as signal coordination and travel time forecasting. There are many known methods in literature for the short-term traffic flow prediction problem, but their efficacy depends heavily on the traffic characteristics. It is difficult, if not impossible, to pick a single method that works well over time. In this paper, we present an automated framework to address this practical issue. Instead of selecting a single method, we combine predictions from multiple methods to generate a consensus traffic flow prediction. We propose an ensemble learning model that exploits the temporal characteristics of the data, and balances the accuracy of individual models and their mutual dependence through a covariance-regularizer. We additionally use a pruning scheme to remove anomalous individual predictions. We apply our proposed model to multi-step-ahead arterial roadway flow prediction. In tests, our method consistently outperforms recently published ensemble prediction methods based on ridge regression and lasso. Our method also produces steady results even when the standalone models and other ensemble methods make wildly exaggerated predictions.

Estimation of Symptom Severity During Chemotherapy From Passively Sensed Data: Exploratory Study.

BackgroundPhysical and psychological symptoms are common during chemotherapy in cancer patients, and real-time monitoring of these symptoms can improve patient outcomes. Sensors embedded in mobile phones and wearable activity trackers could be potentially useful in monitoring symptoms passively, with minimal patient burden.ObjectiveThe aim of this study was to explore whether passively sensed mobile phone and Fitbit data could be used to estimate daily symptom burden during chemotherapy.MethodsA total of 14 patients undergoing chemotherapy for gastrointestinal cancer participated in the 4-week study. Participants carried an Android phone and wore a Fitbit device for the duration of the study and also completed daily severity ratings of 12 common symptoms. Symptom severity ratings were summed to create a total symptom burden score for each day, and ratings were centered on individual patient means and categorized into low, average, and high symptom burden days. Day-level features were extracted from raw mobile phone sensor and Fitbit data and included features reflecting mobility and activity, sleep, phone usage (eg, duration of interaction with phone and apps), and communication (eg, number of incoming and outgoing calls and messages). We used a rotation random forests classifier with cross-validation and resampling with replacement to evaluate population and individual model performance and correlation-based feature subset selection to select nonredundant features with the best predictive ability.ResultsAcross 295 days of data with both symptom and sensor data, a number of mobile phone and Fitbit features were correlated with patient-reported symptom burden scores. We achieved an accuracy of 88.1% for our population model. The subset of features with the best accuracy included sedentary behavior as the most frequent activity, fewer minutes in light physical activity, less variable and average acceleration of the phone, and longer screen-on time and interactions with apps on the phone. Mobile phone features had better predictive ability than Fitbit features. Accuracy of individual models ranged from 78.1% to 100% (mean 88.4%), and subsets of relevant features varied across participants.ConclusionsPassive sensor data, including mobile phone accelerometer and usage and Fitbit-assessed activity and sleep, were related to daily symptom burden during chemotherapy. These findings highlight opportunities for long-term monitoring of cancer patients during chemotherapy with minimal patient burden as well as real-time adaptive interventions aimed at early management of worsening or severe symptoms.

Aggregated Versus Individual Land-Use Models: Modeling Spatial Autocorrelation to Increase Predictive Accuracy

The objective of this paper is to compare the predictive accuracy of individual and aggregated econometric models of land-use choices. We argue that modeling spatial autocorrelation is a comparative advantage of aggregated models due to the smaller number of observation and the linearity of the outcome. The question is whether modeling spatial autocorrelation in aggregated models is able to provide better predictions than individual ones. We consider a complete partition of space with four land-use classes: arable, pasture, forest, and urban. We estimate and compare the predictive accuracies of individual models at the plot level (514,074 observations) and of aggregated models at a regular 12 × 12 km grid level (3,767 observations). Our results show that modeling spatial autocorrelation allows to obtain more accurate predictions at the aggregated level when the appropriate predictors are used.

Bayesian combination for inflation forecasts: The effects of a prior based on central banks’ estimates

Typically, central banks use a variety of individual models (or a combination of models) when forecasting inflation rates. Most of these require excessive amounts of data, time, and computational power, all of which are scarce when monetary authorities meet to decide over policy interventions. In this paper we use a rolling Bayesian combination technique that considers inflation estimates by the staff of the Central Bank of Colombia during 2002–2011 as prior information. Our results show that: (1) the accuracy of individual models is improved by using a Bayesian shrinkage methodology, and (2) priors consisting of staff estimates outperform all other priors that comprise equal or zero vector weights. Consequently, our model provides readily available forecasts that exceed all individual models in terms of forecasting accuracy at every evaluated horizon.

The dominance-based rough set approach to cylindrical plunge grinding process diagnosis

The dominance-based rough set approach is proposed as a methodology for plunge grinding process diagnosis. The process is analyzed and next its diagnosis is considered as a multi-criteria decision making problem based on the modelling of relationships between different process states and their symptoms using a set of rules induced from measured process data. The development of the diagnostic system is characterized by three phases. Firstly, the process experimental data is prepared in the form of a decision table. Using selected methods of signal processing, each process running is described by 17 process state features (condition attributes) and 5 criteria evaluating process state and results (decision attributes). The semantic correlation between all the attributes is modelled. Next, the phase of condition attributes selection and knowledge extraction are strictly integrated with the phase of the model evaluation using an iterative approach. After each loop of the iterative feature selection procedure the induction of rules is conducted using the VC-DomLEM algorithm. The classification capability of the induced rules is carried out using the leave-one-out method and a set of measures. The classification accuracy of individual models is in the range of 80.77–98.72 %. The induced set of rules constitutes a classifier for an assessment of new process run cases.

How accurately can we predict the melting points of drug-like compounds?

This article contributes a highly accurate model for predicting the melting points (MPs) of medicinal chemistry compounds. The model was developed using the largest published data set, comprising more than 47k compounds. The distributions of MPs in drug-like and drug lead sets showed that >90% of molecules melt within [50,250]°C. The final model calculated an RMSE of less than 33 °C for molecules from this temperature interval, which is the most important for medicinal chemistry users. This performance was achieved using a consensus model that performed calculations to a significantly higher accuracy than the individual models. We found that compounds with reactive and unstable groups were overrepresented among outlying compounds. These compounds could decompose during storage or measurement, thus introducing experimental errors. While filtering the data by removing outliers generally increased the accuracy of individual models, it did not significantly affect the results of the consensus models. Three analyzed distance to models did not allow us to flag molecules, which had MP values fell outside the applicability domain of the model. We believe that this negative result and the public availability of data from this article will encourage future studies to develop better approaches to define the applicability domain of models. The final model, MP data, and identified reactive groups are available online at http://ochem.eu/article/55638.

Forecasting malaria: ensemble modelling and predicting the impact of interventions

Background Over the past decade many new mathematical models for malaria have been developed to assess the impact of interventions. Given sufficient data to inform them, these models can be useful tools for policy planning and forecasting the state of malaria in an affected region or to investigate important questions pertaining to controlling and eliminating malaria. But how accurate are these models? And importantly, how do the predictions from different models compare, and what are the uncertainties surrounding their predictions? Materials and methods We present the Swiss TPH malaria modelling group’s initial approach for combining and evaluating a small ensemble of models. The ensemble comprises 14 individual-based stochastic simulation models of P. falciparum dynamics, with varied assumptions about immune decay, transmission heterogeneity, and access to treatment. Models parameters were fit to an extensive library of field data. We present a principled methodology for selecting and using an ensemble of models, which takes the accuracy and uncertainty of single-model predictions into account. The accuracy of individual models, evaluated against clinical data, is used to weight their predictions. The methodology can be applied to an ensemble composed of structural different models, potentially combining the Swiss TPH models with models from other groups. The uncertainty of the stochastic models in the ensemble is used to derive a range of reasonable values for model outcomes, such as predictions of the impact of interventions. Results The proposed methodology has been piloted with an ensemble of Swiss TPH models to investigate the preerythrocytic vaccine RTS,S. Clinical trial data informs the models of the vaccine profile and the ensemble provides predictions of the impact of a pre-erythrocytic vaccine when scaled to national levels. Conclusions This work highlights the requirements, advantages and challenges in using ensemble models for prediction of Malaria. In particular the methodology enables presentation of uncertainty in results, and to predict the health impact of new interventions where their precise influence in the model is unknown but can be discerned from clinical field trials. The methodology is applicable not only to the Swiss TPH models but could be used to make predictions in conjunction with other models.

Sentiment classification using the theory of ANNs

Sentiment classification has attracted increasing interest from natural language processing. The goal of sentiment classification is to automatically identify whether a given piece of text expresses positive or negative opinion on a topic of interest. This paper presents the standpoint that uses individual model (i-model) based on artificial neural networks (ANNs) to determine text sentiment classification. The individual model consists of sentimental features, feature weight and prior knowledge base. During the training process, i-model that makes right sentimental judgment will correct those are wrong, to make more accurate prediction of text sentiment polarity. Experimental results show that the accuracy of individual model is higher than that of support vector machines (SVMs) and hidden Markov model (HMM) classifiers on movie review corpus.

Predictive weighting for cluster ensembles

AbstractAn ensemble of regression models predicts by taking a weighted average of the predictions made by individual models. Calculating the weights such that they reflect the accuracy of individual models (post processing the ensemble) has been shown to increase the ensemble's accuracy. However, post processing cluster ensembles has not received as much attention because of the inherent difficulty in assessing the accuracy of an individual cluster model. By enforcing the notion that clusters must be ‘predictable’, this paper suggests a means of implicitly post processing cluster ensembles by drawing analogies with regression post processing techniques. The product of the post processing procedure is an intelligently weighted co‐occurrence matrix. A new technique, similarity‐based k‐means (SBK), is developed to split this matrix into clusters. The results using three real life datasets underpinned by chemical and biological phenomena show that splitting an intelligently weighted co‐occurrence matrix gives accuracy that approaches supervised classification methods. Copyright © 2007 John Wiley & Sons, Ltd.