Nonparametric Predictive Model Research Articles

Lazy Multi-Label Classification algorithms based on Non-Parametric Predictive Inference

Multi-Label Classification (MLC) extends standard classification in the sense that an instance might belong to multiple labels simultaneously. Many lazy approaches to MLC have been proposed so far. The majority of them, to classify an instance, use statistical estimators from the neighboring instances based on classical probability theory. In this work, we propose lazy algorithms for MLC that employ the Non-Parametric Predictive Inference Model (NPI-M) for the statistical estimators based on the neighboring instances. It is shown that our proposed lazy MLC algorithms are more suitable to tackle the class-imbalance problem that usually arises in MLC, especially when data contain label noise. This issue is corroborated via an exhaustive experimental analysis.

Model Predictive Current Control for PMSM Drives Based on Nonparametric Prediction Model

Model Predictive Current Control for PMSM Drives Based on Nonparametric Prediction Model

The improved quantum genetic algorithm with its application in fault diagnosis

In this paper, by analyzing the characteristics of the simulated annealing algorithm (SA) and the real double-chain coding target gradient quantum genetic algorithm (DCQGA), the real double-chain coding target gradient quantum genetic simulated annealing algorithm (DCQGSAA) is proposed. Because the performance of LSSVR is extraordinarily sensitive to its key parameters, the proposed algorithm is used to optimize these parameters, then a hybrid non-parametric prediction model is put forward. This model is used in fault prediction of liquid rocket engine thrust. The simulation results show that the proposed model is effective for small samples and multi-dimensional fault prediction.

Prediction of Service Life of Thermoplastic Road Markings on Expressways

Currently, historical data and on-site surveys—particularly in the context of China—are heavily relied upon to determine the best time to maintain expressway road markings. This study aims to determine what influences the service life of thermoplastic road markings on expressways in Shandong Province, China, while considering both those motorways’ unique characteristics and the local environment. Additionally, a scientific evaluation of the road markings’ retroreflective coefficient’s decay pattern will be undertaken. We collected the retroreflective data for twelve consecutive months regarding the thermoplastic road markings on five expressways and potential influencing factors such as age of marking and annual average daily traffic. The service life of the markings was forecast using a multiple linear regression. Dominance analysis was used to quantitatively analyze each explanatory factor’s impact on the service life of the markings, and statistically significant variables were also found. Using LightGBM, a machine learning technique, a nonparametric prediction model was also created based on examining the relevance of influencing elements. The modeling results show that LightGBM generates an R2 of 0.942, implying that it offers better interpretability and higher accuracy than the regression-based approach. Additionally, LightGBM outperforms MLR according to final validation accuracies, with a score of 95.02% or more than 8% that of MLR. The results are useful for expressway marking upkeep and for driving safety.

Data-driven nonlinear predictive control for feedback linearizable systems

We present a data-driven nonlinear predictive control approach for the class of discrete-time multi-input multi-output feedback linearizable nonlinear systems. The scheme uses a non-parametric predictive model based only on input and noisy output data along with a set of basis functions that approximate the unknown nonlinearities. Despite the noisy output data as well as the mismatch caused by the use of basis functions, we show that the proposed multi-step robust data-driven nonlinear predictive control scheme is recursively feasible and renders the closed-loop system practically exponentially stable. We illustrate our results on a model of a fully-actuated double inverted pendulum.

Optimizing the quality control of multivariate processes under an improved Mahalanobis–Taguchi system

Quality characteristics in manufacturing are correlated and do not follow a normal distribution. This study proposes a quality control method for multivariate manufacturing processes that are based on an improved Mahalanobis–Taguchi System (IMTS). The MTS has no data distribution assumptions and identifies anomalies through the Mahalanobis distance (MD). However, a covariance distance can consider the correlation between variables. Further, to address the shortcomings of the MTS in feature selection and threshold determination. A joint optimization model is proposed in this paper. Under this approach, the IMTS is employed to perform composite analyses on multiple quality characteristics and reduce dimensionality to identify abnormalities and the key quality characteristics that lead to anomalies. Further, various models are compared to construct the optimal non-parametric prediction models for each key quality characteristic. Finally, a conceptual model of process parameter optimization is proposed, which improves the Taguchi method to obtain the optimal combination of process parameters and their importance ranking, as the basis for process adjustment. By applying the proposed method, results show that the IMTS has an abnormality identification rate of 99.5%, which is higher than other methods such as MTS, support vector machine (SVM), back propagation neural network (BPNN), fast correlation-based filter solution SVM (FCBF-SVM) and sequential backward selection BPNN (SBS-BPNN). The dimensionality reduction rate is 0.5, which is higher than MTS, SVM, BPNN, and SBS-BPNN methods. The random forest (RF) algorithm is used for accurate predictions of all five key quality characteristics, the improved Taguchi method guided adjustments to manufacturing processes objectively, effectively, and economically.

Research on non-parametric prediction method of reciprocating compressor time series based on prediction credibility scale

Aiming at the long-term unpredictability of the reciprocating compressor vibration signal, a non-parametric prediction method of reciprocating compressor time series based on the prediction credibility scale is proposed in this paper. The method is to take the multifractal singular spectrum as the prediction parameter and use the Smoothness Priors Approach (SPA) method to obtain the singular spectrum parameters of different components, and construct the phase space reconstruction dynamic modeling domains. It enables the prediction model to reflect the real-time characteristics of the dynamics evolution of complex systems and highlights the independent influence of each component on the prediction. Meanwhile, the information entropy saturation principle is introduced into the K-Nearest Neighbor (KNN) model to establish the improved K neighborhood dynamic non-parametric prediction model based on the maximum prediction credibility scale, which improves the credibility of the prediction results. Finally, a complete SPA&PSR_KNN prediction algorithm is proposed. Through example validation and error analysis, compared with KNN, BP, and SVM, it can be seen that the prediction results of spectral characteristic parameters obtained by this algorithm have smaller error and higher reliability, and faster operation speed. Thus, the prediction of vibration signal time series of reciprocating compressor is realized.

A nonparametric predictive regression model using partitioning estimators based on Taylor expansions

This article proposes a nonparametric predictive regression model. The unknown function modeling the predictive relationship is approximated using polynomial Taylor expansions applied over disjoint intervals covering the support of the predictor variable. The model is estimated using the theory on partitioning estimators that is extended to a stationary time series setting. We show pointwise and uniform convergence of the proposed estimator and derive its asymptotic normality. These asymptotic results are applied to test for the presence of predictive ability. We develop an asymptotic pointwise test of predictive ability using the critical values of a Normal distribution, and a uniform test with asymptotic distribution that is approximated using a p‐value transformation and Wild bootstrap methods. These theoretical insights are illustrated in an extensive simulation exercise and also in an empirical application to forecasting high‐frequency based realized volatility measures. Our results provide empirical support to the presence of nonlinear autoregressive predictability of these measures for the constituents of the Dow Jones index.

A Comparative Study on Traffic Modeling Techniques for Predicting and Simulating Traffic Behavior

The significant advancements in intelligent transportation systems (ITS) have contributed to the increased development in traffic modeling. These advancements include prediction and simulation models that are used to simulate and predict traffic behaviors on highway roads and urban networks. These models are capable of precise modeling of the current traffic status and accurate predictions of the future status based on varying traffic conditions. However, selecting the appropriate traffic model for a specific environmental setting is challenging and expensive due to the different requirements that need to be considered, such as accuracy, performance, and efficiency. In this research, we present a comprehensive literature review of the research related to traffic prediction and simulation models. We start by highlighting the challenges in the long-term and short-term prediction of traffic modeling. Then, we review the most common nonparametric prediction models. Lastly, we look into the existing literature on traffic simulation tools and traffic simulation algorithms. We summarize the available traffic models, define the required parameters, and discuss the limitations of each model. We hope that this survey serves as a useful resource for traffic management engineers, researchers, and practitioners in this domain.

A cost-sensitive Imprecise Credal Decision Tree based on Nonparametric Predictive Inference

Classifiers sometimes return a set of values of the class variable since there is not enough information to point to a single class value. These classifiers are known as imprecise classifiers. Decision Trees for Imprecise Classification were proposed and adapted to consider the error costs when classifying new instances. In this work, we present a new cost-sensitive Decision Tree for Imprecise Classification that considers the error costs by weighting instances, also considering such costs in the tree building process. Our proposed method uses the Nonparametric Predictive Inference Model, a nonparametric model that does not assume previous knowledge about the data, unlike previous imprecise probabilities models. We show that our proposal might give more informative predictions than the existing cost-sensitive Decision Tree for Imprecise Classification. Experimental results reveal that, in Imprecise Classification, our proposed cost-sensitive Decision Tree significantly outperforms the one proposed so far; even though the cost of erroneous classifications is higher with our proposal, it tends to provide more informative predictions.

Construction a Non-parametric Prediction Model for Spatial Random Field Using Projection Theorem

Identifying the best prediction of unobserved observation is one of the most critical issues in spatial statistics. In this line, various methods have been proposed, that each one has advantages and limitations in application. Although the best linear predictor is obtained according to the Kriging method, this model is applied for the Gaussian random field. The uncertainty in the distribution of random fields makes researchers use a method that makes the nongaussian prediction possible. In this paper, using the Projection theorem, a non-parametric method is presented to predict a random field. Then some models are proposed for predicting the nongaussian random field using the nearest neighbors. Then, the accuracy and precision of the predictor will be examined using a simulation study. Finally, the application of the introduced models is examined in the prediction of rainfall data in Khuzestan province.

How much is the view from the window worth? Machine learning-driven hedonic pricing model of the real estate market

Understanding the customers’ perception of the value of constituent characteristics of a good is among the key questions in any pricing strategy. Hedonic pricing allows such an analysis and is frequently applied in economic fields. Although it is regarded as a benchmark in its original form, the availability of new data sources and the development of machine learning techniques created a space for further improvement. In this study, we propose a general framework for applying machine learning tools to enhance the hedonic pricing model in several directions. We do this, first, by adding image and text sources to conventional data and then by applying an advanced nonparametric prediction model. Lastly, we use model agnostic analysis to uncover new pricing factors and unravel complex relationships that could not be captured by conventional models.

Look-ahead decision making for renewable energy: A dynamic “predict and store” approach

This paper presents an integrative methodology for managing and stabilizing the output of a wind/solar farm using storage devices in a cost effective and real-time manner. We consider the problem where a renewable farm should decide the amount of energy charged into, or withdrawn from, the battery given the stochastic and time-varying nature in the renewable energy power output. Our methodology features a seamless integration of a non-myopic decision framework and a sequential non-parametric predictive model based on functional principal component analysis. A key feature of our algorithm is that it quantifies costs over a rolling horizon where both predictions and decisions are updated on the fly as new data is acquired. Our technology is tested on the California ISO dataset. The case study provides a proof-of-concept that highlights both the benefits and ease of implementation of our forward looking framework.

Uncertainty-based information measures on the approximate non-parametric predictive inference model

ABSTRACT The Non-Parametric Predictive Inference Model for Multinomial Data (NPI-M) is an imprecise probabilities model used to represent the available information about a categorical variable. It presents some advantages over another imprecise probabilities model frequently used in the literature called Imprecise Dirichlet Model (IDM), which assumes previous knowledge about the data through a parameter. The Approximate Non-Parametric Predictive Inference Model for Multinomial Data (A-NPI-M) is a model similar to the NPI-M that can be expressed by reachable sets of probability intervals, is easier to manage, and is non-parametric. As a novelty, in this work, we analyze the main properties of A-NPI-M credal sets, comparing them with the properties of credal sets associated with the IDM. Moreover, we present procedures to calculate the most important uncertainty measures on A-NPI-M credal sets. Those procedures represent useful tools to make the A-NPI-M very suitable to be used in practical applications.

Water-energy benchmarking and predictive modeling in multi-family residential and non-residential buildings

As the threat of climate change grows alongside a continual increase in urban population, the need to ensure access to water and energy resources becomes more crucial. In the context of the water-energy nexus in urban environments, this work addresses current gaps in understanding of coupled water and energy demand patterns and reveals apparent dissimilarities between utilization of water and energy resources for heterogeneous buildings. This study proposes a data-driven approach to identify fundamental water and energy demand profiles, cluster buildings into groups exhibiting similar water and energy use, and predict their demand. The clustering problem was cast as a two-stage cluster ensemble problem, in which several clustering methods with different settings were employed, and then the results obtained from partial view of the data were combined to achieve consensus among the partitionings. The influential drivers for water and energy consumption were identified, parametric and non-parametric prediction models were developed and compared, utilizing high and low temporal data resolution. The clustering analysis performed in this work revealed that water and energy consumption patterns of heterogeneous buildings are not exclusively characterized by general building characteristics. Analysis of the predictive models showed that an overall non-parametric model provides better predictions for water and energy compared with parametric models and that models with high and low data resolution provide comparable demand predictions. The results of this study highlight the value of data-driven modeling for revealing meaningful insights into usage patterns and benchmarking buildings’ performance to provide a meaningful measure of comparison to facilitate multi-utility management. Overall, the methods outlined in this study provide another step towards building greater resiliency within urban areas in preparation for future changes in population and climate.

Non-parametric predictive inference for solving multi-label classification

Decision Trees (DTs) have been adapted to Multi-Label Classification (MLC). These adaptations are known as Multi-Label Decision Trees (ML-DT). In this research, a new ML-DT based on the Nonparametric Predictive Inference Model on Multinomial data (NPI-M) is proposed. The NPI-M is an imprecise probabilities model that provides good results when it is applied to DTs in standard classification. Unlike other models based on imprecise probabilities, the NPI-M is a nonparametric approach and it does not make unjustified assumptions before observing data. It is shown that the new ML-DT based on the NPI-M is more robust to noise than the ML-DT based on precise probabilities. As the intrinsic noise in MLC might be higher than in traditional classification, it is expected that the new ML-DT based on the NPI-M outperforms the already existing ML-DT. This fact is validated with an exhaustive experimentation carried out in this work on different MLC datasets with several levels of added noise. In it, many MLC evaluation metrics are employed in order to measure the performance of the algorithms. The experimental analysis shows that the proposed ML-DT based on NPI-M obtains better results than the ML-DT that uses precise probabilities, especially when we work on data with noise.

Estimating radar precipitation in cold climates: the role of air temperature within a non-parametric framework

Abstract. The use of ground-based precipitation measurements in radar precipitation estimation is well known in radar hydrology. However, the approach of using gauged precipitation and near-surface air temperature observations to improve radar precipitation estimates in cold climates is much less common. In cold climates, precipitation is in the form of snow, rain or a mixture of the two phases. Air temperature is intrinsic to the phase of the precipitation and could therefore be a possible covariate in the models used to ascertain radar precipitation estimates. In the present study, we investigate the use of air temperature within a non-parametric predictive framework to improve radar precipitation estimation for cold climates. A non-parametric predictive model is constructed with radar precipitation rate and air temperature as predictor variables and gauge precipitation as an observed response using a k nearest neighbour (k-nn) regression estimator. The relative importance of the two predictors is ascertained using an information theory-based weighting. Four years (2011–2015) of hourly radar precipitation rates from the Norwegian national radar network over the Oslo region, hourly gauged precipitation from 68 gauges and gridded observational air temperatures were used to formulate the predictive model, hence making our investigation possible. Gauged precipitation data were corrected for wind-induced under-catch before using them as true observed response. The predictive model with air temperature as an added covariate reduces root-mean-square error (RMSE) by up to 15 % compared to the model that uses radar precipitation rate as the sole predictor. More than 80 % of gauge locations in the study area showed improvement with the new method. Further, the associated impact of air temperature became insignificant at more than 85 % of gauge locations when the near-surface air temperature was warmer than 10 ∘C, which indicates that the partial dependence of precipitation on air temperature is most useful for colder temperatures.

Model Discrepancy Quantification in Simulation-Based Design of Dynamical Systems

Discrete-time state-space models have been extensively used in simulation-based design of dynamical systems. These prediction models may not accurately represent the true physics of a dynamical system due to potentially flawed understanding of the system, missing physics, and/or numerical approximations. To improve the validity of these models at new design locations, this paper proposes a novel dynamic model discrepancy quantification (DMDQ) framework. Time-instantaneous prediction models are constructed for the model discrepancies of “hidden” state variables, and are used to correct the discrete-time prediction models at each time-step. For discrete-time models, the hidden state variables and their discrepancies are coupled over two adjacent time steps. Also, the state variables cannot be directly measured. These factors complicate the construction of the model discrepancy prediction models. The proposed DMDQ framework overcomes these challenges by proposing two discrepancy modeling approaches: an estimation-modeling approach and a modeling-estimation approach. The former first estimates the model discrepancy and then builds a nonparametric prediction model of the model discrepancy; the latter builds a parametric prediction model of the model discrepancy first and then estimates the parameters of the prediction model. A subsampling method is developed to reduce the computational effort in building the two types of prediction models. A mathematical example and an electrical circuit dynamical system demonstrate the effectiveness of the proposed DMDQ framework and highlight the advantages and disadvantages of the proposed approaches.

Predictive ability of genome-assisted statistical models under various forms of gene action

Recent work has suggested that the performance of prediction models for complex traits may depend on the architecture of the target traits. Here we compared several prediction models with respect to their ability of predicting phenotypes under various statistical architectures of gene action: (1) purely additive, (2) additive and dominance, (3) additive, dominance, and two-locus epistasis, and (4) purely epistatic settings. Simulation and a real chicken dataset were used. Fourteen prediction models were compared: BayesA, BayesB, BayesC, Bayesian LASSO, Bayesian ridge regression, elastic net, genomic best linear unbiased prediction, a Gaussian process, LASSO, random forests, reproducing kernel Hilbert spaces regression, ridge regression (best linear unbiased prediction), relevance vector machines, and support vector machines. When the trait was under additive gene action, the parametric prediction models outperformed non-parametric ones. Conversely, when the trait was under epistatic gene action, the non-parametric prediction models provided more accurate predictions. Thus, prediction models must be selected according to the most probably underlying architecture of traits. In the chicken dataset examined, most models had similar prediction performance. Our results corroborate the view that there is no universally best prediction models, and that the development of robust prediction models is an important research objective.

What do we know about urban sustainability? A research synthesis and nonparametric assessment

Urban sustainability has become a burgeoning practical and scholarly enterprise over the last two decades. Yet, there have been few attempts to systematically assess what cumulative knowledge this research is generating. We advance our understanding of urban sustainability by synthesising extant empirical findings to gauge progress made towards developing theoretical insight, and then testing a nonparametric predictive model that helps overcome methodological challenges in this literature. Drawing data from two national surveys of US local governments, we find that although organisational capacity appears to be the most important predictor, the broad range of activities grouped under the banner of ‘urban sustainability’ rely on distinct causal mechanisms, and use of composite models and measures of sustainability may hinder theoretical advancement. Implications for future research are discussed.