Local Variable Selection Research Articles

Interpretable transformer-based model for probabilistic short-term forecasting of residential net load

Short-term residential load forecasting is of great significance for the demand-side energy management of the grid and the home energy management of resident customers. The massive penetration of distributed renewable energy, especially small-scale solar photovoltaic (PV), in the residential sector urgently requires us to move from traditional load forecasting to net load forecasting. In recent years, deep learning techniques that can improve model forecasting performance have developed rapidly in the field of residential load forecasting. However, the opaque nature of deep learning makes its practical application very difficult. This paper proposes a Transformer-based probabilistic residential net load forecasting method that utilizes quantile regression to quantify uncertainty in future load demand. Meanwhile, to improve the interpretability of deep learning model, local variable selection network is developed to automatically select relevant features and provide feature-level explanations. Additionally, interpretable sparse self-attention mechanism is proposed to extract long-term temporal dependencies. Numerical experiments are carried out with data from real household smart meters. The results show that the proposed model outperforms other state-of-the-art forecasting models. In terms of point forecasting, compared with the most common deep time series forecasting model LSTM, the proposed model decreases by 21.3%, 27.3% and 20.9% On three point forecasting performance metrics. Compared with Vanilla Transformer, the proposed InterFormer decreases by 9.9%, 10.5% and 9.0% On three point forecasting performance metrics. In terms of probabilistic forecasting, compared with LSTM and Vanilla Transformer, the average pinball loss of the proposed model decreases by 26.2% and 17.0%, respectively. Additionally, and most importantly, our model provides users with explanations in terms of feature importance and temporal patterns.

Sequential Optimization in Locally Important Dimensions

Optimizing an expensive, black-box function is challenging when its input space is high-dimensional. Sequential design frameworks first model with a surrogate function and then optimize an acquisition function to determine input settings to evaluate next. Optimization of both and the acquisition function benefit from effective dimension reduction. Global variable selection detects and removes input variables that do not affect across the input space. Further dimension reduction may be possible if we consider local variable selection around the current optimum estimate. We develop a sequential design algorithm called sequential optimization in locally important dimensions (SOLID) that incorporates global and local variable selection to optimize a continuous, differentiable function. SOLID performs local variable selection by comparing the surrogate’s predictions in a localized region around the estimated optimum with the p alternative predictions made by removing each input variable. The search space of the acquisition function is further restricted to focus only on the variables that are deemed locally active, leading to greater emphasis on refining the surrogate model in locally active dimensions. A simulation study across multiple test functions and an application to the Sarcos robot dataset show that SOLID outperforms conventional approaches. Supplementary materials for this article are available online.

Development of a new multi-layer perceptron based soft sensor for SO2 emissions in power plant

In the paper, a multi-layer perceptron (MLP) based soft sensor for SO2 emission in desulfurization process of thermal power plants is proposed. Firstly, the production process and variables involved in the desulfurization system of thermal power plants are analyzed. Secondly, the MLP network and its input variable selection algorithm with nonnegative garrote (NNG) and extremal optimization (EO) are studied. The proposed algorithm employs MLP to model the complex desulfurization process, and then conducts shrinkage on input weights of MLP by NNG. After that, further local variable selection is performed by EO and the final model is presented. Thirdly, the simulation on actual production data of a power plant and comparisons with other state-of-art soft sensors are made to demonstrate the performance of the proposed algorithm. The simulation results show that the proposed algorithm can accurately predict the target variable and has superior performance to other algorithms. Moreover, the variable importance analysis with our approaches are consistent with the field operating experience and can provide reference for the further optimization of the control system of desulfurization process.

Highly-overlapped, recursive partial least squares soft sensor with state partitioning via local variable selection

We report the use of a soft sensor ensemble based on recursive partial least squares with a large number of overlapping models. The proposed method uses process memory attenuation in the ensemble by varying the number of training samples included in each model, while always including the most recent samples, which are usually the most relevant for prediction of new samples, and also ensures that no local models are invalidated due to drift. To achieve state partitioning of the process data, covariance-based variable selection is performed on each of the model regions to ensure that only variables most relevant to the dominant process state are included in each of the local models. This approach yields a distribution of predictions from the models, permitting a prediction of the target property from the summary statistics of the observed prediction distribution. The effectiveness of the proposed method is demonstrated by testing against a conventional global soft sensor as well as a state-localized soft sensor, both with and without variable selection, on two soft sensing applications developed from real industrial processes employing various model updating frequencies. Results from the experiments demonstrate that the proposed method tends to outperform a global soft sensor in most cases, and is highly competitive with the compared state-localized soft sensor, indicating that the proposed method achieves accurate state partitioning.

Improving the Classification Accuracy for Near-Infrared Spectroscopy of Chinese Salvia miltiorrhiza Using Local Variable Selection.

In order to improve the classification accuracy of Chinese Salvia miltiorrhiza using near-infrared spectroscopy, a novel local variable selection strategy is thus proposed. Combining the strengths of the local algorithm and interval partial least squares, the spectra data have firstly been divided into several pairs of classes in sample direction and equidistant subintervals in variable direction. Then, a local classification model has been built, and the most proper spectral region has been selected based on the new evaluation criterion considering both classification error rate and best predictive ability under the leave-one-out cross validation scheme for each pair of classes. Finally, each observation can be assigned to belong to the class according to the statistical analysis of classification results of the local classification model built on selected variables. The performance of the proposed method was demonstrated through near-infrared spectra of cultivated or wild Salvia miltiorrhiza, which are collected from 8 geographical origins in 5 provinces of China. For comparison, soft independent modelling of class analogy and partial least squares discriminant analysis methods are, respectively, employed as the classification model. Experimental results showed that classification performance of the classification model with local variable selection was obvious better than that without variable selection.

Local variable selection of nonlinear nonparametric systems by first order expansion

Local variable selection by first order expansion for nonlinear nonparametric systems is investigated in the paper. By substantially modifying the algorithms developed in our earlier work (Bai et al., 2014), the previous results have been considerably strengthened under much less restrictive conditions. Firstly, the estimates generated by the modified algorithms are shown to have both the set and parameter convergence with probability one, rather than only the set convergence in probability given in our earlier work. Secondly, several technical assumptions, e.g., the lower and upper bounds on the growth of some random sequences, which practically are uncheckable, have been removed. Thirdly, not only the consistency but also the convergence rate of estimates have been established. Besides, a generalization of the proposed algorithms is also introduced.

Nonparametric Bayes Conditional Distribution Modeling With Variable Selection

This article considers a methodology for flexibly characterizing the relationship between a response and multiple predictors. Goals are (1) to estimate the conditional response distribution addressing the distributional changes across the predictor space, and (2) to identify important predictors for the response distribution change both within local regions and globally. We first introduce the probit stick-breaking process (PSBP) as a prior for an uncountable collection of predictor-dependent random distributions and propose a PSBP mixture (PSBPM) of normal regressions for modeling the conditional distributions. A global variable selection structure is incorporated to discard unimportant predictors, while allowing estimation of posterior inclusion probabilities. Local variable selection is conducted relying on the conditional distribution estimates at different predictor points. An efficient stochastic search sampling algorithm is proposed for posterior computation. The methods are illustrated through simulation and applied to an epidemiologic study.