Pinball Loss Function Research Articles

A novel hybrid model based on evolving multi-quantile long and short-term memory neural network for ultra-short-term probabilistic forecasting of photovoltaic power

Probabilistic forecasting is extremely crucial in eliminating uncertainty in photovoltaic (PV) power generation. Quantile regression long and short-term memory neural network (QRLSTM) is widely recognized as promising methods for PV power probabilistic forecasting due to their strong generalization ability. However, these models train the model for each quantile individually, which lacks consideration of the correlation and monotonicity between quantiles, and multiple training leads to excessive computational complexity. Furthermore, the non-differentiable pinball loss function generated by QR places significant demands on the optimization algorithms. To address these issues, this paper proposes an evolutive distributed chaotic particle swarm optimization (EDCPSO)-optimized multi-quantile LSTM (MQLSTM) to achieve high-quality probabilistic PV power prediction. MQLSTM is a multi-output network structure that simultaneously outputs all quantile estimates and adopts a loss function with all quantile scores and non-crossing constraints to guide the training of the model. This approach not only improves the quality and reasonableness of quantile estimations, but also reduces computational difficulty. Then, from the perspective of evolutionary computation, considering the weight parameters of each connection layer in MQLSTM as decision variables, we convert the probabilistic forecasting into an optimization problem and propose a EDCPSO to solve the training difficulty. It implements a targeted distributed chaos strategy based on the evolutionary state to improve convergence speed and search capability. The proposed model is tested to be superior in real cases.

Ordinal Regression With Pinball Loss.

Ordinal regression (OR) aims to solve multiclass classification problems with ordinal classes. Support vector OR (SVOR) is a typical OR algorithm and has been extensively used in OR problems. In this article, based on the characteristics of OR problems, we propose a novel pinball loss function and present an SVOR method with pinball loss (pin-SVOR). Pin-SVOR is fundamentally different from traditional SVOR with hinge loss. Traditional SVOR employs the hinge loss function, and the classifier is determined by only a few data points near the class boundary, called support vectors, which may lead to a noise sensitive and re-sampling unstable classifier. Distinctively, pin-SVOR employs the pinball loss function. It attaches an extra penalty to correctly classified data that lies inside the class, such that all the training data is involved in deciding the classifier. The data near the middle of each class has a small penalty, and that near the class boundary has a large penalty. Thus, the training data tend to lie near the middle of each class instead of on the class boundary, which leads to scatter minimization in the middle of each class and noise insensitivity. The experimental results show that pin-SVOR has better classification performance than state-of-the-art OR methods.

Sentence level Classification through machine learning with effective feature extraction using deep learning

Social networking website usage has increased dramatically during the past few years. Users can read other users' views, which are categorized into several sentiment classes on this medium with an array of data. These opinions are becoming more and more important while making decisions. To address the above-mentioned issues and improve the sentence-level classification's classification rate, this work introduces a new extensive pinball loss function based twin support vector machine with Deep Learning the (EPLF-TSVM-DL) to identify the polarity (negative and positive) of sentiment sentences. There are four primary components of this technique: The first portion consists of pre-processing the data to minimize noise and improve quality; the second part utilizes word embedding techniques to transform textual data into numerical data. The third part is the CNN for an efficient automatic method of extracting the features-based feature extraction and final is EPLF-TSVM-DL is used for sentence level classification that forms two classes such as Negative and Positive. The findings demonstrated that the EPLF-TSVM-DL outperforms the other classifiers with respect to of time consumption, convergence, complexity, and stability as well as true negative, true positive, error rate, false positive, precision, false negative, and classification rate.

Intuitionistic Fuzzy Extreme Learning Machine with the Truncated Pinball Loss

Fuzzy extreme learning machine (FELM) is an effective algorithm for dealing with classification problems with noises, which uses a membership function to effectively suppress noise in data. However, FELM has the following drawbacks: (a) The membership degree of samples in FELM is constructed by considering only the distance between the samples and the class center, not the local information of samples. It is easy to mistake some boundary samples for noises. (b) FELM uses the least squares loss function, which leads to sensitivity to feature noise and instability to re-sampling. To address the above drawbacks, we propose an intuitionistic fuzzy extreme learning machine with the truncated pinball loss (TPin-IFELM). Firstly, we use the K-nearest neighbor (KNN) method to obtain local information of the samples and then construct membership and non-membership degrees for each sample in the random mapping feature space based on valuable local information. Secondly, we calculate the score value of samples based on the membership and non-membership degrees, which can effectively identify whether the boundary samples are noises or not. Thirdly, in order to maintain the sparsity and robustness of the model, and enhance the stability of the resampling of the model, we introduce the truncated pinball loss function into the model. Finally, in order to solve more efficiently, we employ the concave-convex procedure (CCCP) to solve TPin-IFELM. Extensive comparative experiments are conducted on the benchmark datasets to verify the superior performance of TPin-IFELM.

Pinball-OCSVM for Early-Stage COVID-19 Diagnosis with Limited Posteroanterior Chest X-Ray Images

The conventional way of respiratory coronavirus disease 2019 (COVID-19) diagnosis is reverse transcription polymerase chain reaction (RT-PCR), which is less sensitive during early stages; especially if the patient is asymptomatic, which may further cause more severe pneumonia. In this context, several deep learning models have been proposed to identify pulmonary infections using publicly available chest X-ray (CXR) image datasets for early diagnosis, better treatment and quick cure. In these datasets, the presence of less number of COVID-19 positive samples compared to other classes (normal, pneumonia and Tuberculosis) raises the challenge for unbiased learning of deep learning models. This learning problem can be considered as one-class classification problem where the target class samples are present and other classes are absent or ill-defined. All deep learning models opted class balancing techniques to solve this issue; which however should be avoided in any medical diagnosis process. Moreover, the deep learning models are also data hungry and need massive computation resources. Therefore, for quicker diagnosis, this research proposes a novel pinball loss function based one-class support vector machine (PB-OCSVM), that can work in presence of limited COVID-19 positive CXR samples (target class or class-of-interest (CoI) samples) with objectives to maximize the learning efficiency and to minimize the false predictions. The performance of the proposed model is compared with conventional OCSVM and recent deep learning models, and the experimental results prove that the proposed model outperformed state-of-the-art methods. To validate the robustness of the proposed model, experiments are also performed with noisy CXR images and UCI benchmark datasets.

Learnable quantile polynomial chaos expansion: An uncertainty quantification method for interval reliability analysis

Various uncertainties like random input and data noise in aerospace engineering affect flight vehicle safety, making uncertainty quantification important for improving system reliability. Polynomial chaos expansion (PCE) is a versatile method for quantifying stochastic systems’ uncertainty caused by random input, while it cannot quantify data uncertainty caused by noise. To solve this problem, a novel uncertainty quantification method, learnable quantile PCE (LQPCE), is proposed for interval reliability analysis. Unlike the traditional PCE, the multi-dimensional orthonormal basis is constructed for combination input rather than only random variables, and the expansion coefficients are parameterized to learnable weights, ensuring the LQPCE model can capture data noise’s features. The LQPCE Pinball loss function is constructed to learn parameterized expansion coefficients by quantile level iterative sampling, achieving accurate prediction and data uncertainty quantification. The quantified data uncertainty is used for interval reliability analysis, calculating more reliable results about the influence of input uncertainty on stochastic systems’ reliability. Three numerical examples and one engineering application validate the LQPCE method’s effectiveness. The results show that the LQPCE method can quantify data uncertainty accurately and obtain credible interval reliability.

Granular Ball Twin Support Vector Machine With Pinball Loss Function

Granular Ball Twin Support Vector Machine With Pinball Loss Function

Mechanical Fault Diagnosis With Noisy Multisource Signals via Unified Pinball Loss Intuitionistic Fuzzy Support Tensor Machine

In this paper, a challenging and significant intelligent fault diagnosis task is investigated, in which multisource sensor signals with intense noise and outlier disturbances are jointly analyzed. Such a scenario has hardly been considered in industrial research. To this end, we develop a novel tensor-based nonlinear classifier called unified pinball loss intuitionistic fuzzy support tensor machine (UPIFSTM), which can successfully solve the above tasks and improve the performance of fault diagnosis in practical applications. First, the noisy multisource signals are converted into time-frequency images and reconstructed into tensor samples to mine the time domain, frequency domain features and coupled structure information in the spatial domain. Next, we design two nonlinear forms of nonmembership functions in the tensor space, and obtain an intuitionistic fuzzy score for each training sample to enhance the robustness of the model. Subsequently, pinball loss function is introduced to better handle noise sensitivity and resampling instability problems. Note that, we employ the tensor robust principal component analysis method to accurately recover the low-rank tensors corrupted by sparse noise from the original tensors. Finally, two numerical examples are presented to verify the feasibility and validity of the proposed method.

Short‐term prediction of wind power based on temporal convolutional network and the informer model

AbstractIn this study, a new short‐term wind power prediction model based on a temporal convolutional network (TCN) and the Informer model is proposed to solve the problem of low prediction accuracy caused by large wind speed fluctuations in short‐term prediction. First, an input feature selection method based on the maximum information coefficient is proposed after considering the problem of information interference caused by excessively large input features. A dynamic time planning method is used to select the optimal input step of historical power. Then, the combined forecasting model composed of TCN and the Informer is constructed in accordance with the numerical weather forecast and historical power data. Lastly, the pinball loss function is used to expand the prediction model into a quantile regression model, measure the effect of volatility, quantify the volatility range of prediction, and finally, obtain a deterministic prediction result. The actual measured data of wind farms in the Bohai Sea area are selected for analysis and calculation. The results show that the prediction model proposed in this study achieves better accuracy in deterministic prediction and interval prediction than the traditional model.

Research of a novel combined deterministic and probabilistic forecasting system for air pollutant concentration

Hazy weather seriously threatens human health, providing accurate and reliable air pollutant concentration forecast will be of great scientific value and benefit to residents' livelihood and natural environment. However, existing forecasting models lack outlier detection, feature mapping and uncertainty forecast, which results in unreliable forecasting results. Thus, in this paper, a novel combined deterministic and probabilistic forecasting system based on meta-heuristic algorithm is proposed, which contains multiple data pre-processing, combined deterministic and probabilistic forecast. Firstly, the raw data is pre-processed and analyzed from multiple perspectives to fully extract the chaotic features. Then multi-objective white shark optimizer is utilized to combine the three deep learning models to enhance the forecasting accuracy and stability. Furthermore, the Pin-ball loss function is employed to yield prediction intervals at different confidence levels, and a novel interval pseudo-code is constructed to optimize the prediction intervals thus promoting the probabilistic forecasting reliability. The simulations indicated that the developed system enhances the forecasting accuracy by maximum 60.6772%, 54.9793% and 38.6876% in the three steps of PM2.5 prediction and up to 64.9001%, 68.4637% in the interval sharpness at significance level 0.05 and 0.1 than the tested models.

Adaptive data decomposition based quantile-long-short-term memory probabilistic forecasting framework for power demand side management of energy system

Load management can improve the overall benefit of power system through peak cutting and valley filling, whose performance depends on the accuracy of load forecasting. However, how to efficiently utilize load data to uprate forecasting performance is urgent. Therefore, a peak cutting and valley filling forecasting algorithm based on load operation state is proposed. Firstly, adaptive data decomposition based quantile-long-short-term memory (QLSTM) probabilistic forecasting framework is proposed to reflect the future load information more comprehensively. The method combines long-short-term memory (LSTM) with pinball loss function to provide deterministic forecasting and probability density forecasting. Then, a variable power peak cutting and valley filling algorithm is proposed to suppress the peak–valley difference and optimize the power structure. Finally, the case study of typical working conditions of actual power grid is performed to confirm that the proposed method can better reflect the future load information and reduce peak–valley difference.

Laplacian Twin Support Vector Machine With Pinball Loss for Semi-Supervised Classification

Semi-supervised learning utilizes labeled data and the geometric information in the unlabeled data to construct a model whereas supervised learning makes use of the only label data. So, semi-supervised learning establishes a more reasonable classifier. In recent years, the Laplacian support vector machine (Lap-SVM) has received a lot of interest in the framework of semi-supervised classification. To develop the performance of Lap-SVM, Laplacian twin support vector machine (Lap-TSVM) has shown exceptional performance as an addition to improve the computational complexity. However, dealing with noise sensitivity and instability for resampling due to its hinge loss function is still a challenge. In this paper, we provide a novel Laplacian twin support vector machine by combining pinball loss function, termed as Lap-PTSVM, to effectively handle the aforementioned problems. As a result, it improves a better generalization ability of the classifier. Several experiments have been performed on artificial and UCI datasets. The results show that the proposed model has noise insensitivity comparable to the Lap-TSVM and has a great generalization performance. Furthermore, non-parametric statistical test are also conducted to justify the competitive performance of Lap-PTSVM.

Sparse Pinball Twin Bounded Support Vector Clustering

Analyzing unlabeled data is of prime importance in machine learning. Creating groups and identifying an underlying clustering principle is essential to many fields, such as biomedical analysis and market research. Novel unsupervised machine learning algorithms, also called clustering algorithms, are developed and utilized for this task. Inspired by twin support vector machine (TWSVM) principles, a recently introduced plane-based clustering algorithm, the twin bounded support vector clustering (TBSVC), is used in widespread clustering problems. However, TBSVC is sensitive to noise and suffers from low resampling stability due to usage of hinge loss. Pinball loss is another type of loss function that is less sensitive toward noise in the datasets and is more stable for resampling of datasets. However, the use of pinball loss negatively affects the sparsity of the solution of the problem. In this article, we present a novel plane-based clustering method, the sparse TBSVC using pinball loss (pinSTBSVC). The proposed pinSTBSVC is the sparse version of our recently proposed TBSVC using pinball loss (pinTBSVC). Sparse solutions help create better-generalized solutions to clustering problems; hence, we attempt to use the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\epsilon $ </tex-math></inline-formula> -insensitive pinball loss function to propose pinSTBSVC. The loss function used to propose pinSTBSVC provides sparsity to the solution of the problem and improves the aforementioned plane-based clustering algorithms. Experimental results performed on benchmark University of California, Irvine (UCI) datasets indicate that the proposed method outperforms other existing plane-based clustering algorithms. Additionally, we also give the application of our method in biomedical image clustering and marketing science. We show that the proposed method is more accurate on real-world datasets too. The code for the proposed algorithm is also provided on the author’s Github page: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/mtanveer1</uri> .

Adaptive probabilistic load forecasting for individual buildings

Building-level load forecasting has become essential with the support of fine-grained data collected by widely deployed smart meters. It acts as a basis for arranging distributed energy resources, implementing demand response, etc. Compared to aggregated-level load, the electric load of an individual building is more stochastic and thus spawns many probabilistic forecasting methods. Many of them resort to artificial neural networks (ANN) to build forecasting models. However, a well-designed forecasting model for one building may not be suitable for others, and manually designing and tuning optimal forecasting models for various buildings are tedious and time-consuming. This paper proposes an adaptive probabilistic load forecasting model to automatically generate high-performance NN structures for different buildings and produce quantile forecasts for future loads. Specifically, we cascade the long short term memory (LSTM) layer with the adjusted Differential ArchiTecture Search (DARTS) cell and use the pinball loss function to guide the model during the improved model fitting process. A case study on an open dataset shows that our proposed model has superior performance and adaptivity over the state-of-the-art static neural network model. Besides, the improved fitting process of DARTS is proved to be more time-efficient than the original one.

Learning Pinball TWSVM efficiently using Privileged Information and its Applications

Expert knowledge plays a vital role in any learning framework. However, in the field of machine learning, an expert's knowledge is rarely utilised. Furthermore, most machine learning methods (support vector machine, SVM-based) use a hinge loss function that is sensitive to noise. Thus, in order to benefit from expert knowledge while reducing noise sensitivity, we propose in this paper a fast and novel Twin Support Vector Machine classifier based on privileged information with a pinball loss function, dubbed Pin-TWSVMPI, where expert knowledge is in the form of privileged information. The proposed Pin-TWSVMPI incorporates privileged information into two nonparallel decision hyperplanes by employing a correction function. Furthermore, we employ the Sequential Minimal Optimization (SMO) technique to get the classifier in order to make computations more efficient and faster, and we have demonstrated its applicability for pedestrian detection and handwritten digit recognition. Furthermore, for UCI datasets, we first construct a process that retrieves privileged information from the dataset's features, which is then used by Pin-TWSVMPI, resulting in improved classification accuracy with a reduced computing time.

Learning Pinball TWSVM efficiently using Privileged Information and their applications

In any learning framework, an expert knowledge always plays a crucial role. But, in the field of machine learning, the knowledge offered by an expert is rarely used. Moreover, machine learning algorithms (SVM based) generally use hinge loss function which is sensitive towards the noise. Thus, in order to get the advantage from an expert knowledge and to reduce the sensitivity towards the noise, in this paper, we propose a fast and novel Twin Support Vector Machine classifier based on privileged information with pinball loss function which has been termed as Pin-TWSVMPI where expert's knowledge is in the form of privileged information. The proposed Pin-TWSVMPI incorporates privileged information by using correcting function so as to obtain two nonparallel decision hyperplanes. Further, in order to make computations more efficient and fast, we use Sequential Minimal Optimization (SMO) technique for obtaining the classifier and have also shown its application for Pedestrian detection and Handwritten digit recognition. Further, for UCI datasets, we first implement a procedure which extracts privileged information from the features of the dataset which are then further utilized by Pin-TWSVMPI to which lead to enhancement in classification accuracy with comparatively lesser computational time.

A Novel Twin Support Vector Machine with Generalized Pinball Loss Function for Pattern Classification

We introduce a novel twin support vector machine with the generalized pinball loss function (GPin-TSVM) for solving data classification problems that are less sensitive to noise and preserve the sparsity of the solution. In addition, we use a symmetric kernel trick to enlarge GPin-TSVM to nonlinear classification problems. The developed approach is tested on numerous UCI benchmark datasets, as well as synthetic datasets in the experiments. The comparisons demonstrate that our proposed algorithm outperforms existing classifiers in terms of accuracy. Furthermore, this employed approach in handwritten digit recognition applications is examined, and the automatic feature extractor employs a convolution neural network.

An Improved Technique for Pneumonia Infected Patients Image Recognition Based on Combination Algorithm of Smooth Generalized Pinball SVM and Variational Autoencoders

We present a method based on combining a smooth generalized pinball support vector machine (SVM) and variational autoencoders (VAEs) in chest X-ray (CXR) images. We incorporate generalized pinball into the SVM model to address the hinge loss function’s noise sensitivity and resampling instability. Since the generalized pinball loss is a non-differentiable function, the objective function is non-differentiable. To solve the non-differentiable term in the model’s objective function, we construct a smooth approximation function for the generalized pinball loss function so that the model in the original space can be solved directly. More specifically, using the proposed smoothing functions, we employ the smoothing stochastic quasi-Newton method to solve the primal problem. Moreover, we have introduced verified theorems related to our approaches, and the theoretical convergence of our approaches has been proposed. Numerical comparisons between parameter values of several iteration objective functions are also provided to test the model. Experiments with several benchmark datasets show that our method has the best prediction accuracy 64% and 57% from all of binary datasets and multiclass datasets, respectively. Moreover, our method has the best prediction of Matthews correlation coefficient (MCC) 64% from all of datasets.

Short-term load forecasting based on LSTM networks considering attention mechanism

Reliable and accurate zonal electricity load forecasting is essential for power system operation and planning. Probabilistic load forecasts can present more comprehensive information for decision-making processes by quantifying the uncertainties of the electric load. A suitable feature selection is a critical step in forecasting, especially for data-driven methods. Weather conditions are another major factor related to electricity demand and play an important role in load forecasting. In this paper, we propose a dual-stage attention based long short-term memory (LSTM) network for short-term zonal load probabilistic forecasting. In the first stage, a feature attention based encoder is built to calculate the correlation of input features with electricity load at each time step. The most relevant input features can be adaptively selected. In the second stage, a temporal attention based decoder is developed to mine the time dependencies. Then, an LSTM model integrates these attention results and the probabilistic forecasts can be obtained using a pinball loss function. We also discuss how the proposed method can be utilized for feature and weather station selection. The effectiveness of the proposed method for both point and probabilistic forecasting is adequately verified on an open dataset of GEFCom2014, showing higher accuracy and generalization ability over other state-of-the-art forecasting models.

Intuitionistic fuzzy twin support vector machines with the insensitive pinball loss

Due to the use of membership and nonmembership functions of samples from intuitionistic fuzzy sets(IFSs), intuitionistic fuzzy twin support vector machines (IFTSVMs) can effectively suppress noise in the data. However, the objective function of IFTSVMs partially considers score values of samples and employs the hinge loss function which leads to the sensitivity to feature noise and instability to re-sampling. To enhance the performance of IFTSVMs, we propose novel IFTSVMs with the insensitive pinball loss function. In the proposed convex optimization models, a simple strategy is devised to achieve the score value of each training sample and score values of samples in both classes are defined by using IFSs. Unlike previous methods, we introduce two groups of slack variables to derive the dual formulations of convex models which make them have compact representations. Some properties of the proposed models including geometric properties and noise insensitivity are theoretically analyzed. We also explain the proposed models in terms of the idea of the weighted scatter minimization, which provides theoretical foundations for the proposed models. Experiments on a series of data sets are performed and experimental results demonstrate that the proposed convex models are superior to some existing learning models in the presence of feature noise or label noise.