Pinball Loss Research Articles

Transfer twin support matrix machine using rescaled pinball loss for roller bearing fault diagnosis

Transfer twin support matrix machine using rescaled pinball loss for roller bearing fault diagnosis

Probabilistic oil price forecasting with a variational mode decomposition-gated recurrent unit model incorporating pinball loss

Prediction methods have garnered significant attention in intelligent decision-making. Most existing approaches to predicting crude oil prices prioritize accuracy and stability while providing precise prediction intervals (PIs) that can offer valuable insights. Thus far, we introduce a novel hybrid model to forecast future crude oil prices. Our approach leverages the variational mode decomposition (VMD) to simplify the complexity of the original time series, yielding a set of subseries. These subseries are then modeled using a deep neural network architecture called a gated recurrent unit (GRU). To address the prediction uncertainty, we employed the pinball loss function rather than the mean square error (MSE) to guide the proposed VMD-GRU. This adaptation extends the traditional GRU-based point forecasting to probabilistic forecasting by estimating quantiles. We evaluated our proposed model on a well-established crude oil price series by conducting both single- and multi-step-ahead forecasting analyses. Our findings underscore the efficacy of the combined model, demonstrating its superior predictive performance compared to benchmark models.

A [formula omitted]-means triangular synthesis large margin classifier with unified pinball loss for imbalanced data

Large Margin Distribution Machine (LDM) improves the Support Vector Machine (SVM) by integrating the marginal distribution of samples into the objective function, which exhibits excellent classification and generalization performance. However, most of the existing LDM-based models exhibit inherent flaws: (1) The assumption of a category uniform distribution, resulting in an inability to effectively address the issue of class imbalance. (2) The inheritance of the hinge loss in SVM, leading to inferior anti-noise performance. Given the shortcomings above, this paper constructs a novel K-means Triangular Synthesis (KTS) method for imbalanced data classification and introduces Unified Pinball (UP) loss to ensure robust performance, demonstrated as a novel KTS large margin classifier with UP Loss (KTS-UPLMC) model. Specifically, the KTS method utilizes the triangular synthesis and generates samples based on the cluster density, effectively mitigating the problems of introducing noise samples and strip-shaped sample distribution in the traditional synthetic minority oversampling technique, further alleviating the LDM’s classification line offset. In addition, the UP Loss considers quantile distance, improving the anti-noise performance of the model. Comparative experiments on artificial datasets and benchmark datasets validate the effectiveness and noise resistance of the proposed KTS-UPLMC in imbalanced data classification problems. Furthermore, the stability of the KTS-UPLMC is substantiated by the parameter sensitivity analysis experiment. In conclusion, the KTS method effectively addresses category imbalance, while the integration of the UP Loss enhances noise resistance.

A novel hybrid model based on evolving multi-quantile long and short-term memory neural network for ultra-short-term probabilistic forecasting of photovoltaic power

Probabilistic forecasting is extremely crucial in eliminating uncertainty in photovoltaic (PV) power generation. Quantile regression long and short-term memory neural network (QRLSTM) is widely recognized as promising methods for PV power probabilistic forecasting due to their strong generalization ability. However, these models train the model for each quantile individually, which lacks consideration of the correlation and monotonicity between quantiles, and multiple training leads to excessive computational complexity. Furthermore, the non-differentiable pinball loss function generated by QR places significant demands on the optimization algorithms. To address these issues, this paper proposes an evolutive distributed chaotic particle swarm optimization (EDCPSO)-optimized multi-quantile LSTM (MQLSTM) to achieve high-quality probabilistic PV power prediction. MQLSTM is a multi-output network structure that simultaneously outputs all quantile estimates and adopts a loss function with all quantile scores and non-crossing constraints to guide the training of the model. This approach not only improves the quality and reasonableness of quantile estimations, but also reduces computational difficulty. Then, from the perspective of evolutionary computation, considering the weight parameters of each connection layer in MQLSTM as decision variables, we convert the probabilistic forecasting into an optimization problem and propose a EDCPSO to solve the training difficulty. It implements a targeted distributed chaos strategy based on the evolutionary state to improve convergence speed and search capability. The proposed model is tested to be superior in real cases.

Probabilistic quantile multiple fourier feature network for lake temperature forecasting: incorporating pinball loss for uncertainty estimation

Probabilistic quantile multiple fourier feature network for lake temperature forecasting: incorporating pinball loss for uncertainty estimation

Ordinal Regression With Pinball Loss.

Ordinal regression (OR) aims to solve multiclass classification problems with ordinal classes. Support vector OR (SVOR) is a typical OR algorithm and has been extensively used in OR problems. In this article, based on the characteristics of OR problems, we propose a novel pinball loss function and present an SVOR method with pinball loss (pin-SVOR). Pin-SVOR is fundamentally different from traditional SVOR with hinge loss. Traditional SVOR employs the hinge loss function, and the classifier is determined by only a few data points near the class boundary, called support vectors, which may lead to a noise sensitive and re-sampling unstable classifier. Distinctively, pin-SVOR employs the pinball loss function. It attaches an extra penalty to correctly classified data that lies inside the class, such that all the training data is involved in deciding the classifier. The data near the middle of each class has a small penalty, and that near the class boundary has a large penalty. Thus, the training data tend to lie near the middle of each class instead of on the class boundary, which leads to scatter minimization in the middle of each class and noise insensitivity. The experimental results show that pin-SVOR has better classification performance than state-of-the-art OR methods.

Pinball boosting of regression quantiles

An algorithm for boosting regression quantiles using asymmetric least absolute deviations, better known as pinball loss, is proposed. Existing approaches for boosting regression quantiles are essentially equal to least squares boosting of regression means with the single difference that their working residuals are based on pinball loss. All steps of our boosting algorithm are embedded in the well-established framework of quantile regression, and its main components – sequential base learning, fitting, and updating – are based on consistent scoring rules for regression quantiles. The Monte Carlo simulations performed indicate that the pinball boosting algorithm is competitive with existing approaches for boosting regression quantiles in terms of estimation accuracy and variable selection, and that its application to the study of regression quantiles of hedonic price functions allows the estimation of previously infeasible high-dimensional specifications.

A novel bounded loss framework for support vector machines

This paper introduces a novel bounded loss framework for SVM and SVR. Specifically, using the Pinball loss as an illustration, we devise a novel bounded exponential quantile loss (Leq-loss) for both support vector machine classification and regression tasks. For Leq-loss, it not only enhances the robustness of SVM and SVR against outliers but also improves the robustness of SVM to resampling from a different perspective. Furthermore, EQSVM and EQSVR were constructed based on Leq-loss, and the influence functions and breakdown point lower bounds of their estimators are derived. It is proved that the influence functions are bounded, and the breakdown point lower bounds can reach the highest asymptotic breakdown point of 1/2. Additionally, we demonstrated the robustness of EQSVM to resampling and derived its generalization error bound based on Rademacher complexity. Due to the Leq-loss being non-convex, we can use the concave–convex procedure (CCCP) technique to transform the problem into a series of convex optimization problems and use the ClipDCD algorithm to solve these convex optimization problems. Numerous experiments have been conducted to confirm the effectiveness of the proposed EQSVM and EQSVR.

Probabilistic sunspot predictions with a gated recurrent units-based combined model guided by pinball loss

Sunspots play a crucial role in both weather forecasting and the monitoring of solar storms. In this work, we propose a novel combined model for sunspot prediction using improved gated recurrent units (GRU) guided by pinball loss for probabilistic forecasts. Specifically, we optimize the GRU parameters using the slime mould algorithm and employ a seasonal-trend decomposition procedure based on loess to tackle challenges related to sequence prediction, such as self-correlations and non-stationarity. To address prediction uncertainty, we replace the traditional l2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$l_2$$\\end{document}-norm loss with pinball loss. This modification extends the conventional GRU-based point forecasting to a probabilistic framework expressed as quantiles. We apply our proposed model to analyze a well-established historical sunspot dataset for both single- and multi-step ahead forecasting. The results demonstrate the effectiveness of our combined model in predicting sunspot values, surpassing the performance of other existing methods.

Sentence level Classification through machine learning with effective feature extraction using deep learning

Social networking website usage has increased dramatically during the past few years. Users can read other users' views, which are categorized into several sentiment classes on this medium with an array of data. These opinions are becoming more and more important while making decisions. To address the above-mentioned issues and improve the sentence-level classification's classification rate, this work introduces a new extensive pinball loss function based twin support vector machine with Deep Learning the (EPLF-TSVM-DL) to identify the polarity (negative and positive) of sentiment sentences. There are four primary components of this technique: The first portion consists of pre-processing the data to minimize noise and improve quality; the second part utilizes word embedding techniques to transform textual data into numerical data. The third part is the CNN for an efficient automatic method of extracting the features-based feature extraction and final is EPLF-TSVM-DL is used for sentence level classification that forms two classes such as Negative and Positive. The findings demonstrated that the EPLF-TSVM-DL outperforms the other classifiers with respect to of time consumption, convergence, complexity, and stability as well as true negative, true positive, error rate, false positive, precision, false negative, and classification rate.

A Novel Fuzzy Large Margin Distribution Machine With Unified Pinball Loss

A Novel Fuzzy Large Margin Distribution Machine With Unified Pinball Loss

Intuitionistic Fuzzy Extreme Learning Machine with the Truncated Pinball Loss

Fuzzy extreme learning machine (FELM) is an effective algorithm for dealing with classification problems with noises, which uses a membership function to effectively suppress noise in data. However, FELM has the following drawbacks: (a) The membership degree of samples in FELM is constructed by considering only the distance between the samples and the class center, not the local information of samples. It is easy to mistake some boundary samples for noises. (b) FELM uses the least squares loss function, which leads to sensitivity to feature noise and instability to re-sampling. To address the above drawbacks, we propose an intuitionistic fuzzy extreme learning machine with the truncated pinball loss (TPin-IFELM). Firstly, we use the K-nearest neighbor (KNN) method to obtain local information of the samples and then construct membership and non-membership degrees for each sample in the random mapping feature space based on valuable local information. Secondly, we calculate the score value of samples based on the membership and non-membership degrees, which can effectively identify whether the boundary samples are noises or not. Thirdly, in order to maintain the sparsity and robustness of the model, and enhance the stability of the resampling of the model, we introduce the truncated pinball loss function into the model. Finally, in order to solve more efficiently, we employ the concave-convex procedure (CCCP) to solve TPin-IFELM. Extensive comparative experiments are conducted on the benchmark datasets to verify the superior performance of TPin-IFELM.

A novel dynamic spatio-temporal graph convolutional network for wind speed interval prediction

It is crucial to predict the wind speed for the utilization of renewable wind energy and the operation of transmission lines with increased capacity. The intermittency and stochastic fluctuations of wind speed pose a significant challenge for the high-quality wind speed prediction, and a novel wind speed interval prediction (WSIP) model is constructed in this study by employing the residual estimation (RE)-oriented dynamic spatio-temporal graph convolutional network (DSTGCN) approach. Firstly, a dynamic adjacency matrix is designed to obtain time-varying global spatial weight allocations among each wind speed node. Then, the spatio-temporal features are extracted by using gated recurrent units (GRUs) and GCNs to construct the wind speed graph networks. Moreover, the RE-oriented strategy incorporating the pinball loss is designed to provide a guidance the parameter training of the constructed model, thus eliminating the quantile crossings problem. As a result, the deterministic point prediction of the wind speed is expanded to the quantile-based probabilistic interval prediction. Finally, the experimental results are presented to demonstrate the validity and superiority of proposed scheme in both qualitative and quantitative performance.

Diagnosis of breast cancer using flexible pinball loss support vector machine

Breast cancer is a common disease that affects feminine health, making it an active area of research. Also, support vector machine with pinball loss (pin-SVM) is an efficient classification algorithm to address noise sensitivity and re-sampling instability. The pinball loss function uses a loss parameter τ∈[0,1] which corresponds to the quantile level. However, the non-negativity condition on τ is not necessary, and it can be extended to the negative values for an improvement in classification accuracy. Also, instead of a positive loss parameter τ, two positive parameters, τ1 and τ2 are used in literature, which improve the generalization performance of the pin-SVM. Taking motivation from the aforementioned observations, in this paper, we propose an innovative loss function, termed the flexible pinball loss, which extends the parameters τ1 and τ2 to encompass negative values. This extension enables the function to take τ1 and τ2 values from −1 to 1 while preserving convexity. Subsequently, we integrate the proposed flexible pinball loss function into the support vector machine framework and propose a novel model named flexible pinball loss support vector machine (FP-SVM) for the prediction of breast cancer. FP-SVM provides loss to both incorrectly and correctly classified samples, leveraging the parameters τ1 and τ2, respectively. Importantly, FP-SVM strategically traverses the maximum solution path, ensuring the preservation of convexity within the optimization problem. The proposed FP-SVM outperforms the baseline models in terms of accuracy, which is empirically supported by numerical experiments on 30 UCI and KEEL benchmark datasets. Furthermore, to show the efficacy of the proposed FP-SVM in real-world application, we performed experiments on publicly available breast cancer dataset (BreakHis), and the results demonstrate that the proposed FP-SVM outperforms the baseline models.

Pinball-OCSVM for Early-Stage COVID-19 Diagnosis with Limited Posteroanterior Chest X-Ray Images

The conventional way of respiratory coronavirus disease 2019 (COVID-19) diagnosis is reverse transcription polymerase chain reaction (RT-PCR), which is less sensitive during early stages; especially if the patient is asymptomatic, which may further cause more severe pneumonia. In this context, several deep learning models have been proposed to identify pulmonary infections using publicly available chest X-ray (CXR) image datasets for early diagnosis, better treatment and quick cure. In these datasets, the presence of less number of COVID-19 positive samples compared to other classes (normal, pneumonia and Tuberculosis) raises the challenge for unbiased learning of deep learning models. This learning problem can be considered as one-class classification problem where the target class samples are present and other classes are absent or ill-defined. All deep learning models opted class balancing techniques to solve this issue; which however should be avoided in any medical diagnosis process. Moreover, the deep learning models are also data hungry and need massive computation resources. Therefore, for quicker diagnosis, this research proposes a novel pinball loss function based one-class support vector machine (PB-OCSVM), that can work in presence of limited COVID-19 positive CXR samples (target class or class-of-interest (CoI) samples) with objectives to maximize the learning efficiency and to minimize the false predictions. The performance of the proposed model is compared with conventional OCSVM and recent deep learning models, and the experimental results prove that the proposed model outperformed state-of-the-art methods. To validate the robustness of the proposed model, experiments are also performed with noisy CXR images and UCI benchmark datasets.

Learnable quantile polynomial chaos expansion: An uncertainty quantification method for interval reliability analysis

Various uncertainties like random input and data noise in aerospace engineering affect flight vehicle safety, making uncertainty quantification important for improving system reliability. Polynomial chaos expansion (PCE) is a versatile method for quantifying stochastic systems’ uncertainty caused by random input, while it cannot quantify data uncertainty caused by noise. To solve this problem, a novel uncertainty quantification method, learnable quantile PCE (LQPCE), is proposed for interval reliability analysis. Unlike the traditional PCE, the multi-dimensional orthonormal basis is constructed for combination input rather than only random variables, and the expansion coefficients are parameterized to learnable weights, ensuring the LQPCE model can capture data noise’s features. The LQPCE Pinball loss function is constructed to learn parameterized expansion coefficients by quantile level iterative sampling, achieving accurate prediction and data uncertainty quantification. The quantified data uncertainty is used for interval reliability analysis, calculating more reliable results about the influence of input uncertainty on stochastic systems’ reliability. Three numerical examples and one engineering application validate the LQPCE method’s effectiveness. The results show that the LQPCE method can quantify data uncertainty accurately and obtain credible interval reliability.

A novel day-ahead regional and probabilistic wind power forecasting framework using deep CNNs and conformalized regression forests

Regional forecasting is crucial for a balanced energy delivery system and for achieving the global transition to clean energy. However, regional wind forecasting is challenging due to uncertain weather prediction and its high dimensional nature. Most solutions are limited to single-turbine or farm/park forecasting; therefore, this work proposes a day-ahead regional wind power forecasting framework using deep Convolutional Neural Networks (CNN) with context-aware turbine maps and Conformal Quantile Regression (CQR) to generate quantile forecasts with valid coverage.Additionally, this work introduces the use of the Split Conformal Predictive System (SCPS) to generate valid prediction distributions, which has not yet been proposed for wind power forecasting in general. As well as a new method to generate calibrated prediction distributions based on SCPS and Quantile Regression Forests (QRF). This new method, named Split Conformal Distribution Regression Forests (SCDRF), allows for conditional conformal predictive distribution that increases efficiency compared to SCPS while maintaining valid coverage. SCDRF, together with CNNs and context-aware turbine maps, outperforms the existing models on the evaluated dataset, reducing the pinball loss by 5.89% while having more flexibility due to the generation of prediction distributions that can be used to generate any quantile prediction without retraining the model.

Multi-view universum support vector machines with insensitive pinball loss

Multi-view learning (MVL) is a promising field that seeks to make the most of information shared across different views, ultimately improving generalization performance. Numerous multi-view support vector machines (SVMs) have been proposed and shown great success. However, hinge loss is widely used by existing methods, which makes these SVMs sensitive to noise and unstable when re-sampling. Furthermore, many researchers have demonstrated the effectiveness of universum data for supervised learning. This paper proposes a novel MVL method called multi-view universum support vector machine with insensitive pinball loss (Pin-MvUSVM). Specifically, we first integrate the pinball loss into the multi-view learning framework, which is dependent on the quantization distance and is robust to noise. Then, we utilize universum data to explore prior knowledge for MVL, which improves the generalization performance. Moreover, we develop multi-view universum twin support vector machine with insensitive pinball loss (Pin-MvUTSVM) as a low-complexity alternative to improve computational efficiency. It solves a pair of smaller-scale quadratic programming problems (QPPs) and utilizes the dual form of the model to solve two convex optimization problems for obtaining the prediction classifier. We conduct classification experiments on four multi-view datasets, and our results show that the proposed convex models outperform several state-of-the-art models.

Support matrix machine with truncated pinball loss for classification

With the expansion of vector-based classifiers to matrix-based classifiers, noise insensitivity and sparsity have always been the focal points. Existing SMM and Pin-SMM enjoy the former and the latter separately. To remedy the shortcoming, we propose a support matrix machine based on truncated pinball loss (TPin-SMM) in this paper, which integrates noise insensitivity and sparsity simultaneously. Thanks to the adding of two quantiles, it possesses precious properties including Bayes rule and bounding misclassification error as well. Concerning the non-convexity of TPin-SMM, a targeted CCCP-ADMM algorithm is established, which decomposes the problem into three sub-problems of each sub-iteration. To verify the validity of TPin-SMM, we have conducted numerical experiments on image datasets, EEG signal sets and Daimler Pedestrian Classification Benchmark dataset with different noises, all of them pass the statistical tests.

A novel fatigue design modeling method under small-sample test data with generalized fiducial theory

Understanding the correlation between the fatigue life of engineering materials and the applied stress is a crucial aspect of reliability and safety design. However, small data observations frequently occur in fatigue testing due to factors like time and budget limitations, as well as constraints related to the availability of testing materials and resources. In this article, fiducial inference method is employed for modeling P-S-N curves to deal with scenarios with small sample sizes. Fiducial inference can be seen as a procedure that provides a measure within a parameter space while requiring fewer assumptions than Bayesian inference (no prior). The performance of the proposed method is evaluated by comparing it with the ISO method using statistical simulation data and aluminum alloy 2524-T3 data. In scenarios with ample data (no less than 15 observations at each of the four stress levels), the proposed fiducial-based method yields results comparable to those obtained through the ISO method. When dealing with small-sample data (around 2–4 observations for each of the four stress levels) and medium-sample data (about 5–10 observations per stress level), the proposed fiducial-based method consistently outperforms the ISO method in terms of the quantile scores (also known as pinball loss). This shows the advantageous performance of the fiducial method under conditions of limited data availability. Besides, the consistency in performance across varying data size scenarios underscores the reliability and robustness of the fiducial-based approach in estimating probabilistic S-N curves.