Quadratic Loss Research Articles

Introduction to Neutrosophic Bayes Estimation Theory

This research presents the concept of neutrosophic Bayesian estimation defining the neutrosophic loss function, neutrosophic risk function, neutrosophic posterior risk function and neutrosophic maximum a posteriori estimator. Minimization of the neutrosophic posterior risk of the estimator is also discussed. An algebraic isomorphism is used to simplify equations solving. As an application of the presented theorems, a sample drawn from a neutrosophic gamma distribution with a conjugate prior is discussed and studied and the parameter of the formulated distribution is successfully estimated using neutrosophic quadratic loss function which results an estimator that equals the posterior mean.

Optimal capital allocation for individual risk model using a Mean-Variance Principle

<p style='text-indent:20px;'>In this article, we study the problem of optimal capital allocation for the individual risk model using the Mean-Variance principle with quadratic loss function when a total amount of initial capital is granted for allocation. The determination of the optimal capital allocation strategy proceeds in two phases. First, explicit formulas for the optimal allocations are presented based on the assumption that the claim occurrence indicators are independent of the claim severities when the allocated total capital is fixed. Second, an approximating algorithm is proposed to find out the optimal value of capital used for allocation through minimizing the mean-variance loss function. As a result, the exact allocation policy can be provided by following the first phase again. Numerical examples and applications are provided to illustrate the main results and some discussions on the effect of neglecting the dependence between the claim occurrence indicators and claim severities on the optimal allocations are also given to shed light on future research.</p>

ε-Insensitive Least Squares Support Vector Regression with Sequential Minimal Optimization

The [Formula: see text]-insensitive-Least Squares Support Vector Regression ([Formula: see text]-LSSVR) optimization problem with [Formula: see text]-insensitive quadratic loss function is a non-smooth, piecewise quadratic complex problem. Therefore, the optimal solution of [Formula: see text]-LSSVR takes longer than Classic LSSVR. Additionally, due to its unbounded influence function, [Formula: see text]-LSSVR, like classical LSSVR, shows poor generalization ability in the presence of outliers. To overcome the above-mentioned drawbacks, this paper presents an [Formula: see text]-insensitive Weighted Least Squares Support Vector Regression ([Formula: see text]-WLSSVR) with equality constraints to increase robustness against outliers. To reduce the computational complexity and time, the optimization problem of [Formula: see text]-WLSSVR is solved by the Sequential Minimal Optimization (SMO) algorithm. Extensive simulations have shown that the presented [Formula: see text]-WLSSVR model combined with the SMO algorithm is more effective and efficient than LSSVR and its known sparse variants.

Effect of measurement uncertainty on combined quality control charts

The accuracy of the measurement system is vital for reliable process monitoring using statistical process control charts. The applied chart’s effectiveness depends on the measurement system's performance. Measurement uncertainty can lead to incorrect decisions like unnecessary stops or failure to intervene. In this paper, we investigated the effect of measurement errors on the performance of four well-established combined charts for monitoring the mean of normally distributed processes: Shewhart-CUSUM, Shewhart-Crosier’s CUSUM, Shewhart-EWMA and Shewhart-GWMA charts. To deal with measurement errors we considered the additive measurement error model. Detailed run length profiles of these charts are studied in terms of average run length (ARL), extra quadratic loss, relative ARL, and performance comparison index through Monte Carlo simulations under different sizes of measurement errors. It was found that measurement errors significantly reduce the power of the combined charts. Thus, multiple measurements scheme is incorporated as a remedy to this effect. The Shewhart-Crosier’s CUSUM performed best of four charts, while the Shewhart-EWMA chart did worst. To demonstrate the effect of measurement uncertainty and highlight implications further, a simulated dataset with a shift in the process mean is considered.

Monitoring of Linear Profiles Using Linear Mixed Model in the Presence of Measurement Errors

In the application of control charts, most of the research in profile monitoring is based on accurate measurements. Measurement errors, however, often exist in many manufacturing and service environments. In this paper, we apply linear mixed models in the presence of measurement errors in fixed effects. We discuss three modified multivariate charts, namely Hotelling’s T2, multivariate exponential weighted moving average (MEWMA) control chart, and multivariate cumulative sum (MCUSUM) control chart. Performance comparisons are made in terms of the average run length (ARL) and average extra quadratic loss (AEQL). Finally, a real data example on healthcare expenditures is used to illustrate the implementation of the proposed monitoring schemes.

Bayesian Estimation of a Scale Parameter of the Gumbel-Lomax Distribution Using Informative and Non Informative Priors

Estimating the scale parameter of the Gumbel-Lomax Distribution using the Bayesian method of estimation and evaluating the estimators by assuming two non-informative prior distributions and one informative prior distribution is very important for the general application of the Gumbel-Lomax distribution. These estimators are obtained using the squared error loss function (SELF), Quadratic loss function (QLF) and precautionary loss function (PLF). The posterior distributions of the scale parameter of the Gumbel-Lomax distribution are derived and the Estimators are also obtained using the above mentioned priors and loss functions. Furthermore, a simulation using a package in R software is carried out to assess the performance of the estimators by making use of the Mean Squared Errors of the Estimators under the Bayesian approach and Maximum likelihood method. Our results show that Bayesian Method using PLF under all priors produces the best estimators of the scale parameter compared to estimators using the Maximum Likelihood method, SELF and QLF under all the priors irrespective of the values of the parameters and the different sample sizes. It is also discovered that the other parameters have no effect on the estimators of the scale parameter.

Loss Function Dynamics and Landscape for Deep Neural Networks Trained with Quadratic Loss

Knowledge of the loss landscape geometry makes it possible to successfully explain the behavior of neural networks, the dynamics of their training, and the relationship between resulting solutions and hyperparameters, such as the regularization method, neural network architecture, or learning rate schedule. In this paper, the dynamics of learning and the surface of the standard cross-entropy loss function and the currently popular mean squared error (MSE) loss function for scale-invariant networks with normalization are studied. Symmetries are eliminated via the transition to optimization on a sphere. As a result, three learning phases with fundamentally different properties are revealed depending on the learning step on the sphere, namely, convergence phase, phase of chaotic equilibrium, and phase of destabilized learning. These phases are observed for both loss functions, but larger networks and longer learning for the transition to the convergence phase are required in the case of MSE loss.

Longitudinal regression of covariance matrix outcomes.

In this study, a longitudinal regression model for covariance matrix outcomes is introduced. The proposal considers a multilevel generalized linear model for regressing covariance matrices on (time-varying) predictors. This model simultaneously identifies covariate-associated components from covariance matrices, estimates regression coefficients, and captures the within-subject variation in the covariance matrices. Optimal estimators are proposed for both low-dimensional and high-dimensional cases by maximizing the (approximated) hierarchical-likelihood function. These estimators are proved to be asymptotically consistent, where the proposed covariance matrix estimator is the most efficient under the low-dimensional case and achieves the uniformly minimum quadratic loss among all linear combinations of the identity matrix and the sample covariance matrix under the high-dimensional case. Through extensive simulation studies, the proposed approach achieves good performance in identifying the covariate-related components and estimating the model parameters. Applying to a longitudinal resting-state functional magnetic resonance imaging data set from the Alzheimer's Disease (AD) Neuroimaging Initiative, the proposed approach identifies brain networks that demonstrate the difference between males and females at different disease stages. The findings are in line with existing knowledge of AD and the method improves the statistical power over the analysis of cross-sectional data.

Efficient Representation of Large-Alphabet Probability Distributions

A number of engineering and scientific problems require representing and manipulating probability distributions over large alphabets, which we may think of as long vectors of reals summing to 1. In some cases it is required to represent such a vector with only b bits per entry. A natural choice is to partition the interval 0,1 into 2b uniform bins and quantize entries to each bin independently. We show that a minor modification of this procedure – applying an entrywise non-linear function (compander) f(x) prior to quantization – yields an extremely effective quantization method. For example, for b=8(16) and 105-sized alphabets, the quality of representation improves from a loss (under KL divergence) of 0.5(0.1) bits/entry to 10-4(10-9) bits/entry. Compared to floating point representations, our compander method improves the loss from 10-1(10-6) to 10-4(10-9) bits/entry. These numbers hold for both real-world data (word frequencies in books and DNA k-mer counts) and for synthetic randomly generated distributions. Theoretically, we analyze a minimax optimality criterion and show that the closed-form compander f(x)ArcSinh(cK(KlogK)x) is (asymptotically as b∞) optimal for quantizing probability distributions over a -letter alphabet. Non-asymptotically, such a compander (substituting 1/2 for cfor simplicity) has KL-quantization loss bounded by ≤8∙2-2blog2. Interestingly, a similar minimax criterion for the quadratic loss on the hypercube shows optimality of the standard uniform quantizer. This suggests that the ArcSinh quantizer is as fundamental for KL-distortion as the uniform quantizer for quadratic distortion.

On designing an optimal SPRT control chart with estimated process parameters under guaranteed in-control performance

Being one of the most sophisticated control charts, the sequential probability ratio test (SPRT) chart possesses fast detection ability across a broad range of process shifts. The SPRT chart has the advantage of sampling only a modest number of observations. Thus far, the SPRT chart has been developed under the assumption that the process parameters are known. As process parameters are often unknown in practice, this article reveals that parameter estimation from a limited amount of Phase-I data, leads to excessive false alarms and unstable chart’s performances. To counter these problems, this article advocates the use of adjusted control limits to ensure that a sufficiently high proportion of the in-control conditional average time to signal value, is greater than a pre-specified level. This increasingly prevalent design of control charts is known as Guaranteed In-Control Performance (GICP). In this article, we propose an optimization design for the SPRT chart with estimated process parameters, by minimizing the expected value of the average extra quadratic loss under the GICP framework. Theoretical derivations by means of the Markov chain approach are developed in this article to evaluate the run-length properties of the SPRT chart with estimated process parameters. Results show that the overall performance of the proposed optimal SPRT chart is almost twice as good as the optimal CUSUM chart over a given range of process mean shifts. Finally, an implementation of the proposed optimal SPRT chart is demonstrated with real industrial data obtained from an epitaxial process.

Optimal Bayesian sampling plan for censored competing risks data

There is a substantial amount of literature in the area of acceptance sampling plan with censored lifetime data. However, the optimality of a Bayesian sampling plan in the presence of competing risks has not been considered so far. In this paper, first, the Bayesian sampling plans (BSP) for Type-II and Type-I hybrid censoring schemes are discussed in presence of competing risks when the lifetime distribution is exponential. The closed-form expression of the Bayes decision function is obtained analytically for a linear loss function. Then we consider the Weibull distribution with an unknown shape parameter under Type-I hybrid censoring scheme in presence of competing risks to obtain the BSP. However, the Bayes decision function cannot be obtained in closed-form for a general loss function, and in such cases, a numerical algorithm is proposed. As an illustration, in the exponential case, a quadratic loss function, and in Weibull case, a non-polynomial loss function, are considered for the application of the proposed numerical approach to obtain the optimum BSPs using the Bayes decision function.

Efficient Control Charting Scheme for the Process Location with Application in Automobile Industry

The recently introduced triple exponentially weighted moving average (TEWMA) chart is the extended form of the classical EWMA and double EWMA (DEWMA) charts. On the other hand, the auxiliary information-based (AIB) homogeneously weighted moving average ( HWMA AIB ) chart is used for the monitoring of process location shifts efficiently as compared to the HWMA chart. Combining TEWMA with HWMA features is a new idea and is not seen in the literature until now. So, the objective of this study is to combine the idea of HWMA AIB and TEWMA charts and propose an AIB triple HWMA, symbolized as THWMA AIB chart to further improve the process location shift monitoring. The proposed THWMA AIB chart is developed by combining the AIB double HWMA plotting statistic into the other HWMA chart. The numerical results are computed using the Monte Carlo simulation method. Average run length ( ARL ), relative ARL , performance comparison index, and extra quadratic loss are used as comparison tools for the proposed THWMA AIB chart with the classical EWMA, TEWMA, mixed HWMA-CUSUM (MHC), AIB EWMA ( EWMA AIB ), HWMA, double HWMA (DHWMA), HWMA AIB , AIB mixed EWMA-CUSUM ( MEC AIB ), and AIB mixed CUSUM-EWMA ( MCE AIB ) charts. Finally, a practical application is also provided for users to demonstrate the proposed study’s vitality.

Restricted Stein-rule estimation in ultrastructural linear measurement error models

We consider a multivariate ultrastructural measurement error model when some information on regression coefficients is available in the form of linear restriction. We propose Stein-rule estimation for the regression coefficients. Without any assumption about the distribution for the random components, the asymptotic risk under a specific quadratic loss function is studied and a sufficient condition for the dominance of the proposed estimator over a consistent estimator. We also carry out a simulation study to demonstrate the finite sample properties of the estimators.

Frequentist model averaging for zero‐inflated Poisson regression models

AbstractThis paper considers frequentist model averaging for estimating the unknown parameters of the zero‐inflated Poisson regression model. Our proposed weight choice procedure is based on the minimization of an unbiased estimator of a conditional quadratic loss function. We prove that the resulting model average estimator enjoys optimal asymptotic property and improves finite sample properties over the two commonly used information‐based model selection estimators and their model average estimators via simulation studies. The proposed method is illustrated by a real data example.

A Hybrid Stochastic-Deterministic Minibatch Proximal Gradient Method for Efficient Optimization and Generalization.

Despite the success of stochastic variance-reduced gradient (SVRG) algorithms in solving large-scale problems, their stochastic gradient complexity often scales linearly with data size and is expensive for huge data. Accordingly, we propose a hybrid stochastic-deterministic minibatch proximal gradient~(HSDMPG) algorithm for strongly convex problems with linear prediction structure, e.g.~least squares and logistic/softmax regression. HSDMPG~enjoys improved computational complexity that is data-size-independent for large-scale problems. It iteratively samples an evolving~minibatch of individual losses to estimate the original problem, and efficiently minimizes the sampled smaller-sized subproblems. For strongly convex loss of n components, HSDMPG~attains an ϵ-optimization-error within [Formula: see text] stochastic gradient evaluations, where κ is condition number, ζ = 1 for quadratic loss and ζ = 2 for generic loss. For large-scale problems, our complexity outperforms those of SVRG-type algorithms with/without dependence on data size. Particularly, when ϵ = O(1/√n) which matches the intrinsic excess error of a learning model and is sufficient for generalization, our complexity for quadratic and generic losses is respectively O (n0.5log2(n)) and O (n0.5log3(n)), which for the first time achieves optimal generalization in less than a single pass over data. Besides, we extend HSDMPG~to online strongly convex problems and prove its higher efficiency over the prior algorithms. Numerical results demonstrate the computational advantages of~HSDMPG.

Development of a methodology for assessing the adequacy of electric power systems

This study presents the results of a research into the developing a methodology for assessing the adequacy of advanced electric power systems characterized by the integration of various innovative technologies, which complicates their analysis. The methodology development is aimed at solving two main problems: 1) increase the adequacy of modeling the processes that occur in the electric power system and 2) enhance the computational efficiency of the adequacy assessment methodology. This study proposes a new mathematical model to minimize the power shortage and enhance the adequacy of modeling the processes. The model considers quadratic power transmission losses and network coefficients. The computational efficiency of the adequacy assessment methodology is enhanced using efficient random- number generators to form the calculated states of electric power systems and machine learning methods to assess power shortages and other reliability characteristics in the calculated states.

An Algebraic Estimator for Large Spectral Density Matrices

We propose a new estimator of high-dimensional spectral density matrices, called ALgebraic Spectral Estimator (ALSE), under the assumption of an underlying low rank plus sparse structure, as typically assumed in dynamic factor models. The ALSE is computed by minimizing a quadratic loss under a nuclear norm plus l 1 norm constraint to control the latent rank and the residual sparsity pattern. The loss function requires as input the classical smoothed periodogram estimator and two threshold parameters, the choice of which is thoroughly discussed. We prove consistency of ALSE as both the dimension p and the sample size T diverge to infinity, as well as the recovery of latent rank and residual sparsity pattern with probability one. We then propose the UNshrunk ALgebraic Spectral Estimator (UNALSE), which is designed to minimize the Frobenius loss with respect to the pre-estimator while retaining the optimality of the ALSE. When applying UNALSE to a standard U.S. quarterly macroeconomic dataset, we find evidence of two main sources of comovements: a real factor driving the economy at business cycle frequencies, and a nominal factor driving the higher frequency dynamics. The article is also complemented by an extensive simulation exercise. Supplementary materials for this article are available online.

Data Depth and Multiple Output Regression, the Distorted M-Quantiles Approach

For a univariate distribution, its M-quantiles are obtained as solutions to asymmetric minimization problems dealing with the distance of a random variable to a fixed point. The asymmetry refers to the different weights awarded to the values of the random variable at either side of the fixed point. We focus on M-quantiles whose associated losses are given in terms of a power. In this setting, the classical quantiles are obtained for the first power, while the expectiles correspond to quadratic losses. The M-quantiles considered here are computed over distorted distributions, which allows to tune the weight awarded to the more central or peripheral parts of the distribution. These distorted M-quantiles are used in the multivariate setting to introduce novel families of central regions and their associated depth functions, which are further extended to the multiple output regression setting in the form of conditional and regression regions and conditional depths.

Color Image Inpainting via Robust Pure Quaternion Matrix Completion: Error Bound and Weighted Loss

In this paper, we study color image inpainting as a pure quaternion matrix completion problem. In the literature, the theoretical guarantee for quaternion matrix completion is not well-established. Our main aim is to propose a new minimization problem with an objective combining nuclear norm and a quadratic loss weighted among three channels. To fill the theoretical vacancy, we obtain the error bound in both clean and corrupted regimes, which relies on some new results of quaternion matrices. A general Gaussian noise is considered in robust completion where all observations are corrupted. Motivated by the error bound, we propose to handle unbalanced or correlated noise via a cross-channel weight in the quadratic loss, with the main purpose of rebalancing noise level, or removing noise correlation. Extensive experimental results on synthetic and color image data are presented to confirm and demonstrate our theoretical findings.

Quantile regression version of Hodrick–Prescott filter

Hodrick–Prescott (HP) filter is a popular trend filtering method for univariate macroeconomic time series such as real gross domestic product. This paper considers the quantile regression version of HP filter (qHP filter), which is a filtering method defined by replacing quadratic loss function of HP filter with quantile regression loss function. One of the essential properties of quantile regression is that if the regression includes intercept, then the ratio of negative residuals can be almost controlled. Does the suggested qHP filter also have the property? This paper answers this question. In addition to the main result, we provide an empirical illustration.