Mean Squared Error Loss Function Research Articles

Reconstructing Spectra from RGB Images by Relative Error Least-Squares Regression

Spectral reconstruction (SR) algorithms attempt to map RGB- to hyperspectral-images. Classically, simple pixel-based regression is used to solve for this SR mapping and more recently patch-based Deep Neural Networks (DNN) are considered (with a modest performance increment). For either method, the 'training' process typically minimizes a Mean-Squared-Error (MSE) loss. Curiously, in recent research, SR algorithms are evaluated and ranked based on a relative percentage error, so-called MeanRelative-Absolute Error (MRAE), which behaves very differently from the MSE loss function. The most recent DNN approaches - perhaps unsurprisingly - directly optimize for this new MRAE error in training so as to match this new evaluation criteria.<br/> In this paper, we show how we can also reformulate pixelbased regression methods so that they too optimize a relative spectral error. Our Relative Error Least-Squares (RELS) approach minimizes an error that is similar to MRAE. Experiments demonstrate that regression models based on RELS deliver better spectral recovery, with up to a 10% increment in mean performance and a 20% improvement in worst-case performance depending on the method.

A Supervised Speech Enhancement Approach with Residual Noise Control for Voice Communication

For voice communication, it is important to extract the speech from its noisy version without introducing unnaturally artificial noise. By studying the subband mean-squared error (MSE) of the speech for unsupervised speech enhancement approaches and revealing its relationship with the existing loss function for supervised approaches, this paper derives a generalized loss function that takes residual noise control into account with a supervised approach. Our generalized loss function contains the well-known MSE loss function and many other often-used loss functions as special cases. Compared with traditional loss functions, our generalized loss function is more flexible to make a good trade-off between speech distortion and noise reduction. This is because a group of well-studied noise shaping schemes can be introduced to control residual noise for practical applications. Objective and subjective test results verify the importance of residual noise control for the supervised speech enhancement approach.

ADAPTIVE-NET: deep computed tomography reconstruction network with analytical domain transformation knowledge

Recently, the paradigm of computed tomography (CT) reconstruction has shifted as the deep learning technique evolves. In this study, we proposed a new convolutional neural network (called ADAPTIVE-NET) to perform CT image reconstruction directly from a sinogram by integrating the analytical domain transformation knowledge. In the proposed ADAPTIVE-NET, a specific network layer with constant weights was customized to transform the sinogram onto the CT image domain via analytical back-projection. With this new framework, feature extractions were performed simultaneously on both the sinogram domain and the CT image domain. The Mayo low dose CT (LDCT) data was used to validate the new network. In particular, the new network was compared with the previously proposed residual encoder-decoder (RED)-CNN network. For each network, the mean square error (MSE) loss with and without VGG-based perceptual loss was compared. Furthermore, to evaluate the image quality with certain metrics, the noise correlation was quantified via the noise power spectrum (NPS) on the reconstructed LDCT for each method. CT images that have clinically relevant dimensions of 512×512 can be easily reconstructed from a sinogram on a single graphics processing unit (GPU) with moderate memory size (e.g., 11 GB) by ADAPTIVE-NET. With the same MSE loss function, the new network is able to generate better results than the RED-CNN. Moreover, the new network is able to reconstruct natural looking CT images with enhanced image quality if jointly using the VGG loss. The newly proposed end-to-end supervised ADAPTIVE-NET is able to reconstruct high-quality LDCT images directly from a sinogram.

A Convolutional Neural Network for Real Time Classification, Identification, and Labelling of Vocal Cord and Tracheal Using Laryngoscopy and Bronchoscopy Video.

The use of artificial intelligence, including machine learning, is increasing in medicine. Use of machine learning is rising in the prediction of patient outcomes. Machine learning may also be able to enhance and augment anesthesia clinical procedures such as airway management. In this study, we sought to develop a machine learning algorithm that could classify vocal cords and tracheal airway anatomy real-time during video laryngoscopy or bronchoscopy as well as compare the performance of three novel convolutional networks for detecting vocal cords and tracheal rings. Following institutional approval, a clinical dataset of 775 video laryngoscopy and bronchoscopy videos was used. The dataset was divided into two categories for use for training and testing. We used three convolutional neural networks (CNNs): ResNet, Inception and MobileNet. Backpropagation and a mean squared error loss function were used to assess accuracy as well as minimize bias and variance. Following training, we assessed transferability using the generalization error of the CNN, sensitivity and specificity, average confidence error, outliers, overall confidence percentage, and frames per second for live video feeds. After the training was complete, 22 models using 0 to 25,000 steps were generated and compared. The overall confidence of classification for the vocal cords and tracheal rings for ResNet, Inception and MobileNet CNNs were as follows: 0.84, 0.78, and 0.64 for vocal cords, respectively, and 0.69, 0.72, 0.54 for tracheal rings, respectively. Transfer learning following additional training resulted in improved accuracy of ResNet and Inception for identifying the vocal cords (with a confidence of 0.96 and 0.93 respectively). The two best performing CNNs, ResNet and Inception, achieved a specificity of 0.985 and 0.971, respectively, and a sensitivity of 0.865 and 0.892, respectively. Inception was able to process the live video feeds at 10 FPS while ResNet processed at 5 FPS. Both were able to pass a feasibility test of identifying vocal cords and tracheal rings in a video feed. We report the development and evaluation of a CNN that can identify and classify airway anatomy in real time. This neural network demonstrates high performance. The availability of artificial intelligence may improve airway management and bronchoscopy by helping to identify key anatomy real time. Thus, potentially improving performance and outcomes during these procedures. Further, this technology may theoretically be extended to the settings of airway pathology or airway management in the hands of experienced providers. The researchers in this study are exploring the performance of this neural network in clinical trials.

Estimation of ARMA Model Order via Artificial Neural Network for Modeling Physiological Systems

Model order estimation is the most important but challenging step for system identification using an autoregressive moving average (ARMA) model. In this paper, we propose an artificial neural network (ANN) structure to estimate the best model order for ARMA modeling of linear, time-invariant systems using the system’s input and output data. The proposed algorithm creates an equivalent ANN structure corresponding to an ARMA model and chooses the best model order using the neural network’s mean squared error (MSE) loss function. The proposed method is validated on simulated ARMA model data and the performance is compared with the Akaike Information Criterion (AIC) and Bayesian Information Criterion (BIC). We considered three hypothetical linear systems and performed 100 Monte Carlo simulations for each model, with different data lengths, and with additive noise. For each of the three simulation models, the proposed method significantly outperformed the AIC and BIC in terms of the correct model order selection. Finally, the proposed ANN-based model order estimator was successfully applied to determine the dynamic relationship between heart rate (HR) and instantaneous lung volume (ILV) using an ARMA model. The results indicate that physiological and biological systems can be modeled with appropriate ARMA models obtained by the proposed algorithm to better understand the system dynamics.

A dual-stream deep convolutional network for reducing metal streak artifacts in CT images

Machine learning and deep learning are rapidly finding applications in the medical imaging field. In this paper, we address the long-standing problem of metal artifacts in computed tomography (CT) images by training a dual-stream deep convolutional neural network for streak removal. While many metal artifact reduction methods exist, even state-of-the-art algorithms fall short in some clinical applications. Specifically, proton therapy planning requires high image quality with accurate tumor volumes to ensure treatment success. We explore a dual-stream deep network structure with residual learning to correct metal streak artifacts after a first-pass by a state-of-the-art interpolation-based algorithm, NMAR. We provide the network with a mask of the streaks in order to focus attention on those areas. Our experiments compare a mean squared error loss function with a perceptual loss function to emphasize preservation of image features and texture. Both visual and quantitative metrics are used to assess the resulting image quality for metal implant cases. Success may be due to the duality of information processing, with one network stream performing local structure correction, while the other stream provides an attention mechanism to destreak effectively. This study shows that image-domain deep learning can be highly effective for metal artifact reduction (MAR), and highlights the benefits and drawbacks of different loss functions for solving a major CT reconstruction challenge.

A Coskewness Shrinkage Approach for Estimating the Skewness of Linear Combinations of Random Variables

The coskewness matrix is a (p)x(p^2) parameter matrix useful for studying the third-order interactions between variables and computing the skewness of linear combinations of variables. The classical sample estimator of the coskewness matrix performs poorly in terms of mean squared error (MSE) when the sample size is small, even in moderate dimensions. A possible approach to improve estimation is to use shrinkage estimators that offer a compromise between the (unbiased) sample coskewness matrix and a (possibly biased) target matrix (with lower estimation variance) with the aim of minimizing the MSE. We improve upon the existing literature by proposing unbiased estimators, using multivariate k-statistics and polykays, for the MSE loss function and by including the possibility of having multiple target matrices. Simulations show the superiority of our estimators in both the single-target and the multi-target setting for skewed data. Accurate coskewness estimation is highly valued in the field of portfolio optimization (among others) where we show that the out-of-sample skewness of the maximum skewness optimized portfolio is higher when using the proposed shrinkage coskewness matrix estimates.

Quantile Regression under Misspecification, with an Application to the U.S. Wage Structure

Quantile regression (QR) fits a linear model for conditional quantiles just as ordinary least squares (OLS) fits a linear model for conditional means. An attractive feature of OLS is that it gives the minimum mean-squared error linear approximation to the conditional expectation function even when the linear model is misspecified. Empirical research using quantile regression with discrete covariates suggests that QR may have a similar property, but the exact nature of the linear approximation has remained elusive. In this paper, we show that QR minimizes a weighted mean-squared error loss function for specification error. The weighting function is an average density of the dependent variable near the true conditional quantile. The weighted least squares interpretation of QR is used to derive an omitted variables bias formula and a partial quantile regression concept, similar to the relationship between partial regression and OLS. We also present asymptotic theory for the QR process under misspecification of the conditional quantile function. The approximation properties of QR are illustrated using wage data from the U.S. census. These results point to major changes in inequality from 1990 to 2000.

Quantile Regression under Misspecification, with an Application to the U.S. Wage Structure

Quantile regression(QR) fits a linear model for conditional quantiles, just as ordinary least squares (OLS) fits a linear model for conditional means. An attractive feature of OLS is that it gives the minimum mean square error linear approximation to the conditional expectation function even when the linear model is misspecified. Empirical research using quantile regression with discrete covariates suggests that QR may have a similar property, but the exact nature of the linear approximation has remained elusive. In this paper, we show that QR can be interpreted as minimizing a weighted mean-squared error loss function for specification error. The weighting function is an average density of the dependent variable near the true conditional quantile. The weighted least squares interpretation of QR is used to derive an omitted variables bias formula and a partial quantile correlation concept, similar to the relationship between partial correlation and OLS. We also derive general asymptotic results for QR processes allowing for misspecification of the conditional quantile function, extending earlier results from a single quantile to the entire process. The approximation properties of QR are illustrated through an analysis of the wage structure and residual inequality in US Census data for 1980, 1990, and 2000. The results suggest continued residual inequality growth in the 1990s, primarily in the upper half of the wage distribution and for college graduates.

Hedging Derivative Securities and Incomplete Markets: An ε-Arbitrage Approach

Given a European derivative security with an arbitrary payoff function and a corresponding set of underlying securities on which the derivative security is based, we solve the optimal-replication problem: Find a self-financing dynamic portfolio strategy—involving only the underlying securities—that most closely approximates the payoff function at maturity. By applying stochastic dynamic programming to the minimization of a mean-squared error loss function under Markov-state dynamics, we derive recursive expressions for the optimal-replication strategy that are readily implemented in practice. The approximation error or “ε” of the optimal-replication strategy is also given recursively and may be used to quantify the “degree” of market incompleteness. To investigate the practical significance of these ε-arbitrage strategies, we consider several numerical examples, including path-dependent options and options on assets with stochastic volatility and jumps.

Automatic Lag Selection in Covariance Matrix Estimation

Authors proposes a nonparametric method for automatically selecting the number of autocovariances to use in computing a heteroskedasticity and autocorrelation consistent covariance matrix. For a given kernel for weighting the autocovariances, we prove that our procedure is asymptotically equivalent to one that is optimal under a mean-squared error loss function. Monte Carlo simulations suggest that our procedure performs tolerably well, although it does result in size distortions.

On loss functions which minimize to conditional expected values and posterior probabilities

A loss function, or objective function, is a function used to compare parameters when fitting a model to data. The loss function gives a distance between the model output and the desired output. Two common examples are the squared-error loss function and the cross entropy loss function. Minimizing the mean-square error loss function is equivalent to minimizing the mean square difference between the model output and the expected value of the output given a particular input. This property of minimization to the expected value is formalized as P-admissibility. The necessary and sufficient conditions for P-admissibility, leading to a parametric description of all P-admissible loss functions, are found. In particular, it is shown that two of the simplest members of this class of functions are the squared error and the cross entropy loss functions. One application of this work is in the choice of a loss function for training neural networks to provide probability estimates.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>