Standard Loss Function Research Articles

Suggestion and invention of recipes using bi-directional LSTMs-based frameworks

Choosing which recipe to eat and which recipe to avoid isn’t that simple for anyone. It takes strenuous efforts and a lot of time for people to calculate the number of calories and P.H level of the dish. In this paper, we propose an ensemble neural network architecture that suggests recipes based on the taste of the person, P.H level and calorie content of the recipes. We also propose a bi-directional LSTMs-based variational autoencoder for generating new recipes. We have ensembled three bi-directional LSTM-based recurrent neural networks which can classify the recipes based on the taste of the person, P.H level of the recipe and calorie content of the recipe. The proposed model also predicts the taste ratings of the recipes for which we proposed a custom loss function which gave better results than the standard loss functions and the model also predicts the calorie content of the recipes. The bi-directional LSTMs-based variational autoencoder after being trained with the recipes which are fit for the person generates new recipes from the existing recipes. After training and testing the recurrent neural networks and the variational autoencoder, we have tested the model with 20 new recipes and got overwhelming results in the experimentation, the variational autoencoders generated a couple of new recipes, which are healthy to the specific person and will be liked by the specific person.

A Simple Generic Method for Effective Boundary Extraction in Medical Image Segmentation

Accurate identification of the shape and position of organs and abnormal objects (e.g., tumors) in medical images plays an important role in surgical planning as well as in the diagnosis and prognosis of diseases. However, this is difficult to achieve from two-dimensional medical images as these images present inaccurate and ambiguous organ boundaries. Further, traditional image processing-based boundary detection methods such as the Canny edge detector and Sobel operator exhibit poor boundary detection performance for images with substantial noise. Recently, the use of deep learning has resulted in improvements in semantic segmentation in medical images. In this paper, we propose a generic boundary-aware loss function to facilitate the effective discernment of the boundaries of organs and abnormal objects in medical images. Specifically, the proposed loss function introduces a boundary area and assigns higher weights to the loss of pixels located in the boundary area than to those in the non-boundary areas, thereby promoting effective learning in the boundary area. The results of experiments conducted using public medical datasets comprising colon polyp, skin lesion, and chest X-ray data indicate that the standard loss functions, such as cross-entropy loss and Dice loss, combined with the proposed boundary-aware loss function, achieve comparable or better performance than those without the boundary-aware loss function.

India VIX and Forecasting Ability of Symmetric and Asymmetric GARCH Models

Volatility forecasting plays an important role in decisions concerning risk assessment, asset valuation and monetary policy formulation. Forecasting implied volatility is a key parameter in pricing of options. Thus, through this paper we attempt to model and test the predictive ability of symmetric GARCH(1,1) and asymmetric TGARCH(1,1) and EGARCH(1,1) models in forecasting the India Volatility Index (VIX). The estimated results confirm the dependency of volatility on its past behavior. It discloses that conditional variance takes longer to disintegrate and the innovations to it are highly persistent in nature. The predictive ability of these models to forecast the direction of the VIX series is evaluated by employing a standard (symmetric) loss function, such as the root mean square error (RMSE), the mean absolute error (MAE), the mean absolute percentage error (MAPE) and Theil’s inequality coefficient. The results show that the GARCH(1,1) provides superior forecasts compared to other models.

Constrained Generative Adversarial Networks

Generative Adversarial Networks (GANs) are a powerful subclass of generative models. Yet, how to effectively train them to reach Nash equilibrium is a challenge. A number of experiments have indicated that one possible solution is to bound the function space of the discriminator. In practice, when optimizing the standard loss function without limiting the discriminator's output, the discriminator may suffer from lack of convergence. To be able to reach the Nash equilibrium in a faster way during training and obtain better generative data, we propose constrained generative adversarial networks, GAN-C, where a constraint on the discriminator's output is introduced. We theoretically prove that our proposed loss function shares the same Nash equilibrium as the standard one, and our experiments on mixture of Gaussians, MNIST, CIFAR-10, STL-10, FFHQ, and CAT datasets show that our loss function can better stabilize training and yield even better high-quality images.

Simultaneous Feature Selection and Classification for Data-Adaptive Kernel-Penalized SVM

Simultaneous feature selection and classification have been explored in the literature to extend the support vector machine (SVM) techniques by adding penalty terms to the loss function directly. However, it is the kernel function that controls the performance of the SVM, and an imbalance in the data will deteriorate the performance of an SVM. In this paper, we examine a new method of simultaneous feature selection and binary classification. Instead of incorporating the standard loss function of the SVM, a penalty is added to the data-adaptive kernel function directly to control the performance of the SVM, by firstly conformally transforming the kernel functions of the SVM, and then re-conducting an SVM classifier based on the sparse features selected. Both convex and non-convex penalties, such as least absolute shrinkage and selection (LASSO), moothly clipped absolute deviation (SCAD) and minimax concave penalty (MCP) are explored, and the oracle property of the estimator is established accordingly. An iterative optimization procedure is applied as there is no analytic form of the estimated coefficients available. Numerical comparisons show that the proposed method outperforms the competitors considered when data are imbalanced, and it performs similarly to the competitors when data are balanced. The method can be easily applied in medical images from different platforms.

Learning on dynamic statistical manifolds.

Hyperbolic balance laws with uncertain (random) parameters and inputs are ubiquitous in science and engineering. Quantification of uncertainty in predictions derived from such laws, and reduction of predictive uncertainty via data assimilation, remain an open challenge. That is due to nonlinearity of governing equations, whose solutions are highly non-Gaussian and often discontinuous. To ameliorate these issues in a computationally efficient way, we use the method of distributions, which here takes the form of a deterministic equation for spatio-temporal evolution of the cumulative distribution function (CDF) of the random system state, as a means of forward uncertainty propagation. Uncertainty reduction is achieved by recasting the standard loss function, i.e. discrepancy between observations and model predictions, in distributional terms. This step exploits the equivalence between minimization of the square error discrepancy and the Kullback-Leibler divergence. The loss function is regularized by adding a Lagrangian constraint enforcing fulfilment of the CDF equation. Minimization is performed sequentially, progressively updating the parameters of the CDF equation as more measurements are assimilated.

Hybrid-loss supervision for deep neural network

Multi-loss-joint-optimization has been proven to be valid in computer vision literature. However, the learned deep sub-features usually fit their disjoint constraints, which yield confrontation and spatial inconsistency among the sub-features with nonshared FC layers. In this paper, we propose a Hybrid-loss supervision (HLS) framework in order to obtain smoother and more spatially consistent features with shared FC layers. First, we analyze the shortcomings of the monitoring with single-loss in the existing framework theoretically. Then, we selected two notable loss functions (e.g., Center loss and Weighted loss) to instantiate the HLS framework by linear combination. By instantiating the framework with two standard loss functions, the network has learned more compact intra-class deep features and uniform inter-class deep features. The HLS framework can significantly boost the efficiency of existing convolution networks for both image classification task and object detection task without increasing network parameters and computational complexity. Extensive experimental results on different vision tasks demonstrate consistent improvement can be achieved across a variety of datasets (e.g., CIFAR-10/100, ImageNet-2012, PASCAL VOC and MS-COCO) and different convolutional neural network architectures.

Cost-sensitive support vector machines

Many machine learning applications involve imbalance class prior probabilities, multi-class classification with many classes (often addressed by one-versus-rest strategy), or “cost-sensitive” classification. In such domains, each class (or in some cases, each sample) requires special treatment. In this paper, we use a constructive procedure to extend SVM’s standard loss function to optimize the classifier with respect to class imbalance or class costs. By drawing connections between risk minimization and probability elicitation, we show that the resulting classifier guarantees Bayes consistency. We further analyze the primal and the dual objective functions and derive the objective function in a regularized risk minimization framework. Finally, we extend the classifier to the with cost-sensitive learning with example dependent costs. We perform experimental analysis on class imbalance, cost-sensitive learning with given class and example costs and show that proposed algorithm provides superior generalization performance, compared to conventional methods.

ITM-Net: Deep Inverse Tone Mapping Using Novel Loss Function Considering Tone Mapping Operator

In this paper, we propose a novel inverse tone mapping network, called “iTM-Net.” For training iTM-Net, we also propose a novel loss function that considers the non-linear relation between low dynamic range (LDR) and high dynamic range (HDR) images. For inverse tone mapping with convolutional neural networks (CNNs), we first point out that training CNNs with a standard loss function causes a problem due to the non-linear relation between the LDR and HDR images. To overcome the problem, the novel loss function non-linearly tone-maps target HDR images into LDR ones on the basis of a tone mapping operator, and the distance between the tone-mapped images and predicted ones are then calculated. The proposed loss function enables us not only to normalize the HDR images but also to reduce the non-linear relation between LDR and HDR ones. The experimental results show that the HDR images predicted by the proposed iTM-Net have higher-quality than the HDR ones predicted by conventional inverse tone mapping methods, including the state of the art, in terms of both HDR-VDP-2.2 and PU encoding + MS-SSIM. In addition, compared with loss functions that do not consider the non-linear relation, the proposed loss function is shown to improve the performance of CNNs.

Towards stroke prediction using electronic health records

BackgroundAs of 2014, stroke is the fourth leading cause of death in Japan. Predicting a future diagnosis of stroke would better enable proactive forms of healthcare measures to be taken. We aim to predict a diagnosis of stroke within one year of the patient’s last set of exam results or medical diagnoses.MethodsAround 8000 electronic health records were provided by Tsuyama Jifukai Tsuyama Chuo Hospital in Japan. These records contained non-homogeneous temporal data which were first transformed into a form usable by an algorithm. The transformed data were used as input into several neural network architectures designed to evaluate efficacy of the supplied data and also the networks’ capability at exploiting relationships that could underlie the data. The prevalence of stroke cases resulted in imbalanced class outputs which resulted in trained neural network models being biased towards negative predictions. To address this issue, we designed and incorporated regularization terms into the standard cross-entropy loss function. These terms penalized false positive and false negative predictions. We evaluated the performance of our trained models using Receiver Operating Characteristic.ResultsThe best neural network incorporated and combined the different sources of temporal data through a dual-input topology. This network attained area under the Receiver Operating Characteristic curve of 0.669. The custom regularization terms had a positive effect on the training process when compared against the standard cross-entropy loss function.ConclusionsThe techniques we describe in this paper are viable and the developed models form part of the foundation of a national clinical decision support system.

Comparison of range-based volatility estimators against integrated volatility in European emerging markets

This paper explores the effectiveness of eight range-based volatility estimators for seven European emerging markets. It offers added value by: (i) finding a consistent and asymptotically unbiased estimator of integrated volatility for emerging markets, (ii) employing the upper tail dependence for comparison purposes, in addition to standard loss functions, and (iii) recommending the appropriate ex-post volatility measure in the lack of high-frequency data. When no strong preference for a specific estimator is found, the upper tail dependence measure is consulted, confirming the MSE-based ranking for Czech Republic, Greece, Poland, and Romania; and the QLIKE-based ranking for Bulgaria, Croatia, and Hungary.

Balanced Mixture of Deformable Part Models With Automatic Part Configurations

This paper presents a method to improve the traditional mixture of deformable part models (MDPM) method from the learning perspective. First, an object part configuration learning algorithm based on group sparsity constraint is introduced to automatically discover the object part number, size, and location. The algorithm imposes two additional regularization terms in addition to the standard hinge loss function. The first term focuses on automatic part selection and the second term focuses on automatic part placement. Second, this paper introduces an improved MDPM training framework. The framework applies a learned transformation to normalize the prediction score from each individual deformable part model (DPM) into a pseudo probability such that the partition of the entire object appearance feature space becomes less sensitive to the prior distributions of different DPMs. Finally, the two proposed improvements are combined and formulated under the expectation-maximization framework. We evaluate our method mainly using the PASCAL VOC2007 and VOC2010 detection benchmarks and show that the proposed learning algorithms could increase the detection mean AP score by 2.4% and 0.9%, respectively, on these two data sets when using the proposed part selection method and the training algorithm. We also present further in-depth analysis of the proposed algorithm in the experiments.

Loss functions for predicted click‐through rates in auctions for online advertising

SummaryWe characterize the optimal loss functions for predicted click‐through rates in auctions for online advertising. Whereas standard loss functions such as mean squared error or log likelihood severely penalize large mispredictions while imposing little penalty on smaller mistakes, a loss function reflecting the true economic loss from mispredictions imposes significant penalties for small mispredictions and only slightly larger penalties on large mispredictions. We illustrate that when the model is misspecified using such a loss function can improve economic efficiency, but the efficiency gain is likely to be small.

ProjE: Embedding Projection for Knowledge Graph Completion

With the large volume of new information created every day, determining the validity of information in a knowledge graph and filling in its missing parts are crucial tasks for many researchers and practitioners. To address this challenge, a number of knowledge graph completion methods have been developed using low-dimensional graph embeddings. Although researchers continue to improve these models using an increasingly complex feature space, we show that simple changes in the architecture of the underlying model can outperform state-of-the-art models without the need for complex feature engineering. In this work, we present a shared variable neural network model called ProjE that fills-in missing information in a knowledge graph by learning joint embeddings of the knowledge graph’s entities and edges, and through subtle, but important, changes to the standard loss function. In doing so, ProjE has a parameter size that is smaller than 11 out of 15 existing methods while performing 37% better than the current-best method on standard datasets. We also show, via a new fact checking task, that ProjE is capable of accurately determining the veracity of many declarative statements.

Sub-optimality of some continuous shrinkage priors

Two-component mixture priors provide a traditional way to induce sparsity in high-dimensional Bayes models. However, several aspects of such a prior, including computational complexities in high-dimensions, interpretation of exact zeros and non-sparse posterior summaries under standard loss functions, have motivated an amazing variety of continuous shrinkage priors, which can be expressed as global–local scale mixtures of Gaussians. Interestingly, we demonstrate that many commonly used shrinkage priors, including the Bayesian Lasso, do not have adequate posterior concentration in high-dimensional settings.

Uncertain Potential Output and Simple Rules in Small Open Economy

This paper analyzes the implications of incomplete information for the conduct of monetary policy in small-open economy. I use a standard theoretical DSGE model to evaluate the performance of simple rules, including the exchange rate peg. Incomplete information is modeled assuming that the central bank and the private sector observe domestic inflation and output with a measurement error, while they do not observe potential output. I show that not reacting to the exchange rate yields better outcomes in terms of a standard loss function. For the case of complete information and incomplete information, I quantify for which parameter configuration a Taylor rule reacting to both the exchange rate and the domestic inflation rate yields a higher loss than the fixed exchange rate regime.

Estimating the Public Health Burden Associated With Adverse Pregnancy Outcomes Resulting From Syphilis Infection Across 43 Countries in Sub-Saharan Africa.

Untreated syphilis in pregnancy is associated with adverse clinical outcomes to the infant. The study aimed to estimate the public health burden resulting from adverse pregnancy outcomes due to syphilis infection among pregnant women not screened for syphilis in 43 countries in sub-Saharan Africa. Estimated country-specific incidence of syphilis was generated from annual number of live births, the proportion of women with at least 1 antenatal care (ANC) visit, the syphilis prevalence rate, and the proportion of women screened for syphilis during ANC.Adverse pregnancy outcome data (stillbirth, neonatal death, low birth weight, and congenital syphilis) were obtained from published sources. Disability-adjusted life-year (DALY) estimates were calculated using undiscounted local life expectancy, the neonatal standard loss function, and relevant disability weights. The model assessed the potential impact of raising ANC coverage to at least 95% and syphilis screening to at least 95% (World Health Organization targets). For all 43 sub-Saharan Africa countries, the estimated incidence of adverse pregnancy outcomes was 205,901 (95% confidence interval [CI], 113,256-383,051) per year, including stillbirth (88,376 [95% CI, 60,854-121,713]), neonatal death (34,959 [95% CI, 23,330-50,076]), low birth weight (22,483 [95% CI, 0-98,847]), and congenital syphilis (60,084 [95% CI, 29,073-112,414]), resulting in approximately 12.5 million DALYs. Countries with the greatest burden are (in DALYs, millions) Democratic Republic of the Congo (1.809), Nigeria (1.598), Ethiopia (1.466), and Tanzania (0.961). Attaining World Health Organization targets could reduce the burden by 8.5 million DALYs. Substantial infant mortality and morbidity results from maternal syphilis infection concentrated in countries with low access to ANC or low rates of syphilis screening.

Should the monetary policy rule be different in a financial crisis?

This article reviews the finding that standard loss functions in output and inflation are higher during discretionary periods than in periods during which monetary policy is described by a rule, such as the Taylor rule. It shows that the finding is consistent with earlier research, but argues that we really do not know if the Taylor rule would have improved performance during the recent financial crisis. The article then considers modifications of policy rules to deal with changes in interest rate spreads, credit aggregates and banks׳ balance sheets.

Multiscale support vector regression method in Sobolev spaces on bounded domains

In this paper, we investigate the multiscale support vector regression (SVR) method for approximation of functions in Sobolev spaces on bounded domains. The Vapnik ϵ-intensive loss function, which has been developed well in learning theory, is introduced to replace the standard l2 loss function in multiscale least squares methods. Convergence analysis is presented to verify the validity of the multiscale SVR method with scaled versions of compactly supported radial basis functions. Error estimates on noisy observation data are also derived to show the robustness of our proposed algorithm. Numerical simulations support the theoretical predictions.

The Influence of the Taylor Rule on US Monetary Policy

We analyze the in‡uence of the Taylor rule on US monetary policy by estimating the policy preferences of the Fed within a DSGE framework. The policy preferences are represented by a standard loss function, extended with a term that represents the degree of reluctance to letting the interest rate deviate from the Taylor rule. The empirical support for the presence of a Taylor rule term in the policy preferences is strong and robust to alternative speci…cations of the loss function. Analyzing the Fed’s monetary policy in the period 2001 –2006, we …nd no support for a decreased weight on the Taylor rule, contrary to what has been argued in the literature. The large deviations from the Taylor rule in this period are due to large, negative demand-side shocks, and represent optimal deviations for a given weight on the Taylor rule.