Optimal Loss Function Research Articles

Image enhancement by deep neural networks using high-level information

A method is investigated for training neural networks for image enhancement, based on using information from the features of neural networks trained in image classification. Experiments are performed to identify the optimal loss function that achieves maximum precision in the classification of images superposed with noise or blurring. The dependence of the best configuration of training parameters on the type of detrimental influence and target is demonstrated. This is the first study, to our knowledge, to compare the influence of such a loss function on the precision of the restoration and recognition with the utilization of a single classifier trained under the influence of distorting factors. We show that it is reasonable to correct some simple distortions “outside” the classifier, while others are better corrected “inside.”

Loss Architecture Search for Few-Shot Object Recognition

Few-shot object recognition, which exploits a set of well-labeled data to build a classifier for new classes that have only several samples per class, has received extensive attention from the machine learning community. In this paper, we investigate the problem of designing an optimal loss function for few-shot object recognition and propose a novel few-shot object recognition system that includes the following three steps: (1) generate a loss function architecture using a recurrent neural network (generator); (2) train a base embedding network with the generated loss function on a training set; (3) fine-tune the base embedding network using the few-shot instances from a validation set to obtain the accuracy and use it as a reward signal to update the generator. This procedure is repeated and implemented in the reinforcement learning framework for finding the best loss architecture such that the embedding network yields the highest validation accuracy. Our key insight is to create a search space of the loss function architectures and evaluate the quality of a particular loss function on the dataset of interest. We conduct experiments on three popular datasets for few-shot learning. The results show that the proposed approach achieves better performance than state-of-the-art methods.

Tailored optimal posttreatment surveillance for cancer recurrence.

A substantial rise in the number of cancer survivors has led to urgent management questions regarding effective posttreatment imaging-based surveillance strategies for cancer recurrence. Current surveillance guidelines provided by a number of professional societies all warn against overly aggressive surveillance, especially for low-risk patients, but all fail to provide more specific directions to accommodate underlying heterogeneity of cancer recurrence. Therefore it is imperative to develop data-driven strategies that can tailor the surveillance schedules to recurrence risk in this era of stricter insurance regulations, provider shortages, and rising costs of health care. Due to a lack of statistical methods for optimizing surveillance scheduling in presence of competing risks, we propose a general approach that uses an intuitive loss function for optimization of early detection of recurrence before death. The proposed strategies can tailor to patient risks of recurrence, in terms of both intensity and amount of surveillance. Using general three-state Markov models, our method is flexible and includes earlier works as special cases. We illustrate our method in both simulation studies and an application to breast cancersurveillance.

Robust Time-Consistent Portfolio Selection for an Investor under CEV Model with Inflation Influence

A robust time-consistent optimal investment strategy selection problem under inflation influence is investigated in this article. The investor may invest his wealth in a financial market, with the aim of increasing wealth. The financial market includes one risk-free asset, one risky asset, and one inflation-indexed bond. The price process of the risky asset is governed by a constant elasticity of variance (CEV) model. The investor is ambiguity-averse; he doubts about the model setting under the original probability measure. To dispel this concern, he seeks a set of alternative probability measures, which are absolutely continuous to the original probability measure. The objective of the investor is to seek a time-consistent strategy so as to maximize his expected terminal wealth meanwhile minimizing his variance of the terminal wealth in the worst-case scenario. By using the stochastic optimal control technique, we derive closed-form solutions for the optimal time-consistent investment strategy, the probability scenario, and the value function. Finally, the influences of model parameters on the optimal investment strategy and utility loss function are examined through numerical experiments.

Machine learning approaches to predict peak demand days of cardiovascular admissions considering environmental exposure

BackgroundAccumulating evidence has linked environmental exposure, such as ambient air pollution and meteorological factors, to the development and severity of cardiovascular diseases (CVDs), resulting in increased healthcare demand. Effective prediction of demand for healthcare services, particularly those associated with peak events of CVDs, can be useful in optimizing the allocation of medical resources. However, few studies have attempted to adopt machine learning approaches with excellent predictive abilities to forecast the healthcare demand for CVDs. This study aims to develop and compare several machine learning models in predicting the peak demand days of CVDs admissions using the hospital admissions data, air quality data and meteorological data in Chengdu, China from 2015 to 2017.MethodsSix machine learning algorithms, including logistic regression (LR), support vector machine (SVM), artificial neural network (ANN), random forest (RF), extreme gradient boosting (XGBoost), and light gradient boosting machine (LightGBM) were applied to build the predictive models with a unique feature set. The area under a receiver operating characteristic curve (AUC), logarithmic loss function, accuracy, sensitivity, specificity, precision, and F1 score were used to evaluate the predictive performances of the six models.ResultsThe LightGBM model exhibited the highest AUC (0.940, 95% CI: 0.900–0.980), which was significantly higher than that of LR (0.842, 95% CI: 0.783–0.901), SVM (0.834, 95% CI: 0.774–0.894) and ANN (0.890, 95% CI: 0.836–0.944), but did not differ significantly from that of RF (0.926, 95% CI: 0.879–0.974) and XGBoost (0.930, 95% CI: 0.878–0.982). In addition, the LightGBM has the optimal logarithmic loss function (0.218), accuracy (91.3%), specificity (94.1%), precision (0.695), and F1 score (0.725). Feature importance identification indicated that the contribution rate of meteorological conditions and air pollutants for the prediction was 32 and 43%, respectively.ConclusionThis study suggests that ensemble learning models, especially the LightGBM model, can be used to effectively predict the peak events of CVDs admissions, and therefore could be a very useful decision-making tool for medical resource management.

Spatio-Temporal Deep Residual Network with Hierarchical Attentions for Video Event Recognition

Event recognition in surveillance video has gained extensive attention from the computer vision community. This process still faces enormous challenges due to the tiny inter-class variations that are caused by various facets, such as severe occlusion, cluttered backgrounds, and so forth. To address these issues, we propose a spatio-temporal deep residual network with hierarchical attentions (STDRN-HA) for video event recognition. In the first attention layer, the ResNet fully connected feature guides the Faster R-CNN feature to generate object-based attention (O-attention) for target objects. In the second attention layer, the O-attention further guides the ResNet convolutional feature to yield the holistic attention (H-attention) in order to perceive more details of the occluded objects and the global background. In the third attention layer, the attention maps use the deep features to obtain the attention-enhanced features. Then, the attention-enhanced features are input into a deep residual recurrent network, which is used to mine more event clues from videos. Furthermore, an optimized loss function named softmax-RC is designed, which embeds the residual block regularization and center loss to solve the vanishing gradient in a deep network and enlarge the distance between inter-classes. We also build a temporal branch to exploit the long- and short-term motion information. The final results are obtained by fusing the outputs of the spatial and temporal streams. Experiments on the four realistic video datasets, CCV, VIRAT 1.0, VIRAT 2.0, and HMDB51, demonstrate that the proposed method has good performance and achieves state-of-the-art results.

ADPE: Adaptive Dynamic Projected Embedding on Heterogeneous Information Networks

Network embedding (NE) aims to represent network information appropriately by learning low-dimensional and dense vectors for the nodes and edges of information network. Actually, the real world is almost full of heterogeneous information networks, which stimulates the emergence of heterogeneous information networks (HINs) embedding models. However, parts of existing HIN embedding models like meta-path-based methods only capture limited and aggregated information of relations, whereas some models based on metric or distance learning are usually of high computational complexity and slow training speed. In this paper, we present a novel heterogeneous information network embedding model, which applies dynamic projection metrics and translation mechanisms to the complicated heterogeneous information networks including multiple nodes and different relations. In order to overcome the imbalance of the distribution of relations in HIN and optimize the training process, we introduce an adaptive loss function for model optimization. Further more, we propose a hybrid model with baseline method as the initialization of the model. Experiments have been implemented on some real-world HIN datasets. And empirical results show that our model significantly outperforms the state-of-the-art representation learning models.

Twin Least Squares Support Vector Regression of Heteroscedastic Gaussian Noise Model

The training algorithm of twin least squares support vector regression (TLSSVR) transforms unequal constraints into equal constraints in a pair of quadratic programming problems, it owns faster computational speed. The classical least squares support vector regression (LSSVR) assumpt that the noise is Gaussian with zero mean and the homoscedastic variance. However, it is found that the noise models in some practical applications satisfy Gaussian distribution with zero mean and heteroscedastic variance. In this paper, the LSSVR is combined with the twin hyperplanes, and then an optimal loss function for Gaussian noise with heteroscedasticity is introduced, which is called the twin least squares support vector regression with heteroscedastic Gaussian noise (TLSSVR-HGN). Like LSSVR, TLSSVR-HGN also lacks sparsity. To analyze the generalization ability of the proposed model, the sparse TLSSVR-HGN (STLSSVR-HGN) is proposed with a simple mechanism. The proposed model has been verified using the artificial data set, several benchmark data sets and actual wind speed data. The experimental results show that TLSSVR-HGN is a better technology than the other algorithms.

RolexBoost: A Rotation-Based Boosting Algorithm With Adaptive Loss Functions

We propose a new ensemble algorithm, called RolexBoost (Rotation-Flexible AdaBoost) that can not only secure diversity within an ensemble by rotating the feature axes in conjunction with performing the random subspace method for each bootstrap sample, but can also mitigate the impact of outlying data points by identifying the optimal loss function during the boosting iterations. The experimental results on 30 binary classification problems showed that RolexBoost yielded the best accuracy among the competing ensemble methods (AdaBoost, GentleBoost, Rotation Forest, Random Forest, Rotation Boost, FlexBoost). We statistically verified that the improvements made significant differences in the rankings. Additionally, we empirically verified that Rotation Boost is complemented by adaptive loss functions, demonstrating the superiority of RolexBoost.

ASSESSMENT OF EFFECTIVE BLOOD CONCENTRATION READINGS FROM CLINICAL DATA ON PATIENTS WITH HEART FAILURE DISEASES AFTER DIGOXIN INTAKE: A PROJECTION BASED ON THE INVERSE PROBLEM ALGORITHM

In this study, a projection of effective blood concentration (EBC) readings of digoxin is made using the inverse problem algorithm based on clinical data for patients with heart failure diseases. Seven factors, including body surface area (BSA), blood urine nitrogen (BUN), creatinine, sodium (Na), potassium (K), magnesium (Mg) ion readings, and mean arterial pressure (MAP) were compiled with nonlinear regression fit to develop a projection function having 29 terms obtained from an inverse problem algorithm via the default function run in STATISTICA. Accordingly, data collected from the clinical 168 heart failure patients were normalized to be included in same domain range ([Formula: see text]1 to +1), and then calculated by the specific algorithm to optimize the numerical solution to evaluate EBC readings of digoxin. The evaluated first-order regression fit owned an optimal loss function ([Formula: see text]) coupled with correlation coefficient [Formula: see text] = 0.892 and variance of 89.20%. Furthermore, 45 patients having similar clinical syndromes were also adopted to verify the projection and implied with high agreement. The BUN factor dominated the projection and defined as the most significant coefficient in the analysis, and K ion, MAP, BSA, and Mg ion factors exhibited minor contributions to the projection. The repeated trials to lower number of factors from seven to a smaller number (namely 6, 5, 4, 3, 2, and 1) for simplifying method but resulting with unaccepted outcomes, with high loss function values and low linearity. However, the algorithm held its accuracy to handle the verified data that were out of the original bounds. The proposed algorithm demonstrated a useful analysis to handle the drug administration in pharmaceutical field.

Embedding Learning with Triple Trustiness on Noisy Knowledge Graph

Embedding learning on knowledge graphs (KGs) aims to encode all entities and relationships into a continuous vector space, which provides an effective and flexible method to implement downstream knowledge-driven artificial intelligence (AI) and natural language processing (NLP) tasks. Since KG construction usually involves automatic mechanisms with less human supervision, it inevitably brings in plenty of noises to KGs. However, most conventional KG embedding approaches inappropriately assume that all facts in existing KGs are completely correct and ignore noise issues, which brings about potentially serious errors. To address this issue, in this paper we propose a novel approach to learn embeddings with triple trustiness on KGs, which takes possible noises into consideration. Specifically, we calculate the trustiness value of triples according to the rich and relatively reliable information from large amounts of entity type instances and entity descriptions in KGs. In addition, we present a cross-entropy based loss function for model optimization. In experiments, we evaluate our models on KG noise detection, KG completion and classification. Through extensive experiments on three datasets, we demonstrate that our proposed model can learn better embeddings than all baselines on noisy KGs.

Content-guided deep residual network for single image super-resolution

We propose a content-guided deep residual network for single image super-resolution. Firstly, the relationship between the sparsity of images and the difficulty of reconstruction through a convolution network is studied. On this basis, a guided residual block is built, which can compensate for part of the high-frequency information and improve the convergence of the network. On the other hand, local loss and curvature loss values are integrated into the loss function, enhancing the network's ability to perceive the advanced texture information of the image. Finally, the optimized loss function is used to the network composed of several guided residual blocks. Experimental results demonstrate that the network achieves superior results over some existing deep residual networks in PSNR and SSIM evaluation.

An overview of speaker recognition

Speaker recognition has been studied for many years and has been a hot topic. This paper presents an overview of speaker recognition methods, which include the classical and the state-of-art methods. According to the modular components of speaker recognition system, we fi rstly introduced the fundamentals of speaker recognition, which are mainly divided into two parts: feature extraction and speaker modeling. The most commonly speech features used in speaker recognition were elaborated fi rstly. In particular, the recent progress of deep neural network proposes a new approach of feature extraction and has become the technology trend. Secondly, the classical approaches of speaker recognition model were introduced, and elaborated the recent progress of deep learning speaker recognition. This paper especially provides an in-depth analysis on end-to-end model which consists of a training component to extract features, an enrollment component to training the speaker model, and an evaluation component with appropriate loss function for optimization. The fi nal part concludes the paper with discussion on future trends.

Joint Neural Collaborative Filtering for Recommender Systems

We propose a Joint Neural Collaborative Filtering (J-NCF) method for recommender systems. The J-NCF model applies a joint neural network that couples deep feature learning and deep interaction modeling with a rating matrix. Deep feature learning extracts feature representations of users and items with a deep learning architecture based on a user-item rating matrix. Deep interaction modeling captures non-linear user-item interactions with a deep neural network using the feature representations generated by the deep feature learning process as input. J-NCF enables the deep feature learning and deep interaction modeling processes to optimize each other through joint training, which leads to improved recommendation performance. In addition, we design a new loss function for optimization that takes both implicit and explicit feedback, point-wise and pair-wise loss into account. Experiments on several real-world datasets show significant improvements of J-NCF over state-of-the-art methods, with improvements of up to 8.24% on the MovieLens 100K dataset, 10.81% on the MovieLens 1M dataset, and 10.21% on the Amazon Movies dataset in terms of HR@10. NDCG@10 improvements are 12.42%, 14.24%, and 15.06%, respectively. We also conduct experiments to evaluate the scalability and sensitivity of J-NCF. Our experiments show that the J-NCF model has a competitive recommendation performance with inactive users and different degrees of data sparsity when compared to state-of-the-art baselines.

Deep Learning Network for Multiuser Detection in Satellite Mobile Communication System.

A multiuser detection (MUD) algorithm based on deep learning network is proposed for the satellite mobile communication system. Due to relative motion between the satellite and users, multiple access interference (MUI) introduced by multipath fading channel reduces system performance. The proposed MUD algorithm based on deep learning network firstly establishes the CINR optimal loss function according to the multiuser access mode and then obtains the best multiuser detection weight through the steepest gradient iteration. Multilayer nonlinear learning obtains interference cancellation sharing weights to achieve maximum signal-to-noise ratio through gradient iteration, which is superior than the traditional serial interference cancellation algorithm and parallel interference cancellation algorithm. Then, the weights with multiuser detection through multilayer network forward learning iteration are obtained with traditional multiuser detecting quality characteristics. The proposed multiuser access detection based on deep learning network algorithm improves the MUD accuracy and reduces the number of traditional multiusers. The performance of the satellite multifading uplink system shows that the proposed deep learning network can provide high precision and better iteration times.

A Multi-Task Learning Model for Better Representation of Clothing Images

Clothing images vary in style and everyone has a different understanding of style. Even with the current popular deep learning methods, it is difficult to accurately classify style labels. A style representation learning model based on the deep neural networks called StyleNet is proposed in this paper. We adopt a multi-task learning framework to build the model and make full use of various types of label information to represent the clothing images in a finer-grained manner. Due to the semantic abstraction of image labels in the current fashion field, using a simple migration learning method cannot fully meet the requirements of clothing image classification. An objective function optimization method is put forward by combining the distance confusion loss and the traditional cross entropy loss to improve the accuracy of StyleNet further. The experimental results show that by applying the multi-task representation learning framework, StyleNet can achieve a better classification accuracy, the optimized loss function can also bring performance improvement for deep learning models, and the classification effect of StyleNet becomes better as the size of the data set increases. In order to verify the robustness and effectiveness of the deep learning method in StyleNet, we also apply a Faster R-CNN module to pre-process the clothing images and use the result as the input of StyleNet. The classifier can only get a limited performance improvement, which is negligible compared with the methods proposed in this paper of increasing the depth of the neural network and optimizing the loss function.

Improved Cross-Label Suppression Dictionary Learning for Face Recognition

Cross-label suppression dictionary learning is an effective approach to preserve the label property for signal representation in face recognition. This paper presents a proposed improved dictionary learning algorithm, considering the tradeoffs between the operating time and the signal reconstruction residuals for the face recognition problem that combines an optimal loss function and the cross-label suppression supervised dictionary learning approach. Based on the relationship of the cost time of the dictionary learning algorithm and the residuals of the sparse representations, this paper attempts to select an optimal sparse coding dimension for the original signal to reduce the computational cost. Experiments on face recognition confirm that our proposed algorithm is able to achieve a desired classification results as well as obtain a considerably faster dictionary learning process.

边际赔偿函数和违约风险下的最优再保险

本文考虑了再保险人的违约风险，首先运用失真风险度量和失真保费原理建立了含有违约风险的总风险模型。其次通过边际索赔(MIF)函数与分出函数之间的关系建立了与总风险模型等价的MIF再保险优化模型。然后对MIF再保险优化模型的求解得到最优的边际索赔(MIF)函数，进而得到最优的分出函数。最后应用该方法研究了在VaR风险度量和Wang’s保费原理下的最优分出函数。 In this paper, we consider the default risk of reinsurer. Firstly, we use the distortion risk measure and distortion premium principle to establish the total risk model with default risk. Secondly, by the relationship between the Marginal Indemnification Function (MIF) and the ceded loss function, we build MIF reinsurance optimization model equivalent to total risk model. Then, the optimal MIF function is obtained by solving the MIF reinsurance optimization model. Furthermore, the optimal ceded loss function is obtained. Finally, we apply this method to study the optimal loss function by the VaR risk measure and Wang’s premium principle.

Locally Adaptive Translation for Knowledge Graph Embedding

Knowledge graph embedding aims to represent entities and relations in a large-scale knowledge graph as elements in a continuous vector space. Existing methods, e.g., TransE and TransH, learn embedding representation by defining a global margin-based loss function over the data. However, the optimal loss function is determined during experiments whose parameters are examined among a closed set of candidates. Moreover, embeddings over two knowledge graphs with different entities and relations share the same set of candidate loss functions, ignoring the locality of both graphs. This leads to the limited performance of embedding related applications. In this paper, we propose a locally adaptive translation method for knowledge graph embedding, called TransA, to find the optimal loss function by adaptively determining its margin over different knowledge graphs. Experiments on two benchmark data sets demonstrate the superiority of the proposed method, as compared to the-state-of-the-art ones.

Short-Term Wind Speed or Power Forecasting With Heteroscedastic Support Vector Regression

Wind speed or wind power forecasting plays an important role in large-scale wind power penetration due to their uncertainty. Support vector regression, widely used in wind speed or wind power forecasting, aims at discovering natural structures of wind variation hidden in historical data. Most current regression algorithms, including least squares support vector regression (SVR), assume that the noise of the data is Gaussian with zero mean and the same variance. However, it is discovered that the uncertainty of short-term wind speed satisfies Gaussian distribution with zero mean and heteroscedasticity in this work. This kind of task is called heteroscedastic regression. In order to deal with this problem, we derive an optimal loss function for heteroscedastic regression and develop a new framework of $\nu$ -SVR for learning tasks of Gaussian noise (GN) with heteroscedasticity. In addition, we introduce the stochastic gradient descent (SGD) method to solve the proposed model, which leads the models to be trained online. Finally, we reveal the uncertainty properties of wind speed with two real-world datasets and test the proposed algorithms on these data. The experimental results confirm the effectiveness of the proposed model.