Terms Of Cross Entropy Research Articles

Super-Resolving and Denoising 4D flow MRI of Neurofluids Using Physics-Guided Neural Networks.

To obtain high-resolution velocity fields of cerebrospinal fluid (CSF) and cerebral blood flow by applying a physics-guided neural network (div-mDCSRN-Flow) to 4D flow MRI. The div-mDCSRN-Flow network was developed to improve spatial resolution and denoise 4D flow MRI. The network was trained with patches of paired high-resolution and low-resolution synthetic 4D flow MRI data derived from computational fluid dynamic simulations of CSF flow within the cerebral ventricles of five healthy cases and five Alzheimer's disease cases. The loss function combined mean squared error with a binary cross-entropy term for segmentation and a divergence-based regularization term for the conservation of mass. Performance was assessed using synthetic 4D flow MRI in one healthy and one Alzheimer' disease cases, an in vitro study of healthy cerebral ventricles, and in vivo 4D flow imaging of CSF as well as flow in arterial and venous blood vessels. Comparison was performed to trilinear interpolation, divergence-free radial basis functions, divergence-free wavelets, 4DFlowNet, and our network without divergence constraints. The proposed network div-mDCSRN-Flow outperformed other methods in reconstructing high-resolution velocity fields from synthetic 4D flow MRI in healthy and AD cases. The div-mDCSRN-Flow network reduced error by 22.5% relative to linear interpolation for in vitro core voxels and by 49.5% in edge voxels. The results demonstrate generalizability of our 4D flow MRI super-resolution and denoising approach due to network training using flow patches and physics-based constraints. The mDCSRN-Flow network can facilitate MRI studies involving CSF flow measurements in cerebral ventricles and association of MRI-based flow metrics with cerebrovascular health.

Use of Anomaly Detection algorithms to unveil new physics in Vector Boson Scattering

A new methodology to improve the sensitivity to new physics contributions to the Standard Model processes at LHC is presented. A Variational AutoEncoder trained on Standard Model processes is used to identify Effective Field Theory contributions as anomalies. While the output of the model is supposed to be very similar to the inputs for Standard Model events, it is expected to deviate significantly for events generated through new physics processes. The reconstruction loss can then be used to select a signal enriched region which is by construction independent of the nature of the chosen new physics process. In order to improve further the discrimination power, an adversarial layer is introduced with a cross entropy term added to the loss function, optimizing at the same time the reconstruction of the input variables of the Standard Model and classification of new physics processes. This procedure ensures that the model is optimized for discrimination, with a small price in terms of model dependency to physics process. In this work I will discuss in detail the above-mentioned method using generator level Vector Boson Scattering events produced at LHC assuming an integrated luminosity of 350/fb.

A Compound Loss Function with Shape Aware Weight Map for Microscopy Cell Segmentation.

Microscopy cell segmentation is a crucial step in biological image analysis and a challenging task. In recent years, deep learning has been widely used to tackle this task, with promising results. A critical aspect of training complex neural networks for this purpose is the selection of the loss function, as it affects the learning process. In the field of cell segmentation, most of the recent research in improving the loss function focuses on addressing the problem of inter-class imbalance. Despite promising achievements, more work is needed, as the challenge of cell segmentation is not only the inter-class imbalance but also the intra-class imbalance (the cost imbalance between the false positives and false negatives of the inference model), the segmentation of cell minutiae, and the missing annotations. To deal with these challenges, in this paper, we propose a new compound loss function employing a shape aware weight map. The proposed loss function is inspired by Youden's J index to handle the problem of inter-class imbalance and uses a focal cross-entropy term to penalize the intra-class imbalance and weight easy/hard samples. The proposed shape aware weight map can handle the problem of missing annotations and facilitate valid segmentation of cell minutiae. Results of evaluations on all ten 2D+time datasets from the public cell tracking challenge demonstrate 1) the superiority of the proposed loss function with the shape aware weight map, and 2) that the performance of recent deep learning-based cell segmentation methods can be improved by using the proposed compound loss function.

Improving myocardial pathology segmentation with U-Net++ and EfficientSeg from multi-sequence cardiac magnetic resonance images

Background:Myocardial pathology segmentation plays an utmost role in the diagnosis and treatment of myocardial infarction (MI). However, manual segmentation is time-consuming and labor-intensive, and requires a lot of professional knowledge and clinical experience. Methods:In this work, we develop an automatic and accurate coarse-to-fine myocardial pathology segmentation framework based on the U-Net++ and EfficientSeg model. The U-Net++ network with deep supervision is first applied to delineate the cardiac structures from the multi-sequence cardiac magnetic resonance (CMR) images to generate a coarse segmentation map. Then the coarse segmentation map together with the three-sequence CMR data is sent to the EfficientSeg-B1 for fine segmentation, that is, further segmentation of myocardial scar and edema areas. In addition, we apply the Focal loss to replace the original cross-entropy term, for the purpose of encouraging the model to pay more attention to the pathological areas. Results:The proposed segmentation approach is tested on the public Myocardial Pathology Segmentation Challenge (MyoPS 2020) dataset. Experimental results demonstrate that our solution achieves an average Dice score of 0.7148 ± 0.2213 for scar, an average Dice score of 0.7439 ± 0.1011 for edema + scar, and the final average score of 0.7294 on the overall 20 testing sets, all of which have outperformed the first place method in the competition. Moreover, extensive ablation experiments are performed, which shows that the two-stage strategy with Focal loss greatly improves the segmentation quality of pathological areas. Conclusion:Given its effectiveness and superiority, our method can further facilitate myocardial pathology segmentation in medical practice.

Analysis of the density and distribution of entropy in biomedical infrared imaging for Diabetes Mellitus Type II

We report a methodology to analyze data extracted from infrared images. These pictures show the lower limbs of a cohort of individuals belonging to, (1) voluntary controls and (2) patients diagnosed with diabetes mellitus type II. The analysis is presented in terms of Cross Entropy and temperature distributions; both using the associated thermal histograms. The temperature analysis is placed in terms of comparing the extreme values of dS/dQ, for controls and patients. In this analysis for the frontal view, the values of specificity and sensitivity calculated were 77.77% and 91.66%, respectively. For the back view, the specificity and sensitivity obtained were 88.8% and 83.3%, respectively. Instead of that, the cross-entropy analysis is placed in the modality of self-referencing. In this part of the study we obtained the coefficient of asymmetry and thermal response (ATR). The values of specificity and sensitivity for the ATR quotient in both cases were 83.3%. The results of both studies have a significant correlation with glucose (p < 0.01) and HbA1c (p < 0.01). It means that both approaches have statistical correspondence. By means of the Mann-Whitney U test, for independent samples, we get that the characteristic parameters we analyze can be differentiated among the populations of interest with a significance of p < 0.05. This suggests that both studies show consistency with the clinical diagnosis; exhibiting clear differences between control and patient groups.

A derivational model of discontinuous parsing

The notion of latent-variable probabilistic context-free derivation of syntactic structures is enhanced to allow heads and unrestricted discontinuities. The chosen formalization covers both constituency parsing and dependency parsing. By the new framework, one obtains a probability distribution over the space of all discontinuous parses. This lends itself to intrinsic evaluation in terms of cross-entropy. The derivational model is accompanied by an equivalent automaton model, which can be used for deterministic parsing.

Learning and Evaluation Methodologies for Polyphonic Music Sequence Prediction With LSTMs

Music language models play an important role for various music signal and symbolic music processing tasks, such as music generation, symbolic music classification, or automatic music transcription (AMT). In this article, we investigate Long Short-Term Memory (LSTM) networks for polyphonic music prediction, in the form of binary piano rolls. A preliminary experiment, assessing the influence of the timestep of piano rolls on system performance, highlights the need for more musical evaluation metrics. We introduce a range of metrics, focusing on temporal and harmonic aspects. We propose to combine them into a parametrisable loss to train our network. We then conduct a range of experiments with this new loss, both for polyphonic music prediction (intrinsic evaluation) and using our predictive model as a language model for AMT (extrinsic evaluation). Intrinsic evaluation shows that tuning the behaviour of a model is possible by adjusting loss parameters, with consistent results across timesteps. Extrinsic evaluation shows consistent behaviour across timesteps in terms of precision and recall with respect to the loss parameters, leading to an improvement in AMT performance without changing the complexity of the model. In particular, we show that intrinsic performance (in terms of cross entropy) is not related to extrinsic performance, highlighting the importance of using custom training losses for each specific application. Our model also compares favourably with previously proposed MLMs.

Combo loss: Handling input and output imbalance in multi-organ segmentation

Simultaneous segmentation of multiple organs from different medical imaging modalities is a crucial task as it can be utilized for computer-aided diagnosis, computer-assisted surgery, and therapy planning. Thanks to the recent advances in deep learning, several deep neural networks for medical image segmentation have been introduced successfully for this purpose. In this paper, we focus on learning a deep multi-organ segmentation network that labels voxels. In particular, we examine the critical choice of a loss function in order to handle the notorious imbalance problem that plagues both the input and output of a learning model. The input imbalance refers to the class-imbalance in the input training samples (i.e., small foreground objects embedded in an abundance of background voxels, as well as organs of varying sizes). The output imbalance refers to the imbalance between the false positives and false negatives of the inference model. In order to tackle both types of imbalance during training and inference, we introduce a new curriculum learning based loss function. Specifically, we leverage Dice similarity coefficient to deter model parameters from being held at bad local minima and at the same time gradually learn better model parameters by penalizing for false positives/negatives using a cross entropy term. We evaluated the proposed loss function on three datasets: whole body positron emission tomography (PET) scans with 5 target organs, magnetic resonance imaging (MRI) prostate scans, and ultrasound echocardigraphy images with a single target organ i.e., left ventricular. We show that a simple network architecture with the proposed integrative loss function can outperform state-of-the-art methods and results of the competing methods can be improved when our proposed loss is used.

Instance Weighting Domain Adaptation Using Distance Kernel

Domain adaptation methods aims to improve the accuracy of the target predictive classifier while using the patterns from a related source domain that has large number of labeled data. In this paper, we introduce new kernel weight domain adaptation method based on smoothness assumption of classifier. We propose new simple and intuitive method that can improve the learning of target data by adding distance kernel based cross entropy term in loss function. Distance kernel refers to a matrix which denotes distance of each instances in source and target domain. We efficiently reduced the computational cost by using the stochastic gradient descent method. We evaluated the proposed method by using synthetic data and cross domain sentiment analysis tasks of Amazon reviews in four domains. Our empirical results showed improvements in all 12 domain adaptation experiments.

DeepCXray: Automatically Diagnosing Diseases on Chest X-Rays Using Deep Neural Networks

The automatic detection of diseases in images acquired through chest X-rays can be useful in clinical diagnosis because of a shortage of experienced doctors. Compared with natural images, those acquired through chest X-rays are obtained by using penetrating imaging technology, such that there are multiple levels of features in an image. It is thus difficult to extract the features of a disease for further diagnosis. In practice, healthy people are in a majority and the morbidities of different disease vary, because of which the obtained labels are imbalanced. The two main challenges of diagnosis though chest X-ray images are to extract discriminative features from X-ray images and handle the problem of imbalanced data distribution. In this paper, we propose a deep neural network called DeepCXray that simultaneously solves these two problems. An InceptionV3 model is trained to extract features from raw images, and a new objective function is designed to address the problem of imbalanced data distribution. The proposed objective function is a performance index based on cross entropy loss that automatically weights the ratio of positive to negative samples. In other words, the proposed loss function can automatically reduce the influence of an overwhelming number of negative samples by shrinking each cross entropy terms by a different extent. Extensive experiments highlight the promising performance of DeepCXray on the ChestXray14 dataset of the National Institutes of Health in terms of the area under the receiver operating characteristic curve.

A Chance-Constrained Programming Approach to Preoperative Planning of Robotic Cardiac Surgery Under Task-Level Uncertainty

In this paper, a novel formulation for robust surgical planning of robotics-assisted minimally invasive cardiac surgery based on patient-specific preoperative images is proposed. In this context, robustness is quantified in terms of the likelihood of intraoperative collisions and of joint limit violations. The proposed approach provides a more accurate and complete formulation than existing deterministic approaches in addressing uncertainty at the task level. Moreover, it is demonstrated that the dexterity of robotic arms can be quantified as a cross-entropy term. The resulting planning problem is rendered as a chance-constrained entropy maximization problem seeking a plan with the least susceptibility toward uncertainty at the task level, while maximizing the dexterity (cross-entropy term). By such treatment of uncertainty at the task level, spatial uncertainty pertaining to mismatches between the patient-specific anatomical model and that of the actual intraoperative situation is also indirectly addressed. As a solution method, the unscented transform is adopted to efficiently transform the resulting chance-constrained entropy maximization problem into a constrained nonlinear program without resorting to computationally expensive particle-based methods.

Direct estimation of class membership probabilities for multiclass classification using multiple scores

Accurate estimation of class membership probability is needed for many applications in data mining and decision-making, to which multiclass classification is often applied. Since existing methods for estimation of class membership probability are designed for binary classification, in which only a single score outputted from a classifier can be used, an approach for multiclass classification requires both a decomposition of a multiclass classifier into binary classifiers and a combination of estimates obtained from each binary classifier to a target estimate. We propose a simple and general method for directly estimating class membership probability for any class in multiclass classification without decomposition and combination, using multiple scores not only for a predicted class but also for other proper classes. To make it possible to use multiple scores, we propose to modify or extend representative existing methods. As a non-parametric method, which refers to the idea of a binning method as proposed by Zadrozny et al., we create an “accuracy table” by a different method. Moreover we smooth accuracies on the table with methods such as the moving average to yield reliable probabilities (accuracies). As a parametric method, we extend Platt’s method to apply a multiple logistic regression. On two different datasets (open-ended data from Japanese social surveys and the 20 Newsgroups) both with Support Vector Machines and naive Bayes classifiers, we empirically show that the use of multiple scores is effective in the estimation of class membership probabilities in multiclass classification in terms of cross entropy, the reliability diagram, the ROC curve and AUC (area under the ROC curve), and that the proposed smoothing method for the accuracy table works quite well. Finally, we show empirically that in terms of MSE (mean squared error), our best proposed method is superior to an expansion for multiclass classification of a PAV method proposed by Zadrozny et al., in both the 20 Newsgroups dataset and the Pendigits dataset, but is slightly worse than the state-of-the-art method, which is an expansion for multiclass classification of a combination of boosting and a PAV method, on the Pendigits dataset.

Minimum cross-entropy reconstruction of PET images using prior anatomical information

An algorithm is presented for the reconstruction of PET images using prior anatomical information derived from MR images of the same subject. The cross-entropy or Kullback - Leiber distance is a measure of dissimilarity between two images. We propose to reconstruct PET images by minimizing a weighted sum of two cross-entropy terms. The first is the cross-entropy between the measured emission data and the forward projection of the current estimate of the PET image. Minimizing this term alone is equivalent to the ML - EM reconstruction. The second term is the cross-entropy between the current estimate of the PET image and a prior image model which incorporates anatomical information derived from registered MR images. A weighting parameter determines the relative emphasis given to the emission data and the prior model in the reconstruction. Details of this algorithm are presented as well as test reconstructions for real and simulated data. The performance of the algorithm was evaluated with respect to errors in prior anatomical information. The algorithm provided significant improvement in the quality of reconstructed images as compared with the ML - EM reconstruction technique. The reconstructed images had higher resolution as compared with the images obtained from MAP-like reconstructions which do not utilize anatomical information. The algorithm displayed robustness with respect to errors in prior anatomical information.

Bayesian cross-entropy reconstruction of complex images

BaJkova's generalized maximum entropy method for reconstruction of complex signals is further generalized through the use of Kullback-Leibler cross entropy. This permits apriori information in the form of bias functions to be inserted into the algorithm, with resulting benefits to reconstruction quality. Also, the cross-entropy term is imbedded within an overall maximum aposteriori probability approach that includes a noise-rejection term. A further modification is transformation of the large two-dimensional problem arising from modest-sized two-dimensional images into a sequence of onedimensional problems. Finally the added operation of three-point median window filtration of each intermediary one-dimensional output is shown to suppress edge-top overshoots while augmenting edge gradients. Applications to simulated complex images are shown.