Modified Loss Function Research Articles

Improved CAPSNET model with modified loss function for medical image classification

Improved CAPSNET model with modified loss function for medical image classification

Single Reference Frequency Loss for Multifrequency Wavefield Representation Using Physics-Informed Neural Networks

Physics-informed neural networks (PINNs) can offer approximate multidimensional functional solutions to the Helmholtz equation that are flexible, require low memory, and have no limitations on the shape of the solution space. However, the neural network (NN) training can be costly and the cost dramatically increases as we train for multi-frequency wavefields by adding frequency as an additional input to the NN multi-dimensional function. In this case, the often large variation of the wavefield features (specifically wavelength) with frequency adds more complexity to the NN training. Thus, we propose a new loss function for the NN multidimensional input training that allows us to seamlessly include frequency as a dimension. We specifically utilize the linear relation between frequency and wavenumber (the wavefield space representation) to incorporate a reference frequency scaling to the loss function. As a result, the effective wavenumber of the wavefield solution as a function of frequency remains almost stationary, which reduces the learning burden on the NN function. We demonstrate the effectiveness of this modified loss function on a layered model.

Estimating Weibull Parameters Using Least Squares and Multilayer Perceptron vs. Bayes Estimation

The Weibull distribution is regarded as among the finest in the family of failure distributions. One of the most commonly used parameters of the Weibull distribution (WD) is the ordinary least squares (OLS) technique, which is useful in reliability and lifetime modeling. In this study, we propose an approach based on the ordinary least squares and the multilayer perceptron (MLP) neural network called the OLSMLP that is based on the resilience of the OLS method. The MLP solves the problem of heteroscedasticity that distorts the estimation of the parameters of the WD due to the presence of outliers, and eases the difficulty of determining weights in case of the weighted least square (WLS). Another method is proposed by incorporating a weight into the general entropy (GE) loss function to estimate the parameters of the WD to obtain a modified loss function (WGE). Furthermore, a Monte Carlo simulation is performed to examine the performance of the proposed OLSMLP method in comparison with approximate Bayesian estimation (BLWGE) by using a weighted GE loss function. The results of the simulation showed that the two proposed methods produced good estimates even for small sample sizes. In addition, the techniques proposed here are typically the preferred options when estimating parameters compared with other available methods, in terms of the mean squared error and requirements related to time.

Attention-Based Graph Convolutional Network for Zero-Shot Learning with Pre-Training

Zero-shot learning (ZSL) is a powerful and promising learning paradigm for classifying instances that have not been seen in training. Although graph convolutional networks (GCNs) have recently shown great potential for the ZSL tasks, these models cannot adjust the constant connection weights between the nodes in knowledge graph and the neighbor nodes contribute equally to classify the central node. In this study, we apply an attention mechanism to adjust the connection weights adaptively to learn more important information for classifying unseen target nodes. First, we propose an attention graph convolutional network for zero-shot learning (AGCNZ) by integrating the attention mechanism and GCN directly. Then, in order to prevent the dilution of knowledge from distant nodes, we apply the dense graph propagation (DGP) model for the ZSL tasks and propose an attention dense graph propagation model for zero-shot learning (ADGPZ). Finally, we propose a modified loss function with a relaxation factor to further improve the performance of the learned classifier. Experimental results under different pre-training settings verified the effectiveness of the proposed attention-based models for ZSL.

Rate-constrained learning-based image compression

Rate control is a desirable feature, sometimes a requirement, for several applications in still image coding. Usually, the objective is to achieve rate control for every input data with minimal impact on rate–distortion performance. However, this task can be quite challenging. Learning-based image compression is a new paradigm that needs to be competitive with conventional image coding techniques. A learning-based lossy codec may require several trained models for different quality requirements. Therefore, a coding tool providing the ability to achieve a specific rate can be a deterministic factor to apply such models in practical application scenarios. Hence, in this work, we present a non-constrained solution to solve the constrained problem of training a learning-based image codec for a specific bitrate. The proposed solution requires a modified loss function for autoencoder optimization. This modification allows controlling the deviation from the specified target rate. Experiments performed in Kodak and JPEG AI datasets show that autoencoders trained with the proposed loss function can achieve rate constrained encoding with negligible losses in terms of SSIM and MS-SSIM.

Wave-packet behaviors of the defocusing nonlinear Schrödinger equation based on the modified physics-informed neural networks.

Investigated in this paper is the defocusing nonlinear Schrödinger (NLS) equation, which is used for describing the wave-packet dynamics in certain weakly nonlinear media. With the physics-informed neural networks (PINNs), we modify the corresponding loss function in the existing literature and obtain two types of dark solitons, type-I and type-II solitons. It is demonstrated that the modified loss function presents higher-precision wave-packet behaviors based on fewer initial and boundary data. Taking type-I solitons into consideration, we find that when only a small fraction of initial and boundary data are given, the prediction accuracy of the wave packets will be increased one or two orders of magnitude at least if the modification term of the loss function is introduced. Furthermore, for the inverse problem, the modified loss function provides a better estimate of the nonlinear coefficient of the NLS equation based on fewer observed data of the wave packets. For type-II solitons, we compare the required data and predicted results of the PINNs with those of the conventional time-splitting finite difference (TSFD) method and reveal that achieving the same precision of the wave-packet behavior, the PINNs with the modified loss functions require only one tenth of the amount of the initial and boundary data of the TSFD method. Besides, both unmodified and modified loss functions are exploited for predicting the behaviors of Gaussian wave packets, and it is observed that the predicted result of the modified loss function agrees with the high-precision solution of the time-splitting Fourier pseudospectral method, whereas the unmodified loss function fails.

Deep Learning Phase Compression for MIMO CSI Feedback by Exploiting FDD Channel Reciprocity

Large scale MIMO FDD systems are often hampered by bandwidth required to feedback downlink CSI. Previous works have made notable progresses in efficient CSI encoding and recovery by taking advantage of FDD uplink/downlink reciprocity between their CSI magnitudes. Such framework separately encodes CSI phase and magnitude. To further enhance feedback efficiency, we propose a new deep learning architecture for phase encoding based on limited CSI feedback and magnitude-aided information. Our contribution features a framework with a modified loss function to enable end-to-end joint optimization of CSI magnitude and phase recovery. Our test results show superior performance in indoor/outdoor scenarios.

A Novel Intelligent System for Detection of Type 2 Diabetes with Modified Loss Function and Regularization

A Novel Intelligent System for Detection of Type 2 Diabetes with Modified Loss Function and Regularization

GA-CSPN: generative adversarial monocular depth estimation with second-order convolutional spatial propagation network

Monocular depth estimation, which provides a critical method for understanding 3D scene geometry, is an ill-posed problem. Recent research studies have achieved significant progress by reliable network architecture and optimized constraints, such as spatial propagation network and depth metrics. We propose an effective generative adversarial network for fast and accurate monocular depth estimation. Our approach demonstrates the feasibility of applying a dense-connected UNet for reducing information transmission loss and then fine-tuning the blur depth by the high-order convolutional spatial propagation network (CSPN) that used a modified loss function of discriminator. Furthermore, we modify the loss function of discriminator by adding the correlation loss that is used to measure the similarity of real and fake labels. Compared with the original CSPN, the high-order CSPN reduces the computation complexity and accelerates the convergence of the generator network by increasing the order of kernel, which emphasizes the weight of kernels in the update formula. With these modifications, our generative adversarial second-order convolutional spatial propagation network (GA-CSPN) achieves more accurate results against state-of-the-art methods in both indoor and outdoor scenes on Make3D, KITTI 2015, and NYUv2 datasets.

An improved generative adversarial network with modified loss function for crack detection in electromagnetic nondestructive testing

In this paper, an improved generative adversarial network (GAN) is proposed for the crack detection problem in electromagnetic nondestructive testing (NDT). To enhance the contrast ratio of the generated image, two additional regulation terms are introduced in the loss function of the underlying GAN. By applying an appropriate threshold to the segmentation of the generated image, the real crack areas and the fake crack areas (which are affected by the noises) are accurately distinguished. Experiments are carried out to show the superiority of the improved GAN over the original one on crack detection tasks, where a real-world NDT dataset is exploited that consists of magnetic optical images obtained using the electromagnetic NDT technique.

Fully Automated 3-D Ultrasound Segmentation of the Placenta, Amniotic Fluid, and Fetus for Early Pregnancy Assessment.

Volumetric placental measurement using 3-D ultrasound has proven clinical utility in predicting adverse pregnancy outcomes. However, this metric cannot currently be employed as part of a screening test due to a lack of robust and real-time segmentation tools. We present a multiclass (MC) convolutional neural network (CNN) developed to segment the placenta, amniotic fluid, and fetus. The ground-truth data set consisted of 2093 labeled placental volumes augmented by 300 volumes with placenta, amniotic fluid, and fetus annotated. A two-pathway, hybrid (HB) model using transfer learning, a modified loss function, and exponential average weighting was developed and demonstrated the best performance for placental segmentation (PS), achieving a Dice similarity coefficient (DSC) of 0.84- and 0.38-mm average Hausdorff distances (HDAV). The use of a dual-pathway architecture improved the PS by 0.03 DSC and reduced HDAV by 0.27 mm compared with a naïve MC model. The incorporation of exponential weighting produced a further small improvement in DSC by 0.01 and a reduction of HDAV by 0.44 mm. Per volume inference using the FCNN took 7-8 s. This method should enable clinically relevant morphometric measurements (such as volume and total surface area) to be automatically generated for the placenta, amniotic fluid, and fetus. The ready availability of such metrics makes a population-based screening test for adverse pregnancy outcomes possible.

Leveraging voxel-wise segmentation uncertainty to improve reliability in assessment of paediatric dysplasia of the hip.

Estimating uncertainty in predictions made by neural networks is critically important for increasing the trust medical experts have in automatic data analysis results. In segmentation tasks, quantifying levels of confidence can provide meaningful additional information to aid clinical decision making. In recent work, we proposed an interpretable uncertainty measure to aid clinicians in assessing the reliability of developmental dysplasia of the hip metrics measured from 3D ultrasound screening scans, as well as that of the US scan itself. In this work, we propose a technique to quantify confidence in the associated segmentation process that incorporates voxel-wise uncertainty into the binary loss function used in the training regime, which encourages the network to concentrate its training effort on its least certain predictions. We propose using a Bayesian-based technique to quantify 3D segmentation uncertainty by modifying the loss function within an encoder-decoder type voxel labeling deep network. By appending a voxel-wise uncertainty measure, our modified loss helps the network improve prediction uncertainty for voxels that are harder to train. We validate our approach by training a Bayesian 3D U-Net with the proposed modified loss function on a dataset comprising 92 clinical 3D US neonate scans and test on a separate hold-out dataset of 24 patients. Quantitatively, we show that the Dice score of ilium and acetabulum segmentation improves by 5% when trained with our proposed voxel-wise uncertainty loss compared to training with standard cross-entropy loss. Qualitatively, we further demonstrate how our modified loss function results in meaningful reduction of voxel-wise segmentation uncertainty estimates, with the network making more confident accurate predictions. We proposed a Bayesian technique to encode voxel-wise segmentation uncertainty information into deep neural network optimization, and demonstrated how it can be leveraged into meaningful confidence measures to improve the model's predictive performance.

A novel deep learning neural network for fast-food image classification and prediction using modified loss function

Accurate food image classification is often critical to accurately monitor the dietary assessment to reduce the risk of different heart-related diseases, obesity, diabetes, and other related health conditions. The accuracy and efficiency of image classification results when using traditional deep learning methods were less than optimal. This research aimed at enhancing the classification and prediction accuracy of food images and reducing the processing time by using the Deep Convolutional Neural Network (DCN) algorithm. The solution starts by using the Modified Loss function, the images are fed into the DCN for features extraction through alternating between convolutional layers and pooling layers, then this is followed by a fully connected layer. Finally, the Softmax function is used to classify the images. The result was compared during the classification phase in the DCN. The proposed solution enhanced the accuracy of the classification by using the regularized loss function and lowered the processing time by decreasing the weights of the neurons in the neural network. Probability score is used as the evaluation metric for the accuracy, and total execution time is used as the evaluation metric for the speed of the algorithm. The combination of deep neural network with regularized cross entropy cost function has improved the fast-food images classification by ahcieving better processing time by 40 ~ 50s and accuracy by 5% in average.

Automatic Defect Detection and Depth Visualization in Mild Steel Sample Using Quadratic Frequency Modulated Thermal Wave Imaging

Deeper defect detection and depth resolution capabilities of quadratic frequency-modulated optical stimulus became a viable approach for material inspection in active infrared non-destructive testing modality. But the limitations of complex and non-linear analytical models associated with processing techniques propel towards automated defect assessment techniques in infrared thermography. This paper introduces a deep neural network-based automatic defect detection and depth visualization technique in quadratic frequency modulated thermal wave imaging. The neural network classifier uses the modified loss function of a one-class support vector machine to classify defects. The regression network estimates the depth of classified defects. A mild steel specimen with artificial delaminations is numerically modeled and excited by a quadratic frequency-modulated heat flux. The proposed network classification and regression performances are qualitatively assessed using testing time, accuracy, and mean squared error as a figure of merits.

Identification of Melanoma From Hyperspectral Pathology Image Using 3D Convolutional Networks.

Skin biopsy histopathological analysis is one of the primary methods used for pathologists to assess the presence and deterioration of melanoma in clinical. A comprehensive and reliable pathological analysis is the result of correctly segmented melanoma and its interaction with benign tissues, and therefore providing accurate therapy. In this study, we applied the deep convolution network on the hyperspectral pathology images to perform the segmentation of melanoma. To make the best use of spectral properties of three dimensional hyperspectral data, we proposed a 3D fully convolutional network named Hyper-net to segment melanoma from hyperspectral pathology images. In order to enhance the sensitivity of the model, we made a specific modification to the loss function with caution of false negative in diagnosis. The performance of Hyper-net surpassed the 2D model with the accuracy over 92%. The false negative rate decreased by nearly 66% using Hyper-net with the modified loss function. These findings demonstrated the ability of the Hyper-net for assisting pathologists in diagnosis of melanoma based on hyperspectral pathology images.

An improved tandem neural network for the inverse design of nanophotonics devices

The tandem neural network with a modified loss function is used to inversely design the nanophotonic device from a desired spectrum, to obtain the corresponding structural parameters. High prediction accuracy is demonstrated on the inverse design of grating coupler examples, etc. Different activation functions and loss functions used in the tandem network are compared to find the optimum scheme. The method can be readily applied to many other cases in order to obtain the design parameters instantaneously from the device response curves.

Deep Learning Forecaster-Based Controller for SVC: Wind Farm Flicker Mitigation

One of the major challenges in grid-connected wind farms is the flicker emission caused by the reactive power extremely short time variations. A general used solution is installing a static volt-ampere reactive (VAR) compensator (SVC) as the most cost-saving device, which still holds the risk of incomplete reactive power compensation subject to the operating delay time. To this end, an accurate extreme short-term reactive power forecasting structure is essential. Although there are many studies related to the forecasting of wind power in long, medium, and short terms, however, the extremely short-term reactive power prediction of the wind farms is only addressed in few studies. The main aim in this article is to develop a method based on the convolutional neural network (CNN) to directly learn nonstationary and complex features from the raw wind farm reactive power time series and to contribute a predictive controller to mitigate the voltage flicker through an SVC connected to the wind farm. However, high sensitivity of typical mean-squared-based loss functions might lead to considerable errors in the forecasting. To resolve this issue, a modified loss function is proposed to enhance the performance and, consequently, an improved CNN is presented. The evaluation of the proposed method is based on the actual recorded data of a wind farm in Manjil, Iran. The data are directly used in the modeling of the wind farm. A current source, which is based on the actual records its amplitude and phase angle change every 0.01 s, is utilized to model the wind farm. The numerical results in terms of flicker sensation and short-term flicker perceptibility (Pst) measurement are used to verify the performance of the proposed method through comparison with the wind farm performance without SVC and SVC with a common control system.

Analyzing and Accelerating the Bottlenecks of Training Deep SNNs With Backpropagation.

Spiking neural networks (SNNs) with the event-driven manner of transmitting spikes consume ultra-low power on neuromorphic chips. However, training deep SNNs is still challenging compared to convolutional neural networks (CNNs). The SNN training algorithms have not achieved the same performance as CNNs. In this letter, we aim to understand the intrinsic limitations of SNN training to design better algorithms. First, the pros and cons of typical SNN training algorithms are analyzed. Then it is found that the spatiotemporal backpropagation algorithm (STBP) has potential in training deep SNNs due to its simplicity and fast convergence. Later, the main bottlenecks of the STBP algorithm are analyzed, and three conditions for training deep SNNs with the STBP algorithm are derived. By analyzing the connection between CNNs and SNNs, we propose a weight initialization algorithm to satisfy the three conditions. Moreover, we propose an error minimization method and a modified loss function to further improve the training performance. Experimental results show that the proposed method achieves 91.53% accuracy on the CIFAR10 data set with 1% accuracy increase over the STBP algorithm and decreases the training epochs on the MNIST data set to 15 epochs (over 13 times speed-up compared to the STBP algorithm). The proposed method also decreases classification latency by over 25 times compared to the CNN-SNN conversion algorithms. In addition, the proposed method works robustly for very deep SNNs, while the STBP algorithm fails in a 19-layer SNN.

A novel enhanced region proposal network and modified loss function: threat object detection in secure screening using deep learning

Detection of threat objects concealed in passenger clothing and baggage poses a challenge to aviation security. At present, the detection technology is capable of detecting the presence of threats from the scanned images yet requires the involvement of human in determining what type of threat and where it is located. Deep learning-based object detection technique has not been successfully implemented to detect threats in the security screening processes. This research aims to improve the accuracy and the processing time of threat detection in security screening. Enhanced faster region-based convolutional neural network (faster R-CNN) with improved region proposals is used for better threat localization. The proposed system consists of an improved region proposal network that outputs object’s region proposals with an object score to the detector module to accurately locate the threat in the human body. Furthermore, this system uses a modified loss function that strengthens the classification loss. Results obtained by the proposed model show a 15% improvement in object localization. Therefore, the enhanced faster R-CNN achieves an overall detection accuracy of 0.27 in terms of average precision and reduces the processing time by 0.19 s. The results obtained by the enhanced faster R-CNN for detection accuracy are superior to the state-of-the-art system. Also, this model focuses on localizing the threat and identifying its type, which makes the model suitable for threat detection security screening. Besides, the research also addresses the time consumption issue in detecting the threat object.

A Novel Solution of Using Deep Learning for White Blood Cells Classification: Enhanced Loss Function with Regularization and Weighted Loss (ELFRWL)

Deep learning has been successfully applied in classification of white blood cells (WBCs), however, accuracy and processing time are found to be less than optimal hindering it from getting its full potential. This is due to imbalanced dataset, intra-class compactness, inter-class separability and overfitting problems. The main research idea is to enhance the classification and prediction accuracy of blood images while lowering processing time through the use of deep convolutional neural network (DCNN) architecture by using the modified loss function. The proposed system consists of a deep neural convolution network (DCNN) that will improve the classification accuracy by using modified loss function along with regularization. Firstly, images are pre-processed and fed through DCNN that contains different layers with different activation function for the feature extraction and classification. Along with modified loss function with regularization, weight function aids in the classification of WBCs by considering weights of samples belonging to each class for compensating the error arising due to imbalanced dataset. The processing time will be counted by each image to check the time enhancement. The classification accuracy and processing time are achieved using the dataset-master. Our proposed solution obtains better classification performance in the given dataset comparing with other previous methods. The proposed system enhanced the classification accuracy of 98.92% from 96.1% and a decrease in processing time from 0.354 to 0.216 s. Less time will be required by our proposed solution for achieving the model convergence with 9 epochs against the current convergence time of 13.5 epochs on average, epoch is the formation white blood cells (WBCs) and the development of granular cells. The proposed solution modified loss function to solve the adverse effect caused due to imbalance dataset by considering weight and use regularization technique for overfitting problem.