Enhanced Loss Function Research Articles

Automatic screening of retinal lesions for detecting diabetic retinopathy using adaptive multiscale MobileNet with abnormality segmentation from public dataset

ABSTRACT Owing to the epidemic growth of diabetes, ophthalmologists need to examine the huge fundus images for diagnosing the disease of Diabetic Retinopathy (DR). Without proper knowledge, people are too lethargic to detect the DR. Therefore, the early diagnosis system is requisite for treating ailments in the medical industry. Therefore, a novel deep model-based DR detection structure is recommended to fix the aforementioned difficulties. The developed deep model-based diabetic retinopathy detection process is performed adaptively. The DR detection process is imitated by garnering the images from benchmark sources. The gathered images are further preceded by the abnormality segmentation phase. Here, the Residual TransUNet with Enhanced loss function is used to employ the abnormality segmentation, and the loss function in this structure may be helpful to lessen the error in the segmentation procedure. Further, the segmented images are passed to the final phase of retinopathy detection. At this phase, the detection is carried out through the Adaptive Multiscale MobileNet. The variables in the AMMNet are optimized by the Adaptive Puzzle Optimization to obtain better detection performance. Finally, the effectiveness of the offered approach is confirmed by the experimentation procedure over various performance indices.

B-TBM: A Novel Deep Learning Model with Enhanced Loss Function for HAZOP Risk Classification Using Natural Language Statistical Laws

HAZOP is a paradigm of industrial safety, and the introduction of deep learning-based HAZOP text categorization marks the arrival of an intelligent era of safety analysis. However, existing risk analysis methods have limitations in processing complex texts and extracting deep risk features. To solve this problem, this paper proposes a novel HAZOP risk event classification model based on BERT, BiLSTM, and TextCNN. The complexity of HAZOP text is revealed by introducing statistical laws of natural language, such as Zipf’s law and Heaps’ law, and the outputs of different levels of BERT are further combined linearly to collaborate with BiLSTM and TextCNN to capture long-term dependency and local contextual information for a more accurate classification task. Meanwhile, an improved loss function is proposed to effectively solve the deficiencies of the traditional cross-entropy loss function in the mislabeling process and improve the generalization ability of the model. It is experimentally demonstrated that the accuracy of the model is improved by 3% to 4% compared to the traditional BERT model in the task of severity and possibility classification of HAZOP reports. This study not only improves the accuracy and efficiency of HAZOP risk analysis, but also provides new ideas and methods for the application of natural language processing in industrial safety.

Quantum deep generative prior with programmable quantum circuits

Exploiting the utility of near-term quantum devices is a long-standing challenge whereas hybrid quantum machine learning emerges as a promising candidate. Here we propose a quantum-enhanced deep generative algorithm based on programmable quantum circuit-induced quantum latent codes. To validate the effectiveness of the algorithm, we conduct optical ghost imaging experiments, collecting dataset under varying physical sampling rates. Leveraging a physically enhanced loss function and a pretrained neural network, the quantum algorithm exhibits superior reconstruction performance compared to conventional algorithms. We observe the learnable quantum latent space enhances the out-of-distribution generalization capability of the pretrained model. In the context of computer vision problems, numerical results indicate that quantum latent space can increase the generation diversity of the pretrained model. Furthermore, the algorithm outperforms classical counterparts, particularly in image inpainting and colorization, by a significant margin. Our study demonstrates the utility of hybrid quantum-classical algorithms in enhancing generalization capability, highlighting the potential of near-term quantum devices in large-scale generative artificial intelligence.

A Dual-Branch Self-Boosting Network Based on Noise2Noise for Unsupervised Image Denoising

While unsupervised denoising models have shown progress in recent years, their noise reduction capabilities still lag behind those of supervised denoising models. This limitation can be attributed to the lack of effective constraints during training, which only utilizes noisy images and hinders further performance improvements In this work, we propose a novel dual-branch self-boosting network called DBSNet, which offers a straightforward and effective approach to image denoising. By leveraging task-dependent features, we exploit the intrinsic relationships between the two branches to enhance the effectiveness of our proposed model. Initially, we extend the classic Noise2Noise (N2N) architecture by adding a new branch for noise component prediction to the existing single-branch network designed for content prediction. This expansion creates a dual-branch structure, enabling us to simultaneously decompose a given noisy image into its content (clean) and noise components. This enhancement allows us to establish stronger constraint conditions and construct more powerful loss functions to guide the training process. Furthermore, we replace the UNet structure in the N2N network with the proven DnCNN (Denoising Convolutional Neural Network) sequential network architecture, which enhances the nonlinear mapping capabilities of the DBSNet. This modification enables our dual-branch network to effectively map a noisy image to its content (clean) and noise components simultaneously. To further improve the stability and effectiveness of training, and consequently enhance the denoising performance, we introduce a feedback mechanism where the network’s outputs, i.e., content and noise components, are fed back into the dual-branch network. This results in an enhanced loss function that ensures our model possesses excellent decomposition ability and further boosts the denoising performance. Extensive experiments conducted on both synthetic and real-world images demonstrate that the proposed DBSNet outperforms the unsupervised N2N denoising model as well as mainstream supervised models trained with supervised methods. Moreover, the evaluation results on real-world noisy images highlight the desirable generalization ability of DBSNet for practical denoising applications.

SM-CycleGAN: crop image data enhancement method based on self-attention mechanism CycleGAN

Crop disease detection and crop baking stage judgement require large image data to improve accuracy. However, the existing crop disease image datasets have high asymmetry, and the poor baking environment leads to image acquisition difficulties and colour distortion. Therefore, we explore the potential of the self-attention mechanism on crop image datasets and propose an innovative crop image data-enhancement method for recurrent generative adversarial networks (GANs) fused with the self-attention mechanism to significantly enhance the perception and information capture capabilities of recurrent GANs. By introducing the self-attention mechanism module, the cycle-consistent GAN (CycleGAN) is more adept at capturing the internal correlations and dependencies of image data, thus more effectively capturing the critical information among image data. Furthermore, we propose a new enhanced loss function for crop image data to optimise the model performance and meet specific task requirements. We further investigate crop image data enhancement in different contexts to validate the performance and stability of the model. The experimental results show that, the peak signal-to-noise ratio of the SM-CycleGAN for tobacco images and tea leaf disease images are improved by 2.13% and 3.55%, and the structural similarity index measure is improved by 1.16% and 2.48% compared to CycleGAN, respectively.

A complete artificial intelligence pipeline for radio frequency energy prediction in cellular bands for energy harvesting systems

Radio Frequency (RF) energy harvesting has been used to power wireless and low-powered devices. However, RF energy harvesting has limitations in terms of the amount of power that can be collected based on signal availability. Hence, energy prediction is essential to improve energy harvesting circuits' performance. Previous research has mainly focused on improving power harvesting policies or theoretically estimating the harvested energy. Very few works have considered the prediction of the RF signal as time series data using real RF measurements. Moreover, challenges such as the power consumed by the circuit's harvesting decisions and the impact of outliers on the model performance haven't been addressed yet. This paper presents a complete pipeline for developing the best predictive model for RF energy in cellular frequency bands. Real-time measurements are taken in different frequency bands using software-defined radio technology. We use four artificial intelligence techniques to model the RF energy signal. Additionally, we propose an optimized model with an enhanced loss function, which makes the model more resilient to anomalies, saving computational power and time consumed in cleaning the data. The four algorithms are investigated, and their prediction accuracies are compared. The average power of a period of 5 min is accurately forecasted. Numerical results in the 1960 MHz band show that long short-term memory has the best performance, followed by the DeepAR algorithm with prediction accuracies of 95.76% and 95.02%, respectively. Moreover, the proposed optimized model showed a 32.2% lower prediction error than the traditional models.

Improving Recognition of Defective Epoxy Images in Integrated Circuit Manufacturing by Data Augmentation

This paper discusses the problem of recognizing defective epoxy drop images for the purpose of performing vision-based die attachment inspection in integrated circuit (IC) manufacturing based on deep neural networks. Two supervised and two unsupervised recognition models are considered. The supervised models examined are an autoencoder (AE) network together with a multi-layer perceptron network (MLP) and a VGG16 network, while the unsupervised models examined are an autoencoder (AE) network together with k-means clustering and a VGG16 network together with k-means clustering. Since in practice very few defective epoxy drop images are available on an actual IC production line, the emphasis in this paper is placed on the impact of data augmentation on the recognition outcome. The data augmentation is achieved by generating synthesized defective epoxy drop images via our previously developed enhanced loss function CycleGAN generative network. The experimental results indicate that when using data augmentation, the supervised and unsupervised models of VGG16 generate perfect or near perfect accuracies for recognition of defective epoxy drop images for the dataset examined. More specifically, for the supervised models of AE+MLP and VGG16, the recognition accuracy is improved by 47% and 1%, respectively, and for the unsupervised models of AE+Kmeans and VGG+Kmeans, the recognition accuracy is improved by 37% and 15%, respectively, due to the data augmentation.

An Enhanced Loss Function for Semantic Road Segmentation in Remote Sensing Images

An Enhanced Loss Function for Semantic Road Segmentation in Remote Sensing Images

DENSE SWIN-UNET: DENSE SWIN TRANSFORMERS FOR SEMANTIC SEGMENTATION OF PNEUMOTHORAX IN CT IMAGES

Pneumothorax is a common yet potentially serious lung disease, which makes prompt diagnosis and treatment critical in clinical practice. Deep learning methods have proven effective in detecting pneumothorax lesions in medical images and providing quantitative analysis. However, due to the irregular shapes and uncertain positions of pneumothorax lesions, current segmentation methods must be further improved to increase accuracy. This study aimed to propose a Dense Swin-Unet algorithm that integrated the Dense Swin Transformer Block with the Swin-Unet model. The Dense Swin-Unet algorithm employed a sliding window self-attentiveness mechanism on different scales to enhance multiscale long-range dependencies. We designed an enhanced loss function that accelerated the convergence speed to address the issue of class imbalance. Given the limited availability of data in pneumothorax image processing, we created a new dataset and evaluated the efficacy of our model on this dataset. The results demonstrated that our lesion segmentation algorithm attained a Dice coefficient of 88.8%, representing a 1.5% improvement compared with previous deep learning algorithms. Notably, our algorithm achieved a significant enhancement in segmenting small microlesions.

Generating Defective Epoxy Drop Images for Die Attachment in Integrated Circuit Manufacturing via Enhanced Loss Function CycleGAN

In integrated circuit manufacturing, defects in epoxy drops for die attachments are required to be identified during production. Modern identification techniques based on vision-based deep neural networks require the availability of a very large number of defect and non-defect epoxy drop images. In practice, however, very few defective epoxy drop images are available. This paper presents a generative adversarial network solution to generate synthesized defective epoxy drop images as a data augmentation approach so that vision-based deep neural networks can be trained or tested using such images. More specifically, the so-called CycleGAN variation of the generative adversarial network is used by enhancing its cycle consistency loss function with two other loss functions consisting of learned perceptual image patch similarity (LPIPS) and a structural similarity index metric (SSIM). The results obtained indicate that when using the enhanced loss function, the quality of synthesized defective epoxy drop images are improved by 59%, 12%, and 131% for the metrics of the peak signal-to-noise ratio (PSNR), universal image quality index (UQI), and visual information fidelity (VIF), respectively, compared to the CycleGAN standard loss function. A typical image classifier is used to show the improvement in the identification outcome when using the synthesized images generated by the developed data augmentation approach.

An enhanced loss function in deep learning model to predict PM2.5 in India

Fine particulate matter (PM2.5) is one of the major air pollutants and is an important parameter for measuring air quality levels. High concentrations of PM2.5 show its impact on human health, the environment, and climate change. An accurate prediction of fine particulate matter (PM2.5) is significant to air pollution detection, environmental management, human health, and social development. The primary approach is to boost the forecast performance by reducing the error in the deep learning model. So, there is a need to propose an enhanced loss function (ELF) to decrease the error and improve the accurate prediction of daily PM2.5 concentrations. This paper proposes the ELF in CTLSTM (Chi-Square test Long Short Term Memory) to improve the PM2.5 forecast. The ELF in the CTLSTM model gives more accurate results than the standard forecast models and other state-of-the-art deep learning techniques. The proposed ELFCTLSTM reduces the prediction error of by a maximum of 10 to 25 percent than the state-of-the-art deep learning models.

ILF-LSTM: enhanced loss function in LSTM to predict the sea surface temperature

ILF-LSTM: enhanced loss function in LSTM to predict the sea surface temperature

Improved protein contact prediction using dimensional hybrid residual networks and singularity enhanced loss function.

Deep residual learning has shown great success in protein contact prediction. In this study, a new deep residual learning-based protein contact prediction model was developed. Comparing with previous models, a new type of residual block hybridizing 1D and 2D convolutions was designed to increase the effective receptive field of the residual network, and a new loss function emphasizing the easily misclassified residue pairs was proposed to enhance the model training. The developed protein contact prediction model referred to as DRN-1D2D was first evaluated on 105 CASP11 targets, 76 CAMEO hard targets and 398 membrane proteins together with two in house-developed reference models based on either the standard 2D residual block or the traditional BCE loss function, from which we confirmed that both the dimensional hybrid residual block and the singularity enhanced loss function can be employed to improve the model performance for protein contact prediction. DRN-1D2D was further evaluated on 39 CASP13 and CASP14 free modeling targets together with the two reference models and six state-of-the-art protein contact prediction models including DeepCov, DeepCon, DeepConPred2, SPOT-Contact, RaptorX-Contact and TripleRes. The result shows that DRN-1D2D consistently achieved the best performance among all these models.

An enhanced loss function simplifies the deep learning model for characterizing the 3D organoid models.

Organization of the organoid models, imaged in 3D with a confocal microscope, is an essential morphometric index to assess responses to stress or therapeutic targets. In fact, differentiating malignant and normal cells is often difficult in monolayer cultures. But in 3D culture, colony organization can provide a clear set of indices for differentiating malignant and normal cells. The limiting factors are delineating each cell in a 3D colony in the presence of perceptual boundaries between adjacent cells and heterogeneity associated with cells being at different cell cycles. In a previous paper, we defined a potential field for delineating adjacent nuclei, with perceptual boundaries, in 2D histology images by coupling three deep networks. This concept is now extended to 3D and simplified by an enhanced cost function that replaces three deep networks with one. Validation includes four cell lines with diverse mutations, and a comparative analysis with the UNet models of microscopy indicates an improved performance with the F1-score of 0.83. All software and annotated images are available through GitHub and Bioinformatics online. The software includes the proposed method, UNet for microscopy that was extended to 3D and report generation for profiling colony organization. Supplementary data are available at Bioinformatics online.

A novel solution of enhanced loss function using deep learning in sleep stage classification: predict and diagnose patients with sleep disorders

Sleep stage classification is important to accurately predict and diagnose patients with sleep disorders. Though various deep learning approaches have been implemented to classify sleep classes, these consist limitations that impact the accuracy and processing time of the classification model. The aim of this research is to enhance the accuracy and minimize the training time of the deep learning classification model. The proposed system consists of One Dimensional Convolutional Neural Network (CNN) with enhanced loss function to improve the accuracy of scoring of five different sleep classes. Preprocessing, Feature Extraction and Classification are the main components of the proposed system. Initially, EEG signals are fed to an adaptive filter for preprocessing, in order to remove any noise in signal. Thereafter, feature is extracted through multiple convolutional and pooling layers, and finally the classification is done by fully connected layer using softmax activation with enhanced loss function. The proposed solution is tested on data samples from multiple datasets with five classes of Sleep classification. Based on the obtained results, the proposed solution has found to achieve an accuracy of 96.26% which is almost 4.2% higher than the state-of-the-art solution which is 92.76%. Furthermore, the processing time has been reduced by 11 milliseconds against the state-of-the-art solution. The proposed system focused on classifying sleep stages in five classes using EEG signals with deep learning approach. It enhances the loss function in order to minimize errors in the prediction of sleep classes and improves the accuracy of the model. Furthermore, the training speed of the model has also been reduced by applying batch normalization techniques inside the model. In the future, larger datasets of different sleep disorder patients with varying features can be used for training and implementing the proposed solution. The datasets can also be pre-processed using additional techniques to refine the data before feeding to the neural network model.

Deep learning for liver tumour classification: enhanced loss function

Background and Aim: deep learning has not been successfully implemented in liver tumour feature extraction and classification using computer-aided diagnosis. This study aims to enhance classification accuracy and improves the processing time to better differentiate tumour types. Methodology: This study proposed a hybrid model, which combines the regularization function with the current loss function for the support vector machine (SVM) classifier. Regularization function is used for prioritizing image classes before feeding it to the linear mapping. The proposed model consists of the region growing algorithm to get the region-of-interest (ROI), and Weiner filtering algorithm for image enhancement and noise removal. The gray level co-occurrence matrix (GLCM) was performed to extract the feature from the image. The extracted feature then fed to SVM classifier using selected feature vectors to classify the affected region and neglecting the unwanted areas. Results: classification accuracy was calculated using probability score, and the processing time was calculated based on the total execution time. The proposed system was able to achieve an average classification accuracy of 98.9%, which is about 2–3% higher than the current system. The results showed that 12 ms reduced the processing time on average. Conclusion: The proposed system focused on improving feature extraction and classification for different types of tumours from the MRI images. The study solved the problem in linear mapping of support vector machine and enhanced the classification accuracy and the processing time of early diagnosis of three different types of tumours in liver MRI images.

A novel deep learning system for facial feature extraction by fusing CNN and MB-LBP and using enhanced loss function

Extensive use of surveillance cameras for human tracking and observation have been fostering the research on face recognition technique for individual identification in an unconstrained environment. However, face recognition is a challenging task in an unconstrained environment, where the captured images are affected by illumination effect, varying poses, noise and occlusion. The main objective of this research is to improve the accuracy and processing time in extracting facial features by using the fusion of deep learning and handcrafted architecture for recognizing individuals in unconstrained conditions, thereby providing accurate information about the individuals to security systems. The proposed system consists of Multi-Block Local Binary Pattern (MB-LBP) modules for extracting the handcrafted features and Convolutional Neural Network (CNN) for extracting the high-level distinctive features. The features from both modules are fused and passed through fully connected layer with Softmax classifier to identify individuals. The results show that the enhanced algorithm based on Softmax loss function aided classifier with regularization improves the accuracy and processing time for face recognition. The proposed model improves accuracy by 94.37% against 90.01% for the state-of-the-art solution. In addition to that, it improves the processing time of 307 ms against 357 ms. The proposed system focuses on fusing hand-crafted and deep learned features to extract face features accurately and thus improving the accuracy and overall performance of the proposed system in an unconstrained environment.

A Novel Solution of Using Deep Learning for White Blood Cells Classification: Enhanced Loss Function with Regularization and Weighted Loss (ELFRWL)

Deep learning has been successfully applied in classification of white blood cells (WBCs), however, accuracy and processing time are found to be less than optimal hindering it from getting its full potential. This is due to imbalanced dataset, intra-class compactness, inter-class separability and overfitting problems. The main research idea is to enhance the classification and prediction accuracy of blood images while lowering processing time through the use of deep convolutional neural network (DCNN) architecture by using the modified loss function. The proposed system consists of a deep neural convolution network (DCNN) that will improve the classification accuracy by using modified loss function along with regularization. Firstly, images are pre-processed and fed through DCNN that contains different layers with different activation function for the feature extraction and classification. Along with modified loss function with regularization, weight function aids in the classification of WBCs by considering weights of samples belonging to each class for compensating the error arising due to imbalanced dataset. The processing time will be counted by each image to check the time enhancement. The classification accuracy and processing time are achieved using the dataset-master. Our proposed solution obtains better classification performance in the given dataset comparing with other previous methods. The proposed system enhanced the classification accuracy of 98.92% from 96.1% and a decrease in processing time from 0.354 to 0.216 s. Less time will be required by our proposed solution for achieving the model convergence with 9 epochs against the current convergence time of 13.5 epochs on average, epoch is the formation white blood cells (WBCs) and the development of granular cells. The proposed solution modified loss function to solve the adverse effect caused due to imbalance dataset by considering weight and use regularization technique for overfitting problem.

Deep learning neural network for texture feature extraction in oral cancer: enhanced loss function

The use of a binary classifier like the sigmoid function and loss functions reduces the accuracy of deep learning algorithms. This research aims to increase the accuracy of detecting and classifying oral tumours within a reduced processing time. The proposed system consists of a Convolutional neural network with a modified loss function to minimise the error in predicting and classifying oral tumours by reducing the overfitting of the data and supporting multi-class classification. The proposed solution was tested on data samples from multiple datasets with four kinds of oral tumours. The averages of the different accuracy values and processing times were calculated to derive the overall accuracy. Based on the obtained results, the proposed solution achieved an overall accuracy of 96.5%, which was almost 2.0% higher than the state-of-the-art solution with 94.5% accuracy. Similarly, the processing time has been reduced by 30–40 milliseconds against the state-of-the-art solution. The proposed system is focused on detecting oral tumours in the given magnetic resonance imaging (MRI) scan and classifying whether the tumours are benign or malignant. This study solves the issue of over fitting data during the training of neural networks and provides a method for multi-class classification.

Deep convolutional network for breast cancer classification: enhanced loss function (ELF)

The accurate classification of the histopathological images of breast cancer diagnosis may face a huge challenge due to the complexity of the pathologist images. Currently, computer-aided diagnosis is implemented to get sound and error-less diagnosis of this lethal disease. However, the classification accuracy and processing time can be further improved. This study was designed to control diagnosis error via enhancing image accuracy and reducing processing time by applying several algorithms such as deep learning, K-means, autoencoder in clustering and enhanced loss function (ELF) in classification. Histopathological images were obtained from five datasets and pre-processed by using stain normalisation and linear transformation filter. These images were patched in sizes of 512 × 512 and 128 × 128 and extracted to preserve the tissue and cell levels to have important information of these images. The patches were further pre-trained by ResNet50-128 and ResNet512. Meanwhile, the 128 × 128 were clustered and autoencoder was employed with K-means which used latent feature of image to obtain better clustering result. Classification algorithm is used in current proposed system to ELF. This was achieved by combining SVM loss function and optimisation problem. The current study has shown that the deep learning algorithm has increased the accuracy of breast cancer classification up to 97% compared to state-of-the-art model which gave a percentage of 95%, and the time was decreased to vary from 30 to 40 s. Also, this work has enhanced system performance via improving clustering by employing K-means with autoencoder for the nonlinear transformation of histopathological image.