Loss Function Research Articles

IG‐YOLOv8: Insulator Guardian Based on YOLO for Insulator Fault Detection

AbstractInsulators have an extremely important role in transmission lines, and they are important components for ensuring the safe operation of transmission lines. In order to solve the difficult problem of insulator fault detection under complex background, IG‐YOLOv8 insulator fault detection algorithm is proposed in this paper. First, the Wise‐IoU (WIoU) loss function is introduced to mitigate the adverse impact of low‐quality images by employing a dynamic non‐monotonic focusing mechanism, thereby enhancing the detection performance of the entire model. Second, a novel C2f network is constructed by integrating the receptive field coordination attention (RFCA) convolutional module to address the parameter‐sharing issue associated with large convolutional kernels. Additionally, the data set has been reorganized using k‐fold cross‐validation to ensure that each subset undergoes training and testing, consequently reducing generalization errors. Finally, a deformable attention (DA) mechanism is employed to augment the feature extraction capability pertaining to insulator fault region information. In order to evaluate the detection performance of the improved IG‐YOLOv8 algorithm, this study constructed an insulator target detection data set containing four fault types: Normal, Defect, Dirty, and Aging. The experimental results show that the average accuracy of the improved model is increased from 89.7% to 96.9%, and the Recall value of the Aging type insulator is increased from 71.8% to 89.1%. The occurrence of missed detection is greatly reduced, and the accuracy of detection is improved. © 2024 Institute of Electrical Engineers of Japan and Wiley Periodicals LLC.

Optimal Estimation of Reliability Parameters for Modified Frechet-Exponential Distribution Using Progressive Type-II Censored Samples with Mechanical and Medical Data

The aim of this research is to estimate the parameters of the modified Frechet-exponential (MFE) distribution using different methods when applied to progressive type-II censored samples. These methods include using the maximum likelihood technique and the Bayesian approach, which were used to determine the values of parameters in addition to calculating the reliability and failure functions at time t. The approximate confidence intervals (ACIs) and credible intervals (CRIs) are derived for these parameters. Two bootstrap techniques of parametric type are provided to compute the bootstrap confidence intervals. Both symmetric loss functions such as the squared error loss (SEL) and asymmetric loss functions such as the linear-exponential (LINEX) loss are used in the Bayesian method to obtain the estimates. The Markov Chain Monte Carlo (MCMC) technique is utilized in the Metropolis–Hasting sampler approach to obtain the unknown parameters using the Bayes approach. Two actual datasets are utilized to examine the various progressive schemes and different estimation methods considered in this paper. Additionally, a simulation study is performed to compare the schemes and estimation techniques.

The impact of oral health on depression: A systematic review.

As of 2020, about 21% of adults in the United States have a diagnosable mental health disorder, excluding substance use and developmental disorders. Depression, predicted by the WHO to be the leading cause of disease burden by 2030, is linked to various systemic conditions and has been associated with poor oral health. Both behavioral factors, like poor dental hygiene and irregular visits, and biological mechanisms, such as changes in salivary immunity, contribute to this connection, which impacts overall well-being and quality of life. This systematic review aims include: (1) Does tooth loss affect depression? (2) Does oral pain, such as that experienced during chewing and speaking, impact depression? (3) Does oral functionality, including chewing and speaking, influence depression? (4) Does overall oral health affect depression? We conducted a systematic search of PubMed, EBSCO host, Medline, and Google Scholar databases from January 2000 to June 2024 using relevant keywords. Studies examining the impact of oral health parameters (tooth loss, oral pain, oral functionality, overall oral health) on depression were included. Articles were included if (1) full text manuscripts in English were available, (2) the study described the association of oral health and depression, and (3) the independent value was an oral related factor and the dependent value was depression. The following were excluded from our analysis: (1) any articles where oral factors were not the independent value, (2) systematic reviews, (3) case reports, and (4) duplicate studies among our databases. Thirty-one studies met the inclusion criteria. Tooth loss, oral pain, and impaired oral functionality were consistently associated with increased depressive symptoms across the included studies. Greater tooth loss was linked to higher odds of both onset and progression of depression. Oral pain exacerbated depressive symptoms, while difficulties in chewing or speaking were associated with elevated risks of depression. There is a bidirectional relationship between oral health and depression, highlighting the urgent need for comprehensive public health initiatives. Integrating oral health assessments into routine medical care, and developing targeted interventions are crucial steps to mitigate the impact of poor oral health on mental health outcomes.

CrackNet: A Hybrid Model for Crack Segmentation with Dynamic Loss Function

Cracks are a common form of damage in infrastructure, posing significant risks to both personal safety and property. Along with the development of deep learning, visual-based crack automatic detection has been widely studied. However, this task is still challenging due to complex crack topology, noisy backgrounds, unbalanced categories, etc. To address these challenges, this research proposes a novel hybrid network, named CrackNet, which leverages the strengths of both CNN and transformer. On the encoder side, CNNs are employed to extract multi-level local features, while transformers are used to model global dependencies. Additionally, a strip pooling module is introduced to suppress irrelevant regions and enhance the network’s ability to segment narrow and elongated cracks. On the decoder side, an attention-based skip connection strategy and a mixed up-sampling module are implemented to restore detailed information. Furthermore, a joint learning loss combining Dice and cross-entropy with dynamic weighting is proposed to mitigate the effects of severe class imbalance. CrackNet is trained and evaluated on three public crack datasets, and experimental results show that the proposed model outperforms several well-known deep neural networks, with a particularly noticeable improvement in recall rate.

Assessing Helicopter Turbine Engine Health: A Simple Yet Robust Probabilistic Approach

This paper presents a data-driven approach for assessing the health of helicopter turbine engines, developed for the PHM North America 2024 Conference Data Challenge. The task involves both regression and classification to estimate the torque margin and classify engine health as either nominal or faulty. To quantify the reliability of predictions, probabilistic outputs are generated. We employ a two-stage model where the predicted torque margin serves as an input feature for health classification. For probabilistic torque margin estimation, we introduce an empirical error sampling method to generate torque margin samples, followed by a rule-based distribution selection scheme to evaluate the resulting distributions. For fault classification, logistic regression is used to provide confidence estimates, and we incorporate a score-optimized loss function during training to mitigate penalties for false negatives. Our approach demonstrates strong generalization to unseen assets, ranking 2nd in the competition with a score of 0.94, demonstrating its effectiveness in predicting health conditions and uncertainty for more informed helicopter engine management.

Post-Training Attribute Unlearning in Recommender Systems

With the growing privacy concerns in recommender systems, recommendation unlearning is getting increasing attention. Existing studies predominantly use training data, i.e., model inputs, as unlearning target. However, attackers can extract private information from the model even if it has not been explicitly encountered during training. We name this unseen information as attribute and treat it as unlearning target. To protect the sensitive attribute of users, Attribute Unlearning (AU) aims to make target attributes indistinguishable. In this paper, we focus on a strict but practical setting of AU, namely Post-Training Attribute Unlearning (PoT-AU), where unlearning can only be performed after the training of the recommendation model is completed. To address the PoT-AU problem in recommender systems, we propose a two-component loss function. The first component is distinguishability loss, where we design a distribution-based measurement to make attribute labels indistinguishable from attackers. We further extend this measurement to handle multi-class attribute cases with efficient computational overhead. The second component is regularization loss, where we explore a function-space measurement that effectively maintains recommendation performance compared to parameter-space regularization. We use stochastic gradient descent algorithm to optimize our proposed loss. Extensive experiments on four real-world datasets demonstrate the effectiveness of our proposed methods.

Combination of edge enhancement and cold diffusion model for low dose CT image denoising.

Since the quality of low dose CT (LDCT) images is often severely affected by noise and artifacts, it is very important to maintain high quality CT images while effectively reducing the radiation dose. In recent years, the representation of diffusion models to produce high quality images and stable trainability has attracted wide attention. With the extension of the cold diffusion model to the classical diffusion model, its application has greater flexibility. Inspired by the cold diffusion model, we proposes a low dose CT image denoising method, called CECDM, based on the combination of edge enhancement and cold diffusion model. The LDCT image is taken as the end point (forward) of the diffusion process and the starting point (reverse) of the sampling process. Improved sobel operator and Convolution Block Attention Module are added to the network, and compound loss function is adopted. The experimental results show that CECDM can effectively remove noise and artifacts from LDCT images while the inference time of a single image is reduced to 0.41 s. Compared with the existing LDCT image post-processing methods, CECDM has a significant improvement in all indexes.

Learned phase mask to protect camera under laser irradiation

The electro-optical imaging system works under focus conditions for clear imaging. However, under unexpected laser irradiation, the focused light with extremely high intensity can easily damage the imaging sensor, resulting in permanent degradation of its perceptual capabilities. With the escalating prevalence of compact high-performance lasers, safeguarding cameras from laser damage presents a formidable challenge. Here, we report an end-to-end method to construct the wavefront coding (WFC) imaging systems with simultaneous superior laser protection and imaging performance. In the optical coding part, we employ four types of phase mask parameterization methods: pixel-wise, concentric rings, linear combinations of Zernike bases, and odd-order polynomial bases, with parameters that are learnable. In the algorithm decoding part, a method combined of deconvolution module and residual-Unet is proposed to furthest restore the phase-mask-induced image blurring. The optical and algorithm parts are jointly optimized within the end-to-end framework to determine the performance boundary. The governing rule of the laser protection capability versus imaging quality is revealed by tuning the optimization loss function, and the system database is established for various working conditions. Numerical simulations and experimental validations both demonstrate that the proposed laser-protection WFC imaging system can reduce the peak single-pixel laser power by 99.4% while maintaining high-quality imaging with peak signal-to-noise ratio more than 22 dB. This work pioneers what we believe to be a new path for the design of laser protection imaging systems, with promising applications in security and autonomous driving.

Prediction of laser beam spatial profiles in a high-energy laser facility by use of deep learning

We adapt the significant advances achieved recently in the field of generative artificial intelligence/machine-learning to laser performance modeling in multipass, high-energy laser systems with application to high-shot-rate facilities relevant to inertial fusion energy. Advantages of neural-network architectures include rapid prediction capability, data-driven processing, and the possibility to implement such architectures within future low-latency, low-power consumption photonic networks. Four models were investigated that differed in their generator loss functions and utilized the U-Net encoder/decoder architecture with either a reconstruction loss alone or combined with an adversarial network loss. We achieved inference times of 1.3 ms for a 256 × 256 pixel near-field beam with errors in predicted energy of the order of 1% over most of the energy range. It is shown that prediction errors are significantly reduced by ensemble averaging the models with different weight initializations. These results suggest that including the temporal dimension in such models may provide accurate, real-time spatiotemporal predictions of laser performance in high-shot-rate laser systems.

A Multiclassification Model for Skin Diseases Using Dermatoscopy Images with Inception-v2

Skin cancer represents a significant global public health concern, with over five million new cases diagnosed annually. If not diagnosed at an early stage, skin diseases have the potential to pose a significant threat to human life. In recent years, deep learning has increasingly been used in dermatological diagnosis. In this paper, a multiclassification model based on the Inception-v2 network and the focal loss function is proposed on the basis of deep learning, and the ISIC 2019 dataset is optimised using data augmentation and hair removal to achieve seven classifications of dermatological images and generate heat maps to visualise the predictions of the model. The results show that the model has an average accuracy of 89.04%, a precision of 87.37%, recall of 90.15%, and an F1-score of 88.76%, The accuracy rates of ResNext101, MobileNetv2, Vgg19, and ConvNet are 88.50%, 85.30%, 88.57%, and 86.90%, respectively. These results show that our proposed model performs better than the above models and performs well in classifying dermatological images, which has significant application value.

Deep Learning for Stomatal Opening Recognition in Gynura formosana Kitam Leaves

Gynura formosana Kitam possesses beneficial properties such as heat-clearing, detoxification, and cough suppression, making it a highly nutritious plant with significant economic value. During its growth, the plant’s leaves are prone to infections that can impair stomatal function and hinder growth. Effective identification of stomatal openings and timely application of appropriate chemicals or hormones or indirect environmental adjustments (such as light, temperature, and humidity) to regulate stomatal openings are essential for maintaining the plant’s healthy growth. Currently, manual observation is the predominant method for monitoring stomatal openings of Gynura formosana Kitam, which is complex, labor-intensive, and unsuitable for automated detection. To address this, the study improves upon YOLOv8s by proposing a real-time, high-precision stomatal detection model, Refined GIoU. This model substitutes the original IoU evaluation methods in YOLOv8s with GIoU, DIoU, and EIoU while incorporating the SE (Squeeze-and-Excitation) and SA (Self-Attention) attention mechanisms to enhance understanding of feature representation and spatial relationships. Additionally, enhancements to the P2 layer improve the feature extraction and scale adaptation. The effectiveness of the Refined GIoU is demonstrated through training and validation on a dataset of 1500 images of Gynura formosana Kitam stomata. The results show that the Refined GIoU achieved an average precision (mAP) of 0.935, a recall of 0.98, and an F1-score of 0.88, reflecting an excellent overall performance. The GIoU loss function is better suited to detecting stomatal openings of Gynura formosana Kitam, significantly enhancing the detection accuracy. This model facilitates the automated, real-time monitoring of stomatal openings, allowing for timely control measures and improved economic benefits of Gynura formosana Kitam cultivation.

Association between self-reported weight loss and new long-term care insurance certifications: A 9-year Japanese older adult cohort study.

This cohort study aimed to assess weight loss associated with new long-term care insurance (LTCI) certifications over a 9-year period, accounting for the competing risk of death. We analyzed data from 3749 Japanese individuals aged ≥65 years in Kami Town, Hyogo Prefecture, Japan. Weight loss was assessed using the Kihon Checklist during the baseline survey. Data regarding LTCI certifications were collected until March 2022. Cox proportional hazards models were used to calculate adjusted hazard ratios (HRs) of 9-year LTCI certification because of weight loss, adjusted for confounding factors. To exclude the effect of competing risks, Fine-Gray regression was used to estimate subdistribution HRs. Subgroup analyses were carried out after the examination of potential interactions between subjective cognitive function, body mass index categories and weight loss. The incidence rate of new LTCI certifications was 5.16 per 100 person-years overall - broken down into 7.02 for those with weight loss and 4.97 for those without. The adjusted HR for weight loss to new LTCI certifications was 1.35 (95% CI 1.15-1.59). Considering mortality as a competing risk, the adjusted subdistribution HR was 1.37 (95% CI 1.16-1.61). Conversely, no interaction was observed between weight loss and subjective cognitive function or body mass index categories. Excluding the effect of mortality, weight loss was identified as a risk factor for new LTCI certifications. However, no interaction was observed between weight loss and subjective cognitive function or body mass index categories. Geriatr Gerontol Int 2024; ••: ••-••.

Large-Kernel Central Block Masked Convolution and Channel Attention-Based Reconstruction Network for Anomaly Detection of High-Resolution Hyperspectral Imagery

In recent years, the rapid advancement of drone technology has led to an increasing use of drones equipped with hyperspectral sensors for ground imaging. Hyperspectral data captured via drones offer significantly higher spatial resolution, but this also introduces more complex background details and larger target scales in high-resolution hyperspectral imagery (HRHSI), posing substantial challenges for hyperspectral anomaly detection (HAD). Mainstream reconstruction-based deep learning methods predominantly emphasize spatial local information in hyperspectral images (HSIs), relying on small spatial neighborhoods for reconstruction. As a result, large anomalous targets and background details are often well reconstructed, leading to poor anomaly detection performance, as these targets are not sufficiently distinguished from the background. To address these limitations, we propose a novel HAD network for HRHSI based on large-kernel central block masked convolution and channel attention, termed LKCMCA. Specifically, we first employ the pixel-shuffle technique to reduce the size of anomalous targets without losing image information. Next, we design a large-kernel central block masked convolution to make the network pay more attention to the surrounding background information, enabling better fusion of the information between adjacent bands. This, coupled with an efficient channel attention mechanism, allows the network to capture deeper spectral features, enhancing the reconstruction of the background while suppressing anomalous targets. Furthermore, we introduce an adaptive loss function by down-weighting anomalous pixels based on the mean absolute error. This loss function is specifically designed to suppress the reconstruction of potentially anomalous pixels during network training, allowing our model to be considered an excellent background reconstruction network. By leveraging reconstruction error, the model effectively highlights anomalous targets. Meanwhile, we produced four benchmark datasets specifically for HAD tasks using existing HRHSI data, addressing the current shortage of HRHSI datasets in the HAD field. Extensive experiments demonstrate that our LKCMCA method achieves superior detection performance, outperforming ten state-of-the-art HAD methods on all datasets.

InterLabelGO+: Unraveling label correlations in protein function prediction.

Accurate protein function prediction is crucial for understanding biological processes and advancing biomedical research. However, the rapid growth of protein sequences far outpaces the experimental characterization of their functions, necessitating the development of automated computational methods. We present InterLabelGO+, a hybrid approach that integrates a deep learning-based method with an alignment-based method for improved protein function prediction. InterLabelGO+ incorporates a novel loss function that addresses label dependency and imbalance and further enhances performance through dynamic weighting of the alignment-based component. A preliminary version of InterLabelGO+ achieved a strong performance in the CAFA5 challenge, ranking 6th out of 1,625 participating teams. Comprehensive evaluations on large-scale protein function prediction tasks demonstrate InterLabelGO+'s ability to accurately predict Gene Ontology terms across various functional categories and evaluation metrics. The source code and datasets for InterLabelGO+ are freely available on GitHub at https://github.com/QuanEvans/InterLabelGO. The software is implemented in Python and PyTorch, and is supported on Linux and macOS. Supplementary figures, tables, and data are available at Bioinformatics online.

An Enhanced Shuffle Attention with Context Decoupling Head with Wise IoU Loss for SAR Ship Detection

Synthetic Aperture Radar (SAR) imagery is widely utilized in military and civilian applications. Recent deep learning advancements have led to improved ship detection algorithms, enhancing accuracy and speed over traditional Constant False-Alarm Rate (CFAR) methods. However, challenges remain with complex backgrounds and multi-scale ship targets amidst significant interference. This paper introduces a novel method that features a context-based decoupled head, leveraging positioning and semantic information, and incorporates shuffle attention to enhance feature map interpretation. Additionally, we propose a new loss function with a dynamic non-monotonic focus mechanism to tackle these issues. Experimental results on the HRSID and SAR-Ship-Dataset demonstrate that our approach significantly improves detection performance over the original YOLOv5 algorithm and other existing methods.

Advanced semantic segmentation of aircraft main components based on transfer learning and data-driven approach

Abstract The implementation of Smart Airport and Airport 4.0 visions relies on the integration of automation, artificial intelligence, data science, and aviation technology to enhance passenger experiences and operational efficiency. One essential factor in the integration is the semantic segmentation of the aircraft main components (AMC) perception, which is essential to maintenance, repair, and operations in aircraft and airport operations. However, AMC segmentation has challenges from low data availability, high-quality annotation scarcity, and categorical imbalance, which are common in practical applications, including aviation. This study proposes a novel AMC segmentation solution, employing a transfer learning framework based on a sophisticated DeepLabV3 architecture optimized with a custom-designed Focal Dice Loss function. The proposed solution remarkably suppresses the categorical imbalance challenge and increases the dataset variability with manually annotated images and dynamic augmentation strategies to train a robust AMC segmentation model. The model achieved a notable intersection over union of 84.002% and an accuracy of 91.466%, significantly advancing the AMC segmentation performance. These results demonstrate the effectiveness of the proposed AMC segmentation solution in aircraft and airport operation scenarios. This study provides a pioneering solution to the AMC semantic perception problem and contributes a valuable dataset to the community, which is fundamental to future research on aircraft and airport semantic perception. Graphical abstract

PDGrad: Guiding Diffusion Model for Reference-Based Blind Face Restoration with Pivot Direction Gradient Guidance

Reference-based blind face restoration (RefBFR) has gained considerable attention because it utilizes additional reference images to restore facial images in situations where the degradation is caused by unknown factors, making it particularly useful in real-world applications. Recently, guided diffusion models have demonstrated exceptional performance in this task without requiring training. They achieve this by integrating gradients of the losses where each loss reflects the different desired properties of the additional external images. However, these approaches fail to consider potential conflicts between gradients of multiple losses, which can lead to sub-optimal results. To address this issue, we introduce Pivot Direction Gradient guidance (PDGrad), a novel gradient adjustment method for RefBFR within a guided diffusion framework. To this end, we first define the loss function based on both low-level and high-level features. For loss at each feature level, both the coarsely restored image and the reference image are fully integrated. In cases of conflicting gradients, a pivot gradient is established for each level and other gradients are aligned to it, ensuring that the strengths of both images are maximized. Additionally, if the magnitude of the adjusted gradient exceeds that of the pivot gradient, it is adaptively scaled according to the ratio between the two, placing greater emphasis on the pivot. Extensive experimental results on the CelebRef-HQ dataset show that the proposed PDGrad significantly outperforms competitive approaches both quantitatively and qualitatively.

Residual channel attention based sample adaptation few-shot learning for hyperspectral image classification.

Few-shot learning (FSL) uses prior knowledge and supervised experience to effectively classify hyperspectral images (HSIs), thereby reducing the cost of large numbers of labeled samples. However, existing few-shot methods ignore the correlation between cross-domain feature channels, and the feature representation ability is insufficient. To address above issue, this paper proposes a novel Residual Channel Attention Based Sample Adaptation Few-Shot Learning for Hyperspectral Image Classification(RCASA-FSL) for hyperspectral image classification (HSIC), which can capture and enhance cross-domain dependencies through multi-layer residual connection and random-based feature recalibration. Specifically, a Deep Residual Feature Channel Attention Mechanism (DRFCAM) is designed to obtain cross-domain dependencies by residual concatenation, and further the residual structure is stacked for mining depth discrimination information. Furthermore, a new Random-based Feature Recalibration Module (RFRM) is proposed to reassign the feature weights via random matrix, which fully explore feature weight relationships to guide the sample adaptation process. Besides, we design a joint loss function with combining the FSL loss and domain adaptive loss for further optimization model. Experiments conducted on several standard hyperspectral datasets demonstrate that the proposed RCASA-FSL is superior to other FSL techniques in both quantitative and qualitative aspects.

AIpollen: An Analytic Website for Pollen Identification Through Convolutional Neural Networks

With the rapid development of artificial intelligence, deep learning has been widely applied to complex tasks such as computer vision and natural language processing, demonstrating its outstanding performance. This study aims to exploit the high precision and efficiency of deep learning to develop a system for the identification of pollen. To this end, we constructed a dataset across 36 distinct genera. In terms of model selection, we employed a pre-trained ResNet34 network and fine-tuned its architecture to suit our specific task. For the optimization algorithm, we opted for the Adam optimizer and utilized the cross-entropy loss function. Additionally, we implemented ELU activation function, data augmentation, learning rate decay, and early stopping strategies to enhance the training efficiency and generalization capability of the model. After training for 203 epochs, our model achieved an accuracy of 97.01% on the test set and 99.89% on the training set. Further evaluation metrics, such as an F1 score of 95.9%, indicate that the model exhibits good balance and robustness across all categories. To facilitate the use of the model, we develop a user-friendly web interface. Users can upload images of pollen grains through the URL link provided in this article) and immediately receive predicted results of their genus names. Altogether, this study has successfully trained and validated a high-precision pollen grain identification model, providing a powerful tool for the identification of pollen.

Generating high-quality phase-only holograms of binary images using global loss and stochastic homogenization training strategy

Deep learning presents a highly effective for computer-generated holography (CGH). However, existing learning-based CGH algorithms may encounter significant obstacles in generating high-quality holograms for binary images. This study proposes a novel diffraction model-driven neural network, termed load sharing yielding holography (LSY-Holo), which leverages a dual-branch architecture and a global structural similarity index measure (Global_SSIM) loss function to generate phase-only holograms (POHs). LSY-Holo framework facilitates capturing global information, enhancing the network’s performance to address the ill-posed inverse problem. To further optimize the model for binary images, the research introduces a custom Binary dataset alongside a stochastic homogenization preprocessing method (SHPM), which mitigates the prevalent issue of excessive noise in reconstructed images. Additionally, this deep learning approach is integrated with an iterative algorithm, which employs holograms rapidly generated by LSY-Holo as the initial phase of the AdamW Holography (AH) algorithm. This strategy significantly accelerates the convergence of the iterative process. LSY-Holo is further extended to 3D scenes, enabling dynamic defocusing and focusing effects at varying depths. Results from numerical simulations and full-color optical reconstruction experiments demonstrate that the proposed method can enhance image quality and effectively reduce the artifacts in reconstructed images.