Combined Loss Function Research Articles

MFUnetr: A transformer-based multi-task learning network for multi-organ segmentation from partially labeled datasets

As multi-organ segmentation of CT images is crucial for clinical applications, most state-of-the-art models rely on a fully annotated dataset with strong supervision to achieve high accuracy for particular organs. However, these models have weak generalization when applied to various CT images due to the small scale and single source of the training data. To utilize existing partially labeled datasets to obtain segmentation containing more organs and with higher accuracy and robustness, we create a multi-task learning network called MFUnetr. By directly feeding a union of datasets, MFUnetr trains an encoder-decoder network on two tasks in parallel. The main task is to produce full organ segmentation using a specific training strategy. The auxiliary task is to segment labeled organs of each dataset using label priors. Additionally, we offer a new weighted combined loss function to optimize the model. Compared to the base model UNETR trained on the fully annotated dataset BTCV, our network model, utilizing a combination of three partially labeled datasets, achieved mean Dice on overlapping organs: spleen + 0.35 %, esophagus + 15.28 %, and aorta + 8.31 %. Importantly, without fine-tuning, the mean Dice calculated on 13 organs of BTCV remained + 1.91 % when all 15 organs were segmented. The experimental results show that our proposed method can effectively use existing large partially annotated datasets to alleviate the problem of data scarcity in multi-organ segmentation.

Contactless Palm Vein Recognition Based on Attention-Gated Residual U-Net and ECA-ResNet

Palm vein recognition has received some considerable attention regarding its use in biometric identification. Palm vein characteristics offer a superior level of security and reliability in personal identification compared to extrinsic methods such as fingerprint, face, and palm print recognition, as vein patterns are difficult to duplicate and do not change throughout one’s lifetime. This study proposes both segmentation and recognition methods to enhance the authentication performance and achieve correct identification using palm vein features. First, we propose a segmentation model based on the U-Net model, enhanced with an attention gate, to effectively segment palm vein patterns. The incorporation of both the attention gate and residual block allows the segmentation model for the learning of the essential features required for specific segmentation tasks. The Hessian-based Jerman filtering method is used for ground-truth labeling. The segmentation model extracts the palm vein patterns and filters out the irrelevant and noisy pixels for the purpose of recognition. The efficient channel attention residual network is trained to learn discriminative features for personal identification using combined margin-based loss functions for palm vein recognition. The channel attention module enhances the useful information and suppresses irrelevant features in the feature maps, which overcomes the problem of rotation, position translation, and scale transformation, as well as improves the recognition rate. The combined loss function used in this study increases the similarity between the intra-class samples and the diversity between inter-class samples. The proposed recognition model achieved 100% accuracy for palm vein recognition and an equal error rate of 0.018 for palm vein verification.

An improved YOLOv5 for object detection in visible and thermal infrared images based on contrastive learning

An improved algorithm has been proposed to address the challenges encountered in object detection using visible and thermal infrared images. These challenges include the diversity of object detection perspectives, deformation of the object, occlusion, illumination, and detection of small objects. The proposed algorithm introduces the concept of contrastive learning into the YOLOv5 object detection network. To extract image features for contrastive loss calculation, object and background image regions are randomly cropped from image samples. The contrastive loss is then integrated into the YOLOv5 network, and the combined loss function of both object detection and contrastive learning is used to optimize the network parameters. By utilizing the strategy of contrastive learning, the distinction between the background and the object in the feature space is improved, leading to enhanced object detection performance of the YOLOv5 network. The proposed algorithm has shown pleasing detection results in both visible and thermal infrared images.

FSVM: A Few-Shot Threat Detection Method for X-ray Security Images.

In recent years, automatic detection of threats in X-ray baggage has become important in security inspection. However, the training of threat detectors often requires extensive, well-annotated images, which are hard to procure, especially for rare contraband items. In this paper, a few-shot SVM-constraint threat detection model, named FSVM is proposed, which aims at detecting unseen contraband items with only a small number of labeled samples. Rather than simply finetuning the original model, FSVM embeds a derivable SVM layer to back-propagate the supervised decision information into the former layers. A combined loss function utilizing SVM loss is also created as the additional constraint. We have evaluated FSVM on the public security baggage dataset SIXray, performing experiments on 10-shot and 30-shot samples under three class divisions. Experimental results show that compared with four common few-shot detection models, FSVM has the highest performance and is more suitable for complex distributed datasets (e.g., X-ray parcels).

Artificial Intelligence-Aided Diagnosis Solution by Enhancing the Edge Features of Medical Images.

Bone malignant tumors are metastatic and aggressive. The manual screening of medical images is time-consuming and laborious, and computer technology is now being introduced to aid in diagnosis. Due to a large amount of noise and blurred lesion edges in osteosarcoma MRI images, high-precision segmentation methods require large computational resources and are difficult to use in developing countries with limited conditions. Therefore, this study proposes an artificial intelligence-aided diagnosis scheme by enhancing image edge features. First, a threshold screening filter (TSF) was used to pre-screen the MRI images to filter redundant data. Then, a fast NLM algorithm was introduced for denoising. Finally, a segmentation method with edge enhancement (TBNet) was designed to segment the pre-processed images by fusing Transformer based on the UNet network. TBNet is based on skip-free connected U-Net and includes a channel-edge cross-fusion transformer and a segmentation method with a combined loss function. This solution optimizes diagnostic efficiency and solves the segmentation problem of blurred edges, providing more help and reference for doctors to diagnose osteosarcoma. The results based on more than 4000 osteosarcoma MRI images show that our proposed method has a good segmentation effect and performance, with Dice Similarity Coefficient (DSC) reaching 0.949, and show that other evaluation indexes such as Intersection of Union (IOU) and recall are better than other methods.

Efficient Attention Branch Network with Combined Loss Function for Automatic Speaker Verification Spoof Detection.

Many endeavors have sought to develop countermeasure techniques as enhancements on Automatic Speaker Verification (ASV) systems, in order to make them more robust against spoof attacks. As evidenced by the latest ASVspoof 2019 countermeasure challenge, models currently deployed for the task of ASV are, at their best, devoid of suitable degrees of generalization to unseen attacks. A joint improvement of components of ASV spoof detection systems including the classifier, feature extraction phase, and model loss function may lead to a better detection of attacks by these systems. Accordingly, the present study proposes the Efficient Attention Branch Network (EABN) architecture with a combined loss function to address the model generalization to unseen attacks. The EABN is based on attention and perception branches. The attention branch provides an attention mask that improves the classification performance and at the same time is interpretable from a human point of view. The perception branch, is used for our main purpose which is spoof detection. The new EfficientNet-A0 architecture was optimized and employed for the perception branch, with nearly ten times fewer parameters and approximately seven times fewer floating-point operations than the SE-Res2Net50 as the best existing network. The proposed method on ASVspoof 2019 dataset achieved EER = 0.86% and t-DCF = 0.0239 in the Physical Access (PA) scenario using the logPowSpec as the input feature extraction method. Furthermore, using the LFCC feature, and the SE-Res2Net50 for the perception branch, the proposed model achieved EER = 1.89% and t-DCF = 0.507 in the Logical Access (LA) scenario, which to the best of our knowledge, is the best single system ASV spoofing countermeasure method.

Deep learning on lateral flow immunoassay for the analysis of detection data.

Lateral flow immunoassay (LFIA) is an important detection method in vitro diagnosis, which has been widely used in medical industry. It is difficult to analyze all peak shapes through classical methods due to the complexity of LFIA. Classical methods are generally some peak-finding methods, which cannot distinguish the difference between normal peak and interference or noise peak, and it is also difficult for them to find the weak peak. Here, a novel method based on deep learning was proposed, which can effectively solve these problems. The method had two steps. The first was to classify the data by a classification model and screen out double-peaks data, and second was to realize segmentation of the integral regions through an improved U-Net segmentation model. After training, the accuracy of the classification model for validation set was 99.59%, and using combined loss function (WBCE + DSC), intersection over union (IoU) value of segmentation model for validation set was 0.9680. This method was used in a hand-held fluorescence immunochromatography analyzer designed independently by our team. A Ferritin standard curve was created, and the T/C value correlated well with standard concentrations in the range of 0-500 ng/ml (R 2 = 0.9986). The coefficients of variation (CVs) were ≤ 1.37%. The recovery rate ranged from 96.37 to 105.07%. Interference or noise peaks are the biggest obstacle in the use of hand-held instruments, and often lead to peak-finding errors. Due to the changeable and flexible use environment of hand-held devices, it is not convenient to provide any technical support. This method greatly reduced the failure rate of peak finding, which can reduce the customer's need for instrument technical support. This study provided a new direction for the data-processing of point-of-care testing (POCT) instruments based on LFIA.

Cardiac Magnetic Resonance Image Segmentation Method Based on Multi-Scale Feature Fusion and Sequence Relationship Learning.

Accurate segmentation of the left atrial structure using magnetic resonance images provides an important basis for the diagnosis of atrial fibrillation (AF) and its treatment using robotic surgery. In this study, an image segmentation method based on sequence relationship learning and multi-scale feature fusion is proposed for 3D to 2D sequence conversion in cardiac magnetic resonance images and the varying scales of left atrial structures within different slices. Firstly, a convolutional neural network layer with an attention module was designed to extract and fuse contextual information at different scales in the image, to strengthen the target features using the correlation between features in different regions within the image, and to improve the network's ability to distinguish the left atrial structure. Secondly, a recurrent neural network layer oriented to two-dimensional images was designed to capture the correlation of left atrial structures in adjacent slices by simulating the continuous relationship between sequential image slices. Finally, a combined loss function was constructed to reduce the effect of positive and negative sample imbalance and improve model stability. The Dice, IoU, and Hausdorff distance values reached 90.73%, 89.37%, and 4.803 mm, respectively, based on the LASC2013 (left atrial segmentation challenge in 2013) dataset; the corresponding values reached 92.05%, 89.41% and 9.056 mm, respectively, based on the ASC2018 (atrial segmentation challenge at 2018) dataset.

CADW: CGAN-Based Attack on Deep Robust Image Watermarking

Robust watermarking plays an essential role in copyright protection for digital images. Meanwhile, the studies of robust image watermarking and corresponding attack methods promote and complement each other. Because the traditional attack methods are weak to attack the deep watermarking, in this work, we thus design an effective watermarking attack method based on the conditional generative adversarial nets (CGAN). Also, according to the network structure of CGAN, a targeted and combined loss function is constructed, which can guarantee acceptable visual quality of attacked image and make the watermark extraction fail simultaneously. Experimental results demonstrate that our method is effective for some state-of-the-art deep robust image watermarking methods with the transferability. In addition, as an attack method, it can be regarded as an evaluation standard to measure the robustness of watermarking.

Deep radio signal clustering with interpretability analysis based on saliency map

With the development of information technology, radio communication technology has made rapid progress. Many radio signals that have appeared in space are difficult to classify without manually labeling. Unsupervised radio signal clustering methods have recently become an urgent need for this situation. Meanwhile, the high complexity of deep learning makes it difficult to understand the decision results of the clustering models, making it essential to conduct interpretable analysis. This paper proposed a combined loss function for unsupervised clustering based on autoencoder. The combined loss function includes reconstruction loss and deep clustering loss. Deep clustering loss is added based on reconstruction loss, which makes similar deep features converge more in feature space. In addition, a features visualization method for signal clustering was proposed to analyze the interpretability of autoencoder utilizing Saliency Map. Extensive experiments have been conducted on a modulated signal dataset, and the results indicate the superior performance of our proposed method over other clustering algorithms. In particular, for the simulated dataset containing six modulation modes, when the SNR is 20 dB, the clustering accuracy of the proposed method is greater than 78%. The interpretability analysis of the clustering model was performed to visualize the significant features of different modulated signals and verified the high separability of the features extracted by clustering model.

Semantic Segmentation of Fish and Underwater Environments Using Deep Convolutional Neural Networks and Learned Active Contours

The conservation of marine resources requires constant monitoring of the underwater environment by researchers. For this purpose, visual automated monitoring systems are of great interest, especially those that can describe the environment using semantic segmentation based on deep learning. Although they have been successfully used in several applications, such as biomedical ones, obtaining optimal results in underwater environments is still a challenge due to the heterogeneity of water and lighting conditions, and the scarcity of labeled datasets. Even more, the existing deep learning techniques oriented to semantic segmentation only provide low resolution results, lacking the enough spatial details for a high performance monitoring. To address these challenges, a combined loss function based on the active contour theory and level set methods is proposed to refine the spatial segmentation resolution and quality. To evaluate the method, a new underwater dataset with pixel annotations for three classes (fish, seafloor, and water) was created using images from publicly accessible datasets like SUIM, RockFish, and DeepFish. The performance of architectures of convolutional neural networks (CNNs), such as UNet and DeepLabV3+, trained with different loss functions (cross entropy, dice, and active contours) was compared, finding that the proposed combined loss function improved the segmentation results by around 3%, both in the metric Intercept Over Union (IoU) as in Hausdorff Distance (HD).

Patient Assistance System Based on Hand Gesture Recognition

We propose a two-stage hand gesture recognition architecture to support a patient assistance system. Some medical conditions limit mobility, and the patient must rely on medical staff to meet their needs. In such cases, a phone or intercom is not convenient to call for help. A vision-based system operated by changing the orientation of fingers can be used to send specific messages without making arm movements. However, vision-based hand gesture recognition is hindered by occlusion, background clutter, and variations in illumination. Therefore, we developed a two-stage architecture: The first stage produces a saliency map to simplify recognition and the second stage performs classification. A novel combined loss function optimizes the saliency detection model and makes the saliency map more precise. An adaptive kernel-based channel attention layer is used to emphasize salient features. The proposed architecture achieved precise saliency detection on four benchmark datasets and high-accuracy recognition on two. We designed an interface for patients to send specific messages to the medical staff using hand gestures. The interface help patients request assistance and connect with medical staff without leaving the bed or involving a third party.

Segmentation and differentiation of periventricular and deep white matter hyperintensities in 2D T2-FLAIR MRI based on a cascade U-net.

White matter hyperintensities (WMHs) are a subtype of cerebral small vessel disease and can be divided into periventricular WMHs (pvWMHs) and deep WMHs (dWMHs). pvWMHs and dWMHs were proved to be determined by different etiologies. This study aimed to develop a 2D Cascade U-net (Cascade U) for the segmentation and differentiation of pvWMHs and dWMHs on 2D T2-FLAIR images. A total of 253 subjects were recruited in the present study. All subjects underwent 2D T2-FLAIR scan on a 3.0 Tesla MR scanner. Both contours of pvWMHs and dWMHs were manually delineated by the observers and considered as the gold standard. Fazekas scale was used to evaluate the burdens of pvWMHs and dWMHs, respectively. Cascade U consisted of a segmentation U-net and a differentiation U-net and was trained with a combined loss function. The performance of Cascade U was compared with two other U-net models (Pipeline U and Separate U). Dice similarity coefficient (DSC), Matthews correlation coefficient (MCC), precision, and recall were used to evaluate the performances of all models. The linear correlations between WMHs volume (WMHV) measured by all models and the gold standard were also conducted. Compared with other models, Cascade U exhibited a better performance on WMHs segmentation and pvWMHs identification. Cascade U achieved DSC values of 0.605 ± 0.135, 0.517 ± 0.263, and 0.510 ± 0.241 and MCC values of 0.617 ± 0.122, 0.526 ± 0.263, and 0.522 ± 0.243 on the segmentation of total WMHs, pvWMHs, and dWMHs, respectively. Cascade U exhibited strong correlations with the gold standard on measuring WMHV (R2 = 0.954, p < 0.001), pvWMHV (R2 = 0.933, p < 0.001), and dWMHV (R2 = 0.918, p < 0.001). A significant correlation was found on lesion volume between Cascade U and gold standard (r > 0.510, p < 0.001). Cascade U showed competitive results in segmentation and differentiation of pvWMHs and dWMHs on 2D T2-FLAIR images, indicating potential feasibility in precisely evaluating the burdens of WMHs.

Auxiliary Segmentation Method of Osteosarcoma MRI Image Based on Transformer and U-Net.

One of the most prevalent malignant bone tumors is osteosarcoma. The diagnosis and treatment cycle are long and the prognosis is poor. It takes a lot of time to manually identify osteosarcoma from osteosarcoma magnetic resonance imaging (MRI). Medical image processing technology has greatly alleviated the problems faced by medical diagnoses. However, MRI images of osteosarcoma are characterized by high noise and blurred edges. The complex features increase the difficulty of lesion area identification. Therefore, this study proposes an osteosarcoma MRI image segmentation method (OSTransnet) based on Transformer and U-net. This technique primarily addresses the issues of fuzzy tumor edge segmentation and overfitting brought on by data noise. First, we optimize the dataset by changing the precise spatial distribution of noise and the data-increment image rotation process. The tumor is then segmented based on the model of U-Net and Transformer with edge improvement. It compensates for the limitations of U-semantic Net by using channel-based transformers. Finally, we also add an edge enhancement module (BAB) and a combined loss function to improve the performance of edge segmentation. The method's accuracy and stability are demonstrated by the detection and training results based on more than 4,000 MRI images of osteosarcoma, which also demonstrate how well the method works as an adjunct to clinical diagnosis and treatment.

Self-Training Based Adversarial Domain Adaptation for Radio Signal Recognition

Deep learning has made great progress in the field of radio signal recognition. However, existing works usually assume that testing and training data own the same feature domain, which may not be the case in real-world scenarios. The existing cross-domain radio signal recognition methods based on domain adaptation only consider the global feature alignment. A novel unsupervised adversarial domain-adaptive learning framework named DASig is proposed to address the cross-domain radio signal recognition issue for the data acquired in different channel conditions. To align the fine-grained class distributions of the target and source domains, we designed an efficient combined loss function and iterative self-training strategy via target domain pseudo-labels. Moreover, a non-shared domain self-attention mechanism is designed to preserve domain-specific features. Extensive experiment comparisons with five state-of-the-art (SOTA) methods on both the modulation datasets and protocol datasets show the superiority of the DASig.

A deep learning model for CT-based kidney volume determination in dogs and normal reference definition.

Kidney volume is associated with renal function and the severity of renal diseases, thus accurate assessment of the kidney is important. Although the voxel count method is reported to be more accurate than several methods, its laborious and time-consuming process is considered as a main limitation. In need of a new technology that is fast and as accurate as the manual voxel count method, the aim of this study was to develop the first deep learning model for automatic kidney detection and volume estimation from computed tomography (CT) images of dogs. A total of 182,974 image slices from 386 CT scans of 211 dogs were used to develop this deep learning model. Owing to the variance of kidney size and location in dogs compared to humans, several processing methods and an architecture based on UNEt Transformers which is known to show promising results for various medical image segmentation tasks including this study. Combined loss function and data augmentation were applied to elevate the performance of the model. The Dice similarity coefficient (DSC) which shows the similarity between manual segmentation and automated segmentation by deep-learning model was 0.915 ± 0.054 (mean ± SD) with post-processing. Kidney volume agreement analysis assessing the similarity between the kidney volume estimated by manual voxel count method and the deep-learning model was r = 0.960 (p < 0.001), 0.95 from Lin's concordance correlation coefficient (CCC), and 0.975 from the intraclass correlation coefficient (ICC). Kidney volume was positively correlated with body weight (BW), and insignificantly correlated with body conditions score (BCS), age, and sex. The correlations between BW, BCS, and kidney volume were as follows: kidney volume = 3.701 × BW + 11.962 (R 2 = 0.74, p < 0.001) and kidney volume = 19.823 × BW/BCS index + 10.705 (R 2 = 0.72, p < 0.001). The deep learning model developed in this study is useful for the automatic estimation of kidney volume. Furthermore, a reference range established in this study for CT-based normal kidney volume considering BW and BCS can be helpful in assessment of kidney in dogs.

Neural network based image classifier resilient to destructive perturbation influences – architecture and training method

Modern methods of image recognition are sensitive to various types of disturbances, which actualize the development of resilient intelligent algorithms for safety-critical applications. The current article develops a model and method of training a classifier that exhibits characteristics of resilience to adversarial attacks, fault injection, and concept drift. The proposed model has a hierarchical structure of prototypes and hyperspherical boundaries of classes formed in the space of high-level features. Class boundaries are optimized during training and provide perturbation absorption and graceful degradation. The proposed learning method involves the use of a combined loss function, which allows the use of both labeled and unlabeled data, implements the compression of the feature representation to a discrete form and ensures the compactness of the distribution of classes and the maximization of the buffer zone between classes. The main component of the loss function is the value of the normalized modification of Shannon's information measure, averaged over the alphabet of the classes, expressed as a function of accuracy characteristics. Simultaneously, accuracy characteristics are calculated on the basis of smoothed versions of the distribution of statistical hypothesis testing results. It is experimentally confirmed that the proposed approach provides a certain level of disturbance absorption, graceful degradation and recovery. During testing of the proposed algorithm on the Cifar10 data set, it was established that the integral metric of resilience to tensor damage by inversion of one randomly selected bit is about 0.95 if the share of damaged tensors does not exceed 30%. Also, during testing of the proposed algorithm, it was established that an adversarial attack with a disturbance that does not exceed the L∞-norm threshold equal to 3 provides resilience that exceeds the value of 0.95 according to the integral metric. Additionally, the integral metric of resilience during adaptation to the appearance of two new classes is 0.959. The integral metric of resilience to the real drift of concepts between the two classes is 0.973. The ability to adapt to the appearance of new classes or the concept drift has been confirmed 8 times faster than learning from scratch.

An Image Fusion Method Based on Special Residual Network and Efficient Channel Attention

This paper presents an image fusion network based on a special residual network and attention mechanism. Compared with the traditional fusion network, the image fusion network has the advantages of an end-to-end network and integrates the feature extraction advantages of the attention mechanism residual network. It overcomes the shortcomings of the traditional network that need complex design rules and manual operation. In this method, hierarchical feature fusion is used to achieve effective fusion. A combined loss function is designed to optimize training results and improve image fusion quality. This paper uses many qualitative and quantitative experimental analyses on different data sets. The results show that, compared with the comparison algorithm, the method in this paper has a stronger retention ability of infrared and visible light information and better indexes. 72% of eleven indexes compared with some images in the public TNO data set are optimal or sub-optimal, and 80% are optimal or suboptimal in the RoadScene data set, which is much higher than other algorithms. The overall fusion effect is more in line with human visual perception.

Unsupervised Domain Adaptive Person Re-Identification Method Based on Transformer

Person re-identification (ReID) is the problem of cross-camera target retrieval. The extraction of robust and discriminant features is the key factor in realizing the correct correlation of targets. A model based on convolutional neural networks (CNNs) can extract more robust image features. Still, it completes the extraction of images from local information to global information by continuously accumulating convolution layers. As a complex CNN, a vision transformer (ViT) captures global information from the beginning to extract more powerful features. This paper proposes an unsupervised domain adaptive person re-identification model (ViTReID) based on the vision transformer, taking the ViT model trained on ImageNet as the pre-training weight and a transformer encoder as the feature extraction network, which makes up for some defects of the CNN model. At the same time, the combined loss function of cross-entropy and triplet loss function combined with the center loss function is used to optimize the network; the person’s head is evaluated and trained as a local feature combined with the global feature of the whole body, focusing on the head, to enhance the head feature information. The experimental results show that ViTReID exceeds the baseline method (SSG) by 14% (Market1501 → MSMT17) in mean average precision (mAP). In MSMT17 → Market1501, ViTReID is 1.2% higher in rank-1 (R1) accuracy than a state-of-the-art method (SPCL); in PersonX → MSMT17, the mAP is 3.1% higher than that of the MMT-dbscan method, and in PersonX → Market1501, the mAP is 1.5% higher than that of the MMT-dbscan method.

Uneven wrapped phase pattern denoising using a deep neural network.

The wrapped phase patterns obtained from an object composed of different materials have uneven gray values. In this paper, we improve the dilated-blocks-based deep convolution neural network (DBDNet) and build a new dataset for restoring the uneven gray values of uneven wrapped phase patterns as well as eliminating the speckle noise. In our method, we improve the structure of dilated blocks in DBDNet to enhance the ability of obtaining full scales of gray values and speckle noise information in the uneven phase patterns. We use the combined MS_SSIM+L1 loss function to improve the denoising and restoration performance of our method. We compare three representative networks ResNet-based, ADNet, and BRDNet in denoising with our proposed method. We test the three compared methods and our method on one group of computer-simulated and one group of experimentally obtained uneven noisy wrapped phase patterns from a dynamic measurement. We also conduct the ablation experiments on the improved model structure and the combined loss function used in our method. The denoising performance has been evaluated quantitatively and qualitatively. The denoising results demonstrate that our proposed method can reduce high speckle noise, restore the uneven gray values of wrapped phase patterns, and get better results than the compared methods.