Mixed Loss Function Research Articles

A Low-Illumination Enhancement Method Based on Structural Layer and Detail Layer.

Low-illumination image enhancement technology is a topic of interest in the field of image processing. However, while improving image brightness, it is difficult to effectively maintain the texture and details of the image, and the quality of the image cannot be guaranteed. In order to solve this problem, this paper proposed a low-illumination enhancement method based on structural and detail layers. Firstly, we designed an SRetinex-Net model. The network is mainly divided into two parts: a decomposition module and an enhancement module. Second, the decomposition module mainly adopts the SU-Net structure, which is an unsupervised network that decomposes the input image into a structural layer image and detail layer image. Afterward, the enhancement module mainly adopts the SDE-Net structure, which is divided into two branches: the SDE-S branch and the SDE-D branch. The SDE-S branch mainly enhances and adjusts the brightness of the structural layer image through Ehnet and Adnet to prevent insufficient or overexposed brightness enhancement in the image. The SDE-D branch is mainly denoised and enhanced with textural details through a denoising module. This network structure can greatly reduce computational costs. Moreover, we also improved the total variation optimization model as a mixed loss function and added structural metrics and textural metrics as variables on the basis of the original loss function, which can well separate the structure edge and texture edge. Numerous experiments have shown that our structure has a more significant impact on the brightness and detail preservation of image restoration.

A Salient Object Detection Method Based on Boundary Enhancement.

Visual saliency refers to the human's ability to quickly focus on important parts of their visual field, which is a crucial aspect of image processing, particularly in fields like medical imaging and robotics. Understanding and simulating this mechanism is crucial for solving complex visual problems. In this paper, we propose a salient object detection method based on boundary enhancement, which is applicable to both 2D and 3D sensors data. To address the problem of large-scale variation of salient objects, our method introduces a multi-level feature aggregation module that enhances the expressive ability of fixed-resolution features by utilizing adjacent features to complement each other. Additionally, we propose a multi-scale information extraction module to capture local contextual information at different scales for back-propagated level-by-level features, which allows for better measurement of the composition of the feature map after back-fusion. To tackle the low confidence issue of boundary pixels, we also introduce a boundary extraction module to extract the boundary information of salient regions. This information is then fused with salient target information to further refine the saliency prediction results. During the training process, our method uses a mixed loss function to constrain the model training from two levels: pixels and images. The experimental results demonstrate that our salient target detection method based on boundary enhancement shows good detection effects on targets of different scales, multi-targets, linear targets, and targets in complex scenes. We compare our method with the best method in four conventional datasets and achieve an average improvement of 6.2% on the mean absolute error (MAE) indicators. Overall, our approach shows promise for improving the accuracy and efficiency of salient object detection in a variety of settings, including those involving 2D/3D semantic analysis and reconstruction/inpainting of image/video/point cloud data.

Multi-branch Context Awareness Network for Prostate MRI Segmentation

Prostate image segmentation is a precondition for the diagnosis of prostate diseases and subsequent treatment. However, the blurry organ edges and low image contrast make accurate segmentation difficult. In order to overcome the difficulties, this paper proposes an innovative Multi-branch Context Awareness Network (MBCA-Net) for prostate MRI segmentation. MBCA-Net uses the 3D UNet as the backbone network with an encoder-decoder framework. To improve the feature extraction capacity of the network, a multi-branch residual module with different convolution kernels is proposed. To better make use of extracted features and reduce semantic ambiguity, a context awareness module is used to fuse low-level features in the encoder and high-level features in the decoder in a full process manner. To improve the stability of the model during training, a mixed loss function is adopted. Experiments on the public dataset PROMISE12 show that our model’s Dice coefficient is 90.19%. Compared with other advanced techniques, the proposed model has better segmentation results.

HCTIRdeblur: A hybrid convolution-transformer network for single infrared image deblurring

Infrared images captured by mobile platforms often suffer image blurs such as defocus blur and motion blur, which seriously degrade the quality of infrared images. However, the existing image deblurring methods generally focused on visible image deblurring while failing to perform infrared image deblurring effectively, due to the infrared images with low resolution, lack of detailed textural information, and tend to fused objects with backgrounds when low-level temperature difference. To this end, this study proposed a novel end to end network for single infrared image blind deblurring. An encoder contains multiple hybrid convolution-transformer feature extraction blocks is designed to effectively extract inherent characteristics of infrared image. The bidirectional feature pyramid structured decoder with full scale connections is adopted to achieve fully reuse multi-stage features and reconstructed clear infrared images ideally. The multi-stage training strategy and a novel mixed loss function are introduced to speed up the convergence of training process and obtain better image deblurring performance. Moreover, a dataset dedicated to infrared images blind deblurring is constructed to facilitate the task of infrared image deblurring. Extensive ablation studies and comparison experiments have been conducted on the test set of the proposed infrared image deblurring dataset. The experimental results demonstrated the effectiveness of the proposed network structure and the superiority of the proposed network over other state of the arts deblurring methods. Finally, comparative experiment is conducted on real captured blurred infrared images and the results verified the superiority and robustness of the proposed network over other existing image deblurring methods.

A Double Stream Person Re-Identification Method Based on Attention Mechanism and Multi-Scale Feature Fusion

Person re-identification is the key technology for designing a intelligent video surveillance system to retrieve and recongise a specific person in a series of videos. It mainly plays a role in multiple non-overlapping camera monitoring environments to determine whether the target person of interest that has appeared under a camera appears again under others. However, the image data, in the real scene, taken by the surveillance camera may be occluded or blurred, which increased the difficulty of identifying the pedestrian posturn and then lead to dramatically decrease the accuracy of recognition. To solve the above problems, we propose a dualbranch multi-scale feature fusion network, which improves the expression ability of pedestrian features under partial occlusion by learning discriminative pedestrian features. The feature erasure mechanism is introduced into the branch network to effectively enhance the fine-grained feature learning ability of the network. By embedding the lightweight attention module into Resnet-50, the image sequence features of the channel dimension will be extracted, and the interference caused by the cluttered back ground information will be suppressed. In the training phase, the average pooling layer and the maximum pooling layer of different kernels and strides are utilized for different residual stages, and the mixed pooling strategy and mixed loss function of different kernels are designed. Finally, a novel hybrid loss function which combines triplet loss and ID loss is presented to train the network model. By comparison with the existing representative methods on Market1501, DukeMTMC reID and MTMS17 datasets, the experimental results show that the features extracted by the proposed method are more discriminative and with high recognition accuracy.

A spatiotemporal correlation deep learning network for brain penumbra disease

Brain penumbra is a critical condition that is closely related to stroke. Thus, there is high demand for fast and accurate segmentation of penumbra tissue in magnetic resonance images. However, most convolutional neural networks (CNNs) focus on learning contextual semantic information from two-dimensional imaging slides, ignoring the spatiotemporal correlations among adjacent slides. Here, we propose an encoder-decoder network (ConvLSTM-Net) with a specifically convolutional long short-term memory skip connection to extract the spatiotemporal correlations of features of adjacent slices in a non-linear manner. A mixed loss function is also used to improve the segmentation performance. We test the proposed method on the penumbra segmentation challenge and obtain an average Dice score over 80%, indicating that its performance is superior to or comparable with that of state-of-the-art segmentation methods. A mixed loss function provides positive support for the stability of model training. In addition, we visualize two representative samples to improve the interpretability of the results of ConvLSTM-Net.

Reinforcement Learning Based Diagnosis and Prediction for COVID-19 by Optimizing a Mixed Cost Function From CT Images.

A novel coronavirus disease (COVID-19) is a pandemic disease has caused 4 million deaths and more than 200 million infections worldwide (as of August 4, 2021). Rapid and accurate diagnosis of COVID-19 infection is critical to controlling the spread of the epidemic. In order to quickly and efficiently detect COVID-19 and reduce the threat of COVID-19 to human survival, we have firstly proposed a detection framework based on reinforcement learning for COVID-19 diagnosis, which constructs a mixed loss function that can integrate the advantages of multiple loss functions. This paper uses the accuracy of the validation set as the reward value, and obtains the initial model for the next epoch by searching the model corresponding to the maximum reward value in each epoch. We also have proposed a prediction framework that integrates multiple detection frameworks using parameter sharing to predict the progression of patients' disease without additional training. This paper also constructed a higher-quality version of the CT image dataset containing 247 cases screened by professional physicians, and obtained more excellent results on this dataset. Meanwhile, we used the other two COVID-19 datasets as external verifications, and still achieved a high accuracy rate without additional training. Finally, the experimental results show that our classification accuracy can reach 98.31%, and the precision, sensitivity, specificity, and AUC (Area Under Curve) are 98.82%, 97.99%, 98.67%, and 0.989, respectively. The accuracy of external verification can reach 93.34% and 91.05%. What's more, the accuracy of our prediction framework is 91.54%. A large number of experiments demonstrate that our proposed method is effective and robust for COVID-19 detection and prediction.

Structure-Preserving Random Noise Attenuation Method for Seismic Data Based on a Flexible Attention CNN

The noise attenuation of seismic data is an indispensable part of seismic data processing, directly impacting the following inversion and imaging. This paper focuses on two bottlenecks in the AI-based denoising method of seismic data: the destruction of structural information of seismic data and the inferior generalizability. We propose a flexible attention-CNN (FACNN) and realized the denoising work of seismic data. This paper’s main work and advantages were concentrated on the following three aspects: (i) We propose attention gates (AGs), which progressively suppressed features in irrelevant background parts and improved the denoising performance. (ii) We added a noise level map M as an additional channel, making a single CNN model expected to inherit the flexibility of handling noise models with different parameters, even spatially variant noises. (iii) We propose a mixed loss function based on MS_SSIM to improve the performance of FACNN further. Adding the noise level map can improve the network’s generalization ability, and adding the attention structure with the mixed loss function can better protect the structural information of the seismic data. The numerical tests showed that our method has better generalization and can better protect the details of seismic events.

Generative adversarial networks-based image steganography with multiscale features integration

As an effective method of information hiding, steganography embeds secret information into images in a way that is not perceived by humans. Recent interest in the combination of image steganography and generative adversarial networks (GANs) has yielded rapid progress. However, existing steganography frameworks still suffer from the low quality of the steganographic images and weak resistance to detection by steganalysis algorithms. To overcome these limitations, we propose an effective GAN-based image steganography framework with multiscale features integration. Specifically, we construct the secret image feature extraction network (SfeNet), which is driven by the spatial attention mechanism to extract multiscale features of secret images. And the encoder combined with the efficient channel attention mechanism is presented to embed multiscale features of secret images into the cover image. Subsequently, a steganalyzer is incorporated as the discriminator with the encoder in GAN to strengthen the ability of the model to resist steganalysis. Besides, a mixed loss function is proposed by combining perceptual loss, MS-SSIM, and L1 loss to preserve the structural similarity of images. Experimental results on ImageNet, Pascal VOC2012, and LFW show that the proposed method achieves better quality steganographic images and better resistance to steganalysis compared to some of the state-of-the-art steganography algorithms.

Spatial-Temporal Based Multihead Self-Attention for Remote Sensing Image Change Detection

The neural network-based remote sensing image change detection method faces a large amount of imaging interference and severe class imbalance problems under high-resolution conditions, which bring new challenges to the accuracy of the detection network. In this work, to address the imaging interference caused by different imaging angles and times, the siamese strategy and multi-head self-attention mechanism are used to reduce the imaging differences between the dual-temporal images and fully exploit the inter-temporal information. Secondly, a learnable multi-part feature learning module is used to adaptively exploit features from different scales to obtain more comprehensive features. Finally, a mixed loss function strategy is used to ensure that the network converges effectively and excludes the adverse interference of a large number of negative samples to the network. Extensive experiments show that our method outperforms numerous methods on LEVIR-CD, WHU, and DSIFN datasets.

A Novel RDA-Based Network to Conceal Image Data and Prevent Information Leakage

Image data play an important role in our daily lives, and scholars have recently leveraged deep learning to design steganography networks to conceal and protect image data. However, the complexity of computation and the running speed have been neglected in their model designs, and steganography security still has much room for improvement. For this purpose, this paper proposes an RDA-based network, which can achieve higher security with lower computation complexity and faster running speed. To improve the hidden image’s quality and ensure that the hidden image and cover image are as similar as possible, a residual dense attention (RDA) module was designed to extract significant information from the cover image, thus assisting in reconstructing the salient target of the hidden image. In addition, we propose an activation removal strategy (ARS) to avoid undermining the fidelity of low-level features and to preserve more of the raw information from the input cover image and the secret image, which significantly boosts the concealing and revealing performance. Furthermore, to enable comprehensive supervision for the concealing and revealing processes, a mixed loss function was designed, which effectively improved the hidden image’s visual quality and enhanced the imperceptibility of secret content. Extensive experiments were conducted to verify the effectiveness and superiority of the proposed approach.

CA-BASNet: A Building Extraction Network in High Spatial Resolution Remote Sensing Images

Aiming at the problems of holes, misclassification, and rough edge segmentation in building extraction results from high spatial remote sensing images, a coordinate attention mechanism fusion network based on the BASNet network (CA-BASNet) is designed for building extraction in high spatial remote sensing images. Firstly, the deeply supervised encoder–decoder network was used to create a rough extract of buildings; secondly, to make the network pay more attention to learning building edge features, the mixed loss function composed of binary cross entropy, structural similarity and intersection-over-union was introduced into the network training process; finally, the residual optimization module of fusion coordinate attention mechanism was used for post-processing to realize the fine extraction of buildings from high spatial resolution remote sensing images. Experiments on the WHU building dataset show that the proposed network can achieve mIoU of 93.43%, mPA of 95.86%, recall of 98.79%, precision of 90.13% and F1 of 91.35%. Compared with the existing semantic segmentation networks, such as PSPNet, SegNet, DeepLapV3, SE-UNet, and UNet++, the accuracy of the proposed network and the integrity of object edge segmentation are significantly improved, which proves the effectiveness of the proposed network.

A miniature U-net for k-space-based parallel magnetic resonance imaging reconstruction with a mixed loss function.

BackgroundDeep learning-based magnetic resonance imaging (MRI) methods require in most cases a separate dataset with thousands of images for each anatomical site to train the network model. This paper proposes a miniature U-net method for k-space-based parallel MRI where the network model is trained individually for each scan using scan-specific autocalibrating signal data.MethodsThe original U-net was tailored with fewer layers and channels, and the network was trained using the autocalibrating signal data with a mixing loss function involving magnitude loss and phase loss. The performance of the proposed method was measured using both phantom and in vivo datasets compared to scan-specific robust artificial-neural-networks for k-space interpolation (RAKI) and generalized autocalibrating partially parallel acquisitions (GRAPPA).ResultsThe proposed method alleviates aliasing artifacts and reduces noise with an acceleration factor of four for phantom and in vivo data. Compared with RAKI and GRAPPA, the proposed method represents an improvement with a structural similarity index measure of between 0.02 and 0.05 and a peak signal-to-noise ratio (PSNR) of between 0.1 and 3.ConclusionsThe proposed method introduces a miniature U-net to reconstruct the missing k-space data, which can provide an optimal trade-off between network performance and requirement of training samples. Experimental results indicate that the proposed method can improve image quality compared with the deep learning-based k-space parallel magnitude resonance imaging method.

Deep Learning-Powered Vessel Trajectory Prediction for Improving Smart Traffic Services in Maritime Internet of Things

The maritime Internet of Things (IoT) has recently emerged as a revolutionary communication paradigm where a large number of moving vessels are closely interconnected in intelligent maritime networks. However, the tremendous growth of vessel trajectories, collected from the combined satellite-terrestrial AIS (automatic identification system) base stations, could lead to unsatisfactory maritime safety and efficacy. To promote smart traffic services in maritime IoT, it is necessary to accurately and robustly predict the spatiotemporal vessel trajectories. It is beneficial for collision avoidance, maritime surveillance, and abnormal behavior detection, etc. Motivated by the strong learning capacity of deep neural networks, this work proposes an AIS data-driven trajectory prediction framework, whose main component is a long short term memory (LSTM) network. In particular, the vessel traffic conflict situation modeling, generated using the dynamic AIS data and social force concept, is embedded into the LSTM network to guarantee high-accuracy vessel trajectory prediction. In addition, a mixed loss function is reconstructed to make our prediction results more reliable and robust in different navigation environments. Several quantitative and qualitative experiments have been implemented on realistic AIS-based vessel trajectories. Our results have demonstrated that the proposed method could achieve satisfactory prediction performance in terms of accuracy and robustness.

Multilevel Strong Auxiliary Network for Enhancing Feature Representation to Protect Secret Images

Image data play an important role in the network information, however, some images containing sensitive or confidential information are easy to attract the attention of malicious attackers. Based on deep learning and data hiding technology, a novel hiding-revealing network is designed to protect these secret images. The sender uses the hiding network to conceal the secret image into an ordinary cover image and the receiver uses the revealing network to recover the secret image. Symmetrical shortcut connection is designed to improve both the hiding and the revealing performance without adding any parameters. Consider that some secret images may have complex spatial features, a multi-level strong auxiliary module is designed to enhance feature representation and boost the restoration quality of the secret image. Then, a lifeline is proposed to transform the image hiding task into a residual identity mapping, which reduces the difficulty of network learning and obviously improves the hiding performance. In addition, a mixed loss function is designed to further improve the perceptual quality of both the hidden image and the revealed image, which further completely eliminates the secret content in the residual image and ensures the hiding security. Experimental results demonstrate that, compared with the state-of-the-art methods, our proposed method achieves the best performance in both hidden image synthesis and secret image restoration.

TA-Unet: Integrating Triplet Attention Module for Drivable Road Region Segmentation.

Road segmentation has been one of the leading research areas in the realm of autonomous driving cars due to the possible benefits autonomous vehicles can offer. Significant reduction of crashes, greater independence for the people with disabilities, and reduced traffic congestion on the roads are some of the vivid examples of them. Considering the importance of self-driving cars, it is vital to develop models that can accurately segment drivable regions of roads. The recent advances in the area of deep learning have presented effective methods and techniques to tackle road segmentation tasks effectively. However, the results of most of them are not satisfactory for implementing them into practice. To tackle this issue, in this paper, we propose a novel model, dubbed as TA-Unet, that is able to produce quality drivable road region segmentation maps. The proposed model incorporates a triplet attention module into the encoding stage of the U-Net network to compute attention weights through the triplet branch structure. Additionally, to overcome the class-imbalance problem, we experiment on different loss functions, and confirm that using a mixed loss function leads to a boost in performance. To validate the performance and efficiency of the proposed method, we adopt the publicly available UAS dataset, and compare its results to the framework of the dataset and also to four state-of-the-art segmentation models. Extensive experiments demonstrate that the proposed TA-Unet outperforms baseline methods both in terms of pixel accuracy and mIoU, with 98.74% and 97.41%, respectively. Finally, the proposed method yields clearer segmentation maps on different sample sets compared to other baseline methods.

DCS-TransUperNet: Road Segmentation Network Based on CSwin Transformer with Dual Resolution

Recent advances in deep learning have shown remarkable performance in road segmentation from remotely sensed images. However, these methods based on convolutional neural networks (CNNs) cannot obtain long-range dependency and global contextual information because of the intrinsic inductive biases. Motivated by the success of Transformer in computer vision (CV), excellent models based on Transformer are emerging endlessly. However, patches with a fixed scale limit the further improvement of the model performance. To address this problem, a dual-resolution road segmentation network (DCS-TransUperNet) with a features fusion module (FFM) was proposed for road segmentation. Firstly, the encoder of DCS-TransUperNet was designed based on CSwin Transformer, which uses dual subnetwork encoders of different scales to obtain the coarse and fine-grained feature representations. Secondly, a new FFM was constructed to build enhanced feature representation with global dependencies, using different scale features from the subnetwork encoders. Thirdly, a mixed loss function was designed to avoid the local optimum caused by the imbalance between road and background pixels. Experiments using the Massachusetts dataset and DeepGlobe dataset showed that the proposed DCS-TransUperNet could effectively solve the discontinuity problem and preserve the integrity of the road segmentation results, achieving a higher IoU (65.36% on Massachusetts dataset and 56.74% on DeepGlobe) of road segmentation compared to other state-of-the-art methods. The considerable performance also proves the powerful generation ability of our method.

Multi-Resolution Context Aggregation Network for Single Image Rain Removal

Severe weather such as rain will cause serious degradation of image quality and affect the accuracy of computer vision algorithms. In order to better extract the features of multi-scale rain streaks and restore the important detailed feature information of the image, a new single image rain removal based on multi-resolution context aggregation network is proposed. Firstly, the shuffling operation is used to convert a single resolution input image into a multi-spatial resolution input, which enables the network to rapidly expand the acceptance field at the low spatial resolution and extract more refined rain streak features at high spatial resolution. The low-resolution rain streak features are aggregated from top to bottom into high spatial resolution, and the network is guided to extract multi-scale rain streak information. Secondly, the multi-resolution feature enhancement block is used to refine the details of the image at different resolutions to prevent the loss or blurring of the details in the rain-removed image. This method uses local residual densely connect blocks and squeeze-excitation network to enhance the ability and efficiency of the network to extract features. Finally, the mixed loss function constructed in this paper is used to obtain higher evaluation index values while improving the human visual perception of rain removal images. The experimental results show that proposed method achieves significant rain removal effects on the Rain100H, Rain100L, and Rain12 synthetic datasets and real rain datasets. Compared with the existing algorithms, the proposed method can achieve significant improvement in qualitative and quantitative indicators and has a high degree of detail retention.

Crack Detection Method of Sleeper Based on Cascade Convolutional Neural Network

This work presents a new method for sleeper crack identification based on cascade convolutional neural network (CNN) to address the problem of low efficiency and poor accuracy in the traditional detection method of sleeper crack identification. The proposed algorithm mainly includes improved You Only Look Once version 3 (YOLOv3) and the crack recognition network, where the crack recognition network includes two modules, the crack encoder-decoder network (CEDNet) and the crack residual refinement network (CRRNet). The improved YOLOv3 network is used to identify and locate cracks on sleepers and segment them after the sleeper on the ballast bed is extracted by using the gray projection method. The sleeper is inputted into CEDNet for crack feature extraction to predict the coarse crack saliency map. The prediction graph is inputted into CRRNet to improve its edge information and local region to achieve optimization. The accuracy of the crack identification model is improved by using a mixed loss function of binary cross-entropy (BCE), structural similarity index measure (SSIM), and intersection over union (IOU). Results show that this method can accurately detect the sleeper crack image. During object detection, the proposed method is compared with YOLOv3 in terms of directly locating sleeper cracks. It has an accuracy of 96.3%, a recall rate of 91.2%, a mean average precision (mAP) of 91.5%, and frames per second (FPS) of 76.6/s. In the crack extraction part, the F-weighted is 0.831, mean absolute error (MAE) is 0.0157, and area under the curve (AUC) is 0.9453. The proposed method has better recognition, higher efficiency, and robustness compared with the other network models.

MIEGAN: Mobile Image Enhancement via a Multi-Module Cascade Neural Network

Visual quality of images captured by mobile devices is often inferior to that of images captured by a Digital Single Lens Reflex (DSLR) camera. This paper presents a novel generative adversarial network-based mobile image enhancement method, referred to as MIEGAN. It consists of a novel multi-module cascade generative network and a novel adaptive multi-scale discriminative network. The multi-module cascade generative network is built upon a two-stream encoder, a feature transformer, and a decoder. In the two-stream encoder, a luminance-regularizing stream is proposed to help the network focus on low-light areas. In the feature transformation module, two networks effectively capture both global and local information of an image. To further assist the generative network to generate the high visual quality images, a multi-scale discriminator is used instead of a regular single discriminator to distinguish whether an image is fake or real globally and locally. To balance the global and local discriminators, an adaptive weight allocation is proposed. In addition, a contrast loss is proposed, and a new mixed loss function is developed to improve the visual quality of the enhanced images. Extensive experiments on the popular DSLR photo enhancement dataset and MIT-FiveK dataset have verified the effectiveness of the proposed MIEGAN.