Hybrid Loss Function Research Articles

Automatic segmentation of spine x-ray images based on multiscale feature enhancement network.

Automatic segmentation of vertebrae in spinal x-ray images is crucial for clinical diagnosis, case analysis, and surgical planning of spinal lesions. However, due to the inherent characteristics of x-ray images, including low contrast, high noise, and uneven grey scale, it remains a critical and challenging problem in computer-aided spine image analysis and disease diagnosis applications. In this paper, a Multiscale Feature Enhancement Network (MFENet), is proposed for segmenting whole spinal x-ray images, to aid doctors in diagnosing spinal-related diseases. To enhance feature extraction, the network incorporates a Dual-branch Feature Extraction Module (DFEM) and a Semantic Aggregation Module (SAM). The DFEM has a parallel dual-branch structure. The upper branch utilizes multiscale convolutional kernels to extract features from images. Employing convolutional kernels of different sizes helps capture details and structural information at different scales. The lower branch incorporates attention mechanisms to further optimize feature representation. By modeling the feature maps spatially and across channels, the network becomes more focused on key feature regions and suppresses task-irrelevant information. The SAM leverages contextual semantic information to compensate for details lost during pooling and convolution operations. It integrates high-level feature information from different scales to reduce segmentation result discontinuity. In addition, a hybrid loss function is employed to enhance the network's feature extraction capability. In this study, we conducted a multitude of experiments utilizing dataset provided by the Spine Surgery Department of Henan Provincial People's Hospital. The experimental results indicate that our proposed MFENet demonstrates superior segmentation performance in spinal segmentation on x-ray images compared to other advanced methods, achieving 92.61±0.431 for MIoU, 92.42±0.329 for DSC, and 99.51±0.037 for Global_accuracy. Our model is able to more effectively learn and extract global contextual semantic information, significantly improving spinal segmentation performance, further aiding doctors in analyzing patient conditions.

VascuConNet: an enhanced connectivity network for vascular segmentation.

Medical image segmentation commonly involves diverse tissue types and structures, including tasks such as blood vessel segmentation and nerve fiber bundle segmentation. Enhancing the continuity of segmentation outcomes represents a pivotal challenge in medical image segmentation, driven by the demands of clinical applications, focusing on disease localization and quantification. In this study, a novel segmentation model is specifically designed for retinal vessel segmentation, leveraging vessel orientation information, boundary constraints, and continuity constraints to improve segmentation accuracy. To achieve this, we cascade U-Net with a long-short-term memory network (LSTM). U-Net is characterized by a small number of parameters and high segmentation efficiency, while LSTM offers a parameter-sharing capability. Additionally, we introduce an orientation information enhancement module inserted into the model's bottom layer to obtain feature maps containing orientation information through an orientation convolution operator. Furthermore, we design a new hybrid loss function that consists of connectivity loss, boundary loss, and cross-entropy loss. Experimental results demonstrate that the model achieves excellent segmentation outcomes across three widely recognized retinal vessel segmentation datasets, CHASE_DB1, DRIVE, and ARIA.

A transfer learning approach for data-driven localization of damage areas in plate-like structures of CFRP materials

The application of machine learning techniques based on Lamb waves for structural health monitoring of composite material damage has been widely adopted. However, this method often struggles to acquire sufficient labeled historical Lamb wave signals for training machine learning models. This issue can be addressed through the implementation of transfer learning. This paper proposes a data-driven transfer learning model for locating coordinates of damage areas in plate-like structures with complex geometric features such as rivets and grooves. Based on the differences in conditional distributions in various structural health monitoring scenarios, a Cross-Component Regression for Damage Localization Model (CRDM) is introduced for damage localization across components. Due to the limited availability of labeled target data, CRDM aims to preserve global properties of the target data-dominated conditional distribution, besides considering the predictive accuracy of individual samples. Therefore, a hybrid loss function is constructed by combining Mean Squared Error (MSE) and Conditional Embedding Operator Difference (CEOD). Subsequently, the target model is fine-tuned based on this loss function. Experimental validations are conducted on finite element models and real datasets, confirming the effectiveness of the CRDM model in damage localization for plate-like structures.

MADR-Net: multi-level attention dilated residual neural network for segmentation of medical images

Medical image segmentation has made a significant contribution towards delivering affordable healthcare by facilitating the automatic identification of anatomical structures and other regions of interest. Although convolution neural networks have become prominent in the field of medical image segmentation, they suffer from certain limitations. In this study, we present a reliable framework for producing performant outcomes for the segmentation of pathological structures of 2D medical images. Our framework consists of a novel deep learning architecture, called deep multi-level attention dilated residual neural network (MADR-Net), designed to improve the performance of medical image segmentation. MADR-Net uses a U-Net encoder/decoder backbone in combination with multi-level residual blocks and atrous pyramid scene parsing pooling. To improve the segmentation results, channel-spatial attention blocks were added in the skip connection to capture both the global and local features and superseded the bottleneck layer with an ASPP block. Furthermore, we introduce a hybrid loss function that has an excellent convergence property and enhances the performance of the medical image segmentation task. We extensively validated the proposed MADR-Net on four typical yet challenging medical image segmentation tasks: (1) Left ventricle, left atrium, and myocardial wall segmentation from Echocardiogram images in the CAMUS dataset, (2) Skin cancer segmentation from dermoscopy images in ISIC 2017 dataset, (3) Electron microscopy in FIB-SEM dataset, and (4) Fluid attenuated inversion recovery abnormality from MR images in LGG segmentation dataset. The proposed algorithm yielded significant results when compared to state-of-the-art architectures such as U-Net, Residual U-Net, and Attention U-Net. The proposed MADR-Net consistently outperformed the classical U-Net by 5.43%, 3.43%, and 3.92% relative improvement in terms of dice coefficient, respectively, for electron microscopy, dermoscopy, and MRI. The experimental results demonstrate superior performance on single and multi-class datasets and that the proposed MADR-Net can be utilized as a baseline for the assessment of cross-dataset and segmentation tasks.

Enhancing Road Crack Localization for Sustainable Road Safety Using HCTNet

Road crack detection is crucial for maintaining and inspecting civil infrastructure, as cracks can pose a potential risk for sustainable road safety. Traditional methods for pavement crack detection are labour-intensive and time-consuming. In recent years, computer vision approaches have shown encouraging results in automating crack localization. However, the classical convolutional neural network (CNN)-based approach lacks global attention to the spatial features. To improve the crack localization in the road, we designed a vision transformer (ViT) and convolutional neural networks (CNNs)-based encoder and decoder. In addition, a gated-attention module in the decoder is designed to focus on the upsampling process. Furthermore, we proposed a hybrid loss function using binary cross-entropy and Dice loss to evaluate the model’s effectiveness. Our method achieved a recall, F1-score, and IoU of 98.54%, 98.07%, and 98.72% and 98.27%, 98.69%, and 98.76% on the Crack500 and Crack datasets, respectively. Meanwhile, on the proposed dataset, these figures were 96.89%, 97.20%, and 97.36%.

Motion Correction for Brain MRI Using Deep Learning and a Novel Hybrid Loss Function

Purpose: Motion-induced magnetic resonance imaging (MRI) artifacts can deteriorate image quality and reduce diagnostic accuracy, but motion by human subjects is inevitable and can even be caused by involuntary physiological movements. Deep-learning-based motion correction methods might provide a solution. However, most studies have been based on directly applying existing models, and the trained models are rarely accessible. Therefore, we aim to develop and evaluate a deep-learning-based method (Motion Correction-Net, or MC-Net) for suppressing motion artifacts in brain MRI scans. Methods: A total of 57 subjects, providing 20,889 slices in four datasets, were used. Furthermore, 3T 3D sagittal magnetization-prepared rapid gradient-echo (MP-RAGE) and 2D axial fluid-attenuated inversion-recovery (FLAIR) sequences were acquired. The MC-Net was derived from a UNet combined with a two-stage multi-loss function. T1-weighted axial brain images contaminated with synthetic motions were used to train the network to remove motion artifacts. Evaluation used simulated T1- and T2-weighted axial, coronal, and sagittal images unseen during training, as well as T1-weighted images with motion artifacts from real scans. The performance indices included the peak-signal-to-noise ratio (PSNR), the structural similarity index measure (SSIM), and visual reading scores from three blinded clinical readers. A one-sided Wilcoxon signed-rank test was used to compare reader scores, with p < 0.05 considered significant. Intraclass correlation coefficients (ICCs) were calculated for inter-rater evaluations. Results: The MC-Net outperformed other methods in terms of PSNR and SSIM for the T1 axial test set. The MC-Net significantly improved the quality of all T1-weighted images for all directions (i.e., the mean SSIM of axial, sagittal, and coronal slices improved from 0.77, 0.64, and 0.71 to 0.92, 0.75, and 0.84; the mean PSNR improved from 26.35, 24.03, and 24.55 to 29.72, 24.40, and 25.37, respectively) and for simulated as well as real motion artifacts, both using quantitative measures and visual scores. However, MC-Net performed poorly for images with untrained T2-weighted contrast because the T2 contrast was unseen during training and is different from T1 contrast. Conclusion: The proposed two-stage multi-loss MC-Net can effectively suppress motion artifacts in brain MRI without compromising image quality. Given the efficiency of MC-Net (with a single-image processing time of ~40 ms), it can potentially be used in clinical settings.

AutoDLAR: A Semi-supervised Cross-modal Contact-free Human Activity Recognition System

WiFi-based human activity recognition (HAR) plays an essential role in various applications such as security surveillance, health monitoring, and smart home. Existing HAR methods, though yielding promising performance in indoor scenarios, highly depend on a massive labeled dataset for training which is extremely difficult to acquire in practical applications. In this paper, we present an automatic data labeling and HAR system, termed AutoDLAR. Taking a semi-supervised cross-modal learning framework with a hybrid loss function as the core, AutoDLAR transfers rich visual information to automatically label WiFi signals for WiFi-based HAR. Specifically, we devise a lightweight and multi-view WiFi sensing model with a parallel feature embedding method to accurately identify activities and accelerate recognition speed. Then, we exploit the video data to fine-tune a well-established visual HAR model, generating effective pseudo-labels for guiding the WiFi model’s training. We also build a synchronized Video-WiFi dataset with seven types of human activities under different scenarios to enable training and validating the semi-supervised HAR system. Extensive experiments on our collected activity dataset and the emotion recognition benchmark demonstrate that AutoDLAR attains an average accuracy of over 95.89% without manual labeling and only spends the inference time of 3.35 ms, outperforming the state-of-the-art (SOTA) methods.

WBNet: Weakly-supervised salient object detection via scribble and pseudo-background priors

Weakly supervised salient object detection (WSOD) methods endeavor to boost sparse labels to get more salient cues in various ways. Among them, an effective approach is using pseudo labels from multiple unsupervised self-learning methods, but inaccurate and inconsistent pseudo labels could ultimately lead to detection performance degradation. To tackle this problem, we develop a new multi-source WSOD framework, WBNet, that can effectively utilize pseudo-background (non-salient region) labels combined with scribble labels to obtain more accurate salient features. We first design a comprehensive salient pseudo-mask generator from multiple self-learning features. Then, we pioneer the exploration of generating salient pseudo-labels via point-prompted and box-prompted Segment Anything Models (SAM). Then, WBNet leverages a pixel-level Feature Aggregation Module (FAM), a mask-level Transformer-decoder (TFD), and an auxiliary Boundary Prediction Module (EPM) with a hybrid loss function to handle complex saliency detection tasks. Comprehensively evaluated with state-of-the-art methods on five widely used datasets, the proposed method significantly improves saliency detection performance. The code and results are publicly available at https://github.com/yiwangtz/WBNet.

Efficiently reconstructing high-quality details of 3D digital rocks with super-resolution Transformer

Accurate pore-scale modeling demands high-quality digital rock images, which should possess a broad imaging field of view (FOV) and high resolution to characterize multi-scale rock components. However, achieving both conditions simultaneously is challenging due to hardware constraints. Super-resolution techniques can mitigate this issue by reconstructing high-resolution details from low-resolution images captured with a wide FOV. To reconstruct high-quality 3D digital rocks, we propose a novel Efficient Attention Super-Resolution Transformer (EAST) model. It integrates self-attention and channel attention mechanisms and undergoes structural optimization. Evaluation demonstrates that EAST achieves superior reconstruction quality with a 1.85 × speedup while reducing parameters by 78 % over the advanced model. Additionally, to tailor to the characteristics of digital rocks and the constrained dataset, we employ a hybrid loss function along with two data augmentation techniques. Visualizations reveal that EAST can resist noise and blur interference, highlighting valuable features such as pore edges and textures. Ultimately, we introduce an approach based on self-supervised fine-tuning to enhance the model robustness. Direct flow simulation verifies that the reconstructed results closely align with high-resolution images in terms of physical accuracy. EAST reduces the relative error of the absolute permeability by 18.5 % and 33 % over the Tricubic method on two external samples.

A novel defect prediction method based on semantic feature enhancement

SummaryAlthough cross‐project defect prediction (CPDP) techniques that use traditional manual features to build defect prediction model have been well‐developed, they usually ignore the semantic and structural information inside the program and fail to capture the hidden features that are critical for program category prediction, resulting in poor defect prediction results. Researchers have proposed using deep learning to automatically extract the semantic features of programs and fuse them with traditional features as training data. However, in practice, it is important to explore the effective representation of the semantic features in the programs and how the fusion of a reasonable ratio between the two types of features can maximize the effectiveness of the model. In this paper, we propose a semantic feature enhancement‐based defect prediction framework (SFE‐DP), which augments the semantic feature set extracted from the program code with data. We also introduce a layer of self‐attentive mechanism and a matching layer to filter low‐efficiency and non‐critical semantic features in the model structure. Finally, we combine the idea of hybrid loss function to iteratively optimize the model parameters. Extensive experiments validate that SFE‐DP can outperform the baseline approaches on 90 pairs of CPDP tasks formed by 10 open‐source projects.

Deep improved bidirectional long short‐term memory‐based antenna selection for multiple input multiple output software defined radio system

SummaryMultiple input multiple output (MIMO) constitutes a wireless approach that employs several senders and receivers in order to send numerous data at once. Due to the rapid increase of cellular mobile device usage and the limitation of computing power, antenna selection (AS) has attracted more and more attention recently. AS can keep a balance between communication performance and computational complexity. In this paper, a novel deep learning (DL)‐based AS for MIMO software defined radio (SDR) system is proposed. The main objective of this work is to select the optimal antennas in MIMO systems to balance the efficiency of communication and maximize the secrecy capacity in massive MIMOME channels. First, we set up a DL‐based AS‐aided MIMO SDR environment that ensures intelligent decisions for AS. Accordingly, the hybrid loss function induced in bidirectional long short term memory (HLFI‐Bi‐LSTM) model is proposed to build the decision server, in which it takes some estimated features as input. The features such as channel state information, received signal strength indicator (RSSI), signal‐to‐noise ratio (SNR), and antenna gain are considered accordingly as the input to HLFI‐Bi‐LSTM. The HLFI‐Bi‐LSTM model is the extended version of Bi‐LSTM, which is improved by inducing the hybrid loss function, which is the combination of two functions namely, combo loss and inverse mean square error. The proposed HLFI‐Bi‐LSTM attained the greater accuracy value of 0.968 at 80% for optimal selection of the antenna.

A robust computational framework for variational data assimilation of mean flows with sparse measurements corrupted by strong outliers

A variational framework for the reconstruction of time-averaged mean flows using a sparse set of observations with large magnitudes of noise (referred to as outliers), is presented. The observations constitute a set of point-wise measurements of the mean flow with outliers at certain measurement locations and are incorporated into a numerical simulation governed by the two-dimensional, incompressible Reynolds-averaged Navier-Stokes (RANS) equations with an unknown momentum forcing. This forcing, which corresponds to the divergence of the Reynolds stress tensor, is calculated from a direct-adjoint optimization procedure to reduce the deviation between the measured and estimated mean velocities. ℓ2, ℓ1, Huber, and hybrid loss functions are used to represent the discrepancy in the mean velocity field between the measurements and the predictions. A variety of algorithms are considered to solve the optimization problem with these loss functions and a performance comparison in terms of the quality and physical features of the recovered mean flow is presented. The Huber loss function performed best as it remained robust to strong outliers in the measurements with its ℓ1 contribution and also ensured the uniqueness of the optimal solution with its ℓ2 contribution. Huber loss functions restrict the effect of outliers at the local measurement locations, thereby not affecting the quality of the high-dimensional reconstructed mean flow field. The hybrid loss function, a modified form of the continuous Huber loss function, also recovered the mean flow with high accuracy. We demonstrate the performance of the data assimilation framework for the case of two-dimensional laminar flow around a circular cylinder at Re=100. We then extend the analysis to the case of two-dimensional laminar flow over a backward-facing step at Re=500, to further assess the efficacy and robustness of the data assimilation framework.

PSCAT: a lightweight transformer for simultaneous denoising and super-resolution of OCT images

Optical coherence tomography (OCT), owing to its non-invasive nature, has demonstrated tremendous potential in clinical practice and has become a prevalent diagnostic method. Nevertheless, the inherent speckle noise and low sampling rate in OCT imaging often limit the quality of OCT images. In this paper, we propose a lightweight Transformer to efficiently reconstruct high-quality images from noisy and low-resolution OCT images acquired by short scans. Our method, PSCAT, parallelly employs spatial window self-attention and channel attention in the Transformer block to aggregate features from both spatial and channel dimensions. It explores the potential of the Transformer in denoising and super-resolution for OCT, reducing computational costs and enhancing the speed of image processing. To effectively assist in restoring high-frequency details, we introduce a hybrid loss function in both spatial and frequency domains. Extensive experiments demonstrate that our PSCAT has fewer network parameters and lower computational costs compared to state-of-the-art methods while delivering a competitive performance both qualitatively and quantitatively.

A 3D multi-scale CycleGAN framework for generating synthetic PETs from MRIs for Alzheimer's disease diagnosis

This paper proposes a novel framework for generating synthesized PET images from MRIs to fill in missing PETs and help with Alzheimer's disease (AD) diagnosis. This framework employs a 3D multi-scale image-to-image CycleGAN architecture for the end-to-end translation of MRI and PET domains together. A hybrid loss function is also proposed to enforce structural similarity while preserving voxel-wise similarity and avoiding blurry images. As shown by the quantitative and visual assessment of the synthesized PETs, this framework is superior to the state-of-the-art. Moreover, using these synthesized PETs helps improve the ternary classification of AD subjects (AD vs. MCI vs. NC). Specifically, assuming an extreme case where none of the subjects has a PET, feeding the classifier with MRIs and their corresponding synthetic PETs results in a more accurate diagnosis than feeding it with just available MRIs. Accordingly, the proposed framework can help improve AD diagnosis, which is the final goal of the current study. Ablation investigation of the proposed multi-scale framework as well as the proposed loss function, is also conducted to study their contribution to the quality of synthesized PETs. Furthermore, other factors, such as stopping criteria, the type of normalization layer, the activation function, and dropouts, are examined, concluding that the appropriate use of these factors can significantly improve the quality of synthesized PETs.

S-Net: an S-shaped network for nodule detection in 3D CT images

Objective. Accurate and automatic detection of pulmonary nodules is critical for early lung cancer diagnosis, and promising progress has been achieved in developing effective deep models for nodule detection. However, most existing nodule detection methods merely focus on integrating elaborately designed feature extraction modules into the backbone of the detection network to extract rich nodule features while ignore disadvantages of the structure of detection network itself. This study aims to address these disadvantages and develop a deep learning-based algorithm for pulmonary nodule detection to improve the accuracy of early lung cancer diagnosis. Approach. In this paper, an S-shaped network called S-Net is developed with the U-shaped network as backbone, where an information fusion branch is used to propagate lower-level details and positional information critical for nodule detection to higher-level feature maps, head shared scale adaptive detection strategy is utilized to capture information from different scales for better detecting nodules with different shapes and sizes and the feature decoupling detection head is used to allow the classification and regression branches to focus on the information required for their respective tasks. A hybrid loss function is utilized to fully exploit the interplay between the classification and regression branches. Main results. The proposed S-Net network with ResSENet and other three U-shaped backbones from SANet, OSAF-YOLOv3 and MSANet (R+SC+ECA) models achieve average CPM scores of 0.914, 0.915, 0.917 and 0.923 on the LUNA16 dataset, which are significantly higher than those achieved with other existing state-of-the-art models. Significance. The experimental results demonstrate that our proposed method effectively improves nodule detection performance, which implies potential applications of the proposed method in clinical practice.

Gabor-net with multi-scale hierarchical fusion of features for fundus retinal blood vessel segmentation

This paper proposes a fundus retinal blood vessel segmentation model based on a deep convolutional network structure and biological visual feature extraction mechanism. It aims to solve the multi-scale problem of blood vessels in the fundus retinal blood vessel segmentation task in the field of medical image processing on the basis of increasing the biological interpretability of the model. First, the subject feature information of the retinal blood vessel image is obtained by using the non-subsampled Residual Bolck convolution main channel. Secondly, combined with the study of biological vision mechanisms, an information processing model of the Retina-Exogenius-Primary visual cortex (V1) ventral visual pathway was established. Gabor functions of different scales are used to simulate the structure of different levels of the visual pathway, and the scale information at different levels is integrated into the corresponding hierarchical stages of the convolutional main pathway network to enrich the information of small blood vessels and enhance the semantic information of the overall blood vessels. Finally, considering the imbalance of the ratio of vessel and nonvessel pixels, an adaptive optimization scheme using hybrid loss function weights is proposed to enhance the priority of blood vessel pixels in the calculation of the loss function. According to the experimental results on the STARE, DRIVE and CHASE_DB1 data sets, the model still achieves superior performance evaluation indicators overall compared with the existing optimal methods in the fundus retinal blood vessel segmentation task. This research is of great significance to the field of medical image processing and can provide more accurate auxiliary diagnostic information for clinical diagnosis and treatment.

Shape-aware speckle matching network for cross-domain 3D reconstruction

The learning-based binocular 3D reconstruction method outperforms traditional vision algorithms in terms of precision and efficiency. However, in some cross-domain scenes, the precision of the well-trained network drastically decreases when handling samples in diverse contexts. This paper proposes an end-to-end shape-aware speckle matching network (SSMNet) that combines shape-mask information to achieve improved precision and completeness of disparity calculation in cross-domain applications. The cascade attention mechanism is inserted in the feature extraction stage to concentrate on valuable regions. The shape-aware module is designed to learn additional shape contour information, and multiscale features are integrated simultaneously to construct the cost-volume for the subsequent lightweight 3D aggregation. In addition, instance normalization is adopted to guarantee style migration, and a hybrid loss function is used to supervise the learning process. Furthermore, a high-precision binocular speckle dataset is built, including training and testing sets in different distributions. Extensive quantitative and qualitative experiments demonstrate that SSMNet enhances cross-domain capability and achieves state-of-the-art performance. Measurement precision evaluation illustrates that the proposed method can realize the desired highly precise 3D shape measurement in a real industrial scenario.

Mm-CasGAN: A cascaded adversarial neural framework for mmWave radar point cloud enhancement

Handling and interpreting sparse 3D point clouds, especially from mmWave radar, presents unique challenges due to the inherent data sparsity and the vast domain difference compared to denser point clouds like those from LiDAR. In this paper, we introduce a novel cascaded generative adversarial network (GAN) approach to bridge this domain gap. The core principle is to progressively refine the radar-based point cloud through a series of GANs, each targeting a higher resolution. By leveraging multi-level features and a hybrid loss function that combines adversarial, geometric, and consistency components, our method ensures a smooth transition from the sparse radar representation to a high-resolution LiDAR-like point cloud. Our cascaded approach operates at a patch level, and the integrated loss function ensures that the generated points not only resemble the target domain but also maintain geometric and structural fidelity. Real-life dataset consisting mostly of moving pedestrians were collected using a system made of Radar, LiDAR, and RGB Camera. Through an extensive experiment on the collected real-world pedestrian dataset, we validate the efficacy of our approach. Inference from the network indicates that our method can upsample mmWave radar point clouds with enhanced density, uniformity, and closer alignment to the ground truth LiDAR point clouds, which is the first of its kind network to do so.

Joint Distortion Restoration and Quality Feature Learning for No-reference Image Quality Assessment

No-reference image quality assessment (NR-IQA) methods, inspired by the free energy principle, improve the accuracy of image quality prediction by simulating the human brain’s repair process for distorted images. However, existing methods use separate optimization schemes for distortion restoration and quality prediction, which undermines the accurate mapping of feature representations to quality scores. To address this issue, we propose a joint restoration and quality feature learning NR-IQA (RQFL-IQA) method to jointly tackle distortion image restoration and quality prediction within a unified framework. To accurately establish the quality reconstruction relationship between distorted and restored images, a hybrid loss function based on pixel-wise and structure-wise representations is used to improve the restoration capability of the image restoration network. The proposed RQFL-IQA exploits rich labels, including restored images and quality scores, to enable the model to learn more discriminative features and establish a more accurate mapping from feature representation to quality scores. In addition, to avoid the impact of poor restoration on quality prediction, we propose a module with a cleaning function to reweight the fusion of restored and primitive features to achieve more perceptual consistency in feature fusion. Experimental results on public IQA datasets show that the proposed RQFL-IQA is superior over existing methods.

OENet: An overexposure correction network fused with residual block and transformer

With the wide application of vision in the fields of autonomous driving and medical imaging, the demand for overexposure image correction algorithms is becoming increasingly urgent. However, existing overexposure image correction algorithms can lead to problems such as blurring, color bias, and over-enhancement of the generated images. Optimizing overexposure image quality has a significant impact on improving system performance, accuracy, and safety. In this paper, we propose an overexposure image correction network. First, we built the Detail Enhancement Module (DEM). It adopts global average pooling on each channel of the input feature map. After pooling, an activation function is used for nonlinear mapping to generate a channel attention weight vector. And it is multiplied with the original input feature map to achieve the purpose of enhancing the details of the overexposed image. Second, we construct a context-aware backbone (CAB) to extract features such as color and texture. The linear attention gating mechanism replaces the multi-head attention module in Transformer, and reduces the computational complexity in high-resolution images while maintaining performance by learning linear transformation and attention gating. Finally, we design an attention-guided feature fusion (AGFF) to fuse shallow and deep features. It computes weight vectors for shallow features through an attention module. The calculated result is converted to the same dimension as the input feature by bilinear interpolation, so as to enrich the semantic information and detailed information of the generated image. In addition to designing the network structure, we design a hybrid loss function to improve the quality of the generated image from the spatial and structural aspects, and the exposure function can correct the exposure degree of the generated image. Experiments are conducted on two public datasets and the dataset in this paper. Specifically, the PSNR and SSIM of images generated on the dataset MSEC increased by 1.3813% and 5.56%. The PSNR and SSIM of images generated on the dataset SICE increased by 1.545% and 4.64%. The proposed method can effectively generate clear and high-fidelity images.