Real Image Datasets Research Articles

A degradation-aware enhancement network with fused features for fundus images

Lots of fundus images are not gradable for clinical diagnosis and computer-aided diagnosis of ocular diseases due to poor quality. In order to restore fundus images from different kinds of degradation, a degradation-aware fundus enhancement model with fused features under different receptive fields is proposed in this paper. We obtain fused features from multiple receptive fields by combining a global path with spectral convolution and a local path with degradation attention. Degradation features and degradation labels are calculated on each image and they are applied for a flexible adaption to different degradations. Experiments on both synthetic and real image datasets demonstrate that our method corrects low-quality images effectively and has generalization ability for clinical datasets from different sources.

Underwater image enhancement through color deviation detection-guided peak flattening

Underwater images frequently exhibit color deviation and blurring. Color deviation arises from the selective spectral absorption of light, whereas blurring results from particle scattering within the water. These challenges impede the performance of high-level visual perception tasks. To improve the underwater image quality, we propose an adaptive method for underwater image enhancement that employs color deviation detection-guided peak flattening. Utilizing six features identified through statistical analysis of the latest underwater real-image datasets, we develop a color deviation detection model for underwater images, which employs ensemble learning to quantify the extent of color deviation. Subsequently, we devise the peak flattening algorithm to achieve histogram adaptive partition conversion. The conversion range is determined by the estimated color deviation value, eliminating the need for iteration. For channels with severe color deviation, we employ the weighted fusion to restore partial gray distribution. Comparisons with state-of-the-art methods demonstrate that the proposed method significantly improves contrast and color fidelity, particularly in images with severe degradation.

Image super-resolution based on improved ESRGAN and its application in camera calibration

Camera calibration is a key technique in the field of computer vision. The Zhang’s method is currently the most commonly used camera calibration method, but its accuracy is mainly dependent on the image quality, thusly constrained by camera hardware conditions. Considering that high-performance cameras are often expensive, a cost-effective method to improve the quality of calibration images is of great practical value. Without changing the existing hardware conditions, image super-resolution can be considered to enhance the quality of the calibration image. Image super-resolution (SR) is the process of reconstructing an image from low resolution (LR) to high resolution (HR). It can enhance the clarity and detail of the calibration images, which will be beneficial in improving the accuracy of camera calibration. However, there is very little research on the application of image super-resolution to camera calibration. Therefore, this study innovatively proposed a method and process for applying image super-resolution in camera calibration. Firstly, the ESRGAN (Enhanced Super-Resolution Generative Adversarial Networks) was optimized and improved, followed by the proposal of the MMRSRGAN (Multi-scale Multi-adaptive-weights Residual Super-Resolution Generative Adversarial Networks). Two methods for local image super-resolution were also proposed for efficient training and testing of datasets, and two evaluation metrics based on reprojection error were proposed evaluate the effectiveness of the proposed model. Training and testing experiments of image super-resolution for camera calibration were conducted on both virtual and real image datasets. The proposed MMRSRGAN was compared with several advanced image super-resolution networks developed in recent years across multiple dimensions and indicators. The positive results have demonstrated the feasibility of applying image super-resolution to camera calibration and showcased the good performance of the MMRSRGAN.

Gastrointestinal image stitching based on improved unsupervised algorithm.

Image stitching is a traditional but challenging computer vision task. The goal is to stitch together multiple images with overlapping areas into a single, natural-looking, high-resolution image without ghosts or seams. This article aims to increase the field of view of gastroenteroscopy and reduce the missed detection rate. To this end, an improved depth framework based on unsupervised panoramic image stitching of the gastrointestinal tract is proposed. In addition, preprocessing for aberration correction of monocular endoscope images is introduced, and a C2f module is added to the image reconstruction network to improve the network's ability to extract features. A comprehensive real image data set, GASE-Dataset, is proposed to establish an evaluation benchmark and training learning framework for unsupervised deep gastrointestinal image splicing. Experimental results show that the MSE, RMSE, PSNR, SSIM and RMSE_SW indicators are improved, while the splicing time remains within an acceptable range. Compared with traditional image stitching methods, the performance of this method is enhanced. In addition, improvements are proposed to address the problems of lack of annotated data, insufficient generalization ability and insufficient comprehensive performance in image stitching schemes based on supervised learning. These improvements provide valuable aids in gastrointestinal examination.

A proposed plant classification framework for smart agricultural applications using UAV images and artificial intelligence techniques

Utilizing Wireless Sensor Networks (WSNs), Internet of Things (IoTs) sensors, and Unmanned Aerial Vehicles (UAVs), in conjunction with optimization techniques and machine learning algorithms, can present a novel approach to Precision Agricultural (PA) crops. Agriculture has transitioned from traditional legacy systems to incorporate advanced smart technologies. Several complex farming problems are utilized to promote a variety of UAV sensing and Artificial Intelligence (AI) algorithms in PA applications of smart agriculture, especially in developing countries. This paper proposes the design of a conceptual UAV sensing system for crop management using a novel classification framework. UAV-based image datasets can be implemented and expanded to various types of crops. In addition, it can classify various types of rice species and detect weed farms as it consists of a multistage process for identifying and monitoring different crops. The proposed classification framework is evaluated using three different real UAV image datasets compared with Naive Bayes, Decision Tree (DT), Bagging, and Random Forest (RF) techniques. The classification performance metrics obtained are as follows: i) the WeedNet dataset with 100 % F1-score, 100 % recall, 100 % precision, and 100 % accuracy; ii) the Rice Seedling dataset with 99.5 % F1-score, 99.5 % recall, 99.5 % precision, and 99.5 % accuracy; and iii) the Rice Varieties dataset with 97.99 % F1-score, 97.99 % recall, 98.14 % precision, and 97.99 % accuracy. The success rates achieved by the proposed classification framework outperform those of other recent state-of-the-art techniques, demonstrating its efficacy in crop management applications.

U²PNet: An Unsupervised Underwater Image-Restoration Network Using Polarization.

This article presents U 2PNet, a novel unsupervised underwater image restoration network using polarization for improving signal-to-noise ratio and image quality in underwater imaging environments. Traditional methods for underwater image restoration using polarization require specific cues or pairs of underwater polarization datasets, which limit their practical applications. Our proposed method requires only one mosaicked polarized image of the scene and does not require datasets for pretraining or specific cues. We design two subnetworks (T-net and B textsubscript ∞ -net) to accurately estimate the transmission map and background light, and unique nonreference loss functions to ensure effective restoration. Our experiments are based on an indoor polarization simulated dataset and a real polarization image dataset constructed from our underwater robotic platform equipped with polarization cameras. Experiment results demonstrate that our proposed method achieves state-of-the-art performance on both simulated and real underwater polarization images. The code and datasets will be available at https://github.com/polwork/U-2Pnet.

SAR Image Despeckling Based on Denoising Diffusion Probabilistic Model and Swin Transformer

The speckle noise inherent in synthetic aperture radar (SAR) imaging has long posed a challenge for SAR data processing, significantly affecting image interpretation and recognition. Recently, deep learning-based SAR speckle removal algorithms have shown promising results. However, most existing algorithms rely on convolutional neural networks (CNN), which may struggle to effectively capture global image information and lead to texture loss. Besides, due to the different characteristics of optical images and synthetic aperture radar (SAR) images, the results of training with simulated SAR data may bring instability to the real-world SAR data denoising. To address these limitations, we propose an innovative approach that integrates swin transformer blocks into the prediction noise network of the denoising diffusion probabilistic model (DDPM). By harnessing DDPM’s robust generative capabilities and the Swin Transformer’s proficiency in extracting global features, our approach aims to suppress speckle while preserving image details and enhancing authenticity. Additionally, we employ a post-processing strategy known as pixel-shuffle down-sampling (PD) refinement to mitigate the adverse effects of training data and the training process, which rely on spatially uncorrelated noise, thereby improving its adaptability to real-world SAR image despeckling scenarios. We conducted experiments using both simulated SAR image datasets and real SAR image datasets, evaluating our algorithm from subjective and objective perspectives. The visual results demonstrate significant improvements in noise suppression and image detail restoration. The objective results demonstrate that our method obtains state-of-the-art performance, which outperforms the second-best method by an average peak signal-to-noise ratio (PSNR) of 0.93 dB and Structural Similarity Index (SSIM) of 0.03, affirming the effectiveness of our approach.

Novel digital-based approach for evaluating wine components’ intake: A deep learning model to determine red wine volume in a glass from single-view images

Estimation of wine components’ intake (polyphenols, alcohol, etc.) through Food Frequency Questionnaires (FFQs) may be particularly inaccurate. This paper reports the development of a deep learning (DL) method to determine red wine volume from single-view images, along with its application in a consumer study developed via a web service. The DL model demonstrated satisfactory performance not only in a daily lifelike images dataset (mean absolute error = 10 mL), but also in a real images dataset that was generated through the consumer study (mean absolute error = 26 mL). Based on the data reported by the participants in the consumer study (n = 38), average red wine volume in a glass was 114 ± 33 mL, which represents an intake of 137–342 mg of total polyphenols, 11.2 g of alcohol, 0.342 g of sugars, among other components. Therefore, the proposed method constitutes a diet-monitoring tool of substantial utility in the accurate assessment of wine components’ intake.

A Lightweight Neural Network for the Real-Time Dehazing of Tidal Flat UAV Images Using a Contrastive Learning Strategy

In the maritime environment, particularly within tidal flats, the frequent occurrence of sea fog significantly impairs the quality of images captured by unmanned aerial vehicles (UAVs). This degradation manifests as a loss of detail, diminished contrast, and altered color profiles, which directly impact the accuracy and effectiveness of the monitoring data and result in delays in the execution and response speed of monitoring tasks. Traditional physics-based dehazing algorithms have limitations in terms of detail recovery and color restoration, while neural network algorithms are limited in their real-time application on devices with constrained resources due to their model size. To address the above challenges, in the following study, an advanced dehazing algorithm specifically designed for images captured by UAVs over tidal flats is introduced. The algorithm integrates dense convolutional blocks to enhance feature propagation while significantly reducing the number of network parameters, thereby improving the timeliness of the dehazing process. Additionally, an attention mechanism is introduced to assign variable weights to individual channels and pixels, enhancing the network’s ability to perform detail processing. Furthermore, inspired by contrastive learning, the algorithm employs a hybrid loss function that combines mean squared error loss with contrastive regularization. This function plays a crucial role in enhancing the contrast and color saturation of the dehazed images. Our experimental results indicate that, compared to existing methods, the proposed algorithm has a model parameter size of only 0.005 M and a latency of 0.523 ms. When applied to the real tidal flat image dataset, the algorithm achieved a peak signal-to-noise ratio (PSNR) improvement of 2.75 and a mean squared error (MSE) reduction of 9.72. During qualitative analysis, the algorithm generated high-quality dehazing results, characterized by a natural enhancement in color saturation and contrast. These findings confirm that the algorithm performs exceptionally well in real-time fog removal from UAV-captured tidal flat images, enabling the effective and timely monitoring of these environments.

Transformer Connections: Improving Segmentation in Blurred Near‐Infrared Blood Vessel Image in Different Depth

AbstractHigh‐fidelity segmentation of blood vessels plays a pivotal role in numerous biomedical applications, such as injection assistance, cancer detection, various surgeries, and vein authentication. Near‐infrared (NIR) transillumination imaging is an effective and safe method to visualize the subcutaneous blood vessel network. However, such images are severely blurred because of the light scattering in body tissues. Inspired by the Vision Transformer model, this paper proposes a novel deep learning network known as transformer connection (TRC)‐Unet to capture global blurred and local clear correlations while using multi‐layer attention. Our method mainly consists of two blocks, thereby aiming to remap skip connection information flow and fuse different domain features. Specifically, the TRC extracts global blurred information from multiple layers and suppresses scattering to increase the clarity of vessel features. Transformer feature fusion eliminates the domain gap between the highly semantic feature maps of the convolutional neural network backbone and the adaptive self‐attention maps of TRCs. Benefiting from the long‐range dependencies of transformers, we achieved competitive results in relation to various competing methods on different data sets, including retinal vessel segmentation, simulated blur image segmentation, and real NIR blood vessel image segmentation. Moreover, our method remarkably improved the segmentation results of simulated blur image data sets and a real NIR vessel image data set. The quantitative results of ablation studies and visualizations are also reported to demonstrate the superiority of the TRC‐Unet design. © 2024 The Author(s). IEEJ Transactions on Electrical and Electronic Engineering published by Institute of Electrical Engineers of Japan and Wiley Periodicals LLC.

VIFNet: An end-to-end visible–infrared fusion network for image dehazing

Image dehazing poses significant challenges in environmental perception. Recent research mainly focus on deep learning-based methods with single modality, while they may result in severe information loss especially in dense-haze scenarios. The infrared image exhibits robustness to the haze, however, existing methods have primarily treated the infrared modality as auxiliary information, failing to fully explore its rich information in dehazing. To address this challenge, the key insight of this study is to design a visible–infrared fusion network for image dehazing. In particular, we propose a multi-scale Deep Structure Feature Extraction (DSFE) module, which incorporates the Channel-Pixel Attention Block (CPAB) to restore more spatial and marginal information within the deep structural features. Additionally, we introduce an inconsistency weighted fusion strategy to merge the two modalities by leveraging the more reliable information. To validate this, we construct a visible–infrared multimodal dataset called AirSim-VID based on the AirSim simulation platform. Extensive experiments performed on challenging real and simulated image datasets demonstrate that VIFNet can outperform many state-of-the-art competing methods. The code and dataset are available at https://github.com/mengyu212/VIFNet_dehazing.

ROSE: Multi-level super-resolution-oriented semantic embedding for 3D microvasculature segmentation from low-resolution images

Current state-of-the-art segmentation methods often require high-resolution input to attain the high performance, which pushes the limit of data acquisition and brings large computation budgets. Instead, we present an end-to-end deep learning-based method, ROSE, for robust and precise segmentation of high-quality 3D super-resolution (SR) microvasculatures from low-resolution (LR) images as input, which can transform data from the LR imaging domain to the SR semantic domain (cross different modalities and scales). More specifically, a multi-tasking two-stream deep learning framework is proposed to learn the high-fidelity microvasculature SR image and semantic hybrid features simultaneously. During the proposed joint learning process, the high-resolution features of microvasculatures are further enhanced by the learned fine-grained structural/textural features from the microvasculature SR stream with a multi-level embedding scheme through the oriented feature aggregation at different fusion stages. In the constructed joint multi-level hybrid embedding spaces, the instance semantic embedding and the SR imaging embedding can be connected and integrated synergistically. We have conducted extensive experiments using public and real patient micro-cerebrovascular image datasets and compare our framework with traditional 3D vessel segmentation methods and the other state-of-the-art in deep learning. This robust and precise microvascular visualization in different brain regions by our method demonstrates its potential impact in magnetic resonance (MR) angiography and venography for the diagnosis of microvascular disease.

Adaptive density guided network with CNN and Transformer for underwater fish counting

Accurate assessment of high-density underwater fish resources is vital to the aquaculture industry. It is directly related to the formulation of fishery insurance strategies and the implementation of breeding plans. However, accurately counting fish in high-density environments becomes challenging due to the uneven distribution of fish density and individual fish’s different sizes and postures. To break through this technical bottleneck, we developed an advanced adaptive density-guided high-density fish counting network. In detail, first of all, the network adopts a multi-layer feature fusion structure similar to UNet, which significantly enhances the matching between fish targets of different scales and feature pyramid levels, effectively alleviating the problems caused by scale changes and morphological deformations. Secondly, the network also introduces a density-guided adaptive selection module, which can intelligently judge the applicability of Convolutional Neural Network and Transformer blocks in different density areas, thereby achieving robust information transfer and interaction between blocks. Finally, to verify the effectiveness of this method, we also specially constructed two high-density data sets: a simulated high-density underwater fish image data set (SHUFD) and a real high-density underwater fish image data set (RHUFD). The proposed method has significant improvements over the state-of-the-art method (CUT) on SHUFD and RHUFD datasets, with the mean absolute error, mean square error, background region bias, foreground region bias and density map bias indicators improving by 3.44% and 6.47%, 11.43% and 4.41%, 23.91% and 29.48%, 4.43% and 10.33%, 8.3% and 13.14%, respectively.

Sparse multi-view image clustering with complete similarity information

Multi-view image clustering aims to efficiently divide the collection of images by studying the characteristics of different views. Many studies performed Laplacian dimensionality reduction on the original image to avoid noise interference in constructing the similarity matrix. However, they ignored the problem that local similarity information and isolated image information are lost due to dimensionality reduction. These problems will result in the similarity matrix being insufficient to accurately describe the similarity between images, which in turn affects the clustering accuracy. Therefore, we propose a sparse multi-view spectral clustering model with complete similarity information between images (SMSC-CSI). The model combines the original image space and the low-dimensional spectral embedding space based on the adaptive neighbors method to learn the initial similarity matrices of each view jointly. On the one hand, the original space can retain all the similarity information between images so that the initial similarity matrix can accurately describe the similarity between images, which is conducive to accurate clustering. On the other hand, the low-dimensional space can avoid noise interference and retain the main structure of high-dimensional data, improving the robustness of the initial similarity matrix. Meanwhile, to ensure consistency among the views, the model minimizes the difference between the initial similarity matrix and the central fusion matrix by alternately updating to obtain the optimal weights and central fusion matrix for each view. Finally, we can obtain ideal clustering results directly with low model complexity and without post-processing steps such as K-means by adding non-negative Laplace rank constraints and ℓ0-norm constraints to the model objective function. Experimental results on different real image datasets show that SMSC-CSI can outperform some traditional clustering models and recent models on multi-view image clustering.

Underwater image enhancement based on global features and prior distribution guided

Underwater images often suffer from substantial image blur and color distortion due to the variability of water conditions and the physical location of optical equipment, which significantly impacts the underwater intelligent system's environmental perception. Standard methods exhibit limited generalization capabilities, leading to considerable performance fluctuations when handling images with uncontrolled degradation. In this research, we leverage global features and the prior distribution of ground truth images to guide our enhancement model, introducing a novel conditional Variational Auto-Encoder-based model, named UWG-VAE, to address these challenges. UWG-VAE enhances model controllability by incorporating prior distribution information and classes of degraded styles into the decoder of the enhancement model. We assess the performance of UWG-VAE in underwater image enhancement tasks across four challenging real underwater image datasets, comparing it to state-of-the-art models. UWG-VAE demonstrates a substantial enhancement in visual quality, with notable improvements in UIQM, UCIQE, and URanker evaluation metrics when compared to existing state-of-the-art models.

A high-precision ellipse detection method based on quadrant representation and top-down fitting

Ellipse detection is a basic task in many computer-vision related problems. While widely studied in recent years, accurate and efficient detection in real-world images is still a challenge. In this paper, a novel ellipse detector, with high accuracy and efficiency, is proposed. The detector models edge by block sequences, and extracts a set of elliptical arcs, which are classified into four sets. Then top-down ellipse fitting strategy that also makes the method able to detect small and flat ellipses is designed. A two-level validation process is used to select highly probable potential ellipses, especially for fragmented ellipses. Experiments on four synthetic datasets show that the proposed method performs far better than existing methods. In images with severe cluttering and occlusion, the F-measure can still be around 0.9. On four real image datasets the proposed method achieves better F-measure scores with competitive speed than state-of-the-art techniques.

Investigating Training Datasets of Real and Synthetic Images for Outdoor Swimmer Localisation with YOLO

In this study, we developed and explored a methodical image augmentation technique for swimmer localisation in northern German outdoor lake environments. When it comes to enhancing swimmer safety, a main issue we have to deal with is the lack of real-world training data of such outdoor environments. Natural lighting changes, dynamic water textures, and barely visible swimming persons are key issues to address. We account for these difficulties by adopting an effective background removal technique with available training data. This allows us to edit swimmers into natural environment backgrounds for use in subsequent image augmentation. We created 17 training datasets with real images, synthetic images, and a mixture of both to investigate different aspects and characteristics of the proposed approach. The datasets were used to train YOLO architectures for possible future applications in real-time detection. The trained frameworks were then tested and evaluated on outdoor environment imagery acquired using a safety drone to investigate and confirm their usefulness for outdoor swimmer localisation.

Classification of Microscopic Hyperspectral Images of Blood Cells Based on Lightweight Convolutional Neural Network

Hyperspectral imaging has emerged as a novel imaging modality in the medical field, offering the ability to acquire images of biological tissues while simultaneously providing biochemical insights for in-depth tissue analysis. This approach facilitates early disease diagnosis, presenting advantages over traditional medical imaging techniques. Addressing challenges such as the computational burden of existing convolutional neural networks (CNNs) and imbalances in sample data, this paper introduces a lightweight GhostMRNet for the classification of microscopic hyperspectral images of human blood cells. The proposed model employs Ghost Modules to replace conventional convolutional layers and a cascading approach with small convolutional kernels for multiscale feature extraction, aiming to enhance feature extraction capabilities while reducing computational complexity. Additionally, an SE (Squeeze-and-Excitation) module is introduced to selectively allocate weights to features in each channel, emphasizing informative features and efficiently achieving spatial–spectral feature extraction in microscopic hyperspectral imaging. We evaluated the performance of the proposed GhostMRNet and compared it with other state-of-the-art models using two real medical hyperspectral image datasets. The experimental results demonstrate that GhostMRNet exhibits a superior performance, with an overall accuracy (OA), average accuracy (AA), and Kappa coefficient reaching 99.965%, 99.565%, and 0.9925, respectively. In conclusion, the proposed GhostMRNet achieves a superior classification performance at a smaller computational cost, thereby providing a novel approach for blood cell detection.

RSUIGM: Realistic Synthetic Underwater Image Generation with Image Formation Model

In this paper, we propose to synthesize realistic underwater images with a novel image formation model, considering both downwelling depth and line of sight (LOS) distance as cue and call it as Realistic Synthetic Underwater Image Generation Model, RSUIGM. The light interaction in the ocean is a complex process and demands specific modeling of direct and backscattering phenomenon to capture the degradations. Most of the image formation models rely on complex radiative transfer models and in-situ measurements for synthesizing and restoration of underwater images. Typical image formation models consider only line of sight distance z and ignore downwelling depth d in the estimation of effect of direct light scattering. We derive the dependencies of downwelling irradiance in direct light estimation for generation of synthetic underwater images unlike state-of-the-art image formation models. We propose to incorporate the derived downwelling irradiance in estimation of direct light scattering for modeling the image formation process and generate realistic synthetic underwater images with the proposed RSUIGM, and name it as RSUIGM dataset . We demonstrate the effectiveness of the proposed RSUIGM by using RSUIGM dataset in training deep learning based restoration methods. We compare the quality of restored images with state-of-the-art methods using benchmark real underwater image datasets and achieve improved results. In addition, we validate the distribution of realistic synthetic underwater images versus real underwater images both qualitatively and quantitatively. The proposed RSUIGM dataset is available here.

Progressive feature fusion for SNR-aware low-light image enhancement

Restoring image quality in low-light environments is an intriguing topic. While deep learning models have made significant strides in low-light enhancement, most models do not take into account the inherent characteristics of objects themselves. In this paper, we use the characteristics of the image itself to construct a Signal-to-Noise Ratio (SNR) map that guides the signal space variation to dynamically stretch the pixel values. Specifically, we propose a novel signal-to-noise ratio image-guided enhancement framework that uses the feature information of the original image to guide spatial variations in the image. It involves step-wise guidance for image feature fusion, gradually emphasizing high-frequency feature information within the image. Meanwhile, we introduced a texture optimization module that utilizes the feature information extracted by the feature fusion module to address the issues of overexposure and detail loss. We performed qualitative and quantitative evaluations on synthetic and real low-light image datasets to demonstrate the performance of our method. The experimental results show that our model outperforms other state-of-the-art methods (SOTA) in robust low-light enhancement, especially in processing images captured in complex scenes.