No-reference Image Quality Metrics Research Articles

Key-Point-Descriptor-Based Image Quality Evaluation in Photogrammetry Workflows

Photogrammetry depends critically on the quality of the images used to reconstruct accurate and detailed 3D models. Selection of high-quality images not only improves the accuracy and resolution of the resulting 3D models, but also contributes to the efficiency of the photogrammetric process by reducing data redundancy and computational demands. This study presents a novel approach to image quality evaluation tailored for photogrammetric applications that uses the key point descriptors typically encountered in image matching. Using a LightGBM ranker model, this research evaluates the effectiveness of key point descriptors such as SIFT, SURF, BRISK, ORB, KAZE, FREAK, and SuperPoint in predicting image quality. These descriptors are evaluated for their ability to indicate image quality based on the image patterns they capture. Experiments conducted on various publicly available image datasets show that descriptor-based methods outperform traditional no-reference image quality metrics such as BRISQUE, NIQE, PIQE, and BIQAA and a simple sharpness-based image quality evaluation method. The experimental results highlight the potential of using key-point-descriptor-based image quality evaluation methods to improve the photogrammetric workflow by selecting high-quality images for 3D modeling.

N-LoLiGan: Unsupervised low-light enhancement GAN with an N-Net for low-light tunnel images

Because of low luminance in tunnels, target features in images are not salient, which makes the detection of tunnel surface defects challenging. Most deep-learning-based low-light enhancement methods suffer from contrast distortion and insufficient brightness in extremely dark scenes, as well as artifacts and overexposure in tunnel scenes with large light and dark differences. To alleviate these problems, we propose a novel unsupervised generative adversarial network (GAN), called the N-shape low-light enhancement GAN (N-LoLiGan), which can be trained using unpaired low-/normal-light images and is proven to generalize sufficiently on tunnel images with complex lighting. Experimental results demonstrate that the proposed N-LoLiGan outperforms several state-of-the-art low-light image enhancement methods in terms of visual quality and two no-reference image quality metrics. Additionally, when the proposed method is used as a preprocessing step for the tunnel multi-target detection task, it yields a higher detection rate (88.9% and 97.23%) than other methods.

A Deep Learning-Based No-Reference Quality Metric for High-Definition Images Compressed With HEVC

An accurate no-reference image quality assessment metric for compression artifacts is essential for the broadcasting and streaming industries. Although we have witnessed impressive advances in the capturing, delivery and display technologies, we have not managed to match them with an accurate and perceptual based no-reference image quality metric. In this paper, we propose a unique perceptual based no-reference quality metric for compressed HD frames/images that is based on the DenseNet network architecture. We focus on the effect HEVC (High Efficiency Video Coding) compression artifacts have on the visual quality of a broadcasted and streamed video, as this is a requirement of immense importance for these industries. We chose the Video Multi-Method Assessment Fusion (VMAF) metric as our base measure to map visual quality of HEVC compression artifacts to five visual quality levels. The original VMAF classification was changed to reflect High Definition (HD) resolution images. We trained a DenseNet network to classify compressed images into five visual categories using the dataset generated by the modified VMAF. DenseNet was chosen for its ability to process HD images. Our evaluations have shown that our no-reference metric achieves an impressive average accuracy of 94.13% in classifying the visual quality of compressed images.

A method supporting fault-tolerant optical text recognition from video sequences recorded with handheld cameras

In the paper a method supporting the optical character recognition from video sequences recorded with cameras without good stabilization is proposed. Due to the presence of various distortions, such as motion blur, shadows, lossy compression artifacts, auto-focusing errors, etc., the quality of individual video frames, e.g., recorded by a smartphone camera, differs noticeably, influencing the results of text recognition, causing the presence of additional errors that may be unacceptable even in fault-tolerant applications. Although the quality of individual video frames may be assessed using state-of-the-art no-reference image quality metrics, the obtained results, particularly using the relatively fast metrics, are poorly correlated with recognition results. Therefore, a novel hybrid approach to image quality assessment supporting fault-tolerant optical character recognition is proposed, making it possible to select automatically the video frames leading to the smallest number of errors. Since the data extraction is rarely perfect, the proposed method may be particularly useful in systems where users should identify the faults defined as a too high number of text recognition errors. The proposed method makes it possible to eliminate low quality fragments of video sequences from further analysis, reducing the overall computational burden. It may also be useful in industrial OCR applications based on offline video recording with memory constraints where the use of online OCR services is not possible.

Semi-supervised atmospheric component learning in low-light image problem.

Ambient lighting conditions play a crucial role in determining the perceptual quality of images from photographic devices. In general, inadequate transmission light and undesired atmospheric conditions jointly degrade the image quality. If we know the desired ambient factors associated with the given low-light image, we can recover the enhanced image easily. Typical deep networks perform enhancement mappings without investigating the light distribution and color formulation properties. This leads to a lack of image instance-adaptive performance in practice. On the other hand, physical model-driven schemes suffer from the need for inherent decompositions and multiple objective minimizations. Moreover, the above approaches are rarely data efficient or free of postprediction tuning. Influenced by the above issues, this study presents a semisupervised training method using no-reference image quality metrics for low-light image restoration. We incorporate the classical haze distribution model to explore the physical properties of the given image to learn the effect of atmospheric components and minimize a single objective for restoration. We validate the performance of our network for six widely used low-light datasets. Experimental studies show that our proposed study achieves a competitive performance for no-reference metrics compared to current state-of-the-art methods. We also show the improved generalization performance of our proposed method which is efficient in preserving face identities in extreme low-light scenarios.

Image demosaicing: Subjective analysis and evaluation of image quality metrics

Most cameras use a single-sensor arrangement with Color Filter Array (CFA). Color interpolation techniques performed during image demosaicing are normally the reason behind visual artifacts generated in a captured image. While the severity of the artifacts depends on the demosaicing methods used, the artifacts themselves are mainly zipper artifacts (block artifacts across the edges) and false-color distortions. In this study and to evaluate the performance of demosaicing methods, a subjective pair-comparison method with 15 observers was performed on six different methods (namely Nearest Neighbours, Bilinear interpolation, Laplacian, Adaptive Laplacian, Smooth hue transition, and Gradient-Based image interpolation) and nine different scenes. The subjective scores and scene images are then collected as a dataset and used to evaluate a set of no-reference image quality metrics. Assessment of the performance of these image quality metrics in terms of correlation with the subjective scores show that many of the evaluated no-reference metrics cannot predict perceived image quality.

From synthetic to natural — single natural image dehazing deep networks using synthetic dataset domain randomization

Image dehazing methods aim to solve the problem of poor visibility in images due to haze. Techniques proposed for image dehazing in literature focus on image priors, haze lines or data driven statistical models. Variations of the classical methods relying on prior model or haze line model use no-reference image quality metrics to prove their dehazing performance. Recently developed deep learning models rely on huge amounts of hazy, haze-free pairs for training, and uses PSNR and SSIM like image reconstruction metrics to show their performance. These methods perform poorly on no-reference image quality assessments and also dehazes poorly at the depths of the image. These methods though can be optimized for memory usage and are faster. This work presents a deep learning model (Feature Fusion Attention Network) trained on a domain randomized synthetic dataset generated in simulation. The proposed model achieves the highest scores on blind image assessments through the gradient rationing technique for a deep learning-based approach by a significant margin. The images were evaluated on full-reference metrics as well and obtained favorable results. This approach also yields one of the highest edge sharpness obtained after dehazing. The training procedure adopted to obtain significant gains on real-world dehazing, without using any real-world data is also detailed in this paper.

LANBIQUE: LANguage-based Blind Image QUality Evaluation

Image quality assessment is often performed with deep networks that are fine-tuned to regress a human provided quality score of a given image. Usually, this approach may lack generalization capabilities and, while being highly precise on similar image distribution, it may yield lower correlation on unseen distortions. In particular, they show poor performances, whereas images corrupted by noise, blur, or compression have been restored by generative models. As a matter of fact, evaluation of these generative models is often performed providing anecdotal results to the reader. In the case of image enhancement and restoration, reference images are usually available. Nevertheless, using signal based metrics often leads to counterintuitive results: Highly natural crisp images may obtain worse scores than blurry ones. However, blind reference image assessment may rank images reconstructed with GANs higher than the original undistorted images. To avoid time-consuming human-based image assessment, semantic computer vision tasks may be exploited instead. In this article, we advocate the use of language generation tasks to evaluate the quality of restored images. We refer to our assessment approach as LANguage-based Blind Image QUality Evaluation (LANBIQUE). We show experimentally that image captioning, used as a downstream task, may serve as a method to score image quality, independently of the distortion process that affects the data. Captioning scores are better aligned with human rankings with respect to classic signal based or No-reference image quality metrics. We show insights on how the corruption, by artefacts, of local image structure may steer image captions in the wrong direction.

Automated image quality appraisal through partial least squares discriminant analysis.

Automatic retinal fundus image quality analysis is one of the most essential preliminary stages in automatic computer-aided retinal disease diagnosis system, which allows good-quality fundus images for accurate disease prediction through localization and segmentation of retinal regions. This paper presents new feature extraction methods using full-reference and no-reference image quality metrics for image quality classification. Basic image features, reference and no-reference features are extracted from the fundus image and applied through different classification techniques to determine the image quality for further diagnosis. In this paper, human-made categorization including good and non-good-quality fundus image classification is constructed by considering major features of retinal fundus images are illumination, clarity, image intensity, contrast and region visibility. The proposed system presented fundus image quality classification by automatic extraction of features from fundus images through image processing techniques and automatic classification of image quality through different classification algorithm. This system was thoroughly investigated on 2674 retinal fundus images from publically available datasets, namely MESSIDOR, Drishti-GS1, DRIVE, HRF, DRIONS-DB, DIARETDB0, DIARETDB1, IDRiD, INSPIRE-AVR, CHASE-DB1, ONHSD, DRIMDB and e-ophtha-EX with better performance results in terms of sensitivity, accuracy, precision and F1 score of 99.36%, 96.79%, 96.29% and 97.79%, respectively. The proposed system results were compared to the existing state-of-the-art approaches and outperform existing methods for image quality assessment representing the efficiency and robustness of our system is most suitable for automatic image analysis during retinal disease diagnosis.

Combined No-Reference Image Quality Metrics for Visual Quality Assessment Optimized for Remote Sensing Images

No-reference image quality assessment is one of the most demanding areas of image analysis for many applications where the results of the analysis should be strongly correlated with the quality of an input image and the corresponding reference image is unavailable. One of the examples might be remote sensing since the transmission of such obtained images often requires the use of lossy compression and they are often distorted, e.g., by the presence of noise and blur. Since the practical usefulness of acquired and/or preprocessed images is directly related to their quality, there is a need for the development of reliable and adequate no-reference metrics that do not need any reference images. As the performance and universality of many existing metrics are quite limited, one of the possible solutions is the design and application of combined metrics. Several possible approaches to their composition have been previously proposed and successfully used for full-reference metrics. In the paper, three possible approaches to the development and optimization of no-reference combined metrics are investigated and verified for the dataset of images containing distortions typical for remote sensing. The proposed approach leads to good results, significantly improving the correlation of the obtained results with subjective quality scores.

Transfer learning for no-reference image quality metrics using large temporary image sets

One of the main problems of neural network-based no-reference metrics design for image visual quality assessment is small size of image databases with mean opinion scores (MOS). For large networks which can memorize key features of several thousands of images, usage of the databases for metrics training may lead to overlearning. Since data augmentation for image quality assessment is limited by a horizontal image flipping only, the main way to decrease overlearning is to use transfer learning which can significantly speed up training process. In theis paper, we propose a new technique of transfer learning between networks of different architectures using a large set of images without MOS. We implemented the technique for transfer learning between pre-trained KonCept512 metric and a IMQNet metric proposed in this paper. An effectiveness of the transfer learning is estimated in a numerical analysis. It is shown that the trained IMQNet metric provides significantly better correlation with KonCept512 metric (0.89) than other modern metrics. It is also shown that IMQNet pre-trained by the proposed transfer learning shows better correlation with MOS of KonIQ-10k database (0.86) than IMQNet pre-trained using directly the MOS of KonIQ10k (0.73).

A practical residual block-based no-reference quality metric for neutron radiographic images

Objective image quality metric is a challenging problem for neutron radiography due to the lack of reference images. The existing no-reference image quality metric (NR-IQM) method designed for neutron radiographic images (NRIs) still exists two main shortcomings. Firstly, the limited L/D of the collimator would inevitably lead to geometric blur in the NRIs. However, the special distortion type of mixed Poisson–Gaussian noise is merely considered but ignores the blur distortion. Secondly, the distortion levels of Poisson noise added to an image cannot be adjusted. In order to address these two issues, a more practical objective NR-IQM method is proposed to predict the perceptual quality of NRIs in this paper. Experimental results demonstrate that the proposed method shows good consistency with the quality of authentic NRIs and also performs well in reflecting the tiny quality difference of enhanced NRIs, thus demonstrating its application potential in neutron radiography.

Evaluation of a no-reference image quality metric for projection X-ray imaging using a 3D printed patient-specific phantom.

Feasability of a no-reference image quality metric was assessed on patient-like images using a patient-specific phantom simulating a frame of a coronary angiogram. One background and one contrast-filled frame of a coronary angiogram, acquired using a clinical imaging protocol, were selected from a Philips Integris Allura FD (Philips Healthcare, Best, The Netherlands). The background frame's pixels were extruded to a thickness proportional to their grey value. One phantom was 3D printed using composite 80% bronze filament (max. thickness of 5.1mm), the other was a custom PMMA cast (max thickness of 8.5cm). A vessel mold was created from the contrast-filled frame and injected with a solution of 320mg I/ml contrast fluid (75%), water and gelatin. Still X-ray frames of the vessel mold+background phantom+16cm PMMA were acquired at manually selected different exposure settings using a Philips Azurion (Philips Healthcare, Best, The Netherlands) in User Quality Control Mode and were exported as RAW images. The signal-difference-to-noise-ratio-squared (SDNR2) and a spatial-domain-equivalent of the noise equivalent quanta (NEQSDE) were calculated. The Spearman's correlation of the latter parameters with a no-reference perceptual image quality metric (NIQE) was investigated. The bronze phantom showed better resemblance to the original patient frame selected from a coronary angiogram of an actual patient, with better contrast and less blur than the PMMA phantom. Both phantoms were imaged using a comparable imaging protocol to the one used to acquire the original frame. The bronze phantom was hence used together with the vessel mold for image quality measurements on the 165 still phantom frames. A strong correlation was noted between NEQSDE and NIQE (SROCC=-0.99, p<0.0005) and between SDNR2 and NIQE (SROCC=-0.97, p<0.0005). Using a cost-effective and easy to realize patient-specific phantom we were able to generate patient-like X-ray frames. NIQE as a no-reference image quality model has the potential to predict physical image quality from patient images.

Straightness metric for measurement of turbulence-induced distortion in long-range imaging.

Algorithms used for mitigation of the effects of atmospheric turbulence on video sequences often rely on a process for creating a reference image to register all of the frames. Because such a pristine image is generally not available, no-reference image quality metrics can be used to identify frames in a sequence that have minimum distortion. Here, we propose a metric that quantifies image warping by measuring image straightness based on line detection. The average length of straight lines in a frame is used to select best frames in a sequence and to generate a reference frame for a subsequent dewarping algorithm. We perform tests with this metric on simulated data that exhibits varying degrees of distortion and blur and spans normalized turbulence strengths between 0.75 and 4.5. We show, through these simulations, that the metric can differentiate between weak and moderate turbulence effects. We also show in simulations that the optical flow that uses a reference frame generated by this metric produces consistently improved image quality. This improvement is even higher when we employ the metric to guide optical flow that is applied to three real video sequences taken over a 7 km path.

Evaluation of DISORDER: Retrospective Image Motion Correction for Volumetric Brain MRI in a Pediatric Setting.

Head motion causes image degradation in brain MR imaging examinations, negatively impacting image quality, especially in pediatric populations. Here, we used a retrospective motion correction technique in children and assessed image quality improvement for 3D MR imaging acquisitions. We prospectively acquired brain MR imaging at 3T using 3D sequences, T1-weighted MPRAGE, T2-weighted TSE, and FLAIR in 32 unsedated children, including 7 with epilepsy (age range, 2-18 years). We implemented a novel motion correction technique through a modification of k-space data acquisition: Distributed and Incoherent Sample Orders for Reconstruction Deblurring by using Encoding Redundancy (DISORDER). For each participant and technique, we obtained 3 reconstructions as acquired (Aq), after DISORDER motion correction (Di), and Di with additional outlier rejection (DiOut). We analyzed 288 images quantitatively, measuring 2 objective no-reference image quality metrics: gradient entropy (GE) and MPRAGE white matter (WM) homogeneity. As a qualitative metric, we presented blinded and randomized images to 2 expert neuroradiologists who scored them for clinical readability. Both image quality metrics improved after motion correction for all modalities, and improvement correlated with the amount of intrascan motion. Neuroradiologists also considered the motion corrected images as of higher quality (Wilcoxon z = -3.164 for MPRAGE; z = -2.066 for TSE; z = -2.645 for FLAIR; all P < .05). Retrospective image motion correction with DISORDER increased image quality both from an objective and qualitative perspective. In 75% of sessions, at least 1 sequence was improved by this approach, indicating the benefit of this technique in unsedated children for both clinical and research environments.

No-reference Image Quality Assessment of Underwater Images Using Multi-Scale Salient Local Binary Patterns

Images acquired in underwater scenarios may contain severe distortions due to light absorption and scattering, color distortion, poor visibility, and contrast reduction. Because of these degradations, researchers have proposed several algorithms to restore or enhance underwater images. One way to assess these algorithms’ performance is to measure the quality of the restored/enhanced underwater images. Unfortunately, since reference (pristine) images are often not available, designing no-reference (blind) image quality metrics for this type of scenario is still a challenge. In fact, although the area of image quality has evolved a lot in the last decades, estimating the quality of enhanced and restored images is still an open problem. In this work, we present a no-reference image quality evaluation metric for enhanced underwater images (NR-UWIQA) that uses an adapted version of the multi-scale salient local binary pattern operator to extract image features and a machine learning approach to predict quality. The proposed metric was tested on the UID-LEIA database and presented good accuracy performance when compared to other state-of-the-art methods. In summary, the proposed NR-UWQIA method can be used to evaluate the results of restoration techniques quickly and efficiently, opening a new perspective in the area of underwater image restoration and quality assessment.

COMPARISON OF IMAGE ENHANCEMENT TECHNIQUES FOR RAPID PROCESSING OF POST FLOOD IMAGES

Abstract. Satellite images are widely used for assessing the areal extent of flooded areas. However, presence of clouds and shadow limit the utility of these images. Numerous digital algorithms are available for enhancing such images and highlighting areas of interest. These algorithms range from simple to complex, and the time required to process these images also varies considerably. For disaster response, it is important to select an algorithm that can enhance the quality of the images in relatively short time. This study compared the relative performance of five traditional (Histogram Equalization, Local Histogram Equalization, Contrast Limited Adaptive Histogram Equalization, Gamma Correction, and Linear Contrast Stretch) algorithms for enhancing post-flood satellite images. Flood images with different levels of clouds and shadows were enhanced and output generated were evaluated in terms of processing time and quality as measured by Blind/Reference less Image Spatial Quality Evaluator (BRISQUE), a no-reference image quality metric. Findings from this study will provide valuable information to image analysts for selecting a suitable algorithm for rapidly processing post-flood satellite images.

BITPNet: Unsupervised Bio-Inspired Two-Path Network for Nighttime Traffic Image Enhancement

Due to the low luminance in nighttime traffic images, image features are not salient, making tasks in intelligent transportation systems such as nighttime vehicle detection challenging. Recently, convolutional neural network based methods have been developed for low-light image enhancement. Most of these methods are supervised and require high-light reference images at the same scenes. However, reference images are difficult to be obtained in nighttime traffic scenes because vehicles always move. In the early visual system the input signals are processed by two parallel visual paths in the retina: one path has small receptive fields (RFs) to process the high frequency information and another path has large RFs to deal with the low frequency information. Inspired by this, we design a novel bio-inspired two-path convolutional neural network (BITPNet) for nighttime traffic image enhancement. The high-frequency path with small convolution kernel size is designed to suppress noises and preserve the details. The low-frequency path with large convolution kernel size is used to enhance the luminance of images. Each path includes an encoder-to-decoder network followed by a new multi-level attention module to combine features of levels with different RFs. The outputs of the two paths are summed by learnt weights for generating the final image enhancement result. Several no-reference image quality metrics are utilized to design a new loss function, resulting in an unsupervised approach. The proposed BITPNet is trained on one nighttime traffic image dataset and evaluated on another nighttime dataset. Experimental results demonstrate that the proposed BITPNet outperforms several state-of-the-art low-light image enhancement methods in terms of visual quality and three no-reference image quality metrics. In addition, when the proposed BITPNet is used as pre-processing for the nighttime multi-class vehicle detection task, it achieves higher detection rate (97.18%) than other methods.

No-Reference Image Quality Metric for Tone-Mapped Images

Tone-mapping operators transform high dynamic range (HDR) images into displayable low dynamic range (LDR) images. Image quality evaluation of these LDR images is not possible by comparison with their corresponding high dynamic range images. Hence, a no-reference image quality metric for tone-mapped LDR images is proposed based on the fitting to the present psychophysical results including different visual image quality attributes. Ten images, including HDR natural scenes, were tonemapped using six TMOs. They were used in the assessment and visual attributes were determined to predict the quality of these images. The visual attributes (brightness and Naturalness) were modeled using parameters derived from CAM16-UCS. Results showed that the quality prediction of the model had a reasonable degree of accuracy.

How re-training process affect the performance of no-reference image quality metric for face images

The accuracy of face recognition systems is significantly affected by the quality of face sample images. There are many existing no-reference image quality metrics (IQMs) that are able to assess natural image quality by taking into account similar image-based quality attributes. Previous study showed that IQMs can assess face sample quality according to the biometric system performance. In addition, re-training an IQM can improve its performance for face biometric images. However, only one database was used in the previous study, and it contains only image-based distortions. In this paper, we propose to extend the previous study by use multiple face database including FERET color face database and apply multiple setups for the re-training process in order to investigate how the re-training process affect the performance of no-reference image quality metric for face biometric images. The experimental results show that the performance of the appropriate IQM can be improved for multiple databases, and different re-training setups can influence the IQM's performance