Image Resolution Enhancement Research Articles

Computationally efficient minimum-variance baseband delay-multiply-and-sum beamforming for adjustable enhancement of ultrasound image resolution

Baseband Delay-Multiply-and-Sum (BB-DMAS) beamforming takes advantage of the baseband spatial coherence of receiving aperture to improve image resolution and contrast. Meanwhile, the side-lobe clutter and noise level can also be effectively suppressed in BB-DMAS beamforming due to their low coherence when being detected by channels in different spatial locations. BB-DMAS scales the magnitude of channel signal by p-th root and restores the output dimensionality by p-th power after channel summation. Higher p value introduces more spatial coherence into DMAS beamforming and provides higher image resolution at the cost of background speckle quality. In this study, a computationally efficient integration of BB-DMAS with minimum-variance (MV) beamforming is developed so that the image resolution can be drastically improved with low p value (e.g. p < 2) while maintaining the speckle quality. For each image pixel, the proposed MV-DMAS only requires single MV estimation to optimize the aperture apodization for DMAS beamforming. Our simulation results show that, with p = 1.5, the −6-dB lateral width of wire reflector noticeably improves from 0.22 mm to 0.13 mm by adopting MV estimation in BB-DMAS beamforming. In MV-DMAS, the suppression of uncorrelated random noises also remains effective. Experimental results not only confirm the superior resolution in MV-DMAS beamforming but also demonstrates comparable image contrast and speckle quality to BB-DMAS counterpart. In conclusion, MV-DMAS beamforming can provide improvement in image resolution while maintaining the other image quality metrics using an efficient combination of moderate spatial coherence and MV estimation of receiving aperture apodization in ultrasonic imaging.

Resolution Enhancement of UWB Time-Reversal Microwave Imaging in Dispersive Environments

Time Reversal (TR) techniques allow optimal refocusing of ultrawideband (UWB) electromagnetic waves in complex propagation environments, due to the invariance of the wave equations. Propagation of electromagnetic waves through dispersive or lossy media breaks this invariance. Consequently, such media degrade the refocusing ability of TR techniques. In this work, we propose a novel algorithm for the enhancement of the UWB TR microwave imaging resolution in dispersive environments. The presented algorithm takes into account the frequency-dependent complex permittivity of the propagation medium across the entire bandwidth of the UWB pulse. Using this complex permittivity, it models the medium-, time-, and frequency- dependent attenuation in the wavelet domain to create inverse filters which compensate for the effects of the attenuation. This is the first algorithm that constructs inverse filters in the wavelet domain using the complex permittivity of the dispersive propagation medium across the entire bandwidth, rather than at a center frequency, of the excitation pulse. We also introduce a smart scaling concept to minimize undesired noise amplification. While our proposed approach is agnostic to the application scenario and thus could be utilized in various disciplines, we apply it in a biomedical imaging scenario and achieve enhancement in the resolution of UWB microwave TR imaging of a simulated brain tumor inside the digital human phantom (DHP).

Cascaded Convolutional Neural Network-Based Hyperspectral Image Resolution Enhancement via an Auxiliary Panchromatic Image.

Owing to the limits of incident energy and hardware system, hyperspectral (HS) images always suffer from low spatial resolution, compared with multispectral (MS) or panchromatic (PAN) images. Therefore, image fusion has emerged as a useful technology that is able to combine the characteristics of high spectral and spatial resolutions of HS and PAN/MS images. In this paper, a novel HS and PAN image fusion method based on convolutional neural network (CNN) is proposed. The proposed method incorporates the ideas of both hyper-sharpening and MS pan-sharpening techniques, thereby employing a two-stage cascaded CNN to reconstruct the anticipated high-resolution HS image. Technically, the proposed CNN architecture consists of two sub-networks, the detail injection sub-network and unmixing sub-network. The former aims at producing a latent high-resolution MS image, whereas the latter estimates the desired high-resolution abundance maps by exploring the spatial and spectral information of both HS and MS images. Moreover, two model-training fashions are presented in this paper for the sake of effectively training our network. Experiments on simulated and real remote sensing data demonstrate that the proposed method can improve the spatial resolution and spectral fidelity of HS image, and achieve better performance than some state-of-the-art HS pan-sharpening algorithms.

Artificial Intelligence in Musculoskeletal Radiology: Past, Present, and Future

Musculoskeletal radiology is an important tool for the diagnosis of muscle damage, bone fractures, bone tumors, musculoskeletal infection, and other diseases. However, all currently used radiological techniques, including radiography, ultrasonography, computed tomography, and magnetic resonance imaging are associated with their own challenges. With its ability to address these challenges, artificial intelligence (AI) holds the promise to transform a musculoskeletal radiologist’s job in several areas. In the past, AI-based approaches in musculoskeletal radiology were primarily used for measuring bone mineral density or identifying bone tumors. However, recent studies have expanded the application of AI in several other areas, such as image segmentation, resolution enhancement, and fracture identification as well automatic diagnosis of other forms of musculoskeletal damage. This review article discusses numerous older as well as more recent studies to highlight how the development and application of AI-based approaches have evolved in the field of musculoskeletal radiology and how the applicability of these approaches may be improved in the future.

Cascading and Enhanced Residual Networks for Accurate Single-Image Super-Resolution.

Deep convolutional neural networks (CNNs) have contributed to the significant progress of the single-image super-resolution (SISR) field. However, the majority of existing CNN-based models maintain high performance with massive parameters and exceedingly deeper structures. Moreover, several algorithms essentially have underused the low-level features, thus causing relatively low performance. In this article, we address these problems by exploring two strategies based on novel local wider residual blocks (LWRBs) to effectively extract the image features for SISR. We propose a cascading residual network (CRN) that contains several locally sharing groups (LSGs), in which the cascading mechanism not only promotes the propagation of features and the gradient but also eases the model training. Besides, we present another enhanced residual network (ERN) for image resolution enhancement. ERN employs a dual global pathway structure that incorporates nonlocal operations to catch long-distance spatial features from the the original low-resolution (LR) input. To obtain the feature representation of the input at different scales, we further introduce a multiscale block (MSB) to directly detect low-level features from the LR image. The experimental results on four benchmark datasets have demonstrated that our models outperform most of the advanced methods while still retaining a reasonable number of parameters.

Neural network technology adaptation to the small-size objects identification in satellite images of insufficient resolution within the graphic reference images database

A novel flowchart for small-size objects identification in satellite images of insufficient resolution within the graphic reference images database using neural network technology based on compromise contradiction, i.e. simultaneously the resolution enhancement of the object segment of input image and the resolution reduction of the reference image to joint resolution through the simulation of the imaging system has been proposed. This is necessary due to a significant discrepancy between the resolutions of the input image and the graphic reference images used for identification. The required level of resolution enhancement for satellite images, as a rule, is unattainable, and a significant coarsening of reference images is undesirable because of identification errors. Therefore, a certain intermediate spatial resolution is used for identification, which, on the one hand, can be obtained, and on the other the loss of information contained in the reference image is still acceptable. The intermediate resolution is determined by simulating the process of image acquisition with satellite imaging system. To facilitate such simulation, it is advisable to perform it in the frequency domain, where the advanced Fourier analysis is available and, as a rule, all the necessary transfer properties of the links of image formation chain are known. Three main functional elements are engaged for identification: an artificial neural network for the resolution enhancement of input images, a module of frequency-domain simulating of the graphical reference satellite imaging and an artificial neural network for comparing the enhanced object segment with the reference model images. The feasibility of the described approach is demonstrated by the example of successful identification of the sea vessel image in the SPOT-7 satellite image. Currently, the works are under way to compare the performance of a neural network platforms variety for small-size objects identification in satellite images aa well as to assess achievable accuracy.

Resolution enhancement and realistic speckle recovery with generative adversarial modeling of micro-optical coherence tomography.

A resolution enhancement technique for optical coherence tomography (OCT), based on Generative Adversarial Networks (GANs), was developed and investigated. GANs have been previously used for resolution enhancement of photography and optical microscopy images. We have adapted and improved this technique for OCT image generation. Conditional GANs (cGANs) were trained on a novel set of ultrahigh resolution spectral domain OCT volumes, termed micro-OCT, as the high-resolution ground truth (∼1 μm isotropic resolution). The ground truth was paired with a low-resolution image obtained by synthetically degrading resolution 4x in one of (1-D) or both axial and lateral axes (2-D). Cross-sectional image (B-scan) volumes obtained from in vivo imaging of human labial (lip) tissue and mouse skin were used in separate feasibility experiments. Accuracy of resolution enhancement compared to ground truth was quantified with human perceptual accuracy tests performed by an OCT expert. The GAN loss in the optimization objective, noise injection in both the generator and discriminator models, and multi-scale discrimination were found to be important for achieving realistic speckle appearance in the generated OCT images. The utility of high-resolution speckle recovery was illustrated by an example of micro-OCT imaging of blood vessels in lip tissue. Qualitative examples applying the models to image data from outside of the training data distribution, namely human retina and mouse bladder, were also demonstrated, suggesting potential for cross-domain transferability. This preliminary study suggests that deep learning generative models trained on OCT images from high-performance prototype systems may have potential in enhancing lower resolution data from mainstream/commercial systems, thereby bringing cutting-edge technology to the masses at low cost.

Real-time underwater image resolution enhancement using super-resolution with deep convolutional neural networks

In this paper, a two-step image enhancement is presented. In the first step, color correction and underwater image quality enhancement are conducted if there are artifacts such as darkening, hazing and fogging. In the second step, the image resolution optimized in the previous step is enhanced using the convolutional neural network (CNN) with deep learning capability. The main reason behind the adoption of this two-step technique, which includes image quality enhancement and super-resolution, is the need for a robust strategy to visually improve underwater images at different depths and under diverse artifact conditions. The effectiveness and robustness of the real-time algorithm are satisfactory for various underwater images under different conditions, and several experiments have been undertaken for the two datasets of images. In both stages and for each of image datasets, the mean square error (MSE), peak signal to noise ratio (PSNR), and structural similarity (SSIM) evaluation measures were fulfilled. In addition, the low computational complexity and suitable outputs were obtained for different artifacts that represented divergent depths of water to achieve a real-time system. The super-resolution in the proposed structure for medium layers can offer a proper response. For this reason, time is also one of the major factors reported in the research. Applying this model to underwater imagery systems will yield more accurate and detailed information.

Machine-Learning Image Recognition Enhances Rock Classification

This article, written by JPT Technology Editor Chris Carpenter, contains highlights of paper SPE 196657, “Machine Learning for 3D Image Recognition To Determine Porosity and Lithology of Heterogeneous Carbonate Rock,” by Omar Al-Farisi, SPE, Hongtao Zhang, and Aikifa Raza, Khalifa University of Science and Technology, prepared for the 2019 SPE Reservoir Characterization and Simulation Conference and Exhibition, Abu Dhabi, 17-19 September. The paper has not been peer reviewed. Automated image-processing algorithms can improve the quality and speed in classifying the morphology of heterogeneous carbonate rock. Several commercial products have produced petrophysical properties from 2D images and, to a lesser extent, from 3D images. Images are mainly microcomputed tomography (μCT), optical images of thin sections, or magnetic resonance images (MRI). However, most successful work is from homogeneous and clastic rocks. In the complete paper, the authors have demonstrated a machine-learning-assisted image-recognition (MLIR) approach to determine the porosity and lithology of heterogeneous carbonate rock by analyzing 3D images from µCT and MRI. Introduction The authors’ literature review has revealed the pressing need to perform 3D image processing instead of 2D. Achieving this goal requires an interdisciplinary approach. This study deployed new analysis and verification approaches, including 3D micromodels (3DMM) with various micropore sizes and uses 3DMM as an image-processing calibration reference. Additionally, a new image-resolution enhancement for quality segmentation is developed. Porosity was determined mainly using two methods. The first is a standalone image processing, in which image-information extraction was successful. The second is MLIR. The difference between image processing and image analysis is important. If image processing enables extraction of meaningful data, then image analysis is the ability to interpret this data through numerical analysis.

Scanning microsphere array optical microscope for efficient and large area super-resolution imaging

Dielectric microsphere based super-resolution imaging is a promising approach to overcome the diffraction limit of imaging in an optical microscope. So far, it still remains as a challenge to efficiently implement super-resolution imaging in a large area in a controlled manner. In this study, a home-designed imaging probe which consists of a microsphere array is applied to implement scanning of sample surfaces with an enhanced imaging resolution. The individually scanned images from different microspheres can then be stitched together to construct a large area super-resolution optical image. The experimental results show that the proposed method can correctly stitch together individually scanned images, leading to efficient and feasible super-resolution imaging in an imaging area of 195 μm × 150 μm.

Model for selecting the Earth observation satellite systems by object recognition probability

Model for selecting the Earth observation satellite systems by object recognition probability

Component Decomposition-Based Hyperspectral Resolution Enhancement for Mineral Mapping

Combining both spectral and spatial information with enhanced resolution provides not only elaborated qualitative information on surfacing mineralogy but also mineral interactions of abundance, mixture, and structure. This enhancement in the resolutions helps geomineralogic features such as small intrusions and mineralization become detectable. In this paper, we investigate the potential of the resolution enhancement of hyperspectral images (HSIs) with the guidance of RGB images for mineral mapping. In more detail, a novel resolution enhancement method is proposed based on component decomposition. Inspired by the principle of the intrinsic image decomposition (IID) model, the HSI is viewed as the combination of a reflectance component and an illumination component. Based on this idea, the proposed method is comprised of several steps. First, the RGB image is transformed into the luminance component, blue-difference and red-difference chroma components (YCbCr), and the luminance channel is considered as the illumination component of the HSI with an ideal high spatial resolution. Then, the reflectance component of the ideal HSI is estimated with the downsampled HSI image and the downsampled luminance channel. Finally, the HSI with high resolution can be reconstructed by utilizing the obtained illumination and the reflectance components. Experimental results verify that the fused results can successfully achieve mineral mapping, producing better results qualitatively and quantitatively over single sensor data.

Data Expansion Method Based on Wavelet Transform

For small sample data, data expansion is an effective means to improve the accuracy of target recognition. To image magnification to enhance image resolution as a starting point, the author puts forward two kinds of image magnification method based on wavelet transform. Qualitative analysis on the experimental data and quantitative analysis using average gradient, cross entropy as performance index, show that the proposed image magnification method has obvious advantages in terms of image definition, maintaining image gray scale information. Finally, the author proposes the method of image expansion based on the image magnification data of the four methods, which greatly enriches the diversification of the training set on resolution, clarity, contrast, integrity and so on, and theoretically analyzes the feasibility of improving the recognition accuracy of the data with few samples and small targets.

A GENERATIVE ADVERSARIAL NETWORK APPROACH FOR SUPER-RESOLUTION OF SENTINEL-2 SATELLITE IMAGES

Abstract. High-resolution satellite images have always been in high demand due to the greater detail and precision they offer, as well as the wide scope of the fields in which they could be applied; however, satellites in operation offering very high-resolution (VHR) images has experienced an important increase, but they remain as a smaller proportion against existing lower resolution (HR) satellites. Recent models of convolutional neural networks (CNN) are very suitable for applications with image processing, like resolution enhancement of images; but in order to obtain an acceptable result, it is important, not only to define the kind of CNN architecture but the reference set of images to train the model. Our work proposes an alternative to improve the spatial resolution of HR images obtained by Sentinel-2 satellite by using the VHR images from PeruSat1, a peruvian satellite, which serve as the reference for the super-resolution approach implementation based on a Generative Adversarial Network (GAN) model, as an alternative for obtaining VHR images. The VHR PeruSat-1 image dataset is used for the training process of the network. The results obtained were analyzed considering the Peak Signal to Noise Ratios (PSNR) and the Structural Similarity (SSIM). Finally, some visual outcomes, over a given testing dataset, are presented so the performance of the model could be analyzed as well.

Endoscopic Versus Microscopic Stapedotomy: Our Experience.

With technological improvements leading to enhanced image resolution, as well as recognized acceptance of endoscopes within the field of rhinology, there has been greater implementation of endoscopes in otologic and neurotologic procedures. Surgeons now use endoscopes for a wide range of otological procedure, one such procedure is endoscopic stapedotomy for otosclerosis. This study is done to compare merits and demerits of endoscopic stapedotomy with microscopic stapedotomy. This is a prospective observational study conducted in SMBT Medical College and Hospital, Nashik from January 2018 to January 2020. 40 patients who presented to ENT OPD with clinical features suggestive of otosclerosis and who fulfilled inclusion and exclusion criteria were included in the study. Detailed history, complete ENT examination and audiological investigations were done. Preanesthetic evaluation was done and patients were randomly allotted into two groups. Group A: consisted of twenty patients who underwent microscopic stapedotomy under local anesthesia Group B: consisted of twenty patients who underwent endoscopic stapedotomy under local anesthesia. Data was collected in pre-structured proforma and analyzed. There was no statistically significant difference between microscopic and endoscopic procedure in terms of duration of the surgery, hearing improvements or complications. However, surgeons noted that ease of visualization of middle ear structures and the better image quality favors endoscopic procedure over microscopic procedure. There is no increased risk of complications with endoscopic stapedotomy and hearing improvement is same as that of microscopic stapedotomy however the ease of visualization of middle ear structures and the better image quality favors endoscopic procedure over microscopic procedure. Hence, we conclude that endoscopic stapedotomy is a safe alternative to microscopic stapedotomy.

Super-resolution Quality Criterion (SRQC): a super-resolution image quality assessment metric

Recently, image super-resolution has reinforced image resolution enhancement approaches in real-time and ensuring visual quality of super resolved images has evolved as a key research problem. Most quantitative benchmarks rely on full reference metrics which would work in the presence of a reference image. However, the unavailability of ground truth images in real world applications and the size constraints of low resolution and high resolution images often pose major challenges to such metrics. In order to address these problems, we present a super-resolution image quality index (SRQC –Super-resolution Quality Criterion), which can effectively quantify the efficiency and performance of image super-resolution algorithms. SRQC benchmark evaluates the quality score of a super resolved image according to the perceptual concepts of low-level spatial features in high sharpness space and curvelet based quality-aware features from focal energy bands, which can be used to capture the quality preservation of an SR image. The proposed metric is referenceless, the significance being that the assessment does not require ground-truth image. Explicitly, the SR image is assessed in the curvelet domain which is suitable for the no-reference super-resolution image quality assessment based on human perception. Experimental scores illustrate that the SRQC is more competent in modeling the features from curvelet transform, thus quantifying the quality score of the super resolved image and outperforming the formerly reported image quality assessment metrics.

Image resolution enhancement via sparse interpolation on wavelet domain

The image processing algorithms collectively known as super-resolution (SR) have proven effective in producing high-quality imagery from low-resolution (LR) images. This paper focuses on a novel image resolution enhancement method employing the wavelet domain techniques. In order to preserve more edge information, additional edge extraction step is proposed employing high-frequency (HF) sub-band images - low-high (LH), high-low (HL), and high-high (HH) - via the Discrete Wavelet Transform (DWT). In the designed procedure, the LR image is used in the sparse interpolation for the resolution-enhancement obtaining low-low (LL) sub- band. Additionally, all sub-bands (LL, LH, HL and HH) are performed via the Lanczos interpolation. Finally, the estimated sub-band images are used to form the new high-resolution (HR) image using the inverse DWT (IDWT). Experimental results on real data sets have confirmed the effectiveness of the proposed framework in terms of objective criteria as well as in subjective perception.

Monitoring early stage invasion of exotic Spartina alterniflora using deep-learning super-resolution techniques based on multisource high-resolution satellite imagery: A case study in the Yellow River Delta, China

Over the past decades, Spartina alterniflora, one of the top exotic invasive plants in China, has expanded throughout coastal China. In the Yellow River Delta (YRD), the rapid expansion of S. alterniflora has caused serious negative ecological effects. Current studies have concentrated primarily on mapping the distribution of S. alterniflora with medium-resolution satellite imagery at the regional or landscape scale, which have a limited capability in early detection and monitoring of the invasive process at the patch scale. In this study, we proposed a framework for monitoring the early stage invasion of S. alterniflora patches in the YRD using multiyear multisource high-spatial-resolution satellite imagery with various ground sampling distances (WorldView-2, SPOT-6, GaoFen-1, GaoFen-2, and GaoFen-6 from 2012 to 2019). First, we proposed to use deep-learning-based image super-resolution models to enhance all images to submeter (0.5 m) resolution. Then, we adopted stepwise evolution analysis-based image segmentation and object-based classification rules to detect and delineate S. alterniflora patches from the super-resolved imagery. By investigating Super-Resolution Convolutional Neural Networks (SRCNN) and Fast Super-Resolution Convolutional Neural Networks (FSRCNN) and comparing these methods with the conventional bicubic interpolation method for image resolution enhancement, we concluded that FSRCNN was superior in constructing spectral and structural details from the 1 m/1.5 m/2 m resolution images to 0.5 m resolution. FSRCNN, in particular, was more effective and efficient in discerning and estimating the size of small S. alterniflora patches (<50 m2). Using our method, 76 of 83 field-measured small patches were accurately detected and the delineated S. alterniflora patch perimeters agreed well with the field-measured patch perimeters (root mean square error [RMSE] = 8.29 m, mean absolute percentage error [MAPE] = 23.46 %). The invasion process showed fast expansion from 2012 to 2015 and slow growth from 2016 to 2019. We observed that the landward limits of S. alterniflora patches were influenced by elevation and vicinity to tidal creeks.

Resolution enhancement of images pair based on block cross interpolation

The image resolution enhancement concerns with the improvement of image resolution based on merging of several acquisitions of low-resolution observations by camera. In this paper, an algorithm for increasing the resolution of two images shifted by half a pixel horizontally and vertically is proposed, based on block cross-interpolation. The algorithm aims at reducing the errors in the known (bicubic and averaging) methods, using the criteria of MSE and PSNR.

Image Resolution Enhancement of Highly Compressively Sensed CT/PET Signals

One of the most challenging aspects of medical modalities such as Computed Tomography (CT) as well hybrid techniques such as CT/PET (Computed Tomography/Positron emission tomography) and PET/MRI is finding a balance between examination time, radiation dose, and image quality. The need for a dense sampling grid is associated with two major factors: image resolution enhancement, which leads to a strengthening of human perception, and image features interpretation. All these aspects make an unsupervised image processing much easier. The presented algorithm employs super-resolution-reconstruction with high accuracy motion fields’ estimation at its core for Computed Tomography/Positron Emission Tomography (CT/PET) images enhancement. The suggested method starts with processing compressively sensed input signals. This paper shows that it is possible to achieve higher image resolution while keeping the same radiation dose. The purpose of this paper is to propose a highly effective CT/PET image reconstruction strategy, allowing for simultaneous resolution enhancing and scanning time minimisation. The algorithm aims to overcome two major obstacles—image resolution limitation and algorithm reconstruction time efficiency-by combining a highly-sparse Ridgelet analysis based sampling pattern as well as PET signal sensing with super-resolution (SR) image enhancement. Due to the diverse nature of Computed Tomography, the applied Ridgelet analysis arguing its usability turned out to be efficient in reducing acquisition times in regard to maintaining satisfying scan quality. This paper presents a super-resolution image enhancement algorithm designed for handling highly sensitively compressed hybrid CT/PET scanners raw data. The presented technique allows for improving image resolution while reducing motion artefacts and keeping scanning times at pretty low levels.