Image Interpolation Method Research Articles

Self-supervised medical slice interpolation network using controllable feature flow

Deep learning-based image interpolation methods are confronted with various challenges in its application to anisotropic medical volumetric data (i.e., CT and MR images) out of the complex nonlinear deformation and the scarcity of high-quality images. This paper proposes a self-supervised multiple medical slice interpolation network using controllable feature flow. Firstly, a Controllable Feature Flow Network (CFFNet) is proposed to estimate the complex nonlinear deformation of medical images. In CFFNet, we first use a deformation-aware network and a spatial channel modulation module to predict the bi-directional feature flows from the source slices to the target slices by considering an additional position parameter, and then the learned feature flows are used to synthesize the target intermediate features by using the deformable convolution. Secondly, an improved two-stage self-supervised framework is proposed to alleviate the dependency on high-quality medical images. In the first stage, the synthesized training pairs along the dense sagittal and coronal directions are adopted to pre-train the CFFNet. In the second one, the sparse axial slices are used to fine-tune the CFFNet with the cycle-consistency constraint and a feature domain smooth loss. Experimental results illustrate that the proposed CFFNet assumes superior performance on medical slice interpolation tasks with fewer parameters and the proposed self-supervised CFFNet obtains competitive results compared with other fully supervised algorithms. We also evaluate our model on HepaticVessel segmentation task, the proposed method can effectively enhance the continuity and smoothness of the 3D vessel structures, showing promising results on clinical application. The code is available at: https://codeocean.com/capsule/8945297/tree/v1.

Reconstruction of target scattering centres based on characteristics analysis of inverse synthetic aperture radar images

AbstractDue to the aircraft targets’ uncooperative character, acquiring the complete scattering characteristics of enemy aircraft is challenging. An effective reconstruction method of target scattering centres based on characteristics analysis of inverse synthetic aperture radar (ISAR) images is proposed in this letter. First, a positive definite manifold characterizing ISAR images of different attitudes is constructed. Then, Kullback–Leibler divergence is used as a measure of the variability between ISAR images. Finally, an ISAR image interpolation method based on the characteristics analysis of ISAR images is proposed to obtain the scattering characteristics with more attitudes. Furthermore, simulation results confirm that the proposed method is effective.

Image Super-resolution Methods for FY-3E X-EUVI 195 Å Solar Images

Solar eruptions and the solar wind are sources of space weather disturbances, and extreme-ultraviolet (EUV) observations are widely used to research solar activity and space weather forecasts. Fengyun-3E is equipped with the Solar X-ray and Extreme Ultraviolet Imager, which can observe EUV imaging data. Limited by the lower resolution, however, we research super-resolution techniques to improve the data quality. Traditional image interpolation methods have limited expressive ability, while deep-learning methods can learn to reconstruct high-quality images through training on paired data sets. There is a wide variety of super-resolution models. We try these three representative models: Real-ESRGAN combined with generative adversarial networks, residual channel-attention networks (RCAN) based on channel attention, and SwinIR, based on self-attention. Instruments on different satellites differ in observation time, angle, and resolution, so we selected Solar Dynamics Observatory/Atmospheric Imaging Assembly (SDO/AIA) 193 Å images with similar wavelengths as a reference and used a feature-based method for image registration to eliminate slight deformations to build training data sets. Finally, we compare the above methods in their evaluation metrics and visual quality. RCAN has the highest peak signal-to-noise ratio and structural similarity evaluation. Real-ESRGAN model is the best in the Learned Perceptual Image Patch Similarity index, and its results visually show that it has more highly detailed textures. The corrected super-resolution results can complement the SDO/AIA data to provide solar EUV images with a higher temporal resolution for space weather forecasting and solar physics research.

МЕТОДЫ ИНТЕРПОЛЯЦИИ РАЗРЕЖЕННЫХ ИЗОБРАЖЕНИЙ, РАБОТАЮЩИЕ В ЧАСТОТНОЙ ОБЛАСТИ

The work compares the work of sparse image interpolation methods such as: - interpolation method of sequential computation of the Fourier spectrum (IMSCS), the method of projections onto convex sets (projections onto convex sets, POCS), and the method of amplitude iterations (MAI). All calculations necessary for the reconstruction of sparse images are performed only on spatial spectra. As an example, an aerospace digital image is used, which is typical for the tasks of remote sensing of the earth's surface. A high degree of sparseness is modeled (90 percent of the information is missing). The effectiveness of the studied methods was carried out according to several objective criteria.

Reversible Data Hiding Method for Interpolated Images Based on Modulo Operation and Prediction-Error Expansion

Image reversible data hiding (RDH) using interpolation technology (IT) provides high single-layer embedding capacity and has become a research hotspot. To solve serious image distortion in existing methods caused by secondary embedding based on interpolated pixels, we propose an RDH method for interpolated images based on modulo operation and prediction-error expansion (PEE). Our method differs from existing RDH methods that only consider embedding data into interpolated pixels. First, an image interpolation method is used to generate an interpolated image, which is partitioned into overlapping 3 × 3 blocks. For each block, secret data are embedded into interpolated pixels using the modulo operation and into reference pixel using the PEE. Standard grayscale images of size 512 × 512 were selected from the popular BossBase and BOWS-2 image databases as input images for experiments. Extensive experimental results show that the proposed method avoids pixel values underflow/overflow caused by embedding data and reduces image distortion effectively. For each standard test image, the maximum bits per pixel (BPP) is higher than 1.5, and the corresponding peak signal to noise ratio (PSNR) is higher than 45.0 dB. At the given embedding capacity, the proposed method outperforms recent similar methods regarding the watermarked image quality. Moreover, the proposed method can resist histogram analysis, and the subjective evaluation proves that the proposed method has good imperceptibility.

Convergence Results in Image Interpolation With the Continuous SSIM

Assessing the similarity of two images is a complex task that attracts significant efforts in the image processing community. The widely used structural similarity index measure (SSIM) addresses this problem by quantifying a perceptual structural similarity. In this paper we consider a recently introduced continuous SSIM (cSSIM), which allows one to analyze sequences of images of increasingly fine resolutions, and further extend the definition of the index to encompass the locally weighted version that is used in practice. For both the local and the global versions, we prove that the continuous index includes the classical SSIM as a special case, and we provide a precise connection between image similarity measured by the cSSIM and by the $L_2$ norm. Using this connection, we derive bounds on the cSSIM by means of bounds on the $L_2$ error, and we even prove that the two error measures are equivalent in certain circumstances. We exploit these results to obtain precise rates of convergence with respect to the cSSIM for several concrete image interpolation methods, and we further validate these findings by different numerical experiments. This newly established connection paves the way to obtain novel insights into the features and limitations of the SSIM, including on the effect of the local weighted window on the index performances.

Image Interpolation with Regional Gradient Estimation

This paper proposes an image interpolation method with regional gradient estimation (GEI) to solve the problem of the nonlinear interpolation method not sufficiently considering non-edge pixels. First, the approach presented in this paper expanded on the edge diffusion idea used in CGI and proposed a regional gradient estimation strategy to improve the problem of gradient calculation in the CGI method. Next, the gradient value was used to determine whether a pixel was an edge pixel. Then, a 1D directional filter was employed to process edge pixels while interpolating non-edge pixels using a 2D directionless filter. Finally, we experimented with various representative interpolation methods for grayscale and color images, including the one presented in this paper, and compared them in terms of subjective results, objective criteria, and computational complexity. The experimental results showed that GEI performed better than the other methods in an experiment concerning the visual effect, objective criteria, and computational complexity.

Image Interpolation Based on 2D-DWT with Novel Regularity-Preserving Algorithm Using RLS Adaptive Filters

This paper proposes a novel method for the image interpolation problem based on two-dimensional discrete wavelet transform (DWT) with the edge preserving approach. The purpose of this method is to consider two contrasting issues of over-smoothing and creation of spurious edges at the same time, and offer a novel solution based on statistical dependencies of image sub-bands, and noise behavior. The offered method has a multi-faceted approach for the problem; by sub-band coding, it handles each 2D-DWT image sub-band with a different solution. For LH and HL sub-bands, two algorithms work together in order to preserve regularity. Area_Check algorithm is a four-phase edge-preserving algorithm that aims to recognize and interpolate separating lines of environments and edgy regions in the best possible way. On the other hand, RLS_AVG algorithm interpolates smooth surfaces of the image by keeping the regularity of the image without over-smoothing. In this regard, the offered algorithm has a great power to counter jaggies and annoying artifacts. In the end, in order to demonstrate the capability, and performance of the proposed method, the final results in various metrics are compared with the results of the most famous and the newest image interpolation methods.

Non-rigid Multi-Modal Medical Image Registration Based on Improved Maximum Mutual Information PV Image Interpolation Method.

With the continuous improvement of medical imaging equipment, CT, MRI and PET images can obtain accurate anatomical information of the same patient site. However, due to the fuzziness of medical image physiological evaluation and the unhealthy understanding of objects, the registration effect of many methods is not ideal. Therefore, based on the medical image registration model of Partial Volume (PV) image interpolation method and rigid medical image registration method, this paper established the non-rigid registration model of maximum mutual information Novel Partial Volume (NPV) image interpolation method. The proposed NPV interpolation method uses the Davidon-Fletcher-Powell algorithm (DFP) algorithm optimization method to solve the transformation parameter matrix and realize the accurate transformation of the floating image. In addition, the cubic B-spline is used as the kernel function to improve the image interpolation, which effectively improves the accuracy of the registration image. Finally, the proposed NPV method is compared with the PV interpolation method through the human brain CT-MRI-PET image to obtain a clear CT-MRI-PET image. The results show that the proposed NPV method has higher accuracy, better robustness, and easier realization. The model should also have guiding significance in face recognition and fingerprint recognition.

Three-dimensional tissue volume generation in conventional brightfield microscopy.

The purpose of the study is to develop and automate a series of steps for enabling digital 3D tissue volume generation in conventional Brightfield microscopy for histopathology applications. Tissue samples were retrieved from the General Hospital of Athens "Hippocration", Greece. Samples were placed on a microtome that produced consecutive 2 μm sections. Each section was stained using Hematoxylin and Eosin and placed on microscope slides. A histopathologist specified the region of interest (ROI) on each slide. A 2D image was created from each ROI using a LEICA DM2500 microscope with a LEICA DFC 420C camera. Τhe 3D volume was created by stacking consecutive 2D images using a deep learning image interpolation method. The reconstructed 3D tissue volumes were evaluated by an expert histopathologist. Results showed that the 3D volumes might reveal information that is not clearly visible or even undetectable in the conventional 2D Brightfield images. In contrast to other 3D tissue imaging technologies, the proposed method (a) does not depend on the distance of the sample from the objectives producing 3D tissue volumes at any desired magnification, (b) does not require a special instrument, it may be implemented with any conventional Brightfield microscope, and (c) can be used for any given routine application, not only for some specialized clinical studies. The proposed study provides the basis for a feasible, cost-less and time-less upgrade of any standard 2D microscope into a 3D imaging instrument that may enhance the quality of diagnostic assessments in histopathology. HIGHLIGHTS: A method for 3D tissue volume generation. 3D volumes reveal information not clearly visible or even undetectable in 2D images. A method for feasible, cost-less and time-less upgrade of any Brightfield 2D microscope into a 3D imaging instrument.

Weed Identification by Single-Stage and Two-Stage Neural Networks: A Study on the Impact of Image Resizers and Weights Optimization Algorithms.

The accurate identification of weeds is an essential step for a site-specific weed management system. In recent years, deep learning (DL) has got rapid advancements to perform complex agricultural tasks. The previous studies emphasized the evaluation of advanced training techniques or modifying the well-known DL models to improve the overall accuracy. In contrast, this research attempted to improve the mean average precision (mAP) for the detection and classification of eight classes of weeds by proposing a novel DL-based methodology. First, a comprehensive analysis of single-stage and two-stage neural networks including Single-shot MultiBox Detector (SSD), You look only Once (YOLO-v4), EfficientDet, CenterNet, RetinaNet, Faster Region-based Convolutional Neural Network (RCNN), and Region-based Fully Convolutional Network (RFCN), has been performed. Next, the effects of image resizing techniques along with four image interpolation methods have been studied. It led to the final stage of the research through optimization of the weights of the best-acquired model by initialization techniques, batch normalization, and DL optimization algorithms. The effectiveness of the proposed work is proven due to a high mAP of 93.44% and validated by the stratified k-fold cross-validation technique. It was 5.8% improved as compared to the results obtained by the default settings of the best-suited DL architecture (Faster RCNN ResNet-101). The presented pipeline would be a baseline study for the research community to explore several tasks such as real-time detection and reducing the computation/training time. All the relevant data including the annotated dataset, configuration files, and inference graph of the final model are provided with this article. Furthermore, the selection of the DeepWeeds dataset shows the robustness/practicality of the study because it contains images collected in a real/complex agricultural environment. Therefore, this research would be a considerable step toward an efficient and automatic weed control system.

Research on Printing Defects Inspection of Solder Paste Images

Solder paste printing is the first part of the surface mount process flow; its postprinting defect inspection is particularly important. In this paper, we focus on studying the printing defects inspection algorithm for solder paste on PCB (Printed Circuit Board) images. The work proposes a number of methods to enhance the defects inspection performance of solder paste printing: a regional multidirectional data fusion image interpolation method, which can achieve fast and high precision image interpolation; a method for detecting solder paste areas with better accuracy, efficiency, and robustness; an improved connected domain labeling method to reduce time complexity; and defects detection and types classification method, which extracts features and centroid of every solder paste region and completes the inspection by comparing with a standard image. The experiments show that the defects inspection algorithm can detect the most common types of defects with low time consumption, high inspection accuracy, and classification accuracy.

An adaptive high capacity reversible data hiding algorithm in interpolation domain

Reversible data hiding (RDH) algorithms using interpolation technology have the advantage of high capacity for single-layer embedding. However, existing algorithms usually embed secret data in the non-reference pixels based on simple additions without considering the properties of context-pixels, which results in that the visual quality of watermarked image is not optimal. To deal with this issue, a novel adaptive high capacity RDH algorithm is proposed. Firstly, a location-dependent weighted image interpolation method is proposed to improve the visual quality of the interpolated image. Then, the interpolated image is divided into overlapping blocks, which are sorted in an ascending order according to their standard deviations; secret data is preferentially embedded in the blocks with small deviation so that image quality is not distorted too much. Finally, to enhance embedding capacity as much as possible, the amount of secret data bits that can be embedded in non-reference pixels is calculated adaptively. We theoretically prove that the embedded secret data can be extracted correctly and there is no pixel overflow or underflow problem. Extensive experimental results showed that the proposed algorithm outperforms several state-of-the-art algorithms. In addition, our algorithm can resist steganalysis attacks, and demonstrated the effectiveness of the proposed algorithm.

Imaging depth adaptive resolution enhancement for optical coherence tomography via deep neural network with external attention

Optical coherence tomography (OCT) is a promising non-invasive imaging technique that owns many biomedical applications. In this paper, a deep neural network is proposed for enhancing the spatial resolution of OCT en face images. Different from the previous reports, the proposed can recover high-resolution en face images from low-resolution en face images at arbitrary imaging depth. This kind of imaging depth adaptive resolution enhancement is achieved through an external attention mechanism, which takes advantage of morphological similarity between the arbitrary-depth and full-depth en face images. Firstly, the deep feature maps are extracted by a feature extraction network from the arbitrary-depth and full-depth en face images. Secondly, the morphological similarity between the deep feature maps is extracted and utilized to emphasize the features strongly correlated to the vessel structures by using the external attention network. Finally, the SR image is recovered from the enhanced feature map through an up-sampling network. The proposed network is tested on a clinical skin OCT data set and an open-access retinal OCT dataset. The results show that the proposed external attention mechanism can suppress invalid features and enhance significant features in our tasks. For all tests, the proposed SR network outperformed the traditional image interpolation method, e.g. bi-cubic method, and the state-of-the-art image super-resolution networks, e.g. enhanced deep super-resolution network, residual channel attention network, and second-order attention network. The proposed method may increase the quantitative clinical assessment of micro-vascular diseases which is limited by OCT imaging device resolution.

Research on the Performance of Impressionist Painting Color Visual Communication Based on Wireless Communication and Machine Vision

In order to solve the problem of poor visual communication effect of impressionist painting colors and low accuracy of painting color enhancement by existing methods, this thesis proposes a research on the performance of impressionist painting color visual communication based on machine vision under the background of wireless network. This method can improve the vision, speed, and efficiency of communication, through the analysis of the characteristics of impressionist paintings, and determine the visual communication objects of impressionist painting colors. The process of visual communication is analyzed, and the color matching of impressionist painting is completed with the help of BP neural network algorithm. On this basis, the histogram method is used to process the image brightness of impressionist paintings, the image interpolation method is used to process the image brightness of impressionist paintings, and the image is corrected by gamma correction to complete the image performance research. The color vision communication of impressionist painting needs to correct the gray scale error of impressionist painting and enhance the color of impressionist painting. The experimental results show that the accuracy of this method for the color matching of impressionist painting images is about 98%, which has a certain degree of credibility.

Region- and Pixel-Level Multi-Focus Image Fusion through Convolutional Neural Networks

Capturing all-in-focus images with 3D scenes is typically a challenging task due to depth of field limitations, and various multi-focus image fusion methods have been employed to generate all-in-focus images. However, existing methods have difficulty simultaneously achieving real-time and superior fusion performance. In this paper, we propose a region- and pixel-based method that can recognize the focus and defocus regions or pixels by the neighborhood information in the source images. The proposed method can obtain satisfactory fusion results and achieve improved real-time performance. First, a convolutional neural network (CNN)-based classifier generates a coarse region-based trimap quickly, which contains focus, defocus and boundary regions. Then, precise fine-tuning is implemented at the boundary regions to address the boundary pixels that are difficult to discriminate by existing methods. Based on a public database, a high-quality dataset is constructed that provides abundant precise pixel-level labels, so that the proposed method can accurately classify the regions and pixels without artifacts. Furthermore, an image interpolation method called NEAREST_Gaussian is proposed to improve the recognition ability at the boundary. Experimental results show that the proposed method outperforms other state-of-the-art methods in visual perception and object metrics. Additionally, the proposed method has 80% improved to the conventional CNN-based methods.

Effect of the Pixel Interpolation Method for Downsampling Medical Images on Deep Learning Accuracy

Background: High-resolution medical images often need to be downsampled because of the memory limitations of the hardware used for machine learning. Although various image interpolation methods are applicable to downsampling, the effect of data preprocessing on the learning performance of convolutional neural networks (CNNs) has not been fully investigated. Methods: In this study, five different pixel interpolation algorithms (nearest neighbor, bilinear, Hamming window, bicubic, and Lanczos interpolation) were used for image downsampling to investigate their effects on the prediction accuracy of a CNN. Chest X-ray images from the NIH public dataset were examined by downsampling 10 patterns. Results: The accuracy improved with a decreasing image size, and the best accuracy was achieved at 64 × 64 pixels. Among the interpolation methods, bicubic interpolation obtained the highest accuracy, followed by the Hamming window.

A Super-Resolution Model for Improving the Precision of Wafer Mark Alignment

Wafer bonding machine uses industrial camera to recognize marks on top wafer and bottom wafer, compute deviation, and move top wafer to align the bottom wafer. The alignment precision mainly depends on the camera resolution, high resolution industrial camera is expensive, while classical image up-sampling methods such as bicubic interpolation don’t have good effect. To improve the alignment precision, a super-resolution model is proposed. Main component of this model is convolutional neural network. The first two convolutional layers are to extract feature on wafer image, the next convolutional layer is used for nonlinear mapping, and the final one outputs super-resolution image. Peak Signal to Noise Ratio (PSNR) is used to evaluate the similarity of super resolution image and the target high resolution image. It’s proved by experiments that the super-resolution model has better effect than classical image interpolation methods. This research result can also be applied to other equipment using industrial cameras.

Image interpolation with adaptive k ‐nearest neighbours search and random non‐linear regression

Learning-based image interpolation methods have been proved to be effective in image interpolation. In this study, the authors propose an accurate image interpolation with adaptive k-nearest neighbour searching and non-linear regression. The proposed method aims to find k-nearest neighbours of the input image patch and use them to learn the non-linear mapping between low-resolution and high-resolution image patches. To be specific, they first divide the training image patches into many subspaces, then they utilise an adaptive robust and precise k nearest neighbour searching scheme with proposed normalised Gaussian similarity to find the k nearest neighbours in the matched subspace. The selected k image patch pairs are then used to learn the non-linear regression model through an extreme learning machine. Furthermore, the proposed interpolation method is a cascade framework that consists of two stages. Stage 2 takes the results of Stage 1 as input to further improve the performance. Extensive experimental results on commonly used test images and image datasets indicate that their proposed algorithm obtains competitive performance against the state-of-the-art methods both in terms of objective evaluation values and the subjective effect of reconstructed images.

Volumetric interpolation of tomographic sequences for accurate 3D reconstruction of anatomical parts

Background and objective:Tomographic sequences of biomedical images are commonly used to achieve a three-dimensional visualization of the human anatomy. In some cases, the number of images contained in the sequence is limited, e.g., in low-dose computed tomography acquired on neonatal patients, resulting in a coarse and inaccurate 3D reconstruction.Methods:In this paper, volumetric image interpolation methods, devised to increase the axial resolution of tomographic sequences and achieve a refined 3D reconstruction, are proposed and compared. The techniques taken into consideration are based on motion-compensated frame-interpolation concepts, which have been developed for video applications, mainly frame-rate conversion.Results: The performance of the proposed methods is quantitatively assessed by using sequences with a simulated low axial resolution obtained from the decimation of standard high-resolution computed tomography sequences. Real data with an actual low axial resolution have been used as well for a qualitative evaluation of the proposed methods.Conclusions:The experimental results demonstrate that the proposed methods enable an effective slice interpolation and that the achievable 3D models clearly benefit from the increased axial resolution.