Block Matching 3D Research Articles

Visualizing and quantifying small and nonstationary structural motions in video measurement

AbstractRapid advancement in vision recording technologies is increasing the importance and production of video data in a wide range of applications. This paper proposed a novel perspective of multifrequency phase inference for characterizing especially challenging nonstationary and often small motions in optical measurement. The model estimates and adjusts the phase information by the multi‐frequency phase retrieval, which is derived from the maximum likelihood formulation with block matching 3D sparsity priors. Estimated phase jumps are removed by a robust solution of the 2D phase unwrapping problem. These considerations are supported by applications of dynamic response identification in structural health monitoring. When compared to state‐of‐the‐art techniques, the proposed method readily yielded high‐quality magnifications on real videos, with less noise and better anti‐noise performance. The proposed method also demonstrated uniformly high skill in extracting clearer time‐domain motion estimation of video components.

Dilated Convolution ResNet with Boosting Attention Modules and Combined Loss Functions for LDCT Image Denoising.

With the increasing concern regarding the radiation exposure of patients undergoing computed tomography (CT) scans, researchers have been using deep learning techniques to improve the quality of denoised low-dose CT (LDCT) images. In this paper, a cascaded dilated residual network (ResNet) with integrated attention modules, specifically spatial- and channel- attention modules, is proposed. This experiment demonstrated how these attention modules improved the denoised CT image by testing a simple ResNet with and without the modules. Further, an investigation regarding the effectiveness of per-pixel loss, perceptual loss via VGG16-Net, and structural dissimilarity loss functions is also covered through an ablation experiment. By knowing how these loss functions affect the output denoised images, a combination of the these loss function is then proposed which aims to prevent edge over-smoothing, enhance textural details and finally, preserve structural details on the denoised images. Finally, a bench testing was also done by comparing the visual and quantitative results of the proposed model with the state-of-the-art models such as block matching 3D (BM3D), patch-GAN and dilated convolution with edge detection layer (DRL-E-MP) for accuracy.

Performance of low-dose CT image reconstruction for detecting intracerebral hemorrhage: selection of dose, algorithms and their combinations

To investigate the performance of different low-dose CT image reconstruction algorithms for detecting intracerebral hemorrhage. Low-dose CT imaging simulation was performed on CT images of intracerebral hemorrhage at 30%, 25% and 20% of normal dose level (defined as 100% dose). Seven algorithms were tested to reconstruct low-dose CT images for noise suppression, including filtered back projection algorithm (FBP), penalized weighted least squares-total variation (PWLS-TV), non-local mean filter (NLM), block matching 3D (BM3D), residual encoding-decoding convolutional neural network (REDCNN), the FBP convolutional neural network (FBPConvNet) and image restoration iterative residual convolutional network (IRLNet). A deep learning-based model (CNN-LSTM) was used to detect intracerebral hemorrhage on normal dose CT images and low-dose CT images reconstructed using the 7 algorithms. The performance of different reconstruction algorithms for detecting intracerebral hemorrhage was evaluated by comparing the results between normal dose CT images and low-dose CT images. At different dose levels, the low-dose CT images reconstructed by FBP had accuracies of detecting intracerebral hemorrhage of 82.21%, 74.61% and 65.55% at 30%, 25% and 20% dose levels, respectively. At the same dose level (30% dose), the images reconstructed by FBP, PWLS-TV, NLM, BM3D, REDCNN, FBPConvNet and IRLNet algorithms had accuracies for detecting intracerebral hemorrhage of 82.21%, 86.80%, 89.37%, 81.43%, 90.05%, 90.72% and 93.51%, respectively. The images reconstructed by IRLNet at 30%, 25% and 20% dose levels had accuracies for detecting intracerebral hemorrhage of 93.51%, 93.51% and 93.06%, respectively. The performance of reconstructed low-dose CT images for detecting intracerebral hemorrhage is significantly affected by both dose and reconstruction algorithms. In clinical practice, choosing appropriate dose level and reconstruction algorithm can greatly reduce the radiation dose and ensure the detection performance of CT imaging for intracerebral hemorrhage.

Low Dose CT Denoising by ResNet With Fused Attention Modules and Integrated Loss Functions

X-ray computed tomography (CT) is a non-invasive medical diagnostic tool that has raised public concerns due to the associated health risks of radiation dose to patients. Reducing the radiation dose leads to noise artifacts, making the low-dose CT images unreliable for diagnosis. Hence, low-dose CT (LDCT) image reconstruction techniques have offered a new research area. In this study, a deep neural network is proposed, specifically a residual network (ResNet) using dilated convolution, batch normalization, and rectified linear unit (ReLU) layers with fused spatial- and channel-attention modules to enhance the quality of LDCT images. The network is optimized using the integration of per-pixel loss, perceptual loss via VGG16-net, and dissimilarity index loss. Through an ablation experiment, these functions show that they could effectively prevent edge oversmoothing, improve image texture, and preserve the structural details. Finally, comparative experiments showed that the qualitative and quantitative results of the proposed network outperform state-of-the-art denoising models such as block-matching 3D filtering (BM3D), Markovian-based patch generative adversarial network (patch-GAN), and dilated residual network with edge detection (DRL-E-MP).

In Vivo Ultrasound Localization Microscopy Imaging of the Kidney's Microvasculature With Block-Matching 3-D Denoising.

Structural abnormalities and functional changes of renal microvascular networks play a significant pathophysiologic role in the occurrence of kidney diseases. Super-resolution ultrasound imaging has been successfully utilized to visualize the microvascular network and provide valuable diagnostic information. To prevent the burst of microbubbles, a lower mechanical index (MI) is generally used in ultrasound localization microscopy (ULM) imaging. However, high noise levels lead to incorrect signal localizations in relatively low-MI settings and deep tissue. In this study, we implemented a block-matching 3-D (BM3D) image-denoising method, after the application of singular value decomposition filtering, to further suppress the noise at various depths. The in vitro flow-phantom results show that the BM3D method helps the significant reduction of the error localizations, thus improving the localization accuracy. In vivo rhesus macaque experiments help conclude that the BM3D method improves the resolution more than other image-based denoising techniques, such as the nonlocal means method. The obtained clutter-filtered images with fewer incorrect localizations can enable robust ULM imaging, thus helping in establishing an effective diagnostic tool.

Real Image Denoising With a Locally-Adaptive Bitonic Filter.

Image noise removal is a common problem with many proposed solutions. The current standard is set by learning-based approaches, however these are not appropriate in all scenarios, perhaps due to lack of training data or the need for predictability in novel circumstances. The bitonic filter is a non-learning-based filter for removing noise from signals, with a mathematical morphology (ranking) framework in which the signal is postulated to be locally bitonic (having only one minimum or maximum) over some domain of finite extent. A novel version of this filter is developed in this paper, with a domain that is locally-adaptive to the signal, and other adjustments to allow application to real image sensor noise. These lead to significant improvements in noise reduction performance at no cost to processing times. The new bitonic filter performs better than the block-matching 3D filter for high levels of additive white Gaussian noise. It also surpasses this and other more recent non-learning-based filters for two public data sets containing real image noise at various levels. This is despite an additional adjustment to the block-matching filter, which leads to significantly better performance than has previously been cited on these data sets. The new bitonic filter has a signal-to-noise ratio 2.4dB lower than the best learning-based techniques when they are optimally trained. However, the performance gap is closed completely when these techniques are trained on data sets not directly related to the benchmark data. This demonstrates what can be achieved with a predictable, explainable, entirely local technique, which makes no assumptions of repeating patterns either within an image or across images, and hence creates residual images which are well behaved even in very high noise. Since the filter does not require training, it can still be used in situations where training is either difficult or inappropriate.

Speckle noise reduction in coherent imaging systems via hybrid median–mean filter

Images recorded by coherent imaging systems, including laser-based photography, digital holography (DH), and digital holographic microscopy (DHM), are severely distorted by speckle noise. We present a single-shot image processing method to reduce the speckle noise, coined hybrid median–mean filter (HM2F), which is based on the average of conventional median-filtered images with different kernel sizes. The synergic combination of the median filter and mean approach provides a denoised image with reduced speckle contrast while the spatial resolution is kept at up to 97% of the original value. The HM2F method is compared with the conventional median filter approach, the 3D block matching filter, the nonlocal means filter, the 2D windowed Fourier transform filter, and the Wiener filter using different speckle-distorted images to benchmark its performance. Based on the experimental results and the simplicity of the technique, HM2F is proposed as an effective denoising tool for reducing the speckle noise in laser-based photography, DH, and DHM.

A new HSI denoising method via interpolated block matching 3D and guided filter.

A new hyperspectral images (HSIs) denoising method via Interpolated Block-Matching and 3D filtering and Guided Filtering (IBM3DGF) denoising method is proposed. First, inter-spectral correlation analysis is used to obtain inter-spectral correlation coefficients and divide the HSIs into several adjacent groups. Second, high-resolution HSIs are produced by using adjacent three images to interpolate. Third, Block-Matching and 3D filtering (BM3D) is conducted to reduce the noise level of each group; Fourth, the guided image filtering is utilized to denoise HSI of each group. Finally, the inverse interpolation is applied to retrieve HSI. Experimental results of synthetic and real HSIs showed that, comparing with other state-of-the-art denoising methods, the proposed IBM3DGF method shows superior performance according to spatial and spectral domain noise assessment. Therefore, the proposed method has a potential to effectively remove the spatial/spectral noise for HSIs.

Detect and defense against adversarial examples in deep learning using natural scene statistics and adaptive denoising

Despite the enormous performance of deep neural networks (DNNs), recent studies have shown their vulnerability to adversarial examples (AEs), i.e., carefully perturbed inputs designed to fool the targeted DNN. Currently, the literature is rich with many effective attacks to craft such AEs. Meanwhile, many defense strategies have been developed to mitigate this vulnerability. However, these latter showed their effectiveness against specific attacks and does not generalize well to different attacks. In this paper, we propose a framework for defending DNN classifier against adversarial samples. The proposed method is based on a two-stage framework involving a separate detector and a denoising block. The detector aims to detect AEs by characterizing them through the use of natural scene statistic (NSS), where we demonstrate that these statistical features are altered by the presence of adversarial perturbations. The denoiser is based on block matching 3D (BM3D) filter fed by an optimum threshold value estimated by a convolutional neural network (CNN) to project back the samples detected as AEs into their data manifold. We conducted a complete evaluation on three standard datasets, namely MNIST, CIFAR-10 and Tiny-ImageNet. The experimental results show that the proposed defense method outperforms the state-of-the-art defense techniques by improving the robustness against a set of attacks under black-box, gray-box and white-box settings. The source code is available at: https://github.com/kherchouche-anouar/2DAE.

Removing multi-frame Gaussian noise by combining patch-based filters with optical flow

Patch-based approaches such as 3D block matching (BM3D) and non-local Bayes (NLB) are widely accepted filters for removing Gaussian noise from single-frame images. In this work, we propose three extensions for these filters when there exist multiple frames of the same scene. The first of them employs reference patches on every frame instead of a commonly used single reference frame method, thus utilizing the complete available information. The remaining two techniques use a separable spatio-temporal filter to reduce interactions between dissimilar regions, hence mitigating artifacts. In order to deal with non-registered datasets we combine all our extensions with robust optical flow computation. Two of our proposed multi-frame filters outperform existing extensions on most occasions by a significant margin while also being competitive with a state-of-the-art neural network-based technique. Moreover, one of these two strategies is the fastest among all due to its separable design.

Multi-Scale Superpixels Dimension Reduction Hyperspectral Image Classification Algorithm Based on Low Rank Sparse Representation Joint Hierarchical Recursive Filtering.

The original Hyperspectral image (HSI) has different degrees of Hughes phenomenon and mixed noise, leading to the decline of classification accuracy. To make full use of the spatial-spectral joint information of HSI and improve the classification accuracy, a novel dual feature extraction framework joint transform domain-spatial domain filtering based on multi-scale-superpixel-dimensionality reduction (LRS-HRFMSuperPCA) is proposed. Our framework uses the low-rank structure and sparse representation of HSI to repair the unobserved part of the original HSI caused by noise and then denoises it through a block-matching 3D algorithm. Next, the dimension of the reconstructed HSI is reduced by principal component analysis (PCA), and the dimensions of the reduced images are segmented by multi-scale entropy rate superpixels. All the principal component images with superpixels are projected into the reconstructed HSI in parallel. Secondly, PCA is once again used to reduce the dimension of all HSIs with superpixels in scale with hyperpixels. Moreover, hierarchical domain transform recursive filtering is utilized to obtain the feature images; ultimately, the decision fusion strategy based on a support vector machine (SVM) is used for classification. According to the Overall Accuracy (OA), Average Accuracy (AA) and Kappa coefficient on the three datasets (Indian Pines, University of Pavia and Salinas), the experimental results have shown that our proposed method outperforms other state-of-the-art methods. The conclusion is that LRS-HRFMSuperPCA can denoise and reconstruct the original HSI and then extract the space-spectrum joint information fully.

A Novel Method for Effective Cell Segmentation and Tracking in Phase Contrast Microscopic Images.

Cell migration plays an important role in the identification of various diseases and physiological phenomena in living organisms, such as cancer metastasis, nerve development, immune function, wound healing, and embryo formulation and development. The study of cell migration with a real-time microscope generally takes several hours and involves analysis of the movement characteristics by tracking the positions of cells at each time interval in the images of the observed cells. Morphological analysis considers the shapes of the cells, and a phase contrast microscope is used to observe the shape clearly. Therefore, we developed a segmentation and tracking method to perform a kinetic analysis by considering the morphological transformation of cells. The main features of the algorithm are noise reduction using a block-matching 3D filtering method, k-means clustering to mitigate the halo signal that interferes with cell segmentation, and the detection of cell boundaries via active contours, which is an excellent way to detect boundaries. The reliability of the algorithm developed in this study was verified using a comparison with the manual tracking results. In addition, the segmentation results were compared to our method with unsupervised state-of-the-art methods to verify the proposed segmentation process. As a result of the study, the proposed method had a lower error of less than 40% compared to the conventional active contour method.

Fourier ptychographic reconstruction based on augmented Lagrangian method and sparse approximations for phase and magnitude

Fourier ptychography microscopy provides a large field of view and high-resolution imaging by simultaneously recovering intensity and phase distributions. However, in real setups, the process of capturing large numbers of low-resolution images will inevitably suffer from imaging noise, which could seriously distort the results recovered using the conventional Fourier ptychography approach. To suppress the effects of imaging noise optimally, a novel, to the best of our knowledge, iterative algorithm is proposed. This algorithm consists of two objective functions; one is based on the augmented Lagrangian function for the inverse computation, and its solution is found by utilizing the alternating direction multiplier method; the other is the separate sparse model built for amplitude and absolute phase image; the filtering process is accomplished by exploiting the block-matching 3D frames. In combination with the Nash equilibrium balancing theory, the proposed algorithm is realized by alternately optimizing the two objective functions. The simulated and experimental results demonstrate that the proposed algorithm is robust to noise and is capable of reconstructing complete and good contrast amplitude and phase images.

Frequency division denoising algorithm based on VIF adaptive 2D-VMD ultrasound image.

Ultrasound imaging has developed into an indispensable imaging technology in medical diagnosis and treatment applications due to its unique advantages, such as safety, affordability, and convenience. With the development of data information acquisition technology, ultrasound imaging is increasingly susceptible to speckle noise, which leads to defects, such as low resolution, poor contrast, spots, and shadows, which affect the accuracy of physician analysis and diagnosis. To solve this problem, we proposed a frequency division denoising algorithm combining transform domain and spatial domain. First, the ultrasound image was decomposed into a series of sub-modal images using 2D variational mode decomposition (2D-VMD), and adaptively determined 2D-VMD parameter K value based on visual information fidelity (VIF) criterion. Then, an anisotropic diffusion filter was used to denoise low-frequency sub-modal images, and a 3D block matching algorithm (BM3D) was used to reduce noise for high-frequency images with high noise. Finally, each sub-modal image was reconstructed after processing to obtain the denoised ultrasound image. In the comparative experiments of synthetic, simulation, and real images, the performance of this method was quantitatively evaluated. Various results show that the ability of this algorithm in denoising and maintaining structural details is significantly better than that of other algorithms.

Weighted Schatten p-Norm Low Rank Error Constraint for Image Denoising.

Traditional image denoising algorithms obtain prior information from noisy images that are directly based on low rank matrix restoration, which pays little attention to the nonlocal self-similarity errors between clear images and noisy images. This paper proposes a new image denoising algorithm based on low rank matrix restoration in order to solve this problem. The proposed algorithm introduces the non-local self-similarity error between the clear image and noisy image into the weighted Schatten p-norm minimization model using the non-local self-similarity of the image. In addition, the low rank error is constrained by using Schatten p-norm to obtain a better low rank matrix in order to improve the performance of the image denoising algorithm. The results demonstrate that, on the classic data set, when comparing with block matching 3D filtering (BM3D), weighted nuclear norm minimization (WNNM), weighted Schatten p-norm minimization (WSNM), and FFDNet, the proposed algorithm achieves a higher peak signal-to-noise ratio, better denoising effect, and visual effects with improved robustness and generalization.

Towards Automated Eye Diagnosis: An Improved Retinal Vessel Segmentation Framework Using Ensemble Block Matching 3D Filter.

Automated detection of vision threatening eye disease based on high resolution retinal fundus images requires accurate segmentation of the blood vessels. In this regard, detection and segmentation of finer vessels, which are obscured by a considerable degree of noise and poor illumination, is particularly challenging. These noises include (systematic) additive noise and multiplicative (speckle) noise, which arise due to various practical limitations of the fundus imaging systems. To address this inherent issue, we present an efficient unsupervised vessel segmentation strategy as a step towards accurate classification of eye diseases from the noisy fundus images. To that end, an ensemble block matching 3D (BM3D) speckle filter is proposed for removal of unwanted noise leading to improved detection. The BM3D-speckle filter, despite its ability to recover finer details (i.e., vessels in fundus images), yields a pattern of checkerboard artifacts in the aftermath of multiplicative (speckle) noise removal. These artifacts are generally ignored in the case of satellite images; however, in the case of fundus images, these artifacts have a degenerating effect on the segmentation or detection of fine vessels. To counter that, an ensemble of BM3D-speckle filter is proposed to suppress these artifacts while further sharpening the recovered vessels. This is subsequently used to devise an improved unsupervised segmentation strategy that can detect fine vessels even in the presence of dominant noise and yields an overall much improved accuracy. Testing was carried out on three publicly available databases namely Structured Analysis of the Retina (STARE), Digital Retinal Images for Vessel Extraction (DRIVE) and CHASE_DB1. We have achieved a sensitivity of 82.88, 81.41 and 82.03 on DRIVE, SATARE, and CHASE_DB1, respectively. The accuracy is also boosted to 95.41, 95.70 and 95.61 on DRIVE, SATARE, and CHASE_DB1, respectively. The performance of the proposed methods on images with pathologies was observed to be more convincing than the performance of similar state-of-the-art methods.

The application of plug-and-play ADMM framework and BM3D denoiser for compressed sensing MR image reconstruction

Compressed Sensing Magnetic Resonance Imaging (CS-MRI) is an effective technique to reduce MRI data acquisition time. There is currently growing interest in using Alternating Direction Method of Multiplier (ADMM) for CS-MRI reconstruction. In this paper, we propose a flexible plug-and-play framework to incorporate the block matching 3D (BM3D) denoising algorithm as prior into the plug-and-play ADMM reconstruction procedure for CS-MRI reconstruction, termed BM3D Plug-and-play ADMM (BPA) method. We investigated the performance of the proposed BPA method for the construction of highly under-sampled MRI data of two different sampling masks. Compared with other widely used CS-MRI reconstruction methods, such as, PANO, BM3D-IT, BM3D-MRI and BM3D-AMP-MRI, the proposed framework can reconstruct highly under-sampled CS-MRI data with improved gains in Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM).

Research on X‐Ray Inspection of Basin Insulators and Wireless Image Sensing Technology

This paper presents an in‐depth study and analysis of X‐ray inspection of basin insulators by wireless sensing technology. Aiming at the characteristics of low contrast and many kinds of noise in the basin insulator ray image, this paper proposes an X‐ray basin insulator image denoising method based on improved 3D block matching. Using the RF microcontroller CC2530 chip as the core hardware and networked by ZigBee protocol, the sensor senses and collects various parameters and transmits this information to the monitoring end in real time through wireless. The method proposes an improved wavelet thresholding denoising method to overcome the pseudo‐Gibbs phenomenon caused by the wavelet hard thresholding method in the 3D block matching algorithm cofiltering and retain more details of the image. Aiming at the ringing effect caused by the Wiener filtering method used in the three‐dimensional block matching algorithm collaborative filtering, an improved Kalman filtering method based on anisotropic diffusion is proposed, which avoids the ringing effect, and has clear edges and complete details. An improved Kalman filtering method based on anisotropic diffusion is proposed to avoid the ringing effect, and the edges are clear, and the details are complete. The proposed method is a safe, efficient, accurate, and feasible method for detecting defects in basin insulators by combining X‐ray and improved wireless image sensing technology to detect the internal equipment without disassembling or touching the GIS equipment.

Poisson-Gaussian Noise Reduction for X-Ray Images Based on Local Linear Minimum Mean Square Error Shrinkage in Nonsubsampled Contourlet Transform Domain

Noise reduction is important for X-ray images because it can reduce radiation exposure to patients. X-ray image noise has a Poisson-Gaussian distribution, and recently, noise analysis and removal in multiscale transformations have been widely implemented. The nonsubsampled contourlet transform (NSCT) is a multiscale transformation suitable for medical images that separates the scale and direction. This study proposes a Poisson-Gaussian noise-removal method using NSCT shrinkage that is based on the characteristics of Poisson-Gaussian noise in NSCT domain. It has the structure of a block-matching 3D filtering algorithm in the form of basic estimation and noise removal process; however, the main processes are modified to consider Poisson-Gaussian noise characteristics. In the basic estimation process, an NSCT shrinkage method that is suitable for Poisson-Gaussian noise characteristics is developed by optimizing the local linear minimum mean square error estimator in the NSCT domain. In the denoising step, the noise term of the Wiener filter is determined using the result of the NSCT shrinkage, and finally, the denoised image is obtained. The proposed method is applied to simulated and real X-ray images and is compared with other state-of-the-art Poisson-Gaussian noise removal methods; it exhibits excellent results in both quantitative and qualitative aspects.

Three-Dimensional Mirror Surface Measurement Based on Local Blur Analysis of Phase Measuring Deflectometry System

Despite the marked progress in recent years, structured light-based three-dimensional (3D) measurement techniques still have difficulty in capturing mirror surface reflection. The accuracy of 3D reconstruction for mirror objects should be further improved to adapt to the high reflectivity and curvature of such objects. To improve the stripe definition and reconstruction accuracy of highly reflective mirror objects, this paper analyzes the local blur of defocus stripes in phase measuring deflectometry (PMD) system, and presents a method to analyze the spatially varying defocusing and de-blurring, with the aid of a 3D block matching algorithm, thereby focusing on defocus stripes. Experimental results show that the proposed method can achieve micron-level reconstruction accuracy of standard flat mirrors, and detect the defects on highly reflective mirror objects at a high precision.