Block Matching Research Articles

SHVC performance enhancement using superior step search algorithm

Abstract The SHVC (Scalable High Efficiency Video Coding) standard developed as an extension to H.265 HEVC (High Efficiency Video Coding) that focus on data transmission over heterogeneous network with wide scalability. In this paper a Superior Step Search (SSS) algorithm is proposed to enhance the PSNR performance along with coding efficiency without much increase in the computational complexity. The proposed SSS algorithm performs dynamic search to predict the best matching block using a fine search pattern. Simulation results show 0.5% average increased performance in PSNR and 32% coding efficiency in Bit rate when compared with the SHM reference software.

Quadratic penalty method for intensity-based deformable image registration and 4DCT lung motion recovery.

The purpose of this work is to present a spatially accurate, intensity-based DIR optimization formulation that can be solved with a straightforward gradient-free quadratic penalty algorithm and is suitable for 4D thoracic computed tomography (4DCT) registration. Additionally, a novel regularization strategy based on the well-known leave-one-out robust statistical model cross-validation method is introduced. The proposed Quadratic Penalty DIR (QPDIR) method minimizes both an image dissimilarity term, which is separable with respect to individual voxel displacements, and a regularization term derived from the classical leave-one-out cross-validation statistical method. The resulting DIR problem lends itself to a quadratic penalty function optimization approach, where each subproblem can be solved by straightforward block coordinate descent iteration. The spatial accuracy of the method was assessed using expert-determined landmarks on ten 4DCT datasets available on www.dir-lab.com. The QPDIR algorithm achieved average millimeter spatial errors between 0.69 (0.91) and 1.19 (1.26) on the ten test cases. On all ten 4DCT test cases, the QPDIR method produced spatial accuracies that are superior or equivalent to those produced by current state-of-the-art methods. Moreover, QPDIR achieved accuracies at the resolution of the landmark error assessment (i.e., the interobserver error) on six of the ten cases. The QPDIR algorithm is based on a simple quadratic penalty function formulation and a regularization term inspired by leave-one-out cross validation. The formulation lends itself to a parallelizable, gradient-free, block coordinate descent numerical optimization method. Numerical results indicate that the method achieves a high spatial accuracy on 4DCT inhale/exhale phases.

Improved Parallax Image Stitching Algorithm Based on Feature Block

Image stitching aims at generating high-quality panoramas with the lowest computational cost. In this paper, we present an improved parallax image-stitching algorithm using feature blocks (PIFB), which achieves a more accurate alignment and faster calculation speed. First, each image is divided into feature blocks using an improved fuzzy C-Means (FCM) algorithm, and the characteristic descriptor of each feature block is extracted using scale invariant feature transform (SIFT). The feature matching block of the reference image and the target image are matched and then determined, and the image is pre-registered using the homography calculated by the feature points in the feature block. Finally, the overlapping area is optimized to avoid ghosting and shape distortion. The improved algorithm considering pre-blocking and block stitching effectively reduced the iterative process of feature point matching and homography calculation. More importantly, the problem that the calculated homography matrix was not global has been solved. Ghosting and shape warping are significantly eliminated by re-optimizing the overlap of the image. The performance of the proposed approach is demonstrated using several challenging cases.

Effects of vitamin D supplementation on liver fibrogenic factors in non-alcoholic fatty liver patients with steatohepatitis: study protocol for a randomized clinical trial

BackgroundIt has been suggested that vitamin D and its receptors involve in suppressing fibrogenic signaling in non-alcoholic fatty liver disease (NAFLD). However, the effect of vitamin D supplementation on fibrogenic factors has not been investigated in NAFLD individuals with steatohepatitis. This study was designed to examine the effects on vitamin D supplementation on serum levels of vitamin D receptor (VDR), fibrogenic factors, and fibrogenic microRNAs (MiR) in NAFLD patients.MethodsForty-six NAFLD patients will be recruited in this study. After block matching for sex and BMI, they will be randomly assigned to receive 4000 IU/day vitamin D or placebo for 12 weeks. Weight, height, and waist circumference will be measured. Determination of serum fibrogenic MiRs, laminin, collagen type IV, hyaluronic acid, vitamin D, VDR, calcium, blood glucose, serum insulin, lipid profile, liver markers (ALT, AST, total, direct, and indirect bilirubin) will be done at study baseline and at the end of the trial. Insulin resistance and insulin sensitivity will be determined using the HOMA-IR and QUICKI equation.DiscussionThis is the first randomized controlled trial that will determine the effect of vitamin D supplementation on serum levels of VDR, fibrogenic factors, and fibrogenic MiRs in NAFLD patients. The results of this trial will provide clinical evidence on the effectiveness of vitamin D supplementation in controlling liver fibrosis in NAFLD patients.Trial registrationIranian Registry of Clinical Trials, IRCT201405251485N13. Registered on 14 March 2017.

Dedicated Ultrasound Speckle Tracking for Quantitative Analysis of Uterine Motion Outside Pregnancy.

Fertility problems are nowadays being paralleled by important advances in assisted reproductive technologies. Yet the success rate of these technologies remains low. There is evidence that fertilization outcome is affected by uterine motion, but solutions for quantitative analysis of uterine motion are lacking. This work proposes a dedicated method for uterine-motion quantification by B-mode transvaginal ultrasound. Motion analysis is implemented by speckle tracking based on block matching after speckle-size regularization. Sum of absolute differences is the adopted matching metrics. Prior to the analysis, dedicated singular value decomposition (SVD) filtering is implemented to enhance the uterine motion over noise, clutter, and uncorrelated motion induced by neighboring organs and probe movements. SVD and block matching are first optimized by a dedicated ex vivo setup. Robustness to noise and speckle decorrelation is improved by median filtering of the tracking coordinates from surrounding blocks. Speckle tracking is further accelerated by a diamond search. The method feasibility was tested in vivo with a longitudinal study on nine women, aimed at discriminating between four selected phases of the menstrual cycle known to show different uterine behavior. Each woman was scanned in each phase for 4 min; four sites on the uterine fundus were tracked over time to extract strain and distance signals along the longitudinal and transversal directions of the uterus. Several features were extracted from these signals. Among these features, median frequency and contraction frequency showed significant differences between active and quiet phases. These promising results motivate toward an extended validation in the context of fertilization procedures.

Computational image speckle suppression using block matching and machine learning.

We develop an image despeckling method that combines nonlocal self-similarity filters with machine learning, which makes use of convolutional neural network (CNN) denoisers. It consists of three major steps: block matching, CNN despeckling, and group shrinkage. Through the use of block matching, we can take advantage of the similarity across image patches as a regularizer to augment the performance of data-driven denoising using a pre-trained network. The outputs from the CNN denoiser and the group coordinates from block matching are further used to form 3D groups of similar patches, which are then filtered through a wavelet-domain shrinkage. The experimental results show that the proposed method achieves noticeable improvement compared with state-of-the-art speckle suppression techniques in both visual inspection and objective assessments.

Motion Estimation-Assisted Denoising for an Efficient Combination with an HEVC Encoder.

Noise, which is commonly generated in low-light environments or by low-performance cameras, is a major cause of the degradation of compression efficiency. In previous studies that attempted to combine a denoise algorithm and a video encoder, denoising was used independently of the code for pre-processing or post-processing. However, this process must be tightly coupled with encoding because noise affects the compression efficiency greatly. In addition, this represents a major opportunity to reduce the computational complexity, because the encoding process and a denoise algorithm have many similarities. In this paper, a simple, add-on denoising scheme is proposed through a combination of high-efficiency video coding (HEVC) and block matching three-dimensional collaborative filtering (BM3D) algorithms. It is known that BM3D has excellent denoise performance but that it is limited in its use due to its high computational complexity. This paper employs motion estimation in HEVC to replace the block matching of BM3D so that most of the time-consuming functions are shared. To overcome the challenging algorithmic differences, the hierarchical structure in HEVC is uniquely utilized. As a result, the computational complexity is drastically reduced while the competitive performance capabilities in terms of coding efficiency and denoising quality are maintained.

Reduced-Complexity Intra Block Copy (IntraBC) Mode With Early CU Splitting and Pruning for HEVC Screen Content Coding

A screen content coding (SCC) extension to high efficiency video coding has been developed to incorporate many new coding tools in order to achieve better coding efficiency for videos mixed with camera-captured content and graphics/text/animation. For instance, the Intra Block Copy (IntraBC) mode helps to encode repeating patterns within the same frame while the Palette mode aims at encoding screen content with a few major colors. However, the IntraBC mode brings along high computational complexity due to the exhaustive block matching within the same frame though there are already some constraints and fast approaches applied to the IntraBC mode to reduce its complexity. Thus, we propose a fast intracoding scheme to reduce the complexity of using the IntraBC mode in SCC. Screen content always contains no sensor noise resulting in the characteristics with pixel exactness along both horizontal and vertical directions. These characteristics pave the way for mode skipping and early coding unit (CU) splitting. Besides, early CU pruning and early termination are proposed based on the rate distortion cost to further reduce encoder complexity. Moreover, we also propose reducing the complexity of the IntraBC mode by checking the hash value of each block candidate and the current block during block matching. With our proposed scheme, the encoding time is reduced compared with the SCC while the coding efficiency can still be maintained with a minor increase in the bjontegaard delta bitrate.

Manhattan Distance-Based Adaptive 3D Transform-Domain Collaborative Filtering for Laser Speckle Imaging of Blood Flow.

Laser speckle contrast imaging (LSCI) is a full-field, noncontact imaging technology for mapping blood flow with high spatio-temporal resolution, in which the speckle contrast can be estimated either in spatial domain or temporal domain. Temporal LSCI (tLSCI) provides higher spatial resolution than spatial domain does. However, when the number of sampling frames is limited, it is difficult to obtain accurate blood flow velocity owing to the significant statistical noise. The widely used spatially averaged tLSCI (savg-tLSCI) usually requires a large number of sampling frames to obtain acceptable denoising performance. Here, based on the nonlocal filtering strategy of block-matching and three-dimensional transform-domain collaborative filtering (BM3D), Manhattan distance-based adaptive BM3D (MD-ABM3D) is proposed to effectively manage the complicated inhomogeneous noise in tLSCI image and improve the signal-to-noise ratio. Manhattan distance improves the accuracy of the block matching in strong noise, and the adaptive algorithm adapts to the inhomogeneous noise and estimates suitable parameters for improved denoising. MD-ABM3D improves 4.91 dB in peak signal-to-noise ratio relative to savg-tLSCI. It achieves stability for denoising tLSCI image with different temporal windows. The image-quality evaluation of MD-ABM3D for tLSCI (t = 20 frames) equals that of savg-tLSCI (t = 60 frames). It achieves high signal-to-noise ratio with a reduced number of sampling frames. A reduced number of sampling frames are more practical for biomedical applications. It also offers higher temporal resolution and less disturbance from the motion of the moving object.

Spatial Angular Compounding With Affine-Model-Based Optical Flow for Improvement of Motion Estimation.

Tissue motion estimation is an essential step for ultrasound elastography. Our previous study has shown that the affine-model-based optical flow (OF) method outperforms the normalized cross-correlation-based block matching (BM) method in motion estimation. However, the quality of lateral estimation using OF is still low due to inherent limitation of ultrasound imaging. BM-based spatial angular compounding (SAC) has been developed to obtain better motion estimation. In this paper, OF-based SAC (OF-SAC) is proposed to further improve the performance of lateral (and axial) estimation, and it is compared with BM-based SAC (BM-SAC). Plane wave as well as focused wave is transmitted in both simulations and phantom experiments on a linear array. In order to compare the performance quantitatively, the root-mean-square error (RMSE) of axial/lateral displacement and strain, and signal-to-noise ratio (SNR) and contrast-to-noise ratio (CNR) of axial/lateral strain are used as the evaluation criteria in the simulations. In the phantom experiments, the SNR and CNR are used to assess the quality of axial/lateral strain. The results show that for both OF and BM, SAC improves the performance of motion estimation, regardless of using plane or focused wave transmission. More importantly, OF-SAC is shown to outperform BM-SAC with lower RMSE, higher SNR, and higher CNR. In addition, preliminary in vivo experiments on the carotid artery of a healthy human subject also prove the superiority of OF-SAC. These results suggest that OF-SAC is preferred for both axial and lateral motion estimation to BM-SAC.

Block-based copy–move image forgery detection using DCT

Digital image forensic is a sub-area of multimedia security whose objective is to expose the malicious image manipulations in digital images. Among different types of image forgery, copy–move forgery is the most popular to forge the digital images where a part of the original digital image is copied and pasted at another position in the same image. Different methods have been developed to detect the image forgery in digital images. On the basis of our literature review, we identify that less attention is given to clustering algorithms to speed up the block matching strategy during image forgery detection process. Therefore, to address this issue, we present a pixel-based copy–move image forgery detection method to check the genuineness of digital images. Proposed method includes the following steps: (1) convert the color image into gray-scale image, (2) divide the gray-scale image into overlapping blocks of size 8 × 8, (3) feature extraction using DCT on the basis of different feature sets, (4) block clustering using K-means algorithm, and (5) radix sort for feature matching. Experimental results demonstrate that proposed method can efficiently detect the forged part from digital images.

Hyperspectral Image Classification Using Similarity Measurements-Based Deep Recurrent Neural Networks

Classification is a common objective when analyzing hyperspectral images, where each pixel is assigned to a predefined label. Deep learning-based algorithms have been introduced in the remote-sensing community successfully in the past decade and have achieved significant performance improvements compared with conventional models. However, research on the extraction of sequential features utilizing a single image, instead of multi-temporal images still needs to be further investigated. In this paper, a novel strategy for constructing sequential features from a single image in long short-term memory (LSTM) is proposed. Two pixel-wise-based similarity measurements, including pixel-matching (PM) and block-matching (BM), are employed for the selection of sequence candidates from the whole image. Then, the sequential structure of a given pixel can be constructed as the input of LSTM by utilizing the first several matching pixels with high similarities. The resulting PM-based LSTM and BM-based LSTM are appealing, as all pixels in the whole image are taken into consideration when calculating the similarity. In addition, BM-based LSTM also utilizes local spectral-spatial information that has already shown its effectiveness in hyperspectral image classification. Two common distance measures, Euclidean distance and spectral angle mapping, are also investigated in this paper. Experiments with two benchmark hyperspectral images demonstrate that the proposed methods achieve marked improvements in classification performance relative to the other state-of-the-art methods considered. For instance, the highest overall accuracy achieved on the Pavia University image is 96.20% (using both BM-based LSTM and spectral angle mapping), which is an improvement compared with 84.45% overall accuracy generated by 1D convolutional neural networks.

A Hardware-Efficient Block Matching Algorithm and Its Hardware Design for Variable Block Size Motion Estimation in Ultra-High-Definition Video Encoding

Variable block size motion estimation has contributed greatly to achieving an optimal interframe encoding, but involves high computational complexity and huge memory access, which is the most critical bottleneck in ultra-high-definition video encoding. This article presents a hardware-efficient block matching algorithm with an efficient hardware design that is able to reduce the computational complexity of motion estimation while providing a sustained and steady coding performance for high-quality video encoding. A three-level memory organization is proposed to reduce memory bandwidth requirement while supporting a predictive common search window. By applying multiple search strategies and early termination, the proposed design provides 1.8 to 3.7 times higher hardware efficiency than other works. Furthermore, on-chip memory has been reduced by 96.5% and off-chip bandwidth requirement has been reduced by 39.4% thanks to the proposed three-level memory organization. The corresponding power consumption is only 198mW at the highest working frequency of 500MHz. The proposed design is attractive for high-quality video encoding in real-time applications with low power consumption.

Feature extraction algorithm for fast moving pedestrians with frame drop constraint based on deep learning

When the existing method extracts the information of the fast moving pedestrian, the frame dropping phenomenon may occur, resulting in low extraction precision. A fast moving pedestrian frame loss constrained feature extraction algorithm based on depth tilt is proposed. Block matching and denoising are performed on the pedestrian image. The contour feature extraction method is used to reconstruct the adjacent frames and the reconstructed image frame vector is sub-block fusion. The depth learning algorithm is used to extract the feature quantity of the gray pixel from the frame falling part of the image. Improved feature extraction algorithm for pedestrians with frame loss constraints. The simulation results show that the standard deviation of the frame loss of the extraction result is 8.235 and the standard deviation of the non-drop frame is 4.353. It proves that the algorithm has low frame loss rate and high extraction and recognition ability.

Disparity Map Computation from Stereo Images Using Hill-Climbing Segmentation

Stereo matching is one of the most active research areas in computer vision for decades. The task of stereo matching is to find the point correspondence between two images of the same scene taken from different viewpoints. This paper presents a segment-based stereo matching algorithm. Firstly, the reference image is segmented using hill-climbing algorithm and local stereo matching is performed Scale Invariant Feature Transform (SIFT) feature points with Sum of Absolute Differences (SAD) block matching. Secondly, a set of reliable pixels is constructed by comparing the matching cost and the mutual cross-checking consistent between the left and right initial disparity maps, which can lead to an actual disparity plane. Thirdly, a set of all possible disparity planes are extracted and then plane fitting and neighboring segment merging are performed. Finally, the disparity planes are set in each region using graph cuts to obtain final disparity map. The evaluation of proposed algorithm on the Middlebury data set result shows that the proposed algorithm is competitive with state-of-the-art stereo matching algorithms.

Feature extraction algorithm for fast moving pedestrians with frame drop constraint based on deep learning

When the existing method extracts the information of the fast moving pedestrian, the frame dropping phenomenon may occur, resulting in low extraction precision. A fast moving pedestrian frame loss constrained feature extraction algorithm based on depth tilt is proposed. Block matching and denoising are performed on the pedestrian image. The contour feature extraction method is used to reconstruct the adjacent frames and the reconstructed image frame vector is sub-block fusion. The depth learning algorithm is used to extract the feature quantity of the gray pixel from the frame falling part of the image. Improved feature extraction algorithm for pedestrians with frame loss constraints. The simulation results show that the standard deviation of the frame loss of the extraction result is 8.235 and the standard deviation of the non-drop frame is 4.353. It proves that the algorithm has low frame loss rate and high extraction and recognition ability.

Enhancing 3-D Seismic Data Using the t-SVD and Optimal Shrinkage of Singular Value

We consider the three-dimensional (3-D) seismic data as a tensor data of size ${n_1} \times {n_2} \times {n_3}$ contaminated with the white Gaussian noise. A new version of the tensor robust principal component analysis (TRPCA) is employed for denoising the 3-D seismic data. In the new TRPCA, the singular values are extracted using the optimal shrinkage method. We recover the low-rank matrices from noisy data by shrinkage of the singular values, in which the singular value thresholding in the Fourier domain is exploited to extract the low-rank component of the tensor. The algorithm is as follows. First, by assuming the incoherency conditions the whole tensor is modeled as a combination of a low-rank component and a sparse component. Second, the Fourier transform of the tensor is computed along the third dimension of the tensor, then the singular value decomposition (SVD) is computed in the Fourier domain, then the low-rank component is extracted by shrinkage of the singular values. Finally, the steps mentioned above are repeated until the Frobenius norm of the error matrix reaches the desired value. We evaluate the performance of the proposed method, which is called tensor optimal shrinkage of SVD based on the qualitative and quantitative measurements, and compare it with state-of-the-art methods such as iterative tensor singular value thresholding and 4-D block matching using different types of synthetic and real seismic data.

Comparison of PANAMutyper and PNAClamp for Detecting KRAS Mutations from Patients With Malignant Pleural Effusion.

KRAS is one of the frequently mutated genes in human cancers and often relates with drug resistance and poor prognosis. PANAMutyper™ is a novel technology that integrates PNAClamp™ and PANA S-Melting™. In the present study, PANAMutyper™ and PNAClamp™ were compared for the detection of KRAS mutations using different samples of patients with malignant pleural effusion. A total of 103 patients (including 56 lung adenocarcinoma, 10 lung squamous carcinoma, 17 small cell lung cancer, 3 large cell lung cancer, 3 stomach cancer, 2 ovarian cancer, and others) with malignant pleural effusion were investigated using matched tumor tissue, cell block, and pleural effusion samples. The diagnostic performance of these two methods was compared. KRAS mutations were detected in 18 (17.5%) of 103 patients using tissue, cell block, and pleural effusion samples. All 18 patients with KRAS mutations were detected by PANAMutyper™ using any sample type, however, only 7 cases were detected by PNAClamp™. Among the subtypes of KRAS mutations, substitution in codon 12, 35G>T was the most frequent, followed by substitution in codon 12, 35G>A and codon 12, 34G>A. In pleural effusion specimens, PANAMutyper™ showed a better diagnostic performance compared to PNAClamp™. PANAMutyper™ had a diagnostic superiority for the detection of KRAS mutations in patients with malignant pleural effusion compared to PNAClamp™, although there was a concordance between PANAMutyper™ and PNAClamp™ results. Therefore, PANAMutyper™ can be used for a more sensitive and accurate detection of KRAS mutations.

An Improved Weighted Nuclear Norm Minimization Method for Image Denoising

Patch-based low rank matrix approximation has shown great potential in image denoising. Among state-of-the-art methods in this topic, the weighted nuclear norm minimization (WNNM) has been attracting significant attention due to its competitive denoising performance. For each local patch in an image, the WNNM method groups nonlocal similar patches by block matching to formulate a low-rank matrix. However, the WNNM often chooses irrelevant patches such that it may lose fine details of the image, resulting in undesirable artifacts in the final reconstruction. In this regards, this paper aims to provide a denoising algorithm which further improves the performance of the WNNM method. For this purpose, we develop a new nonlocal similarity measure by exploiting both pixel intensities and gradients and present a filter that enhances edge information in a patch to improve the performance of low rank approximation. The experimental results on widely used test images demonstrate that the proposed denoising algorithm performs better than other state-of-the-art denoising algorithms in terms of PSNR and SSIM indices as well as visual quality.

Location-Based Seeds Selection for Influence Blocking Maximization in Social Networks

Influence blocking maximization (IBM) is a key problem for viral marketing in competitive social networks. Although the IBM problem has been extensively studied, existing works neglect the fact that the location information can play an important role in influence propagation. In this paper, we study the location-based seeds selection for IBM problem, which aims to find a positive seed set in a given query region to block the negative influence propagation in a given block region as much as possible. In order to overcome the low efficiency of the simulation-based greedy algorithm, we propose a heuristic algorithm IS-LSS and its improved version IS-LSS+, both of which are based on the maximum influence arborescence structure and Quadtree index, while IS-LSS+ further improves the efficiency of IS-LSS by using an upper bound method and Quadtree cell lists. The experimental results on real-world datasets demonstrate that our proposed algorithms are able to achieve matching blocking effect to the greedy algorithm as the increase in the number of positive seeds and often better than other heuristic algorithms, whereas they are four orders of magnitude faster than the greedy algorithm.