Block Matching Research Articles

LOW COMPLEXITY HEVC SCALABLE ENCODER BASED ON FSS ALGORITHM

High efficiency video coding (HEVC) provides compressed video bit streams with good quality video that offers an extension of its kind-Scalable extension of high efficiency video coding (SHVC). SHVC offers delivery of compressed video bit streams over various types of networks with broader scalability. Mode decision and motion search in standard HEVC is more exhaustive and complex, and it is even more complex in the HEVC scalable extension. The encoder time increases due to the computational complexity in SHVC to find the best mode in the search algorithm. To overcome this, a forward-looking step search (FSS) algorithm is proposed to offer less computational complexity with acceptable coding efficiency. The FSS algorithm searches for the optimal matching block by fixing nine points around the assumed center point. The optimal block is determined by forward-looking each point that has minimal rate distortion cost (RDC). The simulation results show that the algorithm works better with a 0.5% increase in the peak signal-to-noise ratio (PSNR) and an encoder time savings of 21.16% with the standard SHVC SHM software.

Omnidirectional Stereo Vision Study from Vertical and Horizontal Stereo Configuration

In stereo vision, an omnidirectional camera has high distortion compared to a standard camera, so the camera calibration is very decisive in its stereo matching. In this study, we will perform stereo matching for an omnidirectional camera with vertical and horizontal configuration so that the result of the image's depth has a 360-degree field of view by transforming the image using a calibration-based method. The result is that by using a vertical camera configuration, the image can be stereo matched directly, but by configuring a horizontal image, it is necessary to carry out a different stereo-matching process in each direction. Stereo matching with the semi-global matching method has better image results than block matching with more image objects detectable by the semi-global block matching method with a maximum disparity value of 32 pixels and with a window size of 21 pixels.

EANet: Multiscale autoencoder based edge attention network for fluid segmentation from SD‐OCT images

AbstractMacular edema is a retinal complication that occurs due to the presence of excess fluid between the retinal layers. This might lead to swelling in the retina and cause severe vision impairment if not detected in its early stages. This paper presents a robust Edge Attention network (EANet) for segmenting the different retinal fluids like Intraretinal Fluid (IRF), Subretinal Fluid (SRF), and Pigment Epithelial Detachment (PED) from the Spectral Domain – Optical Coherence Tomography (SD‐OCT) images. The proposed method employs a novel image enhancement technique by filtering OCT images using a BM3D (Block Matching and 3D Filtering) filter followed by Contrast Limited Adaptive Histogram Equalization (CLAHE) and a linear filter based on multivariate Taylor series to acquire the edge maps of the OCT images. A novel autoencoder based multiscale attention mechanism is incorporated with EANet that feeds on both the OCT image and edge‐enhanced OCT image at every level of the encoder. The proposed network, EANet, has been trained and tested for the segmentation of all three types of fluids on the RETOUCH challenge dataset, and the segmentation of the IRF on the OPTIMA challenge and DUKE DME datasets. The average dice coefficient of IRF, SRF, and PED for the RETOUCH dataset is 0.683, 0.873, and 0.756, respectively, whereas it is 0.805, 0.77, and 0.756 for Cirrus, Spectralis, and Topcon vendors, respectively. The proposed method outperformed all the teams that participated in the OPTIMA challenge on all types of vendor images in terms of dice coefficient. The average dice coefficients of IRF on the OPTIMA and DUKE DME datasets are 0.84 and 0.72, respectively.

Image Texture as Quality Indicator for Optical DEM Generation: Geomorphic Applications in the Arid Central Andes

The generation of Digital Elevation Models (DEMs) through stereogrammetry of optical satellite images has gained great popularity across various disciplines. For the analysis of these DEMs, it is important to understand the influence of the input data and different processing steps and parameters employed during stereo correlation. Here, we explore the effects that image texture, as well as the use of different matching algorithms (Block Matching (BM) and More Global Matching (MGM)), can have on optical DEMs derived from the flexible, open-source Ames Stereo Pipeline. Our analysis relies on a ∼2700 km2 clip of a SPOT6 tristereo scene covering the hyperarid, vegetation-free Pocitos Basin and adjacent mountain ranges in the northwestern Argentine Andes. A large, perfectly flat salt pan (paleolake bed) that covers the center of this basin is characterized by strong contrasts in image texture, providing a unique opportunity to quantitatively study the relationship between image texture and DEM quality unaffected by topography. Our findings suggest that higher image texture, measured by panchromatic variance, leads to lower DEM uncertainty. This improvement continues up to ∼103 panchromatic variance, above which further improvements in DEM quality are independent of local image texture but instead may have sensor or geometric origins. Based on this behavior, we propose that image texture may serve as an important proxy of DEM quality prior to stereo correlation and can help to set adequate processing parameters. With respect to matching algorithms, we observe that MGM improves matching in low-texture areas and overall generates a smoother surface that still preserves complex, narrow (i.e., ridge and valley) features. Based on this sharper representation of the landscape, we conclude that MGM should be preferred for geomorphic applications relying on stereo-derived DEMs. However, we note that the correlation kernel selected for stereo-matching must be carefully chosen depending on local image texture, whereby larger kernels generate more accurate matches (less artifacts) at the cost of smoothing results. Overall, our analysis suggests a path forward for the processing and fusion of overlapping satellite images with suitable view-angle differences to improve final DEMs.

HLA gamma block matching in unrelated hematopoietic stem cell transplantation and the development of graft versus host disease

The human leukocyte antigen (HLA) genes routinely typed for hematopoietic stem cell transplantation (HSCT) are HLA-A, -B, -C, -DRB1 and -DQB1. HLA mismatches (MM), which have been associated with many post-transplantation complications, including acute graft-versus-host disease (GvHD), sometimes occur when a fully matched donor is not available. Gamma block (GB) is located in the central HLA region between Beta and Delta blocks and contains many inflammatory and immune regulatory genes, including the C4 gene. C4 was proposed as a marker when predicting haplotype matching due to positive linkage disequilibrium (LD) with its surrounding loci. Our aim was to investigate the association between GB matching in patients and their 9/10 HLA matched unrelated donors (UD) and the occurrence of GvHD. Patients and their UDs were typed using the PCR-SSP kit that detects 25 SNPs within GB. Gamma block mismatch occurred in 25 (75.8%) of the 33 studied patient-UD pairs. There was a significant difference in GvHD occurrence between Gamma type matched (GT-M) and mismatched (GT-MM) patient-UD pairs (p=0.0302). The probability of GvHD occurrence had also shown an increase, although insignificant, along with the number of GT-MM between patient-UD pairs (p=0.0913). These results suggest that GT matching could be useful in reducing the risk of post-HSCT complications.

Методика подсчета площади проекций частиц на двумерных наномасштабных изображениях с использованием пороговых алгоритмов

The growing complexity of studying nanoscale images and meshing the results of experiments processing makes it necessary to automate this process to improve accuracy and reliability of the results obtained. As part of the ongoing research, a method was developed to determine automatically the particles area in the electron microscope images. The threshold algorithm and its modification was considered to identify particles in the nanoscale images. Block matching and 3D filtering algorithm was selected for preprocessing. The optimal sequence of the above algorithms for images application was obtained in order to solve the problem. Several threshold algorithms and methods for calculating the corresponding threshold values were considered forming the base to select the most appropriate algorithms in the context of the problem being solved. The above method resulted in determining dependence of the particles number on their area for each given image. Currently, quite large and constantly growing volumes of nanoscale images were accumulated, which leads to a need to automate the research process. The proposed method is intended to solve practical problems in determining the particles area in the nanoscale images and could be used at various stages of the experiment

RDLNN-based Image Forgery Detection and Forged Region Detection Using MOT

Image forgery detection TEMPhas become an emerging research area due to the increasing number of forged images circulating on the internet and other social media, which leads to legal and social issues. Image forgery detection includes the classification of an image as forged or authentic and as well as localizing the forgery wifin the image. In this paper, we propose a Regression Deep Learning Neural Network (RDLNN) based image forgery detection followed by Modified Otsu Thresholding (MOT) algorithm to detect the forged region. The proposed model comprises five steps that are preprocessing, image decomposition, feature extraction, classification and block matching. In the preprocessing step, the RGB images are converted to YCbCr color format. Then, the images are decomposed using the new Polar Dyadic Wavelet Transform (PDyWT), followed by the extraction of important features. The classification phase called RDLNN effectively classifies the normal image and the forged image. For localization of the forgery, the forged image is divided into a number of blocks, and then Genetic Three Step Search (GTSS) algorithm is exploited to identify the dissimilar blocks. To get the exact forged region in the image, the dissimilar blocks are analyzed by the Modified Otsu Thresholding (MOT) algorithm. The proposed algorithm is compared wif widely used image forgery detection algorithms. The results show that the proposed method improves the forgery detection accuracy and precision by at least 6.04% and 3.77%, respectively, as compared to the already existent techniques such as ANFIS, KNN, ANN, and SVM. Moreover, the training time of the proposed network is lower by at least 64.3 % than the above existing techniques

Approximating a ride-sourcing system with block matching

On-demand matching between waiting passengers and idle drivers is one of the most important components in a ride-sourcing system. A variety of matching mechanisms have been developed to meet different needs of ride-sourcing platforms, e.g. mitigating supply–demand imbalance, maximizing platform revenue. In this paper, we focus on a block matching system, a special type of matching mechanism, where the region of interest is partitioned into blocks, and on-demand matching is separately and simultaneously conducted in each block. Block matching can bring many benefits, such as limiting order assignment with long pick-up distance, simplifying the process of deployment, etc. However, it still remains a challenging yet interesting issue to determine the block size for the matching system, which is a key decision variable governing passengers’ waiting time. To solve the problem, we model the ride-sourcing system with block matching via a M/M/c queue, in which the service rate is endogenous and partially determined by passengers’ average pick-up time. Based on the model, we find that the average queueing time of passengers decreases with block size increasing, while the average pick-up time may increase instead. In addition, the average total waiting time (sum of average queueing and pick-up time) become nearly invariant to the change of block size when the block size is large, which we call plateau phenomenon. In the plateau, ride-sourcing platforms can choose the block size based on other standards while the average total waiting time is always maintained at the nearly lowest value. The findings are verified via an agent-based simulation study, demonstrating that the proposed model can be an effective tool to approximate block matching system.

3D cloud envelope and cloud development velocity from simulated CLOUD (C3IEL) stereo images

Abstract. A method to derive the 3D cloud envelope and the cloud development velocity from high spatial and temporal resolution satellite imagery is presented. The CLOUD instrument of the recently proposed C3IEL mission lends itself well to observing at high spatial and temporal resolutions the development of convective cells. Space-borne visible cameras simultaneously image, under multiple view angles, the same surface domain every 20 s over a time interval of 200 s. In this paper, we present a method for retrieving cloud development velocity from simulated multi-angular, high-resolution top of the atmosphere (TOA) radiance cloud fields. The latter are obtained via the image renderer Mitsuba for a cumulus case generated via the atmospheric research model SAM and via the radiative transfer model 3DMCPOL, coupled with the outputs of an orbit, attitude, and camera simulator for a deep convective cloud case generated via the atmospheric research model Meso-NH. Matching cloud features are found between simulations via block matching. Image coordinates of tie points are mapped to spatial coordinates via 3D stereo reconstruction of the external cloud envelope for each acquisition. The accuracy of the retrieval of cloud topography is quantified in terms of RMSE and bias that are, respectively, less than 25 and 5 m for the horizontal components and less than 40 and 25 m for the vertical components. The inter-acquisition 3D velocity is then derived for each pair of tie points separated by 20 s. An independent method based on minimising the RMSE for a continuous horizontal shift of the cloud top, issued from the atmospheric research model, allows for the obtainment of a ground estimate of the velocity from two consecutive acquisitions. The mean values of the distributions of the stereo and ground velocities exhibit small biases. The width of the distributions is significantly different, with higher a distribution width for the stereo-retrieved velocity. An alternative way to derive an average velocity over 200 s, which relies on tracking clusters of points via image feature matching over several acquisitions, was also implemented and tested. For each cluster of points, mean stereo and ground positions were derived every 20 s over 200 s. The mean stereo and ground velocities, obtained as the slope of the line of best fit to the mean positions, are in good agreement.

Reactive Transport Modeling during Uranium In Situ Leaching (ISL): The Effects of Ore Composition on Mining Recovery

Unconsolidated sandstone uranium deposits exploited by the in situ leaching (ISL) method, contain complex tetravalent and hexavalent uranium compounds, mostly as UO2 and UO3 oxides that have different dissolution rates in sulfuric acid solutions. This work investigates a reactive transport model that takes into account the dissolution of both UO2 and UO3 in sulfuric acid solution together with possible interactions with rock minerals during the ISL uranium extraction. Several empirical reaction rate constants were determined during lab experiments on uranium extraction assays, including dissolution rates of tetravalent and hexavalent uranium oxides, and the dissolution rate of rock components by sulfuric acid solution. Effects on the recovery of solution flow rates and ratios between tetravalent and hexavalent uranium compounds are also investigated. The experimental dissolution constants were then used in the proposed reactive transport model to be applied to a real case study in Kazakhstan for comparing the 16 months history matching of an exploitation block consisting of 18 well injectors and 4 producers. The obtained numerical results show good agreement with empirical data gathered during exploitation.

Robust 3D imaging based on regularization by denoising.

Reconstructing a 3D image from the photon echo is a challenging task due to spurious detections associated with large amounts of background counts. Here, we propose a robust method for estimating the depth and reflectivity by using regularization by the denoising method, where the block matching and the 3D filtering are adopted as denoisers, and in the meantime, the steepest-descent method is implemented to solve the optimization problem. Experimental data with different signal-to-background ratios and different numbers of photons verify that our method is able to accurately recover 3D images. Compared with other existing methods, such as the maximum likelihood estimation algorithm, the photon efficient algorithm by Shin et al. [IEEE Trans. Comput. Imaging1, 112 (2015)2333-940310.1109/TCI.2015.2453093], and the ManiPoP algorithm, our method can effectively remove noise while preserving the edge information of depth images, with better depth image estimation and smaller root mean square error, especially at low signal-to-noise ratios. The superiority of this method over other methods is verified on simulated data sets under different conditions.

Active Contour Extension Basing on Haralick Texture Features, Multi-gene Genetic Programming, and Block Matching to Segment Thyroid in 3D Ultrasound Images

Active Contour Extension Basing on Haralick Texture Features, Multi-gene Genetic Programming, and Block Matching to Segment Thyroid in 3D Ultrasound Images

Modified block-matching algorithm for moving object tracking in video surveillance

Video surveillance has risen as one of the most promising methods for people who live alone in their dwellings. Few video surveillance innovations have recently been introduced. However, due to various changes in illumination, abrupt shifts in target appearance, identical non-target artifacts in the background, and occlusions, developing a reliable video surveillance algorithm remains a difficult challenge. This work attempts to introduce a new framework for moving object detection and tracking by following four major phases: “Video-to-Frame Conversion, Pre-Processing, Background Subtraction, Feature-Based Multi-object Detection, Multi-object Tracking by Filtering”. Initially, in the Video-to-Frame Conversion process, the recorded input video clips are transformed into distinct frames. During pre-processing, the noise is removed from the video frame using a filtering approach, and thereby the nature of the images will be enhanced. In the proposed work, a Weiner filter is used to remove noise and other undesirable features during the pre-processing. Then, to distinguish the frontal areas of objects, background subtraction is performed using the neutrosophic set in noiseless video frames (pre-processed frames). The objects in the background-subtracted frames are separated using Improved Region Growing (IRG) segmentation model in the Feature-Based Multi-object Detection phase. The objects in the frames are determined from this segmented image. The Modified Full Search Algorithm is being used to track the object (motion estimation) on the video frame after it has been identified in the segmented phase. The Modified full search block matching algorithm (MFSA) is introduced in this research work to find the appropriate mobility. Promising results have been obtained by the proposed work, and also the mathematical excellence of the new method is also proven over other state-of-the-art models.

Materials property mapping from atomic scale imaging via machine learning based sub-pixel processing

Direct visualization of the atomic structure in scanning transmission electron microscopy has led to a comprehensive understanding of the structure-property relationship. However, a reliable characterization of the structural transition on a picometric scale is still challenging because of the limited spatial resolution and noise. Here, we demonstrate that the primary segmentation of atomic signals from background, succeeded by a denoising process, enables structural analysis in a sub-pixel accuracy. Poisson noise is eliminated using the block matching and three-dimensional filtering with Anscombe transformation, and remnant noise is removed via morphological filtering, which results in an increase of peak signal-to-noise ratio from 7 to 11 dB. Extracting the centroids of atomic columns segmented via K-means clustering, an unsupervised method for robust thresholding, achieves an average error of less than 0.7 pixel, which corresponds to 4.6 pm. This study will contribute to a profound understanding of the local structural dynamics in crystal structures.

Motion estimation of the common carotid artery wall in ultrasound images using an improved sub-pixel block matching method

To improve the accuracy of the two-dimensional motion estimation of the common carotid artery (CCA) wall, a modified sub-pixel block matching method is proposed in this study. First, the directional asymmetric search (DAS) method is used for the block matching, and the sum of absolute error criterion combined with fuzzy set theory is used to determine the similar blocks. After obtaining the best block matching position, a combination of grid slope and parabolic interpolation (GS15PI) method is used for sub-pixel estimation. Then, the best matching position (point) obtained in the previous step and all points in its 3×3 neighborhood need to be determined. The blocks with the same size as the reference block centered on these points are used for block matching with the reference block from the search frame − 2. After obtaining the best block matching position, the GS15PI method is used for sub-pixel estimation. Finally, the average of sub-pixel positions obtained in the two previous steps is calculated as the final sub-pixel estimation. In simulation experiments, the two-dimensional motion errors of proposed method for simulated ultrasound sequences with the center frequencies of 5 MHz, 7.5 MHz, and 10 MHz are decreased by 8.41%，4.65%, and 11.25%, respectively, compared with the state-of-the-art method. In clinical experiments, as compared with the state-of-the-art method, the radial and longitudinal motion estimation results of proposed method are more consistent with the actual movement trend of the human CCA. In conclusion, the proposed method has good performance and can be used for motion estimation of the CCA wall in ultrasound images.

EDFLOW: Event Driven Optical Flow Camera With Keypoint Detection and Adaptive Block Matching

Event cameras such as the Dynamic Vision Sensor (DVS) are useful because of their low latency, sparse output, and high dynamic range. In this paper, we propose a DVS+FPGA camera platform and use it to demonstrate the hardware implementation of event-based corner keypoint detection and adaptive block-matching optical flow. To adapt sample rate dynamically, events are accumulated in event slices using the area event count slice exposure method. The area event count is feedback controlled by the average optical flow matching distance. Corners are detected by streaks of accumulated events on event slice rings of radius 3 and 4 pixels. Corner detection takes about 6 clock cycles (16 MHz event rate at the 100MHz clock frequency) At the corners, flow vectors are computed in 100 clock cycles (1 MHz event rate). The multiscale block match size is <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$25\times 25$ </tex-math></inline-formula> pixels and the flow vectors span up to 30-pixel match distance. The FPGA processes the sum-of-absolute distance block matching at 123 GOp/s, the equivalent of 1230 Op/clock cycle. EDFLOW is several times more accurate on MVSEC drone and driving optical flow benchmarking sequences than the previous best DVS FPGA optical flow implementation, and achieves similar accuracy to the CNN-based EV-Flownet, although it burns about 100 times less power. The EDFLOW design and benchmarking videos are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://sites.google.com/view/edflow21/home</uri> .

Scalable extension of HEVC with adaptive de-blocking filter for bandwidth-limited applications

A primary latest video coding standard H.265/SHVC (Scalable High Efficiency Video Coding), an extension of High Efficiency Video Coding (HEVC) is dealt in this paper. To transfer high quality video over any network that usually occupies large bandwidth, suffers bandwidth limitation. It can be controlled by trading-off between the quality and the coding efficiency. The trade-off between these two parameters is considered in this paper to keep the difference at the minimum level. A novel Neural Network based Adaptive De-blocking Filter (NNADF) technique is proposed to maintain good PSNR with minimum bit rate in SHVC encoder. The adaptive de-blocking filter (ADF) performs block matching for nonlinear predictions to all frames in the video. Moreover, the ADF performs frame de-noising hence good quality of the video is achieved and trains a neural network (NN) model. The NN model restrains their outputs to achieve bit rate reduction, thereby coding efficiency is achieved. The simulation results show that the proposed NNADF technique delivers an average increment of 0.62 dB in BD-PSNR and an average decrement of 24 % in BD-BR for combined quality and spatial ratios respectively compared with the previously proposed algorithm.

Petroleum Pipeline Interface Recognition and Pose Detection Based on Binocular Stereo Vision

Liquified natural gas (LNG) manipulator arms have been widely used in natural gas transportation. However, the automatic docking technology of LNG manipulator arms has not yet been realized. The first step of automatic docking is to identify and locate the target and estimate its pose. This work proposes a petroleum pipeline interface recognition and pose judgment method based on binocular stereo vision technology for the automatic docking of LNG manipulator arms. The proposed method has three main steps, including target detection, 3D information acquisition, and plane fitting. First, the target petroleum pipeline interface is segmented by using a color mask. Then, color space and Hu moment are used to obtain the pixel coordinates of the contour and center of the target petroleum pipeline interface. The semi-global block matching (SGBM) algorithm is used for stereo matching to obtain the depth information of an image. Finally, a plane fitting and center point estimation method based on a random sample consensus (RANSAC) algorithm is proposed. This work performs a measurement accuracy verification experiment to verify the accuracy of the proposed method. The experimental results show that the distance measurement error is not more than 1% and the angle measurement error is less than one degree. The measurement accuracy of the method meets the requirements of subsequent automatic docking, which proves the feasibility of the proposed method and provides data support for the subsequent automatic docking of manipulator arms.

Image denoising using block matching and convolutional neural network

Image noise removal (denoising) is a significant preprocessing step in image processing research domain. In past decades, various noise removal algorithms have been developed, and they are categorized into conventional and deep learning denoising methods. Most recently, deep learning-based denoising methods outperform the conventional methods. However, deep learning methods have the limitations of requiring large training sample images, difficult to train the deep convolution neural network (DeCNN), and the majority of the convolutional neural network suffers from the performance saturation. Hence, using combination of block matching (BM) and dilated convolutional neural network (DCNN), a unique method for reducing noise in still images is proposed. In this approach, initially, existing algorithm used to preprocess the image. Then, BM technique uses a sliding window size of a four-by-four block and moves it across the image to select for blocks that are similar in the image. The matching blocks are highly correlated with one another. The matching blocks are then fed to the deep DCNN to remove noise and obtain a better noiseless image. Finally, the noiseless image performances are evaluated using standard metrics. From the experimental results, it is observed that the proposed BM and deep DCNN yielded substantially better results than existing state-of-the-art methods to remove the image’s noise.

EXPERIMENT ON PRODUCING DISPARITY MAPS FROM AERIAL STEREO IMAGES USING UNSUPERVISED AND SUPERVISED METHODS

Abstract. Recent advancement in hardware and software provides the possibility of realizing full automation in stereo-image tasks. This paper investigated disparity map generation from aerial images with different methods: unsupervised method and supervised methods. The datasets were from aerial stereo dense matching benchmark dataset for deep learning in ISPRS 2021: Vaihingen dataset and the WHU MVS/Stereo Dataset released in the CVPR 2020. Two neural networks: GC-net and PSMnet have been trained with the Vaihingen dataset and the WHU MVS/Stereo Dataset. With unsupervised methods, stereo block matching(StereoBM) and Stereo Semi-Global Matching (StereoSGM) methods from the OpenCV were studied. We selected seven image pairs from the Vaihingen dataset and six image pairs from the WHU dataset for testing and evaluation. Difficulty scenes such as textureless areas, reflective surfaces, and repetitive patterns were also included in our study. The performance from different methods was compared by both visualization and quantitative means. The advantages and disadvantages are presented.