Motion Estimation Process Research Articles

Efficient motion estimation and discrete cosine transform implementation using the graphics processing units.

Motion Estimation (ME) and the two-dimensional (2D) discrete cosine transform (2D-DCT) are both computationally expensive parts of HEVC standard, therefore real-time performance of the HEVC may not be free from glitches. To address this issue, this study deploys the graphics processing units (GPUs) to perform the ME and 2D-DCT tasks. In this concern, authors probed into four levels of parallelism (i.e., frame, macroblock, search area, and sum of the absolute difference (SAD) levels) existing in ME. For comparative analysis, authors involved full search (FS), test zone search (TZS) of HEVC, and hierarchical diamond search (EHDS) ME algorithms. Similarly, two levels of parallelism (i.e., macroblock and sub-macroblock) are also explored in 2D-DCT. Notably, the least computationally complex multithreaded Loeffler DCT algorithm is utilized for computing 2D-DCT. Experimental results show that ME processing task corresponding to 25 frames, with each frame of size (3840×2160) pixels, is accomplished in 0.15 seconds on the NVIDIA GeForce GTX 1080, whereas the 2D-DCT task along with the image reconstruction and differencing corresponding to 25 frames took 0.1 seconds. Collectively, both ME and 2D-DCT tasks are processed in 0.25 seconds, which still leaves enough room for the encoder's remaining parts to be executed within one second. Due to this enhancement, the resultant encoder can safely be used in real-time applications.

Advanced Patch-Based Affine Motion Estimation for Dynamic Point Cloud Geometry Compression.

The substantial data volume within dynamic point clouds representing three-dimensional moving entities necessitates advancements in compression techniques. Motion estimation (ME) is crucial for reducing point cloud temporal redundancy. Standard block-based ME schemes, which typically utilize the previously decoded point clouds as inter-reference frames, often yield inaccurate and translation-only estimates for dynamic point clouds. To overcome this limitation, we propose an advanced patch-based affine ME scheme for dynamic point cloud geometry compression. Our approach employs a forward-backward jointing ME strategy, generating affine motion-compensated frames for improved inter-geometry references. Before the forward ME process, point cloud motion analysis is conducted on previous frames to perceive motion characteristics. Then, a point cloud is segmented into deformable patches based on geometry correlation and motion coherence. During the forward ME process, affine motion models are introduced to depict the deformable patch motions from the reference to the current frame. Later, affine motion-compensated frames are exploited in the backward ME process to obtain refined motions for better coding performance. Experimental results demonstrate the superiority of our proposed scheme, achieving an average 6.28% geometry bitrate gain over the inter codec anchor. Additional results also validate the effectiveness of key modules within the proposed ME scheme.

Visual Pursuit with Switched Motion Estimation and Rigid Body Gaussian Processes

Visual Pursuit with Switched Motion Estimation and Rigid Body Gaussian Processes

A Fast Gradient Iterative Affine Motion Estimation Algorithm Based on Edge Detection for Versatile Video Coding

In the Versatile Video Coding (VVC) standard, affine motion models have been applied to enhance the resolution of complex motion patterns. However, due to the high computational complexity involved in affine motion estimation, real-time video processing applications face significant challenges. This paper focuses on optimizing affine motion estimation algorithms in the VVC environment and proposes a fast gradient iterative algorithm based on edge detection for efficient computation. Firstly, we establish judging conditions during the construction of affine motion candidate lists to streamline the redundant judging process. Secondly, we employ the Canny edge detection method for gradient assessment in the affine motion estimation process, thereby enhancing the iteration speed of affine motion vectors. The experimentalresults show that the encoding time of the affine motion estimation algorithm is about 15–35% lower than the overall encoding time of the anchor algorithm encoder, the average encoding time of the affine motion estimation part of the inter-frame prediction part is reduced by 24.79%, and the peak signal-to-noise ratio (PSNR) is only reduced by 0.04.

Ant weight-lifting algorithm for motion estimation

Every video coding standard includes and requires motion estimation and compensation. The full search algorithm, which provides the best motion estimation, has a very high computation cost. Researchers have developed several algorithms to reduce the cost of computation. However, most of these algorithms become trapped in local minima during the search. Population-based evolutionary algorithms are widely used to develop a computationally efficient and cost-effective motion estimation strategy. The most recent effort used the Jaya algorithm to develop a motion estimation process that outperformed the state-of-the-art test zone search algorithm. In this study, a motion estimation algorithm based on the ant weight-lifting approach is proposed. Previously, the ant weight-lifting algorithm was used to solve a variety of problems, such as image segmentation, signal compression, and so on. The ant weight-lifting algorithm's computation cost was reduced by adopting a fitness estimation method that uses nearest-neighbor interpolation and an early termination strategy. Compared to Jaya algorithm-based motion estimation, the proposed algorithm executes up to 3% more quickly and exhibits up to 1.2 dB less distortion.

Implementation of a motion estimation algorithm for Intel FPGAs using OpenCL

Motion Estimation is one of the main tasks behind any video encoder. It is a computationally costly task; therefore, it is usually delegated to specific or reconfigurable hardware, such as FPGAs. Over the years, multiple FPGA implementations have been developed, mainly using hardware description languages such as Verilog or VHDL. Since programming using hardware description languages is a complex task, it is desirable to use higher-level languages to develop FPGA applications.The aim of this work is to evaluate OpenCL, in terms of expressiveness, as a tool for developing this kind of FPGA applications. To do so, we present and evaluate a parallel implementation of the Block Matching Motion Estimation process using OpenCL for Intel FPGAs, usable and tested on an Intel Stratix 10 FPGA. The implementation efficiently processes Full HD frames completely inside the FPGA. In this work, we show the resource utilization when synthesizing the code on an Intel Stratix 10 FPGA, as well as a performance comparison with multiple CPU implementations with varying levels of optimization and vectorization capabilities. We also compare the proposed OpenCL implementation, in terms of resource utilization and performance, with estimations obtained from an equivalent VHDL implementation.

Modified Cuckoo Search Algorithm for Motion Vector Estimation

Motion estimation and motion compensation are the accepted process in H.264 and H.265 video coding standard to reduce temporal redundancy. Several fast block matching algorithms have been developed to reduce the calculation cost in the motion estimation process. But quick block matching algorithms often lead to a local minimum. Several researchers used different population-based nature-inspired algorithms to perform block matching. Algorithms like genetic algorithm, differential evolution, particle swarm optimization were used in numerous motion estimation algorithms. Different algorithms used a fitness approximation strategy to reduce computation cost. Jaya algorithm-based block matching is the most efficient block matching algorithm in the available literature. Jaya algorithm is free from algorithmic specific parameter which speeds up the process. This article proposes a few modifications to the traditional cuckoo search algorithm and then, a block matching algorithm was proposed based on the modified cuckoo search algorithm. Fitness approximation, adaptive termination, and zero motion prejudgment modules were used with the modified cuckoo search algorithm to reduce the number of redundant calculations. The performance of the proposed algorithm was compared with the exhaustive search algorithm and other benchmarking algorithms in terms of Peak Signal to Noise Ratio (PSNR), Structure Similarity Index (SSIM), and average search point required to calculate a motion vector for a block. The proposed algorithm delivers better performance compared to the benchmarking algorithms.

A Deep Learning Approach in Scalable High Efficiency Video Coding for Fast Coding Unit Size Decision

The High Efficiency Video Coding (HEVC) with scalable extension known as SHVC allows encoding the same video with different resolutions in a single bitstream. However, this process increases the encoding complexity. The complexity is increased mainly due to the motion estimation process and the Rate-Distortion Optimization (RDO) search process. The Test Zonal (TZ) algorithm with fast search patterns helps to accelerate the motion estimation process. However, the search patterns may get trapped to local minima, leading to inaccurate motion vectors. Moreover, the RDO search process used to determine the Coding Unit (CU) size increases the complexity. We proposed the Horizontal Subsampling Motion Estimation (HSME) method to find the accurate motion vectors with reduced complexity. The experimental results prove that the HSME method saves the encoding time by 53.03% with a 6.45% increase in Bjøntegaard delta bit rate and 0.28 dB loss in Bjøntegaard Delta Peak Signal-to-Noise Ratio (BD-PSNR) compared to standard SHM-12.1. In addition, we designed the Early Terminated Long- and Short-Term Memory (ET-LSTM) network that predicts the CU partition by taking the output features of the Early Terminated Convolutional Neural Network (ET-CNN). The ET-CNN learns the CU partitions from the residual Coding Tree Unit (CTU) using the deep learning approach. Our proposed method (HSME + ET-CNN + ET-LSTM) achieves 53% savings in encoding time, which is significantly higher than state-of-the-art methods.

An intelligent optimization algorithm with a deep learning‐enabled block‐based motion estimation model

AbstractThe motion estimation process has gained significant attention in several video applications such as video compression, global motion estimation, and so on. The deep learning (DL) models that perform effectively in computer vision tasks can be employed for motion estimation. Yet, they have high computational complexity and model parameters since motion estimation in the pixel domain is not effective in practical applications. Instead, a block‐based motion estimation process has been developed where the blocks of sequential images are compared. To that end, this paper presents the Anas platyrhynchos optimizer with deep learning‐enabled block‐based motion estimation (APODL‐BBME) model. The proposed model estimates motion using block‐based concepts and DL approaches. To accomplish this, the training and testing frames are separated into non‐overlapping block categories, and the blocks are filtered via the bilateral filtering (BF) approach. In addition, a histogram of gradients (HOG) and densely connected network (DenseNet) model are employed to extract features, which are then fed into the bidirectional long short‐term memory (BiLSTM) model to classify the input features. Finally, the APO algorithm is applied to optimally tune the hyperparameters of the BiLSTM model, which helps to improve the overall motion estimation efficacy, showing the novelty of the work. To demonstrate the enhanced performance of the APODL‐BBME model, a comprehensive analysis is carried out, and when we compare the results, the APODL‐BBME model outperforms recent motion estimation approaches.

An Improved Fast Motion Estimation Algorithm in H.266/VVC

Motion estimation is a key technique of video compression coding, H.266/VVC (Versatile Video Coding) inter TZsearch fast motion estimation algorithm using the diamond search model, although the search model matching precision is high, but because of the number of search points need to match, so the computational complexity is higher also, for this defect, the article on the basis of the standard algorithm, The fast algorithm of TZSearch is optimized and improved. Firstly, the hexagonal search model with excellent performance and speed is used to replace the diamond search model in the standard algorithm. Secondly, on the basis of the rotation hexagon model, the number of initial matching search points of the rotation hexagon search model is further increased by using the feature of natural video prediction MV center bias. At the same time, a reasonable early termination threshold is set for the whole motion estimation process. Simulation results show that compared with VTM14.0 standard algorithm, the proposed algorithm reduces BDBR by 0.17% and BDPSNR by 0.04dB on average, and the overall coding time is saved by 48.85% on average. The proposed algorithm can effectively reduce the computational complexity of motion estimation process on the premise of guaranteeing the coding quality.

A Motion Estimation based Algorithm for Encoding Time Reduction in HEVC


 
 
 High Efficiency Video Coding (HEVC) is a video compression standard that offers 50% more efficiency at the expense of high encoding time contrasted with the H.264 Advanced Video Coding (AVC) standard. The encoding time must be reduced to satisfy the needs of real-time applications. This paper has proposed the Multi- Level Resolution Vertical Subsampling (MLRVS) algorithm to reduce the encoding time. The vertical subsampling minimizes the number of Sum of Absolute Difference (SAD) computations during the motion estimation process. The complexity reduction algorithm is also used for fast coding the coefficients of the quantised block using a flag decision. Two distinct search patterns are suggested: New Cross Diamond Diamond (NCDD) and New Cross Diamond Hexagonal (NCDH) search patterns, which reduce the time needed to locate the motion vectors. In this paper, the MLRVS algorithm with NCDD and MLRVS algorithm with NCDH search patterns are simulated separately and analyzed. The results show that the encoding time of the encoder is decreased by 55% with MLRVS algorithm using NCDD search pattern and 56% with MLRVS using NCDH search pattern compared to HM16.5 with Test Zone (TZ) search algorithm. These results are achieved with a slight increase in bit rate and negligible deterioration in output video quality.
 
 

Adaptive fractional motion and disparity estimation skipping in MV-HEVC

MV-HEVC can efficiently compress multiview video data captured from different viewpoints. To achieve high coding efficiency, it consists of not only inter coding but also interview coding. The inter coding includes a motion estimation (ME) process that reduces temporal redundancies between consecutive frames, and the interview coding performs a disparity estimation (DE) that reduces interview redundancies between neighboring views. As a result, MV-HEVC needs high encoding complexity to perform both ME and DE. In order to reduce the complexity, this paper proposes an adaptive fractional ME and DE skipping method in a partitioned inter prediction unit (PU) mode, based on a result of a 2 N × 2 N inter PU coding. Experimental results show that the proposed method efficiently reduces the encoding complexity with negligible coding loss, compared to conventional methods.

An efficient field programmable gate array based hardware architecture for efficient motion estimation with parallel implemented genetic algorithm

SummaryThe main idea of this article is to propose a hardware design of block matching (BM) algorithm for an efficient motion estimation (ME) strategy in a field programmable gate array platform based on a parallel implemented genetic algorithm (GA). Easiness and the effectiveness of the BM algorithm while implementing have a major drawback of low quality and computationally cost expensive during the process of ME. Therefore, here in this article, we suggest GA based BM for a quick and cost‐effective computation of motion vectors, without negotiating the quality factor. The ME carried out for various video sequences is implemented by using Xilinx ISE Design Suite 14.1. Delay, time, area, power, PSNR, MSE, SNR, SSIM, and NRMSE are the metrics used for analyzing the performance, and the simulation outcome shows that this parallel implemented BM architecture design shows an exotic improvement in time, quality and in utilization of power on estimating the motion than that of the conventional designs.

An Innovative Approach for Denoising of the Digital Video Stream

Everyday tones of video signals are generated, transmitted, and analyzed. The video contents are created for educational purposes, entertainment purposes, surveillance purposes, medical imaging purposes, weather forecasting, satellite imaging, and many other significant places. During the different phases of video content preparation, transmission, and analysis some unwanted signals get interfered with the true contents. Particularly, the medical imaging signals, since they are weak signals, are more prone to unwanted interferences. Such unwanted interference of the noise signals makes it difficult to analyze the critical information in the video contents and hence, the need for denoising process arises. A decent video denoising framework assures visual improvement in the video signals or it serves as the significant pre-processing step in the video processing steps like compression and analysis. Through this paper, we are about to disclose an efficient video denoising framework that takes the noisy video signal in the form of frames per second and performs the video denoising using shot detection, compensation, intensity calculations, and motion estimation process.

An efficient highly parallelized ReRAM-based architecture for motion estimation of HEVC

Motion estimation (ME) is a high efficiency video coding (HEVC) process for determining motion vectors that describe the blocks transformation direction from one adjacent frame to a future frame in a video sequence. ME is a memory and computation consuming process which accounts for more than 50% of the total running time of HEVC. To conquer the memory and computation challenges, this paper presents ReME, a highly paralleled processing-in-memory (PIM) architecture for the ME process based on resistive random access memory (ReRAM). In ReME, the space of ReRAM is mainly separated into storage engine and ME processing engine. The storage engine is used as conventional memory to store video frames and intermediate data, while the computation operations of ME are performed in ME processing engines. Each ME processing engine in ReME consists of Sum of Absolute Differences (SAD) modules, interpolation modules, and Sum of Absolute Transformed Difference (SATD) modules that transfer ME functions into ReRAM-based logic analog computation units. ReME further cooperates these basic computation units to perform ME processes in a highly parallel manner. Simulation results show that the proposed ReME accelerator significantly outperforms other implementations with time consuming and energy saving.

Block Matching Algorithms for the Estimation of Motion in Image Sequences: Analysis

Several video coding standards and techniques have been introduced for multimedia applications, particularly the h.26x series for video processing. These standards employ motion estimation processing to reduce the amount of data that is required to store or transmit the video. The motion estimation process is an inextricable part of the video coding as it removes the temporal redundancy between successive frames of video sequences. This paper is about these motion estimation algorithms, their search procedures, complexity, advantages, and limitations. A survey of motion estimation algorithms including full search, many fast, and fast full search block-based algorithms has been presented. An evaluation of up-to-date motion estimation algorithms, based on several empirical results on several test video sequences, is presented as well.

Taylor rate-distortion trade-off and adaptive block search for HEVC encoding

The advancement in high-efficiency video coding (HEVC) is adapted for defining the subsequent generation compression model to offer efficient compression without affecting the image quality. The HEVC offers improved performance than the existing compression models. Accordingly, this work develops an approach for video compression by proposing weighted entropy coding and adaptive block search-based rate-distortion (R-D) trade-off, named Taylor R-D trade-off. The adaptive block search R-D trade-off, by integrating the hexagon-based tree search algorithm (HBTSA), along with the Taylor R-D trade-off for initiating the block search process of motion estimation in video coding, which selects the optimal block. Initially, the frames are extorted from the input video. Then, the video frames are divided into macroblocks to perform the adaptive block search. Further, the suitable blocks are selected and given to the encoding process by weighted context-adaptive binary arithmetic coding (CABAC) that employs a weighted entropy function to persist the video quality after the compression. The results analysis shows that the proposed HBTSA method has improved PSNR and SSIM values of 42.717dB, and 0.991, respectively, using football, coast guard, garden, and tennis videos.

MapReduce Analytics Based Fast Search Algorithms

ABSTRACT Using the motion estimation process is still one of the best technologies for video coding standards such as H.26X or MPEG-X. These standards have been supplied to video compression, but this area is still open algorithms. It can provide excellent coding efficiency while maintaining a low distortion. A new motion estimation of a full search algorithm with big data analytics in this research was proposed, which uses MapReduce analysis metrics. The developed method can not only be used to reduce the stored number of motion vectors compared with the full search algorithm but also to perform superior peak signal-to-noise ratio than other recent developed scheme, and achieve the same peak signal-to-noise ratio as the full search algorithm. Simulation exemplifies that our proposed scheme obtains better improvement than the full search algorithm and other well-known methods. The comparison of our proposed algorithms with other methods is evaluated both subjectively and qualitatively and the simulation demonstrated that our proposed method could obtain the desired reconstructed performance subjectively and objectively.

Motion estimation using variable size block matching with cross square search pattern

In recent decades, motion estimation is a major concern in video coding applications. Among all existing techniques, block matching is an emerging technique in motion estimation, because of its simplicity and effectiveness for both hardware and software implementations. In addition, the block matching technique helps in choosing the motion vectors for every macro block instead of utilizing a motion vector for every pixel. In this research article, Variable Size Block Matching (VSBM) with cross square search pattern is proposed for motion estimation in the videos. Compared to other block matching methods, the VSBM results in a low motion compensation error and also significantly speed up the motion estimation process. For reducing the illumination change effect in the video frame, cross square search pattern is developed that prevents macro blocks from over-splitting. In this work, the proposed algorithm performance is analysed by using some real time videos like container, Akiyo, Stefan, kristen-sara, foreman, Johnny, old-town-cross, Carphone, football and crowd-run. From the analysis, the proposed algorithm results in maximum of 14.9 dB improvement in peak signal to noise ratio value as compared to existing algorithms; dynamic pattern (square, diamond and hexagon) and harmony search with block matching.

Motion Estimation From Doppler and Spatial Data in SONAR Images

Motion estimation is critical for the localization of autonomous underwater vehicles. Current SONAR-based techniques exclusively utilize either Doppler or spatial measurements. However, these measurement domains are complementary to each other; Doppler measurements directly measure radial motion, whereas spatial measurements uniquely observe angular motion. Therefore, this article presents SONARODO (SONAR Odometry), a novel real-time motion estimation algorithm for 2-D forward-looking SONARs. It depends on a largely decoupled motion estimation process that better utilizes each measurement domain for their respective strengths. Specifically, it estimates translational motion from Doppler-azimuth images and rotational motion from range-azimuth images. While this method does require a SONAR that can provide both image types, it was designed to ensure robustness to featureless seafloor environments and low-resolution images. This article's validation with high-fidelity simulation data demonstrated that SONARODO offers accuracy and computational cost advantages over related motion estimation techniques.