Motion Compensation Research Articles

Compensation for respiratory motion-induced signal loss and phase corruption in free-breathing self-navigated cine DENSE using deep learning.

To introduce a model that describes the effects of rigid translation due to respiratory motion in displacement encoding with stimulated echoes (DENSE) and to use the model to develop a deep convolutional neural network to aid in first-order respiratory motion compensation for self-navigated free-breathing cine DENSE of the heart. The motion model includes conventional position shifts of magnetization and further describes the phase shift of the stimulated echo due to breathing. These image-domain effects correspond to linear and constant phase errors, respectively, in k-space. The model was validated using phantom experiments and Bloch-equation simulations and was used along with the simulation of respiratory motion to generate synthetic images with phase-shift artifacts to train a U-Net, DENSE-RESP-NET, to perform motion correction. DENSE-RESP-NET-corrected self-navigated free-breathing DENSE was evaluated in human subjects through comparisons with signal averaging, uncorrected self-navigated free-breathing DENSE, and breath-hold DENSE. Phantom experiments and Bloch-equation simulations showed that breathing-induced constant phase errors in segmented DENSE leads to signal loss in magnitude images and phase corruption in phase images of the stimulated echo, and that these artifacts can be corrected using the known respiratory motion and the model. For self-navigated free-breathing DENSE where the respiratory motion is not known, DENSE-RESP-NET corrected the signal loss and phase-corruption artifacts and provided reliable strain measurements for systolic and diastolic parameters. DENSE-RESP-NET is an effective method to correct for breathing-associated constant phase errors. DENSE-RESP-NET used in concert with self-navigation methods provides reliable free-breathing DENSE myocardial strain measurement.

Enhanced Motion Compensation for Deep Video Compression

Most of the existing deep learning-based video compression frameworks rely on motion estimation and compensation. However, the artifacts of the warped frames after motion compensation, which propagate the errors to the next frame, limit the video coding performance. In this work, we propose enhanced motion compensation for reduced error propagation in deep video compression. More specifically, we incorporate the designed convolutional neural network into Open DVC as the motion compensation enhancement network to remove noise in the predicted frame. With the enhanced frame, we jointly optimize the whole framework with a single loss function by considering the trade-off between bit cost and frame quality. Experiments show that the proposed enhanced motion compensation model reduces error propagation within a group of frames. Compared with Open DVC, our model can achieve 8.94% bit savings on average for standard test videos in terms of PSNR. Regarding MS-SSIM, our model outperforms Open DVC with 5.67% bit rate savings.

Ray-Space Motion Compensation for Lenslet Plenoptic Video Coding.

Plenoptic images and videos bearing rich information demand a tremendous amount of data storage and high transmission cost. While there has been much study on plenoptic image coding, investigations into plenoptic video coding have been very limited. We investigate the motion compensation (or so-called temporal prediction) for plenoptic video coding from a slightly different perspective by looking at the problem in the ray-space domain instead of in the conventional pixel domain. Here, we develop a novel motion compensation scheme for lenslet video under two sub-cases of ray-space motion, that is, integer ray-space motion and fractional ray-space motion. The proposed new scheme of light field motion-compensated prediction is designed such that it can be easily integrated into well-known video coding techniques such as HEVC. Experimental results compared to relevant existing methods have shown remarkable compression efficiency with an average gain of 20.03% and 21.76% respectively under "Low delayed B " and "Random Access" configurations of HEVC.

A Two-Step Discrete Cosine Basis Oriented Motion Modeling Approach for Enhanced Motion Compensation.

Video coding algorithms attempt to minimize the significant commonality that exists within a video sequence. Each new video coding standard contains tools that can perform this task more efficiently compared to its predecessors. Modern video coding systems are block-based wherein commonality modeling is carried out only from the perspective of the block that need be coded next. In this work, we argue for a commonality modeling approach that can provide a seamless blending between global and local homogeneity information in terms of motion. For this purpose, at first a prediction of the current frame, the frame that need be coded, is generated by performing a two-step discrete cosine basis oriented (DCO) motion modeling. The DCO motion model is employed rather than traditional translational or affine motion model since it has the ability to efficiently model complex motion fields by providing a smooth and sparse representation. Moreover, the proposed two-step motion modeling approach can yield better motion compensation at a reduced computational complexity since an informed guess is designed for initializing the motion search procedure. After that the current frame is partitioned into rectangular regions and the conformance of these regions to the learned motion model is investigated. Depending on the non-conformance to the estimated global motion model, an additional DCO motion model is introduced to increase the local motion homogeneity. In this way, the proposed approach generates a motion compensated prediction of the current frame through the minimization of both global and local motion commonality. Experimental results show an improved rate-distortion performance of a reference high efficiency video coding (HEVC) encoder, specifically up to around 9% savings in bit rate, that employs the DCO prediction frame as a reference frame for encoding the current frame. When compared to the more recent video coding standard, the versatile video coding (VVC) encoder, a bit rate savings of 2.37% is reported.

Segmental Aperture Imaging Algorithm for Multirotor UAV-Borne MiniSAR

This article takes on the challenges of synthetic aperture radar (SAR) imaging for miniaturized SAR (MiniSAR) onboard a multirotor unmanned aerial vehicle (UAV). Several unique challenges are systematically analyzed, and a corresponding analytical phase error model is established, which accurately models the effects of both translational and rotational motions of UAVs. A segmental aperture imaging (SAI) algorithm, an autofocus algorithm based on strong scatterers, is proposed. It simply divides the platform trajectory into uneven segments, which are first independently focused with motion compensation and then stitched together to form a complete SAR image. Both the theoretical derivation of the signal model and the implementation of the imaging algorithm are presented. A simulation analysis with actual UAV trajectory and attitude data is conducted, which demonstrates the efficacy and performance of the proposed imaging algorithm. It shows that the ideal focusing effect can be achieved as evaluated by various metrics, and the proposed algorithm has superior performance compared to the subaperture phase gradient autofocus (PGA) and minimum entropy autofocus (MEA) methods. Finally, the multirotor-borne MiniSAR system FUSAR-Ku is used for experiments to verify the proposed algorithm. Experimental results show that the proposed algorithm can achieve the theoretical decimeter-resolution imaging performance as measured by various metrics.

Dual-Channel Airborne SAR Imaging of Ground Moving Targets on Perturbed Platform

A dual-channel airborne SAR imaging approach is proposed to acquire well focused images of ground moving targets after compensating the platform trajectory deviation. The phase error attributed to either platform trajectory deviation or target motion is given by a polynomial of slow time. The SAR image before motion compensation is segmented into multiple sub-images, and a compressive-sensing (CS) technique is applied on each sub-image to estimate the polynomial coefficients featuring platform trajectory deviation. A range-frequency reversal transform (RFRT) and fractional Fourier transform (FrFT) are combined to estimate the motion parameters and position of each moving target. The proposed approach is verified with real data and simulated data, respectively, and the moving targets in the acquired image are well recognized.

Media resources in video information systems

The article describes methods for increasing the contrast of images in video information systems, the main compression of the video stream is provided by eliminating inter-frame redundancy using methods for compensating the movement of image fragments of adjacent frames. However, the use of motion compensation methods requires the formation of additional data (metadata) containing information about the types of image blocks used, their movement coordinates, etc. At the same time, in order to increase the compression of the video stream without sacrificing its quality, a higher accuracy of motion compensation is required, which leads to an increase in the number of blocks and, accordingly, an increase in the volume of the video stream metadata that reduces the effectiveness of motion compensation. This is the main problem of compressing streaming video without compromising image quality. In addition, higher positioning accuracy of blocks with motion compensation dramatically reduces the speed of image processing, which is not always feasible in real-time systems.

3-D High Frame Rate Imaging With Motion Compensation (3-D HFR With MoCo): An Experimental Evaluation

3-D High Frame Rate Imaging With Motion Compensation (3-D HFR With MoCo): An Experimental Evaluation

Motion Compensation in Six Degrees of Freedom for a MIMO Radar Mounted on a Hovering UAV

This paper deals with the motion of a hovering UAV with six degrees of freedom. The effects of the motion on the measured signal of a MIMO radar mounted underneath the UAV are analyzed. For each degree of freedom, namely the translation in x-, y- and z-direction as well as the rotation around x-, y- and z-axis, an algorithm is proposed to compensate the considered motion. The effectiveness of the proposed algorithms is demonstrated by measuring validated vital signs independent of the current UAV motion.

DYNAMIC CONTRAST-ENHANCED MRI OF THE SYNOVIUM IN KNEE OSTEOARTHRITIS: SEMI-AUTOMATIC SEGMENTATION OF SYNOVIAL SUBREGIONS AND TEST-RETEST REPEATABILITY

Inflammation of the joint lining (synovitis) is often seen in OA, and is strongly associated with both knee pain severity and disease progression. Dynamic contrast-enhanced MRI (DCE-MRI) can be used to visualize the uptake and washout of gadolinium-based contrast, and enables quantification of blood perfusion as a surrogate measure of inflammation. DCE-MRI could therefore be used to characterize the extent of synovitis and serve as a biomarker for response to treatment in clinical trials. However, (semi)-automatic segmentation methods and thorough technical and clinical validation are needed to fully adopt DCE-MRI in this setting. We aim to evaluate semi-automatic segmentation of synovial subregions and test-retest repeatability of DCE-MRI to measure synovial perfusion in patients with knee OA. Patients with mild-to-moderate knee OA (KLG 1-3) who were allocated to the control group in a prospective randomized controlled trial of genicular artery embolization were included. Patients underwent MRI at 3T (GE Healthcare) at baseline and after 1 month. Image acquisition consisted of native T1 mapping with variable flip angle, followed by a dual-echo SPGR sequence (DISCO; DIfferential Sub-sampling with Cartesian Ordering), in which the spatial resolution was optimized to allow for visualization of small vessels. After the initial phase, 0.1 ml/kg gadovist was injected and subsequently 34 dynamic phases were acquired, with a temporal resolution of 10.2 seconds. Segmentation of the synovium comprised an initial rough manual segmentation followed by selection of enhancing voxels using a shuffle transform method. Synovial subregions were created by semi-automatic vessel mapping using MeVisLab and mathematically assigning synovial voxels to the corresponding closest genicular artery. Motion compensation was performed using Elastix (Elastix, version 5.0.1), and native T1 mapping and pharmacokinetic modelling was done using open source software for DCE analysis (MADYM, version 4.21.1). The commonly used combination of the Parker literature arterial input function, and the Extended Toft's model pharmacokinetic model was implemented. Voxelwise perfusion parameters were calculated, after which median Ktrans values were extracted for the whole synovium and separately for each synovial subregion. Agreement was visualized with a Bland-Altman plot, and test-retest repeatability was evaluated using intraclass correlation coefficients (ICC; single rating random sample) and within-subject coefficients of variation (CV). 30 patients were included, of which two patients were lost to follow-up and one patient was excluded due to errors during image acquisition. For the remaining 27 participants, median Ktrans values ranged from 0.017 to 0.121. ICC of Ktrans for the whole synovium was 0.51, and CV was 0.14. Sensitivity analysis by exclusion of two statistical outliers yielded an ICC of 0.66 and a CV of 0.11. For the individual synovial regions, ICCs ranged between 0.27 and 0.58 while CVs ranged between 0.13 and 0.25. Semi-automatic segmentation of the synovium and synovial subregions through vessel mapping on DCE-MRI in knee OA is feasible. Perfusion parameters determined through pharmacokinetic modeling have moderate repeatability and should be interpreted with caution. Stichting Coolsingel, Cook Medical, Erasmus MRace Doelmatigheid E.H.G. Oei has received research support from GE Healthcare. The University of Wisconsin receives research support from Bracco Diagnostics. CORRESPONDENCE ADDRESS: [email protected]

Modified Cuckoo Search Algorithm for Motion Vector Estimation

Motion estimation and motion compensation are the accepted process in H.264 and H.265 video coding standard to reduce temporal redundancy. Several fast block matching algorithms have been developed to reduce the calculation cost in the motion estimation process. But quick block matching algorithms often lead to a local minimum. Several researchers used different population-based nature-inspired algorithms to perform block matching. Algorithms like genetic algorithm, differential evolution, particle swarm optimization were used in numerous motion estimation algorithms. Different algorithms used a fitness approximation strategy to reduce computation cost. Jaya algorithm-based block matching is the most efficient block matching algorithm in the available literature. Jaya algorithm is free from algorithmic specific parameter which speeds up the process. This article proposes a few modifications to the traditional cuckoo search algorithm and then, a block matching algorithm was proposed based on the modified cuckoo search algorithm. Fitness approximation, adaptive termination, and zero motion prejudgment modules were used with the modified cuckoo search algorithm to reduce the number of redundant calculations. The performance of the proposed algorithm was compared with the exhaustive search algorithm and other benchmarking algorithms in terms of Peak Signal to Noise Ratio (PSNR), Structure Similarity Index (SSIM), and average search point required to calculate a motion vector for a block. The proposed algorithm delivers better performance compared to the benchmarking algorithms.

Doppler and Pair-wise Optical Flow Constrained 3D Motion Compensation for 3D Ultrasound Imaging.

Volumetric (3D) ultrasound imaging using a 2D matrix array probe is increasingly developed for various clinical procedures. However, 3D ultrasound imaging suffers from motion artifacts due to tissue motions and a relatively low frame rate. Current Doppler-based motion compensation (MoCo) methods only allow 1D compensation in the in-range dimension. In this work, we propose a new 3D-MoCo framework that combines 3D velocity field estimation and a two-step compensation strategy for 3D diverging wave compounding imaging. Specifically, our framework explores two constraints of a round-trip scan sequence of 3D diverging waves, i.e., Doppler and pair-wise optical flow, to formulate the estimation of the 3D velocity fields as a global optimization problem, which is further regularized by the divergence-free and first-order smoothness. The two-step compensation strategy is to first compensate for the 1D displacements in the in-range dimension and then the 2D displacements in the two mutually orthogonal cross-range dimensions. Systematical in-silico experiments were conducted to validate the effectiveness of our proposed 3D-MoCo method. The results demonstrate that our 3D-MoCo method achieves higher image contrast, higher structural similarity, and better speckle patterns than the corresponding 1D-MoCo method. Particularly, the 2D cross-range compensation is effective for fully recovering image quality.

A Novel Motion Compensation Method Applicable to Ground Cartesian Back-Projection Algorithm for Airborne Circular SAR

The Ground-Cartesian factorized back-projection (G-CFBP) is an efficient time-domain processing algorithm without image interpolation, and can realize accurate imaging for curved trajectory synthetic aperture radar (SAR). Its superiority shows good potential in airborne circular SAR (CSAR) imaging. However, the motion compensation (MoCo) based on ground Cartesian back-projection (GCBP) in the airborne CSAR is still a challenge. There are two main problems: one is that the existence of the image spectrum aliasing makes the processing of the phase error estimation inaccurate; the other is that the mapping relationship of the phase error between the image spectrum domain and the azimuth time domain still needs to be studied within GCBP processing chain. To tackle the above two problems, a novel MoCo method applicable to the GCBP algorithm is proposed and can be mainly divided into two steps: the first step is to remove the sub-aperture image spectrum aliasing by a spectrum compression operation; the second step is to establish an analytical phase error structure, which includes an auto-selection criterion of the effective support region for GCBP image. The first step ensures the accuracy of the phase error estimation, and the second step establishes the inverse-mapping relationship of the phase error between the image spectrum and azimuth time. These two procedures are both vital in improving the accuracy and robustness of the GCBP-based MoCo for the CSAR imaging. The processed results of simulated and real data are provided to verify the effectiveness of the proposed method.

Motion Analysis and Compensation of Space-borne Single Pixel Imaging

Motion Analysis and Compensation of Space-borne Single Pixel Imaging

Nonrigid Registration-Based Progressive Motion Compensation for Point Cloud Geometry Compression

Nonrigid Registration-Based Progressive Motion Compensation for Point Cloud Geometry Compression

ViCo-MoCo-DL: Video Coding and Motion Compensation Solutions for Human Activity Recognition Using Deep Learning

ViCo-MoCo-DL: Video Coding and Motion Compensation Solutions for Human Activity Recognition Using Deep Learning

Motion Compensation Method Using Direct Wave Signal for CTSR Bistatic HFSWR

Coast-transmit ship-receive (CTSR) bistatic high-frequency surface wave radar (HFSWR) can fully exploit the flexibility of moving platform and the anti-interference advantages of bistatic radar. However, the platform motion can cause the spread of target echo in the frequency spectrum and reduction of amplitude, which is not conducive to detection. In this letter, a motion compensation method using direct wave signal is proposed for CTSR bistatic HFSWR. First, the frequency shift characteristics of direct wave signal and target echo are analyzed by simulation. The simulation results show that the platform motion has a similar modulation form on both target echo and direct wave signal, and their difference is mainly caused by different azimuth. In addition, the direct wave signal is only affected by the platform motion, so it can be used to obtain the platform motion effect. Then, the proposed method uses direct wave signal as the reference information to estimate and eliminate the phase modulation caused by the platform motion. At last, the effectiveness of the method is verified by the experiment based on simulated and measured attitude data.

Learning Cross-Scale Weighted Prediction for Efficient Neural Video Compression.

Neural video codecs have demonstrated great potential in video transmission and storage applications. Existing neural hybrid video coding approaches rely on optical flow or Gaussian-scale flow for prediction, which cannot support fine-grained adaptation to diverse motion content. Towards more content-adaptive prediction, we propose a novel cross-scale prediction module that achieves more effective motion compensation. Specifically, on the one hand, we produce a reference feature pyramid as prediction sources and then transmit cross-scale flows that leverage the feature scale to control the precision of prediction. On the other hand, for the first time, a weighted prediction mechanism is introduced even if only a single reference frame is available, which can help synthesize a fine prediction result by transmitting cross-scale weight maps. In addition to the cross-scale prediction module, we further propose a multi-stage quantization strategy, which improves the rate-distortion performance with no extra computational penalty during inference. We show the encouraging performance of our efficient neural video codec (ENVC) on several benchmark datasets. In particular, the proposed ENVC can compete with the latest coding standard H.266/VVC in terms of sRGB PSNR on UVG dataset for the low-latency mode. We also analyze in detail the effectiveness of the cross-scale prediction module in handling various video content, and provide a comprehensive ablation study to analyze those important components. Test code is available at https://github.com/USTC-IMCL/ENVC.

An Effective Translational Motion Compensation Approach for High-Resolution ISAR Imaging With Time-Varying Amplitude

An Effective Translational Motion Compensation Approach for High-Resolution ISAR Imaging With Time-Varying Amplitude

A Comparative Analysis of End-To-End Trainable Motion Compensation based Video Super-Resolution Techniques

Video Super-Resolution (VSR) is gaining massive attention in the digital age across various application domains such as video surveillance and ultra-high-definition displays. Deep learning (DL) has recently made significant progress in research and industry domains. Existing researchers used numerous VSR methods based on deep learning to improve the resolution. VSR is based on inter-frame information utilization. Video frames provide the leverage information of data that modalities record sequentially, i.e.; the object's temporal changes cause inconsistent measurements. Consequently, suitable models and algorithms must be developed to provide artifact-free images. This paper goes through two reconstruction methods incorporating various motion data sources. Initially, it describes an iterative approach that implicitly treats dynamic behavior as uncertainty in the forward model. Further direct reconstruction techniques can be derived for an explicit motion map. This paper compares both explicit and implicit motion compensation-based methods. Convolutional neural networks (CNN) and recurrent, residual network (RRN) based methods from a deep learning environment are analyzed to reconstruct the High-Resolution (HR) frames. This paper shows numerical findings for both methods using simulated data. Observations show that RRN can boost the super-resolution (SR) performance and achieve considerable visual quality after reconstruction; however, performance is limited to small datasets due to shallow network parameters. Extensive experiments show that RRN significantly impacts SR results due to its substantial computational efficiency.