Motion Compensation Step Research Articles

Solid-State-LiDAR-Inertial-Visual Odometry and Mapping via Quadratic Motion Model and Reflectivity Information

This paper proposes a solid-state-LiDAR-inertial-visual fusion framework containing two subsystems: the solid-state-LiDAR-inertial odometry (SSLIO) subsystem and the visual-inertial odometry (VIO) subsystem. Our SSLIO subsystem has two novelties that enable it to handle drastic acceleration and angular velocity changes: (1) the quadratic motion model is adopted in the in-frame motion compensation step of the LiDAR feature points, and (2) the system has a weight function for each residual term to ensure consistency in geometry and reflectivity. The VIO subsystem renders the global map in addition to further optimizing the state output by the SSLIO. To save computing resources, we calibrate our VIO subsystem’s extrinsic parameter indirectly in advance, instead of using real-time estimation. We test the SSLIO subsystem using publicly available datasets and a steep ramp experiment, and show that our SSLIO exhibits better performance than the state-of-the-art LiDAR-inertial SLAM algorithm Point-LIO in terms of coping with strong vibrations transmitted to the sensors due to the violent motion of the crawler robot. Furthermore, we present several outdoor field experiments evaluating our framework. The results show that our proposed multi-sensor fusion framework can achieve good robustness, localization and mapping accuracy, as well as strong real-time performance.

A Monostatic Forward-Looking Staring Spotlight SAR Raw Data Generation and Hybrid-Domain Image Formation Modifications Based on Extended Azimuth Nonlinear Chirp Scaling Autofocusing

Forward-looking staring spotlight synthetic aperture radar (FSS-SAR) offers operational advantages over conventional squinted SAR counterparts. However, due to its complex observation configuration, pixel anomalies occur when processing raw data. For instance, in the forward-looking geometry, symmetrically distributed point scatterers along the flight path have the same Doppler history, resulting in azimuth ambiguities, whereas staring observation narrows and skews the 2D received spectrum, reducing signal orthogonality and causing spectral folding. As a result, not only is the Doppler frequency gradient too small to be precisely measured in the FSS-SAR mode, limiting the ability to achieve high azimuth angular resolution, but it also suffers from azimuth ambiguities when forming an image. To overcome such azimuth-variant characteristics known as inherent resolutional anomalies, a hybrid-domain image formation algorithm (IFA) based on azimuth scaling is modified here. As an approach, radiometric bias correction (RBC) techniques in conjunction with an extended azimuth nonlinear chirp scaling (ANCS) algorithm capable of equalizing range space variance in the azimuth direction, which benefits from an embedded autofocusing step, have been designed for raw data processing. Given the fact that the platform experiences trajectory deviations under a real-flight condition compared to an ideal trajectory, a pre-processing motion compensation (MOCO) step based on Kalman filtering is included within the IFA. The entire proposed IFA solution tackled the inherent resolutional anomalies of FSS-SAR and succeeded in reconstructing the focused image without azimuthal anomalies. To further assess the IFA, a modified Cartesian back-projection (BP) algorithm was developed along with other objective evaluation scenarios, all of which confirm that the proposed hybrid-domain extended ANCS algorithm is valid for the FSS-SAR application.

High-Resolution SAR Imaging with Azimuth Missing Data Based on Sub-Echo Segmentation and Reconstruction

Due to the substantial electromagnetic interference, radar interruptions, and other factors, the SAR system may fail to receive valid data in some azimuth areas. This phenomenon is known as Azimuth Missing Data (AMD). If classical SAR imaging algorithms are performed directly using AMD echo, the imaging results may be defocused or even display false targets, which seriously affects the accuracy of the image. Thus, we proposed a Sub-echo Segmentation and Reconstruction Azimuth Missing Data SAR Imaging Algorithm (SSR-AMDIA) to solve the problem of incomplete echo SAR imaging in this article. Instead of using the motion compensation step of the Polar Format algorithm (PFA) to recover the full echo from the AMD echo, the proposed SSR-AMDIA eliminates the effect of the planar approximation in PFA and expands the maximum depth of focus (DOF). The raw AMD echo was first subjected to range compression and Range Cell Migration Correction (RCMC), after which the AMD-RCMC echo was divided along the range direction. Then, we constructed a series of phase compensation functions based on the sub-segment AMD-RCMC echoes to guarantee the perfect recovery of the full RCMC echoes corresponding to the sub-scenes. Finally, we combined them to obtain the complete RCMC echo, and an excellent focused imaging result was then obtained via azimuth compression. Simulation and experimental data verified the effectiveness of the proposed algorithm. Furthermore, we derived the mathematical expressions for the two-dimensional maximum DOFs of the proposed algorithm. In contrast to the State-Of-the-Art (SOA) AMDIA, the SSR-AMDIA can obtain a superior imaging performance in a larger imaging scope under the conditions of most AMD cases.

Scalable Video Coding Using Accordion Discrete Wavelet Transform and Tucker Decomposition for Multimedia Applications

The digital world demands the transmission and storage of high quality video for streaming and broadcasting applications, the constraints are the network bandwidth and the memory of devices for the various multimedia and scientific applications, the video consists of spatial and temporal redundancies. The objective of any video compression algorithm is to eliminate the redundant information from the video signal during compression for effective transmission and storage. The correlation between the successive frames has not been exploited enough by the current compression algorithms. In this paper, a novel method for video compression is presented. The proposed model, applies the transformation on set of group of pictures (GOP). The high spatial correlation is achieved from the spatial and temporal redundancy of GOP by accordion representation and this helps to bypass the computationally demanding motion compensation step. The core idea of the proposed technique is to apply Tucker Decomposition (TD) on the Discrete Wavelet Transform (DWT) coefficients of the Accordion model of the GOP. We use DWT to separate the video in to different sub-images and TD to efficiently compact the energy of sub-images. The blocking artifacts will be considerably eliminated as the block size is huge. The proposed method attempts to reduce the spatial and temporal redundancies of the video signal to improve the compression ratio, computation time, and PSNR. The experimental results prove that the proposed method is efficient especially in high bit rate and with slow motion videos.

Head motion correction based on filtered backprojection for x-ray CT imaging.

For head x-ray CT imaging, the head needs to remain motionless during the scan. In clinical practice, however, head motion is sometimes unavoidable depending on the patient. The motion can occur abruptly during the scan and can be unpredictable. It thereby causes motion artifacts such as tissue blurring or doubled edges around the skull area. To mitigate this problem, we propose a 3D head motion estimation (ME) and compensation algorithm based on filtered backprojection. If a patient moves his or her head during the scan, a motion-corrupted sinogram is obtained. Modeling the head motion as a 3D rigid transformation, we develop a motion-compensated (MC) reconstruction algorithm based on the FDK algorithm. To determine the head motion of a rigid transformation, we propose two optimization-based ME schemes depending on the degree of head motion, both of which are performed by updating motion parameters and the corresponding MC reconstructed image alternatively until the proposed cost function is minimized for the MC reconstructed image. In particular, to improve the robustness in the case of large motion, we propose attaching a fiducial marker to the head so that more reliable motion parameters can be initialized by determining the marker position, before the optimization. To evaluate the proposed algorithm, a numerical phantom with realistic, continuous, and smoothly varying motion, and a moving physical phantom are used with a gantry rotation time of 1s. In the simulation using a numerical phantom and in the experiment using a physical phantom, the proposed algorithm provides well-restored 3D motion-compensated images in both cases of small and large motion. In particular, in the case of large motion of the physical phantom, using a fiducial marker, we obtain remarkable improvement of image quality in cerebral arteries and a lesion as well as the skull. Quantitative evaluations using the image sharpness and root-mean-square error also show noticeable improvement of image quality in both simulations and experiments. We propose a framework for head motion correction in an axial CT scan, which consists of motion estimation and compensation steps. Two image-based ME algorithms for rigid motion tracking are developed according to the degree of head motion. The estimated motion information is then used for MC image reconstruction. Both motion estimation and compensation algorithms are based on computationally efficient filtered backprojection. Excellent performance of the proposed framework is illustrated by means of simulations using a numerical phantom and experiments using a physical phantom.

A novel moving object segmentation framework utilizing camera motion recognition for H.264 compressed videos

This paper presents a novel coarse to fine moving object segmentation framework for H.264/AVC compressed videos. The proposed framework integrates the global motion estimation and global motion compensation steps in the segmentation pipeline unlike previous techniques which did not consider such an integration. The integration is based on testing for presence of global motion by classifying the interframe motion vectors into moving camera class and still camera class. The decision boundary separating these two classes is learnt from the training video data. The integration automates the moving object segmentation to be applicable for static, moving and combination of static/moving camera cases which to the best of our knowledge has not been carried out earlier. Further, a novel coarse segmentation technique is proposed by decomposing the inter-frame motion vectors into wavelet sub-bands and utilizing logical operations on LH, HL and HH sub-band wavelet coefficients. The premise is based on the fact that since the LH, HL and HH sub-bands contain the detail information pertaining to horizontal, vertical and diagonal moving blocks respectively, they can be exploited to identify the coarse moving boundaries. The coarse segmentation is fast in comparison to state-of-the-art coarse segmentation methods as demonstrated by our experiments. Finally, these coarse boundaries are modeled in an energy minimization framework and shown that by minimizing the energy using graph cut optimization the segmentation is refined to obtain the fine segmentation. The proposed framework is tested on a number of standard video sequences encoded with H.264/AVC JM encoder and comparison is carried out with state-of-the-art compressed domain moving object segmentation methods as well as with an existing state-of-the-art pixel domain method to establish and validate the proposed moving object segmentation framework.

An R-D optimized transcoding resilient motion vector selection

Selection of motion vector (MV) has a significant impact on the quality of an encoded, and particularly a transcoded video, in terms of rate-distortion (R-D) performance. The conventional motion estimation process, in most existing video encoders, ignores the rate of residuals by utilizing rate and distortion of motion compensation step. This approach implies that the selected MV depends on the quantization parameter. Hence, the same MV that has been selected for high bit rate compression may not be suitable for low bit rate ones when transcoding the video with motion information reuse technique, resulting in R-D performance degradation. In this paper, we propose an R-D optimized motion selection criterion that takes into account the effect of residual rate in MV selection process. Based on the proposed criterion, a new two-piece Lagrange multiplier selection is introduced for motion estimation process. Analytical evaluations indicate that our proposed scheme results in MVs that are less sensitive to changes in bit rate or quantization parameter. As a result, MVs in the encoded bitstream may be used even after the encoded sequence has been transcoded to a lower bit rate one using re-quantization. Simulation results indicate that the proposed technique improves the quality performance of coding and transcoding without any computational overhead.

Segmenting moving objects from a freely moving camera with an effective segmentation cue

It is challenging to segment moving objects in a video sequence viewed from a freely moving camera. Within the conditional random field framework, we propose a new segmentation cue to solve this problem. The cue is defined as a spatial–color likelihood ratio map by combining the foreground and background appearance models which are extracted from the previous segmentation result. Specifically, to make the cue suitable for the current segmentation task in the presence of complex camera motions, we introduce a camera motion compensation step in the process of evaluating the likelihood ratio map. We validate the proposed segmentation cue using several video sequences taken by hand-held cameras in outdoor urban scenes.

A complete non-predictive video compression scheme based on a 3D to 2D geometric transform

The proposed method consists on ‘3D-to-2D’ transformation of the temporal frames that allows exploring the temporal redundancy of the video using 2D transforms and avoiding the computationally demanding motion compensation step. This transformation turns video spatial temporal correlation into high spatial correlation. In this paper, we explore the proposed method performances and try to better show what it actually offers to users. The paper presents also the extensions chosen to reduce the perceived artefacts and increase the perceptual as well as objective (PSNR) decoded video quality, which is actually competitive with state-of-the-art video coders, especially when operating complexity is taken into account.

Focusing of Airborne Synthetic Aperture Radar Data From Highly Nonlinear Flight Tracks

Standard focusing of data from synthetic aperture radar (SAR) assumes a straight recording track of the sensor platform. Small nonlinearities of airborne platform tracks are corrected for during a motion-compensation step while maintaining the assumption of a linear flight path. This paper describes the processing of SAR data acquired from nonlinear tracks, typical of sensors mounted on small aircraft or drones flying at low altitude. Such aircraft do not fly along straight tracks, but the trajectory depends on topography, influences of weather and wind, or the shape of areas of interest such as rivers or traffic routes. Two potential approaches for processing SAR data from such highly nonlinear flight tracks are proposed, namely, a patchwise frequency-domain processing and mosaicking technique and a time-domain back-projection-based technique. Both are evaluated with the help of experimental data featuring tracks with altitude changes, a double bend, a 90deg curve, and a linear flight track. In order to assess the quality of the focused data, close-ups of amplitude images are compared, impulse response functions of a point target are analyzed, and the coherence is evaluated. The experimental data were acquired by the German Aerospace Center's E-SAR L-band system.