Combined Motion Estimation Research Articles

Motion Correction for Separate Mandibular and Cranial Movements in Cone Beam CT Reconstructions.

Patient motions are a repeatedly reported phenomenon in oral and maxillofacial cone beam CT scans, leading to reconstructions of limited usability. In certain cases, independent movements of the mandible induce unpredictable motion patterns. Previous motion correction methods are not able to handle such complex cases of patient movements. Our goal was to design a combined motion estimation and motion correction approach for separate cranial and mandibular motions, solely based on the 2D projection images from a single scan. Our iterative three-step motion correction algorithm models the two articulated motions as independent rigid motions. First of all, we segment cranium and mandible in the projection images using a deep neural network. Next, we compute a 3D reconstruction with the poses of the object's trajectories fixed. Third, we improve all poses by minimizing the projection error while keeping the reconstruction fixed. Step two and three are repeated alternately. We find that our marker-free approach delivers reconstructions of up to 85% higher quality, with respect to the projection error, and can improve on already existing techniques, which model only a single rigid motion. We show results of both synthetic and real data created in different scenarios. The reconstruction of motion parameters in a real environment was evaluated on acquisitions of a skull mounted on a hexapod, creating a realistic, easily reproducible motion profile. The proposed algorithm consistently enhances the visual quality of motion impaired CBCT scans, thus eliminating the need for a re-scan in certain cases, considerably lowering radiation dosage for the patient. It can flexibly be used with differently sized regions of interest and is even applicable to localtomography. This article is protected by copyright. All rights reserved.

Multi-Frame Blind Super-Resolution Based on Joint Motion Estimation and Blur Kernel Estimation

Multi-frame super-resolution makes up for the deficiency of sensor hardware and significantly improves image resolution by using the information of inter-frame and intra-frame images. Inaccurate blur kernel estimation will enlarge the distortion of the estimated high-resolution image. Therefore, multi-frame blind super resolution with unknown blur kernel is more challenging. For the purpose of reducing the impact of inaccurate motion estimation and blur kernel estimation on the super-resolved image, we propose a novel method combining motion estimation, blur kernel estimation and super resolution. The confidence weight of low-resolution images and the parameter value of the motion model obtained in image reconstruction are added to the modified motion estimation and blur kernel estimation. At the same time, Jacobian matrix, which can better describe the motion change, is introduced to further correct the error of motion estimation. Based on the results acquired from the experiments on synthetic data and real data, the superiority of the proposed method over others is obvious. The reconstructed high-resolution image retains the details of the image effectively, and the artifacts are greatly reduced.

Advanced 4-dimensional cone-beam computed tomography reconstruction by combining motion estimation, motion-compensated reconstruction, biomechanical modeling and deep learning

4-Dimensional cone-beam computed tomography (4D-CBCT) offers several key advantages over conventional 3D-CBCT in moving target localization/delineation, structure de-blurring, target motion tracking, treatment dose accumulation and adaptive radiation therapy. However, the use of the 4D-CBCT in current radiation therapy practices has been limited, mostly due to its sub-optimal image quality from limited angular sampling of cone-beam projections. In this study, we summarized the recent developments of 4D-CBCT reconstruction techniques for image quality improvement, and introduced our developments of a new 4D-CBCT reconstruction technique which features simultaneous motion estimation and image reconstruction (SMEIR). Based on the original SMEIR scheme, biomechanical modeling-guided SMEIR (SMEIR-Bio) was introduced to further improve the reconstruction accuracy of fine details in lung 4D-CBCTs. To improve the efficiency of reconstruction, we recently developed a U-net-based deformation-vector-field (DVF) optimization technique to leverage a population-based deep learning scheme to improve the accuracy of intra-lung DVFs (SMEIR-Unet), without explicit biomechanical modeling. Details of each of the SMEIR, SMEIR-Bio and SMEIR-Unet techniques were included in this study, along with the corresponding results comparing the reconstruction accuracy in terms of CBCT images and the DVFs. We also discussed the application prospects of the SMEIR-type techniques in image-guided radiation therapy and adaptive radiation therapy, and presented potential schemes on future developments to achieve faster and more accurate 4D-CBCT imaging.

Compressive Sensing Reconstruction for Video: An Adaptive Approach Based on Motion Estimation

This paper focuses on the problem of causally reconstructing compressive sensing (CS) captured video. The state-of-the-art causal approaches usually assume that the signal support is static or changing sufficiently slowly over time, where magnetic resonance imaging is widely used as a motivating example. However, such an assumption is too restrictive for many other video applications, where the signal support changes rapidly. In this paper, we propose a framework that combines motion estimation (ME), the Kalman filter (KF), and CS to adapt the reconstruction process to motions in the video so that the slowly changing assumption on the signal support is relaxed and consequently is more suitable for video reconstruction. Explicit and implicit ME are designed to provide motion-aware predictions, upon which a modified KF procedure is applied. Furthermore, three CS algorithms with embedded ME and KF are developed, and theoretical analyses are conducted via reconstruction error upper bounds to characterize the various factors that affect reconstruction accuracy. Extensive simulations utilizing actual videos are carried out, and the superiority of our methods is demonstrated.

A MAP Approach for Joint Motion Estimation, Segmentation, and Super Resolution

Super resolution image reconstruction allows the recovery of a high-resolution (HR) image from several low-resolution images that are noisy, blurred, and down sampled. In this paper, we present a joint formulation for a complex super-resolution problem in which the scenes contain multiple independently moving objects. This formulation is built upon the maximum a posteriori (MAP) framework, which judiciously combines motion estimation, segmentation, and super resolution together. A cyclic coordinate descent optimization procedure is used to solve the MAP formulation, in which the motion fields, segmentation fields, and HR images are found in an alternate manner given the two others, respectively. Specifically, the gradient-based methods are employed to solve the HR image and motion fields, and an iterated conditional mode optimization method to obtain the segmentation fields. The proposed algorithm has been tested using a synthetic image sequence, the "Mobile and Calendar" sequence, and the original "Motorcycle and Car" sequence. The experiment results and error analyses verify the efficacy of this algorithm.

Object based refinement of motion vector fields applying probabilistic homogenization rules

This paper presents a new image analysis method, combining motion estimation and image segmentation. Whereas none of these methods, used on its own, delivers error free meta-information, an appropriate combination leads to an orthogonalization of these errors. This method is applied to improve the quality of motion vector fields.

Automatic reconstruction of stationary 3-D objects from multiple uncalibrated camera views

A system for the automatic reconstruction of real-world objects from multiple uncalibrated camera views is presented. The camera position and orientation for all views, the 3-D shape of the rigid object, as well as the associated color information, are recovered from the image sequence. The system proceeds in four steps. First, the internal camera parameters describing the imaging geometry are calibrated using a reference object. Second, an initial 3-D description of the object is computed from two views. This model information is then used in a third step to estimate the camera positions for all available views using a novel linear 3-D motion and shape estimation algorithm. The main feature of this third step is the simultaneous estimation of 3-D camera-motion parameters and object shape refinement with respect to the initial 3-D model. The initial 3-D shape model exhibits only a few degrees of freedom and the object shape refinement is defined as flexible deformation of the initial shape model. Our formulation of the shape deformation allows the object texture to slide on the surface, which differs from traditional flexible body modeling. This novel combined shape and motion estimation using sliding texture considerably improves the calibration data of the individual views in comparison to fixed-shape model based camera-motion estimation. Since the shape model used for model based camera-motion estimation is only approximate, a volumetric 3-D reconstruction process is initiated in the fourth step that combines the information from ail views simultaneously. The recovered object consists of a set of voxels with associated color information that describes even fine structures and details of the object. New views of the object can be rendered from the recovered 3-D model, which has potential applications in virtual reality or multimedia systems and the emerging field of video coding using 3-D scene models.

Stereo image analysis for multi-viewpoint telepresence applications

Abstract An improved method for combined motion and disparity estimation in stereo sequences to synthesize temporally and perspectively intermediate views is presented. The main problems of matching methods for motion and disparity analysis are summarised. The improved concept is based on a modified block matching algorithm in which a cost function consisting of feature- and area-based correlation together with an appropriately weighted temporal smoothness term is applied. Considerable improvements have been obtained with respect to the motion and disparity assignments by introducing a confidence measure to evaluate the reliability of estimated correspondences. In occluded image areas, enhanced results are obtained applying an edge-assisted vector interpolation strategy. Two different image synthesis concepts are presented. The first concept is suitable for processing natural stereo sequences. It comprises the detection of covered and uncovered image areas caused by motion or disparity. This information is used to switch between different interpolation and extrapolation modes during the computation of intermediate views. The proposed object-based approach is suitable for processing typical video conference scenes containing extremely large occluded image regions and keeping implementation costs low. A set of stereo sequences has been processed. The performed computer simulations show that a continuous motion parallax can be obtained with good image quality by using sequences taken with stereo cameras having large interaxial distances.

A semi-automatic system for edge tracking with snakes

Active contour models, or “snakes,” developed in (Kass et al. 1988), use a simple physical model to track edges in image sequences. Snakes as originally defined however, tend to shrink, stretch and slide back and forth in unwanted ways along a tracked edge and are also confused by multiple edges, always grabbing the nearest one. In this paper a semi-automatic system is presented that combines motion estimation techniques with snakes to overcome these problems. An algorithm is presented that uses a block matching technique to guide the endpoints of the snake, optical flow to push the snake in the direction of the underlying motion, followed by the traditional snake edge-fitting minimization process. We use this technique for tracking facial features of an actor for driving computer animated characters.