Estimated Motion Vector Field Research Articles

Anomaly detection in structural dynamic systems via nonlinearity occurrence analysis using video data

Dynamic loads in disaster events, such as earthquakes, often cause structural damages that severely affect serviceability and even reduce the safety of civil structures or render them unsafe. Most of these damage scenarios can cause the transition from a linear to nonlinear behavior in structural dynamics. Therefore, extracting such nonlinear behaviors is an effective strategy for structural anomaly detection. In recent years, the rapid development of media devices has led to the widespread use of video data, primarily because of the ease of acquisition and lack of contact measurement. This study developed a novel method for detecting anomalies due to structural nonlinearity from video data that could capture target dynamic behaviors via non-contact measurement. The method is based on optical flow, extended repulsive force networks, and morphological operations for extracting singularities due to nonlinear events in the estimated motion vector field in video data. Subsequently, shaking table tests on a three-story shear building model containing a novel controllable hinge bearing verified the effectiveness of the developed method. The results of the feature enhancement process demonstrate the accurate localization of anomalous regions and effective mitigation of unwanted interference using the proposed method. The proposed method facilitates the use of video-based technology to evaluate rapid damage conditions post-earthquake.

Motion vector field phase-to-amplitude resampling for 4D motion-compensated cone-beam CT

We propose a phase-to-amplitude resampling (PTAR) method to reduce motion blurring in motion-compensated (MoCo) 4D cone-beam CT (CBCT) image reconstruction, without increasing the computational complexity of the motion vector field (MVF) estimation approach. PTAR is able to improve the image quality in reconstructed 4D volumes, including both regular and irregular respiration patterns. The PTAR approach starts with a robust phase-gating procedure for the initial MVF estimation and then switches to a phase-adapted amplitude gating method. The switch implies an MVF-resampling, which makes them amplitude-specific. PTAR ensures that the MVFs, which have been estimated on phase-gated reconstructions, are still valid for all amplitude-gated reconstructions. To validate the method, we use an artificially deformed clinical CT scan with a realistic breathing pattern and several patient data sets acquired with a TrueBeamTM integrated imaging system (Varian Medical Systems, Palo Alto, CA, USA). Motion blurring, which still occurs around the area of the diaphragm or at small vessels above the diaphragm in artifact-specific cyclic motion compensation (acMoCo) images based on phase-gating, is significantly reduced by PTAR. Also, small lung structures appear sharper in the images. This is demonstrated both for simulated and real patient data. A quantification of the sharpness of the diaphragm confirms these findings. PTAR improves the image quality of 4D MoCo reconstructions compared to conventional phase-gated MoCo images, in particular for irregular breathing patterns. Thus, PTAR increases the robustness of MoCo reconstructions for CBCT. Because PTAR does not require any additional steps for the MVF estimation, it is computationally efficient. Our method is not restricted to CBCT but could rather be applied to other image modalities.

Motion estimation for cardiac functional analysis using two x-ray computed tomography scans.

This work concerns computed tomography (CT)-based cardiac functional analysis (CFA) with a reduced radiation dose. As CT-CFA requires images over the entire heartbeat, the scans are often performed at 10-20% of the tube current settings that are typically used for coronary CT angiography. A large image noise then degrades the accuracy of motion estimation. Moreover, even if the scan was performed during the sinus rhythm, the cardiac motion observed in CT images may not be cyclic with patients with atrial fibrillation. In this study, we propose to use two CT scan data, one for CT angiography at a quiescent phase at a standard dose and the other for CFA over the entire heart beat at a lower dose. We have made the following four modifications to an image-based cardiac motion estimation method we have previously developed for a full-dose retrospectively gated coronary CT angiography: (a) a full-dose prospectively gated coronary CT angiography image acquired at the least motion phase was used as the reference image; (b) a three-dimensional median filter was applied to lower-dose retrospectively gated cardiac images acquired at 20 phases over one heartbeat in order to reduce image noise; (c) the strength of the temporal regularization term was made adaptive; and (d) a one-dimensional temporal filter was applied to the estimated motion vector field in order to decrease jaggy motion patterns. We describe the conventional method iME1 and the proposed method iME2 in this article. Five observers assessed the accuracy of the estimated motion vector field of iME2 and iME1 using a 4-point scale. The observers repeated the assessment with data presented in a new random order 1 week after the first assessment session. The study confirmed that the proposed iME2 was robust against the mismatch of noise levels, contrast enhancement levels, and shapes of the chambers. There was a statistically significant difference between iME2 and iME1 (accuracy score, 2.08±0.81 versus 2.77±0.98, P<0.01) and the improvement by the score of +0.69 seemed clinically relevant. Inter-observer concordance was good: The inter-class correlation coefficient was 0.63 and Kendall's rank correlation coefficients were in the range of 0.41-0.67 (P<0.01), respectively. Intra-observer reproducibility between sessions was good with the inter-class correlation coefficient of 0.76. We have proposed iME2 method for CT-CFA with two CT scans. The observer study verified the robustness and accuracy of iME2 method and its improved performance over iME1 method.

Registration and Summation of Respiratory-Gated or Breath-Hold PET Images Based on Deformation Estimation of Lung from CT Image

Lung motion due to respiration causes image degradation in medical imaging, especially in nuclear medicine which requires long acquisition times. We have developed a method for image correction between the respiratory-gated (RG) PET images in different respiration phases or breath-hold (BH) PET images in an inconsistent respiration phase. In the method, the RG or BH-PET images in different respiration phases are deformed under two criteria: similarity of the image intensity distribution and smoothness of the estimated motion vector field (MVF). However, only these criteria may cause unnatural motion estimation of lung. In this paper, assuming the use of a PET-CT scanner, we add another criterion that is the similarity for the motion direction estimated from inhalation and exhalation CT images. The proposed method was first applied to a numerical phantom XCAT with tumors and then applied to BH-PET image data for seven patients. The resultant tumor contrasts and the estimated motion vector fields were compared with those obtained by our previous method. Through those experiments we confirmed that the proposed method can provide an improved and more stable image quality for both RG and BH-PET images.

Cardiorespiratory motion-compensated micro-CT image reconstruction using an artifact model-based motion estimation.

Cardiac in vivo micro-CT imaging of small animals typically requires double gating due to long scan times and high respiratory rates. The simultaneous respiratory and cardiac gating can either be done prospectively or retrospectively. In any case, for true 5D imaging, i.e., three spatial dimensions plus one respiratory-temporal dimension plus one cardiac temporal dimension, the amount of information corresponding to a given respiratory and cardiac phase is orders of magnitude lower than the total amount of information acquired. Achieving similar image quality for 5D than for usual 3D investigations would require increasing the amount of data and thus the applied dose to the animal. Therefore, the goal is phase-correlated imaging with high image quality but without increasing the dose level. To achieve this, the authors propose a new image reconstruction algorithm that makes use of all available projection data, also of that corresponding to other motion windows. In particular, the authors apply a motion-compensated image reconstruction approach that sequentially compensates for respiratory and cardiac motion to decrease the impact of sparsification. In that process, all projection data are used no matter which motion phase they were acquired in. Respiratory and cardiac motion are compensated for by using motion vector fields. These motion vector fields are estimated from initial phase-correlated reconstructions based on a deformable registration approach. To decrease the sensitivity of the registration to sparse-view artifacts, an artifact model-based approach is used including a cyclic consistent nonrigid registration algorithm. The preliminary results indicate that the authors' approach removes the sparse-view artifacts of conventional phase-correlated reconstructions while maintaining temporal resolution. In addition, it achieves noise levels and spatial resolution comparable to that of nongated reconstructions due to the improved dose usage. By using the proposed motion estimation, no sensitivity to streaking artifacts has been observed. Using sequential double gating combined with artifact model-based motion estimation allows to accurately estimate respiratory and cardiac motion from highly undersampled data. No sensitivity to streaking artifacts introduced by sparse angular sampling has been observed for the investigated dose levels. The motion-compensated image reconstruction was able to correct for both, respiratory and cardiac motion, by applying the estimated motion vector fields. The administered dose per animal can thus be reduced for 5D imaging allowing for longitudinal studies at the highest image quality.

Motion Vector Field Estimation Using Brightness Constancy Assumption and Epipolar Geometry Constraint

Abstract. In most Photogrammetry and computer vision tasks, finding the corresponding points among images is required. Among many, the Lucas-Kanade optical flow estimation has been employed for tracking interest points as well as motion vector field estimation. This paper uses the IMU measurements to reconstruct the epipolar geometry and it integrates the epipolar geometry constraint with the brightness constancy assumption in the Lucas-Kanade method. The proposed method has been tested using the KITTI dataset. The results show the improvement in motion vector field estimation in comparison to the Lucas-Kanade optical flow estimation. The same approach has been used in the KLT tracker and it has been shown that using epipolar geometry constraint can improve the KLT tracker. It is recommended that the epipolar geometry constraint is used in advanced variational optical flow estimation methods.

Artifact‐resistant motion estimation with a patient‐specific artifact model for motion‐compensated cone‐beam CT

In image-guided radiation therapy (IGRT) valuable information for patient positioning, dose verification, and adaptive treatment planning is provided by an additional kV imaging unit. However, due to the limited gantry rotation speed during treatment the typical acquisition time is quite long. Tomographic images of the thorax suffer from motion blurring or, if a gated 4D reconstruction is performed, from significant streak artifacts. Our purpose is to provide a method that reliably estimates respiratory motion in presence of severe artifacts. The estimated motion vector fields are then used for motion-compensated image reconstruction to provide high quality respiratory-correlated 4D volumes for on-board cone-beam CT (CBCT) scans. The proposed motion estimation method consists of a model that explicitly addresses image artifacts because in presence of severe artifacts state-of-the-art registration methods tend to register artifacts rather than anatomy. Our artifact model, e.g., generates streak artifacts very similar to those included in the gated 4D CBCT images, but it does not include respiratory motion. In combination with a registration strategy, the model gives an error estimate that is used to compensate the corresponding errors of the motion vector fields that are estimated from the gated 4D CBCT images. The algorithm is tested in combination with a cyclic registration approach using temporal constraints and with a standard 3D-3D registration approach. A qualitative and quantitative evaluation of the motion-compensated results was performed using simulated rawdata created on basis of clinical CT data. Further evaluation includes patient data which were scanned with an on-board CBCT system. The model-based motion estimation method is nearly insensitive to image artifacts of gated 4D reconstructions as they are caused by angular undersampling. The motion is accurately estimated and our motion-compensated image reconstruction algorithm can correct for it. Motion artifacts of 3D standard reconstruction are significantly reduced, while almost no new artifacts are introduced. Using the artifact model allows to accurately estimate and compensate for patient motion, even if the initial reconstructions are of very low image quality. Using our approach together with a cyclic registration algorithm yields a combination which shows almost no sensitivity to sparse-view artifacts and thus ensures both high spatial and high temporal resolution.

Motion Segmentation and Tracking with Edge Relaxation and Optimization using Fully Parallel Methods in the Cellular Nonlinear Network Architecture

In this paper we outline a fully parallel and locally connected computation model for the segmentation of motion events in video sequences based on spatial and motion information. Extraction of motion information from video series is very time consuming. Most of the computing effort is devoted to the estimation of motion vector fields, defining objects and determining the exact boundaries of these objects. The split and merge segmentation of different small areas, those obtained by oversegmentation, needs an optimization process. In our proposed algorithm the process starts from an oversegmented image, then the segments are merged by applying the information coming from the spatial and temporal auxiliary data: motion fields and motion history, calculated from consecutive image frames. This grouping process is defined through a similarity measure of neighboring segments, which is based on intensity, speed and the time-depth of motion-history. There is also a feedback for checking the merging process, by this feedback we can accept or refuse the cancellation of a segment-border. Our parallel approach is independent of the number of segments and objects, since instead of graph representation of these components, image features are defined on the pixel level. We use simple VLSI implementable functions like arithmetic and logical operators, local memory transfers and convolution. These elementary instructions are used to build up the basic routines such as motion displacement field detection, disocclusion removal, anisotropic diffusion, grouping by stochastic optimization. This relaxation-based motion segmentation can be a basic step of the effective coding of image series and other automatic motion tracking systems. The proposed system is ready to be implemented in a Cellular Nonlinear Network chip-set architecture.

An efficient true-motion estimator using candidate vectors from a parametric motion model

Some efficient motion estimation algorithms select their output motion vector from a limited number of likely correct candidate, or prediction, vectors. In this paper, next to the known spatial and temporal prediction vectors, an additional and independent prediction is proposed. This candidate is generated with a parametric model describing the global motion in a previously estimated motion vector field. The proposal is elaborated as an addition to the three-dimensional (3-D) recursive search block-matching algorithm. The evaluation shows that a subpixel accurate, true-motion estimator results with a very low operations count.

Robust Estimation of Motion Vector Fields with Discontinuity and Occlusion Using Local Outliers Rejection

This paper proposes the use of local outliers rejection to solve two fundamental problems in estimating motion vector fields: motion vector discontinuity and occlusion. This approach does not introduce extra unknowns and parameters as do another approaches, such as the line process and the occlusion process. Since the objective function of this approach can be calculated locally for each pixel, a highly parallel implementation can be achieved. For the problem of motion vector discontinuity, we argue that since the outliers occur at the motion boundaries, we should reject them locally. We reject outliers at each pixel by thresholding with a threshold reference based on the surrounding neighbors. Thus, our approach is not sensitive to the amplitude of motion, and the normal smoothness assumptions can be fully used on smooth areas. Also, since we apply outlier rejection to the motion model instead of to the measurements, we can reject outliers even when there are more outliers than nonoutliers for a given pixel, such as at a moving corner for an 8-neighborhood system. The proposed approach has been extended to the problem of occlusion using three frames. Simulated annealing using a Gibbs sampler is used to solve the minimization problem. Experiments for synthetic and real image sequences have been conducted to illustrate the effectiveness of the proposed approach.

Bayesian estimation of motion vector fields

A stochastic approach to the estimation of 2D motion vector fields from time-varying images is presented. The formulation involves the specification of a deterministic structural model along with stochastic observation and motion field models. Two motion models are proposed: a globally smooth model based on vector Markov random fields and a piecewise smooth model derived from coupled vector-binary Markov random fields. Two estimation criteria are studied. In the maximum a posteriori probability (MAP) estimation, the a posteriori probability of motion given data is maximized, whereas in the minimum expected cost (MEC) estimation, the expectation of a certain cost function is minimized. Both algorithms generate sample fields by means of stochastic relaxation implemented via the Gibbs sampler. Two versions are developed: one for a discrete state space and the other for a continuous state space. The MAP estimation is incorporated into a hierarchical environment to deal efficiently with large displacements.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>