Temporal Sequence Of Images Research Articles

TH-CD-303-11: Simultaneous Reference-Image and Deformation-Field Reconstruction for 4DCBCT with Periodic and Non-Periodic Breathing Based On 5D Respiratory Motion Model

Purpose: 4DCBCT reconstructs a temporal sequence of CBCT images, and often involves the binning of projection data to each temporal phase. Thus it suffers from binning errors during non-periodic breathing. The 5D model parameterizes respiratory motion by breathing amplitude and rate (e.g. tidal volume and airflow) independent of breathing irregularity. We propose to develop a new 4DCBCT reconstruction method based on the 5D model. Methods: The proposed CBCT technique requires that the breathing amplitude and rate are measured during CBCT. The proposed 5D-model-based method reconstructs a reference breathing phase, and time-independent deformation fields corresponding to the motion due to amplitude and rate. Therefore, the new method reconstructs 7 three-dimensional spatial maps, with reduced numbers of unknowns compared with the standard 4DCBCT method when 10 or more breathing phases are reconstructed. In addition, the 5D method explicitly manages non-periodic breathing.The 5D method is formulated as a simultaneous reference-image and deformation-field reconstruction problem, with total-variation (TV) regularization for both images and motions. Then the 4DCBCT reconstruction is carried out by alternately solving image reconstruction and image registration, in which the split Bregman method is used to reconstruct the reference image, and the Chambolle's duality-based algorithm is used to reconstruct the differential deformation fields. Moreover, the proximal terms are applied to ensure the convergence of the developed optimization algorithm for this nonconvex problem. Results: The proposed 5D method was validated in simulation based on measurements of a lung patient, in comparison with the state-of-art spatiotemporal-TV-based 4DCBCT iterative method. And the results show that the 5D method had significantly improved image quality and quantitative error, for both periodic and non-periodic breathing studies. Conclusion: We have developed a new 4DCBCT method based on 5D respiratory motion model, with improved image reconstruction during both periodic and non-periodic breathing studies. Jiulong Liu and Hao Gao were partially supported by the NSFC (#11405105), the 973 Program (#2015CB856000) and the Shanghai Pujiang Talent Program (#14PJ1404500).

A Hidden Markov Models Approach for Crop Classification: Linking Crop Phenology to Time Series of Multi-Sensor Remote Sensing Data

Vegetation monitoring and mapping based on multi-temporal imagery has recently received much attention due to the plethora of medium-high spatial resolution satellites and the improved classification accuracies attained compared to uni-temporal approaches. Efficient image processing strategies are needed to exploit the phenological information present in temporal image sequences and to limit data redundancy and computational complexity. Within this framework, we implement the theory of Hidden Markov Models in crop classification, based on the time-series analysis of phenological states, inferred by a sequence of remote sensing observations. More specifically, we model the dynamics of vegetation over an agricultural area of Greece, characterized by spatio-temporal heterogeneity and small-sized fields, using RapidEye and Landsat ETM+ imagery. In addition, the classification performance of image sequences with variable spatial and temporal characteristics is evaluated and compared. The classification model considering one RapidEye and four pan-sharpened Landsat ETM+ images was found superior, resulting in a conditional kappa from 0.77 to 0.94 per class and an overall accuracy of 89.7%. The results highlight the potential of the method for operational crop mapping in Euro-Mediterranean areas and provide some hints for optimal image acquisition windows regarding major crop types in Greece.

Dynamic imaging of accommodation by swept-source anterior segment optical coherence tomography

To study the accommodation process in normal eyes using a commercially available clinical system based on swept-source anterior segment optical coherence tomography (AS-OCT). Ophthalmology Department, University of Parma, Italy. Evaluation of diagnostic technology. Right eyes were analyzed using swept-source AS-OCT (Casia SS-1000). The optical vergence of the internal coaxial fixation target was adjusted during imaging to obtain monocular accommodation stimuli with different amplitudes (0, 3.0, 6.0, and 9.0 diopters [D]). Overlapping of real and conjugate OCT images enabled imaging of all the anterior segment optical surfaces in a single frame. Central corneal thickness (CCT), anterior chamber depth (ACD), and lens thickness were extracted from the OCT scans acquired at different static accommodation stimulus amplitudes. The crystalline lens was analyzed dynamically during accommodation and disaccommodation by acquiring sequential OCT images of the anterior segment at a rate of 8 frames per second. The lens thickness was extracted from the temporal sequence of OCT images and plotted as a function of time. The study analyzed 14 eyes of 14 subjects aged 18 to 46 years. During accommodation, the decrease in the ACD was statistically significant (P < .05), as were the increase in the lens thickness (P < .001) and the slight movement forward of the lens central point (P < .01). The CCT and anterior chamber width measurements did not change statistically significantly during accommodation. The lens thickness at 0 D was positively correlated with age (P < .01). High-resolution real-time imaging and biometry of the accommodating anterior segment can be effectively performed using a commercially available swept-source AS-OCT clinical device. No author has a financial or proprietary interest in any material or method mentioned.

Optimized Comics-Based Storytelling for Temporal Image Sequences

We propose a system to transform any temporal image sequence into a comics-based presentation, as an effective and interesting storytelling manner. Three main components, including page allocation, layout selection, and speech balloon placement, are respectively formulated as optimization problems, and systematic approaches are proposed to find solutions. Page allocation is viewed as a labeling problem, and the best solution is determined by the genetic algorithm. Importance values of images and predefined layouts are both represented in vector forms, and the best layout is selected by finding the best match between vectors. Feasible solutions of speech balloons constitute a solution space, and the best solution that jointly describes the best locations of all balloons in a page is determined by the particle swarm optimization algorithm. Objective evaluation and subjective evaluation are designed from various perspectives to demonstrate effectiveness and superiority of the proposed system.

Calculation of Cardiac Kinetic Energy Index from PET images.

Cardiac function can be assessed from displacement measurements in imaging modalities from nuclear medicine Using positron emission tomography (PET) image sequences with Rubidium-82, we propose and estimate the total Kinetic Energy Index (KEf) obtained from the velocity field, which was calculated using 3D optical flow(OF) methods applied over the temporal image sequence. However, it was found that the brightness of the image varied unexpectedly between frames, violating the constant brightness assumption of the OF method and causing large errors in estimating the velocity field. Therefore total brightness was equalized across image frames and the adjusted configuration tested with rest perfusion images acquired from individuals with normal (n=30) and low (n=33) cardiac function. For these images KEf was calculated as 0.5731±0.0899 and 0.3812±0.1146 for individuals with normal and low cardiac function respectively. The ability of KEf to properly classify patients into the two groups was tested with a ROC analysis, with area under the curve estimated as 0.906. To our knowledge this is the first time that KEf has been applied to PET images.

A Kalman Filter-Based Method to Generate Continuous Time Series of Medium-Resolution NDVI Images

A data assimilation method to produce complete temporal sequences of synthetic medium-resolution images is presented. The method implements a Kalman filter recursive algorithm that integrates medium and moderate resolution imagery. To demonstrate the approach, time series of 30-m spatial resolution NDVI images at 16-day time steps were generated using Landsat NDVI images and MODIS NDVI products at four sites with different ecosystems and land cover-land use dynamics. The results show that the time series of synthetic NDVI images captured seasonal land surface dynamics and maintained the spatial structure of the landscape at higher spatial resolution. The time series of synthetic medium-resolution NDVI images were validated within a Monte Carlo simulation framework. Normalized residuals decreased as the number of available observations increased, ranging from 0.2 to below 0.1. Residuals were also significantly lower for time series of synthetic NDVI images generated at combined recursion (smoothing) than individually at forward and backward recursions (filtering). Conversely, the uncertainties of the synthetic images also decreased when the number of available observations increased and combined recursions were implemented.

Active contour-based cell segmentation during freezing and its application in cryopreservation.

Water permeability of the plasma membrane plays an important role in making optimal cryopreservation protocols for different types of cells. To quantify water permeability effectively, automated cell volume segmentation during freezing is necessary. Unfortunately, there exists so far no efficient and accurate segmentation method to handle this kind of image processing task gracefully. The existence of extracellular ice and variable background present significant challenges for most traditional segmentation algorithms. In this paper, we propose a novel approach to reliably extract cells from the extracellular ice, which attaches to or surrounds cells. Our method operates on temporal image sequences and is composed of two steps. First, for each image from the sequence, a greedy search strategy is employed to track approximate locations of cells in motion. Second, we utilize a localized competitive active contour model to obtain the contour of each cell. Based on the first step's result, the initial contour for level set evolution can be determined appropriately, thus considerably easing the pain of initialization for an active contour model. Experimental results demonstrate that the proposed method is efficient and effective in segmenting cells during freezing.

Crop Phenology Estimation Using a Multitemporal Model and a Kalman Filtering Strategy

In this letter, a new approach for crop phenology estimation with remote sensing is presented. The proposed methodology is aimed to exploit tools from a dynamical system context. From a temporal sequence of images, a geometrical model is derived, which allows us to translate this temporal domain into the estimation problem. The evolution model in state space is obtained through dimensional reduction by a principal component analysis, defining the state variables, of the observations. Then, estimation is achieved by combining the generated model with actual samples in an optimal way using a Kalman filter. As a proof of concept, an example with results obtained with this approach over rice fields by exploiting stacks of TerraSAR-X dual polarization images is shown.

On-Vehicle Video Localization Technique based on Video Search using Real Data on the Web

Recently, the mounting of on-vehicle camera is increasing to general cars. Because of this, some users start to upload the on-vehicle videos to web. So that, a number of on-vehicle videos are available nowadays. In this paper, in order to localize car, we propose the efficient matching method for such on-vehicle videos using Temporal Height Image (THI), Affine SIFT and Bag of Feature. THI retains information of relative building height from temporal image sequence. Then we extract robust features from the THI by using Affine SIFT. We realize efficient matching by expressing their features using Bag of features. We conducted experiments to show the efficiency of the proposed method by real image sequences of the city.

Migration of Ca II H bright points in the internetwork

The migration of magnetic bright point-like features (MBP) in the lower solar atmosphere reflects the dispersal of magnetic flux as well as the horizontal flows of the atmospheric layer they are embedded in. We analyse trajectories of the proper motion of intrinsically magnetic, isolated internetwork Ca II H MBPs (mean lifetime 461 +- 9 s) to obtain their diffusivity behaviour. We use seeing-free high spatial and temporal resolution image sequences of quiet-Sun, disc-centre observations obtained in the Ca II H 3968 {\AA} passband of the Sunrise Filter Imager (SuFI) onboard the Sunrise balloon-borne solar observatory. Small MBPs in the internetwork are automatically tracked. The trajectory of each MBP is then calculated and described by a diffusion index ({\gamma}) and a diffusion coefficient (D). We further explore the distribution of the diffusion indices with the help of a Monte Carlo simulation. We find {\gamma} = 1.69 +- 0.08 and D = 257 +- 32 km^2/s averaged over all MBPs. Trajectories of most MBPs are classified as super-diffusive, i.e., {\gamma} > 1, with the determined {\gamma} being to our knowledge the largest obtained so far. A direct correlation between D and time-scale ({\tau}) determined from trajectories of all MBPs is also obtained. We discuss a simple scenario to explain the diffusivity of the observed, relatively short-lived MBPs while they migrate within a small area in a supergranule (i.e., an internetwork area). We show that the scatter in the {\gamma} values obtained for individual MBPs is due to their limited lifetimes. The super-diffusive MBPs can be well-described as random walkers (due to granular evolution and intergranular turbu- lence) superposed on a large systematic (background) velocity, caused by granular, mesogranular and supergranular flows.

Diaphragmatic Surface Reconstruction from Massive Temporal Registration of Orthogonal MRI Sequences

The diaphragm is an important component of the respiratory pumping process. Orthogonal temporal sequences of MR images are acquired in free breathing without use of contrast. The vertical intersection between two orthogonal temporal sequences is determined, the diaphragmatic level in the vertical segment is determined and used to perform the temporal registration. Previous works used similar temporal registration algorithms to reconstruct the diaphragmatic surface. However, it is shown that the available information was not efficiently used. The proposed method massively performs temporal registrations, and then validates the determined temporal registrations. The images to be used in the diaphragmatic surface reconstruction are determined and its associated diaphragmatic levels are collected. Positions with inconsistencies have their diaphragmatic level determined by interpolation. The proposed algorithm is tested with MR image sequences obtained from a healthy subject.

Groupwise Elastic Registration by a New Sparsity-Promoting Metric: Application to the Alignment of Cardiac Magnetic Resonance Perfusion Images

This paper proposes a methodology for the joint alignment of a sequence of images based on a groupwise registration procedure by using a new family of metrics that exploit the expected sparseness of the temporal intensity curves corresponding to the aligned points. Therefore, this methodology is able to tackle the alignment of temporal sequences of images in which the represented phenomenon varies in time. Specifically, we have applied it to the correction of motion in contrast-enhanced first-pass perfusion cardiac magnetic resonance images. The time sequence is elastically registered as a whole by using the aforementioned family of multi-image metrics and jointly optimizing the parameters of the transformations involved. The proposed metrics are able to cope with dynamic changes in the intensity content of corresponding points in the sequence guided by the assumption that these changes allow for a sparse representation in a properly selected frame. Results have shown the statistically significant improvement in the performance of the proposed metric with respect to previous groupwise registration metrics for the problem at hand, which is especially relevant to correct for elastic deformations.

Automatic metal parts inspection: Use of thermographic images and anomaly detection algorithms

A fully-automatic approach based on the use of induction thermography and detection algorithms is proposed to inspect industrial metallic parts containing different surface and sub-surface anomalies such as open cracks, open and closed notches with different sizes and depths. A practical experimental setup is developed, where lock-in and pulsed thermography (LT and PT, respectively) techniques are used to establish a dataset of thermal images for three different mockups. Data cubes are constructed by stacking up the temporal sequence of thermogram images. After the reduction of the data space dimension by means of denoising and dimensionality reduction methods; anomaly detection algorithms are applied on the reduced data cubes. The dimensions of the reduced data spaces are automatically calculated with arbitrary criterion. The results show that, when reduced data cubes are used, the anomaly detection algorithms originally developed for hyperspectral data, the well-known Reed and Xiaoli Yu detector (RX) and the regularized adaptive RX (RARX), give good detection performances for both surface and sub-surface defects in a non-supervised way.

Complex monitoring of the state of sea water basins by optical methods. Part 2. Sea surface pollution registration

Optical instruments and methods for a rapid and complex monitoring of coastal waters are analyzed. Usability of polarization panoramic shore-based optical systems for mapping and estimating the dynamic parameters of sea surface pollution with organic films is shown. A possibility of estimating the near-surface wind velocity by processing a temporal sequence of panoramic images is considered. In addition, this video system can be used for real-time direction of research vessels to hydrologic objects (vortices, internal waves, sea surface pollution, etc.) in coastal zones via mobile Internet.

Using DInSAR to Separate Surface and Subsurface Features

We report on an investigation into the use of differential interferometric synthetic aperture radar (SAR) (DInSAR) for the discrimination between surface and subsurface features in a soil, undertaken at the Ground-Based SAR Microwave Measurement Facility. A temporal sequence of C-band VV SAR images of a drying soil containing a buried target was collected. While the phase record of the signal identified with the soil return showed almost no variation, in stark contrast, the phase from the buried target showed a strongly linear change with time. A model is presented, which describes the observed phase changes in terms of retardation of the signal by the soil dielectric properties, which are dependent upon the moisture content. The model confirms a strongly linear relationship between phase and volumetric soil moisture. The linearity promises to greatly simplify any exploitation scheme, and such a DInSAR scheme would be applicable at large standoff distances from airborne and spaceborne platforms, in contrast to current subsurface techniques which rely on close-in measurement to spatially isolate returns vertically in backscatter.

Temporal sparse free-form deformations

FFD represent a widely used model for the non-rigid registration of medical images. The balance between robustness to noise and accuracy in modelling localised motion is typically controlled by the control point grid spacing and the amount of regularisation. More recently, TFFD have been proposed which extend the FFD approach in order to recover smooth motion from temporal image sequences. In this paper, we revisit the classic FFD approach and propose a sparse representation using the principles of compressed sensing. The sparse representation can model both global and local motion accurately and robustly. We view the registration as a deformation reconstruction problem. The deformation is reconstructed from a pair of images (or image sequences) with a sparsity constraint applied to the parametric space. Specifically, we introduce sparsity into the deformation via L1 regularisation, and apply a bending energy regularisation between neighbouring control points within each level to encourage a grouped sparse solution. We further extend the sparsity constraint to the temporal domain and propose a TSFFD which can capture fine local details such as motion discontinuities in both space and time without sacrificing robustness. We demonstrate the capabilities of the proposed framework to accurately estimate deformations in dynamic 2D and 3D image sequences. Compared to the classic FFD and TFFD approach, a significant increase in registration accuracy can be observed in natural images as well as in cardiac images.

Quantitative Measurement of the Surface Self-Diffusion on Au Nanoparticles by Aberration-Corrected Transmission Electron Microscopy

We present a method that allows for a quantitative measurement of the surface self-diffusion on nanostructures, such as nanoparticles, at the atomic scale using aberration-corrected high-resolution transmission electron microscopy (HRTEM). The diffusion coefficient can be estimated by measuring the fluctuation of the atom column occupation at the surface of Au nanoparticles, which is directly observable in temporal sequences of HRTEM images. Both a Au icosahedron and a truncated Au octahedron are investigated, and their diffusion coefficients are found to be in the same order of magnitude, D = 10(-17) to 10(-16) cm(2)/s. It is to be assumed that the measured surface diffusion is affected by the imaging electron beam. This assumption is supported by the observed instability of a (5 × 1) surface reconstruction on a {100} Au facet.

Statistical grey level and noise evaluation of Foveon X3 and CFA image sensors

This paper presents a practical, comprehensive and flexible statistical framework to analyze the radiometric response values and their uncertainty effects on image-based sensor systems. The statistical framework is based on experimental techniques and is tested on two single lens reflex (SLR) cameras powered by different image sensors, Foveon X3 and Color Filter Array (CFA). A colorchecker inside an indoor light booth is used as target. Design of Experiments (DoE) for Linear Models (LM) are derived to analyze and characterize the variability of the output grey level signal (digital number) and their uncertainty effects according to influential factors such as scene reflectance, wavelength range and time. The digital numbers are statistically modeled, and their noise and temporal trend are the uncertainty effects analyzed. Experiments are carried out under laboratory conditions to minimize the rest of uncertainty sources that might affect digital numbers. The flexibility of the statistical framework is confirmed to model and characterize the digital numbers, as well as the noise of a single image and the stability (trend and noise) of a temporal sequence of images.

Dayside and nightside segments of a polar arc: The particle characteristics

Using multiple satellites and a ground‐based radar network, the physical characteristics of a polar arc and its evolution are investigated under quiet solar wind conditions. The average electron energy is higher at the near‐midnight segment than at the sunward extension of the polar arc. Both polar arc segments are positioned primarily on closed magnetic field lines. The convection reversal boundary in the fitted convection pattern was located close to the transition region in the average electron energy seen by the satellites. The topside plasma density at an altitude of 680 km is clearly enhanced around the polar arc, although the enhancement boundary does not coincide precisely with that of the precipitation. The temporal sequence of auroral images suggests that the polar arcs emerged from a double oval structure. This suggestion is supported by the observed physical characteristics of the polar arc, which are quite similar to those of double oval structures that have been reported previously.

Non-equilibrium fluctuations on earth and in micro-gravity. The GRADFLEX experiment

We present the results of measurements of giant nonequilibrium fluctuations in a single component fluid (density fluctuations) and a mixture (concentration fluctuations) driven by applied temperature gradients, both on Earth and in space. Flight data were obtained during the September 2007 FOTON M3 mission. Spatial power spectra obtained using the shadowgraph method, during flight, confirm that the asymptotic behaviour extends to such low wave vector q, as to be limited by the sample thickness. Quantitative comparison with theory is provided, and is generally quite good. Temporal sequences of shadowgraph images for the mixture, both on Earth and during flight will be presented to emphasize the dramatic differences. Fluctuation lifetimes of thousands of seconds were observed during flight. The rugged but sensitive shadowgraph scattering method (phase fluctuations below 10 milliradians measured to within a few percent absolute accuracy) will be described briefly.