Temporal Sequence Of Images Research Articles

A low-cost experiment to visualise the Fourier series: video analysis of a real plucked coiled spring

In the present work, we develop a low-cost and simple experiment to visualise Fourier’s synthesis using a short, soft, and light plastic coiled spring oscillating in a horizontal plane, and a basic camera (120 fps). It is shown that the spring obeys a linear wave differential equation, as gravitational influence is neglected. A nonlinear criterion is evaluated to determine if magnitudes of the parameters in the initial conditions satisfy the linear wave equation. Our setup promotes some desirable characteristics that make Fourier’s synthesis experiments feasible, visual, and enlightening: (i) it requires few, common, and cheap resources, and the experiment can be carried out even in a high-school laboratory; (ii) since the spring’s tension is small (∼1 N, on average), the frequencies of normal modes are low (close to 2 Hz), and therefore, it is possible to record the oscillations just with the camera and extract a considerable number of position and time data in just one cycle; (iii) when the video is loaded in the Tracker free software, it can be reproduced in slow motion. Since the frequencies involved are low, an interesting and instructive temporal sequence of images of the spring displaying the typical trapezoidal shape appears clearly; (iv) the tools associated with the Tracker software tools can yield the relevant oscillation parameters, such as the damping constant, amplitudes, frequencies, and phases; and (v) it is possible to carry out superposition of a snapshot of the spring in Tracker at any time, and to draw the related Fourier synthesis graphs. The visual match between the shape of the spring and the theoretical graph is remarkable, and can be enhanced by adding the damping term.

Assessment and Enhancement of SAR Noncoherent Change Detection of Sea-Surface Oil Spills

Oil spills are one of the most dangerous catastrophes that threaten the oceans. Therefore, detecting and monitoring oil spills by means of remote sensing techniques that provide large-scale assessments is of critical importance to predict, prevent, and clean oil contamination. In this study, the detection of an oil spill using synthetic aperture radar (SAR) imagery is considered. Detection of the oil spill is performed using change detection algorithms between imagery acquired at different times. The specific algorithms used are the correlation coefficient change statistic and the intensity ratio change statistic algorithms. These algorithms and the probabilistic selection of threshold criteria are reviewed and discussed. A recently offered change detection method that depends on generating change maps of two images in a temporal sequence is used. An initial change map is obtained by cumulatively adding sequences in such a manner that common change areas are excluded and uncommon change areas are included. A final change map is obtained by comparing the first and the last images in the temporal sequence. This method requires at least three images to be employed and can be generalized to longer temporal image sequences. The purpose of this approach is to provide a double-check mechanism to the conventional approach and, thus, reduce the probability of false alarm while enhancing change detection. The algorithms are tested on 2010 Gulf of Mexico oil spill imagery. It is shown that the intensity ratio change statistic is a better tool for identification of the changes due to the oil spill compared to the correlation coefficient change statistic. It is also shown that the proposed method can reduce the probability of false alarm.

Ego-Lane Analysis System (ELAS): Dataset and algorithms

Decreasing costs of vision sensors and advances in embedded hardware boosted lane related research – detection, estimation, tracking, etc. – in the past two decades. The interest in this topic has increased even more with the demand for advanced driver assistance systems (ADAS) and self-driving cars. Although extensively studied independently, there is still need for studies that propose a combined solution for the multiple problems related to the ego-lane, such as lane departure warning (LDW), lane change detection, lane marking type (LMT) classification, road markings detection and classification, and detection of adjacent lanes (i.e., immediate left and right lanes) presence. In this paper, we propose a real-time Ego-Lane Analysis System (ELAS) capable of estimating ego-lane position, classifying LMTs and road markings, performing LDW and detecting lane change events. The proposed vision-based system works on a temporal sequence of images. Lane marking features are extracted in perspective and Inverse Perspective Mapping (IPM) images that are combined to increase robustness. The final estimated lane is modeled as a spline using a combination of methods (Hough lines with Kalman filter and spline with particle filter). Based on the estimated lane, all other events are detected. To validate ELAS and cover the lack of lane datasets in the literature, a new dataset with more than 20 different scenes (in more than 15,000 frames) and considering a variety of scenarios (urban road, highways, traffic, shadows, etc.) was created. The dataset was manually annotated and made publicly available to enable evaluation of several events that are of interest for the research community (i.e., lane estimation, change, and centering; road markings; intersections; LMTs; crosswalks and adjacent lanes). Moreover, the system was also validated quantitatively and qualitatively on other public datasets. ELAS achieved high detection rates in all real-world events and proved to be ready for real-time applications.

Heart Bounding Box Automatic Determination in Coronal Magnetic Resonance Temporal Sequences

The determination of the heart bounding box is a key task that can be used in several applications. Balanced steady-state free precession sequence without contrast produces artifacts originated from the blood flow. For this reason, very few research studies using this type of MR sequence have been conducted. However, this drawback can be used to help in the determination of the heart bounding box. The proposed algorithm is very simple, and averagely determines the heart bounding box in the entire MR temporal sequence of images. After that, it is necessary to adjust the vertical position in each image. The adjustment is a consequence of the diaphragmatic movement. The proposed algorithm is evaluated with several sets of MR temporal sequences without contrast.

Joint reconstruction of dynamic PET activity and kinetic parametric images using total variation constrained dictionary sparse coding

Dynamic positron emission tomography (PET) is capable of providing both spatial and temporal information of radio tracers in vivo. In this paper, we present a novel joint estimation framework to reconstruct temporal sequences of dynamic PET images and the coefficients characterizing the system impulse response function, from which the associated parametric images of the system macro parameters for tracer kinetics can be estimated. The proposed algorithm, which combines statistical data measurement and tracer kinetic models, integrates a dictionary sparse coding (DSC) into a total variational minimization based algorithm for simultaneous reconstruction of the activity distribution and parametric map from measured emission sinograms. DSC, based on the compartmental theory, provides biologically meaningful regularization, and total variation regularization is incorporated to provide edge-preserving guidance. We rely on techniques from minimization algorithms (the alternating direction method of multipliers) to first generate the estimated activity distributions with sub-optimal kinetic parameter estimates, and then recover the parametric maps given these activity estimates. These coupled iterative steps are repeated as necessary until convergence. Experiments with synthetic, Monte Carlo generated data, and real patient data have been conducted, and the results are very promising.

Bold-Independent Computational Entropy Assesses Functional Donut-Like Structures in Brain fMRI Images.

We introduce a novel method for the measurement of information level in fMRI (functional Magnetic Resonance Imaging) neural data sets, based on image subdivision in small polygons equipped with different entropic content. We show how this method, called maximal nucleus clustering (MNC), is a novel, fast and inexpensive image-analysis technique, independent from the standard blood-oxygen-level dependent signals. MNC facilitates the objective detection of hidden temporal patterns of entropy/information in zones of fMRI images generally not taken into account by the subjective standpoint of the observer. This approach befits the geometric character of fMRIs. The main purpose of this study is to provide a computable framework for fMRI that not only facilitates analyses, but also provides an easily decipherable visualization of structures. This framework commands attention because it is easily implemented using conventional software systems. In order to evaluate the potential applications of MNC, we looked for the presence of a fourth dimension's distinctive hallmarks in a temporal sequence of 2D images taken during spontaneous brain activity. Indeed, recent findings suggest that several brain activities, such as mind-wandering and memory retrieval, might take place in the functional space of a four dimensional hypersphere, which is a double donut-like structure undetectable in the usual three dimensions. We found that the Rényi entropy is higher in MNC areas than in the surrounding ones, and that these temporal patterns closely resemble the trajectories predicted by the possible presence of a hypersphere in the brain.

Low-Rank and Adaptive Sparse Signal (LASSI) Models for Highly Accelerated Dynamic Imaging.

Sparsity-based approaches have been popular in many applications in image processing and imaging. Compressed sensing exploits the sparsity of images in a transform domain or dictionary to improve image recovery fromundersampledmeasurements. In the context of inverse problems in dynamic imaging, recent research has demonstrated the promise of sparsity and low-rank techniques. For example, the patches of the underlying data are modeled as sparse in an adaptive dictionary domain, and the resulting image and dictionary estimation from undersampled measurements is called dictionary-blind compressed sensing, or the dynamic image sequence is modeled as a sum of low-rank and sparse (in some transform domain) components (L+S model) that are estimated from limited measurements. In this work, we investigate a data-adaptive extension of the L+S model, dubbed LASSI, where the temporal image sequence is decomposed into a low-rank component and a component whose spatiotemporal (3D) patches are sparse in some adaptive dictionary domain. We investigate various formulations and efficient methods for jointly estimating the underlying dynamic signal components and the spatiotemporal dictionary from limited measurements. We also obtain efficient sparsity penalized dictionary-blind compressed sensing methods as special cases of our LASSI approaches. Our numerical experiments demonstrate the promising performance of LASSI schemes for dynamicmagnetic resonance image reconstruction from limited k-t space data compared to recent methods such as k-t SLR and L+S, and compared to the proposed dictionary-blind compressed sensing method.

Three-dimensional thermography for non-destructive testing and evaluation

Three-dimensional (3D) vision based scanning for metrology and inspection applications is an area that has attracted increasing interest in the industry. This interest is driven by the recent advances in 3D technologies, which enable high precision measurements at an affordable cost. 3D vision allows for the modelling and inspection of the visible surface of objects. When it is necessary to detect subsurface defects, active infrared (IR) thermography is one of the most commonly used tools today for the non-destructive testing and evaluation (NDT&E) of materials. Fusion of these two modalities allows the simultaneous detection of surface and subsurface defects and the visualisation of these defects overlaid on the 3D model of the scanned and modelled parts or their 3D computer-aided design (CAD). In this work, we present a framework for automatically fusing 3D data (scanned or CAD) with the infrared thermal images for an NDT&E process in 3D space. The captured 3D images and their thermal infrared counterparts are aligned and fused using automatically detected features. This fusion is undergone on 3D space, thus allowing 3D visualisation of subsurface defects on a 3D model (or CAD). Additionally, the defects are extracted using image processing techniques and overlaid over their virtual position in 3D space. Their positioning at a certain distance from the part’s 3D surface is proportional to the computed depth using phase image analysis in the Fourier domain. This depth represents the real position of the detected subsurface defect and is extracted using thermograms (temporal sequence of thermal images). The results obtained are promising and show how this new technology can be used efficiently in a combined NDT&E-Metrology analysis of manufactured parts, in areas such as aerospace and automotive, among others.

Boosting Hankel matrices for face emotion recognition and pain detection

Studies in psychology have shown that the dynamics of emotional expressions play an important role in face emotion recognition in humans. Motivated by these studies, in this paper the dynamics of face expressions are modeled and used for automatic emotion recognition and pain detection.Given a temporal sequence of face images, several appearance-based descriptors are computed at each frame. Over the sequence, the descriptors corresponding to the same feature type and spatial scale define a time series. The Hankel matrix built upon each time series is used to represent the dynamics of face expressions with respect to the used feature-scale pair.The set of Hankel matrices obtained by varying the feature type and the scale is used within a boosting approach to train a strong classifier. During training, random subspace projection is adopted for feature and scale selection.Experiments on two challenging publicly available datasets show that the dynamics of appearance-based face expression representations can be used to discriminate among different emotion classes and, within a boosting approach, attain state-of-the-art average accuracy values in classification.

Real-time Functional Analysis of Inertial Microfluidic Devices via Spectral Domain Optical Coherence Tomography.

We report the application of spectral-domain optical coherence tomography (SD-OCT) technology that enables real-time functional analysis of sorting microparticles and cells in an inertial microfluidic device. We demonstrated high-speed, high-resolution acquisition of cross-sectional images at a frame rate of 350 Hz, with a lateral resolution of 3 μm and an axial resolution of 1 μm within the microfluidic channel filled with water. We analyzed the temporal sequence of cross-sectional SD-OCT images to determine the position and diameter of microspheres in a spiral microfluidic channel under various flow rates. We used microspheres with known diameters to validate the sub-micrometer precision of the particle size analysis based on a scattering model of spherical microparticles. An additional investigation of sorting live HT-29 cells in the spiral microfluidic channel indicated that the distribution of cells within in the microchannel has a close correspondence with the cells’ size distribution. The label-free real-time imaging and analysis of microscale particles in flow offers robustness for practical applications with live cells and allows us to better understand the mechanisms of particle separations in microfluidic sorting systems.

Simultaneous Temporal Superresolution and Denoising for Cardiac Fluorescence Microscopy

Due to low-light emission of fluorescent samples, live fluorescence microscopy imposes a tradeoff between spatio-temporal resolution and signal-to-noise ratio. This can result in images and videos containing motion blur or Poisson-type shot noise, depending on the settings used during acquisition. Here, we propose an algorithm to simultaneously denoise and temporally superresolve movies of repeating microscopic processes that is compatible with any conventional microscopy setup that can achieve imaging at a rate of at least twice that of the fundamental frequency of the process (above 4 frames per second for a 2-Hz process). Our method combines low temporal resolution frames from multiple cycles of a repeating process to reconstruct a denoised, higher temporal resolution image sequence which is the solution to a linear program that maximizes the consistency of the reconstruction with the measurements, under a regularization constraint. This paper describes, in particular, a parallelizable superresolution reconstruction algorithm and demonstrates its application to live cardiac fluorescence microscopy. Using our method, we experimentally show temporal resolution improvement by a factor of 1.6, resulting in a visible reduction of motion blur in both on-sample and off-sample frames.

Frame rate required for speckle tracking echocardiography: A quantitative clinical study with open-source, vendor-independent software

ObjectivesTo determine the optimal frame rate at which reliable heart walls velocities can be assessed by speckle tracking. BackgroundAssessing left ventricular function with speckle tracking is useful in patient diagnosis but requires a temporal resolution that can follow myocardial motion. In this study we investigated the effect of different frame rates on the accuracy of speckle tracking results, highlighting the temporal resolution where reliable results can be obtained. Material and methods27 patients were scanned at two different frame rates at their resting heart rate. From all acquired loops, lower temporal resolution image sequences were generated by dropping frames, decreasing the frame rate by up to 10-fold. ResultsTissue velocities were estimated by automated speckle tracking. Above 40 frames/s the peak velocity was reliably measured. When frame rate was lower, the inter-frame interval containing the instant of highest velocity also contained lower velocities, and therefore the average velocity in that interval was an underestimate of the clinically desired instantaneous maximum velocity. ConclusionsThe higher the frame rate, the more accurately maximum velocities are identified by speckle tracking, until the frame rate drops below 40 frames/s, beyond which there is little increase in peak velocity. We provide in an online supplement the vendor-independent software we used for automatic speckle-tracked velocity assessment to help others working in this field.

Rapid dynamic changes of the geometry of the anterior segment of the eye: A method of automatic spatial correction of a temporal sequence of OCT images

IntroductionOcular dynamics is a very complex phenomenon which has not been well studied and understood yet. The way in which the eye responds to pulsatile changes of the blood pressure or even the electric activity of the heart depends not only on the mechanical properties of each individual structure of the eye globe, but also on its internal conditions, such as the degree of accommodation or intraocular pressure (IOP). MethodThis paper presents a method for correcting these undesired movements that could increase the sensitivity of the technique and its reliability in the estimation of pulsatile dynamics. The presented algorithm uses fully automatic detection of the structures that form an angle and fully automatic stabilisation of each individual image being a part of a captured sequence. Results and conclusionsThe procedures described above were applied to the data of nine subjects taking part in the study. For all of them it turned out that the most representative area for fitting the images is the area of the irido-corneal angle apex. The presented algorithm significantly improved the spatial stability of the images in the temporal sequences of the tomographic images. The use of such correction makes it possible to distinguish the subtle pulsatile fluctuations of the ocular structures in the anterior segment, that can be associated with the activity of the retinal blood vessels and/or electric activity of the heart, from the undesired involuntary movements of the eye or the whole head.

Estimating iceberg paths using a wind-driven drift model

Icebergs are hazards to ship traffic and offshore operations. Their monitoring using satellite data is hence important for marine safety and for protection of the marine environment. The repeated search for smaller icebergs in a temporal sequence of satellite images, however, can be time-consuming if the expected drift path of the berg is not known. Computer simulations of iceberg drift can be applied to determine the approximate new iceberg position to narrow the search radius for icebergs. Best suited for this purpose are relatively simple, fast-running models. For three different regions of the Weddell Sea, Antarctica, we used an iceberg drift model driven by wind forecasts and analyzed predicted iceberg positions in comparison to positions retrieved from GPS buoys placed on the icebergs. As an error measure, we took the difference between the modeled and observed iceberg positions and drift angles. As the model was developed for open water regions, we added a sea-ice component. The application of the model was tested on two ENVISAT Wide Swath SAR scenes from the Weddell Sea, simulating and tracking iceberg positions for a period of five days. Although simulation results differ from direct observations of iceberg drift, they are still useful for narrowing the search area for icebergs in satellite images. We attribute the deviations between simulations and observations to the inherent large uncertainties in the input data for the model.

Cloud Computing for Earth Surface Deformation Analysis via Spaceborne Radar Imaging: A Case Study

We present a case study on the migration to a Cloud Computing environment of the advanced differential synthetic aperture radar interferometry (DInSAR) technique, referred to as Small BAseline Subset (SBAS), which is widely used for the investigation of Earth surface deformation phenomena. In particular, we focus on the SBAS parallel algorithmic solution, namely P-SBAS, that allows the production of mean deformation velocity maps and the corresponding displacement time-series from a temporal sequence of radar images by exploiting distributed computing architectures. The Cloud migration is carried out by encapsulating the overall P-SBAS application in virtual machines running on the Cloud; moreover, the Cloud resources provisioning and configuration phases are implemented in an automatic way. Such an approach allows us to preserve the P-SBAS parallelization strategy and to straightforwardly evaluate its performance within a Cloud environment by comparing it with those achieved on a HPC in-house cluster. The results we present were achieved by using the Amazon Elastic Compute Cloud (EC2) of the Amazon Web Services (AWS) to process SAR datasets collected by the ENVISAT satellite and show that, thanks to the Cloud resources availability and flexibility, large DInSAR data volumes can be processed through the P-SBAS algorithm in short time frames and at reduced costs. As a case study, the mean deformation velocity map of the southern California area has been generated by processing 172 ENVISAT images. By exploiting 32 EC2 instances this processing took less than 17 hours to complete, with a cost of USD 850. Considering the available PB-scale archives of SAR data and the upcoming huge SAR data flow relevant to the recently launched (April 2014) Sentinel-1A and the forthcoming Sentinel-1B satellites, the exploitation of Cloud Computing solutions is particularly relevant because of the possibility to provide Cloud-based multi-user services allowing worldwide scientists to quickly process SAR data and to manage and access the achieved DInSAR results.

Removing Noises Induced by Gamma Radiation in Cerenkov Luminescence Imaging Using a Temporal Median Filter.

Cerenkov luminescence imaging (CLI) can provide information of medical radionuclides used in nuclear imaging based on Cerenkov radiation, which makes it possible for optical means to image clinical radionuclide labeled probes. However, the exceptionally weak Cerenkov luminescence (CL) from Cerenkov radiation is susceptible to lots of impulse noises introduced by high energy gamma rays generating from the decays of radionuclides. In this work, a temporal median filter is proposed to remove this kind of impulse noises. Unlike traditional CLI collecting a single CL image with long exposure time and smoothing it using median filter, the proposed method captures a temporal sequence of CL images with shorter exposure time and employs a temporal median filter to smooth a temporal sequence of pixels. Results of in vivo experiments demonstrated that the proposed temporal median method can effectively remove random pulse noises induced by gamma radiation and achieve a robust CLI image.

5D respiratory motion model based image reconstruction algorithm for 4D cone-beam computed tomography

4D cone-beam computed tomography (4DCBCT) reconstructs a temporal sequence of CBCT images for the purpose of motion management or 4D treatment in radiotherapy. However the image reconstruction often involves the binning of projection data to each temporal phase, and therefore suffers from deteriorated image quality due to inaccurate or uneven binning in phase, e.g., under the non-periodic breathing. A 5D model has been developed as an accurate model of (periodic and non-periodic) respiratory motion. That is, given the measurements of breathing amplitude and its time derivative, the 5D model parametrizes the respiratory motion by three time-independent variables, i.e., one reference image and two vector fields. In this work we aim to develop a new 4DCBCT reconstruction method based on 5D model. Instead of reconstructing a temporal sequence of images after the projection binning, the new method reconstructs time-independent reference image and vector fields with no requirement of binning. The image reconstruction is formulated as a optimization problem with total-variation regularization on both reference image and vector fields, and the problem is solved by the proximal alternating minimization algorithm, during which the split Bregman method is used to reconstruct the reference image, and the Chambolle's duality-based algorithm is used to reconstruct the vector fields. The convergence analysis of the proposed algorithm is provided for this nonconvex problem. Validated by the simulation studies, the new method has significantly improved image reconstruction accuracy due to no binning and reduced number of unknowns via the use of the 5D model.

Fast frame scanning camera system for light-sheet microscopy.

In the interest of improving the temporal resolution for light-sheet microscopy, we designed a fast frame scanning camera system that incorporated a galvanometer scanning mirror into the imaging path of a home-built light-sheet microscope. This system transformed a temporal image sequence to a spatial one so that multiple images could be acquired during one exposure period. The improvement factor of the frame rate was dependent on the number of sub-images that could be tiled on the sensor without overlapping each other and was therefore a trade-off with the image size. As a demonstration, we achieved 960 frames/s (fps) on a CCD camera that was originally capable of recording images at only 30 fps (full frame). This allowed us to observe millisecond or sub-millisecond events with ordinary CCD cameras.

Lifetime estimation of moving subcellular objects in frequency-domain fluorescence lifetime imaging microscopy.

Fluorescence lifetime is usually defined as the average nanosecond-scale delay between excitation and emission of fluorescence. It has been established that lifetime measurements yield numerous indications on cellular processes such as interprotein and intraprotein mechanisms through fluorescent tagging and Förster resonance energy transfer. In this area, frequency-domain fluorescence lifetime imaging microscopy is particularly appropriate to probe a sample noninvasively and quantify these interactions in living cells. The aim is then to measure the fluorescence lifetime in the sample at each location in space from fluorescence variations observed in a temporal sequence of images obtained by phase modulation of the detection signal. This leads to a sensitivity of lifetime determination to other sources of fluorescence variations such as intracellular motion. In this paper, we propose a robust statistical method for lifetime estimation for both background and small moving structures with a focus on intracellular vesicle trafficking.

Remote sensing proxies of productivity and moisture predict forest stand type and recovery rate following experimental harvest

Site productivity, as affected by soil nutrients and available moisture is often characterized using an edatopic grid. A challenge for forest ecologists and managers working across large areas and in complex landscapes is the need to identify spatially different ecological environments that follow an edatopic classification. Recent advances in remote sensing offer some potential approaches for mapping ecological environments and landscape conditions. It is now feasible to compile long temporal image sequences using Landsat imagery for reconstruction of forest stands and derivation of long term indices of landscape productivity; and the increasing proliferation of airborne laser scanning (ALS) technology also allows for the acquisition of detailed information on topographic elevation and vegetation structure with sub-meter accuracy. In a large area of boreal mixedwood forest in northwestern Alberta, Canada, we examined the utility of using Landsat-derived vegetation greenness (indicating productivity), and ALS-derived cartographic depth-to-water (indicating moisture), to determine forest cover type and vegetation responses following variable retention harvesting. Our results demonstrate that both long-term image sequences from Landsat and ALS-derived topography and vegetation structure act as proxies for edatopic grid components, and are well-suited to differentiating forest cover types. Deciduous-dominated, deciduous-dominated with conifer understory, mixed, and conifer-dominated forests were generally distributed (in order) across a gradient of increasing moisture and decreasing greenness. Landscape greenness was the strongest predictor of vegetation regrowth after disturbance, followed by depth to water and other terrain factors, such as elevation and slope. New advances in, and complementary use of, different remotely sensed information provides a better understanding of both the landscape-scale distribution of forest cover types and patterns of vegetation regrowth following disturbance.