Motion Magnification Technique Research Articles

Volumetric Motion Magnification: Subtle Motion Extraction from 4D Data

The extension of Phase-based Video Motion Magnification into three-dimensions is presented in this work. The technique uses a 3D complex steerable pyramid to decompose volumetric frames of 4D dataset. The resulting decomposition can be processed to filter, amplify, or attenuate subtle motions occurring locally within the 4D data, before inversion of the transform is performed to recreate the volumetric images. Recent research has used Phase Based Video Motion Magnification to extract subtle motion from 2D slices of internal deformation data. This work demonstrates how Phase Based Video Motion Magnification can be extended to work on full volumetric images to extract full 3D volumetric motions. The Volumetric Motion Magnification technique is demonstrated qualitatively, to amplify and extract the indiscernible motion of a cylinder for visual identification. Then, the ability of the technique to enhance the motion noise floor of Digital Volume Correlation by more than a factor of 10 is demonstrated.

Implementation of video motion magnification technique for non-contact operational modal analysis of light poles

Damages on lights and utility poles mounted on the elevated highway or railway bridges were observed in the past several large earthquakes. The damages could have serious consequences to public safety, travelling vehicles or trains, and nearby properties. A previous study shows that the damages were caused by buckling and yielding of the pole due to excessive response amplification during large earthquake. Such amplification occurs when the bridge's natural frequency is close to the light pole's fundamental frequency. An investigation of the seismic performance of existing light pole mounted on elevated highway bridges is needed to avoid the response amplification. This includes the identification of the light pole's natural frequency and damping ratio. Vibration testing of the light pole using conventional contact sensors individually would require enormous effort and is time-consuming. Moreover, such vibration testing on a highway bridge deck would require traffic disruption to provide access. Video camera-based non-contact vision sensing is seen as a promising alternative to the conventional contact sensors for this purpose. The objective of this paper is to explore the use of non-contact vision sensing for operational modal analysis of light pole on highway viaduct. The phase-based video motion magnification method is implemented to obtain the light pole response in an ambient condition. Using this method, small and invisible displacement is magnified for a certain range of frequency of interest. Based on the magnified video frames, structural displacement is extracted using the image processing technique. The natural frequency and damping ratio of the light pole are estimated using the random decrement technique. The method is verified in a laboratory-scale experiment and implemented to practical field measurements of a light pole on a highway viaduct in Kanagawa, Japan. The results are compared with measurement by Laser Doppler Vibrometer. Both experiments suggest that the method could effectively obtain the natural frequency and damping ratio of the structures under the ambient condition where vibration amplitudes are very small and invisible with reasonable accuracy.

Evaluating tool point dynamics using smartphone-based visual vibrometry

Modern smartphones can record high megapixel videos at high frame rates. This paper leverages these capabilities and presents a new method to evaluate dynamics of a representative grooving tool using a smartphone’s video camera. Pixels within images are treated as vibration sensors measuring response to impact-based excitation. Motion is detected and tracked using image processing techniques to give pixel-displacement time series data. Spectra of these data give natural frequencies, and damping is estimated using the logarithmic decrement method. These parameters compare well with those extracted using experimental modal analysis procedures. Unscaled tool vibration shapes are visualized using motion magnification techniques.

Manifestation of Flexural Vibration Modes of Rails by the Phase-Based Magnification Method

With the recent development of high performance photography devices, vibration measurements using digital image processing can be utilized for the condition monitoring and the analysis of generation mechanisms. High-quality image recording provides very dense spatial information unlike physical sensors with a limited number of point-oriented measurements and output channels such as accelerometers. Moreover, image acquisition is a non-contact technique and ensures that the installed sensors do not affect the mass or stiffness. In this study, the vibration mode of rails is investigated using the phase-based video magnification method. An extensive amount of information that cannot be detected by human eyes, can be obtained with this imaging motion magnification technique by amplifying and visualising the minute vibrational movements. Flexural vibration modes in rails generate rolling noise, especially at high frequencies where the displacement amplitudes are small. The motion magnification method is applied using a high-speed camera to measure the rail mode shapes. The measured mode shapes are compared to the numerical results obtained from the waveguide finite element method. This method helps in understanding the vibration generation and the transfer mechanisms in the waveguide structures with a uniform cross-section in the lengthwise direction such as a railway track.

DeepMag

Many important physical phenomena involve subtle signals that are difficult to observe with the unaided eye, yet visualizing them can be very informative. Current motion magnification techniques can reveal these small temporal variations in video, but require precise prior knowledge about the target signal, and cannot deal with interference motions at a similar frequency. We present DeepMag, an end-to-end deep neural video-processing framework based on gradient ascent that enables automated magnification of subtle color and motion signals from a specific source, even in the presence of large motions of various velocities. The advantages of DeepMag are highlighted via the task of video-based physiological visualization. Through systematic quantitative and qualitative evaluation of the approach on videos with different levels of head motion, we compare the magnification of pulse and respiration to existing state-of-the-art methods. Our method produces magnified videos with substantially fewer artifacts and blurring whilst magnifying the physiological changes by a similar degree.

Effect of broad-band phase-based motion magnification on modal parameter estimation

In this paper, broad-band phase-based motion magnification (BPMM) is used to improve the modal parameter estimation from high-speed video of a structure undergoing low amplitude vibration. The authors use a novel application of the phase-based motion magnification technique to obtain improved results in the presence of image noise. A numerical simulation is performed to demonstrate and quantify the effect of broad-band motion magnification. A clear correlation is found between the magnification factor and the error in the motion obtained from 2-dimensional point tracking (2DPT). In a laboratory experiment, operational modal analysis is performed on a metallic nozzle with a 5.3 in exit diameter and a constant wall thickness of 0.03 inches. Fluorescent markers are painted on the nozzle lip. The nozzle is attached to a test stand and excited by pressurized air. The induced vibrations are captured by a single high-speed camera, which takes images of the nozzle lip. Using a 2DPT algorithm, the displacement history of the markers in the nozzle exit plane is extracted. As is typical for small and stiff structures, the resonant frequencies are quite high and the resulting vibration amplitudes are fairly low. This leads to low signal-to-noise ratio in the higher frequencies, which makes the following operational modal analysis much harder. By using BPMM as a preprocessor, the authors demonstrate that the outcome of modal analysis is greatly improved. Comparison with Finite Element analysis shows that the mode shapes agree much better with the experimental results when motion magnification is used. The modal parameters of the first six modes are found in the frequency range 0–1400 Hz using the new methodology, whereas without motion magnification only the first five modes are found.

Observations of a Quasi-periodic Pulsation in the Coronal Loop and Microwave Flux during a Solar Preflare Phase

Abstract We report a quasi-periodic pulsation (QPP) event simultaneously detected from the spatial displacements of the coronal loop at both EUV images and microwave emission during the preflare phase of a C1.1 flare on 2016 March 23. Using the motion magnification technique, a low-amplitude transverse oscillation with the growing period is discovered in a diffuse coronal loop in Atmospheric Imaging Assembly (AIA) image sequences at wavelength of 171 Å, and the initial oscillation period is estimated to be ∼397 s with a slow growth rate of 0.045. At the same time, a QPP with growing periods from roughly 300 s to nearly 500 s is discovered in the microwave flux in the same active region. Based on the imaging observations measured at EUV wavelengths by the AIA and at microwave 17 GHz by Nobeyama Radioheliograph, the diffuse coronal loop and the microwave radiation source are found to be connected through a hot loop seen in AIA images at wavelength of 94 Å. The growing period of the QPP should be related to the modulation of LRC-circuit oscillating process in a current-carrying plasma loop. The existence of electric currents may imply the non-potentialities in the source region during the preflare phase.

Imaging techniques for defect detection of fiber reinforced polymer‐bonded civil infrastructures

The present article provides a state-of-the-art review of imaging techniques used for defect detection of fiber reinforced polymer (FRP)-bonded civil engineering structures. Compared to the conventional techniques by equipping a structure with stress wave sensors, the contactless imaging techniques feature efficient instrumentation, convenient data acquisition, and rapid evaluation in the procedures of nondestructive testing (NDT). Recently, a substantial progress in utilizing electromagnetic waves for development of imaging NDT techniques (e.g., synthetic aperture radar, infrared thermography, laser shearography, and laser reflection technique) with the purpose to identify debonding or delamination of FRP-bonded structural systems has been made. As another electromagnetic wave-based imaging technique, X-ray computed tomography is promising for exploration of the damage evolution in this structural system, despite little application in structural health monitoring of real infrastructures. Apart from these imaging techniques, more recently, there have been computer-aided motion magnification techniques for defect identification. The motion magnification technique only requires a digital camera and a computer with vision algorithm, which can amplify the motion of defect region and render it visible in a video. This advanced imaging technique achieves high-resolution measurement, simultaneous full-field inspection, and straightforward defect identification in the scene. In this review, both Eulerian motion magnification and phase-based motion magnification techniques for structural visual inspection are presented and discussed. Furthermore, the present article recommends the combination of imaging NDT techniques with artificial intelligence approaches (e.g., deep learning algorithms) to realize the automated and efficient defect detection towards FRP-bonded civil infrastructures.

Seeing the Invisible

The latest techniques in video motion magnification and relevant small motion analysis are surveyed. The main motion magnification techniques are discussed in chronological fashion, highlighting the inherent limitations of predecessor techniques in comparison with subsequent variants. The focus is then shifted to the specific stages within the motion magnification framework to discuss advancements that have been proposed in the literature, namely for spatial decomposition and for emphasizing, representing, and distinguishing different motion signals. The survey concludes with a treatment of different problems in varying application contexts that have benefited from motion magnification and small motion analysis.

Magnetohydrodynamic Seismology of Quiet Solar Active Regions

Abstract The ubiquity of recently discovered low-amplitude decayless kink oscillations of plasma loops allows for the seismological probing of the corona on a regular basis. In particular, in contrast to traditionally applied seismology that is based on the large-amplitude decaying kink oscillations excited by flares and eruptions, decayless oscillations can potentially provide the diagnostics necessary for their forecasting. We analyzed decayless kink oscillations in several distinct loops belonging to active region NOAA 12107 on 2010 July 10 during its quiet time period, when it was observed on the west limb in extreme ultraviolet by the Atmospheric Imaging Assembly on board the Solar Dynamics Observatory. The oscillation periods were estimated with the use of the motion magnification technique. The lengths of the oscillating loops were determined within the assumption of its semicircular shape by measuring the position of their footpoints. The density contrast in the loops was estimated from the observed intensity contrast accounting for the unknown spatial scale of the background plasma. The combination of those measurements allows us to determine the distribution of kink and Alfvén speeds in the active region. Thus, we demonstrate the possibility of obtaining seismological information about coronal active regions during the quiet periods of time.

Time-Varying Motion Filtering for Vision-Based Nonstationary Vibration Measurement

In the field of vibration measurement, the vision-based approach can achieve high spatial resolution compared to traditional accelerometers and lasers and thus has attracted much attention during the past decades. A recently developed phase-based video motion magnification (PVMM) technique to estimate subtle motions from videos is only suitable for stationary vibration measurement, because it is limited to analyze the well-separated vibration modes. In this article, a time-varying motion filtering (TVMF) method is presented to endow the PVMM technique with the ability to decompose nonstationary vibrations and visualize the time-dependent mode shapes of nonstationary systems. Specifically, we first build a parameterized mathematical model for the video motions and then extract each vibration mode by optimizing a set of parameters. After modulating each mode with an amplification factor, the time-varying characteristics of the vibration mode shapes can be visualized to naked eyes. Compared to the traditional PVMM technique, the merit of the proposed TVMF is verified to be able to produce less noise and fewer artifacts in nonstationary vibration measurement. The performance of TVMF is demonstrated on both a simulated experiment and a real nonstationary moving mass system.

A New Approach to the Phase-Based Video Motion Magnification for Measuring Microdisplacements

Video motion magnification (VMM) techniques have recently gained attention as a low-cost accurate method for measuring microdisplacements. VMM techniques are known to have incorrect frequency responses when handling dynamic measurements consisting of multiple frequencies. New phase-based VMM method utilizing radial fractional steerable 2-D Hilbert transform is proposed and contrasted, in terms of the frequency response, with two other established VMM methods. Typical VMM workflow includes amplifying phase changes between subsequent video frames on many spatial frequencies separately using image pyramids, which are then collapsed forming new, motion-magnified video. In this paper, Hilbert transform is used to extract local phase changes. The modified Kullback–Leibler divergence was calculated between the spectral estimates of the time series of vibrating plate measured using a high-speed video camera and a reference time series synchronously acquired with high-precision air-coupled ultrasonic transducer. Time series consisted of micrometer range multicomponent vibrations (one, two, and three sine frequencies) recorded with different motion magnification factors. The results show that in the majority of cases, the proposed method, out of three considered, resulted in the most uniform frequency response and generated the lowest amount of unwanted frequencies. To conclude, a novel approach to VMM is proposed, which substantially improves frequency response and makes this technique more suitable for the measurements of phenomena consisting of multiple frequencies.

Defect detection in FRP-bonded structural system via phase-based motion magnification technique

Defect detection in FRP-bonded structural system via phase-based motion magnification technique

Frequency-based damage detection in cantilever beam using vision-based monitoring system with motion magnification technique

This article proposes a method for damage detection using vision-based monitoring with motion magnification technique. The methods based on the vibration characteristics of structures such as natural frequency, mode shapes, and modal damping have been applied to structural damage detection. However, the conventional methods have limitations for practical applications. Vision-based monitoring system can be employed as a new structural monitoring system because of its simplicity, potentially low cost, and unique capability of collecting high-resolution data. A methodology called video motion magnification has been developed to amplify non-visible small motions in a video to reveal the dynamic response. The video motion magnification method can be applied to measure small displacements to calculate the natural frequencies and the operational deflection shapes of the structures. Unlike conventional optimization methods, a genetic algorithm explores the entire solution space and can obtain the global optimum. In this article, identification of the location and magnitude of damage in a cantilever beam is formulated as an optimization problem using a real-value genetic algorithm by minimizing the objective function, which directly compares the first three natural frequencies changes from the phase-based motion magnification measurement and from the analytical model of a damaged cantilever beam.

Euler Renk ve Hareket Büyütme Yöntemlerinin Performans Analizi

Bu calismasinda, insan gozu tarafindan algilanmasi zor olan bir goruntudeki, gizli renk ve hareket degisimlerini aciga cikarmada kullanilan goruntu buyutme yontemlerinin performans analizi yapilmistir. Bu analiz isleminde, goruntu buyutme yontemi olarak Euler renk buyutme ve Euler hareket buyutme yontemleri kullanilmistir. Her iki goruntu buyutme yonteminde, goruntu ayristirma yontemi olarak hem Laplace piramidi hem de Gauss piramidi kullanilmis ve bu piramitlerin performanslari karsilastirilmistir. Tum piramit ayristirmalarindan elde edilen cikis goruntuleri hesapsal olarak analiz edilir ve PSNR (En Buyuk Isaret-Gurultu Orani) ve SSIM (Yapisal Benzerlik Indeksi Olcumu) degerleri uzerinden birbirleriyle karsilastirilir. Goruntu isleme zamani, goruntu ayristirma yontemlerini karsilastirmada kullanilan bir parametredir. Goruntu ayristirma yontemleriyle yapilan deneylerin sonuclari goruntu isleme zamani acisindan incelendiginde, Gauss piramit ayristirmasiyla goruntu buyutmenin daha iyi bir performansa sahip oldugu gorulmustur. Bu yontemler goruntu kalitesi olarak PSNR degerleri acisindan incelendiginde, Gauss piramidinin renk buyutme isleminde daha iyi sonuclar verdigi Laplace piramidinin ise hareket buyutme isleminde daha iyi sonuclar verdigi gorulmustur.

Feasibility of extracting operating shapes using phase-based motion magnification technique and stereo-photogrammetry

Optically-based vibration measurements can be used to determine the full-field dynamic response of structures without the need of mounted transducers (e.g. accelerometers, strain gages, and LVDTs) that can add cost, have wiring, power signal transmission issues, and may affect the dynamic characteristics by adding mass or stiffness. The three-dimensional digital image correlation (3D DIC) and three-dimensional point tracking (3DPT) methods based on a stereo-vision system can provide the full-field dynamic displacements of a structure with sub-pixel accuracy. However, stereo-photogrammetry systems are limited by camera resolution and intrinsic noise of the acquired images. Moreover, exciting a structure with only a small amount of energy may result in a subtle mechanical response that is not perceptible by observing the raw data of the conventional optical measurement systems (i.e. 3D DIC and 3DPT). Thus, in order to use the optical-sensing techniques to identify dynamic characteristics of a structure at high frequencies, the signal-to-noise ratio (SNR) in the sequence of images captured with a stereo-vision system needs to be improved. Within this paper, phase-based video magnification in conjunction with the stereo-photogrammetry techniques are used to measure the higher-frequency operating shapes of a cantilever beam and a 2.3-m long Skystream 4.7 wind turbine blade. The phase-based motion estimation method uses complex steerable pyramids to decompose the original sequence of images to amplitude and phase at different spatial resolutions (subsampled images). Motion information is conserved within the phase values which can be filtered in the time domain and amplified to detect the subtle motions. The magnified sequence of images using the motion magnification technique are post-processed using 3D DIC or 3DPT to quantify infinitesimal deformations that are not recognizable using only the traditional stereo-photogrammetry methods. The results obtained within this paper reveal the great potential of motion magnification in conjunction with stereo-photogrammetry techniques in detecting small motions of structures and extracting 3D operating shapes from the optically measured data with low SNR. Compared to traditional stereo-photogrammetry, additional modes of the test structures are obtainable by means of combining the phase based motion magnification and conventional image-sensing techniques (i.e. 3D DIC and 3DPT). The proposed methodology could be used to enhance the established procedure to extract more information about the dynamic characteristics of structures (only once effectively calibrated), compared to existing image-sensing techniques and contact measurement sensors.

Output-only computer vision based damage detection using phase-based optical flow and unscented Kalman filters

A damage detection methodology is proposed by integrating a nonlinear recursive filter and a non-contact computer vision based algorithm to measure structural dynamic responses. A phase-based optical flow algorithm inspired by the motion magnification technique is used to measure structural displacements, and the unscented Kalman filter is used to predict structural properties such as stiffness and damping coefficients. This non-contact displacement measurement methodology does not require an intensive instrumentation process, does not add any additional mass to the structure which may skew measurements, and can measure more signals compared to traditional methods. This measurement methodology still needs improvement as a tool due to its higher noise level relative to traditional accelerometer and laser vibrometer measurements. In order to detect structural damage using measured displacements from video, an unscented Kalman filter is used to remove noise from the displacement measurement and simultaneously detect damage by identifying the current stiffness and damping coefficient values, given a known mass, which are used to detect damage. To validate the proposed damage detection method state-space equations are derived without external excitation input and experimental tests are carried out. The experimental results show reasonable and accurate predictions of the stiffness and damping properties compared to dynamic analysis calculations.

Effective recognition of facial micro-expressions with video motion magnification

Facial expression recognition has been intensively studied for decades, notably by the psychology community and more recently the pattern recognition community. What is more challenging, and the subject of more recent research, is the problem of recognizing subtle emotions exhibited by so-called micro-expressions. Recognizing a micro-expression is substantially more challenging than conventional expression recognition because these micro-expressions are only temporally exhibited in a fraction of a second and involve minute spatial changes. Until now, work in this field is at a nascent stage, with only a few existing micro-expression databases and methods. In this article, we propose a new micro-expression recognition approach based on the Eulerian motion magnification technique, which could reveal the hidden information and accentuate the subtle changes in micro-expression motion. Validation of our proposal was done on the recently proposed CASME II dataset in comparison with baseline and state-of-the-art methods. We achieve a good recognition accuracy of up to 75.30 % by using leave-one-out cross validation evaluation protocol. Extensive experiments on various factors at play further demonstrate the effectiveness of our proposed approach.

Remote respiratory monitoring system based on developing motion magnification technique

Abstract The aim of this study is to detect and measure the rate and timing parameters of the respiratory cycle at a distance from different sleeping positions of a baby based on video imagery. This study relied on amplifying motion resulting from movement of the chest caused by inhalation and exhalation. A motion magnification technique based on a wavelet decomposition and an elliptic filter was used to magnify breathing movement that is difficult to see with the naked eye. A novel measuring method based on motion detection was used to measure respiratory rate and its time parameters by detecting the fastest moving areas in the magnified video frame sequences. The video frames were converted into a corresponding logical matrix. The experimental results on several videos for the baby at different sleeping positions show that the remote respiratory monitoring system has an accuracy of 99%. The proposed system has very low computational complexity, is feasible and safe making it suitable for the design of next generation non-contact vital signs monitoring systems.