Phase-based Motion Magnification Research Articles

3D vibration modes estimation of complex-shaped structure by Phase-based motion magnification and DIC method

Knowing the 3D mode shapes of a structure is important in solving unwanted vibration and noise problems in mechanical systems. Accordingly, various analysis methods for estimating the vibration modes of complex-shaped structures have been proposed. However, it is difficult to attach accelerometers to a complex-shaped structure, and to cover a large area, many accelerometers must be attached. So experimental verification of the mode shapes of an actual structure is insufficient. Therefore, the purpose of this study is to experimentally estimate the 3D vibration modes of a complex structure using two cameras by Phase-based image magnification and DIC method. In addition, by using non-contact sensor such as a camera, it is possible to estimate the modes without affecting the mass or stiffness of the structure.

Holistically Nested Edge Detection and particle filtering for subtle vibration extraction

Extracting and analyzing subtle motion via optical sensing techniques is becoming a more important research area within the vibration community. Computer vision approaches and machine learning architecture have been adopted to contest the complexity concerning hand instrumentation of sensors. Pre-treatment using a stochastic pattern for optical analysis or traditional wired sensing is not always permitted for large structures. In this work, Holistically Nested Edge Detection (HED) is utilized to highlight high-spatial-frequency content features in images for structural dynamic analysis. Following the accentuation of edge features, phase-based motion magnification (PMM) is used to amplify motion that is not visible to the naked eye. This permits the band-passing of higher order resonant frequencies for structural dynamic evaluation. Implementation of two-dimensional particle filtering (PF) is utilized to track the accentuated edge features in the presence of ghosting artifacts that appear post magnification. Within this work, compared to traditional edge-thresholding, the combination of HED and PF (HED-PF) is more proficient to handle noise and ultimately improves magnified image quality and modal parameter extraction. Uncertainties related to using PF for vibration measurement are analytically presented to provide guidance on the behavior of the algorithm. To demonstrate the effectiveness of HED-PF, experimental modal testing takes place on both a tier-structure and cantilever beam to demonstrate the approaches’ capability in handling varying sorts of excitation. Finally, introduction of background occlusions and non-ideal lighting scenarios are used to test the limitations of the approach. Ultimately, this will highlight HED-PF’s robust tracking capabilities compared to the current state of the art.

Video Motion Magnification and Subpixel Edge Detection-Based Full-Field Dynamic Displacement Measurement

Noncontact measurement techniques in structural dynamics field have progressed significantly in the past few decades. Vision-based measurement techniques are unique in that they have the ability to achieve full-field measurement and possess the typical advantages associated with noncontact measurement techniques. Recently, vision-based techniques have also been applied to streaming of videos for structural dynamic displacement measurement. The most recent trends in vision-based measurements include target tracing, digital image correlation, and target-less approaches. There are, however, some shortcomings of the vision-based techniques such as susceptibilities to image noise, prevailing light conditions, and limit in measurement resolution. To reduce these shortcomings, a method known as video motion magnification (MM) can be used to amplify small structural motions. Using the phase-based motion magnification (PBMM) and subpixel edge detection methods, the full-field dynamic displacements of the structure can be obtained. The deep convolutional long short-term memory (ConvLSTM) network is applied to aid in the selection of the frequency band for magnification in the PBMM algorithm. To achieve higher measurement accuracy, the displacement results with and without MM are combined with the finite impulse response (FIR) filter which can reduce the error caused by the PBMM procedure. In the tests, plastic optical fiber (POF) displacement sensors are introduced and used as reference measurements to compare the dynamic displacement results from the proposed vision-based method. Compared with the measured displacements with POF sensors, the proposed method offers high level of accuracy for full-field displacement measurement.

Cable vibration measurement based on broad-band phase-based motion magnification and line tracking algorithm

Cables are important components of cable-supported bridges and the measurement of cable vibration is very useful for the condition assessment of bridges. The recently developed computer vision (CV) based vibration measurement method has many merits over the traditional contact method. However, most existing CV-based methods require installing the artificial targets on the structure and are only suitable for situations with large vibration amplitude. Therefore, this study proposes a method based on the broad-band phase-based video motion magnification (BPVMM) and the line tracking algorithms for the vibration measurement of cables with small-amplitude vibration. The BPVMM algorithm is used to magnify the small vibrations and the line tracking algorithm is used to extract the vibration displacement from the magnified video for the final identification of cable frequencies. The proposed method is then verified by the laboratory cable test and the field test of a cable-stayed bridge. The results indicate that using the BPVMM algorithm to magnify the small vibration can significantly improve the identification accuracy of displacements and frequencies. The proposed method performs motion magnification over a wide frequency band including all the frequencies of interest, without prior knowledge of the structure frequencies, which greatly improves the computational efficiency and facilitate the measurement of cable vibration under environmental excitation in practical applications.

Motion magnification-based nonlinear ultrasonic signal enhancement and its application to remaining fatigue life estimation of a steel padeye

In this study, a nonlinear ultrasonic signal enhancement method based on phase-based motion magnification was proposed. First, a two-dimensional (2D) video was constructed from a one-dimensional time-domain ultrasonic signal with an automatically selected pixel scale. The constructed 2D video was spatially decomposed into multiple sub-bands with different spatial scales. The subtle motions of the nonlinear responses of interest were enhanced by phase denoising and magnification in each sub-band. Then, the magnified 2D video was reconstructed by collapsing all the amplified sub-bands. Finally, a magnified ultrasonic signal was extracted from the magnified 2D video for further nonlinear ultrasonic analysis. The proposed method was validated with a group of synthetic signals at different noise levels, and applied in the estimation of the remaining fatigue life of a steel padeye with improved accuracy.

Vibration signal separation of rotating cylindrical structure through target-less photogrammetric approach

Vibration analysis is one of the most widely used methodologies for structural health monitoring. For vibration monitoring of structures, contact-type sensors, such as gap sensors, are commonly utilized. However, these devices can add mass loading to a lightweight structure, resulting in degradation in performance of the structure. As an alternative, photogrammetric approaches, such as digital image correlation and point-tracking techniques have been proposed for vibration measurement. Despite the many advantages of image-based techniques applied by non-contact-type sensors, these are ineffective in detecting vibrations at the outer edge of a rotating cylinder-shaped structure because tracking of the speckles or targets on the backside of a structure invisible to a camera is impossible. Furthermore, vibrations occurring in structures are usually attributed to multiple factors; to identify these factors individually, the vibration signals should be detected separately. In this study, we introduce a noncontact-based vibration signal separation technique that can be used to recognize multiple factors that cause a structure to vibrate at different frequency rates. For this purpose, the vibration occurring in a structure is first detected using an edge-based vibration measurement technique. A phase-based motion magnification technique is then used to generate multiple magnified videos at different dominant frequencies. Subsequently, vibrations were detected in multiple magnified videos to separate vibration signals. To demonstrate the effectiveness of the proposed method, quantitative and qualitative assessments were conducted on both simulation and real video datasets containing vibrations with multiple frequency rates. The results generated by this method can be utilized for health monitoring of various types of structures on which targets, necessary for the general photogrammetry approach for monitoring purposes, are not attached. Furthermore, the magnified videos generated in this study can be used for the visualization and mode identification of existing vibration signals at different frequencies in a structure.

Vision-Based Structural Modal Identification Using Hybrid Motion Magnification

As a promising alternative to conventional contact sensors, vision-based technologies for a structural dynamic response measurement and health monitoring have attracted much attention from the research community. Among these technologies, Eulerian video magnification has a unique capability of analyzing modal responses and visualizing modal shapes. To reduce the noise interference and improve the quality and stability of the modal shape visualization, this study proposes a hybrid motion magnification framework that combines linear and phase-based motion processing. Based on the assumption that temporal variations can represent spatial motions, the linear motion processing extracts and manipulates the temporal intensity variations related to modal responses through matrix decomposition and underdetermined blind source separation (BSS) techniques. Meanwhile, the theory of Fourier transform profilometry (FTP) is utilized to reduce spatial high-frequency noise. As all spatial motions in a video are linearly controllable, the subsequent phase-based motion processing highlights the motions and visualizes the modal shapes with a higher quality. The proposed method is validated by two laboratory experiments and a field test on a large-scale truss bridge. The quantitative evaluation results with high-speed cameras demonstrate that the hybrid method performs better than the single-step phase-based motion magnification method in visualizing sound-induced subtle motions. In the field test, the vibration characteristics of the truss bridge when a train is driving across the bridge are studied with a commercial camera over 400 m away from the bridge. Moreover, four full-field modal shapes of the bridge are successfully observed.

Edge-based 3D vibration measurement of rotating cylinder-shaped structure through epipolar line-based corresponding point extraction between two camera images

Three-dimensional (3D) vibration measurements are important for monitoring rotating structures in order to reproduce the directional displacements of vibration signals such that they are similar to the actual movement. Photogrammetric techniques, which can be used to determine the relationship between 3D objects and their two-dimensional images, allow 3D vibrations in a structure to be measured without direct contact. Therefore, we herein present a multipoint 3D vibration measurement approach for rotating cylinder-shaped structures, which performs the imaging of a structure from two directions using two cameras. First, an epipolar line-based corresponding point extraction technique is applied to extract the corresponding regions of interest (ROIs) in videos acquired by the two cameras placed at 90° to the structure. Subsequently, an edge-based vibration measurement approach is used to detect vibrations in the corresponding ROIs. Noise reduction is then applied to reduce the noise, induced by the cameras or edge-based vibration measurement technique, by extracting and employing the frequencies at which the vibrations were occurring to the phase-based motion magnification technique. Subsequently, vibrations in the corresponding ROIs of the magnified videos are detected. For 3D plotting, the two vibration signals are combined. Simulation and real datasets are prepared for the qualitative and quantitative assessments of the proposed method. The simulation data results are compared with the results obtained before noise reduction, those obtained after noise reduction via bandpass filtering, and the reference data. A 3D vibration analysis is performed on the real data in multiple ROIs. The results yielded by the proposed method can be used to identify the weak points in a rotating cylinder-shaped structure that cause the structure to function abnormally.

Hybrid-driven structural modal shape visualization using subtle variations in high-speed video.

The phase-based motion magnification technique can exaggerate specific structural vibrations and obtain potential applications in visualizing and understanding modal shapes. However, the quality of motion magnification is affected by noise and clipping artifacts, especially in large amplifications. We propose a hybrid-driven motion magnification framework that combines Eulerian and Lagrangian motion processing. Since the structural global spatial vibration corresponding to different modal shapes usually accumulates energy differences in the timeline, from a Eulerian perspective, temporal intensity variations are denoised and separated according to the energy distribution to control spatial motions. Meanwhile, from a Lagrangian perspective, the motion magnification is realized by compensating spatial motion according to the magnified inter-frame motion vector field. By utilizing both Eulerian and Lagrangian motion processing, the proposed framework supports a larger amplification factor and achieves better performance in perceiving subtle vibrations in controlled modal tests.

A robust structural vibration recognition system based on computer vision

Vibration-based structural health monitoring (SHM) systems are useful tools for assessing structural safety performance quantitatively. When employing traditional contact sensors, achieving high-resolution spatial measurements for large-scale structures is challenging, and fixed contact sensors may also lose dependability when the lifetime of the host structure is surpassed. Researchers have paid close attention to computer vision because it is noncontact, saves time and effort, is inexpensive, and has high efficiency in giving visual perception. In advanced noncontact measurements, digital cameras can capture the vibration information of structures remotely and swiftly. Thus, this work studies a system for recognizing structural vibration. The system ensures acquiring high-quality structural vibration signals by the following: 1) Establishing a novel image preprocessing, which includes visual partitioning measurement and image enhancement techniques; 2) initial recognition of structural vibration using phase-based optical flow estimation (POFE), which introduces 2-D Gabor wavelets to extract the independent phase information of the image to track the natural texture targets on the surface of the structure; 3) extracting the practical vibration information of the structure using mode decomposition to remove the complex environment of the camera vibration and other noises; 4) employing phase-based motion magnification (PMM) techniques to magnify small vibration signals, and then recognizing the complete information on the vibration time range of the structure. The research results of the laboratory experiments and field testing conducted under three different cases reveal that the system can recognize structural vibration in complicated environments.

Towards phase-based defect detection: A feasibility study in vibrating panels

This work discusses the possibility of using phase-based motion magnification as a non-destructive inspection tool for defect detection and identification in vibrating panels. Phase-based motion magnification has recently emerged as a potentially disruptive technology in the field of optical methods for vibration engineering. In fact, the method allows to post-process high-speed video recordings in order to magnify small motions happening in a prescribed bandwidth. In particular, our strategy relies on measuring the full-field low-frequency eigen-shapes, and extracting their aberration as resulting from small defects. Effects of defects are usually very localized in space but may appear even at lower frequencies, making the approach particularly suitable for any kind of high-resolution full-field optical technique. Within this work, a novel phase-based processing pipeline for defect detection is described, and a set of preliminary tests is discussed to assess the feasibility and advantages of the methodology. All validations were carried out by means of numerical simulations relying upon a photo-realistic dynamic finite element model of a rectangular plate.

Exploiting phase-based motion magnification for the measurement of subtle 3D deformation maps with FP + 2D-DIC

Phase-Based Motion Magnification (PBMM) is an effective methodology to visualise imperceptible phenomena based on magnifying periodic subtle movements on image sequences. Moreover, the integration of the optical techniques Fringe Projection and 2D Digital Image Correlation (FP + 2D-DIC) make it possible to obtain 3D displacement maps employing a single camera.In this study, the integration of PBMM with FP + 2D-DIC is exploited for the determination of subtle 3D displacements. An initial solid-rigid test validates of the combination of those techniques. A later test, employing a cantilever beam, demonstrates its correct performing in the determining of Operational Deflection Shapes. In both cases, the results are compared with those obtained with 3D-DIC and Scanning Laser Doppler Vibrometer (SLDV) to evaluate the relation between the magnification factor and the result obtained. Finally, the potential of the proposed integration for the determination of complex mode shapes is demonstrated by obtaining the ODSs of an industrial component.

Phase-Based Motion Magnification for Structural Vibration Monitoring at a Video Streaming Rate

Here we propose a novel approach to image magnification that enables frame-by-frame motion magnification application at a video streaming rate of around 30 fps. This novel approach can be used instead of batch processing an image file in conventional phase-based magnification (PBM). This new PBM method can instantly show a magnified video streaming on display at 30 fps, which is helpful for vibration measurement and for monitoring tasks where magnified images need to be viewed simultaneously. To accomplish video streaming-rate magnification, the proposed PBM uses time-domain convolution in temporal bandpass filtering for frame-by-frame operation whereas conventional PBM employs a frequency-domain filter that is applied to the entire image file at once. An experiment was conducted to monitor the vibration of a cantilever using a webcam streaming at the same frame rate, and data were collected simultaneously using a laser Doppler vibrometer for comparison. The experiment confirmed that the proposed PBM approach is more effective than the conventional magnification method and it also analyzed the system performance for vibration measurement. Additionally, the proper orthogonal mode could be found through the singular value decomposition from vibration displacement data of the cantilever that was collected instantly from the magnified image frames. Furthermore, the dominant mode could be effectively extracted from excitation at the resonance frequency. Because of magnification factors, the vibration displacements from the proposed method were estimated using linear regression and the accuracy of the estimated displacements was within the permissible error bound. Since the proposed PBM is a frame-by-frame operation, instantaneous adjustment of magnification parameters is available, even while the magnification is being processed. In addition, the proposed PBM is independently adjustable to the number of image frames for temporal FIR filter order.

Quantification of phase-based magnified motion using image enhancement and optical flow techniques

Phase-based motion magnification (PMM) has been widely implemented in the field of vibration and structural health monitoring for its non-invasive nature to reveal hidden system dynamics. The approach has shown success in magnifying subtle structural oscillatory motions for system identification and observation of operating shapes. Although this method has been implemented and is becoming increasingly popular, the amount of physical motion associated with the degree of magnification has yet to be quantified. Within this work, a synthetic simulation containing an oscillating geometry is presented to quantify its magnified pixel displacement. Computer vision techniques including centroid detection and edge-feature tracking via optical flow are adopted to quantify the relation between amplification and true motion. The quantification techniques are also tested and verified on an experimental structure with the use of a high-speed optical sensing system. Motion artifacts distort the integrity of the magnified motion, which can pose problems for accurate quantification. Image enhancement techniques such as the two-dimensional Wiener filter and Total Variation Denoising (TVD) are used to smooth the high-frequency content that is observed following magnification. Associative error concerning a discrete shift of the Gabor wavelet is analytically derived to show the justification of spatial aliasing. An adjusted bound on magnification is presented to display the limitations of the technique, while providing insight into associated error. The results of this work will help to enhance PMM from a qualitative evaluation tool to a quantitative measurement tool of magnified displacements.

An Experimental Validation of Phase-Based Motion Magnification for Structures with Developing Cracks and Time-Varying Configurations

In this study, Computer Vision and Phase-Based Motion Magnification (PBMM) are validated for continuous Structural Health Monitoring (SHM) purposes. The aim is to identify the exact instant of occurrence for damage or abrupt structural changes from video-extracted, very low amplitude (barely visible) vibrations. The study presents three experimental datasets: a box beam with multiple saw cuts of different lengths and angles, a beam with a full rectangular cross section and a mass added at the tip, and the spar of a prototype High-Aspect-Ratio wing. Both mode-shape- and frequency-based approaches are considered, showing the potential to identify the severity and position of the damage as well A high-definition, high-speed camera and a low-cost commercial alternative have been successfully utilised for these video acquisitions. Finally, the technique is also preliminarily tested for outdoor applications with smartphone cameras.

3D-DIC analysis for BIM-oriented SHM of a lab-scale aluminium frame structure

In recent years, three-dimensional Digital Image Correlation (3D-DIC) has proven to be a reliable technique for Structural Health Monitoring (SHM) by allowing full line-of-sight, high-resolution, and contactless measurements of structures. The large amount of raw output resulting from 3D-DIC requires appropriate methods to visualise and analyse the data systematically. In this context, Building Information Modeling (BIM) emerges as a robust repository and data management tool to store, query, and assess long sequences of SHM data. Different sensors have been used to collect SHM data and digital sensors have been created in the BIM environment to mimic the behaviour of the physical ones on site. This research focuses on the BIM implementation of virtual sensors from information retrieved using 3D-DIC to store frequency and time domain data and monitor structural changes in a targeted system over time. The application of the Phase-based Motion Magnification (PMM) technique is proposed as a preprocessing tool for 3D-DIC analysis videos to obtain displacement information in specific frequency bands and make those values structurally effective. As a case study, a three-story aluminium frame structure with a damaged and undamaged configuration is used to validate the developed methodology.

Image Motion Extraction of Structures Using Computer Vision Techniques: A Comparative Study.

Vibrational measurements play an important role for structural health monitoring, e.g., modal extraction and damage diagnosis. Moreover, conditions of civil structures can be mostly assessed by displacement responses. However, installing displacement transducers between the ground and floors in real-world buildings is unrealistic due to lack of reference points and structural scales and complexity. Alternatively, structural displacements can be acquired using computer vision-based motion extraction techniques. These extracted motions not only provide vibrational responses but are also useful for identifying the modal properties. In this study, three methods, including the optical flow with the Lucas–Kanade method, the digital image correlation (DIC) with bilinear interpolation, and the in-plane phase-based motion magnification using the Riesz pyramid, are introduced and experimentally verified using a four-story steel-frame building with a commercially available camera. First, the three displacement acquiring methods are introduced in detail. Next, the displacements are experimentally obtained from these methods and compared to those sensed from linear variable displacement transducers. Moreover, these displacement responses are converted into modal properties by system identification. As seen in the experimental results, the DIC method has the lowest average root mean squared error (RMSE) of 1.2371 mm among these three methods. Although the phase-based motion magnification method has a larger RMSE of 1.4132 mm due to variations in edge detection, this method is capable of providing full-field mode shapes over the building.

Noncontact operational modal analysis of light poles by vision-based motion-magnification method

Light and sign poles on elevated highway bridge are secondary structural elements that support functionality of a highway system. Over the time, they may suffer from fatigue damage due to traffic, wind, and earthquake-induced vibration in a direct or indirect manner. As a secondary structural component, vibrations of light and sign poles are influenced by the bridge vibration, the primary structure. Structural condition assessment of the light and sign poles are normally conducted by visual inspection and/or vibration measurement using contact or noncontact method. In this paper, a vision-based vibration measurement of the light poles on elevated highway bridges using motion magnification and dynamic mode decomposition methods is proposed. Using the method, vibrations of the light poles are captured by video camera and the phase-based motion magnification method is implemented to magnify micro-vibration of the structure. By employing discretized centroid searching method, spatial displacements of the light poles are extracted. Modal parameters of the structure are then identified from the displacement responses by dynamic mode decomposition method. The study includes laboratory experiments for verification, and the full-scale implementation on existing light poles on elevated highway bridges. The vision-based results are compared with the noncontact vibration measurement using Laser-Doppler vibrometers. Results of experiments and full-scale tests demonstrate the capability of the proposed method in extracting multi-mode vibration characteristics of the light poles accurately in a noncontact long-distance measurement.

Development of an image-based experimental modal analysis method based on phase-based motion magnification

Development of an image-based experimental modal analysis method based on phase-based motion magnification

Vibration Measurement With Video Processing Based on Alternating Optimization of Frequency and Phase Shifts

Vision-based measurement methods have attracted growing interests over the past years because of their non-contact sensing capability for acquiring the full-field vibrations of structure. Existing vision-based measurement techniques, such as point tracking (PT) and digital image correlation (DIC), require preparation for the surface with high-contrast markers or speckle patterns before measurement. The phase-based motion magnification, enabling to extract the vibration of the structure without surface pre-processing, is a newly developed vision-based measurement method. However, it suffers from the problems such as phase noise, phase instability, and phase unwrapping. In this paper, a novel iterative algorithm called alternating optimization of frequency and phase shifts (AOFPS) is developed to measure the vibration signals from the video instead of explicitly manipulating the phase. The method of Tikhonov regularization for estimating pixel-to-pixel sinusoid patterns and the approach of the least squares for extracting frame-to-frame random phase shifts are alternately implemented to track the motion of the object. This algorithm provides robust and accurate vibration measurements without expensive computation. Simulated and laboratory experiments analyses illustrate the effectiveness and accuracy of the proposed algorithm.