Vibrational Imaging Research Articles

Tool health monitoring in lathe turning process by artificial intelligence techniques — a review

Monitoring tool health is essential for maintaining efficiency, productivity, and quality in lathe turning operations. Traditional methods rely on manual assessments and subjective judgments, which can be time-consuming, inconsistent, and inadequate for detecting subtle tool wear. Therefore, this review discusses the literature review on predicting tool wear in the turning process, comprehensively examining the methods documenting for sensing and testing parameter design, image processing, and classification methods. The review outlines the use of vibration signals and images as datasets and advanced artificial intelligence techniques like machine learning, computer vision, deep learning, and expert systems to predict the accurate wear percentage in the tool. It also discusses the benefits and limitations of methods used in reviewed papers. To conclude, the performance of AI techniques from the reviewed papers, RNN from deep learning, gives more accuracy, with 97.04% predicting the tool wear. Within the Industry 4.0 framework, after a detailed review of the AI techniques, the combination of deep learning techniques that ensemble vibration signals and image information develops as a vital technology for evolving intelligent manufacturing.

Vibration displacement measurement of bridge structural models using image super-resolution reconstruction and visual object detection network

Abstract Visual vibration measurement has emerged in the field of structural health monitoring in recent years, but it still has some shortcomings in terms of resolution, recognition rate and real-time performance. Considering the three aspects of recovering high-frequency image details, improving the compactness of the target bounding box, and reducing the computational time, we use the constructed image super-resolution reconstruction model and target detection model to measure the vibration displacement of the bridge structural model. First, we integrate the Transformer module into the Unet network with a simple structure. The Swin and Global Transformer Unet (SGTU) module constructed in this form can reduce the computational cost while reconstructing the large-resolution feature map target, and it can sharply edge information of the vibration target. We use the framework of the YOLOv5 algorithm as the backbone, and use the GhostBottleneck (GB) module to reduce the time for convolution operations to generate similar features. In addition, the proposed DWCBottleneck (DWCB) fusion module is also able to achieve high-level semantic fusion and network depth expansion with minimal computational cost. Finally, the center point offset of the bounding box predicted by the model can be used to obtain the displacement offset of the object in the image sequence. The position information of the target in the first frame image is used as the reference frame for calculating the offset, and the vibration displacement of the flexible structure in the image coordinate system is obtained by calculating the deviation of the displacement between the remaining frames and the first frame. We perform qualitative and quantitative comparisons in three aspects: video super-resolution reconstruction, visual detection robustness, and sensor vibration measurement displacement using a homemade vibration image dataset. The time–frequency domain displacement curves regressed by the visual vibration measurement algorithm are compared with the curves acquired after accelerometer acquisition, indicating the necessity of super-resolution reconstruction in visual vibration measurement.

Rapid determination of fat content: Advanced spectroscopic methods across diverse fish species

The rapid estimation of fat content using spectroscopic-based sensors, irrespective of fish species (salmon, trout, sea bass, sea bream, tuna, cod, and mackerel), was primarily investigated in the present study. The fat content and fatty acid (FA) composition was quantified with reference methods. Fourier-Transform Infrared (FTIR) and Fourier-Transform Near-Infrared (FT-NIR) spectroscopy, along with multispectral imaging (MSI) instruments (both benchtop and portable), were evaluated for their ability to rapidly predict fat content in ground samples using data analysis. The performance of the PLS-R models was evaluated according to root mean square error (RMSE), coefficient of determination (R2) and residual prediction deviation (RPD). Also fish were classified as fat/low-fat and low/medium/high-fat using partial least squares discriminant analysis (PLS-DA). All fish species exhibited a consistent pattern of unsaturated fatty acids (UNFA) > monounsaturated fatty acids (MUFA) > polyunsaturated fatty acids (PUFA) > saturated fatty acids (SFA) and the range of fat content was from 22.8 (salmon) to 0.02 (cod), expressed in g/100 g of sample based on the reference methods. In terms of fat content prediction using rapid sensors the best performance indices of the test set were obtained from the benchtop-MSI (RMSE = 1.475, R2 = 0.847, RPD = 2.581) and FT-NIR (RMSE = 1.638, R2 = 0.855, RPD = 2.651) instruments, while FTIR and portable-MSI had scores of RMSE = 1.874, R2 = 0.815, RPD = 2.309 and RMSE = 1.737, R2 = 0.786, RPD = 2.191, respectively. All sensors discriminated fat from low-fat samples (accuracy = 100%). For low/medium/high fat the best results were achieved by benchtop-MSI (accuracy = 94.12%), followed by portable-MSI (accuracy = 88.24%) and FTIR (accuracy = 84.21%). Results were less satisfactory for the FT-NIR method (accuracy = 68.75%). This study demonstrates that vibrational and multispectral imaging spectroscopies, when coupled with data analysis, have potential to provide information about the nutritional quality (e.g., fat content) independent of the fish species. Such insights could contribute to the automation and potential digitalization of processes within the fields of food nutrition and the seafood industry.

Hyperspectral acquisition with ScanImage at the single pixel level: application to time domain coherent Raman imaging.

We present a comprehensive strategy and its practical implementation using the commercial ScanImage software platform to perform hyperspectral point scanning microscopy when a fast time-dependent signal varies at each pixel level. In the proposed acquisition scheme, the scan along the X-axis is slowed down while the data acquisition is maintained at a high pace to enable the rapid acquisition of the time-dependent signal at each pixel level. The ScanImage generated raw 2D images have a very asymmetric aspect ratio between X and Y, the X axis encoding both for space and time acquisition. The results are X-axis macro-pixel where the associated time-dependent signal is sampled to provide hyperspectral information. We exemplified the proposed hyperspectral scheme in the context of time-domain coherent Raman imaging, where a pump pulse impulsively excites molecular vibrations that are subsequently probed by a time-delayed probe pulse. In this case, the time-dependent signal is a fast acousto-optics delay line that can scan a delay of 4.5ps in 25μs at each pixel level. With this acquisition scheme, we demonstrate ultra-fast hyperspectral vibrational imaging in the low frequency range [10cm-1, 150 cm-1] over a 500 μm field of view (64 x 64 pixels) in 130ms (∼ 7.5 frames/s). The proposed acquisition scheme can be readily extended to other applications requiring the acquisition of a fast-evolving signal at each pixel level.

Image representation of vibration signals and its application on fault diagnosis of rotating machinery with convolutional neural network: A benchmark study

Intelligent fault diagnosis methods on the basis of two-dimensional (2D) image representation of vibration signals (IRVS) and the convolutional neural network (CNN) have been extensively applied in rotating machinery. However, as a large number of IRVS methods are being used, their performance has not been fairly and comprehensively evaluated, which leads to difficulties for researchers to make choices. To address this issue, this paper conducted a benchmark study aimed at comparing the comprehensive performance of different IRVS methods, including computation time, fault diagnosis accuracy, and noise resistance performance. Firstly, the IRVS methods and the fault diagnosis method combining IRVS and CNN were summarized. Then, the general process of the IRVS-CNN method was proposed. Finally, 17 types of IRVS methods were selected, and the performance of the IRVS method was compared based on three datasets and two classic CNN models. The results indicate that the computing time taken to generate 2D images using unthresholded recurrence plot (UTRP), and frequency-domain unthresholded recurrence plot (FDUTRP) methods is relatively longer. The vast majority of frequency-domain methods and time-frequency domain methods can achieve or approach a high accuracy rate in all experiments, demonstrating their outstanding performance. Frequency-domain color vibration image (FDCVI), FDUTRP, and continuous wavelet transform (CWT) have good noise resistance. These results provide a reference for future researchers in selecting IRVS methods.

Fourier Ptychographic Coherent Anti-Stokes Raman Scattering Microscopy with Point-Scanning for Super-Resolution Imaging.

Fourier ptychography (FP) is a high resolution wide-field imaging method based on the extended aperture in the Fourier space, which is synthesized from raw images with varying illumination angles. If FP is extended to coherent nonlinear optical imaging, the resolution could be further improved due to the increase of the cutoff frequency of the synthesized coherent optical transfer function (C-OTF) with respect to the order of nonlinear optical processes. However, there is a fundamental conflict between wide-field FP and nonlinear optical imaging, whereby the nonlinear optical imaging typically requires a focused excitation laser beam with high power density. To tackle the problem, in this work, a unique point-scanning FP (PS-FP) method is presented for super-resolution nonlinear optical imaging, in which the nonlinear optical signal is obtained by using focused laser beam, while the conventional FP algorithm can still be used to retrieve the super-resolution image. PS-FP coherent anti-Stokes Raman scattering (PS-FP-CARS) imaging on a variety of samples, where a 1.8-fold expansion of the OTF is achieved experimentally for enhancing vibrational imaging. Further theoretical calculation shows that the C-OTF of PS-FP higher-order CARS (PS-FP-HO-CARS) can be expanded up to ≈4.9-fold, thereby improving the spatial resolution by ≈3-fold in comparison with conventional point-scanning CARS with under tightly focused beams. The generality of PS-FP method developed in this work can be adapted to other coherent nonlinear optical imaging modalities for super-resolution imaging in tissue and cells.

Phase nonlinearity–weighted optical flow for enhanced full-field displacement measurement and vibration imaging

In this study, we developed a technique to utilize phase nonlinearity as a quantifiable confidence measure of the reliability of component displacement vectors extracted from multi-scale and multi-direction phases. The proposed approach includes three novel concepts. First, relative confidence values are established based on phase nonlinearity to quantify the reliability of the displacement components across various environmental conditions. Second, by analyzing the distribution of the relative confidence values, adaptive confidence measures are extracted and are used to reject unreliable displacement components; these confidence measures can dynamically adapt to various environmental conditions, such as varying image noise and large amplitudes of motion. Third, phase nonlinearity–weighted motion integration is conducted to ensure accurate estimation of the full displacement vector from the component vectors. By integrating these three approaches, a remarkable enhancement was achieved in full-field displacement measurement in terms of accuracy, robustness, and signal-to-noise ratio, especially in environments with high noise levels. The capability of the proposed technique was evaluated through numerical experiments using artificial patterns that simulate identical motions under varying noise levels. Experimental validation was also performed on an air compressor to substantiate the accuracy and robustness of the proposed technique. The findings verify its improvements over conventional techniques for full-field displacement measurement.

Time-resolved imaging of GHz acoustic vibrations at arbitrary frequencies using asynchronous periodic probe laser pulses

We present a technique for imaging gigahertz surface acoustic waves in the time domain at arbitrary frequencies, in which acoustic waves are excited asynchronously to the optical probe pulse train used to detect them. We apply this approach to the two-dimensional imaging of electrically excited single-frequency surface acoustic waves on a GaAs substrate with a deposited interdigital transducer. This technique significantly increases the possible fields of application for gigahertz acoustic wave device imaging.

Molecular Fingerprinting of Mouse Brain Using Ultrabroadband Coherent Anti-Stokes Raman Scattering (CARS) Microspectroscopy Empowered by Multivariate Curve Resolution-Alternating Least Squares (MCR-ALS).

The Raman fingerprint spectral region provides abundant structural information on molecules. However, analyzing vibrational images within this region using coherent Raman imaging remains challenging due to the small Raman cross section and congested spectral features. In this study, we combined ultrabroadband coherent anti-Stokes Raman scattering (CARS) microspectroscopy across the spectral range of 500-4000 cm-1 with multivariate curve resolution-alternating least-squares (MCR-ALS) to reveal hidden Raman bands in the fingerprint region. Applying this method to mouse brain tissue, we extracted information on cholesterol and collagen, leveraging their distinctive molecular signatures, as well as on key molecules such as lipids, proteins, water, and nucleic acids. Moreover, the simultaneous detection of second harmonic generation facilitated label-free visualization of organelles, including arachnoid membrane and Rootletin filaments.

Towards Quantitative Imaging of Atomic Vibrations with Multi-dimensional STEM Detectors

Towards Quantitative Imaging of Atomic Vibrations with Multi-dimensional STEM Detectors

Real-time bearing fault classification of induction motor using enhanced inception ResNet-V2

ABSTRACT The rolling bearing is a vital part used in different rotating electrical devices. Detecting defects in bearings is crucial for the safe operation of these machines. However, it is challenging to use Deep Learning techniques to identify bearing defects when the machine is not under load. To resolve this issue, this paper presents the Constant-Q Non-stationary Gabor Transform with enhanced Inception ResNet-V2, proposed for the early-stage classification of ball bearing faults in induction motors. The proposed model obtained the vibration images, i.e. time-frequency images of unfiltered vibration signals from the laboratory experimental setup. These images are applied to the proposed model, which classifies the ball bearing faults under various load conditions while adjusting its hyperparameter values instead of employing default ones. Furthermore, the model underwent training using k-fold method to assess its resilience with the use of optimal values obtained from hyperparameter tuning. The model is evaluated by performance metrics like F1-score, Recall, Precision, Confusion Matrix and Training time. The proposed model accomplished an average classification accuracy of 99.84% in low load and full load conditions within a few epochs. Ultimately, when compared to Inception-V4 and ResNet-50, which achieved 91.41% and 91.65%, respectively, the experimental findings unambiguously demonstrate the superior performance of the proposed model over both models.

Phase-based Full-field Displacement Measurement with Phase Nonlinearity Weighting Process

The camera, as a non-contact sensor, has evolved into a robust tool for measuring the full-field vibration of complex structures. It offers distinct advantages over traditional contact sensors, including flexible positioning, simultaneous multi-point tracking, and high spatial resolution. Vibration imaging techniques have been developed to identify structural operational conditions through feature extraction methods. Phase-based vibration imaging techniques can visualize full-field operational shapes or vibrational features in a less-supervised manner. While phase-based optical flow with Gabor filters provides accurate displacement estimation, managing a large amount of image data can pose challenges in extracting full-field displacement efficiently. In this paper, we propose a compressive sensing approach for full-field phase-based optical flow and vibration imaging. Compressive sensing serves as an efficient sampling method that requires significantly fewer frames than traditional approaches, selecting frames at less than one third of the original video. Features are extracted from the displacement of the compressed video frames. Subsequently, the vibration features are transformed into images. The proposed methodology is validated through experiments conducted on an air compressor. Comparative results between vibration imaging from video with compressive sensing and that from the original video demonstrate that vibrational features can be extracted using only a few frames of the original video. Consequently, this approach can reduce processing time and data volume while preserving the performance of vibration feature extraction.

Vibrational imaging of metabolites for improved microbial cell strains.

Biomanufacturing utilizes modified microbial systems to sustainably produce commercially important biomolecules for use in agricultural, energy, food, material, and pharmaceutical industries. However, technological challenges related to non-destructive and high-throughput metabolite screening need to be addressed to fully unlock the potential of synthetic biology and sustainable biomanufacturing. This perspective outlines current analytical screening tools used in industrial cell strain development programs and introduces label-free vibrational spectro-microscopy as an alternative contrast mechanism. We provide an overview of the analytical instrumentation currently used in the "test" portion of the design, build, test, and learn cycle of synthetic biology. We then highlight recent progress in Raman scattering and infrared absorption imaging techniques, which have enabled improved molecular specificity and sensitivity. Recent developments in high-resolution chemical imaging methods allow for greater throughput without compromising the image contrast. We provide a roadmap of future work needed to support integration with microfluidics for rapid screening at the single-cell level. Quantifying the net expression of metabolites allows for the identification of cells with metabolic pathways that result in increased biomolecule production, which is essential for improving the yield and reducing the cost of industrial biomanufacturing. Technological advancements in vibrational microscopy instrumentation will greatly benefit biofoundries as a complementary approach for non-destructive cell screening.

Millimetre-deep micrometre-resolution vibrational imaging by shortwave infrared photothermal microscopy

Millimetre-deep micrometre-resolution vibrational imaging by shortwave infrared photothermal microscopy

Overtone photothermal microscopy for high-resolution and high-sensitivity vibrational imaging

Photothermal microscopy is a highly sensitive pump-probe method for mapping nanostructures and molecules through the detection of local thermal gradients. While visible photothermal microscopy and mid-infrared photothermal microscopy techniques have been developed, they possess inherent limitations. These techniques either lack chemical specificity or encounter significant light attenuation caused by water absorption. Here, we present an overtone photothermal (OPT) microscopy technique that offers high chemical specificity, detection sensitivity, and spatial resolution by employing a visible probe for local heat detection in the C-H overtone region. We demonstrate its capability for high-fidelity chemical imaging of polymer nanostructures, depth-resolved intracellular chemical mapping of cancer cells, and imaging of multicellular C. elegans organisms and highly scattering brain tissues. By bridging the gap between visible and mid-infrared photothermal microscopy, OPT establishes a new modality for high-resolution and high-sensitivity chemical imaging. This advancement complements large-scale shortwave infrared imaging approaches, facilitating multiscale structural and chemical investigations of materials and biological metabolism.

Preliminary Findings of Vocal Fold Vibratory Characteristics of Singers Analyzed by Laryngeal High-Speed Videoendoscopy

ObjectivesThis study investigates the vocal fold vibratory dynamics of singers, which are postulated to differ from those of normal speakers due to the singers' regular vocal training. The measurement of vocal fold vibration involved the utilization of laryngeal high-speed video endoscopy (LHSV) and subsequent LHSV-based analysis. The focus is to characterize and compare the LHSV-based measures derived from the glottal area waveform (GAW), namely fundamental frequency (F0GAW), glottal perturbation (jitterGAW and shimmerGAW), open quotient (OQGAW), and Nyquist plots, between singers and normal speakers across genders. MethodsParticipants comprised 13 singers from a local cultural and heritage academy and 56 normal speakers from a local university, all were evaluated to have normal voices. Each participant underwent LHSV procedures to capture images of vocal fold vibration, which were subsequently analyzed to generate the LHSV-based measures. ResultsMale singers exhibited lower F0GAW, jitterGAW, shimmerGAW, and OQGAW than female singers. When compared to normal speakers, male singers demonstrated higher F0GAW, and lower jitterGAW and shimmerGAW. No difference in OQGAW was found between male singers and normal speakers. Female singers exhibited lower jitterGAW compared to normal speakers, but no differences were observed in shimmerGAW and OQGAW. The results of Nyquist plots indicated no gender-related associations between types of rim width and among singers. However, for rim pattern, male singers were associated with a higher percentage of clustered rim, suggesting more regular vocal fold vibration, compared to female singers and male normal speakers. ConclusionsSingers, particularly male singers, demonstrate distinct and potentially superior vocal fold vibrations compared to normal speakers, likely attributed to their regular vocal training, resulting in refined vocal fold configurations even during speaking. Despite the limited sample of singers, the study offers valuable insights into the vocal fold vibratory behaviors in singers analyzed using LHSV.

Application of ultrasonic vibration in magnesium alloy lap welding

In order to enhance the performance of magnesium alloy and galvanized steel welds, ultrasonic vibration was applied to the laser welding process, and the effect of ultrasonic on mechanical properties was verified through experimental methods. The laser welding system for the lap weld seam was designed, and the ultrasonic vibration module and image detection module were added, which can obtain the influence of ultrasonic vibration on the molten pool area. Under the conditions of ultrasonic vibration power of 1000 W and 0 W, the characteristics of the weld pool area, metallographic structure, tensile strength, fracture morphology, hardness, residual stress, wear resistance, and corrosion resistance of the weld specimen were compared. The image data acquisition structure indicated that ultrasonic vibration can effectively reduce the ineffective area of the molten pool and make the energy in the molten pool more concentrated. Under the influence of ultrasonic vibration, the maximum molten pool area decreased to 5.38 mm2, with a variation range of 3.9 %, and the proportion of pores was greatly reduced. Research found that ultrasonic vibration can significantly improve the microstructure characteristics of the fusion welding zone, with an average grain size reduced to 23 μm. The reduction of grain size and refinement of microstructure were beneficial to the improvement of mechanical properties of magnesium alloy joints, with a yield strength increase of 6.5 %. Ultrasonic vibration had little effect on the hardness of the heat affected zone, it can increase the average hardness of the weld zone by more than 5 % and reduce the maximum residual stress by more than 50 %. Under different pressure and friction speed conditions, the maximum wear amount can be reduced by more than 25 %. At the same time, the resistance to oxidation corrosion and electrochemical corrosion also can be improved to a certain extent.

Detection of jelly orange granulation disease using a dual-input Resnet-Transformer model (DresT) based on acoustic vibration images and a novel acoustic vibration device

Granulation is a common internal disease in citrus fruits, and it is difficult to distinguish fruits with granulation disease from their appearance. In this study, a novel acoustic vibration device based on a micro-LDV, a microphone and a resonance speaker was employed to collect acoustic vibration response signals of "Aiyuan 38" jelly orange. The one-dimensional acoustic vibration response signal was converted into acoustic vibration images, and a double-input Resnet-Transformer network (DresT) was constructed for extracting deep features in acoustic vibration images for identifying jelly-orange granulation disease. Firstly, train Drest and Resnet50 models using acoustic vibration images and compare the performance of Drest with that of Resnet50 (based on CNN). Then PLS-DA and SVM models are trained using acoustic vibration image texture features or acoustic vibration spectral features, and the performance is compared with the DresT model. The results showed that the DresT model trained using acoustic vibration images can accurately identify jelly orange granulation disease with a detection accuracy of 99.31 %. The F1 of the model is 99.5 %, the accuracy is 99.01 %, and the recall is 100 %.

Photoluminescence Emission Efficiency Analysis Methodology by Integrating Raman Spectroscopy of the A1(LO) and E2(high) Phonons in a GaInN/GaN Heterostructure

Microscopic lattice vibration images of the E2(high) mode (E2H) and another mode of A1(LO) (A1L) or the higher energy branch of LO‐phonon−plasmon coupling mode (LOPC+) in a Ga0.95In0.05N film on a GaN template are obtained by Raman scattering spectroscopy using a 325 nm laser. The increase in temperature by increasing the laser power is obtained from the decrease in the energy of E2H and the theoretical formula comprising two terms based on the mode energy variation of the bulk material and the thermal strain effect. Using the obtained temperature and the energy shift of the LOPC+, the mapping images of the temperature and electron density in the x–y plane are simultaneously obtained. This image provides the spatial variation of photoluminescence (PL) emission efficiency, given as PL intensity per electron. This method enables the quantitative discussion on photo‐emission efficiency even in the regions of low or high carrier density affected by carrier transport. In the investigated area, a region with a lower PL efficiency is found despite a higher electron density and lower temperature increase than the surrounding region. This imaging analysis is feasible in integrating the carrier and thermal energy transports and recombination processes in carrier dynamics study.

A novel diagnostic framework based on vibration image encoding and multi-scale neural network

Intelligent fault diagnosis employing CNN-based techniques has demonstrated promising results in rotating machinery maintenance and management. However, most approaches that rely on time-series vibration signals fail to incorporate the physical knowledge of faults into feature learning, and are subject to obtaining the correlation between input and output characteristics. Furthermore, it is challenging for the neural network to improve feature extraction due to the rigid convolution kernel and the limited receptive field. To address these issues, an intelligent fault diagnosis framework is developed based on vibration image encoding and multi-scale neural networks. Firstly, a novel image encoding rule is designed to convert time-series signals into vibration images, which incorporate the impact characteristics of faults into the model inputs to enhance fault information. Secondly, a multi-scale feature fusion network with dilatated and irregular sampling convolution block is constructed for feature extraction and classification. In addition, a new regularization term is designed to avoid training overfitting. Finally, Grad-CAM is introduced to visualize the region of interest of the model. The proposed method is verified on the datasets with different fault types from our laboratory and DIRG bearing dataset with different fault severities from Politecnico di Torino. The experimental results demonstrate that our diagnostic framework exhibits good generalization and noise robustness, and can achieve high-precision fault diagnosis.