Thermal Feature Extraction Research Articles

Evaluation of design information disclosure through thermal feature extraction in metal based additive manufacturing

Manufacturing-as-a-Service (MaaS) can accelerate additive manufacturing (AM) process-defect modeling by augmenting training data to all collaborating users via a data sharing network. However, sharing process data may disclose product design information. This paper aims to evaluate design information disclosure of various thermal history-based feature extraction methods for metal-based AM anomaly detection. This is accomplished by evaluating the design information (i.e., printing orientation) retained, and the overall data usability (i.e., anomaly detection) preserved in the extracted features for various state-of-the-art feature extraction methods. The evaluation results indicate that there are urgent needs in privacy preserving data sharing for additive MaaS (AMaaS).

Emissivity Correction and Thermal Pattern Reconstruction in Eddy Current Pulsed Thermography

Emissivity variations are one of the most critical challenges in thermography technologies; this is due to the temperature calculation strongly depending on emissivity settings for infrared signal extraction and evaluation. This paper describes an emissivity correction and thermal pattern reconstruction technique based on physical process modelling and thermal feature extraction, for eddy current pulsed thermography. An emissivity correction algorithm is proposed to address the pattern observation issues of thermography in both spatial and time domains. The main novelty of this method is that the thermal pattern can be corrected based on the averaged normalization of thermal features. In practice, the proposed method brings benefits in enhancing the detectability of the faults and characterization of the materials without the interference of the emissivity variation problem at the object's surfaces. The proposed technique is verified in several experimental studies, such as the case-depth evaluation of heat-treatment steels, failures, and fatigues of gears made of the heat-treated steels that are used for rolling stock applications. The proposed technique can improve the detectability of the thermography-based inspection methods and would improve the inspection efficiency for high-speed NDT&E applications, such as rolling stock applications.

BOSF-SVM: A thermal image-based fault diagnosis method of circuit boards

In recent years, the lack of thermal images and the difficulty of thermal feature extraction have led to low accuracy and efficiency in the fault diagnosis of circuit boards using thermal images. To address the problem, this paper presents a simple and efficient intelligent fault diagnosis method combined with computer vision, namely the bag-of-SURF-features support vector machine (BOSF-SVM). Firstly, an improved BOF feature extraction based on SURF is proposed. The preliminary fault features of the abnormally hot components are extracted by the speeded-up robust features algorithm (SURF). In order to extract the ultimate fault features, the preliminary fault features are clustered into K clusters by K-means and substituted into the bag-of-features model (BOF) to generate a bag-of-SURF-feature vector (BOSF) for each image. Then, all of the BOSF vectors are fed into SVM to train the fault classification model. Finally, extensive experiments are conducted on two homemade thermal image datasets of circuit board faults. Experimental results show that the proposed method is effective in extracting the thermal fault features of components and reducing misdiagnosis and underdiagnosis. Also, it is economical and fast, facilitating savings in labour costs and computing resources in industrial production.

Thermal-Inertial SLAM for the Environments With Challenging Illumination

In recent years, longwave infrared (LWIR) cameras have become potential in visual simultaneous localization and mapping (SLAM) research since the delivered thermal images can provide information beyond the visible spectrum and are robust to environment illumination. However, due to modality differences, SLAM methods designed for visible cameras cannot be directly applied to thermal data. In this paper, we propose a thermal-inertial SLAM method for all-day autonomous systems. To overcome the challenge of the thermal data association, the proposed method represents several improvements, including singular-value-decomposition-based (SVD-based) image processing and ThermalRAFT tracking methods. Based on the characteristics of the thermal images, the SVD-based image processing method can exploit the fixed noise pattern of thermal images and enhance the image quality to improve the performance of subsequent steps, including thermal feature extraction and loop detection. To achieve real-time and robust feature tracking, we develop ThermalRAFT, an efficient optical flow network with iterative optimization. Moreover, the system introduces a bag-of-words-based loop detection method to maintain global consistency in long-term operation. The experimental results demonstrate that the proposed method can provide competitive performance in indoor and outdoor environments and is robust under challenging illumination conditions.

Finite element modeling of direct metal laser solidification process: Sensor data replication and use in defect detection and data reduction via machine learning

In the additive manufacturing (AM) industry, the direct metal laser solidification (DMLS) process has received significant research interest because of its ultra-high precision and geometry variability. However, many possible process errors can happen during the printing process and in-situ quality control is quite challenging because of the lack of appropriate monitoring and analysis capabilities. Thus, in this work, a part-scale finite element method (FEM) model is developed to investigate the heat transfer behavior of the DMLS process and to extract experimentally relevant thermal features. Specifically, a microscopic and a mesoscale sub-model are initially developed to describe powder properties and the laser behaviors, respectively, and their outputs are directly incorporated in the part-scale FEM model. The FEM model-generated data are then processed to replicate the long-wave infrared (LWIR) camera sensor outputs. Finally, both the experimental and FEM model-generated images are used to train machine learning algorithms for in-situ defect detection applications. In addition, we are endeavoring to look into a convolutional neural network utilizing such realistic thermal features from LWIR images with and without a variety of simulated data augmentation to examine its contribution to the CNN training. The thermal feature extraction and the machine learning algorithm are then utilized for the data reduction purpose. Also, a transmission strategy is demonstrated to filter out a significant amount of redundant data while maintaining high model prediction quality.

A Novel Facial Thermal Feature Extraction Method for Non-Contact Healthcare System

The non-contact healthcare system is a system that can avoid germ infection, and can also provide comfortable and convenient health care services for the caregivers. In the current thermal imaging research, local or small area images are used to represent the overall temperature information of the participants, but the temperature feature of the face should not only be used in a small part and ignore other parts. The facial thermal image can show clear temperature feature but is not conducive to meaningful feature extraction. Therefore, this research proposed a novel facial thermal image feature extraction method, which is used facial landmarks to detect and cut 12 blocks to establish a new feature matrix based on color mean values and standard deviation values. It can establish clear features on facial thermal images. The core part of the proposed healthcare system is the use of a deep learning framework, which is based on CAFFE under the DIGITS platform. The CAFFE runs the classic CNN, GoogLeNet. Based on the acquired images and new feature types, four models were trained, which were used for the raw RGB image, raw thermal image, RGB feature image, and thermal feature image. In the experiment, 800 images were used for training and validation, and 200 images were used for testing. An additional 40 images were used for random testing. The experimental results show that RGB images cannot be effectively used, thermal images can effectively predict the health status, and thermal feature images have the highest prediction accuracy.

Thermal strain extraction methodologies for bridge structural condition assessment

This paper presents a feature extraction method to uncover the temperature effects on bridge responses, which combines mode decomposition, data reduction, and blind separation. For mode decomposition, empirical mode decomposition (EMD) and ensemble empirical mode decomposition (EEMD) have been executed, followed by principal component analysis (PCA) for data size compression. Independent component analysis (ICA) is then employed for blind separation. The unique feature of the proposed method is the blind separation, which enables temperature-induced response to be extracted from the mixed structural responses without any prior information of the loading conditions and structural physical models. This study further evaluates the effects of extracting a temperature-induced response on damage detectability when using Moving Principal Component Analysis (MPCA). Numerical analysis for a truss bridge model is first conducted to evaluate the proposed method for thermal feature extraction, followed by a real truss bridge test in the structural laboratory at the University of Warwick. Results from the numerical study show that the method enables the separation of a temperature-induced response; and furthermore, the EEMD, utilized in mode decomposition, has a positive influence on the blind separation compared with EMD when combined with PCA and ICA. Finally, the real truss bridge test demonstrates that the feature extraction method can enhance the probability of MPCA to uncover the damage as the MPCA fails to discern damage without the proposed method.

Ensemble variational Bayes tensor factorization for super resolution of CFRP debond detection

The carbon fiber reinforced polymer (CFRP) is widely used in aircraft and wind turbine blades. The common type of CFRP defect is debond. Optical pulse thermographic nondestructive evaluation (OPTNDE) and relevant thermal feature extraction algorithms are generally used to detect the debond. However, the resolution of detection performance remain as challenges. In this paper, the ensemble variational Bayes tensor factorization has been proposed to conduct super resolution of the debond detection. The algorithm is based on the framework of variational Bayes tensor factorization and it constructs spatial-transient multi-layer mining structure which can significantly enhance the contrast ratio between the defective regions and sound regions. In order to quantitatively evaluate the results, the event based F-score is computed. The different information regions of the extracted thermal patterns are considered as different events and the purpose is to objectively evaluate the detectability for different algorithms. Experimental tests and comparative studies have been conducted to prove the efficacy of the proposed method.

Quantitative validation of Eddy current stimulated thermal features on surface crack

Thermography is an established technique in Non-Destructive Testing and Evaluation (NDT&E) for detecting and visualizing defects. The acquired thermal image sequences contain valuable information in the spatial, time and frequency domains. Many thermal feature extraction methods have been previously proposed to detect and quantify defects. However, the quantitative assessment and error analysis remain as challenges. In the present work, a quantitative analysis strategy is proposed for validating detection performance of various thermal feature extraction techniques based on Eddy Current Stimulated Thermography. F-score is adopted as a global quantitative evaluation indicator to assess the crack detectability. The quantitative validation is carried out based on a large number of simulations and experiments for cracks on the nonferromagnetic material. The detectability in relation to the variations in crack size and the different thermal feature extraction methods have been critically analyzed. The insights obtained from the present findings can serve as guidelines to other types of thermal-based NDT&E to improve the defect visibility and detectability.

Automated detection and recognition of wildlife using thermal cameras.

In agricultural mowing operations, thousands of animals are injured or killed each year, due to the increased working widths and speeds of agricultural machinery. Detection and recognition of wildlife within the agricultural fields is important to reduce wildlife mortality and, thereby, promote wildlife-friendly farming. The work presented in this paper contributes to the automated detection and classification of animals in thermal imaging. The methods and results are based on top-view images taken manually from a lift to motivate work towards unmanned aerial vehicle-based detection and recognition. Hot objects are detected based on a threshold dynamically adjusted to each frame. For the classification of animals, we propose a novel thermal feature extraction algorithm. For each detected object, a thermal signature is calculated using morphological operations. The thermal signature describes heat characteristics of objects and is partly invariant to translation, rotation, scale and posture. The discrete cosine transform (DCT) is used to parameterize the thermal signature and, thereby, calculate a feature vector, which is used for subsequent classification. Using a k-nearest-neighbor (kNN) classifier, animals are discriminated from non-animals with a balanced classification accuracy of 84.7% in an altitude range of 3–10 m and an accuracy of 75.2% for an altitude range of 10–20 m. To incorporate temporal information in the classification, a tracking algorithm is proposed. Using temporal information improves the balanced classification accuracy to 93.3% in an altitude range 3–10 of meters and 77.7% in an altitude range of 10–20 m

Classifying pretended and evoked facial expressions of positive and negative affective states using infrared measurement of skin temperature

Earlier researchers were able to extract the transient facial thermal features from thermal infrared images (TIRIs) to make binary distinctions between the expressions of affective states. However, effective human-computer interaction would require machines to distinguish between the subtle facial expressions of affective states. This work, for the first time, attempts to use the transient facial thermal features for recognizing a much wider range of facial expressions. A database of 324 time-sequential, visible-spectrum, and thermal facial images was developed representing different facial expressions from 23 participants in different situations. A novel facial thermal feature extraction, selection, and classification approach was developed and invoked on various Gaussian mixture models constructed using: neutral and pretended happy and sad faces, faces with multiple positive and negative facial expressions, faces with neutral and six (pretended) basic facial expressions, and faces with evoked happiness, sadness, disgust, and anger. This work demonstrates that (1) infrared imaging can be used to observe the affective-state-specific facial thermal variations, (2) pixel-grey level analysis of TIRIs can help localise significant facial thermal feature points along the major facial muscles, and (3) cluster-analytic classification of transient thermal features can help distinguish between the facial expressions of affective states in an optimized eigenspace of input thermal feature vectors. The observed classification results exhibited influence of a Gaussian mixture model's structure on classifier-performance. The work also unveiled some pertinent aspects of future research on the use of facial thermal features in automated facial expression classification and affect recognition.