Thermal Face Images Research Articles

A Preliminary Study on Correlation Between Independent Components of Facial Thermal Images and Subjective Assessment of Chronic Stress

A Preliminary Study on Correlation Between Independent Components of Facial Thermal Images and Subjective Assessment of Chronic Stress

Driver drowsiness is associated with altered facial thermal patterns: Machine learning insights from a thermal imaging approach

Driver drowsiness is a significant factor in road accidents. Thermal imaging has emerged as an effective tool for detecting drowsiness by enabling the analysis of facial thermal patterns. However, it is not clear which facial areas are most affected and correlate most strongly with drowsiness. This study examines the variations and importance of various facial areas and proposes an approach for detecting driver drowsiness. Twenty participants underwent tests in a driving simulator, and temperature changes in various facial regions were measured. The random forest method was employed to evaluate the importance of each facial region. The results revealed that temperature changes in the nasal area exhibited the highest value, while the eyes had the most correlated changes with drowsiness. Furthermore, drowsiness was classified with an accuracy of 88 % utilizing thermal variations in the facial region identified as the most important regions by the random forest feature importance model. These findings provide a comprehensive overview of facial thermal imaging for detecting driver drowsiness and introduce eye temperature as a novel and effective measure for investigating cognitive activities.

Analysis of thermal videos for detection of lie during interrogation

The lie-detection tests are traditionally carried out by well-trained experts using polygraph machines. However, it is time-consuming, invasive, and, overall, a cumbersome process, not admissible by the court of law. Trained individuals can easily flaunt these tests. On the other hand, facial thermal imaging could be more effective as it is noninvasive and could be a stealth method of tracking the facial blood flow patterns, which have been proven to detect deceit. This paper presents a method based on facial thermal imaging to detect deception in human subjects. The major issue in such research is the lack of proper real-life databases which emulate the crime scenes. In this paper, first, we have developed a database based on almost real-life theft incidents with due diligence using isolated subjects over a period of time at a government hospital under the plea of free health checkup. The experiment has been conducted at Midnapore Medical College and Hospital, West Bengal, India, with proper ethical committee approval. The participants are selected at the behest of the police department with habitual crime records. Most of them have been repeatedly charged with petty crimes of pick-pocketing and stealing. They are invited individually at different instances of time under the plea of medical checkup where they have been enticed to steal cash. It is followed by a two-stage process, a friendly interaction followed by a slightly tougher interrogation. Their forehead and periorbital region skin surface temperature are recorded by a hidden thermal camera. Upon analysis, conspicuous changes in the temperature profile and blood flow pattern have been observed in the individuals who stole money and those who did not.

Dual-phase prediction model of passenger thermal sensation using facial thermal imaging and environmental factors

The concept of passenger-centric design has become increasingly significant in the intelligent automotive industry. Enhancing in-cabin thermal comfort while maintaining energy efficiency is crucial, and to this end, we leverage real-time passenger thermal sensation as a basis for HVAC control strategies. This study introduces a non-invasive, dual-phase prediction model that evaluates passenger thermal comfort by utilizing facial thermal imaging and environmental factors. The proposed model employs a deep learning neural network to extract pertinent features from the facial thermal images for an initial prediction. Then this prediction is refined by integrating it with environmental data through a machine learning algorithm, facilitating real-time personal thermal comfort assessment. Data collection for the model was performed across two distinct seasons, with 22 participants in summer and 18 in winter, gathering facial thermal images, environmental parameters, and thermal sensation votes to construct a comprehensive dataset. Analysis revealed a set of critical environmental factors that correlate strongly with thermal sensation votes, which were used as auxiliary features in the model. The performance of the proposed dual-phase approach, combining deep learning and machine learning techniques, demonstrated robust effectiveness on both summer and winter datasets. This substantiates the method's practicality for real-world application within vehicle environments.

Performance enhancement of thermal image analysis for noncontact cardiopulmonary signal extraction

Infrared thermography (IRT) has emerged as a tool for the rapid and non-contact detection of febrile individuals in places of mass gathering. This study explores the feasibility of extracting cardiopulmonary signals (i.e., respiration and cardiac pulse signals) by analyzing thermal image sequences. A high IR spatial resolution (320 × 240 pixels) IRT was applied to acquire facial thermal image sequences at a frame rate of 60 frames per second (fps). The respiration signal was extracted by monitoring the temperature changes in the region of interest (ROI) of the nostrils caused by breathing, that is, the temperature increase during the exhalation phase and the temperature decrease during the inhalation phase. The respiration signal was extracted by picking up the rise and fall motions of the shoulders caused by breathing. The cardiac pulse signal was extracted by measuring tiny head motions accompanying the cardiac cycle on the thermal image sequences. The system was tested on 11 subjects in a laboratory setting to assess the respiration performance and pulse signal extraction. Bland–Altman and Pearson's correlation plots were used to compare the noncontact respiration rate (RR) and pulse rate (PR) measurements with contact-type ECG and respiration belt. The mean difference in the RR was − 0.2 bpm with the 95 % limit of − 5.0 to 4.6 bpm, correlation of 0.83, root mean square error (RMSE) of 2.4 bpm, and mean absolute percentage error (MAPE) of 22 %. Regarding the PR, the mean difference of 4.2 bpm with the 95 % limit of − 18 to 26 bpm, correlation of 0.67, RMSE of 12 bpm, and MAPE of 15 % were determined from comparison of the ECG and thermal camera. In summary, our findings suggest that the proposed thermal image analysis method for measuring RR and PR is promising, particularly for RR measurements. Further validation and refinement are required for the PR measurements. The results indicate that this noncontact approach has potential for various medical applications.

An Optimization for Resting Blood Pressure Estimation Model Based on Independent Components of Facial Thermal Images

The increasing number of patients with hypertension is a problem worldwide. Daily blood pressure monitoring is important for the prevention of this disease, and a non‐contact measurement method is desirable. A previous study applied independent components analysis to facial skin temperatures measured remotely using a thermography device for extracting features related to resting blood pressure. Then, a blood pressure estimation model was constructed and obtained an accuracy of 7.73 mmHg root mean square error. However, this previous study did not consider the variables selection methods and models to be constructed. Considering these factors is expected to improve the accuracy of estimation. The objective of this study is to optimize the resting blood pressure estimation model. Two variable selection methods and three modeling methods were applied to independent components of facial thermal images, and the accuracy was compared. The result showed an accuracy of 6.80 mmHg root mean square error with a pattern of support vector regression model using independent components selected by Boruta. In addition, eight explanatory variables were selected by Boruta, and suggesting that the lip, orbital, and lateral nasal regions were related to resting blood pressure. © 2024 Institute of Electrical Engineer of Japan and Wiley Periodicals LLC.

Thermal-based gender recognition using drones: advancing biometric recognition in challenging outdoor environments

While biometric recognition typically uses features such as face, fingerprint, and iris to identify individuals, this study focuses on utilising specific characteristics to identify gender. The aim of this article is to propose a procedure for gender recognition under specific conditions. The specific condition addressed is outdoor area monitoring, which presents challenges such as varying lighting conditions and limited camera placement options. To tackle this, a proposed procedure utilises thermal images captured by the drone equipped with a thermal camera. The advantage of thermal images is their independence from ambient light conditions. The captured images are resized and processed using convolutional neural networks (CNNs) (AlexNet, VGG-16, VGG-19) for feature extraction and binary classification. A freely available database of thermal face images is used for training the CNNs, while a own created dataset of thermal images obtained by the drone is used for testing. The findings indicate that the optimised CNNs achieve classification accuracies of 82.4% (VGG-16), 82.9% (AlexNet), and 85.5% (VGG-19). The original contribution of this study lies in demonstrating the suitability of face thermal images obtained through drones for gender recognition purposes.

Advanced Deep Learning Techniques for High-Quality Synthetic Thermal Image Generation

In this paper, we introduce a cutting-edge system that leverages state-of-the-art deep learning methodologies to generate high-quality synthetic thermal face images. Our unique approach integrates a thermally fine-tuned Stable Diffusion Model with a Vision Transformer (ViT) classifier, augmented by a Prompt Designer and Prompt Database for precise image generation control. Through rigorous testing across various scenarios, the system demonstrates its capability in producing accurate and superior-quality thermal images. A key contribution of our work is the development of a synthetic thermal face image database, offering practical utility for training thermal detection models. The efficacy of our synthetic images was validated using a facial detection model, achieving results comparable to real thermal face images. Specifically, a detector fine-tuned with real thermal images achieved a 97% accuracy rate when tested with our synthetic images, while a detector trained exclusively on our synthetic data achieved an accuracy of 98%. This research marks a significant advancement in thermal image synthesis, paving the way for its broader application in diverse real-world scenarios.

Classification of User Emotional Experiences on B2C Websites Utilizing Infrared Thermal Imaging.

The acquisition of physiological signals for analyzing emotional experiences has been intrusive, and potentially yields inaccurate results. This study employed infrared thermal images (IRTIs), a noninvasive technique, to classify user emotional experiences while interacting with business-to-consumer (B2C) websites. By manipulating the usability and aesthetics of B2C websites, the facial thermal images of 24 participants were captured as they engaged with the different websites. Machine learning techniques were leveraged to classify their emotional experiences, with participants' self-assessments serving as the ground truth. The findings revealed significant fluctuations in emotional valence, while the participants' arousal levels remained consistent, enabling the categorization of emotional experiences into positive and negative states. The support vector machine (SVM) model performed well in distinguishing between baseline and emotional experiences. Furthermore, this study identified key regions of interest (ROIs) and effective classification features in machine learning. These findings not only established a significant connection between user emotional experiences and IRTIs but also broadened the research perspective on the utility of IRTIs in the field of emotion analysis.

Development of abnormal facial temperature detection technology using thermal imaging to prevent the spread of infectious diseases

Substantial research has been conducted on image-based augmentation, segmentation, detection, and classification of COVID-19. Thermal image classification methods have been used to identify people with high fever. However, these methods cannot always identify infected patients because aberrant images cannot be detected using face temperatures. For example, if the individual in the thermal image cleansed his/her face a few minutes earlier or was perspiring, the method cannot reliably determine their facial temperature. Moreover, the active movements of the human body can produce temperature variations in the face. Thus, achieving high accuracy remains challenging in thermal facial image identification. Therefore, in this study, a novel research has been conducted to solve the problems. Based on preliminary experimental results, a deep-learning-based classification system was proposed for detecting faces with abnormal facial temperatures in thermal pictures. Two different experiments including abnormal face images detection and classification were conducted. To conduct these experiments, a self-collected thermal face-image dataset including abnormal (faces after exercise, sweaty and wet faces) and normal images was used. In the detection method, a binary image classification was conducted to classify the abnormal and normal images. Moreover, in the classification method, an input image is classified into four classes, such as normal faces (class 1) and three abnormal faces, namely, after physical activity (class 2), sweaty face (class 3), and wet face (class 4). Testing on the dataset collected in this study revealed that the proposed detection and classification methods achieved F1-score of 95.71 and 96.17, respectively. These performances are superior to those of the state-of-the-art methods.

Building a Database with Thermal Images for the Classification of Emotional States

Abstract This research paper aims primarily to highlight the stages regarding the development of a database with facial images in the thermal spectrum, presenting various emotional states (for example, neutral, happy and sad). A series of exploratory results was presented, carried out on five databases with thermal images (DBold-2019, DBnew-2023, DBnew-2023_Crop, DBnew- 2023_Noise and DBnew-2023_ROI), belonging to the CCETIC research center (CCETIC, 2023). The database development process includes both the thermal image acquisition part and the pre-processing steps (cutting, alignment, selection of areas of interest, removal of annotations/marking elements, etc). And secondly, it aims to present an analysis regarding the general structure of the thermal image processing application, together with a series of image processing methods and techniques, addressed in certain research works. Some experimental results obtained with the help of the five databases are presented, as well as a series of general conclusions regarding the development of a database with facial thermal images.

Spatial Feature Extraction for Acute Blood Pressure Fluctuations in Facial Visible Images Using Sparse Coding

Remote blood pressure measurement using visible images facilitates routine blood pressure monitoring and leads to early detection of hypertension, a risk factor for lifestyle diseases. The previous study that attempted to estimate blood pressure by applying CNN to facial thermal images found that facial images contain two types of features, physiological responses and expression changes, which need to be separated. In contrast, we found that these features could be separated by using sparse coding on facial thermal images. This study used sparse coding to extract physiological response areas during acute blood pressure fluctuations from facial visible images by examining preprocessing. The results indicated that sparse coding and the proposed preprocessing methods for images were effective. © 2023 Institute of Electrical Engineer of Japan and Wiley Periodicals LLC.

Utilizing Alignment Loss to Advance Eye Center Detection and Face Recognition in the LWIR Band

Geometric normalization is an integral part of most of the face recognition (FR) systems. To geometrically normalize a face, it is essential to detect the eye centers, since one way to align the face images is to make the line joining the eye centers horizontal. This paper proposes a novel approach to detect eye centers in the challenging Long-Wave Infrared (LWIR) spectrum (8-14 μm). While using thermal band images for face recognition is a feasible approach in low-light and nighttime conditions, where visible face images cannot be used, there are not many thermal or dual band (visible and thermal) face datasets available to train and test new eye center detection models. This work takes advantage of the available deep learning based eye center detection algorithms in the visible band to detect the eye centers in thermal face images through image synthesis. While we empirically evaluate different image synthesis models, we determine that StarGAN2 yields the highest eye center detection accuracy, when compared to the other state-of-the-art models. We incorporate alignment loss that captures the normalized error between the detected and actual eye centers as an additional loss term during training (using the generated images during training, ground truth annotations, and an eye center detection model), so that the model learns to align the images to minimize this error. During test phase, visible images are generated from the thermal images using the trained model. Then, the available landmark detection algorithms in the visible band, namely, MT-CNN and HR-Net are used to detect the eye centers. Next, these eye centers are used to geometrically normalize the source thermal face images before performing same-spectral (thermal-to-thermal) face recognition. The proposed method improved the eye center detection accuracy by 60% over the baseline model, and by 14% over training only the StarGAN2 model without the alignment loss. The proposed approach also reports the highest improvement in the face recognition accuracy by 36% and 3% over the baseline and original StarGAN2 models, respectively, when using deep learning based face recognition models, namely, Facenet, ArcFace, and VGG-Face. We also perform experiments by augmenting the train and test datasets with images rotated in-plane to further demonstrate the efficiency of the proposed approach. When CycleGAN (another unpaired image translation network) is used to generate images before incorporating the alignment loss, it failed to preserve the alignment at the slightest, therefore the eye center detection accuracy was extremely low. With the alignment loss, the accuracy increased by 20%, 50%, and 80% when the normalized error (e) ≤ 0.05, 0.10 and 0.25 respectively.

A novel method based on thermal image to predict the personal thermal comfort in the vehicle

A passenger-centered Heating, Ventilation, and Air Conditioning (HVAC) system is an urgent need with the innovative development of intelligent vehicles. A prerequisite for such a system is precise real-time estimation of human thermal comfort. This paper introduces a new method based on deep learning to predict the personal thermal comfort in the vehicle via facial thermal image. Specifically, a deep learning-based ResNet34 coupled with a spatial attention mechanism (SAM-ResNet34) is designed for feature extraction from the facial thermal image. Unlike the existing methods, this method extracts the gray features from the different areas of facial thermal images instead of measuring the facial skin temperature to predict the thermal comfort states. The thermal comfort data were collected from the experiment with 22 subjects. It has highlighted that the model trained with the thermal image dataset can achieve an accuracy as high as 93.75% in the test set. The result suggests that the non-invasive method proposed in this study could accurately predict personal thermal comfort in the vehicular environment and holds great potential to be applied to the HVAC control system of a vehicle in the future.

Analysis of Facial Occlusion Challenge in Thermal Images for Human Affective State Recognition.

Several studies have been conducted using both visual and thermal facial images to identify human affective states. Despite the advantages of thermal facial images in recognizing spontaneous human affects, few studies have focused on facial occlusion challenges in thermal images, particularly eyeglasses and facial hair occlusion. As a result, three classification models are proposed in this paper to address the problem of thermal occlusion in facial images, with six basic spontaneous emotions being classified. The first proposed model in this paper is based on six main facial regions, including the forehead, tip of the nose, cheeks, mouth, and chin. The second model deconstructs the six main facial regions into multiple subregions to investigate the efficacy of subregions in recognizing the human affective state. The third proposed model in this paper uses selected facial subregions, free of eyeglasses and facial hair (beard, mustaches). Nine statistical features on apex and onset thermal images are implemented. Furthermore, four feature selection techniques with two classification algorithms are proposed for a further investigation. According to the comparative analysis presented in this paper, the results obtained from the three proposed modalities were promising and comparable to those of other studies.

Human Face Emotions Recognition from Thermal Images Using DenseNet

In the current scenario face identification and recognition is an important technique in surveillance. The face is a necessary biometric in humans. Therefore face detection plays a major job in computer vision applications. Several face recognition and emotions classification approaches have been presented throughout the last few decades of research to improve the rate of face recognition for thermal pictures. However, in real-time, lighting conditions might change due to several factors, such as the different times of capture, weather, etc. Due to variations in lighting intensity, the performance of the facial expression recognition system is not good. This paper proposed a model for human thermal face detection and expression classification. Four main steps were involved in this research. Initially, the Difference of the Gaussian (DOG) filter is utilized to crop the input thermal images and then normalize the images using the median filter in pre-processing step. Then, Efficient Net is used for extracting features such as shape, location, and occurrences from thermal face images. After that, detect human faces utilized by the YOLOv4 technique to better emotions classification. Finally, classify the emotions on faces by using the DenseNet technique into seven emotions such as happy, sad, disgust, surprise, anger, fear, and neutral. The proposed method outperforms state-of-art techniques for face recognition on thermal pictures, and classifies the expressions, according to experimentations on the RGB-D-T database. The accuracy, precision, recall, and f1-score metrics will be utilized with the database to assess the efficacy of the proposed methodology. The proposed models achieve a high classification accuracy of 95.97% on the RGB-D-T database. Furthermore, the outcomes show good precision for various face recognition tasks.

Potential of using facial thermal imaging in patient triage of flu-like syndrome during the COVID-19 pandemic crisis.

The screening of flu-like syndrome is difficult due to nonspecific symptoms or even oligosymptomatic presentation and became even more complex during the Covid-19 pandemic. However, an efficient screening tool plays an important role in the control of highly contagious diseases, allowing more efficient medical-epidemiological approaches and rational management of global health resources. Infrared thermography is a technique sensitive to small alterations in the skin temperature which may be related to early signs of inflammation and thus being relevant in the detection of infectious diseases. Thus, the objective of this study was to evaluate the potential of facial thermal profiles as a risk evaluator of symptoms and signs of SARs diseases, using COVID-19 as background disease. A total of 136 patients were inquired about the most common symptoms of COVID-19 infection and were submitted to an infrared image scanning, where the temperatures of 10 parameters from different regions of the face were captured. We used RT-qPCR as the ground truth to compare with the thermal parameters, in order to evaluate the performance of infrared imaging in COVID-19 screening. Only 16% of infected patients had fever at the hospital admission, and most infrared thermal variables presented values of temperature significantly higher in infected patients. The maximum eye temperature (MaxE) showed the highest predictive value at a cut-off of >35.9°C (sn = 71.87%, sp = 86.11%, LR+ = 5.18, LR- = 0.33, AUC = 0.850, p < 0.001). Our predictive model reached an accuracy of 86% for disease detection, indicating that facial infrared thermal scanning, based on the combination of different facial regions and the thermal profile of the face, has potential to act as a more accurate diagnostic support method for early COVID-19 screening, when compared to classical infrared methods, based on a single spot with the maximum skin temperature of the face.

Mask Detection and Classification in Thermal Face Images

Face masks are recommended to reduce the transmission of many viruses, especially SARS-CoV-2. Therefore, the automatic detection of whether there is a mask on the face, what type of mask is worn, and how it is worn is an important research topic. In this work, the use of thermal imaging was considered to analyze the possibility of detecting (localizing) a mask on the face, as well as to check whether it is possible to classify the type of mask on the face. The previously proposed dataset of thermal images was extended and annotated with the description of a type of mask and a location of a mask within a face. Different deep learning models were adapted. The best model for face mask detection turned out to be the Yolov5 model in the "nano" version, reaching mAP higher than 97% and precision of about 95%. High accuracy was also obtained for mask type classification. The best results were obtained for the convolutional neural network model built on an autoencoder initially trained in the thermal image reconstruction problem. The pretrained encoder was used to train a classifier which achieved an accuracy of 91%.

Biometric authentication based with thermal facial images based on biometrics to code converters

Biometric authentication based with thermal facial images based on biometrics to code converters

Learning Domain and Pose Invariance for Thermal-to-Visible Face Recognition

Interest in thermal to visible face recognition has grown significantly over the last decade due to advancements in thermal infrared cameras and analytics beyond the visible spectrum. Despite large discrepancies between thermal and visible spectra, existing approaches bridge domain gaps by either synthesizing visible faces from thermal faces or by learning the cross-spectrum image representations. These approaches typically work well with frontal facial imagery collected at varying ranges and expressions, but exhibit significantly reduced performance when matching thermal faces with varying poses to frontal visible faces. We propose a novel Domain and Pose Invariant Framework that simultaneously learns domain and pose invariant representations. Our proposed framework is composed of modified networks for extracting the most correlated intermediate representations from off-pose thermal and frontal visible face imagery, a sub-network to jointly bridge domain and pose gaps, and a joint-loss function comprised of cross-spectrum and pose-correction losses. We demonstrate efficacy and advantages of the proposed method by evaluating on three thermal-visible datasets: ARL Visible-to-Thermal Face, ARL Multimodal Face, and Tufts Face. Although DPIF focuses on learning to match off-pose thermal to frontal visible faces, we also show that DPIF enhances performance when matching frontal thermal face images to frontal visible face images.