Thermal Infrared Face Research Articles

Dual Modal Face Recognition Based on Visible Light and Thermal Infrared

Abstract In recent years, face recognition has received widespread attention, in the use of visible light for face recognition, there are a series of problems that require auxiliary light sources, may be attacked by photos, and camouflage is difficult to penetrate. Compared with visible light recognition, thermal infrared face recognition shows usability, and it can still be stable in low light or dark extreme environmental conditions, but due to low resolution, it can not capture rich face feature information. In view of this paper proposes that dual mode face recognition can effectively integrate two single mode information and make up for the deficiency of single feature. Using a Bimodal Face network model based on visible light and thermal infrared, the idea of feature decoupling is introduced into image fusion. The cross-modal information is decomposed into common information and unique information, and the common information and unique information of different modes are fused respectively according to the feature fusion method. It is tested on the self-built dual modal dataset that this fusion strategy can achieve the optimal effect, and the accuracy rate can reach 97.08%, which is better than other five face recognition network models.

Application of Visual Transformer in Low-resolution Thermal Infrared Image Recognition

Abstract Addressing the challenges of inadequate accuracy and limited robustness exhibited by current lightweight object detection networks specifically tailored for low-resolution thermal infrared face detection scenarios, this paper delves into developing an ultra-lightweight thermal infrared face detection algorithm that leverages visual attention mechanisms. To ascertain the optimal neural network complexity, a series of comparative experiments are meticulously conducted. With Yolo-FastestDet serving as the benchmark, this study endeavors to compress the backbone network, striking a delicate balance between network depth and detection speed. Additionally, to bolster the network’s capacity for profound feature extraction and precise discrimination of target edges and small objects, a Transformer-Encoder-based visual attention module is seamlessly integrated. Consequently, a lightweight face detection algorithm, enriched with attention mechanisms, is formulated. Furthermore, to mitigate the scarcity of low-resolution infrared face image data, a self-constructed visible-infrared face dataset is employed for training and evaluation purposes. The experimental outcomes reveal that the proposed algorithm attains an impressive mAP@0.5 score of 0.953 on the test dataset while satisfying the stringent real-time detection criterion of 30 frames per second (FPS) when deployed on an embedded Raspberry Pi CPU.

FeverNet: Enabling accurate and robust remote fever screening

Remote human fever screening via thermal infrared imaging helps reduce the risk of respiratory disease transmission and plays an important role in public health monitoring. However, the accuracy of such systems often falls prey to variations in measurement distance and environment temperature. Most previous methods tend to employ sensors to overcome these variations, which are expensive schemes and have limited performance improvement. To address above problems, this paper presents a novel and robust remote fever screening framework named FeverNet. Specifically, FeverNet introduces depth estimation network and temperature distribution constraints across time periods to reduce the influence of distance variations and environment temperature changes. The fever attention module is thus proposed to enhance feature representation and expand the difference between fever faces and normal ones. In addition, we provide the Extended Thermal Infrared Face dataset (ETIF), which further gives visible images (paired with thermal infrared images) for depth estimation and improve the fever face generated method based on the maximum temperature of the face. Extensive experiments on ETIF demonstrate the advantages of our FeverNet over the state-of-the-art methods.

Dynamic Group Difference Coding Based on Thermal Infrared Face Image for Fever Screening

Recently, noncontact temperature measurement methods based on infrared face perception have received widely attentions since fever screening plays an important role in the early prediction of respiratory infections, such as SARS, H1N1, and COVID-19. However, the performance of these methods always significantly degrades when facing the changes of environment. Thus, the majority of these methods leverage the block-body and sensors to reduce the influence of environment changes. It is a pity that the increased instrument complexity leads to higher costs and failure rate. To address the aforementioned issues, this article presents a novel fever screening method, named dynamic group difference coding (DGDC), which is based on the analysis about the influencing factors. The key idea of DGDC is to compute the temperature differences between the target person and the recently passed crowd (dynamic group). Specifically, we develop the face temperature encoder (FTE) to describe the face temperature and thus construct the difference matrix of the embedding feature between the target person and the dynamic group. Multilayer perceptions (MLP) are employed to capture the intrinsic information by characterizing the difference matrix in vertical and horizontal directions, respectively. Finally, we provide a dataset of thermal infrared face (TIF) images and conduct extensive experiments to demonstrate the advantages of the proposed method over the competing methods.

Charlotte-ThermalFace: A Fully Annotated Thermal Infrared Face Dataset with Various Environmental Conditions and Distances

Facial thermography is one of the most popular research areas in infrared thermal imaging, with diverse applications in medical, surveillance, and environmental monitoring. However, in contrast to facial imagery in the visual spectrum, the lack of public datasets on facial thermal images is an obstacle to research improvement in this area. Thermal face imagery is still a relatively new research area to be evaluated and studied in different domains.The current thermal face datasets are limited in regards to the subjects’ distance from the camera, the ambient temperature variation, and facial landmarks’ localization. We address these gaps by presenting a new facial thermography dataset.This article makes two main contributions to the body of knowledge. First, it presents a comprehensive review and comparison of current public datasets in facial thermography. Second, it introduces and studies a novel public dataset on facial thermography, which we call it Charlotte-ThermalFace. Charlotte-ThermalFace contains more than10000 infrared thermal images in varying thermal conditions, several distances from the camera, and different head positions. The data is fully annotated with the facial landmarks, ambient temperature, relative humidity, the air speed of the room, distance to the camera, and subject thermal sensation at the time of capturing each image. Our dataset is the first publicly available thermal dataset annotated with the thermal sensation of each subject in different thermal conditions and one of the few datasets in raw 16-bit format. Finally, we present a preliminary analysis of the dataset to show the applicability and importance of the thermal conditions in facial thermography. The full dataset, including annotations, are freely available for research purpose at https://github.com/TeCSAR-UNCC/UNCC-ThermalFace.

Thermal Infrared Face Recognition.

The technology for deep learning in the field of thermal infrared face recognition has recently become more available for use in research, therefore allowing for the many groups working on this subject to achieve many novel findings. Thermal infrared face recognition helps recognize faces that are not able to be recognized in visible light and can additionally recognize facial blood vessel structure. Previous research regarding temperature variations, mathematical formulas, wave types, and methods in thermal infrared face recognition is reviewed.

Convolutional Neural Networks for Semantic Segmentation as a Tool for Multiclass Face Analysis in Thermal Infrared

Convolutional neural networks were used for multiclass segmentation in thermal infrared face analysis. The principle is based on existing image-to-image translation approaches, where each pixel in an image is assigned to a class label. We show that established networks architectures can be trained for the task of multiclass face analysis in thermal infrared. Created class annotations consisted of pixel-accurate locations of different face classes. Subsequently, the trained network can segment an acquired unknown infrared face image into the defined classes. Furthermore, face classification in live image acquisition is shown, in order to be able to display the relative temperature in real-time from the learned areas. This allows a pixel-accurate temperature face analysis e.g. for infection detection like Covid-19. At the same time our approach offers the advantage of concentrating on the relevant areas of the face. Areas of the face irrelevant for the relative temperature calculation or accessories such as glasses, masks and jewelry are not considered. A custom database was created to train the network. The results were quantitatively evaluated with the intersection over union (IoU) metric. The methodology shown can be transferred to similar problems for more quantitative thermography tasks like in materials characterization or quality control in production.

Thermal face segmentation based on circular shortest path

Recently, research on thermal infrared human face analysis grows rapidly. However, due to the low contrast and limited information, human face in thermal infrared images is difficult to segment precisely. To overcome these shortcomings of thermal images, we propose an improved circular shortest path method in this paper. In our method, the proposed gradient based cost function enhances the gradient information and extracts detail of the original image. In addition, we propose a shape constraint by using ellipse derivative in the cost function. The constraint helps the contour to conform to the real human face. Moreover, the proposed certainty penalty term and straight path penalty term restrain the effect of the local minima regions and improve the robustness of our method. Our method could effectively extract the precise human face contour and thus segment the complete human face. Experimental results show that our method performs well for thermal infrared face segmentation, in both visual and quantitative ways.

Regional parallel structure based CNN for thermal infrared face identification

Regional parallel structure based CNN for thermal infrared face identification

Composite multilobe descriptors for cross-spectral recognition of full and partial face

Cross-spectral image matching is a challenging research problem motivated by various applications, including surveillance, security, and identity management in general. An example of this problem includes cross-spectral matching of active infrared (IR) or thermal IR face images against a dataset of visible light images. A summary of recent developments in the field of cross-spectral face recognition by the authors is presented. In particular, it describes the original form and two variants of a local operator named composite multilobe descriptor (CMLD) for facial feature extraction with the purpose of cross-spectral matching of near-IR, short-wave IR, mid-wave IR, and long-wave IR to a gallery of visible light images. The experiments demonstrate that the variants of CMLD outperform the original CMLD and other recently developed composite operators used for comparison. In addition to different IR spectra, various standoff distances from close-up (1.5 m) to intermediate (50 m) and long (106 m) are also investigated. Performance of CMLD I to III is evaluated for each of the three cases of distances. The newly developed operators, CMLD I to III, are further utilized to conduct a study on cross-spectral partial face recognition where different facial regions are compared in terms of the amount of useful information they contain for the purpose of conducting cross-spectral face recognition. The experimental results show that among three facial regions considered in the experiments the eye region is the most informative for all IR spectra at all standoff distances.

Study of Local Matching-Based Facial Recognition Methods Using Thermal Infrared Imagery

This paper shows a comparative study among different local matching-based methods for thermal infrared face recognition. The principal assumption of this work is that the thermal face corresponds to the diffuse energy emission captured by an infrared camera, where the thermal signature is unique for each subject and it can be addressed as a texture descriptor with thermal images. Local matching-based methods find inter-class differences that improve the face recognition rate in thermal spectrum. Specifically, this work considers four methods: Local Binary Pattern (LBP), Local Derivative Pattern (LDP), Weber Linear Descriptor (WLD) and Histograms of Oriented Gradients Descriptors (HOG). The methods are evaluated and compared using the UCHThermalFace database, that considers real-world conditions and unconstrained environments, such as indoor and outdoor setups, natural variations in illumination, facial expression, pose, accessories, occlusions, and background. Results indicate that HOG variants followed by LBP method achieved the best recognition rates for face recognition systems.

Good match exploration for thermal infrared face recognition based on YWF-SIFT with multi-scale fusion

Stable local feature detection is a critical prerequisite in the problem of infrared (IR) face recognition. Recently, Scale Invariant Feature Transform (SIFT) is introduced for feature detection in an infrared face frame, which is achieved by applying a simple and effective averaging window with SIFT termed as Y-styled Window Filter (YWF). However, the thermal IR face frame has an intrinsic characteristic such as lack of feature points (keypoints); therefore, the performance of the YWF-SIFT method will be inevitably influenced when it was used for IR face recognition. In this paper, we propose a novel method combining multi-scale fusion with YWF-SIFT to explore more good feature matches. The multi-scale fusion is performed on a thermal IR frame and a corresponding auxiliary visual frame generated from an off-the-shelf low-cost visual camera. The fused image is more informative, and typically contains much more stable features. Besides, the use of YWF-SIFT method enables us to establish feature correspondences more accurately. Quantitative experimental results demonstrate that our algorithm is able to significantly improve the quantity of feature points by approximately 38%. As a result, the performance of YWF-SIFT with multi-scale fusion is enhanced about 12% in infrared human face recognition.

TIR/VIS cross-modality modelling via correlative subspace learning

A novel TIR/VIS cross-modality model is presented. It has a live signature and is illumination invariant for the thermal infrared spectrum. It can be used to solve several difficult problems in face biometrics, e.g. face recognition under weak illumination while lacking a thermal infrared face template of enrollment, liveness assurance against photograph and video face spoofing, face disguise identification, etc. In the presented approach, correlative subspace for TIR/VIS cross-modality is learned by canonical correlation analysis and correlation coefficients between TIR and VIS face variables on different face patches are presented by leave-one-out cross-validation. The experiment results show that it is effective to model TIR/VIS cross-modality using canonical correlation analysis via patch correlation coefficient based weighting.

Recent advances in visual and infrared face recognition—a review

Face recognition is a rapidly growing research area due to increasing demands for security in commercial and law enforcement applications. This paper provides an up-to-date review of research efforts in face recognition techniques based on two-dimensional (2D) images in the visual and infrared (IR) spectra. Face recognition systems based on visual images have reached a significant level of maturity with some practical success. However, the performance of visual face recognition may degrade under poor illumination conditions or for subjects of various skin colors. IR imagery represents a viable alternative to visible imaging in the search for a robust and practical identification system. While visual face recognition systems perform relatively reliably under controlled illumination conditions, thermal IR face recognition systems are advantageous when there is no control over illumination or for detecting disguised faces. Face recognition using 3D images is another active area of face recognition, which provides robust face recognition with changes in pose. Recent research has also demonstrated that the fusion of different imaging modalities and spectral components can improve the overall performance of face recognition.