Fusion Of Thermal Images Research Articles

Fusion of visible and thermal images for facial expression recognition

Most present research into facial expression recognition focuses on the visible spectrum, which is sensitive to illumination change. In this paper, we focus on integrating thermal infrared data with visible spectrum images for spontaneous facial expression recognition. First, the active appearance model AAM parameters and three defined head motion features are extracted from visible spectrum images, and several thermal statistical features are extracted from infrared (IR) images. Second, feature selection is performed using the F-test statistic. Third, Bayesian networks BNs and support vector machines SVMs are proposed for both decision-level and feature-level fusion. Experiments on the natural visible and infrared facial expression (NVIE) spontaneous database show the effectiveness of the proposed methods, and demonstrate thermal IR images’ supplementary role for visible facial expression recognition.

Finite asymmetric generalized Gaussian mixture models learning for infrared object detection

The interest in automatic surveillance and monitoring systems has been growing over the last years due to increasing demands for security and law enforcement applications. Although, automatic surveillance systems have reached a significant level of maturity with some practical success, it still remains a challenging problem due to large variation in illumination conditions. Recognition based only on the visual spectrum remains limited in uncontrolled operating environments such as outdoor situations and low illumination conditions. In the last years, as a result of the development of low-cost infrared cameras, night vision systems have gained more and more interest, making infrared (IR) imagery as a viable alternative to visible imaging in the search for a robust and practical identification system. Recently, some researchers have proposed the fusion of data recorded by an IR sensor and a visible camera in order to produce information otherwise not obtainable by viewing the sensor outputs separately. In this article, we propose the application of finite mixtures of multidimensional asymmetric generalized Gaussian distributions for different challenging tasks involving IR images. The advantage of the considered model is that it has the required flexibility to fit different shapes of observed non-Gaussian and asymmetric data. In particular, we present a highly efficient expectation–maximization (EM) algorithm, based on minimum message length (MML) formulation, for the unsupervised learning of the proposed model’s parameters. In addition, we study its performance in two interesting applications namely pedestrian detection and multiple target tracking. Furthermore, we examine whether fusion of visual and thermal images can increase the overall performance of surveillance systems.

Application of G100/120 thermal imaging camera in energy efficiency measuring in building construction

A measurement of energy efficiency in the construction industry aims to reduce permanently energy requirements in design, construction and use of new buildings, sannation and reconstruction of the existing ones. Over the long term, with the expected increasing in the price of energy and the development of awareness about energy conservation and environmental protection, thermal imaging methods will certainly find their wide application in the construction industry. Infrared thermal camera is used to estimate the temperature. However, we do not see all the details of interest on the thermal image, therefore we suggest using the fusion of visual and thermal images of the same part of the building recorded simultaneously. The analysis images of the object obtained by fusion of television and infrared thermal imaging shows the areas of interest for further processing.

Multi-focus thermal image fusion

This paper proposes a novel algorithm for multi-focus thermal image fusion. The algorithm is based on local activity analysis and advanced pre-selection of images into fusion process. The algorithm improves the object temperature measurement error up to 5°C. The proposed algorithm is evaluated by half total error rate, root mean squared error, cross correlation and visual inspection. To the best of our knowledge, this is the first work devoted to multi-focus thermal image fusion. For testing of proposed algorithm we acquire six thermal image set with objects at different focal depth.

Image fusion of visible and thermal images for fruit detection

Image fusion is the combination of two or more images of a scene to improve visual perception or feature extraction. A thermal image and a visible image of an orange canopy scene were fused to improve fruit detection. Visible images are formed by reflection in the visible spectrum while thermal images are created from thermal radiation. A digital colour camera captured the visible source image and a thermal infrared camera acquired the thermal source image. Because the scene was acquired by two different cameras with different fields of view and spatial resolutions, image registration was performed prior to image fusion. Two image fusion approaches were applied, Laplacian pyramid transform (LPT) and fuzzy logic. Results showed that both image fusion methods improved fruit detection when compared to using the thermal image alone. Based on image fusion evaluation indices, the fuzzy logic approach performed better than the LPT.

A Computer Tool for the Fusion and Visualization of Thermal and Magnetic Resonance Images

The measurement of temperature variation along the surface of the body, provided by digital infrared thermal imaging (DITI), is becoming a valuable auxiliary tool for the early detection of many diseases in medicine. However, DITI is essentially a 2-D technique and its image does not provide useful anatomical information associated with it. However, multimodal image registration and fusion may overcome this difficulty and provide additional information for diagnosis purposes. In this paper, a new method of registering and merging 2-D DITI and 3-D MRI is presented. Registration of the images acquired from the two modalities is necessary as they are acquired with different image systems. Firstly, the body volume of interest is scanned by a MRI system and a set of 2-D DITI of it, at orthogonal angles, is acquired. Next, it is necessary to register these two different sets of images. This is done by creating 2-D MRI projections from the reconstructed 3-D MRI volume and registering it with the DITI. Once registered, the DITI is then projected over the 3-D MRI. The program developed to assess the proposed method to combine MRI and DITI resulted in a new tool for fusing two different image modalities, and it can help medical doctors.

Multiscale Fusion of Visible and Thermal IR Images for Illumination-Invariant Face Recognition

This paper describes a new software-based registration and fusion of visible and thermal infrared (IR) image data for face recognition in challenging operating environments that involve illumination variations. The combined use of visible and thermal IR imaging sensors offers a viable means for improving the performance of face recognition techniques based on a single imaging modality. Despite successes in indoor access control applications, imaging in the visible spectrum demonstrates difficulties in recognizing the faces in varying illumination conditions. Thermal IR sensors measure energy radiations from the object, which is less sensitive to illumination changes, and are even operable in darkness. However, thermal images do not provide high-resolution data. Data fusion of visible and thermal images can produce face images robust to illumination variations. However, thermal face images with eyeglasses may fail to provide useful information around the eyes since glass blocks a large portion of thermal energy. In this paper, eyeglass regions are detected using an ellipse fitting method, and replaced with eye template patterns to preserve the details useful for face recognition in the fused image. Software registration of images replaces a special-purpose imaging sensor assembly and produces co-registered image pairs at a reasonable cost for large-scale deployment. Face recognition techniques using visible, thermal IR, and data-fused visible-thermal images are compared using a commercial face recognition software (FaceIt®) and two visible-thermal face image databases (the NIST/Equinox and the UTK-IRIS databases). The proposed multiscale data-fusion technique improved the recognition accuracy under a wide range of illumination changes. Experimental results showed that the eyeglass replacement increased the number of correct first match subjects by 85% (NIST/Equinox) and 67% (UTK-IRIS).

Visibility of road hazards in thermal, visible, and sensor-fused night-time imagery

Sensor fusion combines the output of multiple imaging sensors within a single composite display. Ideally, a fused image will retain important spatial information provided by individual input images, and will convey useful spatial or chromatic emergent information derived from the contrast between input images. The present experiment assessed the potential benefits of sensor fusion as a method of enhancing drivers’ night-time detection of road hazards. Observers were asked to detect a pedestrian within thermal and visible images of a night-time scene, and within chromatic and achromatic renderings created by sensor fusion of grayscale thermal and visible images. Results indicated that fusion can both improve spatial image content, and can effectively embellish spatial content with emergent chromatic information. The benefits of both sensor fusion and of color rendering, however, were inconsistent, varying substantially with quality of input images submitted for fusion.