Infrared Video Research Articles

An improved object tracking algorithm based on adaptive weighted strategy and occlusion detection mechanism

Due to complex background and volatile object shape-appearance in image, the stability and accuracy of tracking algorithm is often disturbed and reduced. So how to accurately and robustly track object in object tracking application is a challenge topic at home and abroad. Built upon the methodologies of compressive tracking and spatio-temporal context, a simple yet robust object tracking method is proposed for solving the drift and occlusion problems in paper. It combines two existing classical ideas into a single framework: adaptive weighted idea and occlusion detection mechanism. In order to weaken interference problems of object background, object area is firstly partitioned into equal-sized sub-patches and the different weight related with location information is assigned for each patch; Then, for improving its robustness, Bhattacharyya distance is adopted to find out these samples with maximum discrimination; In addition, our proposed occlusion detection mechanism is for recapturing the tracked object when occlusion occurs. Many simulation experiments show that our proposed algorithm achieves more favorable performance than these existing state-of-the-art algorithms in handing various challenging infrared videos, especially occlusion and shape deformation.

Thermal Infrared Video Stabilization for Aerial Monitoring of Active Wildfires

Measuring wildland fire behavior is essential for fire science and fire management. Aerial thermal infrared (TIR) imaging provides outstanding opportunities to acquire such information remotely. Variables such as fire rate of spread (ROS), fire radiative power (FRP), and fireline intensity may be measured explicitly both in time and space, providing the necessary data to study the response of fire behavior to weather, vegetation, topography, and firefighting efforts. However, raw TIR imagery acquired by unmanned aerial vehicles (UAVs) requires stabilization and georeferencing before any other processing can be performed. Aerial video usually suffers from instabilities produced by sensor movement. This problem is especially acute near an active wildfire due to fire-generated turbulence. Furthermore, the nature of fire TIR video presents some specific challenges that hinder robust interframe registration. Therefore, this article presents a software-based video stabilization algorithm specifically designed for TIR imagery of forest fires. After a comparative analysis of existing image registration algorithms, the KAZE feature-matching method was selected and accompanied by pre- and postprocessing modules. These included foreground histogram equalization and a multireference framework designed to increase the algorithm's robustness in the presence of missing or faulty frames. The performance of the proposed algorithm was validated in a total of nine video sequences acquired during field fire experiments. The proposed algorithm yielded a registration accuracy between 10 and 1000× higher than other tested methods, returned 10× more meaningful feature matches, and proved robust in the presence of faulty video frames. The ability to automatically cancel camera movement for every frame in a video sequence solves a key limitation in data processing pipelines and opens the door to a number of systematic fire behavior experimental analyses. Moreover, a completely automated process supports the development of decision support tools that can operate in real time during an emergency.

Enhancing Small Moving Target Detection Performance in Low-Quality and Long-Range Infrared Videos Using Optical Flow Techniques

The detection of small moving objects in long-range infrared videos is challenging due to background clutter, air turbulence, and small target size. In this paper, we summarize the investigation of efficient ways to enhance the performance of small target detection in long-range and low-quality infrared videos containing moving objects. In particular, we focus on unsupervised, modular, flexible, and efficient methods for target detection performance enhancement using motion information extracted from optical flow methods. Three well-known optical flow methods were studied. It was found that optical flow methods need to be combined with contrast enhancement, connected component analysis, and target association in order to be effective for target detection. Extensive experiments using long-range mid-wave infrared (MWIR) videos from the Defense Systems Information Analysis Center (DSIAC) dataset clearly demonstrated the efficacy of our proposed approach.

Edge and Corner Awareness-Based Spatial–Temporal Tensor Model for Infrared Small-Target Detection

Infrared (IR) small-target detection has been a widely studied task in IR search and tracking systems. It remains a challenging problem, especially in heterogeneous scenarios, where it is very difficult to discriminate true targets from sparse residuals in the background. A novel edge and corner awareness-based spatial–temporal tensor (ECA-STT) model is presented in this article. First, we construct an STT based on a spatial–temporal correlation analysis of the IR video background. Then, we propose an indicator to highlight the target through adjustable importance measurements of the edge and corner. The tensor-based nonlocal total variation is also adopted to describe the edges in the background. The target–background separation problem is modeled as a tensor robust principal component analysis (TRPCA) problem with the tensor rank function replaced by the tensor truncated nuclear norm. The proposed model is solved by an effective optimization algorithm derived from the alternating direction method of multipliers (ADMM). Extensive experiments verify the superior abilities of the proposed model in target enhancement and background suppression.

Automated video-based heart rate tracking for the anesthetized and behaving monkey

Heart rate (HR) is extremely valuable in the study of complex behaviours and their physiological correlates in non-human primates. However, collecting this information is often challenging, involving either invasive implants or tedious behavioural training. In the present study, we implement a Eulerian video magnification (EVM) heart tracking method in the macaque monkey combined with wavelet transform. This is based on a measure of image to image fluctuations in skin reflectance due to changes in blood influx. We show a strong temporal coherence and amplitude match between EVM-based heart tracking and ground truth ECG, from both color (RGB) and infrared (IR) videos, in anesthetized macaques, to a level comparable to what can be achieved in humans. We further show that this method allows to identify consistent HR changes following the presentation of conspecific emotional voices or faces. EVM is used to extract HR in humans but has never been applied to non-human primates. Video photoplethysmography allows to extract awake macaques HR from RGB videos. In contrast, our method allows to extract awake macaques HR from both RGB and IR videos and is particularly resilient to the head motion that can be observed in awake behaving monkeys. Overall, we believe that this method can be generalized as a tool to track HR of the awake behaving monkey, for ethological, behavioural, neuroscience or welfare purposes.

Extraction of foreground area of pedestrian objects under thermal infrared video surveillance

Extraction of foreground area of pedestrian objects under thermal infrared video surveillance

A New Roadway Eventual Obstacle Detection System Based on Computer Vision.

A new roadway eventual obstacle detection system based on computer vision is described and evaluated. This system uses low-cost hardware and open-source software to detect and classify moving elements in roads using infra-red and colour video images as input data. This solution represents an important advancement to prevent road accidents due to eventual obstacles which have considerably increased in the past decades, mainly with wildlife. The experimental evaluation of the system demonstrated that the proposed solution detects and classifies correctly different types of moving obstacles on roads, working robustly under different weather and illumination conditions.

Non-Contact Assessment of Peripheral Artery Haemodynamics Using Infrared Video Thermography

Skin temperature has long been used as a natural indicator of vascular diseases in the extremities. Considerable correlation between oscillations in skin surface temperature and oscillations of skin blood flow has previously been demonstrated. We hypothesised that the impairment of blood flow in stenotic (subcutaneous) peripheral arteries would influence cutaneous temperature such that, by measuring gradients in the temperature distribution over skin surfaces, one may be able to diagnose or quantify the progression of vascular conditions in whose pathogenesis a reduction in subcutaneous blood perfusion plays a critical role (e.g. peripheral artery disease). As proof of principle, this study investigates the local changes in the skin temperature of healthy humans (15 male, [Formula: see text] years old, BMI [Formula: see text]kg/m 2) undergoing two physical challenges designed to vary their haemodynamic status. Skin temperature was measured in four central regions (forehead, neck, chest, and left shoulder) and four peripheral regions (left upper arm, forearm, wrist, and hand) using an infrared thermal camera. We compare inter-region patterns. Median temperature over the peripheral regions decreased from baseline after both challenges (maximum decrease: [Formula: see text]°C at 60s after exercise; [Formula: see text] and [Formula: see text]°C at 180s of cold-water immersion; [Formula: see text]). Median temperature over the central regions showed no significant changes. Our results show that the non-contact measurement of perfusion-related changes in peripheral temperature from infrared video data is feasible. Further research will be directed towards the thermographic study of patients with symptomatic peripheral vascular disease.

Detection and Confirmation of Multiple Human Targets Using Pixel-Wise Code Aperture Measurements

Compressive video measurements can save bandwidth and data storage. However, conventional approaches to target detection require the compressive measurements to be reconstructed before any detectors are applied. This is not only time consuming but also may lose information in the reconstruction process. In this paper, we summarized the application of a recent approach to vehicle detection and classification directly in the compressive measurement domain to human targets. The raw videos were collected using a pixel-wise code exposure (PCE) camera, which condensed multiple frames into one frame. A combination of two deep learning-based algorithms (you only look once (YOLO) and residual network (ResNet)) was used for detection and confirmation. Optical and mid-wave infrared (MWIR) videos from a well-known database (SENSIAC) were used in our experiments. Extensive experiments demonstrated that the proposed framework was feasible for target detection up to 1500 m, but target confirmation needs more research.

Distinguishing Obstructive Versus Central Apneas in Infrared Video of Sleep Using Deep Learning: Validation Study

BackgroundSleep apnea is a respiratory disorder characterized by an intermittent reduction (hypopnea) or cessation (apnea) of breathing during sleep. Depending on the presence of a breathing effort, sleep apnea is divided into obstructive sleep apnea (OSA) and central sleep apnea (CSA) based on the different pathologies involved. If the majority of apneas in a person are obstructive, they will be diagnosed as OSA or otherwise as CSA. In addition, as it is challenging and highly controversial to divide hypopneas into central or obstructive, the decision about sleep apnea type (OSA vs CSA) is made based on apneas only. Choosing the appropriate treatment relies on distinguishing between obstructive apnea (OA) and central apnea (CA).ObjectiveThe objective of this study was to develop a noncontact method to distinguish between OAs and CAs.MethodsFive different computer vision-based algorithms were used to process infrared (IR) video data to track and analyze body movements to differentiate different types of apnea (OA vs CA). In the first two methods, supervised classifiers were trained to process optical flow information. In the remaining three methods, a convolutional neural network (CNN) was designed to extract distinctive features from optical flow and to distinguish OA from CA.ResultsOvernight sleeping data of 42 participants (mean age 53, SD 15 years; mean BMI 30, SD 7 kg/m2; 27 men and 15 women; mean number of OA 16, SD 30; mean number of CA 3, SD 7; mean apnea-hypopnea index 27, SD 31 events/hour; mean sleep duration 5 hours, SD 1 hour) were collected for this study. The test and train data were recorded in two separate laboratory rooms. The best-performing model (3D-CNN) obtained 95% accuracy and an F1 score of 89% in differentiating OA vs CA.ConclusionsIn this study, the first vision-based method was developed that differentiates apnea types (OA vs CA). The developed algorithm tracks and analyses chest and abdominal movements captured via an IR video camera. Unlike previously developed approaches, this method does not require any attachment to a user that could potentially alter the sleeping condition.

Male European badger churrs: insights into call function and motivational basis

Determining the contexts of emission and information content of vocal signals can yield insights into the function of different call types, and remains an important step towards understanding the diversification of mammalian vocal repertoires. In this study, we used infra-red video cameras and remote audio recorders to document seasonal and contextual variation in male European badger (Meles meles) churr production over a 24-month period, and acoustic analysis based on source-filter theory to examine whether churr acoustic structure varies according to the caller’s arousal state and identity. Our behavioural observations revealed that male churrs are produced almost exclusively during the breeding season. Further contextual analysis showed that males emit churrs during close-range interactions with female conspecifics, often during copulation attempts, and churr directly into sett entrances. In addition, males involved in close-range social interactions delivered churrs with more call units per second than those vocalising without other conspecifics in close proximity. Discriminant function analysis also revealed that male churrs are individually distinctive, and confirmed that the formants (vocal tract resonances) contribute the most to caller identity. These findings indicate that badger churrs are sexual calls with the potential to signal male arousal state and identity in reproductive contexts. They also add to an increasing body of literature on the importance of formants for identity cueing in nonhuman mammals.

Non-Destructive Investigation of Dispersion, Bonding, and Thermal Properties of Emerging Polymer Nanocomposites Using Close-Up Lens Assisted Infrared Thermography

AbstractPolymer nanocomposites possess unique sets of properties that make them suitable for different applications, including structural and flame-retardant material, electromagnetic wave reflector, sensors, thin film transistor, flexible display, and many more. The properties of these nanocomposite are dependent on nanofiller dispersion and bonding with polymer matrix (i.e. particle-matrix interaction). Thermography is a non-destructive method that may be used to gain insight into dispersion and particle-matrix interaction. Infrared (IR) radiation emitted from these nanomaterial polymer composite depends on the emissivity of the individual components. In addition, during flash heating and cooling, different thermal conductivity of components in the nanocomposite can influence pixel intensity differently in the IR image or video being captured. We have used an economical mid wavelength IR camera Fluke RSE600 equipped with a close-up macro lens and algorithm based on MATLAB image processing toolbox to analyse dispersion, voids and thermal diffusivity of patented graphene polymer nanocomposite materials (G-PMC) in micro-scale. These G-PMCs can act as a standard material to determine the potential of our IR thermography technique due to their homogeneity and lack of impurity due to unique fabrication process. Thermal diffusivity and dispersion of nanoparticles in our G-PMCs was estimated after irradiation with a xenon flash lamp by spatially mapping transient IR radiations from different G-PMCs using the Fluke RSE600 thermal imager. Results from thermography experiments were compared with scanning electron microscope (SEM) and Raman spectroscopy results. Micro-scale thermography was able to detect millimetre scale thermal diffusivity variation in the injection molded G-PMC samples and relate it to change in dispersion of nanofillers, unlike SEM and Raman, where micro-scale measurements could not determine the reason behind millimetre scale property variation. We believe this low cost, fast, micro-scale, non-destructive technique will provide valuable insight into functional polymer nanocomposite fabrication and corresponding electrical and thermal properties.

Can pregnant women accurately report snoring? A pilot study

Sleep-Disordered Breathing (SDB) has been associated with possible negative outcomes, such as preeclampsia and foetal growth restriction. SDB screening tools have been developed for use within general populations. These included the use of self-reports and objective measurements. Seventeen pregnant women within their 34th to 37th week of pregnancy were recruited. Participants undertook an overnight study within their home and SDB symptoms were monitored using the Watch-PAT 200 and an infra-red video camera. The women were administered an online questionnaire comprised of the Multivariable Apnea Risk Index (MAP Index) and the Basic Nordic Sleep Questionnaire (BNSQ). More than half of our participants (n = 10) were identified as snorers while much fewer (n = 4) met the current cut off for diagnosis of mild SDB. Investigation of concordance and predictive value of self-report measures compared to standard video-scoring and Watch-PAT 200 determined SDB indicators suggests that self-reports may not provide an accurate assessment of SBD symptoms in late pregnancy. Self-report in this study, resulted in an underestimation of the number of participants who experienced SDB symptoms. This was a pilot study, with a small sample size. However, our study lends weight to others that found poor predictive value of common scales to detect SDB in pregnancy. Future study is therefore needed to validate screening tools, which may need to be a combination of measures.

Modelling and Validation of Computer Vision Techniques to Assess Heart Rate, Eye Temperature, Ear-Base Temperature and Respiration Rate in Cattle.

Simple SummaryAnimal monitoring normally requires procedures that are time- and labour-consuming. The implementation of novel non-invasive technologies could be a good approach to monitor animal health and welfare. This study aimed to evaluate the use of images and computer-based methods to track specific features of the face and to assess temperature; respiration rate and heart rate in cattle. The measurements were compared with measures obtained with conventional methods during the same time period. The data were collected from ten dairy cows that were recorded during six handling procedures across two consecutive days. The results from this study show over 92% of accuracy from the computer algorithm that was developed to track the areas selected on the videos collected. In addition, acceptable correlation was observed between the temperature calculated from thermal infrared images and temperature collected using intravaginal loggers. Moreover, there was acceptable correlation between the respiration rate calculated from infrared videos and from visual observation. Furthermore, a low to high relationship was found between the heart rate obtained from videos and from attached monitors. The study also showed that both the position of the cameras and the area analysed on the images are very important, as both had large impact on the accuracy of the methods. The positive outcomes and the limitations observed in this study suggest the need for further researchPrecision livestock farming has emerged with the aim of providing detailed information to detect and reduce problems related to animal management. This study aimed to develop and validate computer vision techniques to track required features of cattle face and to remotely assess eye temperature, ear-base temperature, respiration rate, and heart rate in cattle. Ten dairy cows were recorded during six handling procedures across two consecutive days using thermal infrared cameras and RGB (red, green, blue) video cameras. Simultaneously, core body temperature, respiration rate and heart rate were measured using more conventional ‘invasive’ methods to be compared with the data obtained with the proposed algorithms. The feature tracking algorithm, developed to improve image processing, showed an accuracy between 92% and 95% when tracking different areas of the face of cows. The results of this study also show correlation coefficients up to 0.99 between temperature measures obtained invasively and those obtained remotely, with the highest values achieved when the analysis was performed within individual cows. In the case of respiration rate, a positive correlation (r = 0.87) was found between visual observations and the analysis of non-radiometric infrared videos. Low to high correlation coefficients were found between the heart rates (0.09–0.99) obtained from attached monitors and from the proposed method. Furthermore, camera location and the area analysed appear to have a relevant impact on the performance of the proposed techniques. This study shows positive outcomes from the proposed computer vision techniques when measuring physiological parameters. Further research is needed to automate and improve these techniques to measure physiological changes in farm animals considering their individual characteristics.

Deep Learning-Based Target Tracking and Classification for Low Quality Videos Using Coded Aperture Cameras.

Compressive sensing has seen many applications in recent years. One type of compressive sensing device is the Pixel-wise Code Exposure (PCE) camera, which has low power consumption and individual control of pixel exposure time. In order to use PCE cameras for practical applications, a time consuming and lossy process is needed to reconstruct the original frames. In this paper, we present a deep learning approach that directly performs target tracking and classification in the compressive measurement domain without any frame reconstruction. In particular, we propose to apply You Only Look Once (YOLO) to detect and track targets in the frames and we propose to apply Residual Network (ResNet) for classification. Extensive simulations using low quality optical and mid-wave infrared (MWIR) videos in the SENSIAC database demonstrated the efficacy of our proposed approach.

Temporal-Spatial Nonlinear Filtering for Infrared Focal Plane Array Stripe Nonuniformity Correction

In this work, we introduce a temporal-spatial approach for infrared focal plane array (IRFPA) stripe nonuniformity correction in infrared images that generates visually appealing results. We posit that the nonuniformity appears as a striped structure in the spatial domain and that the pixel values change slowly in the temporal domain. Based on this, we formulate our correction method in two steps. In the first step, weighted guided image filtering with our adaptive weight is utilized to predict the stripe nonuniformity using a single frame. In the second step, the temporal profile of each pixel can be formed using a few frames of successive nonuniformity images. Further, we present a temporal nonlinear diffusion equation to remove scene residuals from the temporal profile of nonuniformity images in order to estimate a more accurate value of the stripe nonuniformity. The results of extensive experiments demonstrate that the proposed nonuniformity correction algorithm substantially outperforms many state-of-the-art approaches, including both traditional and deep convolution-neural-network-based methods, on four popular infrared videos. In addition, the proposed method only requires a fraction (less than ten) of the video frames.

Burrow Wars and Sinister Behaviour among Burrow-Nesting Petrels at Sub-Antarctic Marion Island

Competition for nest sites is relatively common amongst burrow-nesting Procellariiformes, especially on some sub-Antarctic islands where there is limited availability of good burrow-nesting habitat. Where space is limited, petrels may even successfully share a common burrow entrance or nest chamber and burrow densities can reach >7000 burrows/ha. Interspecies burrow competition and chick evictions generally occur as a result of an overlap in breeding seasons, yet there are few documented records of this behaviour and even within study colonies many evictions are unconfirmed or probably go undetected. Here we report on interactions among three burrow-nesting petrels (White-chinned Petrels Procellaria aequinoctialis, Grey Petrels P. cinerea and Great-winged Petrels Pterodroma macroptera) at Marion Island which we observed through regular nest checks with a burrowscope and using infra-red video cameras inside burrow chambers. Despite relatively low petrel densities, White-chinned Petrels were responsible for 17% (8/46) of the Great-winged Petrel chick mortalities over the five breeding seasons (3% of the breeding attempts), but two were also recorded feeding Great-winged Petrel chicks. A pair of White-chinned Petrels evicted a Grey Petrel chick, but then had their own chick killed by Grey Petrels the following season, who went on to breed successfully in the same burrow. Feral Cats Felis catus were eradicated in 1991 and the greatly reduced petrel populations are slowly recovering, which could exacerbate competition for burrows on Marion.

Object Detection in a Maritime Environment: Performance Evaluation of Background Subtraction Methods

This paper provides a benchmark of the performance of 23 classical and state-of-the-art background subtraction (BS) algorithms on visible range and near infrared range videos in the Singapore Maritime dataset. Importantly, our study indicates the limitations of the conventional performance evaluation criteria for maritime vision and proposes new performance evaluation criteria that is better suited to this problem. This paper provides insight into the specific challenges of BS in maritime vision. We identify four open challenges that plague BS methods in maritime scenario. These include spurious dynamics of water, wakes, ghost effect, and multiple detections. Poor recall and extremely poor precision of all the 23 methods, which have been otherwise successful for other challenging BS situations, allude to the need for new BS methods custom designed for maritime vision.

Foreground Detection for Infrared Videos With Multiscale 3-D Fully Convolutional Network

Foreground detection for infrared (IR) videos is an important and fundamental problem in many applications, e.g., IR surveillance, IR object tracking, and so on. Conventional foreground detection algorithms developed for visible videos do not focus on the problems for IR videos, e.g., low contrast, coarse texture, lack of color information, and so on. Recent foreground detection methods based on deep neural network (DNN) demonstrated significant improvement, but most of them still use only spatial features, which is less obvious in IR images. In this letter, we add deeply learned multiscale temporal features to improve the performance of background subtraction for IR videos. We propose a novel multiscale 3-D fully convolutional network (MFC3-D) to establish a mapping from image sequences to pixelwise classification results and to learn deep and hierarchical multiscale spatial–temporal features of the input images sequence. The experimental results show that the MFC3-D can learn spatial–temporal features effectively and achieved state-of-the-art results on the test data set, comparing to other DNN-based methods and traditional background subtraction methods.

Real-Time Geophysical Monitoring of Particle Size Distribution During Volcanic Explosions at Stromboli Volcano (Italy)

Of all the key parameters needed to inform forecast models, real-time tracking particle size distribution (PSD) of pyroclasts leaving the vent coupled with plume modeling has probably the highest potential for effective management of volcanic crises. This paper presents an algorithm capable of providing syn-emission horizontal size and velocity of particles in real time, converted in mass discharge rates, and its evolution during an explosion, using thermal infrared videos. We present data on explosions that occurred at the SW crater of Stromboli volcano (Italy) in 2012. PSDs and mass eruption rate data, collected at frequencies of 40 Hz, are then coupled with particle and gas speed data collected with traditional image analysis techniques. The dataset is used to quantify the dynamics of the explosions and magma fragmentation. We find that explosive evacuation of magma from a Strombolian conduit during a single explosion proceeds at a constant rate while the explosive dynamics are marked by a pattern that includes an initial transient and short phase until the system stabilizes at equilibrium. These stationary conditions dominate the emission. All explosions begin with a gas jet (onset phase), with maximum recorded vertical velocities above 150 m/s. These high velocities are for small particles carried by the faster-moving gas or pressure wave, and larger particles typically have slower velocities. The gas jets are followed by a particle-loaded plume. The particles increase in number until the explosion dynamics become almost constant (in the stationary phase). Mass eruption rate is either stable or increases during the onset to become stable in the stationary phase. The shearing at the interface between the magma and the gas jets controls fragmentation dynamics and particles sizes. Quantification of the Reynolds and Weber numbers suggest that the fragmentation regime changes during an explosion to affect particle shape. This method relates to lapilli and bomb-dominated emissions associated with low energy primary fragmentation. In general terms, its applicability is limited by the size of individual pixels recorded by the camera, which depends on the detector, the recording distance and the optical system, particle temperature, which has to be significantly higher than the background.