Contactless monitoring of human respiration using infrared thermography and deep learning

Abstract

Objective. To monitor the human respiration rate (RR) using infrared thermography (IRT) and artificial intelligence, in a completely contactless, automated, and non-invasive manner. Approach. The human breathing signals (BS) were obtained using IRT, by plotting the change in temperature occurring across the nostrils during breathing, with respect to time. The RR was monitored under extreme conditions (random head motion, involuntary body movements, etc), by developing deep learning (DL) based ‘Residual network50+Facial landmark detection’ (ResNet 50+FLD) model. This model was built and evaluated on 10 000 thermograms and is the first work that documents the use of a DL classifier on a large thermal dataset for nostril tracking. Further, the acquired BS were filtered using the moving average filter (MAF), and the Butterworth filter (BF), and a comparative analysis of their performance was done. The novel ‘breathing signal characterization algorithm (BSCA)’ was proposed to obtain the RR in an automated manner. This algorithm is the first work that identifies the breaths in the thermal BS as regular, prolonged, or rapid, using machine learning (ML). The ‘exploratory data analysis’ was performed to choose an appropriate ML algorithm for the BSCA. The performance of the ‘BSCA’ was evaluated for both ‘decision tree (DT)’ and ‘support vector machine(SVM)’ models. Main results. The ‘ResNet 50+FLD model’ had Validation and Testing accuracy, of 99.5%, and 99.4% respectively. The Precision, Sensitivity, Specificity, F-measure, and G- mean values were computed as well. The comparative analysis of the filters revealed that the BF performed better than the MAF. The ‘BSCA’ performed better with the SVM classifier, than the DT classifier, with validation accuracy, and testing accuracy of 99.5%, and 98.83%, respectively. Significance. The ever-increasing number of critical cases and the limited availability of skilled medical attendants, advocates in favor of an automated and harmless health monitoring system. The proposed methodology is completely contactless, thus eliminating the risk of infections that spread through contact. There is a wide scope of using this methodology in complete darkness, and in remote areas as well, where there is a lack of medical attendants.