Thermal Sensor Array Research Articles

Identifying Human Indoor Daily Life Behavior Employing Thermal Sensor Arrays (TSAs)

Identifying Human Indoor Daily Life Behavior Employing Thermal Sensor Arrays (TSAs)

A study of oil–water two-phase flow pattern recognition based on thermal sensors

To understand the flow characteristics and kinetic modelling of oil–water two-phase flow(OWTPF), this paper employs thermal array sensors to identify the flow pattern (FP) in OWTPF horizontal pipelines. Firstly, the thermal array sensor installation was constructed. Secondly, the time-varying characteristics of multi-feature signals were qualitatively analyzed using the adaptive optimal kernel time–frequency representation (AOK-TFR) method and unthresholded recurrence plot (URP) method. Finally, the sparrow search algorithm-support vector machine (SSA-SVM) machine learning algorithm is used for quantitative prediction of multi-feature classification, which leads to high prediction accuracy. The experiment proves that the correct rate of the thermal method in recognizing six types of OWTPF is above 91%, the correct rate of five types of flow is above 97.33%. This paper adopts the thermal method to identify the multiphase FP, which is a major innovation in this paper and also lays the foundation for recognizing the flow characteristics of multiphase flow.

Reduced CPU Workload for Human Pose Detection with the Aid of a Low-Resolution Infrared Array Sensor on Embedded Systems

Modern embedded systems have achieved relatively high processing power. They can be used for edge computing and computer vision, where data are collected and processed locally, without the need for network communication for decision-making and data analysis purposes. Face detection, face recognition, and pose detection algorithms can be executed with acceptable performance on embedded systems and are used for home security and monitoring. However, popular machine learning frameworks, such as MediaPipe, require relatively high usage of CPU while running, even when idle with no subject in the scene. Combined with the still present false detections, this wastes CPU time, elevates the power consumption and overall system temperature, and generates unnecessary data. In this study, a low-cost low-resolution infrared thermal sensor array was used to control the execution of MediaPipe’s pose detection algorithm using single-board computers, which only runs when the thermal camera detects a possible subject in its field of view. A lightweight algorithm with several filtering layers was developed, which allowed the effective detection and isolation of a person in the thermal image. The resulting hybrid computer vision proved effective in reducing the average CPU workload, especially in environments with low activity, almost eliminating MediaPipe’s false detections, and reaching up to 30% power saving in the best-case scenario.

Ultrasensitive Flexible Thermal Sensor Arrays based on High-Thermopower Ionic Thermoelectric Hydrogel.

Ionic circuits using ions as charge carriers have demonstrated great potential for flexible and bioinspired electronics. The emerging ionic thermoelectric (iTE) materials can generate a potential difference by virtue of selective thermal diffusion of ions, which provide a new route for thermal sensing with the merits of high flexibility, low cost, and high thermopower. Here, ultrasensitive flexible thermal sensor arrays based on an iTE hydrogel consisting of polyquaternium-10 (PQ-10), a cellulose derivative, as the polymer matrix and sodium hydroxide (NaOH) as the ion source are reported. The developed PQ-10/NaOH iTE hydrogel achieves a thermopower of 24.17mV K-1 , which is among the highest values reported for biopolymer-based iTE materials. The high p-type thermopower can be attributed to thermodiffusion of Na+ ions under a temperature gradient, while the movement of OH- ions is impeded by the strong electrostatic interaction with the positively charged quaternary amine groups of PQ-10. Flexible thermal sensor arrays are developed through patterning the PQ-10/NaOH iTE hydrogel on flexible printed circuit boards, which can perceive spatial thermal signals with high sensitivity. A smart glove integrated with multiple thermal sensor arrays is further demonstrated, which endows a prosthetic hand with thermal sensation for human-machine interaction.

Enhanced Performance of Ru-Based Infrared Imaging Sensor Array With Electrospun Thermal Isolation Structure

Predicting and optimizing thermal conductivity of materials used in infrared thermal imaging sensors is an effective method to improve thermal isolation performance, which can substantially reduce the thermal crosstalk among pixels. This paper reports an electrospun P(VDF-TrFE) nanofiber film used as adiabatic support structure in thermal sensor array, and investigates its effect on imaging performance of thermal sensor. Based on solid heat transfer simulation, the thermal conductivities of electrospun P(VDF-TrFE) films with different packing ratios were theoretically calculated, showing a significant decrease compared with the spin-coated counterpart. The temperature increment of thermal sensor using nanofiber film with 43.71% packing ratio is calculated as 1.68 times higher than that using spin-coated film. An average temperature rise of 1.73 times was experimentally obtained, and the thermal response time was also reduced to half, compared with the sensor using spin-coated film. The method of improving thermal isolation performance using electrospun fibrous film could be of great advantage to high sensitivity infrared imaging sensor.

A robotic honeycomb for interaction with a honeybee colony.

Robotic technologies have shown the capability to interact with living organisms and even to form integrated mixed societies composed of living and artificial agents. Biocompatible robots, incorporating sensing and actuation capable of generating and responding to relevant stimuli, can be a tool to study collective behaviors previously unattainable with traditional techniques. To investigate collective behaviors of the western honeybee (Apis mellifera), we designed a robotic system capable of observing and modulating the bee cluster using an array of thermal sensors and actuators. We initially integrated the system into a beehive populated with about 4000 bees for several months. The robotic system was able to observe the colony by continuously collecting spatiotemporal thermal profiles of the winter cluster. Furthermore, we found that our robotic device reliably modulated the superorganism's response to dynamic thermal stimulation, influencing its spatiotemporal reorganization. In addition, after identifying the thermal collapse of a colony, we used the robotic system in a "life-support" mode via its thermal actuators. Ultimately, we demonstrated a robotic device capable of autonomous closed-loop interaction with a cluster comprising thousands of individual bees. Such biohybrid societies open the door to investigation of collective behaviors that necessitate observing and interacting with the animals within a complete social context, as well as for potential applications in augmenting the survivability of these pollinators crucial to our ecosystems and our food supply.

The Design of a Low-Cost Sensing and Control Architecture for a Search and Rescue Assistant Robot

At a disaster site, unforeseen circumstances can severely limit the activities of rescue workers. The best solution is for a cooperative team of robots and rescue workers to complete the rescue work. Therefore, in this paper, we propose a simple and low-cost sensing and control architecture for a search and rescue assistant robot using a thermal infrared sensor array, an ultrasonic sensor, and a three-axis accelerometer. In the proposed architecture, we estimate the location of human survivors using a low-cost thermal IR sensor array and generate and control the trajectory of approaching the searched human survivors. Obstacle avoidance and control are also possible through 3D position estimation of obstacles using 1D ultrasonic sensor integration. In addition, a three-axis accelerometer is used to estimate the tilt angle of the robot according to terrain conditions, and horizontal control of the storage box angle is performed using this feature. A prototype robot was implemented to experimentally validate its performance and can be easily constructed from inexpensive, commonly available parts. The implementation of this system is simple and cost-effective, making it a viable solution for search and rescue operations. The experimental results have demonstrated the effectiveness of the proposed method, showing that it is capable of achieving a level storage box and identifying the location of survivors while moving on a sloped terrain.

Privacy-Preserving, Thermal Vision With Human in the Loop Fall Detection Alert System

To support the independent living of older adults in their own homes, it is essential to identify their abnormal behaviors before triggering an automated alert system. Existing normal vision sensing approaches to detect human falls in the activities of daily living (ADL) experienced acceptability issues due to outstanding privacy concerns when they are deployed in personal environments. Besides, false alerts (false-positive) fall detection has not been addressed thoroughly in systems that report abnormal human behaviors as emergency alerts to the information support. This article proposes a novel human-in-the-loop fall detection approach in the ADLs using a low-resolution thermal sensor array. The motivation for enabling a human interactive model, fall detection confirmation, is to influence resource efficiency by reducing false-positive alerts while keeping the false-negative fall predictions as low as possible. The proposed approach is based on the motion sequence classification of human movements using a recurrent neural network. The proposed approach is evaluated with comprehensive experiments using different learning techniques, users, and domestic environment conditions. This article shows a performance accuracy of 99.7% to detect human falls from various typical ADLs.

Image Recognition Based on Deep Learning with Thermal Camera Sensing

As the COVID-19 epidemic spread across the globe, people around the world were advised or mandated to wear masks in public places to prevent its spreading further. In some cases, not wearing a mask could result in a fine. To monitor mask wearing, and to prevent the spread of future epidemics, this study proposes an image recognition system consisting of a camera, an infrared thermal array sensor, and a convolutional neural network trained in mask recognition. The infrared sensor monitors body temperature and displays the results in real-time on a liquid crystal display screen. The proposed system reduces the inefficiency of traditional object detection by providing training data according to the specific needs of the user and by applying You Only Look Once Version 4 (YOLOv4) object detection technology, which experiments show has more efficient training parameters and a higher level of accuracy in object recognition. All datasets are uploaded to the cloud for storage using Google Colaboratory, saving human resources and achieving a high level of efficiency at a low cost.

An Unobtrusive Human Activity Recognition System Using Low Resolution Thermal Sensors, Machine and Deep Learning.

Given the aging population, healthcare systems need to be established to deal with health issues such as injurious falls. Wearable devices can be used to detect falls. However, most wearable devices are obtrusive, and patients generally do not like or may forget to wear them. In this study, we developed an unobtrusive monitoring system using infrared technology to unobtrusively detect locations and recognize human activities such as sitting, standing, walking, lying, and falling. We prototyped a system consisting of two 24×32 thermal array sensors and collected data from healthy young volunteers performing ten different scenarios. A supervised deep learning (DL)-based approach classified activities and detected locations from images. The performance of the DL approach was also compared with the machine learning (ML)-based methods. In addition, we fused the data of two sensors and formed a stereo system, which resulted in better performance compared to a single sensor. Furthermore, to detect critical activities such as falling and lying on floor, we performed a binary classification in which one class was falling plus lying on floor and another class was all the remaining activities. Using the DL-based algorithm on the stereo dataset to recognize activities, overall average accuracy and F1-score were achieved as 97.6%, and 0.935, respectively. These scores for location detection were 97.3%, and 0.927, respectively. These scores for binary classification were 97.9%, and 0.945, respectively. Our results suggest the proposed system recognized human activities, detected locations, and detected critical activities namely falling and lying on floor accurately.

SmartAct: Energy Efficient and Real-Time Hand-to-Mouth Gesture Detection Using Wearable RGB-T.

Researchers have been leveraging wearable cameras to both visually confirm and automatically detect individuals' eating habits. However, energy-intensive tasks such as continuously collecting and storing RGB images in memory, or running algorithms in real-time to automate detection of eating, greatly impacts battery life. Since eating moments are spread sparsely throughout the day, battery life can be mitigated by recording and processing data only when there is a high likelihood of eating. We present a framework comprising a golf-ball sized wearable device using a low-powered thermal sensor array and real-time activation algorithm that activates high-energy tasks when a hand-to-mouth gesture is confirmed by the thermal sensor array. The high-energy tasks tested are turning on the RGB camera (Trigger RGB mode) and running inference on an on-device machine learning model (Trigger ML mode). Our experimental setup involved the design of a wearable camera, 6 participants collecting 18 hours of data with and without eating, the implementation of a feeding gesture detection algorithm on-device, and measures of power saving using our activation method. Our activation algorithm demonstrates an average of at-least 31.5% increase in battery life time, with minimal drop of recall (5%) and without impacting the accuracy of detecting eating (a slight 4.1% increase in F1-Score).

Multiple Thermal Sensor Array Fusion Toward Enabling Privacy-Preserving Human Monitoring Applications

Human-centric applications of a single thermal sensor array (TSA) have performed extremely well in many areas. However, most of these works have not yet reached the real applicability stage of the Internet of Things (IoT) applications. The main limitation of deploying such systems on a large scale is the challenge of fusing multiple TSAs to cover a wide inspection area, e.g., smart homes, hospitals, and many other domestic environments. On the other hand, objects that appear in the low-resolution thermal images acquired from TSA have low intraclass variations and high interclass similarities, making the identification of the overlapping regions through matching a comparable template image in multiple images very difficult. This article proposes a motion-based approach to fuse multiple TSAs and learn the domestic environment layout to enable further human-centred IoT applications to run in the cloud. Besides, a privacy improvement on utilizing these sensors in IoT applications is proposed. The proposed approach is evaluated with comprehensive experiments on different sensor placements and domestic environment conditions. This article shows an average performance of 92.5% accuracy using various machine learning techniques and use case scenarios.

Operating Temperature Analysis for Microbolometric Focal Plane Array

The operating temperature is numerically calculated and theoretically processed for the microbolometer thermal sensor array with pulse voltage bias. According to the physically thermal model and equation of heat balance for a microbolometer under a list of biasing voltage pulses, the operating temperature equation is theoretically derived for numerical calculations. Above these exact numerical calculations, the operating temperature expressed in theory is proposed for the thermal sensor in stable thermal cycles with pulse voltage bias. The theoretical expression is useful to predict the accurate infrared sensitivity of the microbolometric sensor in normal operating state, which has been proved to be exact and usable, and is feasible for being applied to the infrared sensing performance promotions of microbolometric array.

Fabric-inspired thermoelectric two-dimensional imaging array based on carbon nanotube

A thermal imaging array can be a convenient tool for health monitoring and security applications. In this work, a two-dimensional wearable thermal imaging sensor array design comprised of the carbon nanotube (CNT) harnessing thermoelectric effect is proposed. The proposed sensor device was constructed of an array of p-type CNT fibers that are woven across another array of n-type CNT fibers in the similar manner fibers that are woven in textile fabric. Electromagnetic (EM) wave emitted from the subject of detection increases the temperature of certain array nodes of the sensor device, which causes voltage differences between different p–n fiber junctions. Mapping the voltages of all the p–n junctions, thermal images of the subject can be obtained. Though there is a trade-off between responsivity and detection resolution, our proposed sensor can provide a responsivity of 57 V/W for a nanowatt range EM power source with submicrometer level detection resolution according to our calculation. Moreover, we theoretically investigated the effect of the junction distance in the sensor and the size of the hotspot on the resultant thermoelectric voltage.

Flexible Skin for Flight Parameter Estimation Based on Pressure and Velocity Data Fusion

Accurate perception of flight parameters is essential for the safe and stable flight of small unmanned aerial systems (SUASs). However, traditional flush air data sensing (FADS) systems require complex tubing and fuselage openings. Herein this study, a flexible skin system integrated with a pressure and thermal flow sensor array, and a dual‐sensor fusion algorithm for determining the angle of attack (AOA) and airspeed (V ∞ ), are proposed. By establishing an error back‐propagation neural network, the dual‐sensor fusion modality demonstrates higher estimation accuracy than other modalities that utilize only pressure data or only flow velocity data, achieving mean absolute errors of the AOA and V ∞ of less than 0.16° and 0.37 ms−1, respectively. Moreover, the simulation and experimental results show that sensors placed closer to the leading edge of the wing can provide higher estimation accuracy of the flight parameters. The flight parameters determined using the flexible air data sensing system and dual‐sensor fusion algorithm demonstrate potential application in the flight control of SUASs.

Calibration of Low-Resolution Thermal Imaging for Human Monitoring Applications

Thermal imaging has recently come to light to measure high human body temperature (fever) in responses to the global public health issues. This is normally achieved by very expensive high-resolution thermal cameras. Lately, there has been a new commercial low-resolution thermal sensor array (TSA) that has gained growing interest in indoor human monitoring applications due to their low-cost and human privacy-preserving claims. However, there has not been sufficient independent empirical calibration of low-resolution TSA and high-resolution images for human-centred applications. This letter provides empirical calibration of low- and high-resolution thermal imaging techniques in terms of their visible outputs, accuracy in temperature values, and stability. Besides, this letter assesses the claimed privacy-preserving feature of TSA by experimentally validating the possibility of revoking the human identity from the TSA’s output. Thus, this letter aims to understand better the advantages, limitations, and future trends of using TSA in human monitoring applications.

CNN-LSTM 기반 낙상 감지 시스템 구현

본 논문에서는 영상 데이터 처리 모델인 CNN(convolutional neural networks)과 시퀀스 데이터 처리 모델인 LSTM(long short term memory)을 결합하여 낙상 감지 시스템 모델을 구현한다. 비접촉 적외선 열상 카메라 모듈(thermal array sensor)을 이용하여 인간의 보편적인 3가지 자세(서 있는 자세, 앉은 자세, 누운 자세)의 데이터세트를 수집하였다. 수집된 데이터를 CNN 알고리즘의 입력 값으로 사용하여 나온 출력 값을 LSTM 알고리즘의 입력 값으로 취하는 CNN-LSTM 결합형 알고리즘을 사용하였다. 본 논문에서는 비접촉 적외선 열상 카메라 모듈을 이용하여 수집한 비디오 데이터세트를 통해서 CNN-LSTM, ResNetCNN-LSTM 두 가지 딥 러닝 모델에서 낙상감지 시스템을 구현한 후 각각 정확도를 비교 분석하였다. 실험 결과를 통한 정확도를 기반으로 향후 연구 방향에 대해 논의한다.

Sensors-on-paper: Fabrication of graphite thermal sensor arrays on cellulose paper for large area temperature mapping

This paper reports on a fabrication method to obtain multiple thermal sensors by employing an array of graphite thermocouple patterns on commonly available Xerox paper. The graphite thermocouples are patterned using two different grade graphite pencils, which show a stable and reproducible thermal sensitivity. The fabricated paper devices with multiple thermocouple arrays are capable of producing temperature mapping of the desired area. Different thermal conditions were applied to test and confirm the working of these devices. The present work shows that simple graphite trace patterns can convert a piece of paper into a thermal mapping device.

A Non-Invasive Breast Cancer Detection System Using FBG Thermal Sensor Array: A Feasibility Study

Breast cancer is one of the most challenging health problems in women worldwide. Early detection and regular breast cancer screening can considerably improve the survival rate. Many advanced imaging technologies are currently being used in the detection of breast malignancy. However, these methods are costly, uncomfortable, and time-consuming. Therefore, in this work, a simple and noninvasive thermographic technique, based on a Fiber Bragg Grating (FBG) thermal sensor array, has been developed to detect breast cancer. Breast tissue is simulated using agar gel phantom, and the heaters are embedded inside the phantom in different locations to mimic the tumors. FBG thermal sensor array is employed to acquire the surface temperature distribution of the simulated breast phantom. Also, modeling of the tissue-mimicking phantom and tumor in COMSOL Multiphysics software is carried out to provide a detailed 3D thermal view of the proposed prototype. Both experimental and simulated results are found to be in agreement and display the same trend as expected. Experimental results show that a temperature variation of ≥0.3°C is detected at the location of the tumor’s presence, which can be easily picked up by FBG interrogators which have sensitivity of ~1 pm, indicating the efficacy of the proposed sensing prototype as a potential screening tool for the diagnosis of breast cancer.

Enhanced precision of real-time control photothermal therapy using cost-effective infrared sensor array and artificial neural network

Photothermal therapy (PTT) requires tight thermal dose control to achieve tumor ablation with minimal thermal injury on surrounding healthy tissues. In this study, we proposed a real-time closed-loop system for monitoring and controlling the temperature of PTT using a non-contact infrared thermal sensor array and an artificial neural network (ANN) to induce a predetermined area of thermal damage on the tissue. A cost-effective infrared thermal sensor array was used to monitor the temperature development for feedback control during the treatment. The measured and predicted temperatures were used as inputs of fuzzy control logic controllers that were implemented on an embedded platform (Jetson Nano) for real-time thermal control. Three treatment groups (continuous wave = CW, conventional fuzzy logic = C-Fuzzy, and ANN-based predictive fuzzy logic = P-Fuzzy) were examined and compared to investigate the laser heating performance and collect temperature data for ANN model training. The ex vivo experiments validated the efficiency of fuzzy control with temperature method on maintaining the constant interstitial tissue temperature (80 ± 1.4 °C) at a targeted surface of the tissue. The linear relationship between coagulation areas and the treatment time was indicated in this study, with the averaged coagulation rate of 0.0196 cm2/s. A thermal damage area of 1.32 cm2 (diameter ∼1.3 cm) was observed under P-Fuzzy condition for 200 s, which covered the predetermined thermal damage area (diameter ∼1 cm). The integration of real-time feedback temperature control with predictive ANN could be a feasible approach to precisely induce the preset extent of thermal coagulation for treating papillary thyroid microcarcinoma.