Infrared Sensor Array Research Articles

Gait Anomaly Detection With Low-Cost and Low-Resolution Infrared Sensor Arrays

Gait Anomaly Detection With Low-Cost and Low-Resolution Infrared Sensor Arrays

Fall detection on embedded platform using infrared array sensor for healthcare applications

Fall detection on embedded platform using infrared array sensor for healthcare applications

Reduced CPU Workload for Human Pose Detection with the Aid of a Low-Resolution Infrared Array Sensor on Embedded Systems

Modern embedded systems have achieved relatively high processing power. They can be used for edge computing and computer vision, where data are collected and processed locally, without the need for network communication for decision-making and data analysis purposes. Face detection, face recognition, and pose detection algorithms can be executed with acceptable performance on embedded systems and are used for home security and monitoring. However, popular machine learning frameworks, such as MediaPipe, require relatively high usage of CPU while running, even when idle with no subject in the scene. Combined with the still present false detections, this wastes CPU time, elevates the power consumption and overall system temperature, and generates unnecessary data. In this study, a low-cost low-resolution infrared thermal sensor array was used to control the execution of MediaPipe’s pose detection algorithm using single-board computers, which only runs when the thermal camera detects a possible subject in its field of view. A lightweight algorithm with several filtering layers was developed, which allowed the effective detection and isolation of a person in the thermal image. The resulting hybrid computer vision proved effective in reducing the average CPU workload, especially in environments with low activity, almost eliminating MediaPipe’s false detections, and reaching up to 30% power saving in the best-case scenario.

Skeleton-based 3D human pose estimation with low-resolution infrared array sensor using attention based CNN-BiGRU

Skeleton-based 3D human pose estimation with low-resolution infrared array sensor using attention based CNN-BiGRU

Application of visible near-infrared spectroscopy combined with colorimetric sensor array for the aroma quality evaluation in tencha drying process

The drying process is a critical stage in developing the aroma quality of tencha. In our research, visible near infrared (Vis-NIR) and colorimetric sensor array (Vis-NIR-CSA) were used for evaluating the aroma quality of tencha drying process. Vis-NIR recorded the spectral signal of CSA after the reaction in samples. Subsequently, the aroma quality was predicted by a combination of different data fusion strategies and classification and regression tree (CART) in tencha drying process. The high-level fusion strategy showed the best performance, with calibration and prediction set accuracy of 94.68% and 93.48%, respectively. The results indicated that Vis-NIR-CSA combined with high-level data fusion could be applied satisfactorily in the aroma quality evaluation of tencha. Moreover, pentanal was identified to be highly correlated with aroma quality during tencha drying process, which verified the sensor identification results. This study contributed to controlling good manufacturing practices and designing optimal tencha processing systems.

Self-powered ultra-flexible infrared sensor based on PVA-PEDOT: PSS/Ti3C2Tx composite film

Flexibility and power supply needless are two demands of the emerging electronics, including sensor, processer and so on. Wearable infrared sensor is an important part of intelligent and medical sensing. In this work, a self-powered flexible infrared sensor with photo-thermo-electric energy conversion is designed based on the composite film of polyvinyl alcohol (PVA)/poly-(3,4-ethylene dioxythiophene): polystyrene sulfonate (PEDOT: PSS)/Ti3C2Tx. Appropriate content of Ti3C2Tx nanoflakes improves the Seebeck coefficient and greatly enhances light absorption simultaneously. Bionic micro-dome structure is employed to the surface of the film to further enhance the light absorption for better photo-thermal conversion. As results, the composite film with 5 mg Ti3C2Tx shows a Seebeck coefficient up to 37.57 μV K−1, a response time <0.75 s and great stretchability (up to 200 %). At 2 mW cm−2 power density of an 850 nm laser, the film generates 220 μV thermal voltage, which is 44 times of that of the film without Ti3C2Tx. An infrared sensor array based on this composite film is designed for the successful imaging of a human palm. This self-powered infrared sensor shows great application prospects in the fields of wearable non-visual perception and intelligent sensing.

Human Activity Recognition With Low-Resolution Infrared Array Sensor Using Semi-Supervised Cross-Domain Neural Networks for Indoor Environment

Low-resolution infrared-based human activity recognition (HAR) attracted enormous interests due to its low-cost and private. In this paper, a novel semi-supervised crossdomain neural network (SCDNN) based on 8 × 8 low-resolution infrared sensor is proposed for accurately identifying human activity despite changes in the environment at a low-cost. The SCDNN consists of feature extractor, domain discriminator and label classifier. In the feature extractor, the unlabeled and minimal labeled target domain data are trained for domain adaptation to achieve a mapping of the source domain and target domain data. The domain discriminator employs the unsupervised learning to migrate data from the source domain to the target domain. The label classifier obtained from training the source domain data improves the recognition of target domain activities due to the semi-supervised learning utilized in training the target domain data. Experimental results show that the proposed method achieves 92.12% accuracy for recognition of activities in the target domain by migrating the source and target domains. The proposed approach adapts superior to cross-domain scenarios compared to the existing deep learning methods, and it provides a low-cost yet highly adaptable solution for cross-domain scenarios.

Privacy-preserving activity recognition using multimodal sensors in smart office

Developing a human activity recognition (HAR) system for employees is essential to incorporate intelligence into smart office environments, enabling various human-centered applications to enhance employees’ well-being. Although remarkable progress has been made for the HAR in the smart office, several issues still exist, including lacking a privacy-preserving and unobtrusive method and demanding enhanced generalization performance across users. Therefore, a novel privacy-preserving HAR method based on multimodal sensors is investigated in this study, containing an infrared array sensor, a sensing chair, and a triaxial accelerometer built in a smartphone. The effectiveness of different multimodal combinations is examined. Moreover, a deep learning model is developed for multimodal data fusion to enhance the generalization performances across users. The model contains a residual 3D convolutional neural network (CNN) and 1D CNN for learning spatial–temporal feature representation of different modalities. Additionally, external memory units and an adaptive decision fusion operation are utilized for multimodal data fusion. Finally, extensive experiments are conducted to examine the performance of the proposed model using a self-collected dataset and the leave-one-subject-out cross-validation approach. The results verify the effectiveness of the proposed model.

The Design of a Low-Cost Sensing and Control Architecture for a Search and Rescue Assistant Robot

At a disaster site, unforeseen circumstances can severely limit the activities of rescue workers. The best solution is for a cooperative team of robots and rescue workers to complete the rescue work. Therefore, in this paper, we propose a simple and low-cost sensing and control architecture for a search and rescue assistant robot using a thermal infrared sensor array, an ultrasonic sensor, and a three-axis accelerometer. In the proposed architecture, we estimate the location of human survivors using a low-cost thermal IR sensor array and generate and control the trajectory of approaching the searched human survivors. Obstacle avoidance and control are also possible through 3D position estimation of obstacles using 1D ultrasonic sensor integration. In addition, a three-axis accelerometer is used to estimate the tilt angle of the robot according to terrain conditions, and horizontal control of the storage box angle is performed using this feature. A prototype robot was implemented to experimentally validate its performance and can be easily constructed from inexpensive, commonly available parts. The implementation of this system is simple and cost-effective, making it a viable solution for search and rescue operations. The experimental results have demonstrated the effectiveness of the proposed method, showing that it is capable of achieving a level storage box and identifying the location of survivors while moving on a sloped terrain.

A low-cost calibration method for the infrared sensor array for quantitative analysis of natural gas

Infrared-based measurement technology is a promising approach to natural gas analysis, and high-precision calibration models for infrared sensor arrays require sufficient calibration data, which is costly. Therefore, we propose a calibration-transfer-based low-cost calibration method for infrared gas sensor arrays. We construct four identical infrared sensor arrays, one as the master array and three as slave arrays. A dedicated multi-layer perceptual neural network undertakes the data space mapping between the slave array and the master array. Thus, the slave sensor arrays can reuse the existing high-precision calibration model of the master array, reducing the calibration cost. The root mean squared errors of methane, ethane, and propane of the calibrated slave array are 2.87%(detection range: 0%–100%), 1.56%(detection range: 0%–24%), and 1.54%(detection range: 0%–12%), respectively, which are close to the accuracy level of the master array. The results show that our approach outperforms related algorithms in infrared sensor array calibration. In addition, experimental results on actual natural gas samples validate the method. The proposed low-cost and high-precision calibration method provides a basis for the generalization of infrared sensor arrays for mixed gas concentration monitoring. Furthermore, the reported scheme can be extended to quantitative analysis of other gas mixtures.

Uncooled self-powered hemispherical biomimetic pit organ for mid- to long-infrared imaging.

Infrared vision is highly desirable for applications in multifarious fields. Of the few species with this visual capability, snakes have exceptional infrared perception with the assistance of pit organs. Inspired by the pit organ design we present here a hemispherical biomimetic infrared imaging device. The devices use high-density ionic thermoelectric polymer nanowire arrays that serve as the sensing nerve cells. The individual nanowires exhibit notable voltage response to temperature variation in test objects. An infrared sensor array with 625 pixels on the hemispherical substrate is successfully demonstrated with an ultrawide field of view up to 135°. The device can image body temperature objects without a cooling system and external power supply. This work opens up opportunities for the design and fabrication of bioinspired infrared imaging devices based on emerging ionic thermoelectric materials.

Monitoring Social Distancing in Queues Using Infrared Array Sensor

Since the emergence of the coronavirus disease (COVID-19), more than 165 million people have been infected all around the world (as of May 2021). The COVID-19 virus is highly contagious, especially in large crowded spaces. To prevent the infection and slow the transmission of the virus, the World Health Organization recommends wearing a mask, washing hands regularly, and maintaining a distance of at least one meter between people. To support this prevention, we propose a connected object based on LoRa technology to monitor the social distancing of one meter between people in queues. The proposed system uses infrared array sensor MLX60940 to detect people through their body temperature. Based on this output, we developed an algorithm to verify social distancing. The main advantage of this node is its ability to preserve the privacy of detected people and consume less energy compared to camera-based systems. Experimental results show that the proposed node can detect up to six persons per queue with an accuracy of 93% when the distance between people is below 80 cm.

A privacy-preserving and unobtrusive sitting posture recognition system via pressure array sensor and infrared array sensor for office workers

Sitting posture recognition is essential in preventing work-related musculoskeletal disorders (WMSDs). WMSDs are of huge concern for office workers whose working process is averagely 81.8% sedentary. Prevailing studies have utilized cameras, wearables, and pressure sensors to recognize sitting postures. The cameras and wearables can achieve accurate recognition results, while personal privacy concerns and inconvenience for long-term use impede their adoption. Meanwhile, the pressure sensors are privacy-preserving and convenient. However, they cannot accurately recognize the sitting posture with different states of the trunk, head, upper extremity, and lower extremity. Considering the pros and cons of those approaches, this study proposes a novel privacy-preserving and unobtrusive sitting posture recognition system, which combines a pressure array sensor with another privacy-preserving sensing technology, i.e., an infrared array (IRA) sensor. Moreover, a deep learning-based sitting posture recognition algorithm is developed, which adopts a feature-level fusion strategy and does not require a complex handcrafted feature extraction process. Based on the ergonomics studies, ten daily sitting postures with the states of different body parts are selected. This system achieved an overall 90.6% accuracy using the leave-subject-out validation approach based on the self-collected dataset from 21 subjects. It has a great potential for privacy-preserving and unobtrusive related applications for sitting posture management.

An Infrared Array Sensor-Based Approach for Activity Detection, Combining Low-Cost Technology with Advanced Deep Learning Techniques.

In this paper, we propose an activity detection system using a 24 × 32 resolution infrared array sensor placed on the ceiling. We first collect the data at different resolutions (i.e., 24 × 32, 12 × 16, and 6 × 8) and apply the advanced deep learning (DL) techniques of Super-Resolution (SR) and denoising to enhance the quality of the images. We then classify the images/sequences of images depending on the activities the subject is performing using a hybrid deep learning model combining a Convolutional Neural Network (CNN) and a Long Short-Term Memory (LSTM). We use data augmentation to improve the training of the neural networks by incorporating a wider variety of samples. The process of data augmentation is performed by a Conditional Generative Adversarial Network (CGAN). By enhancing the images using SR, removing the noise, and adding more training samples via data augmentation, our target is to improve the classification accuracy of the neural network. Through experiments, we show that employing these deep learning techniques to low-resolution noisy infrared images leads to a noticeable improvement in performance. The classification accuracy improved from 78.32% to 84.43% (for images with 6 × 8 resolution), and from 90.11% to 94.54% (for images with 12 × 16 resolution) when we used the CNN and CNN + LSTM networks, respectively.

Near Infrared Sensor Setup for General Interface Detection in Automatic Liquid-Liquid Extraction Processes

This work presents a novel sensor setup for the general detection of liquid-liquid interfaces in different mixes of liquids as part of a liquid-liquid extraction device. The sensor setup is applied to a laboratory scale separatory funnel. It uses a near infrared sensor array which receives light going through the liquids inside the funnel, which are illuminated by a light source located on the other side and below the funnel. Light refracts inside the funnel and the liquids and reflects on the interface creating changing patterns in the light intensity measured by the sensor, providing a way of locating the liquid-liquid interface. Liquid mixes with different optical features, from transparent to opaque, emulsion and clean, are used to test whether different types of interfaces produce a distinguishable response on the sensor, allowing to detect interfaces in different situations that can occur as part of an Artificial Intelligence orchestrated battery chemical synthesis process. Emulsion interfaces create a discernible change in the sensor input by lowering the light intensity registered when crossing in front of the sensor making them easier to locate than with other optical techniques. The setup opens the possibility of detecting a liquid-liquid interface as it is forming and can be miniaturized to be attached to laboratory funnels as a manual aid or used with other transparent vessels in automatic solutions like liquid handling robots or pipetting robots.

Automatic Detection System for Vehicular Heatstroke Prevention

Many children and animals had died from Vehicular Heatstroke every year. According to the “Not Heatstroke organization,” 882 children have died due to pediatric Vehicular Heatstroke since 1998 in just in the US. Vehicle manufacturers have not given this tragic problem the needed attention despite the massive development of the technology of vehicles. Nowadays, the technology of vehicles has been improved by different types of systems such as Anti-Lock Brakes, Electronic Stability Control, Adaptive cruise control, Lane-departure warning system, and Self-parking which assist and protect the driver's life during the trip. In this paper, a comparison between this system and other vehicular heatstroke detection systems will be executed. An automatic vehicular heatstroke detection system will be designed based on an Infrared Array Sensor. This system will detect any child inside the vehicle by using the Infrared Array Sensor “Grid EYE” as soon as the driver leaves the vehicle. After that, the system will activate the light and horn of the vehicle as an alarm. And then, the system will send an SMS to the driver by GSM signal. Finally, the system will send a signal to turn the air condition on only in case there is no response from the driver.

Automatic Detection System for Vehicular Heatstroke Prevention

Many children and animals had died from Vehicular Heatstroke every year. According to the “Not Heatstroke organization,” 882 children have died due to pediatric Vehicular Heatstroke since 1998 in just in the US. Vehicle manufacturers have not given this tragic problem the needed attention despite the massive development of the technology of vehicles. Nowadays, the technology of vehicles has been improved by different types of systems such as Anti-Lock Brakes, Electronic Stability Control, Adaptive cruise control, Lane-departure warning system, and Self-parking which assist and protect the driver's life during the trip. In this paper, a comparison between this system and other vehicular heatstroke detection systems will be executed. An automatic vehicular heatstroke detection system will be designed based on an Infrared Array Sensor. This system will detect any child inside the vehicle by using the Infrared Array Sensor “Grid EYE” as soon as the driver leaves the vehicle. After that, the system will activate the light and horn of the vehicle as an alarm. And then, the system will send an SMS to the driver by GSM signal. Finally, the system will send a signal to turn the air condition on only in case there is no response from the driver.

A smart reheating and defrosting microwave oven based on infrared temperature sensor and humidity sensor

Microwave oven is widely used around the world since the functions of reheating and defrosting are convenient and efficient. However, these functions in traditional microwave oven are not intelligent for the control effects deeply rely on the users' experience. Besides, heating without food is very dangerous since there is no no-load detection. Therefore, in this paper, infrared array sensor and humidity sensor are combined together to achieve a smart microwave oven. Besides, no-load detection method, detection method for humidity sensor, pre-processing method for infrared sensor, location identification method for the food, and intelligent control methods for defrosting and reheating are proposed in detail. Experimental results demonstrate that the eigenvalues of load tests are all smaller than 1.5, while they are all larger than 1.5 in the no-load tests. The detection accuracy is 100% in the 200 test cases. In addition, the final temperatures of the meats (i.e. pork, fish, drumstick and chicken) after intelligent defrosting control are all between −2 °C and 25 °C, and the mean temperature ranges from 1 °C to 2.5 °C, which meets the expected target. Besides, the maximum control error of “Heating Target” is 3.5 °C, and the maximum difference between the mean temperature of multiple measurements and the corresponding target is 1.56 °C. The control error and control deviation of the temperature range for “Milk” are both less than 1 °C. As for “Reheating” control, the maximum control error and control deviation of the temperature range for the food with less moisture content are 0.18 °C and 2.5 °C, while they are 2.7 °C and 3.2 °C for the food with more moisture content. Moreover, the food quality after reheating or defrosting with the proposed smart microwave oven is better than that with the traditional microwave oven. In general, the proposed no-load detection method, and the intelligent control methods for defrosting and reheating are effective.

Low-Resolution Infrared Array Sensor for Counting and Localizing People Indoors: When Low End Technology Meets Cutting Edge Deep Learning Techniques

In this paper, we propose a method that uses low-resolution infrared (IR) array sensors to identify the presence and location of people indoors. In the first step, we introduce a method that uses 32 × 24 pixels IR array sensors and relies on deep learning to detect the presence and location of up to three people with an accuracy reaching 97.84%. The approach detects the presence of a single person with an accuracy equal to 100%. In the second step, we use lower end IR array sensors with even lower resolution (16 × 12 and 8 × 6) to perform the same tasks. We invoke super resolution and denoising techniques to faithfully upscale the low-resolution images into higher resolution ones. We then perform classification tasks and identify the number of people and their locations. Our experiments show that it is possible to detect up to three people and a single person with accuracy equal to 94.90 and 99.85%, respectively, when using frames of size 16 × 12. For frames of size 8 × 6, the accuracy reaches 86.79 and 97.59%, respectively. Compared to a much complex network (i.e., RetinaNet), our method presents an improvement of over 8% in detection.

Fall detection system based on infrared array sensor and multi-dimensional feature fusion

In recent years, with the acceleration of the aging of the population, the safety of the elderly living alone has attracted great attention, and the falls have become one of the main factors leading to elderly casualties. In order to obtain a high precision and low cost fall detection system for the elderly, a fall detection system based on infrared array sensor and multi-dimensional feature fusion is proposed in this paper. First, we propose a new data acquisition method using infrared array sensor, which effectively enlarges the detection area. Then the personnel positioning is performed before fall detection, which can ensure real-time detection while reducing computational complexity. In addition, a sliding window algorithm is developed and four representative features of a fall are extracted from the collected data, which is fitful to the online detection. Among them, the four characteristics include the change in the center of mass of the falling process, the change in the speed, the change in the area of the person, and the change in variance. Finally, based on the refined features, the support vector machine (SVM) classifier is introduced to identify falls and improve the classification accuracy. The experimental results validate that the proposed fall detection system shows good fall detection accuracy and great practicability.