Occupant Detection Research Articles

Vehicle Occupant Detection Based on MM-Wave Radar.

With the continuous development of automotive intelligence, vehicle occupant detection technology has received increasing attention. Despite various types of research in this field, a simple, reliable, and highly private detection method is lacking. This paper proposes a method for vehicle occupant detection using millimeter-wave radar. Specifically, the paper outlines the system design for vehicle occupant detection using millimeter-wave radar. By collecting the raw signals of FMCW radar and applying Range-FFT and DoA estimation algorithms, a range-azimuth heatmap was generated, visually depicting the current status of people inside the vehicle. Furthermore, utilizing the collected range-azimuth heatmap of passengers, this paper integrates the Faster R-CNN deep learning networks with radar signal processing to identify passenger information. Finally, to test the performance of the detection method proposed in this article, an experimental verification was conducted in a car and the results were compared with those of traditional machine learning algorithms. The findings indicated that the method employed in this experiment achieves higher accuracy, reaching approximately 99%.

Exploring occupant detection model generalizability for residential buildings using supervised learning with IEQ sensors

This study explores two modeling approaches for occupancy detection at room level for residential buildings in Denmark. The aim is to assess the performance and generalizability of occupant detection models using XGBoost method trained on a rich dataset comprising indoor environmental quality (IEQ) variables and occupancy ground truth. A global approach (all rooms) and a room-specific approach are considered. After a thorough feature- selection and importance analysis process, the occupancy detection models (ODMs) are trained and tested in a nested cross-validation schema. The time of the day, indoor CO2 concentration, and feature transforms related to short-term IEQ dynamics were found to be the most important features of the ODMs. Both the global and room-specific models show good occupancy prediction performance, especially for bedrooms. When tested for generalizability with an unseen dataset from a different residential building, the ODMs maintain very good performance for the bedroom but not for the office room. This discrepancy could be explained by significant differences in occupancy and ventilation patterns, and large air infiltration from adjacent rooms. Although currently limited in terms of generalizability, XGBoost-based ODMs using IEQ data have the potential to provide robust and scalable occupancy detection for occupant-centric control and occupant-aware building performance assessments. The IEQ dataset with occupancy ground truth collected for this study is made available in open access.

Zonal demand-controlled ventilation strategy to minimize infection probability and energy consumption: A coordinated control based on occupant detection

Due to the outbreak of COVID-19, an increased risk of airborne transmission has been experienced in buildings, particularly in confined public places. The need for ventilation as a means of infection prevention has become more pronounced given that some basic precautions (like wearing masks) are no longer mandatory. However, ventilating the space as a whole (e.g., using a unified ventilation rate) may lead to situations where there is either insufficient or excessive ventilation in localized areas, potentially resulting in localized virus accumulation or large energy consumption. It is of urgent need to investigate real-time control of ventilation systems based on local demands of the occupants to strike a balance between infection risk and energy saving. In this work, a zonal demand-controlled ventilation (ZDCV) strategy was proposed to optimize the ventilation rates in sub-zones. A camera-based occupant detection method was developed to detect occupants (with eight possible locations in sub-zones denoted as ‘A’ to ‘H’). Linear ventilation model (LVM), dimension reduction, and artificial neural network (ANN) were integrated for rapid prediction of pollutant concentrations in sub-zones with the identified occupants and ventilation rates as inputs. Coordinated ventilation effects between sub-zones were optimized to improve infection prevention and energy savings. Results showed that rapid prediction models achieved an average prediction error of 6 ppm for CO2 concentration fields compared with the simulation under different occupant scenarios (i.e., occupant locations at ABH, ABCFH, and ABCDEFH). ZDCV largely reduced the infection risk to 2.8% while improved energy-saving efficiency by 34% compared with the system using constant ventilation rate. This work can contribute to the development of building environmental control systems in terms of pollutant removal, infection prevention, and energy sustainability.

A novel occupant-centric stratum ventilation system using computer vision: Occupant detection, thermal comfort, air quality, and energy savings

Traditional ventilation and air conditioning systems typically operate on a predetermined schedule with fixed operating parameters. Occupant-centric control (OCC) strategies have been proposed to reduce system operation energy consumption without sacrificing thermal comfort. Indoor occupancy detection in real time is a critical step in successfully implementing the OCC strategy. Thus, the deep learning-based computer vision method was adopted in the first step of the study, and the detection performance and camera position were analyzed in an office scenario. Next, the proposed OCC strategy was used to regulate the supply air parameters and outdoor air volume in stratum ventilation based on the monitored occupant number. The traditional static control strategy was then compared to two control strategies: constant air volume and variable air volume. Occupant detection performance results showed the mean NRMSD for the five most common relative positions of the occupants and camera was 0.1109, with sitting back to camera having the lowest accuracy. Subjective response results demonstrated that, when compared to the traditional control strategy, thermal comfort was improved by 43%–73%, perceived air quality was maintained at an acceptable level, CO2 concentration was less than 700 ppm, and energy could be saved by 2.3%–8.1%. Furthermore, the lower the occupancy, the greater the improvement in comfort and the greater the energy savings. This research focused on how the stratum ventilation system responds to dynamic changes in occupancy and provided insights into reducing unnecessary energy waste while maintaining comfort.

An Improved ANN Approach for Occupancy Detection of A Smart Building

Building energy performance can be improved with a reduction in energy consumption. The heating and cooling loads of a building are important factors to consider in the field of energy conservation. It is possible to estimate energy consumption by predicting the presence of occupants in a room based on information provided by the HVAC (Heating, Ventilation, and Air Conditioning) system using standard information. Temperature, humidity, light, and CO 2 levels from various sensors are taken as input parameters. In addition, the output of the network is programmed to be "0" when the building is not occupied and "1" when the building is occupied for the purpose of occupant detection. Pattern recognition using an Elman back propagation network is being proposed for occupancy detection. The data sets were used for training and testing (with the office door open and closed) the models during occupancy. The proposed ANN-based method is trained and tested and was found to be more effective, with an accuracy of 98.5% and 97.5% in cases of closed and opened doors, respectively.

Machine learning-based indoor localization and occupancy estimation using 5G ultra-dense networks

Nowadays, mobile applications need the location of the running devices to operate properly. This has increased the interest in indoor localization. Furthermore, the ability to sense mobile devices in indoor environments opens the door for building occupancy-count estimation. Studies have shown that occupant's detection and building occupancy-count estimation can be utilized to improve the efficiency of building operation and management. This research introduces new models to study the performance of such indoor localization and building occupancy-count estimation using the available technological advances in 5G Ultra-Dense Networks (UDNs). We propose an algorithm to collect the Received Signal Strength Indicator (RSSI) from User Equipments (UEs) and use it to build a fingerprinting database. We then use Machine Learning (ML) to estimate the location of the UEs in buildings from their RSSI values. Detecting users in the building is treated as a binary-classification problem. We then use various ML algorithms to build models for indoor occupancy-count estimation. Finally, the localization of users is used to estimate occupancy in specific sections of the building. The simulation results show that UDNs can provide accurate indoor localization, occupancy-count estimation in a building and in parts within the building.

Survey of Exposure to RF Electromagnetic Fields in the Connected Car

Future vehicles will be increasingly connected to enable new applications and improve safety, traffic efficiency and comfort, through the use of several wireless access technologies, ranging from vehicle-to-everything (V2X) connectivity to automotive radar sensing and Internet of Things (IoT) technologies for intra-car wireless sensor networks. These technologies span the radiofrequency (RF) range, from a few hundred MHz as in intra-car network of sensors to hundreds of GHz as in automotive radars used for in-vehicle occupant detection and advanced driver assistance systems. Vehicle occupants and road users in the vicinity of the connected vehicle are thus daily immersed in a multi-source and multi-band electromagnetic field (EMF) generated by such technologies. This paper is the first comprehensive and specific survey about EMF exposure generated by the whole ensemble of connectivity technologies in cars. For each technology we describe the main characteristics, relevant standards, the application domain, and the typical deployment in modern cars. We then extensively describe the EMF exposure scenarios resulting from such technologies by resuming and comparing the outcomes from past studies on the exposure in the car. Results from past studies suggested that in no case EMF exposure was above the safe limits for the general population. Finally, open challenges for a more realistic characterization of the EMF exposure scenario in the connected car are discussed.

A framework for occupancy detection and tracking using floor-vibration signals

In sensed buildings, information related to occupant movement helps optimize important functionalities such as caregiving, energy management, and security enhancement. Typical sensing approaches for occupant tracking rely on mobile devices and cameras. These systems compromise the privacy of building occupants and may affect their behavior. Occupant detection and tracking using floor-vibration measurements that are induced by footsteps is a non-intrusive and inexpensive sensing method. Detecting the presence of occupants on a floor is challenging due to ambient noise that may mask footstep-induced floor vibrations. In addition, spurious events such as door closing and falling objects may produce vibrations that are similar to footstep impacts. These events have to be detected and disregarded. Tracking occupants is complicated due to uncertainties associated with walking styles, walking speed, shoe type, health, and mood. Also, spatial variation in structural behavior of floor slabs adds ambiguity to the task of occupant tracking, which cannot be addressed using data-driven strategies alone. In this paper, a framework for occupant detection and tracking is developed. Occupant detection is carried out based on signal information. This method outperforms existing threshold-based methods. Support-vector-machine classifiers, trained with time and frequency-domain features, successfully distinguish footsteps from spurious events and determine the number of occupants walking simultaneously. A model-based data-interpretation approach is used for occupant tracking. Structural-mechanics models are used to identify a population of possible occupant locations and trajectories. Up to two occupants can be tracked by accommodating systematic bias and uncertainties from sources such as modeling assumptions and variability in walking gaits. A hybrid framework for occupant detection and tracking that combines model-free approaches for occupancy detection with structural behavior models for tracking is developed and tested on two full-scale case studies. These studies successfully validate the utility of the framework for buildings having sparse sensor configurations that measure floor vibrations.

Using footstep-induced vibrations for occupant detection and recognition in buildings

Occupant detection and recognition support functional goals such as security, healthcare, and energy management in buildings. Typical sensing approaches, such as smartphones and cameras, undermine the privacy of building occupants and inherently affect their behavior. To overcome these drawbacks, a non-intrusive technique using floor-vibration measurements, induced by human footsteps, is outlined. Detection of human-footstep impacts is an essential step to estimate the number of occupants, recognize their identities and provide an estimate of their probable locations. Detecting the presence of occupants on a floor is challenging due to ambient noise that may mask footstep-induced floor vibrations. Also, signals from multiple occupants walking simultaneously overlap, which may lead to inaccurate event separation. Signals corresponding to events, once extracted, can be used to identify the number of occupants and their locations. Spurious events such as door closing, chair dragging and falling objects may produce vibrations similar to footstep-impacts. Signals from such spurious events have to be discarded as outliers to prevent inaccurate interpretations of floor vibrations for occupant detection. Walking styles differ among occupants due to their anatomies, walking speed, shoe type, health and mood. Thus, footstep-impact vibrations from the same person may vary significantly, which adds uncertainty and complicates occupant recognition. In this paper, efficient strategies for event-detection and event-signal extraction have been described. These strategies are based on variations in standard deviations over time of measured signals (using a moving window) that have been filtered to contain only low-frequency components. Methods described in this paper for event detection and event-signal extraction perform better than existing threshold-based methods (fewer false positives and false negatives). Support vector machine classifiers are used successfully to distinguish footsteps from other events and to determine the number of occupants on a floor. Convolutional neural networks help recognize the identity of occupants using footstep-induced floor vibrations. The utility of these strategies for footstep-event detection, occupant counting, and recognition is validated successfully using two full-scale case studies.

Convolutional Neural Network-Based In-Vehicle Occupant Detection and Classification Method using Second Strategic Highway Research Program Cabin Images

This paper describes an approach for automatic detection and localization of drivers and passengers in automobiles using in-cabin images. We used a convolutional neural network (CNN) framework and conducted experiments based on the Faster R-CNN and Cascade R-CNN detectors. Training and evaluation were performed using the Second Strategic Highway Research Program (SHRP 2) naturalistic dataset. In SHRP 2, the cabin images have been blurred to maintain privacy. After detecting occupants inside the vehicle, the system classifies each occupant as driver, front-seat passenger, or back-seat passenger. For one SHRP 2 test set, the system detected occupants with an accuracy of 94.5%. Those occupants were correctly classified as front-seat passenger with an accuracy of 97.3%, as driver with 99.5% accuracy, and as back-seat passenger with 94.3% accuracy. The system performed slightly better for daytime images than for nighttime images. Unlike previous work, this method is capable of presence classification and location prediction of occupants. By fine-tuning the object detection model, there is also significant improvement in detection accuracy as compared with pretrained models. The study also provides a fully annotated dataset of in-cabin images. This work is expected to facilitate research involving interactions between drivers and passengers, particularly related to driver attention and safety.

A Framework for Detecting Vehicle Occupancy Based on the Occupant Labeling Method

High-occupancy vehicle (HOV) lanes or congestion toll discount policies are in place to encourage multipassenger vehicles. However, vehicle occupancy detection, essential for implementing such policies, is based on a labor-intensive manual method. To solve this problem, several studies and some companies have tried to develop an automated detection system. Due to the difficulties of the image treatment process, those systems had limitations. This study overcomes these limits and proposes an overall framework for an algorithm that effectively detects occupants in vehicles using photographic data. Particularly, we apply a new data labeling method that enables highly accurate occupant detection even with a small amount of data. The new labeling method directly labels the number of occupants instead of performing face or human labeling. The human labeling, used in existing research, and occupant labeling, this study suggested, are compared to verify the contribution of this labeling method. As a result, the presented model’s detection accuracy is 99% for the binary case (2 or 3 occupants or not) and 91% for the counting case (the exact number of occupants), which is higher than the previously studied models’ accuracy. Basically, this system is developed for the two-sided camera, left and right, but only a single side, right, can detect the occupancy. The single side image accuracy is 99% for the binary case and 87% for the counting case. These rates of detection are also better than existing labeling.

Monitoring of Occupant States in Autonomous Vehicles Using Capacitance-Sensing Imaging

The increasing popularity of autonomous vehicles has heightened the need to deploy robust systems that guarantee the safety and comfort of vehicle occupants. Such systems are required to monitor the presence and posture of occupants in real-time. This paper proposes a novel system that combines capacitive sensing with machine learning to achieve real-time occupant detection and posture recognition. The main constituent of the system is a capacitance-sensing mat that spans both the seat base (SB) and the backrest (BR) of a typical vehicle seat. The mat is connected to a sensing circuitry that continuously monitors variations in capacitance inside the mat to generate grayscale capacitance-sensing images (CSIs). The CSIs are real-time pictorial representations of the variations in capacitance caused within the mat due to changes in the position and the posture of occupants. The real-time CSIs then serve as inputs to a k-nearest-neighbors-based (k-NN) classifier that is capable of identifying different posture classes. The classifier is pre-trained using a previously acquired database consisting of a collection of CSIs belonging to known posture classes. Subsequently, the performance of the classifier is validated, and the system is deployed for real-time occupant detection and posture recognition.

UMoDT: an unobtrusive multi-occupant detection and tracking using robust Kalman filter for real-time activity recognition

Human activity recognition (HAR) is an important branch of human-centered research. Advances in wearable and unobtrusive technologies offer many opportunities for HAR. While much progress has been made in HAR using wearable technology, it still remains a challenging task using unobtrusive (non-wearable) sensors. This paper investigates detection and tracking of multi-occupant HAR in a smart-home environment, using a novel low-resolution Thermal Vision Sensor (TVS). Specifically, the research presents the development and implementation of a two-step framework, consisting of a Computer Vision-based method to detect and track multiple occupants combined with Convolutional Neural Network (CNN)-based HAR. The proposed algorithm uses frame difference over consecutive frames for occupant detection, a set of morphological operations to refine identified objects, and features are extracted before applying a Kalman filter for tracking. Laterally, a 19-layer CNN architecture is used for HAR and afterward the results from both methods are fused using time interval-based sliding window. This approach is evaluated through a series of experiments based on benchmark Thermal Infrared datasets (VOT-TIR2016) and multi-occupant data collected from TVS. Results demonstrate that the proposed framework is capable of detecting and tracking 88.46% of multi-occupants with a classification accuracy of 90.99% for HAR.

Occupant counting modelling for intelligent buildings based on data from multiple WiFi sniffers

Knowing the occupancy information in each room can create energy saving through intelligent control of HVAC systems. In this paper, a classification-based occupant counting method using multiple WiFi sniffers is proposed firstly to get a coarse estimation of occupancy. Then, to deal with the false negative problem, i.e., those occupants who do not carry a smartphone or if the WiFi module is not enabled and cannot be counted, a p-persistent frequent itemsets with 1-right-hand-side (RHS)-based occupant correction algorithm is further proposed to improve the occupant detection accuracy using association analysis. Finally, our methods are validated through real experiments. Results show that our classification-based occupant detection method using multiple WiFi sniffers outperforms the 1-WiFi-sniffer-based method, and the association analysis-based correction algorithm can improve the accuracy because it can see occupants in buildings that the naive WiFi-based occupant detection method cannot see, which makes it suffice to be a viable approach to occupant estimation for intelligent buildings.

Model-based characterisation of vehicle occupants using a depth camera

Due to recent advances in sensing technologies, modern vehicle occupant classification systems enable personalised vehicle experiences and adaptive occupant crash protection. However, most systems are limited to occupant detection and simple classification, and thus, accurate estimation of body characteristics is needed to support more advanced occupant classification. This paper presents a model-based characterisation method for vehicle occupants using a 3D depth camera. This method automatically estimates standard anthropometric data of an occupant such as stature and weight along with the body shape by fitting a statistical body shape model to depth image data. The system is even robust to a wide range of clothing and is capable of generating accurate results. A variety of other algorithms were developed to improve the fitting result, including seat geometry detection and head location estimation. The new capability has a range of potential applications for improving occupant safety and providing an optimised interior configuration for the occupant.

Step-Level Occupant Detection across Different Structures through Footstep-Induced Floor Vibration Using Model Transfer

AbstractThis paper presents a floor-vibration-based step-level occupant-detection approach that enables detection across different structures through model transfer. Detecting the occupants through...

Development of a Child Detection System with Artificial Intelligence Using Object Detection Method

The issue of children dying due to vehicular heatstroke has raised significant concerns of public interest. The advancement of artificial intelligence (AI) technology, particularly in image classification and object detection, could be applied to overcome the current flaws of the vehicular occupant detection devices that often failed to serve as a triggering system to caretakers. In this paper, a technique for child detection with transfer learning is proposed. A real-time child detection system that consisted of a camera as an input medium, a classifier to detect the presence of a child and a triggering system in audio and visual forms was established. The modern convolutional object detector, SSD Mobilenet v1 was trained with Microsoft Common Objects in Context (MS COCO) dataset as a starting point of the training process. The model was then assessed and retrained to possess the ability to classify human into an adult or a child. The accuracy of the model was measured by counting the percentage of pixels labelled correctly per class. Based on the mean Average Precision (mAP), the detection system achieved an overall precision of 0.969 and the experimental results obtained showed a precision of 0.883, giving an error of less than ten percent.

A novel occupancy detection solution using low-power IR-FPA based wireless occupancy sensor

Significant energy savings can be achieved by operating heating, ventilation, and air conditioning control systems with indoor occupancy measurement information. This paper presents a novel plug-and-play occupancy sensing method which will enable temporal minimization of building energy consumption to meet building usage behavior without privacy concerns. The proposed wireless occupancy sensing platform is based on long-wave infra-red (LWIR) focal-plane arrays (FPAs), or thermal imagers, that detect thermal energy rather than visible light. We developed an advanced sensor package consisting of multiple thermal imagers with low-cost optical enhancements to increase field of view and increase sensitivity to occupant detection (filtering building clutter). These imagers can be coupled with radio frequency and ultrasonic-based radar to enhance data collection at key occupant zone boundaries to improve accuracy. Standard filtering and estimation techniques from the image processing and computer vision communities are introduced to overcome the accuracy issues suffered by traditional PIR based sensing, especially when occupants remain relatively still. Accurate low-level counting of individuals can be achieved with minimal impact on privacy. The proposed occupancy detection method can work as a retrofit to enable real-time understanding of the building operational state and usage behavior. Sensor data obtained from real test building zones was used to test the efficacy of the method.

Robust Vehicle Occupant Detection based on RGB-Depth-Thermal Camera

Robust Vehicle Occupant Detection based on RGB-Depth-Thermal Camera

Turning a pyroelectric infrared motion sensor into a high-accuracy presence detector by using a narrow semi-transparent chopper

Pyroelectric infrared (PIR) sensors are the most popular presence detectors. However, their applications are limited to motion detection only, as the pyroelectric sensing element is only sensitive to radiated heat power fluctuation. This letter reports a chopped PIR (C-PIR) presence sensor, capable of detecting both the stationary and the moving occupants at high accuracy by introducing a narrow semi-transparent optical chopper to shutter the incident radiation periodically. More importantly, even though the narrow chopper can efficiently shutter the unidirectional infrared (IR) power radiated by stationary occupants, it has near-zero influence on the omnidirectional power radiated from the ambient environment. Therefore, the voltage difference generated by the C-PIR sensor between occupied and unoccupied scenarios can be directly used for identifying the presence of stationary occupants with high-accuracy. For moving occupant detection, the optical chopper can barely affect the variation of the radiated power and thus lead to the same high-accuracy detection as the on-board PIR. An approximate theoretical model is created to illustrate the physics and dynamics of IR radiation shuttered by the optical chopper. Parametric studies, experimental characterization, and empirical analysis were performed to obtain the optimal setting of the chopper in terms of its thickness and chopping frequency. Experimental results reveal that the C-PIR sensor can detect stationary occupants for up to 4.0 m and moving occupants for up to 8.0 m, while maintaining an accuracy of 100%. The field of view of the C-PIR was found at 110° horizontally and 90° vertically, consistent with that of the on-board PIR sensor.