Occupancy Detection Research Articles

Background-foreground segmentation for interior sensing in automotive industry

To ensure safety in automated driving, the correct perception of the situation inside the car is as important as its environment. Thus, seat occupancy detection and classification of detected instances play an important role in interior sensing. By the knowledge of the seat occupancy status, it is possible to, e.g., automate the airbag deployment control. Furthermore, the presence of a driver, which is necessary for partially automated driving cars at the automation levels two to four can be verified. In this work, we compare different statistical methods from the field of image segmentation to approach the problem of background-foreground segmentation in camera based interior sensing. In the recent years, several methods based on different techniques have been developed and applied to images or videos from different applications. The peculiarity of the given scenarios of interior sensing is, that the foreground instances and the background both contain static as well as dynamic elements. In data considered in this work, even the camera position is not completely fixed. We review and benchmark three different methods ranging, i.e., Gaussian Mixture Models (GMM), Morphological Snakes and a deep neural network, namely a Mask R-CNN. In particular, the limitations of the classical methods, GMM and Morphological Snakes, for interior sensing are shown. Furthermore, it turns, that it is possible to overcome these limitations by deep learning, e.g. using a Mask R-CNN. Although only a small amount of ground truth data was available for training, we enabled the Mask R-CNN to produce high quality background-foreground masks via transfer learning. Moreover, we demonstrate that certain augmentation as well as pre- and post-processing methods further enhance the performance of the investigated methods.

Non-Intrusive Room Occupancy Prediction Performance Analysis Using Different Machine Learning Techniques

Recent advancements in the Internet of Things and Machine Learning techniques have allowed the deployment of sensors on a large scale to monitor the environment and model and predict individual thermal comfort. The existing techniques have a greater focus on occupancy detection, estimations, and localization to balance energy usage and thermal comfort satisfaction. Different sensors, actuators, and analytic data methods are often non-invasively utilized to analyze data from occupant surroundings, identify occupant existence, estimate their numbers, and trigger the necessary action to complete a task. The efficiency of the non-invasive strategies documented in the literature, on the other hand, is rather poor due to the low quality of the datasets utilized in model training and the selection of machine learning technology. This study combines data from camera and environmental sensing using interactive learning and a rule-based classifier to improve the collection and quality of the datasets and data pre-processing. The study compiles a new comprehensive public set of training datasets for building occupancy profile prediction with over 40,000 records. To the best of our knowledge, it is the largest dataset to date, with the most realistic and challenging setting in building occupancy prediction. Furthermore, to the best of our knowledge, this is the first study that attained a robust occupancy count by considering a multimodal input to a single output regression model through the mining and mapping of feature importance, which has advantages over statistical approaches. The proposed solution is tested in a living room with a prototype system integrated with various sensors to obtain occupant-surrounding environmental datasets. The model’s prediction results indicate that the proposed solution can obtain data, and process and predict the occupants’ presence and their number with high accuracy values of 99.7% and 99.35%, respectively, using random forest.

Non-Intrusive Detection of Occupants’ On/Off Behaviours of Residential Air Conditioning

Understanding occupants’ behaviours (OBs) of heating and cooling use in dwellings is essential for effectively promoting occupants’ behavioural change for energy saving and achieving efficient demand response operation. Thus, intensive research has been conducted on data collection, statistical analysis, and modelling of OBs. However, the majority of smart metres currently deployed worldwide monitor only the total household consumption rather than appliance-level load. Therefore, estimating the turn-on/off state of specific home appliances from the measured household total electricity referred to as non-intrusive load monitoring (NILM), has gained research attention. However, the current NILM methods overlook the specific features of inverter-controlled heat pumps (IHPs) used for space heating/cooling; thus, they are unsuitable for detecting OBs. This study presents a rule-based method for identifying the occupants’ intended operation states of IHPs based on a statistical analysis of load data monitored at 423 dwellings. This method detects the state of IHPs by subtracting the power of sequential-operation appliances other than IHPs from the total household power. Three time-series characteristics, including the durations of power-on/off states and power differences between power-off/on states, were used for this purpose. The performance of the proposed method was validated, indicating an F-score of 0.834.

Indoor Occupancy Detection Based on Environmental Data Using CNN-XGboost Model: Experimental Validation in a Residential Building

Indoor occupancy prediction can play a vital role in the energy-efficient operation of building engineering systems and maintaining satisfactory indoor climate conditions at the lowest possible energy use by operating these systems on the basis of occupancy data. Many methods have been proposed to predict occupancy in residential buildings according to different data types, e.g., digital cameras, motion sensors, and indoor climate sensors. Among these proposed methods, those with indoor climate data as input have received significant interest due to their less intrusive and cost-effective approach. This paper proposes a deep learning method called CNN-XGBoost to predict occupancy using indoor climate data and compares the performance of the proposed method with a range of supervised and unsupervised machine learning algorithms plus artificial neural network algorithms. The comparison is performed using mean absolute error, confusion matrix, and F1 score. Indoor climate data used in this work are CO2, relative humidity, and temperature measured by sensors for 13 days in December 2021. We used inexpensive sensors in different rooms of a residential building with a balanced mechanical ventilation system located in northwest Copenhagen, Denmark. The proposed algorithm consists of two parts: a convolutional neural network that learns the features of the input data and a scalable end-to-end tree-boosting classifier. The result indicates that CNN-XGBoost outperforms other algorithms in predicting occupancy levels in all rooms of the test building. In this experiment, we achieved the highest accuracy in occupancy detection using inexpensive indoor climate sensors in a mechanically ventilated residential building with minimum privacy invasion.

Edge–Artificial Intelligence-Powered Parking Surveillance With Quantized Neural Networks

The rapid development of urbanization has raised challenges for existing parking facilities to serve increasing parking demand. Being constrained by the limited urban land resources for newly constructed parking facilities, improving the efficiency of existing parking infrastructures relies more on an advanced parking management strategy. Video-based parking surveillance technology, with abundant information, easy installation, and powerful algorithms, is the most popular method to provide real-time parking information as the required data feed for a parking management system. Meanwhile, deep learning-based algorithms gradually replace traditional computer vision methods for video processing since the detection accuracy has been increased considerably by learning fine-grained features. However, the excessive computational complexity of deep learning-based algorithms occupies considerable computational resources, which certainly hurts the entire system’s efficiency. Due to the limited computing power of edge devices, most parking surveillance systems deploy video processing algorithms on a server or cloud platform, which raises concerns about data transmission latency and central computation pressure. Deploying efficient algorithms on an edge-side device is a potential solution to solve these problems. This article proposes an edge computing parking occupancy detection system with a quantized deep learning model. Model quantization is employed to boost the inference speed while maintaining accuracy. In addition, knowledge distillation is applied to improve the quantized model’s training performance. Experiments are conducted to demonstrate the model’s superiority compared to state-of-the-art algorithms and the feasibility of edge computing. The proposed method can improve the accuracy and efficiency of parking surveillance systems. It is a systematic solution for obtaining parking information with limited computational resources.

AI-big data analytics for building automation and management systems: a survey, actual challenges and future perspectives

In theory, building automation and management systems (BAMSs) can provide all the components and functionalities required for analyzing and operating buildings. However, in reality, these systems can only ensure the control of heating ventilation and air conditioning system systems. Therefore, many other tasks are left to the operator, e.g. evaluating buildings’ performance, detecting abnormal energy consumption, identifying the changes needed to improve efficiency, ensuring the security and privacy of end-users, etc. To that end, there has been a movement for developing artificial intelligence (AI) big data analytic tools as they offer various new and tailor-made solutions that are incredibly appropriate for practical buildings’ management. Typically, they can help the operator in (i) analyzing the tons of connected equipment data; and; (ii) making intelligent, efficient, and on-time decisions to improve the buildings’ performance. This paper presents a comprehensive systematic survey on using AI-big data analytics in BAMSs. It covers various AI-based tasks, e.g. load forecasting, water management, indoor environmental quality monitoring, occupancy detection, etc. The first part of this paper adopts a well-designed taxonomy to overview existing frameworks. A comprehensive review is conducted about different aspects, including the learning process, building environment, computing platforms, and application scenario. Moving on, a critical discussion is performed to identify current challenges. The second part aims at providing the reader with insights into the real-world application of AI-big data analytics. Thus, three case studies that demonstrate the use of AI-big data analytics in BAMSs are presented, focusing on energy anomaly detection in residential and office buildings and energy and performance optimization in sports facilities. Lastly, future directions and valuable recommendations are identified to improve the performance and reliability of BAMSs in intelligent buildings.

Research on Hotel Management Based on Internet of Things and Big Data Analysis

With one-tap check-ins, digital concierge services, voice-activated gadgets, chatbots, smart in-room technology, and advanced analytics, the hotel sector has been quickly embracing new technologies to meet and exceed consumer expectations and digitize the customer experiences. The traditional hotel management with internet-based customer service could not handle dynamic real-time data efficiently due to increased data volume. Thus, this work analyzes hotel management practices with the internet of things (IoT) and big data. The IoT and big data significantly impact the guest experience since businesses can provide consumers with unique services to their needs. Automated check-in and checkout, pre-booking, registration, and user-chosen payment methods are just a few self-services that improve the visitor experience. For enhancing visitor satisfaction and offering tailored services, this paper looks at how IoT and big data analytics can help the hotel sector. It moreover examines how IoT can be used within the business. This extended research finds excellent results in hotel management through IoT and big data. A client occupancy detection model (CODM) simulation scenario finds the best detection accuracy of 97.51%.

A fusion framework for vision-based indoor occupancy estimation

Building occupancy information is essential for energy saving, comfort improvement, and security management. Existing vision-based indoor occupancy measurement methods have achieved remarkable progress; however, they mainly focus on single-vision situations, i.e., cameras at room entrances or interiors. These methods struggle to achieve high accuracy because of the complex indoor environments. For example, they often fail to detect occupants and generate many false positives. In this paper, to address these issues, we propose a novel fusion framework for occupancy detection and estimation based on two different perspectives. First, we design a head detection method combined with indoor scene knowledge to filter false positives and recover missed detection. Second, we propose a two-vision entrance counting method to refine the predicted results. Finally, we propose a cumulative error clearing strategy named dynamic Bayesian fusion (DBF), which integrates entrance counting and static estimation. Our framework achieves superior performance through ablation studies compared to existing methods on practical building surveillance videos, with occupancy estimation SCOREs of 99.2%, 98.5%, and 94.9%. Our framework can clear cumulative errors and stabilize estimation results. Practical experiments validate its potential for building energy saving and comfort improvement. The code is available at https://github.com/kailaisun/FFO.

A health monitoring system with posture estimation and heart rate detection based on millimeter-wave radar

Millimeter-wave radar is a device gaining popularity in many applications such as occupancy detection and people counting, vital sign detection and robotics. It has recently been used for human detection and tracking within home health care as a potential application area. Furthermore, increasing number of researchers utilize mmWave radar for heart rate detection. In this paper, we introduce a new home health monitoring system based on millimeter-wave radar, which is composed of posture estimation and heart rate detection. We focus on categorization of postures (standing, sitting and lying) for a single person in indoor environments. When the person is estimated as lying down, our system can further detect the heart rate. We establish a new point cloud dataset and show that, with simple algorithms employed, the system has an accuracy of around 99% for posture estimation and approximately 91% accuracy for detecting heart rate with only a single millimeter-wave radar.

Automatic detection of indoor occupancy based on improved YOLOv5 model.

Indoor occupancy detection is essential for energy efficiency control and Coronavirus Disease 2019 traceability. The number and location of people can be accurately identified and determined through classroom surveillance video analysis. This information is used to manage environmental equipment such as HVAC and lighting systems to reduce energy use. However, the mainstream one-stage YOLO algorithm still uses an anchor-based mechanism and couples detection heads to predict. This results in slow model convergence and poor detection performance for densely occluded targets. Therefore, this paper proposed a novel decoupled anchor-free VariFocal loss convolutional network algorithm DFV-YOLOv5 for occupancy detection to tackle these problems. The proposed method uses the YOLOv5 algorithm as a baseline. It uses the anchor-free mechanism to reduce the number of design parameters needing heuristic tuning. Afterwards, to reduce the coupling of the model, speed up the model’s convergence ability, and improve the model detection performance, the detection head is decoupled based on the YOLOv5 model. It can resolve the conflict between classification and regression tasks. In addition, we use the VariFocal loss to assign more weights to difficult data points to optimize the class imbalance problem and use the training target q to measure positive samples, treating positive and negative samples asymmetrically. The total loss function is redesigned, the L_{1} loss is increased, and the ablation experiment verifies the effect of the improved loss. By applying a hybrid activation function of the sigmoid linear unit and rectified linear unit, we improved the model’s nonlinear representation and reduced the model’s inference time. Finally, a classroom dataset was constructed to validate the occupancy detection performance of the model. The proposed model was compared with mainstream target detection models regarding average mean precision, memory allocation, execution time, and the number of parameters on the VOC2012, CrowdHuman and self-built datasets. The experimental results show that the method significantly improves the detection accuracy and robustness, shortens the inference time, and proves the practicality of the algorithm in occupancy detection compared with the mainstream target detection model and related variants of the model.

MRI Liver Image Assisted Diagnosis Based on Improved Faster R-CNN

In response to challenges in liver occupancy such a variety of types and manifestations and difficulties in differentiating benign and malignant ones, this paper takes liver images of enhanced MRI scan as the research object, targets on the detection and identification of liver occupancy lesion areas and determining if it is benign or malignant. Accordingly, the paper proposes an auxiliary diagnosis method for liver image combining deep learning and MRI medical imaging. The first step is to establish a reusable standard dataset for MRI liver occupancy detection by pre-processing, image denoising, lesion annotation and data augmentation. Then it improves the classical region-based convolutional neural network (R-CNN) algorithm Faster R-CNN by incorporating CondenseNet feature extraction network, custom-designed anchor size and transfer learning pre-training. This is to further improve the detection accuracy and benign and malignant classification performance of liver occupancy. Experiments show that the improved model algorithm can effectively identify and localise liver occupancies in MRI images, and achieves a mean average precision (mAP) of 0.848 and an Area Under the Curve (AUC) of 0.926 on the MRI standard dataset. This study has important research significance and application value for reducing manual misses and misdiagnosis and improving the early clinical diagnosis rate of liver cancer.

Indoor fire detection utilizing computer vision-based strategies

Fires are an ever-increasing risk in the world for both indoor and outdoor environments. Current technologies for detection in indoor environments are smoke and flame detectors. However, these detectors have several limitations during both the ignition phase of a fire and propagation. These systems cannot detect an exact position of the fire nor how the fire is spreading, or its size, all of which is necessary information for fire services when dealing with these incidents. A potential solution is to use artificial intelligence techniques such as computer vision, which has shown the potential to detect and recognize objects and activities in indoor spaces. This study aims to develop a vision-based indoor fire and smoke detection system. Existing models based on the Faster R–CNN Inception V2 and the SSD MobileNet V2 models were explored and adopted in this work. This study utilized small training and testing datasets (for indoor specific fire cases) of varying pixel density images. Initial evaluation of the approach was carried out by testing both models on videos, including a mock-up bedroom and living room and a CCTV video of office space. Both high-density smoke environments and flame density scenarios were recognized. The promising results were achieved from using only 480 training images. Despite the success achieved by the Faster R–CNN Inception V2, the SSD MobileNet V2 model showed low accuracy and missed detection results. Future works can focus on integrating this with our previously developed approach, such as occupancy detection, enhancing training data and models, using more advanced detection models, and integrating the proposed approach with the fire fighting and HVAC control systems.

Hybrid cooperative spectrum sensing for improving cognitive power line communication systems

This paper introduces a hybrid cooperative spectrum sensing technique that exploits the existing diversity between power line and wireless media with the purpose of improving spectrum sensing functionalities in power line communication (PLC) systems, which operate in the frequency band between 1.7 and 100 MHz. To evaluate the performance of the proposed technique, a data set obtained from a measurement campaign is analyzed, and a feature evaluation reveals that the signal energy is the most relevant feature of all extracted ones. Also, the performance evaluation of the proposed technique shows that the exploitation of the existing diversity between signals acquired from both electric power grid and air can remarkably improve the detection of spectrum occupancy or holes in comparison to the sole use of one of them.

PORÓWNANIE METOD DETEKCJI ZAJĘTOŚCI WIDMA RADIOWEGO Z WYKORZYSTANIEM UCZENIA FEDERACYJNEGO Z ORAZ BEZ WĘZŁA CENTRALNEGO

Dynamic spectrum access systems typically require information about the spectrum occupancy and thus the presence of other users in order to make a spectrum al-location decision for a new device. Simple methods of spectrum occupancy detection are often far from reliable, hence spectrum occupancy detection algorithms supported by machine learning or artificial intelligence are often and successfully used. To protect the privacy of user data and to reduce the amount of control data, an interesting approach is to use federated machine learning. This paper compares two approaches to system design using federated machine learning: with and without a central node.

Deep and transfer learning for building occupancy detection: A review and comparative analysis

The building internet of things (BIoT) is quite a promising concept for curtailing energy consumption, reducing costs, and promoting building transformation. Besides, integrating artificial intelligence (AI) into the BIoT is essential for data analysis and intelligent decision-making. Thus, data-driven approaches to infer occupancy patterns usage are gaining growing interest in BIoT applications. Typically, analyzing big occupancy data gathered by BIoT networks helps significantly identify the causes of wasted energy and recommend corrective actions. Within this context, building occupancy data aids in the improvement of the efficacy of energy management systems, allowing the reduction of energy consumption while maintaining occupant comfort. Occupancy data might be collected using a variety of devices. Among those devices are optical/thermal cameras, smart meters, environmental sensors such as carbon dioxide (CO2), and passive infrared (PIR). Even though the latter methods are less precise, they have generated considerable attention owing to their inexpensive cost and low invasive nature. This article provides an in-depth survey of the strategies used to analyze sensor data and determine occupancy. The article’s primary emphasis is on reviewing deep learning (DL), and transfer learning (TL) approaches for occupancy detection. This work investigates occupancy detection methods to develop an efficient system for processing sensor data while providing accurate occupancy information. Moreover, the paper conducted a comparative study of the readily available algorithms for occupancy detection to determine the optimal method in regards to training time and testing accuracy. The main concerns affecting the current occupancy detection system in terms of privacy and precision were thoroughly discussed. For occupancy detection, several directions were provided to avoid or reduce privacy problems by employing forthcoming technologies such as edge devices, Federated learning, and Blockchain-based IoT.

Stability of crop pollinator occurrence is influenced by bee community composition

Bees provide a vital ecosystem service to agriculture by contributing to the pollination of many leading global crops. Human wellbeing depends not only on the quantity of agricultural yields, but also on the stability and resilience of crop production. Yet a broad understanding of how the diversity and composition of pollinator communities may influence crop pollination service has previously been hindered by a scarcity of standardized data. We used outputs from Bayesian occupancy detection models to examine patterns in the inter-annual occupancy dynamics of the bee pollinator communities of four contrasting crops (apples, field bean, oilseed and strawberries) in Great Britain between 1985 and 2015. We compared how the composition and species richness of different crop pollinator communities may affect the stability of crop pollinator occurrence. Across the four crops, we found that the inter-annual occupancy dynamics of the associated pollinator communities tended to be more similar in smaller communities with closely related pollinator species. Our results indicate that crop pollinator communities composed of a small number of closely related bee species show greater variance in mean occupancy compared to crops with more diverse pollinator communities. Lower variance in the occurrence of crop pollinating bee species may lead to more stable crop pollination services. Finally, whilst our results initially indicated some redundancy within most crop pollinator communities, with no, or little, increase in the variance of overall mean occupancy when species were initially removed, this was followed by a rapid acceleration in the variance of crop pollinator occurrence as each crop's bee pollinator community was increasingly depreciated. High inter-annual variations in pollination services have negative implications for crop production and food security. High bee diversity could ensure more stable and resilient crop pollination services, yet current agri-environment schemes predominantly benefit a limited suite of common species. Management may therefore benefit from targeting a wider diversity of solitary species in order to safeguard crop pollination service in the face of increasing environmental change.

Enhancing the detection performance of a vision-based occupancy detector for buildings

Occupant behaviour is one of the key parameters that significantly impact the operation of heating, ventilation and air-conditioning (HVAC) systems and the energy performance of buildings. Detailed occupancy information can improve HVAC operation and utilisation of building spaces. Strategies such as vision-based occupancy detection and recognition have recently garnered much interest. This study investigated the performance of a vision-based deep learning detection technique for enhancing building system operations and energy performances. The model used was the Faster region-based convolutional neural network with Inception V2. Two occupancy detection model configurations were developed, tested and evaluated. Both models were analysed based on the application of the detector within a selected case study building, along with an evaluation based on different evaluation metrics. The results suggest the occupancy detector using model 1 provided an overall accuracy of 95.23%, while model 2 (the occupancy activity detector) provided an accuracy of 89.37%. In addition, the F 1 score, which represents the harmonic mean between the precision and recall values, was 0.9756 for model 1 and 0.8298 for model 2. Building energy simulation was used to evaluate and compare the impact of such an approach on indoor occupancy heat gains. The study highlighted the potential of the detection approaches, but further development is necessary, including optimisation of the model, full integration with HVAC controls and further model training and field testing.

MPSN: Motion-aware Pseudo-Siamese Network for indoor video head detection in buildings

Indoor video head detection is an essential component of building occupancy detection. While deep learning models have achieved remarkable progress in general object detection, their performance is limited in complex indoor scenes. Indoor surveillance videos often contain cluttered background objects, among which heads have small scales and diverse poses. In this paper, we propose Motion-aware Pseudo-Siamese Network (MPSN), an end-to-end approach that leverages head motion information to guide deep learning models to extract effective head features in indoor scenarios. By taking the pixel-wise difference between adjacent frames as the auxiliary input, MPSN effectively enhances human head motion information and removes irrelevant background objects. Its performance is compared with that of existing methods. Compared with existing methods, MPSN achieves superior performance on two indoor video datasets. It successfully suppresses static background objects and highlights the moving instances, especially human heads in indoor videos. Different methods to capture head motion also were compared. MPSN demonstrates simplicity and flexibility compared to other methods. We validate its robustness through adversarial experiments with a mathematical solution of small perturbations for robust model selection. To confirm its potential in building control systems, we apply MPSN to occupancy counting. The code is available at https://github.com/pl-share/MPSN.

Comparison of different training data sets from simulation and experimental measurement with artificial users for occupancy detection — Using machine learning methods Random Forest and LASSO

The applications for occupancy detection range from controlling building automation and systems, determining heat transfer coefficients and even assessing the risk of infection in rooms. Studies in the literature use various statistical models, physical models and machine learning techniques to detect occupancy. All these methods require data for training the occupancy detection models. However, data generation is time-consuming and expensive. This study demonstrates the feasibility of using simulated learning data. Using three different data sources, we tested the suitability of different methods for generating learning data. We conducted two experiments in two office spaces with a real user and an artificial user, and we generated a third data set using a building simulation model. In addition, this study compares two different machine learning approaches (Random Forest and LASSO) using environmental parameters. Both machine learning approaches could develop models with a sensitivity of at least 83% and a specificity of at least 97% with both training data sets. This work shows that it is possible to determine the presence in rooms using simulated data. The results compared to measured data were just slightly less accurate, and the added value due to the lower effort was considerable.

Automatic detection of vehicle occupancy and driver's seat belt status using deep learning

Automatic detection of vehicle occupancy and driver's seat belt status using deep learning