Faster Region-based Convolutional Neural Network Research Articles

Defect Detection Scheme for Key Equipment of Transmission Line for Complex Environment

Aiming at the difficulty in detecting defects of key equipment of transmission lines in small samples and complex environments, and the problems of low accuracy and unreliability in one-time detection using traditional deep learning-based methods, an image detection scheme combining optimized deep convolutional neural networks and Kalman filtering is proposed. The convolutional neural network architecture is based on Faster Region-based Convolutional Neural Networks (R-CNNs). First, the model backbone network is constructed by MobileNet, which effectively reduces the computational cost. Secondly, a soft nonmaximum suppression algorithm is integrated to solve the occlusion problem of target parts, and the context-aware ROI pooling layer replaces the original pooling layer, maintaining the original structure of small-sized components. Finally, the detection results are corrected twice by Kalman filtering to further improve the detection accuracy and reliability. The experimental results show that this method can realize the accurate detection of components in complex transmission line equipment, the mean Average Precision (mAP) reaches 91.10%, which is 11.05% higher than the original model, and the detection time of each picture is only 0.05 s. Compared with other detection algorithms under the same conditions, the comprehensive performance of the proposed method can be improved by 20%.

MULTIPLE OIL PAD DETECTION USING DEEP LEARNING

Abstract. Deep learning (DL) algorithms are widely used in object detection such as roads, vehicles, buildings, etc., in aerial images. However, the object detection task is still considered challenging for detecting complex structures, oil pads are one such example: due to its shape, orientation, and background reflection. A recent study used Faster Region-based Convolutional Neural Network (FR-CNN) to detect a single oil pad from the center of the image of size 256 × 256. However, for real-time applications, it is necessary to detect multiple oil pads from aerial images irrespective of their orientation. In this study, FR-CNN was trained to detect multiple oil pads. We cropped images from high spatial resolution images to train the model containing multiple oil pads. The network was trained for 100 epochs using 164 training images and tested with 50 images under 3 different categories. with images containing: single oil pad, multiple oil pad and no oil pad. The model performance was evaluated using standard metrics: precision, recall, F1-score. The final model trained for multiple oil pad detection achieved a weighted average for 50 images precision of 0.67, recall of 0.80, and f1 score of 0.73. The 0.80 recall score indicates that 80% of the oil pads were able to identify from the given test set. The presence of instances in test images like cleared areas, rock structures, and sand patterns having high visual similarity with the target resulted in a low precision score.

3cDe-Net: a cervical cancer cell detection network based on an improved backbone network and multiscale feature fusion

BackgroundCervical cancer cell detection is an essential means of cervical cancer screening. However, for thin-prep cytology test (TCT)-based images, the detection accuracies of traditional computer-aided detection algorithms are typically low due to the overlapping of cells with blurred cytoplasmic boundaries. Some typical deep learning-based detection methods, e.g., ResNets and Inception-V3, are not always efficient for cervical images due to the differences between cervical cancer cell images and natural images. As a result, these traditional networks are difficult to directly apply to the clinical practice of cervical cancer screening.MethodWe propose a cervical cancer cell detection network (3cDe-Net) based on an improved backbone network and multiscale feature fusion; the proposed network consists of the backbone network and a detection head. In the backbone network, a dilated convolution and a group convolution are introduced to improve the resolution and expression ability of the model. In the detection head, multiscale features are obtained based on a feature pyramid fusion network to ensure the accurate capture of small cells; then, based on the Faster region-based convolutional neural network (R-CNN), adaptive cervical cancer cell anchors are generated via unsupervised clustering. Furthermore, a new balanced L1-based loss function is defined, which reduces the unbalanced sample contribution loss.ResultBaselines including ResNet-50, ResNet-101, Inception-v3, ResNet-152 and the feature concatenation network are used on two different datasets (the Data-T and Herlev datasets), and the final quantitative results show the effectiveness of the proposed dilated convolution ResNet (DC-ResNet) backbone network. Furthermore, experiments conducted on both datasets show that the proposed 3cDe-Net, based on the optimal anchors, the defined new loss function, and DC-ResNet, outperforms existing methods and achieves a mean average precision (mAP) of 50.4%. By performing a horizontal comparison of the cells on an image, the category and location information of cancer cells can be obtained concurrently.ConclusionThe proposed 3cDe-Net can detect cancer cells and their locations on multicell pictures. The model directly processes and analyses samples at the picture level rather than at the cellular level, which is more efficient. In clinical settings, the mechanical workloads of doctors can be reduced, and their focus can be placed on higher-level review work.

Pavement Surface Defect Detection Using Mask Region-Based Convolutional Neural Networks and Transfer Learning

Pavement defect detection is critical for pavement maintenance and management. Meanwhile, the accurate and timely detection of pavement defects in complex backgrounds is a huge challenge for maintenance work. Therefore, this paper used a mask region-based convolutional neural network (Mask R-CNN) and transfer learning to detect pavement defects in complex backgrounds. Twelve hundred pavement images were collected, and a dataset containing corresponding instance labels of the defects was established. Based on this dataset, the performance of the Mask R-CNN was compared with faster region-based convolutional neural networks (Faster R-CNNs) under the transfer of six well-known backbone networks. The results confirmed that the classification accuracy of the two algorithms (Mask R-CNN and Faster R-CNN) was consistent and reached 100%; however, the average precision (AP) of the Mask R-CNN was higher than that of Faster R-CNNs. Meanwhile, the testing time of the models using a feature pyramid network (FPN) was lower than that of other models, which reached 0.21 s per frame (SPF). On this basis, the segmentation performance of the Mask R-CNN was further analyzed at three learning rates (LRs). The Mask R-CNN performed best with ResNet101 plus FPN as its backbone structure, and its AP reached 92.1%. The error rate of defect quantification was between 4% and 16%. It has an ideal detection effect on multi-object and multi-class defects on pavement surfaces, and the quantitative results of the defects can provide a reference for pavement maintenance personnel.

End-to-end deep learning framework for printed circuit board manufacturing defect classification

We report a complete deep-learning framework using a single-step object detection model in order to quickly and accurately detect and classify the types of manufacturing defects present on Printed Circuit Board (PCBs). We describe the complete model architecture and compare with the current state-of-the-art using the same PCB defect dataset. These benchmark methods include the Faster Region Based Convolutional Neural Network (FRCNN) with ResNet50, RetinaNet, and You-Only-Look-Once (YOLO) for defect detection and identification. Results show that our method achieves a 98.1% mean average precision(mAP[IoU = 0.5]) on the test samples using low-resolution images. This is 3.2% better than the state-of-the-art using low-resolution images (YOLO V5m) and 1.4% better than the state-of-the-art using high-resolution images (FRCNN-ResNet FPN). While achieving better accuracies, our model also requires roughly 3× fewer model parameters (7.02M) compared with the state-of-the-art FRCNN-ResNet FPN (23.59M) and YOLO V5m (20.08M). In most cases, the major bottleneck of the PCB manufacturing chain is quality control, reliability testing and manual rework of defective PCBs. Based on the initial results, we firmly believe that implementing this model on a PCB manufacturing line could significantly increase the production yield and throughput, while dramatically reducing manufacturing costs.

Enhancing the detection performance of a vision-based occupancy detector for buildings

Occupant behaviour is one of the key parameters that significantly impact the operation of heating, ventilation and air-conditioning (HVAC) systems and the energy performance of buildings. Detailed occupancy information can improve HVAC operation and utilisation of building spaces. Strategies such as vision-based occupancy detection and recognition have recently garnered much interest. This study investigated the performance of a vision-based deep learning detection technique for enhancing building system operations and energy performances. The model used was the Faster region-based convolutional neural network with Inception V2. Two occupancy detection model configurations were developed, tested and evaluated. Both models were analysed based on the application of the detector within a selected case study building, along with an evaluation based on different evaluation metrics. The results suggest the occupancy detector using model 1 provided an overall accuracy of 95.23%, while model 2 (the occupancy activity detector) provided an accuracy of 89.37%. In addition, the F 1 score, which represents the harmonic mean between the precision and recall values, was 0.9756 for model 1 and 0.8298 for model 2. Building energy simulation was used to evaluate and compare the impact of such an approach on indoor occupancy heat gains. The study highlighted the potential of the detection approaches, but further development is necessary, including optimisation of the model, full integration with HVAC controls and further model training and field testing.

Detection of live breast cancer cells in bright-field microscopy images containing white blood cells by image analysis and deep learning.

.SignificanceCirculating tumor cells (CTCs) are important biomarkers for cancer management. Isolated CTCs from blood are stained to detect and enumerate CTCs. However, the staining process is laborious and moreover makes CTCs unsuitable for drug testing and molecular characterization.AimThe goal is to develop and test deep learning (DL) approaches to detect unstained breast cancer cells in bright-field microscopy images that contain white blood cells (WBCs).ApproachWe tested two convolutional neural network (CNN) approaches. The first approach allows investigation of the prominent features extracted by CNN to discriminate in vitro cancer cells from WBCs. The second approach is based on faster region-based convolutional neural network (Faster R-CNN).ResultsBoth approaches detected cancer cells with higher than 95% sensitivity and 99% specificity with the Faster R-CNN being more efficient and suitable for deployment presenting an improvement of 4% in sensitivity. The distinctive feature that CNN uses for discrimination is cell size, however, in the absence of size difference, the CNN was found to be capable of learning other features. The Faster R-CNN was found to be robust with respect to intensity and contrast image transformations.ConclusionsCNN-based DL approaches could be potentially applied to detect patient-derived CTCs from images of blood samples.

Online sorting of drilled lotus seeds using deep learning

For defective drilled lotus seeds, the inner bitter lotus plumule cannot be removed normally, leading to difficulties in subsequent nutrient extraction and food processing. There is an obvious difference in visibility of drilled hole between normal and defective drilled lotus seeds in the top view; thus, an online sorting method for drilled lotus seeds based on drilled hole detection is proposed in this study. First, a drilled hole detection model based on You Only Look Once (YOLOv3) is developed to detect the drilled hole features on the lotus seed surface. The model was tested and compared with the Faster Region-based Convolutional Neural Network (Faster R-CNN) and Single Shot MultiBox Detector (SSD) models, and it showed a better comprehensive performance in terms of accuracy and speed. A sorting control algorithm is also proposed to perform online sorting based on real-time drilled hole detection results. In addition, an auxiliary algorithm is proposed to prevent the boundary misjudgement of detected hole ownership between adjacent lotus seeds during continuous sorting. An online sorting system was designed, and sorting tests were performed. A sorting accuracy of 95.8% was achieved for the mixed defective and normal drilled lotus seed samples. The proposed method is expected to not only fulfil the practical requirements for the online sorting of drilled lotus seeds but also provide references for other agricultural products that require continuous online sorting based on the detection of local features.

Fracture R-CNN: An anchor-efficient anti-interference framework for skull fracture detection in CT images.

Skull fracture, as a common traumatic brain injury, can lead to multiple complications including bleeding, leaking of cerebrospinal fluid, infection, and seizures. Automatic skull fracture detection (SFD) is of great importance, especially in emergencymedicine. Existing algorithms for SFD, developed based on hand-crafted features, suffer from low detection accuracy due to poor generalizability to unseen samples. Deploying deep detectors designed for natural images like Faster Region-based Convolutional Neural Network (R-CNN) for SFD can be helpful but are of high redundancy and with nonnegligible false detections due to the cranial suture and skull base interference. Therefore, we, for the first time, propose an anchor-efficient anti-interference deep learning framework named Fracture R-CNN for accurate SFD with low computationalcost. The proposed Fracture R-CNN is developed by incorporating the prior knowledge utilized in clinical diagnosis into the original Faster R-CNN. Specifically, based on the distributions of skull fractures, we first propose an adaptive anchoring region proposal network (AA-RPN) to generate proposals for diverse-scale fractures with low computational complexity. Then, based on the prior knowledge that cranial sutures exist in the junctions of bones and usually contain sclerotic margins, we design an anti-interference head (A-Head) network to eliminate the cranial suture interference for better SFD detection. In addition, to further enhance the anti-interference ability of the proposed A-Head, a difficulty-balanced weighted loss function is proposed to emphasize more on distinguishing the interference areas from the skull base and the cranial sutures duringtraining. Experimental results demonstrate that the proposed Fracture R-CNN outperforms the current state-of-the-art (SOTA) deep detectors for SFD with a higher recall and fewer false detections. Compared to Faster R-CNN, the proposed Fracture R-CNN improves the average precision (AP) by 11.74% and the free-response receiver operating characteristic (FROC) score by 11.08%. Through validating on various backbones, we further demonstrate the architecture independence of Fracture R-CNN, making it extendable to other detectionapplications. As the customized deep learning-based framework for SFD, Fracture R-CNN can effectively overcome the unique challenges in SFD with less computational cost, leading to a better detection performance compared to the SOTA deep detectors. Moreover, we believe the prior knowledge explored for Fracture R-CNN would shed new light on future deep learning approaches for SFD.

Object Tracking in Crowd Environment Using Deep Learning

Abstract: Object tracking is fetching a primary set of object identification, allocating a unique ID to each one of them, and then ensuing each of the objects as they move around frames in a movie while safe keeping the ID assignment. Person tracking is a difficult task in video surveillance. In recent years, many computer vision, deep and machine learning have been developed. Convolutional Neural Networks (CNN) are transforming target tracking. The project's purpose is to recognize and track images utilizing object identification techniques such as Region based Convolutional Neural Networks (RCNN), Faster RCNN, Single Shot Detector (SSD), and You Only Look Once (YOLO). Faster-RCNN and SSD have superior accuracy among them, whereas YOLO performs better when speed is prioritized above accuracy. Deep learning combines SSD and Mobile Nets for efficient identification and tracking implementation. This technique detects objects quickly and efficiently without sacrificing performance. Convolutionally default boxex are passed over several feature maps. If a detected object matches one of the object classifiers during prediction, a score is produced..

An ıntelligent system for detecting Mediterranean fruit fly [Medfly; Ceratitis Capitata (Wiedemann)

Nowadays, the most critical agriculture-related problem is the harm caused in fruit, vegetable, nut, and flower crops by harmful pests, particularly the Mediterranean fruit fly, Ceratitis capitata, named in short as Medfly. Medfly existence in agricultural fields must be monitored systematically for effective combat against it. Special traps are utilized in the field to catch Medflies which will reveal their presence, and applying pesticides at the right time will help reduce their population. A technologically supported automated remote monitoring system should eliminate frequent site visits as a more economical solution. In this paper, a machine learning system that can detect Medfly images on a picture and count their numbers is developed. A special trap equipped with an integrated camera that can take photos of the sticky band where Medflies are caught daily is utilized. Obtained pictures are then transmitted by an electronic circuit containing a SIM card to the central server where the object detection algorithm runs. This study employs a faster region-based convolutional neural network (Faster R-CNN) model in identifying trapped Medflies. When Medflies or other insects stick on the sticky band of the trap, they continue to spend extraordinary effort trying to release themselves in a panic until they die. Therefore, their shape is badly distorted as their bodies, wings, and legs are all buckled. The challenge here is that the machine learning system should detect these Medflies of distorted shape with high accuracy. Therefore, it is crucial to utilize pictures that contain trapped Medfly images that possess distorted shapes for training and validation. In this paper, the success rate in identifying Medflies when other insects are also present is approximately 94% that is achieved by the machine learning system training process, owing to the considerable amount of purpose-specific photographic data. This rate may be seen as quite favorable when compared to the success rates provided in the literature.

Weed Detection by Faster RCNN Model: An Enhanced Anchor Box Approach

To apply weed control treatments effectively, the weeds must be accurately detected. Deep learning (DL) has been quite successful in performing the weed identification task. However, various aspects of the DL have not been explored in previous studies. This research aimed to achieve a high average precision (AP) of eight classes of weeds and a negative (non-weed) class, using the DeepWeeds dataset. In this regard, a DL-based two-step methodology has been proposed. This article is the second stage of the research, while the first stage has already been published. The former phase presented a weed detection pipeline and consisted of the evaluation of various neural networks, image resizers, and weight optimization techniques. Although a significant improvement in the mean average precision (mAP) was attained. However, the Chinee apple weed did not reach a high average precision. This result provided a solid ground for the next stage of the study. Hence, this paper presents an in-depth analysis of the Faster Region-based Convolutional Neural Network (RCNN) with ResNet-101, the best-obtained model in the past step. The architectural details of the Faster RCNN model have been thoroughly studied to investigate each class of weeds. It was empirically found that the generation of anchor boxes affects the training and testing performance of the Faster RCNN model. An enhancement to the anchor box scales and aspect ratios has been attempted by various combinations. The final results, with the addition of 64 × 64 scale size, and aspect ratio of 1:3 and 3:1, produced the best classification and localization of all classes of weeds and a negative class. An enhancement of 24.95% AP was obtained in Chinee apple weed. Furthermore, the mAP was improved by 2.58%. The robustness of the approach has been shown by the stratified k-fold cross-validation technique and testing on an external dataset.

Sustainable Oil Palm Resource Assessment Based on an Enhanced Deep Learning Method

Knowledge of the number and distribution of oil palm trees during the crop cycle is vital for sustainable management and predicting yields. The accuracy of the conventional image processing method is limited for the hand-crafted feature extraction method and the overfitting problem occurs due to the insufficient dataset. We propose a modification of the Faster Region-based Convolutional Neural Network (FRCNN) for palm tree detection to reduce the overfitting problem and improve the detection accuracy. The enhanced FRCNN (EFRCNN) leads to improved performance for detecting objects (in the same image) when they are of multiple sizes by using a feature concatenation method. Transfer learning based on a ResNet50 model is used to extract the features of the input image. High-resolution images of oil palm trees from a drone are used to form the data set, containing mature, young, and mixed oil palm tree regions. We train and test the EFRCNN, the FRCNN, a CNN used recently for oil palm image detection, and two standard methods, namely, the support vector machine (SVM) and template matching (TM). The results reveal an overall accuracy of ≥96.8% for the EFRCNN on the three test sets. The accuracy is higher than the CNN and FRCNN and substantially higher than SVM and TM. For large-scale plantations, the accuracy improvement is significant. This research provides a method for automatically counting the oil palm trees in large-scale plantations.

Energy conservation for existing cooling and lighting loads

Energy consumption by cooling and lighting loads can be significantly reduced if real time information on the occupancy is applied in their control. To meet this goal smart cooling and lighting devices, which have built-in sensors and controllers and which make decisions on individual basis, are being developed. However, it appears that no attempt has yet been taken for auto-operation of the huge number of already existent so called “non smart” air conditioning (AC) and lighting loads considering the count and location of occupants. A good deal of research has been reported in literature on the line of deploying or improving occupant detecting sensors/cameras, processing the captured images and developing high performance occupant identification (only detection or detection and counting) algorithms. Those were focused on locating users and surveillance in a building. A comparison of those works reveals that cameras combined with faster RCNN (Region based Convolutional Neural Network) algorithm can best identify the occupants. However, none of the works reported any prototyping considering occupancy with or without other ambient conditions such as comfort level, temperature, humidity and luminance for cooling load set point temperature update and/or lighting load control. Furthermore, the way the existing cooling and lighting loads can be made amenable to occupancy sensitive operation for energy saving in a noninvasive way is not yet reported. The novelties and contributions of the present paper are on bridging the gaps viz. (i) prototyping using traditional cooling and lighting loads of any brand already existent in a space for achieving occupancy sensitive energy saving, (ii) developing a model using the total count of occupants in a space along with ambient temperature and humidity for update of the set point temperature of cooling loads such that energy will be saved while the occupants will feel comfort, (iii) generation of ON/OFF commands in real time for lighting loads considering zone wise location of occupants along with the difference between ambient and required luminance in a zone, (iv) auto transmission of the updated set point temperature and on/off commands wirelessly from a common GUI (Graphical User Interface) respectively to the cooling and lighting loads without requiring any “hand held remote”, and (v) noninvasive implementation i.e. no need to modify or open the existing cooling and lighting loads at all and fixing any gadget inside these. The method proposed has been implemented on 3 traditional split type AC units of different brands and 16 composite units of LED (Light Emitting Diode) tube lights already existent in a lab. Compared with the manual mode of control which is insensitive to occupancy, the proposed method shows an energy saving in the range of 12.7%–36.15% for cooling loads while in the range of 35%–87.5% for lighting loads as the occupancy varies from high to low. The main economic and commercial impact of the present research is avoiding or postponing the replacement of the huge number of existing cooling and lighting devices by their smart counterparts which is often unaffordable for the entities in many countries.

Identification of Smoke from Straw Burning in Remote Sensing Images with the Improved YOLOv5s Algorithm

Controlling straw burning is important for ensuring the ambient air quality and for sustainable agriculture. Detecting burning straw is vital for managing and controlling straw burning. Existing methods for detecting straw combustion mainly look for combustion products, especially smoke. In this study, the improved You Only Look Once version 5 (YOLOv5s) algorithm was used to detect smoke in Sentinel-2 images captured by remote sensing. Although the original YOLOv5s model had a faster detection speed, its detection accuracy was poor. Thus, a convolutional block attention module was added to the original model. In addition, in order to speed up the convergence of the model, this study replaced the leaky Rectified Linear Unit (leaky ReLU) activation function with the Mish activation function. The accuracy of the improved model was approximately 4% higher for the same detection speed. The improved YOLOv5s had a higher detection accuracy and speed compared to common target detection algorithms, such as RetinaNet, mask Region-Based Convolutional Neural Network (R-CNN), Single-Shot Multibox Detector (SSD), and faster R-CNN. The improved YOLOv5s analyzed an image in 2 ms. In addition, mAP50 exceeded 94%, demonstrating that with this study’s improved method, smoke can be quickly and accurately identified. This work may serve as a reference for improving smoke detection, and for the effective management and control of straw burning.

Helminth egg analysis platform (HEAP): An opened platform for microscopic helminth egg identification and quantification based on the integration of deep learning architectures.

Millions of people throughout the world suffer from parasite infections. Traditionally, technicians use manual eye inspection of microscopic specimens to perform a parasite examination. However, manual operations have limitations that hinder the ability to obtain precise egg counts and cause inefficient identification of infected parasites on co-infections. The technician requirements for handling a large number of microscopic examinations in countries that have limited medical resources are substantial. We developed the helminth egg analysis platform (HEAP) as a user-friendly microscopic helminth eggs identification and quantification platform to assist medical technicians during parasite infection examination. Multiple deep learning strategies including SSD (Single Shot MultiBox Detector), U-net, and Faster R-CNN (Faster Region-based Convolutional Neural Network) are integrated to identify the same specimen allowing users to choose the best predictions. An image binning and egg-in-edge algorithm based on pixel density detection was developed to increase the performance. Computers with different operation systems can be gathered to lower the computation time using our easy-to-deploy software architecture. A user-friendly interface is provided to substantially increase the efficiency of manual validation. To adapt to low-cost computers, we architected a distributed computing structure with high flexibilities. HEAP serves not only as a prediction service provider but also as a parasitic egg database of microscopic helminth egg image collection, labeling data and pretrained models. All images and labeling resources are free and accessible at http://heap.cgu.edu.tw. HEAP can also be an ideal education and training resource for helminth egg examination.

Comparison of two deep learning methods for detecting fire hotspots

Every high-rise building must meet construction requirements, i.e. it must have good safety to prevent unexpected events such as fire incident. To avoid the occurrence of a bigger fire, surveillance using closed circuit television (CCTV) videos is necessary. However, it is impossible for security forces to monitor for a full day. One of the methods that can be used to help security forces is deep learning method. In this study, we use two deep learning methods to detect fire hotspots, i.e. you only look once (YOLO) method and faster region-based convolutional neural network (faster R-CNN) method. The first stage, we collected 100 image data (70 training data and 30 test data). The next stage is model training which aims to make the model can recognize fire. Later, we calculate precision, recall, accuracy, and F1 score to measure performance of model. If the F1 score is close to 1, then the balance is optimal. In our experiment results, we found that YOLO has a precision is 100%, recall is 54.54%, accuracy is 66.67%, and F1 score is 0.70583667. While faster R-CNN has a precision is 87.5%, recall is 95.45%, accuracy is 86.67%, and F1 score is 0.913022.

Detection of common bleeding points based on a faster region-based convolutional neural network.

Detection of common bleeding points based on a faster region-based convolutional neural network.

Detection of complicated bleeding points based on a faster region-based convolutional neural network.

Detection of complicated bleeding points based on a faster region-based convolutional neural network.

Deep learning and computer vision based occupancy CO2 level prediction for demand-controlled ventilation (DCV)

The present study investigated the potential of the application of a live occupancy detection approach to assist the operations of demand-controlled ventilation (DCV) systems to ensure that sufficient interior thermal conditions and air quality were attained while reducing unnecessary building energy loads to improve building energy performance. Faster region-based convolutional neural network (RCNN) models were trained to detect the number of people and occupancy activities respectively, and deployed to an artificial intelligence (AI)-powered camera. Experimental tests were carried out within a case study room to assess the performance of this approach. Due to the less complexity of people counting model, it achieved an average intersection over union (IoU) detection accuracy of about 98.9%, which was higher than activity detection model of about 88.5%. During the detection, the count-based occupancy profiles were produced according to the real-time information about the number of people and their activities. To estimate the effect of this approach on indoor air quality and energy demand, scenario-based modelling of the case study building under four ventilation scenarios was carried out via building energy simulation (BES). Results showed that the proposed approach could provide demand-driven ventilation controls data on the dynamic changes of occupancy to improve the indoor air quality (IAQ) and address the problem of under- or over-estimation of the ventilation demand when using the static or fixed profiles.