Faster Region-based Convolutional Neural Network Research Articles

Artificial intelligence for automated detection of large mammals creates path to upscale drone surveys

Imagery from drones is becoming common in wildlife research and management, but processing data efficiently remains a challenge. We developed a methodology for training a convolutional neural network model on large-scale mosaic imagery to detect and count caribou (Rangifer tarandus), compare model performance with an experienced observer and a group of naïve observers, and discuss the use of aerial imagery and automated methods for large mammal surveys. Combining images taken at 75 m and 120 m above ground level, a faster region-based convolutional neural network (Faster-RCNN) model was trained in using annotated imagery with the labels: “adult caribou”, “calf caribou”, and “ghost caribou” (animals moving between images, producing blurring individuals during the photogrammetry processing). Accuracy, precision, and recall of the model were 80%, 90%, and 88%, respectively. Detections between the model and experienced observer were highly correlated (Pearson: 0.96–0.99, P value < 0.05). The model was generally more effective in detecting adults, calves, and ghosts than naïve observers at both altitudes. We also discuss the need to improve consistency of observers’ annotations if manual review will be used to train models accurately. Generalization of automated methods for large mammal detections will be necessary for large-scale studies with diverse platforms, airspace restrictions, and sensor capabilities.

An AI-Aided Diagnostic Framework for Hematologic Neoplasms Based on Morphologic Features and Medical Expertise

A morphologic examination is essential for the diagnosis of hematological diseases. However, its conventional manual operation is time-consuming and laborious. Herein, we attempt to establish an artificial intelligence (AI)-aided diagnostic framework integrating medical expertise. This framework acts as a virtual hematological morphologist (VHM) for diagnosing hematological neoplasms. Two datasets were established as follows: An image dataset was used to train the Faster Region-based Convolutional Neural Network to develop an image-based morphologic feature extraction model. A case dataset containing retrospective morphologic diagnostic data was used to train a support vector machine algorithm to develop a feature-based case identification model based on diagnostic criteria. Integrating these 2 models established a whole-process AI-aided diagnostic framework, namely, VHM, and a 2-stage strategy was applied to practice case diagnosis. The recall and precision of VHM in bone marrow cell classification were 94.65% and 93.95%, respectively. The balanced accuracy, sensitivity, and specificity of VHM were 97.16%, 99.09%, and 92%, respectively, in the differential diagnosis of normal and abnormal cases, and 99.23%, 97.96%, and 100%, respectively, in the precise diagnosis of chronic myelogenous leukemia in chronic phase. This work represents the first attempt, to our knowledge, to extract multimodal morphologic features and to integrate a feature-based case diagnosis model for designing a comprehensive AI-aided morphologic diagnostic framework. The performance of our knowledge-based framework was superior to that of the widely used end-to-end AI-based diagnostic framework in terms of testing accuracy (96.88% vs 68.75%) or generalization ability (97.11% vs 68.75%) in differentiating normal and abnormal cases. The remarkable advantage of VHM is that it follows the logic of clinical diagnostic procedures, making it a reliable and interpretable hematological diagnostic tool.

A Conceptual Multi-Layer Framework for the Detection of Nighttime Pedestrian in Autonomous Vehicles Using Deep Reinforcement Learning

The major challenge faced by autonomous vehicles today is driving through busy roads without getting into an accident, especially with a pedestrian. To avoid collision with pedestrians, the vehicle requires the ability to communicate with a pedestrian to understand their actions. The most challenging task in research on computer vision is to detect pedestrian activities, especially at nighttime. The Advanced Driver-Assistance Systems (ADAS) has been developed for driving and parking support for vehicles to visualize sense, send and receive information from the environment but it lacks to detect nighttime pedestrian actions. This article proposes a framework based on Deep Reinforcement Learning (DRL) using Scale Invariant Faster Region-based Convolutional Neural Networks (SIFRCNN) technologies to efficiently detect pedestrian operations through which the vehicle, as agents train themselves from the environment and are forced to maximize the reward. The SIFRCNN has reduced the running time of detecting pedestrian operations from road images by incorporating Region Proposal Network (RPN) computation. Furthermore, we have used Reinforcement Learning (RL) for optimizing the Q-values and training itself to maximize the reward after getting the state from the SIFRCNN. In addition, the latest incarnation of SIFRCNN achieves near-real-time object detection from road images. The proposed SIFRCNN has been tested on KAIST, City Person, and Caltech datasets. The experimental results show an average improvement of 2.3% miss rate of pedestrian detection at nighttime compared to the other CNN-based pedestrian detectors.

Performance analysis of object detection algorithms for robotic welding applications in planar environment

ABSTRACT Weld seam detection and tracking using intelligent techniques is a common task in robotic welding. In this study, vision-based robotic welding is implemented using a 2D Camera to reduce the experimental cost in a planar environment. Firstly, different multi-class object recognition algorithms namely Faster Region-Based Convolutional Neural Network (Faster R-CNN), Single Shot Detector (SSD) and You Look Only Once (YOLOv3) are trained with 1530 manually labeled images. To recognize the weld seam under different brightness levels addressing arc light and splash, Contrast-Limited Adaptive Histogram Equalization (CLAHE) method has been used. Further, an extensive experimental study is conducted to find the optimal hyperparameters among three deep learning models. The performance metrics like Accuracy, Root Mean Square Error, Precision, Recall, Mean Average Precision and F1-score are calculated for different test cases. Finally, homogeneous coordinate transformation is implemented to obtain robot coordinates from weld seam pixel coordinates. The analysis is also extended to detect the bad welding conditions for improper shapes and track the weld seam for good welding shapes using TAL BRABO manipulator. It was found that the weld shapes were accurately detected and tracked precisely with 99.9% accuracy using YOLOv3 than Faster R-CNN and SSD.

An Appropriate Approach to Recognize Coke Size Distribution in a Blast Furnace

The size distribution of coke is important in order to decide the burden layer structure and the burden porosity in the shaft of a blast furnace (BF), which fluctuates daily and can be determined by several parameters. It is measured two or three times per shift by screening the raw material. However, the screening method used is random and takes a lot of time and manpower, resulting in the susceptive size distribution of the raw material and delayed operation of the BF. Therefore, in this paper, a new online approach used to measure the size distribution of particles was selected through comparison. Four common algorithms were used to detect the coke particles from images, including the Marker-based Watershed (MW), Histogram of Oriented Gradient + Support Vector Machine (HOG + SVM), Faster Region-based Convolutional Neural Networks (Faster R-CNN), and You Only Look Once (YOLOv3). The results show that the MW and HOG + SVM were not suitable for coke image detection. The average mean average precisions (mAPs) of Faster R-CNN and YOLOv3 were 93.391% and 91.348%, respectively, which meet the requirements of coke particle recognition. However, the YOLOv3 (5.419 fps) was selected as the final coke particle image detection algorithm, which is about 4.3 times faster than the average detection speed of Faster RCNN (1.269 fps). After this, the YOLOv3 and screening were used to detect 100 coke images and to generate particle size distribution statistics. The results show that the two methods are basically consistent. YOLOv3 can be used in the online measurement of BF coke. This research, which is of important value, provides a basis for the online measurement of the particle size distribution of raw material in a BF.

Comparative Study of Some Deep Learning Object Detection Algorithms: R-CNN, FAST R-CNN, FASTER R-CNN, SSD, and YOLO

Due to its numerous applications and new technological advancements, object detection has gained more attention in the last few years. This study examined various uses of some deep learning object detection algorithms. These algorithms are divided into two-stage detectors like Region Based Convolutional Neural Network (R-CNN), Fast Region Based Convolutional Neural Network (Faster R-CNN), and Faster Region Based Convolutional Neural Network (Faster R-CNN), and one-stage detectors like Single Shot MultiBox Detector (SSD) and You Only Look Once (YOLO) algorithms that are used in text and face detection, image retrieval, security, surveillance, traffic control, traffic sign/light detection, pedestrian detection and in medical areas among others. This research primarily focuses on three applications: drone surveillance, applications relating to traffic, and medical fields. From the analysis performed, it was found that of the various deep learning models used in the various application areas, YOLO, a one-stage detector, is the most popular algorithm for drone surveillance. SSD, another one-stage detector, was primarily used in applications related to traffic, and Faster R-CNN, a two-stage detector, is the most popular algorithm for applications in the medical field

An Adaptive Traffic Lights System using Machine Learning

Traffic congestion is a major problem in many cities of the Hashemite Kingdom of Jordan as in most countries. The rapidly increase of vehicles and dealing with the fixed infrastructure have caused traffic congestion. One of the main problems is that the current infrastructure cannot be expanded further. Therefore, there is a need to make the system work differently with more sophistication to manage the traffic better, rather than creating a new infrastructure. In this research, a new adaptive traffic lights system is proposed to determine vehicles type, calculate the number of vehicles in a traffic junction using patterns detection methods, and suggest the necessary time for each side of the traffic junction using machine learning tools. In this context, the contributions of this paper are: (a) creating a new image-based dataset for vehicles, (b) proposing a new time management formula for traffic lights, and (c) providing literature of many studies that contributed to the development of the traffic lights system in the past decade. For training the vehicle detector, we have created an image-based dataset related to our work and contains images for traffic. We utilized Region-Based Convolutional Neural Networks (R-CNN), Fast Region-Based Convolutional Neural Networks (Fast R-CNN), Faster Region-Based Convolutional Neural Networks (Faster R-CNN), Single Shot Detector (SSD), and You Only Look Once v4 (YOLO v4) deep learning algorithms to train the model and obtain the suggested mathematical formula to the required process and give the appropriate timeslot for every junction. For evaluation, we used the mean Average Precision (mAP) metric. The obtained results were as follows: 78.2%, 71%, 75.2%, 79.8%, and 86.4% for SSD, R-CNN, Fast R-CNN, Faster R-CNN, and YOLO v4, respectively. Based on our experimental results, it is found that YOLO v4 achieved the highest mAP of the identification of vehicles with (86.4%) mAP. For time division (the junctions timeslot), we proposed a formula that reduces about 10% of the waiting time for vehicles.

Detection Method Based on Image Enhancement and an Improved Faster R-CNN for Failed Satellite Components

The components detection of a failed satellite is an important work in space on-orbit service. However, the current detection methods for failed satellite components do not consider the effects of low illumination and small targets on the detection accuracy of components at the same time, and most of the datasets used are manually designed. This article proposes a detection method based on image enhancement and an improved faster region-based convolutional neural network (R-CNN) for small components of a failed satellite in low illumination. First, the dataset of failed satellite components dataset containing low-illumination scenarios is established in a simulated real space environment. Second, an image enhancement based on reflection model and principal component analysis is proposed, which further enhances images while conserving richer details. Finally, an improved faster R-CNN for small components is proposed. To improve the detection accuracy of small components, the original faster R-CNN is improved by modifying three modules: backbone, region proposal network, and region of interest (RoI) pooling layer, respectively [i.e., modified high-resolution neural network (M-HRNet), intersection over union (IoU)-balanced sampling, and RoI align]. Experimental results show that the proposed method can accurately detect all the small components on satellite capture plane, and the detection performance for low illumination and small components is improved significantly compared to the state-of-the-art methods and original faster R-CNN.

Faster Region Based Convolutional Neural Network for Skin Lesion Segmentation

The diagnostic interpretation of dermoscopic images is a complex task as it is very difficult to identify the skin lesions from the normal. Thus the accurate detection of potential abnormalities is required for patient monitoring and effective treatment. In this work, a Two-Tier Segmentation (TTS) system is designed, which combines the unsupervised and supervised techniques for skin lesion segmentation. It comprises preprocessing by the median filter, TTS by Colour K-Means Clustering (CKMC) for initial segmentation and Faster Region based Convolutional Neural Network (FR-CNN) for refined segmentation. The CKMC approach is evaluated using the different number of clusters (k = 3, 5, 7, and 9). An inception network with batch normalization is employed to segment melanoma regions effectively. Different loss functions such as Mean Absolute Error (MAE), Cross Entropy Loss (CEL), and Dice Loss (DL) are utilized for performance evaluation of the TTS system. The anchor box technique is employed to detect the melanoma region effectively. The TTS system is evaluated using 200 dermoscopic images from the PH2 database. The segmentation accuracies are analyzed in terms of Pixel Accuracy (PA) and Jaccard Index (JI). Results show that the TTS system has 90.19% PA with 0.8048 JI for skin lesion segmentation using DL in FR-CNN with seven clusters in CKMC than CEL and MAE.

A Construction of Object Detection Model for Acute Myeloid Leukemia

The evolution of bone marrow morphology is necessary in Acute Myeloid Leukemia (AML) prediction. It takes an enormous number of times to analyze with the standardization and inter-observer variability. Here, we proposed a novel AML detection model using a Deep Convolutional Neural Network (D-CNN). The proposed Faster R-CNN (Faster Region-Based CNN) models are trained with Morphological Dataset. The proposed Faster R-CNN model is trained using the augmented dataset. For overcoming the Imbalanced Data problem, data augmentation techniques are imposed. The Faster R-CNN performance was compared with existing transfer learning techniques. The results show that the Faster R-CNN performance was significant than other techniques. The number of images in each class is different. For example, the Neutrophil (segmented) class consists of 8,486 images, and Lymphocyte (atypical) class consists of eleven images. The dataset is used to train the CNN for single-cell morphology classification. The proposed work implies the high-class performance server called Nvidia Tesla V100 GPU (Graphics processing unit).

Ship Detection Based on Faster R-CNN Using Range-Compressed Airborne Radar Data

Near real-time ship monitoring is crucial for ensuring safety and security at sea. Established ship monitoring systems are the automatic identification system (AIS) and marine radars. However, not all ships are committed to carry an AIS transponder and the marine radars suffer from limited visibility. For these reasons, airborne radars can be used as an additional and supportive sensor for ship monitoring, especially on the open sea. State-of-the-art algorithms for ship detection in radar imagery are based on constant false alarm rate (CFAR). Such algorithms are pixel-based and therefore it can be challenging in practice to achieve near real-time detection. This letter presents two object-oriented ship detectors based on the faster region-based convolutional neural network (R-CNN). The first detector operates in time domain and the second detector operates in Doppler domain of airborne Range-Compressed (RC) radar data patches. The Faster R-CNN models are trained on thousands of real X-band airborne RC radar data patches containing several ship signals. The robustness of the proposed object-oriented ship detectors is tested on multiple scenarios, showing high recall performance of the models even in very dense multitarget scenarios in the complex inshore environment of the North Sea.

A Secure Method for Data Storage and Transmission in Sustainable Cloud Computing

Cloud computing is a technology that provides secure storage space for the customer’s massive data and gives them the facility to retrieve and transmit their data efficiently through a secure network in which encryption and decryption algorithms are being deployed. In cloud computation, data processing, storage, and transmission can be done through laptops and mobile devices. Data Storing in cloud facilities is expanding each day and data is the most significant asset of clients. The important concern with the transmission of information to the cloud is security because there is no perceivability of the client’s data. They have to be dependent on cloud service providers for assurance of the platform’s security. Data security and privacy issues reduce the progression of cloud computing and add complexity. Nowadays; most of the data that is stored on cloud servers is in the form of images and photographs, which is a very confidential form of data that requires secured transmission. In this research work, a public key cryptosystem is being implemented to store, retrieve and transmit information in cloud computation through a modified Rivest-Shamir-Adleman (RSA) algorithm for the encryption and decryption of data. The implementation of a modified RSA algorithm results guaranteed the security of data in the cloud environment. To enhance the user data security level, a neural network is used for user authentication and recognition. Moreover; the proposed technique develops the performance of detection as a loss function of the bounding box. The Faster Region-Based Convolutional Neural Network (Faster R-CNN) gets trained on images to identify authorized users with an accuracy of 99.9% on training.

Detection of abnormal chicken droppings based on improved Faster R-CNN

The characteristics of chicken droppings are closely related to the health of chickens. Veterinarians often judge the health of a chicken by looking at whether the chicken poop is normal. At present, the inspection of abnormal chicken droppings in chicken coops relies on manual observation, which is inefficient, accurate varies from person to person, labor-intensive, and has the risk of cross-infection. To achieve efficient, accurate, and intelligent identification of abnormal chicken droppings, an abnormal chicken droppings detection method based on improved Faster Region-based Convolutional Neural Network (Faster R-CNN) was proposed in this study. In the feature extraction network stage, deformable convolution was used and combined with Path Augmentation-Feature Pyramid Network (PA-FPN) to improve the extraction ability of features at different scales. In the Region Proposal Network (RPN) stage, the K-means++ algorithm was used to cluster the dataset and obtain the Anchor-ratio which is more suitable for the chicken poop object, and the FocalLoss classification loss function was used to improve the classification ability of difficult samples. In the regional convolutional network stage, the region of interest calibration algorithm was used instead to obtain more accurate localization information. The experimental results show that the improved Faster R-CNN structure can reach an accuracy of 98.8% for abnormal chicken poop detection, and the average accuracy mean value was improved by 27.8%. The results can provide a key core technology support for establishing an efficient abnormal chicken droppings online detection system. Keywords: abnormal chicken droppings, Faster R-CNN, detection, non-destructive monitoring, PA-FPN DOI: 10.25165/j.ijabe.20231601.7732 Citation: Zhu J H, Cui Z H, Wang H Y, Zhou M, Sun X Q. Detection of abnormal chicken droppings based on improved Faster R-CNN. Int J Agric & Biol Eng, 2023; 16(1): 243–249.

Detection of Tomato Leaf Diseases for Agro-Based Industries Using Novel PCA DeepNet

The advancement of Deep Learning and Computer Vision in the field of agriculture has been found to be an effective tool in detecting harmful plant diseases. Classification and detection of healthy and diseased crops play a very crucial role in determining the rate and quality of production. Thus the present work highlights a well-proposed novel method of detecting Tomato leaf diseases using Deep Neural Networks to strengthen agro-based industries. The present novel framework is utilized with a combination of classical Machine Learning model Principal Component Analysis (PCA) and a customized Deep Neural Network which has been named as PCA DeepNet. The hybridized framework also consists of Generative Adversarial Network (GAN) for obtaining a good mixture of datasets. The detection is carried out using the Faster Region-Based Convolutional Neural Network (F-RCNN). The overall work generated a classification accuracy of 99.60% with an average precision of 98.55%; giving a promising Intersection over Union (IOU) score of 0.95 in detection. Thus the presented work outperforms any other reported state-of-the-art.

Optimizing Internet of Things-Based Intelligent Transportation System’s Information Acquisition Using Deep Learning

This work first discusses the Intelligent Transportation System (ITS)-oriented dynamic and static Information Acquisition Models (IAMs) and explains the information collection mechanism of the Internet of Things (IoT)-based ITS. The goal is to improve travel conditions and contribute to a better urban environment. In order to do so, the Faster Region-based Convolutional Neural Network (Faster R-CNN) is introduced to extract the IoT-based ITS’s electronic data features. It is observed that the Faster R-CNN has excellent recall and accuracy in extracting the features from the ITS electronic data sets. Specifically, the Faster R-CNN’s average recall and accuracy reach 83.89% and 86.79%. The accuracy is 6.20% higher than the R-CNN method. Thus, the Faster R-CNN algorithm features more robust and reliable performance for collecting and analyzing ITS data. Overall, this work examines ITS-oriented electronic information collection and automatic detection against the technological background of applying Computer Vision, Deep Learning, and IoT in urban traffic management. In particular, it explains the IoT-based ITS’s electronic information collection mechanism under Deep Learning (Faster R-CNN). The finding offers a theoretical foundation for implementing Deep Learning technologies in collecting ITS-oriented big data and smart city construction.

The application of artificial neural networks in the detection of mandibular fractures using panoramic radiography.

Panoramic radiography is a standard diagnostic imaging method for dentists. However, it is challenging to detect mandibular trauma and fractures in panoramic radiographs due to the superimposed facial skeleton structures. The objective of this study was to develop a deep learning algorithm that is capable of detecting mandibular fractures and trauma automatically and compare its performance with general dentists. This is a retrospective diagnostic test accuracy study. This study used a two-stage deep learning framework. To train the model, 190 panoramic images were collected from four different sources. The mandible was first segmented using a U-net model. Then, to detect fractures, a model named Faster region-based convolutional neural network was applied. In the end, a comparison was made between the accuracy, specificity, and sensitivity of artificial intelligence and general dentists in trauma diagnosis. The mAP50 and mAP75 for object detection were 98.66% and 57.90%, respectively. The classification accuracy of the model was 91.67%. The sensitivity and specificity of the model were 100% and 83.33%, respectively. On the other hand, human-level diagnostic accuracy, sensitivity, and specificity were 87.22 ± 8.91, 82.22 ± 16.39, and 92.22 ± 6.33, respectively. Our framework can provide a level of performance better than general dentists when it comes to diagnosing trauma or fractures.

A Deep Learning-Based Efficient Firearms Monitoring Technique for Building Secure Smart Cities

Violence, in any form, is a disgrace to our civilized world. Nevertheless, even in modern days, violence is an integral part of our society and causes the deaths of many innocent lives. One of the conventional means of violence is using a firearm. Firearm-related death is currently a global phenomenon. It is a threat to society and a challenge to law enforcement agencies. A significant portion of such crimes happens in semi-urban areas or cities. Nowadays, CCTV-based surveillance is widely used by governments and private organizations for monitoring and prevention. However, human-based monitoring requires a significant amount of person-hours as a resource and is prone to mistakes. On the other hand, automated smart surveillance for violent activities is more suitable for scale and reliability. The paper’s main focus is to showcase that deep learning-based techniques can be used in combination to detect firearms (particularly guns). This paper uses different detection techniques, such as Faster Region-Based Convolutional Neural Networks (Faster RCNN) and the latest EfficientDet-based architectures for detecting guns and human faces. An ensemble (stacked) scheme has improved the detection performance to identify human faces and guns at the post-processing level using Non-Maximum Suppression, Non-Maximum Weighted, and Weighted Boxes Fusion techniques. This paper has empirically discussed the comparative results of various detection techniques and their ensembles. It helps the police to gather quick intelligence about the incident and take preventive measures at the earliest. Also, the same method can be used to identify social-media videos for gun-based content detection. Here, theWeighted Boxes Fusion-based ensemble detection scheme provides mean average precision 77.02%, 16.40%, 29.73% for the mAP <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0.5</sub> , mAP <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0.75</sub> and mAP <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">[0.500.95]</sub> , respectively. The results achieve the best performance among all the experimented alternatives. The model has been rigorously tested with unknown test images and movie clips. The obtained ensemble schemes are satisfactory and consistently improve over primary models.

Towards the development of obstacle detection in railway tracks using thermal imaging

To prevent collisions between trains and objects on the railway line, rugged trains require an intelligent rail protection system. To improve railway safety and reduce the number of accidents, studies are underway. Machine learning (ML) had progressed rapidly, creating new perspectives on the subject. A technique called speed up robust features (SURF) is proposed by researchers to collect regionally and globally relevant information. In addition, taking advantage of the Ohio State University (OSU) heat walker benchmarking dataset, the effectiveness of this technique was examined under various lighting scenarios. This technology could help in reducing train accident rates and financial costs. The findings of the proposed methodology are very specific, in addition to the ability to quickly identify items (obstacles) on the railway line, both of which contribute to rail security. The proposed faster region based convolutional neural network (FR-CNN) with 2D singular spectrum analysis (SSA) improves the performance analysis of an accuracy of 90.2%, recall 95.6% and precision 94.6% when compared with an existing system YOLOv2 and YOLOv5 with 2D SSA.

Location Recognition of Indoor Firefighting Facilities based on RGB-D Camera and 3D LiDAR

Recently, as fires frequently occur in large buildings, digital twin (DT) technology that enables remote and real-time monitoring and control similar to the real world environment is being studied as a disaster response technology in buildings. In order to use DT technology, it is essential to collect the spatial data of actual building indoor environments and firefighting facilities. This study proposes an indoor spatial data collecting system that can generate the modeling data inside the building and location data of firefighting facilities using laser imaging detection and ranging (LiDAR) and RGB-D cameras. First, point clouds from three-dimensional (3D) LiDAR and the FAST-LIO2 (Fast LiDAR-Inertial Odometry) algorithm are used to obtain odometry information in an indoor environment. The firefighting facilities located inside the building are detected using RGB images and the deep learning model Faster region-based convolutional neural network (R-CNN) with Inception V2 architecture trained using RGB images of four types of firefighting facilities: fire extinguishers, fire hydrants, exit signs, and fire detectors. When a firefighting facility is detected, the relative distance between the RGB-D camera and the firefighting facility is calculated through the depth image and intrinsic parameters of the RGB-D camera. Afterwards, odometry information obtained from FAST-LIO2 and the relative distance are combined to obtain the 3D location of the firefighting facility. Point clouds of the FAST-LIO2 algorithm are then converted into models of the building indoor environment. Through this method, spatial data of an actual building can be constructed and used with DT technology.

Noninvasive Tracking of Every Individual in Unmarked Mouse Groups Using Multi-Camera Fusion and Deep Learning.

Accurate and efficient methods for identifying and tracking each animal in a group are needed to study complex behaviors and social interactions. Traditional tracking methods (e.g., marking each animal with dye or surgically implanting microchips) can be invasive and may have an impact on the social behavior being measured. To overcome these shortcomings, video-based methods for tracking unmarked animals, such as fruit flies and zebrafish, have been developed. However, tracking individual mice in a group remains a challenging problem because of their flexible body and complicated interaction patterns. In this study, we report the development of a multi-object tracker for mice that uses the Faster region-based convolutional neural network (R-CNN) deep learning algorithm with geometric transformations in combination with multi-camera/multi-image fusion technology. The system successfully tracked every individual in groups of unmarked mice and was applied to investigate chasing behavior. The proposed system constitutes a step forward in the noninvasive tracking of individual mice engaged in social behavior.