Terms Of Mean Average Precision Research Articles

Identifying and Counting Avian Blood Cells in Whole Slide Images via Deep Learning

Avian blood analysis is a fundamental method for investigating a wide range of topics concerning individual birds and populations of birds. Determining precise blood cell counts helps researchers gain insights into the health condition of birds. For example, the ratio of heterophils to lymphocytes (H/L ratio) is a well-established index for comparing relative stress load. However, such measurements are currently often obtained manually by human experts. In this article, we present a novel approach to automatically quantify avian red and white blood cells in whole slide images. Our approach is based on two deep neural network models. The first model determines image regions that are suitable for counting blood cells, and the second model is an instance segmentation model that detects the cells in the determined image regions. The region selection model achieves up to 97.3% in terms of F1 score (i.e., the harmonic mean of precision and recall), and the instance segmentation model achieves up to 90.7% in terms of mean average precision. Our approach helps ornithologists acquire hematological data from avian blood smears more precisely and efficiently.

L-Net: A lightweight convolutional neural network for devices with low computing power

Deep learning (DL) has demonstrated exceptional success across various domains, including computer vision, natural language processing, and speech recognition. However, the training and inference processes of DL models typically require substantial computational resources and storage space, presenting a significant challenge within the Internet of Things (IoT) domain. This study contributes theoretically to the field of lightweight DL by proposing L-Net, a lightweight convolutional neural network designed specifically for low-compute devices. The L-Net addresses challenges associated with channel interaction disparities and vanishing gradients. To further improve the network performance, we introduce the residual enhanced channel attention (or R-ECA) module, which combines a bypass mechanism derived from simplified residual learning with the attention mechanism's cross-channel interaction. Additionally, we replace the rectified linear unit function (or ReLU) with an exponential linear unit (or ELU) function to enhance the network's nonlinear expression capability and training speed. We conducted object recognition experiments and compared the accuracy and prediction stability of L-Net with well-known models, such as AlexNet, VGG11, SqueezeNet, ResNet, and MobileNet, to assess its efficacy. Using the CIFAR-10 dataset and our custom dataset of apple tree leaf diseases, our experimental results demonstrate that, with relatively smaller model parameters, L-Net performs exceptionally well in terms of mean Average Precision (mAP), achieving 0.906. Furthermore, when applied to our custom dataset, L-Net exhibits relatively consistent performance across various dataset splits under different ratios, outperforming the majority of models.

Lightweight Detection Methods for Insulator Self-Explosion Defects.

The accurate and efficient detection of defective insulators is an essential prerequisite for ensuring the safety of the power grid in the new generation of intelligent electrical system inspections. Currently, traditional object detection algorithms for detecting defective insulators in images face issues such as excessive parameter size, low accuracy, and slow detection speed. To address the aforementioned issues, this article proposes an insulator defect detection model based on the lightweight Faster R-CNN (Faster Region-based Convolutional Network) model (Faster R-CNN-tiny). First, the Faster R-CNN model's backbone network is turned into a lightweight version of it by substituting EfficientNet for ResNet (Residual Network), greatly decreasing the model parameters while increasing its detection accuracy. The second step is to employ a feature pyramid to build feature maps with various resolutions for feature fusion, which enables the detection of objects at various scales. In addition, replacing ordinary convolutions in the network model with more efficient depth-wise separable convolutions increases detection speed while slightly reducing network detection accuracy. Transfer learning is introduced, and a training method involving freezing and unfreezing the model is employed to enhance the network's ability to detect small target defects. The proposed model is validated using the insulator self-exploding defect dataset. The experimental results show that Faster R-CNN-tiny significantly outperforms the Faster R-CNN (ResNet) model in terms of mean average precision (mAP), frames per second (FPS), and number of parameters.

Joint-Guided Distillation Binary Neural Network via Dynamic Channel-Wise Diversity Enhancement for Object Detection

Through truncating the weights and activations of a deep neural network, conventional binary quantization imposes limitations on the representation capability of the network parameters, which hence deteriorates the detection performance of the network. In this paper, a joint-guided distillation binary neural network via dynamic channel-wise diversity enhancement for object detection (JDBNet) is proposed to mitigate the gap of quantization errors. Our JDBNet includes a dynamic channel-wise diversity scheme and real-valued joint-guided teacher assistance to enhance the representation capability of the binary neural network in the object detection tasks. In the dynamic diversity scheme, the learning channel-wise bias (LCB) layer supports adjusting the magnitude of the parameters in which the sensitivity of the model parameters to the arbitrary quantization method is reduced, thereby improving the diversity expression ability of the feature parameters. In the joint-guided strategy, the single-precision implicit knowledge from the guiding teacher in the multilevel layer is utilized to supervise and penalize the quantitative model, enhancing the fitting performance of parameters in the binary quantized model. Extensive experiments on the PASCAL VOC, MS COCO, and VisDrone-DET datasets demonstrate that our JDBNet outperforms the state-of-the-art binary object detection networks in terms of mean Average Precision.

Metal Surface Defect Detection Based on Metal-YOLOX

Article Metal Surface Defect Detection Based on Metal-YOLOX Xiaoli Yue , Jiandong Chen , and Guoqiang Zhong * 1 College of Computer Science and Technology, Ocean University of China, Qingdao 266000, China * Correspondence: gqzhong@ouc.edu.cn Received: 14 June 2023 Accepted: 13 September 2023 Published: 21 December 2023 Abstract: Due to the limitations of manufacture technologies, working environments and other conditions, metals (such as steel and aluminum) are susceptible to surface defects during the production process. Therefore, defect detection is an indispensable part of metal manufacturing. This paper innovatively proposes a one-stage defect detection model named Metal-YOLOX. Metal-YOLOX addresses the limitations in existing models posed by large variances in defect features and inadequate balance between detection accuracy and efficiency. Firstly, the composite convolution module of Metal-YOLOX integrates texture, dilated and deformable convolutions to filter out irrelevant features and extract effective feature information. Secondly, the feature cross-fusion module (HCNet) alleviates the problem of large dimensional differences in defects. HCNet uses skip connections to establish the connection between the original multi-scale features and the output nodes, and reduces the addition of redundant information. Thirdly, Metal-YOLOX adopts the deep separable convolution and global channel reduction. This lightweight design helps reduce computational complexity. Finally, detailed experiments demonstrate that, in terms of mean average precision, Metal-YOLOX achieves 79.83, 69.14, and 81.22 on the NEU-DET, GC-10 and Aluminum datasets, respectively. Furthermore, Metal-YOLOX dramatically reduces parameter number and computational complexity. The experiments validate that the Metal-YOLOX model improves the detection performance, maintains the detection speed, and meets the real-time requirements.

Monocular 3D object detection for construction scene analysis

AbstractThree‐dimensional (3D) object detection, that is, localizing and classifying all critical objects in a 3D space, is essential for downstream construction scene analysis tasks. However, accurate instance segmentation, few 2D object segmentation and 3D object detection data sets, high‐quality feature representations for depth estimation, and limited 3D cues from a single red‐green‐blue (RGB) image pose significant challenges to 3D object detection and severely hinder its practical applications. In response to these challenges, an improved cascade‐based network with a transformer backbone and a boundary‐patch‐refinement method is proposed to build hierarchical features and refine object boundaries, resulting in better results in 2D object detection and instance segmentation. Furthermore, a novel self‐supervised monocular depth learning method is proposed to extract better feature representations for depth estimation from construction site video data with unknown camera parameters. Additionally, a pseudo‐LiDAR point cloud method and a 3D object detection method with a density‐based clustering algorithm are proposed to detect 3D objects in a construction scene without help from 3D labels, which will serve as a good foundation for other downstream 3D tasks. Finally, the proposed model is evaluated for object instance segmentation and depth estimation on the moving objects in construction sites (MOCS) and construction scene data sets. It brings a 9.16% gain in terms of mean average precision (mAP) for object detection and a 4.92% gain in mask mAP for object instance segmentation. The average order accuracy and relative mean error for depth estimation are improved by 0.94% and 60.56%, respectively. This study aims to overcome the challenges and limitations of 3D object detection and facilitate practical applications in construction scene analysis.

Feasibility of Detecting Sweet Potato (Ipomoea batatas) Virus Disease from High-Resolution Imagery in the Field Using a Deep Learning Framework

The sweet potato is an essential food and economic crop that is often threatened by the devastating sweet potato virus disease (SPVD), especially in developing countries. Traditional laboratory-based direct detection methods and field scouting are commonly used to rapidly detect SPVD. However, these molecular-based methods are costly and disruptive, while field scouting is subjective, labor-intensive, and time-consuming. In this study, we propose a deep learning-based object detection framework to assess the feasibility of detecting SPVD from ground and aerial high-resolution images. We proposed a novel object detector called SPVDet, as well as a lightweight version called SPVDet-Nano, using a single-level feature. These detectors were prototyped based on a small-scale publicly available benchmark dataset (PASCAL VOC 2012) and compared to mainstream feature pyramid object detectors using a leading large-scale publicly available benchmark dataset (MS COCO 2017). The learned model weights from this dataset were then transferred to fine-tune the detectors and directly analyze our self-made SPVD dataset encompassing one category and 1074 objects, incorporating the slicing aided hyper inference (SAHI) technology. The results showed that SPVDet outperformed both its single-level counterparts and several mainstream feature pyramid detectors. Furthermore, the introduction of SAHI techniques significantly improved the detection accuracy of SPVDet by 14% in terms of mean average precision (mAP) in both ground and aerial images, and yielded the best detection accuracy of 78.1% from close-up perspectives. These findings demonstrate the feasibility of detecting SPVD from ground and unmanned aerial vehicle (UAV) high-resolution images using the deep learning-based SPVDet object detector proposed here. They also have great implications for broader applications in high-throughput phenotyping of sweet potatoes under biotic stresses, which could accelerate the screening process for genetic resistance against SPVD in plant breeding and provide timely decision support for production management.

A hierarchical attention detector for bearing surface defect detection

Multi-scale defect detection on bearing surfaces is a challenging task due to the varying sizes and shapes of the defects and character-induced noise backgrounds. It has been proven that introducing attention mechanism in convolutional neural networks can often enhance feature extraction performance. However, most existing detectors use attention mechanisms at only one specific level of abstraction, which limits their capability to extract all defect features. In response to this issue, inspired by natural language processing, we present a novel visual hierarchical attention detector for multi-scale defect location and classification. The detector leverages texture, semantic, and instance features of defects through a hierarchical attention mechanism, enabling multi-scale defect detection in bearing images with complex backgrounds. First, a multi-attentional backbone uses spatial attention mechanisms to extract the spatial and semantic information of defects. Second, a semantic alignment and channel attention based feature pyramid network aligns the semantic information of the extracted features and concentrates on the important layers of the feature pyramid. Third, a semantic guided anchoring based region proposal network uses instance-level attention to refine the anchors. Furthermore, the distance-intersection over union loss optimizes the detector to locate defects. The experiments on an industrial bearing surface defect image dataset demonstrated that the proposed method outperformed the baseline by 6.03 % and 8.76 % in terms of mean average precision (mAP) and mean average recall (mAR), achieving 0.7838 mAP and 0.9588 mAR, respectively.

A deep-learning–based antifraud system for car-insurance claims

The annual cost of vehicle insurance fraud is estimated to exceed 40 billion dollars. This is an enormous amount considering the number of new vehicles insured yearly. In terms of higher premiums, it implies that insurance fraud incurs an additional annual cost to each U.S. family of $400 to $700, on average. Many frauds can be attributed to previously reported damages, which are submitted a second time to the insurance company. In these cases, it does not suffice to check the customer’s history to identify them: Damaged car panels can be removed from one vehicle and reassembled on another to make an insurance claim on the second car. To deal with these fraud attempts, in this paper, we propose an end-to-end solution to support the special investigation unit of the insurance companies in their antifraud investigations. For each claim, we organize the images sent to the insurance company and analyze them to extract basic vehicle information. Subsequently, we use these images to identify any damage to the bodywork, and, finally, we verify that the damage has not already been processed in previous claims. To the best of our knowledge, this is the first published work that deals with this problem through a pipeline that covers the entire claim management process. To validate our proposal, we compare our solution with other state-of-the-art models for estimating image similarity. Our results show that our solution is, on average, superior by 15.27% in terms of mean average precision (mAP). In addition, we report the challenges faced in scaling such a system in a production environment. This aspect is often ignored, but applying these solutions to industrial settings is of fundamental importance. Our proposed end-to-end system can reduce by up to 18% the number of false positives produced by the damage reidentification system. We show that encapsulating several specialized components and merging their intermediate results leads to a 72% reduction in possible alerts. Finally, to support the discussion and comparison of explanations for this new task, we introduce a new dataset as a benchmark for damage reidentification.

Exploiting sublimated deep features for image retrieval

Deep learning techniques can be used to describe image content, which is a good way to reduce the semantic gap between low-level and high-level features. However, convolutional neural networks (CNNs) exhibit texture bias and largely ignore global object shape. This does not conform to human perception and can fail to gain the advantages of both low-level features and deep features. To address this problem, in this work, the main focus was shifted from using simple deep features to the use of sublimated deep features, which incorporate global object shape and color features. Along these lines, in this work, a novel image retrieval method named the sublimated deep feature histogram (SDFH) was proposed. The main highlights are: 1) An effect orientation feature was introduced, namely, orientation-selective feature, to mimic the orientation-selection mechanism. This provides a good representation of global object shape and reduces the side-effects of texture bias. 2) A new concept was developed, namely color perceptual feature, to address the shortcoming of deep features—that they discard color features. This includes color cues in the deep features and provides a more discriminating representation. 3) The orientation selective and color perception mechanisms were effectively mimicked to provide a compact yet efficient representation, and propose an effective transfer learning method called gain whitening learning. By carrying out comparative experiments, it was demonstrated that sublimated deep features can provide highly competitive retrieval performance (in terms of mean average precision) using a pre-trained CNN model applied to well-known benchmark datasets. Our results provide new insights into image retrieval based on the mechanisms of the primary visual cortex (V1). Furthermore, the method is more in line with human perception than other methods.

Analysis of Stable Diffusion-derived fake weeds performance for training Convolutional Neural Networks

Weeds challenge crops by competing for resources and spreading diseases, impacting crop yield and quality. Effective weed detection can enhance herbicide application, thus reducing environmental and health risks. A major challenge in Site-Specific Weed Management (SSWM) is developing a reliable weed identification system, especially given the diversity and similarity between certain weeds and crops during early growth stages. Image-based deep learning (DL) methods have become vital for weed classification. However, accurate weed classification and detection using DL techniques face the bottleneck of requiring large labeled data. Furthermore, labeling this specific extensive data is a time-consuming and tedious task apart from necessitating weed science experts. This research's central focus is to present a novel approach to weed detection using convolutional neural network (CNN) classifiers, specifically Yolov8l and RetinaNet, augmented with Stable Diffusion data i.e., artificial weed images. Stable Diffusion enhanced the training data, increasing the classifiers' adaptability. The study targeted specific weeds (Solanum nigrum L.; Portulaca oleracea L.; Setaria Verticillata L.) found in tomato crops, using a limited number of real images (30 samples) to produce artificial training images for the CNNs. All validation and test sets are comprised of real weed images. Results showed high performance when using only artificial images in terms of Mean Average Precision (mAP). In isolated conditions (0.91 mAP), i.e., only one weed species per image, an average performance gain of about 3% in all tests is obtained. When adding the artificial images to the real ones (mixed dataset), a mAP of 0.99 is obtained. In contrast, results using only artificial images obtained 0.81 mAP when detecting more than a single weed species. However, when implementing the trained CNNs with a mixed dataset, a 6% − 9% performance gain was achieved in all cases. A mAP of up to 0.93 was achieved in the most challenging conditions where weed species could overlap. The results indicate that the proposed approach outperformed existing methods, such as Generative Adversarial Networks (GANs) regarding mAP. Furthermore, the Yolov8l model distinctly emerged as the most favorable option for real-time detection systems considering Frame Detection Speed (FDS). Specifically, the Yolov8l model registered an FDS of 10.2 ms, which is considerably faster when compared to the 21.2 ms that the RetinaNet model exhibited. Additionally, the method is versatile and applicable to various crops and weed species, thereby enhancing automated weed management systems. This research illustrates that Stable Diffusion can efficiently expand small image sets, significantly reducing field imaging. The study offers valuable insights for future SSWM efforts utilizing artificially generated images for weed detection and classification.

Aquaculture defects recognition via multi-scale semantic segmentation

Aquaculture net pen defects such as biofouling, vegetation, and holes are key challenges to efficient and sustainable fish production in aquaculture. These defects must be monitored to avoid problems with aquaculture net safety as well as fish growth. Recently, deep learning methods have been adopted to solve detection and classification problems in different applications. However, the conventional methods are challenging to meet the demands of high precision and real-time detection of aquaculture net defects in a complex marine environment. Towards this end, this paper proposes an autonomous net pen defect detection system that contains a novel multi-scale semantic segmentation topology for detecting the biofouling, vegetation, and hole problems in the aquaculture environment. In particular, we emphasize fusing the attention maps obtained across different decomposition levels of the network to generate rich feature distributions that enable the accurate extraction of biofouling, vegetation, and holes within aquaculture nets. Moreover, the proposed model is thoroughly tested on two private datasets and two public datasets which were acquired from a real-field fish farms. Across all four datasets, the proposed framework showed remarkable biofouling, vegetation, and hole detection performance, where it outperformed state-of-the-art methods by 6.58%, 3.69%, 6.44%, and 4.78% in terms of mean average precision across LABUST, KU, NDv1, and NDv2 datasets, respectively.

Detect occluded items in X-ray baggage inspection

X-ray baggage inspection automatically determines whether there are prohibited items in passenger luggage and has recently achieved good progress due to the development of deep convolutional neural networks (DCNNs). However, the performance of X-ray baggage inspection degrades with heavily occluded and cluttered baggage owing to the interaction of the imaged objects. Inspired by the idea that discriminant cues may still exist in multiscale perceptions, discrepancies among different classes, and associations among different tasks, we propose a novel approach that exploits pseudo semantic masks to enhance the discriminant ability of feature representations. Moreover, the features at neighboring scales interact to further explore context information. We verify our method on the publicly available SIXray, OPIXray, and HIXray datasets. The results show that our method outperforms other state-of-the-art X-ray baggage inspection methods by 1.02% and 0.85% in terms of mean average precision (mAP).

Hierarchical Detection of Gastrodia elata Based on Improved YOLOX

Identifying the grade of Gastrodia elata in the market has low efficiency and accuracy. To address this issue, an I-YOLOX object detection algorithm based on deep learning and computer vision is proposed in this paper. First, six types of Gastrodia elata images of different grades in the Gastrodia elata planting cooperative were collected for image enhancement and labeling as the model training dataset. Second, to improve feature information extraction, an ECA attention mechanism module was inserted between the backbone network CSPDarknet and the neck enhancement feature extraction network FPN in the YOLOX model. Then, the impact of the attention mechanism and application position on model improvement was investigated. Third, the 3 × 3 convolution in the neck enhancement feature extraction network FPN and the head network was replaced by depthwise separable convolution (DS Conv) to reduce the model size and computation amount. Finally, the EIoU loss function was used to predict boundary frame regression at the output prediction end to improve the convergence speed of the model. The experimental results indicated that compared with the original YOLOX model, the mean average precision of the improved I-YOLOX network model was increased by 4.86% (97.83%), the model computation was reduced by 5.422 M (reaching 3.518 M), the model size was reduced by 20.6 MB (reaching 13.7 MB), and the image frames detected per second increased by 3 (reaching 69). Compared with other target detection algorithms, the improved model outperformed Faster R-CNN, SSD-VGG, YOLOv3s, YOLOv4s, YOLOv5s, and YOLOv7 algorithms in terms of mean average precision, model size, computation amount, and frames per second. The lightweight model improved the detection accuracy and speed of different grades of Gastrodia elata and provided a theoretical basis for the development of online identification systems of different grades of Gastrodia elata in practical production.

Do you get what you see? Insights of using mAP to select architectures of pretrained neural networks for automated aerial animal detection.

The vast amount of images generated by aerial imagery in the context of regular wildlife surveys nowadays require automatic processing tools. At the top of the mountain of different methods to automatically detect objects in images reigns deep learning's object detection. The recent focus given to this task has led to an influx of many different architectures of neural networks that are benchmarked against standard datasets like Microsoft's Common Objects in COntext (COCO). Performance on COCO, a large dataset of computer vision images, is given in terms of mean Average Precision (mAP). In this study, we use six pretrained networks to detect red deer from aerial images, three of which have never been used, to our knowledge, in a context of aerial wildlife surveys. We compare their performance along COCO's mAP and a common test metric in animal surveys, the F1-score. We also evaluate how dataset imbalance and background uniformity, two common difficulties in wildlife surveys, impact the performance of our models. Our results show that the mAP is not a reliable metric to select the best model to count animals in aerial images and that a counting-focused metric like the F1-score should be favored instead. Our best overall performance was achieved with Generalized Focal Loss (GFL). It scored the highest along both metrics, combining most accurate counting and localization (with average F1-score of 0.96 and 0.97 and average mAP scores of 0.77 and 0.89 on both datasets respectively) and is therefore very promising for future applications. While both imbalance and background uniformity improved the performance of our models, their combined effect had twice as much impact as the choice of architecture. This finding seems to confirm that the recent data-centric shift in the deep learning field could also lead to performance gains in wildlife surveys.

A novel attention-based enhancement framework for face mask detection in complicated scenarios

In the context of COVID-19 pandemic prevention and control, it is of vital significance to realize accurate face mask detection via computer vision technique. In this paper, a novel attention improved Yolo (AI-Yolo) model is proposed, which can handle existing challenges in the complicated real-world scenarios with dense distribution, small-size object detection and interference of similar occlusions. In particular, a selective kernel (SK) module is set to achieve convolution domain soft attention mechanism with split, fusion and selection operations; a spatial pyramid pooling (SPP) module is applied to enhance the expression of local and global features, which enriches the receptive field information; and a feature fusion (FF) module is utilized to promote sufficient fusions of multi-scale features from each resolution branch, which adopts basic convolution operators without excessive computational complexity. In addition, the complete intersection over union (CIoU) loss function is adopted in the training stage for accurate positioning. Experiments are carried out on two challenging public face mask detection datasets, and the results demonstrate the superiority of the proposed AI-Yolo against other seven state-of-the-art object detection algorithms, which achieves the best results in terms of mean average precision and F1 score on both datasets. Furthermore, effectiveness of the meticulously designed modules in AI-Yolo is validated through extensive ablation studies. In a word, the proposed AI-Yolo is competent to accomplish face mask detection tasks under extremely complex situations with precise localization and accurate classification.

Marine ship instance segmentation by deep neural networks using a global and local attention (GALA) mechanism.

Marine ships are the transport vehicle in the ocean and instance segmentation of marine ships is an accurate and efficient analysis approach to achieve a quantitative understanding of marine ships, for example, their relative locations to other ships or obstacles. This relative spatial information is crucial for developing unmanned ships to avoid crashing. Visible light imaging, e.g. using our smartphones, is an efficient way to obtain images of marine ships, however, so far there is a lack of suitable open-source visible light datasets of marine ships, which could potentially slow down the development of unmanned ships. To address the problem of insufficient datasets, here we built two instance segmentation visible light datasets of marine ships, MariBoats and MariBoatsSubclass, which could facilitate the current research on instance segmentation of marine ships. Moreover, we applied several existing instance segmentation algorithms based on neural networks to analyze our datasets, but their performances were not satisfactory. To improve the segmentation performance of the existing models on our datasets, we proposed a global and local attention mechanism for neural network models to retain both the global location and semantic information of marine ships, resulting in an average segmentation improvement by 4.3% in terms of mean average precision. Therefore, the presented new datasets and the new attention mechanism will greatly advance the marine ship relevant research and applications.

D-S Augmentation: Density-Semantics Augmentation for 3-D Object Detection

Cameras and light detection and ranging (LiDAR) sensors are commonly used together in autonomous vehicles to provide rich color and texture information and information on the location of objects, respectively. However, fusing image and point-cloud information remains a key challenge. In this article, we propose D-S augmentation as a 3-D object detection method based on point-cloud density and semantic augmentation. Our proposed approach first performs 2-D bounding box detection and instance segmentation on an image. Then, a LiDAR point cloud is projected onto an instance segmentation mask, and a fixed number of random points are generated. Finally, a global <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${N}$ </tex-math></inline-formula> -nearest neighbor clustering is used to associate random and projected points to give depth to virtual points and complete point-cloud density augmentation (P-DA). Then, point-cloud semantic augmentation (P-SA) is performed, in which the instance segmentation mask of an object is used to associate it with the point cloud. The instance-segmented class labels and segmentation scores are assigned to the projected cloud, and the projection points added with 1-D features are inversely mapped to the point-cloud space to obtain a semantically augmented point cloud. We conducted extensive experiments on the nuScenes (Caesar et al., 2020) and KITTI (Geiger et al., 2012) datasets. The results demonstrate the effectiveness and efficiency of our proposed method. Notably, D-S augmentation outperformed a LiDAR-only baseline detector by +7.9% in terms of mean average precision (mAP) and +5.1% in terms of nuScenes detection score (NDS) and outperformed the state-of-the-art multimodal fusion-based methods. We also present the results of ablation studies to show that the fusion module improved the performance of a baseline detector.

Textual Context-Aware Dense Captioning With Diverse Words

Dense captioning generates more detailed spoken descriptions for complex visual scenes. Despite several promising leads, existing methods still have two broad limitations: 1) The vast majority of prior arts only consider visual contextual clues during captioning but ignore potentially important textual context; 2) current imbalanced learning mechanisms limit the diversity of vocabulary learned from the dictionary, thus giving rise to low language-learning efficiency. To alleviate these gaps, in this paper, we propose an end-to-end enhanced dense captioning architecture, namely Enhanced Transformer Dense Captioner (ETDC), which obtains textual context from surrounding regions and dynamically diversifies the vocabulary bank during captioning. Concretely, we first propose the Textual Context Module (TCM), which is integrated into each self-attention layer of the Transformer decoder, to capture the surrounding textual context. Moreover, we take full advantage of the class information of object context and propose a Dynamic Vocabulary Frequency Histogram (DVFH) re-sampling strategy during training to balance words with different frequencies. The proposed method is tested on the standard dense captioning datasets and surpasses the state-of-the-art methods in terms of mean Average Precision (mAP).

Road users detection for traffic congestion classification

One of the important problems that urban residents suffer from is Traffic Congestion. It makes their life more stressful, it impacts several sides including the economy: by wasting time, fuel and productivity. Moreover, the psychological and physical health. That makes road authorities required to find solutions for reducing traffic congestion and guaranteeing security and safety on roads. To this end, detecting road users in real-time allows for providing features and information about specific road points. These last are useful for road managers and also for road users about congested points. The goal is to build a model to detect road users including vehicles and pedestrians using artificial intelligence especially machine learning and computer vision technologies. This paper provides an approach to detecting road users using as input a dataset of 22983 images, each image contains more than one of the target objects, generally about 81000 target objects, distributed on persons (pedestrians), cars, trucks/buses (vehicles), and also motorcycles/bicycles. The dataset used in this study is known as Common Objects in Context (MS COCO) published by Microsoft. Furthermore, six different models were built based on the approaches RCNN, Fast RCNN, Faster RCNN, Mask RCNN, and the 5th and the 7th versions of YOLO. In addition, a comparison of these models using evaluation metrics was provided. As a result, the chosen model is able to detect road users with more than 55% in terms of mean average precision.