CNN-based Object Detection Research Articles

Assessment of deep convolutional neural network models for mandibular fracture detection in panoramic radiographs

The aim of this study was to develop automated models for the identification and detection of mandibular fractures in panoramic radiographs using convolutional neural network (CNN) algorithms. A total of 1710 panoramic radiograph images from the years 2016 to 2020, including 855 images containing mandibular fractures, were obtained retrospectively from the regional trauma centre. CNN-based classification models, DenseNet-169 and ResNet-50, were fabricated to identify fractures in the radiographic images. The CNN-based object detection models Faster R-CNN and YOLOv5 were trained to automate the placement of the bounding boxes to detect fractures in the radiographic images. The performance of the models was evaluated on a hold-out test set and also by comparison with residents in oral and maxillofacial surgery and oral and maxillofacial surgeons (experts) on a 100-image subset. The binary classification performance of the models achieved promising results with an area under the receiver operating characteristics curve (AUC), sensitivity, and specificity of 100%. The detection performance of the models achieved an AUC of approximately 90%. When compared with the accuracy of clinician observers, the identification performance of the models outperformed even an expert-level classification. In conclusion, CNN-based models identified mandibular fractures above expert-level performance. It is expected that these models will be used as an aid to improve clinician performance, with aided resident performance approximating that of expert level.

Video Analysis and Rule-Based Reasoning for Driving Maneuver Classification at Intersections

We propose a system for monitoring the driving maneuver at road intersections using rule-based reasoning and deep learning-based computer vision techniques. Along with detecting and classifying turning movements online, the system also detects violations such as ignoring STOP signs and failing to yield the right-of-way to other drivers. There is no distinction between temporarily and permanently stopped vehicles in the majority of frameworks proposed in the literature. Therefore, to conduct an accurate right-of-way study, permanently stopped vehicles should be excluded not to confound the results. Moreover, we also propose in this work a low-cost Convolutional Neural Network (CNN)-based object detection framework able to detect moving and temporally stopped vehicles. The detection framework combines the reasoning system with background subtraction and a CNN-based object detector. The obtained results are promising. Compared to the conventional CNN-based methods, the detection framework reduces the execution time of the object detection module by about 30% (i.e., 54.1 instead of 75ms/image) while preserving the same detection reliability. The accuracy of trajectory recognition is 95.32%, that of the zero-speed detection is 96.67%, and the right-of-way detection was perfect.

Ovarian Structures Detection using Convolutional Neural Networks

The detection of ovarian structures from ultrasound images is an important task in gynecological and reproductive medicine. An automatic detection system of ovarian structures can work as a second opinion for less experienced physicians or complex ultrasound interpretations. This work presents a study of three popular CNN-based object detectors applied to the detection of healthy ovarian structures, namely ovary and follicles, in B-mode ultrasound images. The Faster R-CNN presented the best results, with a precision of 95.5% and a recall of 94.7% for both classes, being able to detect all the ovaries correctly. The RetinaNet showed competitive results, exceeding 90% of precision and recall. Despite being very fast and suitable for real-time applications, YOLOv3 was ineffective in detecting ovaries and had the worst results detecting follicles. We also compare CNN results with classical computer vision methods presented in the ovarian follicle detection literature.

Multiscale Semantic Guidance Network for Object Detection in VHR Remote Sensing Images

With the development of convolutional neural network (CNN), many CNN-based object detection methods have made a remarkable success in very high-resolution (VHR) remote sensing images (RSIs). However, the standard convolution has a fixed receptive field, which makes it deficient in dynamic feature capture; complex backgrounds may also lead to the degradation of detection performance. Accordingly, this letter proposes a novel multiscale semantic guidance network (MSGN) to tackle these problems, wherein, based on the deformable convolution, an improved feature extraction backbone is proposed to capture features dynamically. Moreover, features from different layers are used to ensure the ability for detecting multiscale objects. Furthermore, a multilevel semantic guidance filtering subnetwork is proposed based on the designed backward semantic guidance filtering (BSGF) module, to suppress the complex backgrounds. Experimental results show that the proposed MSGN has stronger robustness and a better accuracy for multiscale object detection, compared with other reference methods.

Convergence of Stereo Vision-Based Multimodal YOLOs for Faster Detection爋f Potholes

Road potholes can cause serious social issues, such as unexpected damages to vehicles and traffic accidents. For efficient road management, technologies that quickly find potholes are required, and thus researches on such technologies have been conducted actively. The three-dimensional (3D) reconstruction method has relatively high accuracy and can be used in practice but it has limited application owing to its long data processing time and high sensor maintenance cost. The two-dimensional (2D) vision method has the advantage of inexpensive and easy application of sensor. Recently, although the 2D vision method using the convolutional neural network (CNN) has shown improved pothole detection performance and adaptability, large amount of data is required to sufficiently train the CNN. Therefore, we propose a method to improve the learning performance of CNN-based object detection model by artificially generating synthetic data similar to a pothole and enhancing the learning data. Additionally, to make the defective areas appear more contrasting, the transformed disparity map (TDM) was calculated using stereo-vision cameras, and the detection performance of the model was further improved through the late fusion with RGB (Red, Green, Blue) images. Consequently, through the convergence of multimodal You Only Look Once (YOLO) frameworks trained by RGB images and TDMs respectively, the detection performance was enhanced by 10.7% compared with that when using only RGB. Further, the superiority of the proposed method was confirmed by showing that the data processing speed was two times faster than the existing 3D reconstruction method.

Toward Performing Image Classification and Object Detection With Convolutional Neural Networks in Autonomous Driving Systems: A Survey

Nowadays Convolutional Neural Networks (CNNs) are being employed in a wide range of industrial technologies for a variety of sectors, such as medical, automotive, aviation, agriculture, space, etc. This paper reviews the state-of-the-art in both the field of CNNs for image classification and object detection and Autonomous Driving Systems (ADSs) in a synergetic way. Layer-based details of CNNs along with parameter and floating-point operation number calculations are outlined. Using an evolutionary approach, the majority of the outstanding image classification CNNs, published in the open literature, is introduced with a focus on their accuracy performance, parameter number, model size, and inference speed, highlighting the progressive developments in convolutional operations. Results of a novel investigation of the convolution types and operations commonly used in CNNs are presented, including a timing analysis aimed at assessing their impact on CNN performance. This extensive experimental study provides new insight into the behaviour of each convolution type in terms of training time, inference time, and layer level decomposition. Building blocks for CNN-based object detection are also discussed, such as backbone networks and baseline types, and then representative state-of-the-art designs are outlined. Experimental results from the literature are summarised for each of the reviewed models. This is followed by an overview of recent ADSs related works and current industry activities, aiming to bridge academic research and industry practice on CNNs and ADSs. Design approaches targeted at solving problems of automakers in achieving real-time implementations are also proposed based on a discussion of design constraints, human vs. machine evaluations and trade-off analysis of accuracy vs. size. Current technologies, promising directions, and expectations from the literature on ADSs are introduced including a comprehensive trade-off analysis from a human-machine perspective.

Human pose estimation for mitigating false negatives in weapon detection in video-surveillance

Applying CNN-based object detection models to the task of weapon detection in video-surveillance is still producing a high number of false negatives. In this context, most existing works focus on one type of weapons, mainly firearms, and improve the detection using different pre- and post-processing strategies. One interesting approach that has not been explored in depth yet is the exploitation of the human pose information for improving weapon detection. This paper proposes a top-down methodology that first determines the hand regions guided by the human pose estimation then analyzes those regions using a weapon detection model. For an optimal localization of each hand region, we defined a new factor, called Adaptive pose factor, that takes into account the distance of the body from the camera. Our experiments show that this top-down Weapon Detection over Pose Estimation (WeDePE) methodology is more robust than the alternative bottom-up approach and state-of-the art detection models in both indoor and outdoor video-surveillance scenarios.

Fetal Heart and Descending Aorta Detection in Four-Chamber View of Fetal Echocardiography.

Automatic analysis of fetal heart and related components in fetal echocardiography can help cardiologists to reach a diagnosis for Congenital Heart Disease (CHD). Previous studies mainly focused on cardiac chamber segmentation, while few researches deal with the cardiac component detection. In this paper, we tackle the task of simultaneous detection of the fetal heart and descending aorta in four-chamber view of fetal echocardiography, which is useful to analyze some kinds of CHD, such as left/right atrial isomerism, dextroversion of heart, etc. Several CNN-based object detection methods with different backbones are thoroughly evaluated, and finally, the Hybrid Task Cascade method with HRNet is selected as the detection method. Experiments on a fetal echocardiography dataset show that the method can achieve superior performance according to common-used evaluation metrics.Clinical relevance-This can be used to help the cardiologists to estimate the position of the fetal heart and the descending aorta, which is also useful to estimate the direction of the cardiac axis and apex and analyze some kinds of CHD, such as left/right atrial isomerism, dextroversion of heart, etc.

Improving the Ability of a Laser Ultrasonic Wave-Based Detection of Damage on the Curved Surface of a Pipe Using a Deep Learning Technique.

With the advent of the Fourth Industrial Revolution, the economic, social, and technological demands for pipe maintenance are increasing due to the aging of the infrastructure caused by the increase in industrial development and the expansion of cities. Owing to this, an automatic pipe damage detection system was built using a laser-scanned pipe’s ultrasonic wave propagation imaging (UWPI) data and conventional neural network (CNN)-based object detection algorithms. The algorithm used in this study was EfficientDet-d0, a CNN-based object detection algorithm which uses the transfer learning method. As a result, the mean average precision (mAP) was measured to be 0.39. The result found was higher than COCO EfficientDet-d0 mAP, which is expected to enable the efficient maintenance of piping used in construction and many industries.

Concept Towards Segmenting Arm Areas for Robot-Based Dermatological In Vivo Measurements

Dermatological in vivo measurements are used for various purposes, e.g. health care, development and testing of skin care products or claim support in marketing. Especially for the last two purposes, in vivo measurements are extensive due to the quantity and repeatability of the measurement series. Furthermore, they are performed manually and therefore represent a nonnegligible time and cost factor. A solution to this is the implementation of collaborative robotics for the measurement execution. Due to various body shapes and surface conditions, common static control procedures are not applicable. To solve this problem, spatial information obtained from a stereoscopic camera can be integrated into the robot control process. However, the designated measurement area has to be detected and the spatial information processed. Therefore the authors propose a concept towards segmenting arm areas through a CNN-based object detector and their further processing to perform robot-based in vivo measurements. The paper gives an overview of the utilization of RGB-D images in 2D object detectors and describes the selection of a suitable model for the application. Furthermore the creation, annotation and augmentation of a custom dataset is presented.

Automatic detection of burial mounds (kurgans) in the Altai Mountains

The Altai Mountains are one of the most impressive and valuable archaeological areas in the world. Kurgans (burial mounds) of ancient civilizations, which are scattered across the vast Altai area, are an exceptionally valuable source of information for archaeology. These precious archaeological resources, which sometimes have been preserved intact in the permafrost underground for over two millennia, are now under various threats, such as natural disasters, farmland expansion, touristic development, and most notably global warming. A detailed map or inventory of the mounds is essential but is still not available. In this study, we test the deep convolutional neural network (CNN) technique for automatic detection of stone mounds from high-resolution satellite images in four regions in the Altai Mountains. We propose three improvement techniques to increase the performance of off-the-shelf object detection methods that are originally proposed for daily-life objects. Our results demonstrate that it is feasible to apply CNN to detect stone mounds, and the detection results are good enough to capture their spatial distribution. CNN-based object detection can largely narrow down the search area for archaeologists in yet un-surveyed regions, and is therefore useful for preparing field survey campaigns and directing archaeological fieldwork. We also applied the method to an un-surveyed Altai Mountain area and successfully discovered stone mounds that are yet undocumented. Our method can potentially be applied to construct an inventory for all stone mounds present in the whole Altai Mountain region.

Monitoring social distancing through human detection for preventing/reducing COVID spread.

COVID-19 is a severe epidemic that has put the world in a global crisis. Over 42 Million people are infected, and 1.14 Million deaths are reported worldwide as on Oct 23, 2020. A deeper understanding of the epidemic suggests that a person’s negligence can cause widespread harm that would be difficult to negate. Since no vaccine is yet developed, social distancing must be practiced to detain COVID-19 spread. Therefore, we aim to develop a framework that tracks humans for monitoring the social distancing being practiced. To accomplish this objective of social distance monitoring, an algorithm is developed using object detection method. Here, CNN based object detector is explored to detect human presence. The object detector’s output is used for calculating distances between each pair of humans detected. This approach of social distancing algorithm will red mark the persons who are getting closer than a permissible limit. Experimental results prove that CNN based object detectors with our proposed social distancing algorithm exhibit promising outcomes for monitoring social distancing in public areas.

A 500-Fps Pan-Tilt Tracking System With Deep-Learning-Based Object Detection

In this letter, we propose a fast mirror-drive pan-tilt target tracking system that can robustly track an object whose appearance varies in a complex background at 500 fps. By assuming a small image displacement between frames, which is a property of high-frame rate vision, we develop an fast object tracking algorithm by hybridizing the convolutional-neural-network (CNN) based object detection with template-matching (TM) based tracking operating at hundreds of frames per second (fps). For object tracking with high-speed visual feedback, the proposed tracking algorithm can remarkably reduce dozens-of-milliseconds-latency in the CNN-based object detection by simultaneously executing TM-based tracking for several images at consecutive frames within a few milliseconds. In the proposed pan-tilt tracking system, when the current tracked objects are occluded or out of the camera view, it can recognize objects to be newly tracked with CNN-based object detection at the rate of 33 fps with acceleration using graphic processing units (GPUs). Controlling the pan-tilt tracking system via visual feedback at 500 Hz, fast moving objects can be robustly tracked at the center of the camera view. The effectiveness of our method was experimentally demonstrated via several results when fast-moving pre-learned objects, such as toy cars were tracked in complex backgrounds.

Practical aspects of detection and grasping objects by a mobile manipulating robot

PurposeThe purpose of this paper is to propose and evaluate the method for grasping a defined set of objects in an unstructured environment. To this end, the authors propose the method of integrating convolutional neural network (CNN)-based object detection and the category-free grasping method. The considered scenario is related to mobile manipulating platforms that move freely between workstations and manipulate defined objects. In this application, the robot is not positioned with respect to the table and manipulated objects. The robot detects objects in the environment and uses grasping methods to determine the reference pose of the gripper.Design/methodology/approachThe authors implemented the whole pipeline which includes object detection, grasp planning and motion execution on the real robot. The selected grasping method uses raw depth images to find the configuration of the gripper. The authors compared the proposed approach with a representative grasping method that uses a 3D point cloud as an input to determine the grasp for the robotic arm equipped with a two-fingered gripper. To measure and compare the efficiency of these methods, the authors measured the success rate in various scenarios. Additionally, they evaluated the accuracy of object detection and pose estimation modules.FindingsThe performed experiments revealed that the CNN-based object detection and the category-free grasping methods can be integrated to obtain the system which allows grasping defined objects in the unstructured environment. The authors also identified the specific limitations of neural-based and point cloud-based methods. They show how the determined properties influence the performance of the whole system.Research limitations/implicationsThe authors identified the limitations of the proposed methods and the improvements are envisioned as part of future research.Practical implicationsThe evaluation of the grasping and object detection methods on the mobile manipulating robot may be useful for all researchers working on the autonomy of similar platforms in various applications.Social implicationsThe proposed method increases the autonomy of robots in applications in the small industry which is related to repetitive tasks in a noisy and potentially risky environment. This allows reducing the human workload in these types of environments.Originality/valueThe main contribution of this research is the integration of the state-of-the-art methods for grasping objects with object detection methods and evaluation of the whole system on the industrial robot. Moreover, the properties of each subsystem are identified and measured.

Learning to Match Anchors for Visual Object Detection.

Modern CNN-based object detectors assign anchors for ground-truth objects under the restriction of object-anchor Intersection-over-Union (IoU). In this study, we propose a learning-to-match (LTM) method to break IoU restriction, allowing objects to match anchors in a flexible manner. LTM updates hand-crafted anchor assignment to "free" anchor matching by formulating detector training in the Maximum Likelihood Estimation (MLE) framework. During the training phase, LTM is implemented by converting the detection likelihood to anchor matching loss functions which are plug-and-play. Minimizing the matching loss functions drives learning and selecting features which best explain a class of objects with respect to both classification and localization. LTM is extended from anchor-based detectors to anchor-free detectors, validating the general applicability of learnable object-feature matching mechanism for visual object detection. Experiments on MS COCO dataset demonstrate that LTM detectors consistently outperform counterpart detectors with significant margins. The last but not the least, LTM requires negligible computational cost in both training and inference phases as it does not involve any additional architecture or parameter. Code has been made publicly available.

Multichannel Object Detection for Detecting Suspected Trees With Pine Wilt Disease Using Multispectral Drone Imagery

In this article, a multichannel convolutional neural network (CNN) based object detection was used to detect suspected trees of pine wilt disease after acquiring aerial photographs through a rotorcraft drone equipped with a multispectral camera. The acquired multispectral aerial photographs consist of RGB, green, red, NIR, and red edge spectral bands per shooting point. The aerial photographs for each band performed image calibration to correct radiation distortion, image alignment to correct the distance error of the lenses of a multispectral camera, and image enhancement to edge enhancement to highlight the features of objects in the image. After that, a large amount of data obtained through data augmentation were put into multichannel CNN-based object detection for training and test. As a result of verifying the detection performance of the trained model, excellent detection results were obtained with mAP 86.63% and average intersection over union 71.47%.

Energy Drain of the Object Detection Processing Pipeline for Mobile Devices: Analysis and Implications

Applying deep learning to object detection provides the capability to accurately detect and classify complex objects in the real world. However, currently, few mobile applications use deep learning because such technology is computation-intensive and energy-consuming. This article, to the best of our knowledge, presents the first detailed experimental study of a mobile augmented reality (AR) client's energy consumption and the detection latency of executing Convolutional Neural Networks (CNN) based object detection, either locally on the smartphone or remotely on an edge server. In order to accurately measure the energy consumption on the smartphone and obtain the breakdown of energy consumed by each phase of the object detection processing pipeline, we propose a new measurement strategy. Our detailed measurements refine the energy analysis of mobile AR clients and reveal several interesting perspectives regarding the energy consumption of executing CNN-based object detection. Furthermore, several insights and research opportunities are proposed based on our experimental results. These findings from our experimental study will guide the design of energy-efficient processing pipeline of CNN-based object detection.

CNN-based single object detection and tracking in videos and its application to drone detection

This paper presents convolutional neural network (CNN)-based single object detection and tracking algorithms. CNN-based object detection methods are directly applicable to static images, but not to videos. On the other hand, model-free visual object tracking methods cannot detect an object until a ground truth bounding box of the target is provided. Moreover, many annotated video datasets of the target object are required to train both the object detectors and visual trackers. In this work, three simple yet effective object detection and tracking algorithms for videos are proposed to efficiently combine a state-of-the-art object detector and visual tracker for circumstances in which only a few static images of the target are available for training. The proposed algorithms are tested using a drone detection task and the experimental results demonstrated their effectiveness.

Layer-Specific Optimization for Mixed Data Flow With Mixed Precision in FPGA Design for CNN-Based Object Detectors

Convolutional neural networks (CNNs) require both intensive computation and frequent memory access, which lead to a low processing speed and large power dissipation. Although the characteristics of the different layers in a CNN are frequently quite different, previous hardware designs have employed common optimization schemes for them. This paper proposes a layer-specific design that employs different organizations that are optimized for the different layers. The proposed design employs two layer-specific optimizations: layer-specific mixed data flow and layer-specific mixed precision. The mixed data flow aims to minimize the off-chip access while demanding a minimal on-chip memory (BRAM) resource of an FPGA device. The mixed precision quantization is to achieve both a lossless accuracy and an aggressive model compression, thereby further reducing the off-chip access. A Bayesian optimization approach is used to select the best sparsity for each layer, achieving the best trade-off between the accuracy and compression. This mixing scheme allows the entire network model to be stored in BRAMs of the FPGA to aggressively reduce the off-chip access, and thereby achieves a significant performance enhancement. The model size is reduced by 22.66-28.93 times compared to that in a full-precision network with a negligible degradation of accuracy on VOC, COCO, and ImageNet datasets. Furthermore, the combination of mixed dataflow and mixed precision significantly outperforms the previous works in terms of both throughput, off-chip access, and on-chip memory requirement.

CNN Based Object Detection and Localization of Aerial Images

CNN Based Object Detection and Localization of Aerial Images