Real-time Object Detection Model Research Articles

Hyperparameter optimization of YOLOv4 tiny for palm oil fresh fruit bunches maturity detection using genetics algorithms

Indonesia's economic progress is closely linked to palm oil, a crucial plantation commodity. The price of this oil relies on the oil content found in the fruit of the oil palm, which is gauged by the fruit's maturity level, a task usually undertaken by experts. The maturity of oil palm FFB is generally divided into several classes, commonly unripe, under-ripe, ripe, and over-ripe. However, the detection of maturity still uses human vision, which causes detection errors. Therefore, this research will use the computer vision method with the YOLOv4 model to detect the maturity of the oil palm fruit. YOLOv4 itself is a real-time object detection model that has been widely used to detect fruit maturity, including oil palm fruit; this research uses 4,160 images of oil palm fruit maturity which is divided into six categories, namely empty fruit, underripe, abnormal, ripe, under-ripe, and over-ripe. The YOLOv4 model trained using the dataset will be optimized using GA for looking at the optimal hyperparameter to produce a better model. The result is YOLOv4 model experienced an increase in mAP compared to before using GA hyperparameter optimization.

Intelligent recognition of drill cores and automatic RQD analytics based on deep learning

Rock quality designation (RQD), as a well-accepted and appliable rock quality index, is crucial in geotechnical engineering. Current RQD estimation mainly relies on either manual statistics or the image binarisation method, while the former approach surrenders high labour intensity and low efficiency and the latter one is constrained by image acquisition. Considering the above-mentioned limitations in RQD estimation, this study proposed a novel convolutional neural network (CNN) approach to automatically perform core recognition and RQD cataloguing with significant improvement in accuracy and efficiency. Firstly, the proposed neural network automatically identified the prefabricated round markers to distinct drilling rounds. To maximumly strengthen the engineering capability of CNN without losing generality, we considered image inversion, rotation, noise addition, and RGB conversion of 200 core box samples in total. Secondly, replacing the unstable image binarisation method, the advanced YOLO V2 object detection model, a single-stage real-time object detection model, was adopted in this study. We also proposed the modified four-layer downsampling structure as our CNN, and then developed an automatic recognition approach for both cores and the round markers, resulting in a 93.1% accuracy according to the validation set. Thirdly, this study proposed an auto-ranking algorithm to sequence the core sample according to the confidence of core recognition by the CNN and row-scanning results for subsequent RQD cataloguing. In addition, the optimal scan width was proved to be 1.33 times larger than the average core width. Finally, a quick cataloguing platform for drill cores was developed. Compared with manual measurement and visual statistics, intelligent RQD cataloguing is characterised by its unparalleled accuracy and efficiency, which is merited by the low relative error (1.84%) and fast processing time (around 0.2 s). Moreover, the application presented in this paper is applicable to most geotechnical engineering scenarios. This is attributed to its low requirements in image acquisition, high efficiency, precise recognition, and robustness.

YOLOSA: Object detection based on 2D local feature superimposed self-attention

We analyzed the network structure of real-time object detection models and found that the features in the feature concatenation stage are very rich. Applying an attention module here can effectively improve the detection accuracy of the model. However, the commonly used attention module or self-attention module shows poor performance in detection accuracy and inference efficiency. Therefore, we propose a novel self-attention module, called 2D local feature superimposed self-attention, for the feature concatenation stage of the neck network. This self-attention module reflects global features through local features and local receptive fields. We also propose and optimize an efficient decoupled head and AB-OTA, and achieve SOTA results. Average precisions of 49.0% (71FPS, 14ms), 46.1% (85FPS, 11.7ms), and 39.1% (107FPS, 9.3ms) were obtained for large, medium, and small-scale models built using our proposed improvements. Our models exceeded YOLOv5 by 0.8% – 3.1% in average precision.

A real-time object detection model for orchard pests based on improved YOLOv4 algorithm

Accurate and efficient real-time detection of orchard pests was essential and could improve the economic benefits of the fruit industry. The orchard pest dataset, PestImgData, was built through a series of methods such as web crawler, specimen image collection and data augmentation. PestImgData was composed of two parts, PestImgData-1 and PestImgData-2. It contained 24,796 color images and covered 7 types of orchard pests. Based on the PestImgData and YOLOv4 algorithm, this paper conducted a preliminary study on the real-time object detection of orchard pests from 4 perspectives: transfer learning, activation function, anchor box, and batch normalization. In addition, this paper also visualized the feature learning ability of the detection models. On the basis of the above research, three improvement measures were adopted: the post-processing NMS algorithm was upgraded to DIoU-NMS, the training method was upgraded to 2-time finetuning training and the training data was enhanced. The performance of the improved model, F-D-YOLOv4-PEST, had been effectively improved. The mean average precision of F-D-YOLOv4-PEST was 92.86%, and the detection time of a single picture was 12.22 ms, which could meet the real-time detection requirements. In addition, in the case of high overlap area or high density, F-D-YOLOv4-PEST still maintained good performance. In the testing process of the laboratory and the greenhouse, including the wired network and the wireless network, F-D-YOLOv4-PEST could locate and classify pests as expected. This research could provide technical reference for the intelligent identification of agricultural pests based on deep learning.

Yolosa: Object Detection Based on 2d Local Feature Superimposed Self-Attention

We analyzed the network structure of real-time object detection models and found that the features in the feature concatenation stage are very rich. Applying an attention module here can effectively improve the detection accuracy of the model. However, the commonly used attention module or self-attention module shows poor performance in detection accuracy and inference efficiency. Therefore, we propose a novel self-attention module, called 2D local feature superimposed self-attention, for the feature concatenation stage of the neck network. This self-attention module reflects global features through local features and local receptive fields. We also propose and optimize an efficient decoupled head and AB-OTA, and achieve SOTA results. Average precisions of 49.0\% (66.2 FPS), 46.1\% (80.6 FPS), and 39.1\% (100 FPS) were obtained for large, medium, and small-scale models built using our proposed improvements. Our models exceeded YOLOv5 by 0.8\% -- 3.1\% in average precision.

An Efficient Deep Convolutional Neural Network Approach for Object Detection and Recognition Using a Multi-Scale Anchor Box in Real-Time

Deep learning is a relatively new branch of machine learning in which computers are taught to recognize patterns in massive volumes of data. It primarily describes learning at various levels of representation, which aids in understanding data that includes text, voice, and visuals. Convolutional neural networks have been used to solve challenges in computer vision, including object identification, image classification, semantic segmentation and a lot more. Object detection in videos involves confirming the presence of the object in the image or video and then locating it accurately for recognition. In the video, modelling techniques suffer from high computation and memory costs, which may decrease performance measures such as accuracy and efficiency to identify the object accurately in real-time. The current object detection technique based on a deep convolution neural network requires executing multilevel convolution and pooling operations on the entire image to extract deep semantic properties from it. For large objects, detection models can provide superior results; however, those models fail to detect the varying size of the objects that have low resolution and are greatly influenced by noise because the features after the repeated convolution operations of existing models do not fully represent the essential characteristics of the objects in real-time. With the help of a multi-scale anchor box, the proposed approach reported in this paper enhances the detection accuracy by extracting features at multiple convolution levels of the object. The major contribution of this paper is to design a model to understand better the parameters and the hyper-parameters which affect the detection and the recognition of objects of varying sizes and shapes, and to achieve real-time object detection and recognition speeds by improving accuracy. The proposed model has achieved 84.49 mAP on the test set of the Pascal VOC-2007 dataset at 11 FPS, which is comparatively better than other real-time object detection models.

STetro-Deep Learning Powered Staircase Cleaning and Maintenance Reconfigurable Robot.

Staircase cleaning is a crucial and time-consuming task for maintenance of multistory apartments and commercial buildings. There are many commercially available autonomous cleaning robots in the market for building maintenance, but few of them are designed for staircase cleaning. A key challenge for automating staircase cleaning robots involves the design of Environmental Perception Systems (EPS), which assist the robot in determining and navigating staircases. This system also recognizes obstacles and debris for safe navigation and efficient cleaning while climbing the staircase. This work proposes an operational framework leveraging the vision based EPS for the modular re-configurable maintenance robot, called sTetro. The proposed system uses an SSD MobileNet real-time object detection model to recognize staircases, obstacles and debris. Furthermore, the model filters out false detection of staircases by fusion of depth information through the use of a MobileNet and SVM. The system uses a contour detection algorithm to localize the first step of the staircase and depth clustering scheme for obstacle and debris localization. The framework has been deployed on the sTetro robot using the Jetson Nano hardware from NVIDIA and tested with multistory staircases. The experimental results show that the entire framework takes an average of 310 ms to run and achieves an accuracy of for staircase recognition tasks and accuracy for obstacle and debris detection tasks during real operation of the robot.

Lightweight and real-time object detection model on edge devices with model quantization.

Environment perception is of vital importance in autonomous driving as it serves as autonomous car’s “eye” to perceive the surrounding environment. We propose a solution to achieve real-time environment perception on resource constrained edge devices like mobile phones, even though they have limited computation resources. We used float-to-int quantization based on TensorFlow Lite to quantilize the SSD model. TensorFlow Lite achieves accuracy lossless quantization with quantization-aware training by inserting fake quant operator during training, which imitates the loss introduced by quantization in feedforward training. Our experiment shows that superiority of our proposed method, which is highly speed, light weight and relatively high accuracy.

Non-destructive thermal imaging for object detection via advanced deep learning for robotic inspection and harvesting of chili peppers

Deep Learning has been utilized in computer vision for object detection for almost a decade. Real-time object detection for robotic inspection and harvesting has gained interest during this time as a possible technique for high-quality machine assistance during agriculture applications.We utilize RGB and thermal images of chili peppers in an environment of various amounts of debris, pepper overlapping, and ambient lighting, train this dataset, and compare object detection methods. Results are presented from the real-time and less than real-time object detection models. Two advanced deep learning algorithms, Mask-Regional Convolutional Neural Networks (Mask-RCNN) and You Only Look Once version 3 (YOLOv3)are compared in terms of object detection accuracy and computational costs. When utilizing the YOLOv3 architecture, an overall training mean average precision (mAP) value of 1.0 is achieved. Most testing images from this model score within a range from 97 to 100% confidence levels in natural environment. It is shown that the YOLOv3 algorithm has superior capabilities to the Mask-RCNN with over 10 times the computational speed on the chili dataset. However, some of the RGB test images resulted in low classification scores when heavy debris is present in the image. A significant improvement in the real-time classification scores was observed when the thermal images were used, especially with heavy debris present. We found and report improved prediction scores with a thermal imagery dataset where YOLOv3 struggled on the RGB images. It was shown that mapping temperature differences between the pepper and plant/debris can provide significant features for object detection in real-time and can help improve accuracy of predictions with heavy debris, variant ambient lighting, and overlapping of peppers. In addition, successful thermal imaging for real-time robotic harvesting could allow the harvesting period to become more efficient and open up harvesting opportunity in low light situations.

Real-time object detection based on YOLO-v2 for tiny vehicle object

Abstract In Automatic Driving System (ADS) and Driver Assistance System (DAS), object detection plays a vital part. Nevertheless, existing real-time detection models for tiny vehicle objects have the problems of low precision and poor performance. To solve these issues, we propose a novel real-time object detection model based on You Only Look Once Version 2 (YOLO-v2) deep learning framework for tiny vehicle objects, called Optimized You Only Look Once Version 2 (O-YOLO-v2). In the proposed model, a new structure is introduced to strengthen the feature extraction ability of the network by adding convolution layers at different locations. Meanwhile, the problem of gradient disappearance or dispersion caused by increasing network depth is solved by adding residual modules. Furthermore, in order to promote the detection accuracy of tiny vehicle objects, we combine the low-level features and high-level features of the network. The experimental findings and analysis on a KITTI dataset show that the model not only promotes the accuracy of tiny vehicle object detection but also improves the accuracy of vehicle detection (the accuracy reaches 94%) without decreasing the detection speed.

Towards the sustainable development of smart cities through mass video surveillance: A response to the COVID-19 pandemic

Sustainable smart city initiatives around the world have recently had great impact on the lives of citizens and brought significant changes to society. More precisely, data-driven smart applications that efficiently manage sparse resources are offering a futuristic vision of smart, efficient, and secure city operations. However, the ongoing COVID-19 pandemic has revealed the limitations of existing smart city deployment; hence; the development of systems and architectures capable of providing fast and effective mechanisms to limit further spread of the virus has become paramount. An active surveillance system capable of monitoring and enforcing social distancing between people can effectively slow the spread of this deadly virus. In this paper, we propose a data-driven deep learning-based framework for the sustainable development of a smart city, offering a timely response to combat the COVID-19 pandemic through mass video surveillance. To implementing social distancing monitoring, we used three deep learning-based real-time object detection models for the detection of people in videos captured with a monocular camera. We validated the performance of our system using a real-world video surveillance dataset for effective deployment.

Real-Time Vehicle Detection Framework Based on the Fusion of LiDAR and Camera

Vehicle detection is essential for driverless systems. However, the current single sensor detection mode is no longer sufficient in complex and changing traffic environments. Therefore, this paper combines camera and light detection and ranging (LiDAR) to build a vehicle-detection framework that has the characteristics of multi adaptability, high real-time capacity, and robustness. First, a multi-adaptive high-precision depth-completion method was proposed to convert the 2D LiDAR sparse depth map into a dense depth map, so that the two sensors are aligned with each other at the data level. Then, the You Only Look Once Version 3 (YOLOv3) real-time object detection model was used to detect the color image and the dense depth map. Finally, a decision-level fusion method based on bounding box fusion and improved Dempster–Shafer (D–S) evidence theory was proposed to merge the two results of the previous step and obtain the final vehicle position and distance information, which not only improves the detection accuracy but also improves the robustness of the whole framework. We evaluated our method using the KITTI dataset and the Waymo Open Dataset, and the results show the effectiveness of the proposed depth completion method and multi-sensor fusion strategy.

TF-YOLO: An Improved Incremental Network for Real-Time Object Detection

In recent years, significant advances have been gained in visual detection, and an abundance of outstanding models have been proposed. However, state-of-the-art object detection networks have some inefficiencies in detecting small targets. They commonly fail to run on portable devices or embedded systems due to their high complexity. In this workpaper, a real-time object detection model, termed as Tiny Fast You Only Look Once (TF-YOLO), is developed to implement in an embedded system. Firstly, the k-means++ algorithm is applied to cluster the dataset, which contributes to more excellent priori boxes of the targets. Secondly, inspired by the multi-scale prediction idea in the Feature Pyramid Networks (FPN) algorithm, the framework in YOLOv3 is effectively improved and optimized, by three scales to detect the earlier extracted features. In this way, the modified network is sensitive for small targets. Experimental results demonstrate that the proposed TF-YOLO method is a smaller, faster and more efficient network model increasing the performance of end-to-end training and real-time object detection for a variety of devices.

An optimization framework of video advertising: using deep learning algorithm based on global image information

Object detection and recognition technology plays an important role in video advertising. Based on the deep learning algorithm, this paper first builds a real-time object detection model (Yes-Net) based on image global information which combines the advantages of CNN and RNN algorithm together and has achieved significant effects compared with other object detection algorithms. We used this model to build an accurate video advertising framework and applied the framework in an IPTV platform of a telecommunication operator to collect comparative data. We first performed image detection and recognition in video playing, and then associated the recognition results with the ads database to conduct real-time classification and accurate ads delivery. It proves that compared with traditional advertisement delivery method, this new framework dramatically improves the click-through rate, and achieves better results.