Neural Networks For Object Detection Research Articles

Lane Detection Based on Two-Stage Noise Features Filtering and Clustering

Lane detection is essential for autonomous vehicles, and vision-based lane detection is widely used in the field of intelligent driving cars because of its low cost. Aiming to solve the problem of false detection caused by surrounding vehicles during lane detection, a new multi-lane detection method with strong anti-interference ability is proposed. The main contribution of this work is the utilization of object detection neural network which is applied to remove vehicles from road images as much as possible. Compared with the traditional feature-based detection algorithms, this method can preserve lane line features more accurately and effectively. Firstly, binarization of global optimal threshold method is utilized to extract the basic features of lanes. In order to further remove non-lane line noises and extract lane features as clear as possible, YOLOv4-tiny object detection network is introduced to detect and filter vehicles on the road. Then the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm with denoising function is applied to cluster the different lane features. Finally, an improved RANSAC (Random Sample Consensus) method is used to perform quadratic curve fitting on the clustered features. The experimental results on dataset show that consumed time of a single frame image is only 0.119 second and the precision of the proposed method can reach 94.4%. Compared with other detection algorithms, the proposed algorithm has good real-time performance, stability and robustness in a variety of complex scenarios.

Convolutional Neural Network for Object Detection in Garlic Root Cutting Equipment.

Traditional manual garlic root cutting is inefficient and can cause food safety problems. To develop food processing equipment, a novel and accurate object detection method for garlic using deep learning—a convolutional neural network—is proposed in this study. The you-only-look-once (YOLO) algorithm, which is based on lightweight and transfer learning, is the most advanced computer vision method for single large object detection. To detect the bulb, the YOLOv2 model was modified using an inverted residual module and residual structure. The modified model was trained based on images of bulbs with varied brightness, surface attachment, and shape, which enabled sufficient learning of the detector. The optimum minibatches and epochs were obtained by comparing the test results of different training parameters. Research shows that IRM-YOLOv2 is superior to the SqueezeNet, ShuffleNet, and YOLOv2 models of classical neural networks, as well as the YOLOv3 and YOLOv4 algorithm models. The confidence score, average accuracy, deviation, standard deviation, detection time, and storage space of IRM-YOLOv2 were 0.98228, 99.2%, 2.819 pixels, 4.153, 0.0356 s, and 24.2 MB, respectively. In addition, this study provides an important reference for the application of the YOLO algorithm in food research.

Apply Yolov4-Tiny on an FPGA-Based Accelerator of Convolutional Neural Network for Object Detection

With the continuous expansion of Neural Network technology in the artificial intelligence field, for example, image recognition and retrieval, object detection, pixel processing, automatic speech generation, etc., Convolutional Neural Networks (CNN) and Deep Learning of Neural Networks (DNN) have made apparent breakthroughs. To improve the inference speed of images, the combination of FPGA-based acceleration and multiple model quantization methods has become one of the most contemporary alternative methods. This paper designed an FPGA-Based acceleration scheme combining software and hardware and effectively applied it to the Yolov4-Tiny object detection model, realizing the accelerated detection inference process from the original 6-7mins to 383ms. First, it chose the static quantization method of fixed-point numbers, fixed the position of the decimal point, and then added Batch Norm between the convolutional layer and the activation function to form a connection structure. Second, it further improved inference speed on an FPGA with a version of ZYNQ-7020 by increasing the bandwidth cap and reducing bandwidth requirements, employing a massive pipeline design. Finally, in the test of the Coco dataset, the plan has completed a substantial acceleration of the average inference speed of the Yolov4-Tiny object detection model from 7.13mins/Picture to 498.89ms/Picture, which has a high application value in the field of object detection. It dramatically improves the inference speed as well as keeps the average accuracy above 0.95.

Using Synthetic Images to Evaluate and Improve Object Detection Neural Network Performance on Aerial Image Datasets

Machine learning-based models for object detection heavily rely on large datasets of labeled images. When models trained on these datasets are applied to Unmanned Aerial Vehicle (UAV) imagery, the problem arises that the conditions under which the training images were created (lighting, altitude, angle) may be different to the UAVs applied conditions, leading to misclassifications. This problem becomes even more pressing in safety critical applications where failures can have huge negative impacts and constitute obstacles for certification of cognitive UAV components. Along a case study on car detection in low-altitude aerial imagery, we show that using, both, artificial and real images for model training has a positive effect on the performance of object detection algorithms when the trained model is applied on images from another domain. Additionally, we show that weak points from object detection neural networks trained on real-world images transfer to synthetic images and that synthetic data can be used to evaluate neural networks trained on real-world data. Since simulated images are easy to create and object labels are inherently given, the presented approaches show a promising direction for scenarios where adequate datasets are difficult to obtain, as well as for the targeted exploration of weak points of object detection algorithms.

SyntEO: Synthetic dataset generation for earth observation and deep learning – Demonstrated for offshore wind farm detection

With the emergence of deep learning in the last years, new opportunities arose in Earth observation research. Nevertheless, they also brought with them new challenges. The data-hungry training processes of deep learning models demand large, resource expensive, annotated datasets and partly replaced knowledge-driven approaches so that model behaviour and the final prediction process became a black box. The proposed SyntEO approach enables Earth observation researchers to automatically generate large deep learning ready datasets by merging existing and procedural data. SyntEO does this by including expert knowledge in the data generation process in a highly structured manner to control the automatic image and label generation by employing an ontology. In this way, fully controllable experiment environments are set up, which support insights in the model training on the synthetic datasets. Thus, SyntEO makes the learning process approachable, which is an important cornerstone for explainable machine learning. We demonstrate the SyntEO approach by predicting offshore wind farms in Sentinel-1 images on two of the worlds largest offshore wind energy production sites. The largest generated dataset has 90,000 training examples. A basic convolutional neural network for object detection, that is only trained on this synthetic data, confidently detects offshore wind farms by minimising false detections in challenging environments. In addition, four sequential datasets are generated, demonstrating how the SyntEO approach can precisely define the dataset structure and influence the training process. SyntEO is thus a hybrid approach that creates an interface between expert knowledge and data-driven image analysis.

Development of a Machine Learning-Based Cysticidal Assay and Identification of an Amebicidal and Cysticidal Marine Microbial Metabolite against Acanthamoeba.

ABSTRACTTraditional cysticidal assays for Acanthamoeba species revolve around treating cysts with compounds and manually observing the culture for evidence of excystation. This method is time-consuming, labor-intensive, and low throughput. We adapted and trained a YOLOv3 machine learning, object detection neural network to recognize Acanthamoeba castellanii trophozoites and cysts in microscopy images to develop an automated cysticidal assay. This trained neural network was used to count trophozoites in wells treated with compounds of interest to determine if a compound treatment was cysticidal. We validated this new assay with known cysticidal and noncysticidal compounds. In addition, we undertook a large-scale bioluminescence-based screen of 9,286 structurally unique marine microbial metabolite fractions against the trophozoites of A. castellanii and identified 29 trophocidal hits. These hits were then subjected to this machine learning-based automated cysticidal assay. One marine microbial metabolite fraction was identified as both trophocidal and cysticidal.IMPORTANCE The free-living Acanthamoeba can exist as a trophozoite or cyst and both stages can cause painful blinding keratitis. Infection recurrence occurs in approximately 10% of cases due to the lack of efficient drugs that can kill both trophozoites and cysts. Therefore, the discovery of therapeutics that are effective against both stages is a critical unmet need to avert blindness. Current efforts to identify new anti-Acanthamoeba compounds rely primarily upon assays that target the trophozoite stage of the parasite. We adapted and trained a machine learning, object detection neural network to recognize Acanthamoeba trophozoites and cysts in microscopy images. Our machine learning-based cysticidal assay improved throughput, demonstrated high specificity, and had an exquisite ability to identify noncysticidal compounds. We combined this cysticidal assay with our bioluminescence-based trophocidal assay to screen about 9,000 structurally unique marine microbial metabolites against A. castellanii. Our screen identified a marine metabolite that was both trophocidal and cysticidal.

Image Recognition by Using a Convolutional Neural Network to Identify Objects for Driverless Car

Abstract: The concept of the paper was inspired by the recent surge in the automated car industry. The designed car was capable of detecting the road signals and taking the right and left turns accordingly. Object detection is a key ability required by most computer used in automated vehicles. The latest research in this area has been making great progress in many directions. Object detection and tracking has a variety of uses, our paper explain how to use convolutional neural network for object detection in autonomous vehicles. Automatic car always has the potential to solve traffic problems with the help of Convolution Neural Network (CNN). However, in the current scenario complete autonomy is still to be achieved. Although today's CNN have brought us closer to autonomy than ever before. CNN contain artificial neurons which are trained using preset rules and these rules determine whether it will provide an output or not when given a set of inputs. CNN will analyze various road footages, which include various scenarios such as collisions, empty roads, traffic, etc. CNN will analyze and send appropriate instructions to the car such as brake, accelerate, slow down, etc. Keywords: Object Detection System, Face Recognition, Object Detection, Convolution Neural Network

A Pipeline Approach to Context-Aware Handwritten Text Recognition

Despite concerted efforts towards handwritten text recognition, the automatic location and transcription of handwritten text remain a challenging task. Text detection and segmentation methods are often prone to errors, affecting the accuracy of the subsequent recognition procedure. In this paper, a pipeline that locates texts on a page and recognizes the text types, as well as the context of the texts within the detected region, is proposed. Clinical receipts are used as the subject of study. The proposed model is comprised of an object detection neural network that extracts text sequences present on the page regardless of size, orientation, and type (handwritten text, printed text, or non-text). After that, the text sequences are fed to a Residual Network with a Transformer (ResNet-101T) model to perform transcription. Next, the transcribed text sequences are analyzed using a Named Entity Recognition (NER) model to classify the text sequences into their corresponding contexts (e.g., name, address, prescription, and bill amount). In the proposed pipeline, all the processes are implicitly learned from data. Experiments performed on 500 self-collected clinical receipts containing 15,297 text segments reported a character error rate (CER) and word error rate (WER) of 7.77% and 10.77%, respectively.

An object detection network based on YOLOv4 and improved spatial attention mechanism

In recent years, the research on object detection has been intensified. A large number of object detection results are applied to our daily life, which greatly facilitates our work and life. In this paper, we propose a more effective object detection neural network model ENHANCE_YOLOV4. We studied the effects of several attention mechanisms on YOLOV4, and finally concluded that spatial attention mechanism had the best effect on YOLOV4. Therefore, based on previous studies, this paper introduces Dilated Convolution and one-by-one convolution into the spatial attention mechanism to expand the receptive field and combine channel information. Compared with CBAM and BAM, which are composed of spatial attention and channel attention, this improved spatial attention module reduces model parameters and improves detection capabilities. We built a new network model by embedding improved spatial attention module in the appropriate place in YOLOV4. And this paper proves that the detection accuracy of this network structure on the VOC data set is increased by 0.8%, and the detection accuracy on the coco data set is increased by 7%when the calculation performance is increased a little.

Development smart eyeglasses for visually impaired people based on you only look once

Visually impaired people are facing many problems in their life. One of these problems is how they can find the objects in their indoor environment. This research was presented to assists visually impaired people in finding the objects in office. Object detection is a method used to detect the objects in images and videos. Many algorithms used for object detection such as convolutional neural network (CNN) and you only look once (YOLO). The proposed method was YOLO which outperforms the other algorithms such as CNN. In CNN the algorithm splits the image into regions. These regions sequentially enters the neural network for object detection and recognition so CNN does not deal with all the regions at the same time but YOLO looks the entire image then it produces the bounding boxes with convolutional network and the probabilities of these boxes, this makes YOLO faster than other algorithms. Open source computer vision (OpenCV) used to capture frames by using camera. Then YOLO used to detect and recognize the objects in each frame. Finally, the sound in Arabic language was generated to tell the visually impaired people about the objects. The proposed system can detect 6 objects and achieve an accuracy of 99%.

Automatic Bunch Detection in White Grape Varieties Using YOLOv3, YOLOv4, and YOLOv5 Deep Learning Algorithms

Over the last few years, several Convolutional Neural Networks for object detection have been proposed, characterised by different accuracy and speed. In viticulture, yield estimation and prediction is used for efficient crop management, taking advantage of precision viticulture techniques. Convolutional Neural Networks for object detection represent an alternative methodology for grape yield estimation, which usually relies on manual harvesting of sample plants. In this paper, six versions of the You Only Look Once (YOLO) object detection algorithm (YOLOv3, YOLOv3-tiny, YOLOv4, YOLOv4-tiny, YOLOv5x, and YOLOv5s) were evaluated for real-time bunch detection and counting in grapes. White grape varieties were chosen for this study, as the identification of white berries on a leaf background is trickier than red berries. YOLO models were trained using a heterogeneous dataset populated by images retrieved from open datasets and acquired on the field in several illumination conditions, background, and growth stages. Results have shown that YOLOv5x and YOLOv4 achieved an F1-score of 0.76 and 0.77, respectively, with a detection speed of 31 and 32 FPS. Differently, YOLO5s and YOLOv4-tiny achieved an F1-score of 0.76 and 0.69, respectively, with a detection speed of 61 and 196 FPS. The final YOLOv5x model for bunch number, obtained considering bunch occlusion, was able to estimate the number of bunches per plant with an average error of 13.3% per vine. The best combination of accuracy and speed was achieved by YOLOv4-tiny, which should be considered for real-time grape yield estimation, while YOLOv3 was affected by a False Positive–False Negative compensation, which decreased the RMSE.

A Method to Automatic Create Dataset for Training Object Detection Neural Networks

Numerous high-accuracy neural networks for object detection have been proposed in recent years. Creating a specific training dataset seems to be the main obstacle in putting them into practice. This paper aims to overcome this obstacle and proposes an automatic dataset creation method. This method first extracts objects from the source images and then combines them as synthetic images. These synthetic images are automatic annotated using the data flow in the process and can be used directly for training. To the best of our knowledge, this is the first automatic dataset creation method for object detection tasks. In addition, the adaptive object extraction method and created natural synthetic images make the proposed method maintain strong adaptation and generalization ability. To validate the feasibility, a dataset that includes 44 categories of objects is created for the object detection task in a vending supermarket. Under the strictest metric <i>AP₇₅</i>, the trained EfficientDet achieves 98.7% accuracy on the common difficulty testing set and 93.1% accuracy on the high difficulty testing set.

Object Detection in Large-Scale Remote-Sensing Images Based on Time-Frequency Analysis and Feature Optimization

Recently, optical remote-sensing images have been steadily growing in size, as they contain massive data and complex backgrounds. This trend presents several problems for object detection, for example, increased computation time and memory consumption and more false positives due to the complex backgrounds of large-scale images. Inspired by deep neural networks combined with time-frequency analysis, we propose a time-frequency analysis-based object detection method for large-scale remote-sensing images with complex backgrounds. We utilize wavelet decomposition to carry out a time-frequency transform and then integrate it with deep learning in feature optimization. To effectively capture the time-frequency features, we propose a feature optimization method based on deep reinforcement learning to select the dominant time-frequency channels. Furthermore, we design a discrete wavelet multiscale attention mechanism (DW-MAM), enabling the detector to concentrate on the object area rather than the background. Extensive experiments show that the proposed method of learning from time-frequency channels not only solves the challenges of large-scale and complex backgrounds, but also improves the performance compared to the original state-of-the-art object detection methods. In addition, the proposed method can be used with almost all object detection neural networks, regardless of whether they are anchor-based or anchor-free detectors, horizontal or rotation detectors.

Автоматическая разметка данных для сегментации изображений документов с использованием глубоких нейронных сетей

The article proposes a new method for automatic data annotation for solving the problem of document image segmentation using deep object detection neural networks. The format of marked PDF files is considered as the initial data for markup. The peculiarity of this format is that it includes hidden marks that describe the logical and physical structure of the document. To extract them, a tool has been developed that simulates the operation of a stack-based printing machine according to the PDF format specification. For each page of the document, an image and annotation are generated in PASCAL VOC format. The classes and coordinates of the bounding boxes are calculated during the interpretation of the labeled PDF file based on the labels. To test the method, a collection of marked up PDF files was formed from which images of document pages and annotations for three segmentation classes (text, table, figure) were automatically obtained. Based on these data, a neural network of the EfficientDet D2 architecture was trained. The model was tested on manually labeled data from the same domain, which confirmed the effectiveness of using automatically generated data for solving applied problems.

Preoperative diagnosis of hepatocellular carcinoma patients with bile duct tumor thrombus using deep learning method

Preoperative diagnosis of bile duct tumor thrombus (BDTT) is clinically important as the surgical prognosis of hepatocellular carcinoma (HCC) patients with BDTT is significantly different from that of patients without BDTT. Although dilated bile ducts (DBDs) can act as biomarkers for diagnosing BDTT, it is easy for doctors to ignore DBDs when reporting the imaging scan result, leading to a high missed diagnosis rate in practice. This study aims to develop an artificial intelligence (AI) pipeline for automatically diagnosing HCC patients with BDTT using medical images. The proposed AI pipeline includes two stages. First, the object detection neural network Faster R-CNN was adopted to identify DBDs; then, an HCC patient was diagnosed with BDTT if the proportion of images with at least one identified DBD exceeded some threshold value. Based on 2354 CT images collected from 32 HCC patients (16 with BDTT and 16 without BDTT, 1∶1 matched), the proposed AI pipeline achieves an average true positive rate of 0.92 for identifying DBDs per patient and a patient-level true positive rate of 0.81 for diagnosing BDTT. The AUC value of the patient-level diagnosis of BDTT is 0.94 (95% CI: 0.87, 1.00), compared with 0.71 (95% CI: 0.51, 0.90) achieved by random forest based on preoperative clinical variables. The high accuracies demonstrate that the proposed AI pipeline is successful in the diagnosis and localization of BDTT using CT images.

Implementation of the PointPillars Network for 3D Object Detection in Reprogrammable Heterogeneous Devices Using FINN

In this paper, we present a hardware-software implementation of a deep neural network for object detection based on a point cloud obtained by a LiDAR sensor. The PointPillars network was used in the research, as it is a reasonable compromise between detection accuracy and calculation complexity. The Brevitas / PyTorch tools were used for network quantisation (described in our previous paper) and the FINN tool for hardware implementation in the reprogrammable Zynq UltraScale+ MPSoC device. The obtained results show that quite a significant computation precision limitation along with a few network architecture simplifications allows the solution to be implemented on a heterogeneous embedded platform with maximum 19% AP loss in 3D, maximum 8% AP loss in BEV and execution time 375ms (the FPGA part takes 262ms). We have also compared our solution in terms of inference speed with a Vitis AI implementation proposed by Xilinx (19 Hz frame rate). Especially, we have thoroughly investigated the fundamental causes of differences in the frame rate of both solutions. The code is available at https://github.com/vision-agh/pp-finn.

Crop stem detection and tracking for precision hoeing using deep learning

Developing alternatives to the chemical weeding process usually carried out in vegetable crop farming is necessary in order to reach a more sustainable agriculture. However, a precise mechanical weeding requires specific sensors and advanced computer vision algorithms to process crop and weed discrimination in real-time.In this paper we propose an algorithm able to detect, locate, and track the stem position of crops in images which is suitable for precision actions in vegetable fields such as mechanical hoeing within crop rows. The algorithm is twofold: (i) a deep neural network for object detection is first used to detect crop stems in individual RGB images and then (ii) an aggregation algorithm further refines the detections taking advantage of the temporal redundancy in consecutive frames.We evaluated the pipeline on images of maize and bean crops at an early stage of development, acquired in field conditions with a camera embedded in an experimental mechanical weeding system. We reported F1-scores of respectively 94.74% and 93.82% with a location accuracy around 0.7 cm when compared with human annotation. Moreover, this pipeline can operate in real-time on an embedded computer consuming as little power as 30 W.

LPNet: Retina Inspired Neural Network for Object Detection and Recognition

The detection of rotated objects is a meaningful and challenging research work. Although the state-of-the-art deep learning models have feature invariance, especially convolutional neural networks (CNNs), their architectures did not specifically design for rotation invariance. They only slightly compensate for this feature through pooling layers. In this study, we propose a novel network, named LPNet, to solve the problem of object rotation. LPNet improves the detection accuracy by combining retina-like log-polar transformation. Furthermore, LPNet is a plug-and-play architecture for object detection and recognition. It consists of two parts, which we name as encoder and decoder. An encoder extracts images which feature in log-polar coordinates while a decoder eliminates image noise in cartesian coordinates. Moreover, according to the movement of center points, LPNet has stable and sliding modes. LPNet takes the single-shot multibox detector (SSD) network as the baseline network and the visual geometry group (VGG16) as the feature extraction backbone network. The experiment results show that, compared with conventional SSD networks, the mean average precision (mAP) of LPNet increased by 3.4% for regular objects and by 17.6% for rotated objects.

Region-aggregated attention CNN for disease detection in fruit images.

BackgroundDiseases and pests have a profound effect on a yearly harvest and productivity in agriculture. A precise and accurate detection of the diseases and pests could facilitate timely treatment and management of the diseases and pests and lessen the resultant loss in economy and health. Herein, we propose an improved design of the disease detection system for plant images.MethodsBuilt upon the two-stage framework of object detection neural networks such as Mask R-CNN, the proposed network involves three types of extensions, including the addition of additional level of feature pyramids to improve the exploration and proposal of candidate regions, the aggregation of feature maps from all levels of feature pyramids per candidate region to fully exploit the information from feature pyramids, and the introduction of a squeeze-and-excitation block to the construction of feature pyramids and the aggregated feature maps to improve the representation of feature maps.ResultsThe proposed network was evaluated using 74 images of infected apple fruits. In 3-fold cross-validation, the proposed network achieved averaged precision (AP) of 72.26, AP at 0.5 threshold of 88.51 and AP at 0.75 threshold of 82.30. In the comparative experiments, the proposed network outperformed the other competing networks. The utility of the three extensions was also demonstrated in comparison to Mask R-CNN.ConclusionsThe experimental results suggest that the proposed network could identify and localize the symptom of the disease with high accuracy, leading to an early diagnosis and treatment of the disease, and thus holding the potential for improving crop yield and quality.

Road pollution estimation from vehicle tracking in surveillance videos by deep convolutional neural networks

Air quality and reduction of emissions in the transport sector are determinant factors in achieving a sustainable global climate. The monitoring of emissions in traffic routes can help to improve route planning and to design strategies that may make the pollution levels to be reduced. In this work, a method which detects the pollution levels of transport vehicles from the images of IP cameras by means of computer vision techniques and neural networks is proposed. Specifically, for each sequence of images, a homography is calculated to correct the camera perspective and determine the real distance for each pixel. Subsequently, the trajectory of each vehicle is computed by applying convolutional neural networks for object detection and tracking algorithms. Finally, the speed in each frame and the pollution emitted by each vehicle are determined. Experimental results on several datasets available in the literature support the feasibility and scalability of the system as an emission control strategy.