You Only Look Once V3 Research Articles

Three-Dimensional Target Detection Based on RGB-D Data

The current three-dimensional (3D) target detection model has a low accuracy, because the surface information of the target can only be partially represented by its two-dimensional (2D) image detector. To solve the problem, this paper studies the 3D target detection in the RGB-D data of indoor scenes, and modifies the frustum PointNet (F-PointNet), a model superior in point cloud data processing, to detect indoor targets like sofa, chair, and bed. The 2D image detector of F-PointNet was replaced with you only look once (YOLO) v3 and faster region-based convolutional neural network (R-CNN) respectively. Then, the F-PointNet models with the two 2D image detectors were compared on SUN RGB-D dataset. The results show that the model with YOLO v3 did better in target detection, with a clear advantage in mean average precision (>6.27).

Individual Detection and Tracking of Group Housed Pigs in Their Home Pen Using Computer Vision

Modern welfare definitions not only require that the Five Freedoms are met, but animals should also be able to adapt to changes (i. e., resilience) and reach a state that the animals experience as positive. Measuring resilience is challenging since relatively subtle changes in animal behavior need to be observed 24/7. Changes in individual activity showed potential in previous studies to reflect resilience. A computer vision (CV) based tracking algorithm for pigs could potentially measure individual activity, which will be more objective and less time consuming than human observations. The aim of this study was to investigate the potential of state-of-the-art CV algorithms for pig detection and tracking for individual activity monitoring in pigs. This study used a tracking-by-detection method, where pigs were first detected using You Only Look Once v3 (YOLOv3) and in the next step detections were connected using the Simple Online Real-time Tracking (SORT) algorithm. Two videos, of 7 h each, recorded in barren and enriched environments were used to test the tracking. Three detection models were proposed using different annotation datasets: a young model where annotated pigs were younger than in the test video, an older model where annotated pigs were older than the test video, and a combined model where annotations from younger and older pigs were combined. The combined detection model performed best with a mean average precision (mAP) of over 99.9% in the enriched environment and 99.7% in the barren environment. Intersection over Union (IOU) exceeded 85% in both environments, indicating a good accuracy of the detection algorithm. The tracking algorithm performed better in the enriched environment compared to the barren environment. When false positive tracks where removed (i.e., tracks not associated with a pig), individual pigs were tracked on average for 22.3 min in the barren environment and 57.8 min in the enriched environment. Thus, based on proposed tracking-by-detection algorithm, pigs can be tracked automatically in different environments, but manual corrections may be needed to keep track of the individual throughout the video and estimate activity. The individual activity measured with proposed algorithm could be used as an estimate to measure resilience.

High-Accuracy Real-Time Fish Detection Based on Self-Build Dataset and RIRD-YOLOv3

To better detect fish in an aquaculture environment, a high-accuracy real-time detection model is proposed. An experimental dataset was collected for fish detection in laboratory aquaculture environments using remotely operated vehicles. To overcome the inaccuracy of the You Only Look Once v3 (YOLOv3) algorithm in underwater farming environment, a suitable set of hyperparameters was obtained through multiple sets of experiments. Then, a real-time image recovery algorithm is applied before YOLOv3 to reduce the effects of both noise and light on images whilst keeping the real-time capability, leading to a mean average precision of 0.85 and frame rate of 17.6 fps, respectively. Finally, compared with the base detection model using only the YOLOv3 algorithm, the enhanced detection model presented results in a reduction of miss detection rate from 23% to only 9% across different environments and with the detection accuracy of the target in different environments being improved from 8% to 37%.

Automatic Detection and Tracking of Marker Seeds Implanted in Prostate Cancer Patients using a Deep Learning Algorithm.

Purpose:Fiducial marker seeds are often used as a surrogate to identify and track the positioning of prostate volume in the treatment of prostate cancer. Tracking the movement of prostate seeds aids in minimizing the prescription dose spillage outside the target volume to reduce normal tissue complications. In this study, You Only Look Once (YOLO) v2™ (MathWorks™) convolutional neural network was employed to train ground truth datasets and develop a program in MATLAB that can visualize and detect the seeds on projection images obtained from kilovoltage (kV) X-ray volume imaging (XVI) panel (Elekta™).Methods:As a proof of concept, a wax phantom containing three gold marker seeds was imaged, and kV XVI seed images were labeled and used as ground truth to train the model. The projection images were corrected for any panel shift using flex map data. Upon successful testing, labeled marker seeds and projection images of three patients were used to train a model to detect fiducial marker seeds. A software program was developed to display the projection images in real-time and predict the seeds using YOLO v2 and determine the centers of the marker seeds on each image.Results:The fiducial marker seeds were successfully detected in 98% of images from all gantry angles; the variation in the position of the seed center was within ± 1 mm. The percentage difference between the ground truth and the detected seeds was within 3%.Conclusion:Our study shows that deep learning can be used to detect fiducial marker seeds in kV images in real time. This is an ongoing study, and work is underway to extend it to other sites for tracking moving structures with minimal effort.

Deep learning approaches for challenging species and gender identification of mosquito vectors

Microscopic observation of mosquito species, which is the basis of morphological identification, is a time-consuming and challenging process, particularly owing to the different skills and experience of public health personnel. We present deep learning models based on the well-known you-only-look-once (YOLO) algorithm. This model can be used to simultaneously classify and localize the images to identify the species of the gender of field-caught mosquitoes. The results indicated that the concatenated two YOLO v3 model exhibited the optimal performance in identifying the mosquitoes, as the mosquitoes were relatively small objects compared with the large proportional environment image. The robustness testing of the proposed model yielded a mean average precision and sensitivity of 99% and 92.4%, respectively. The model exhibited high performance in terms of the specificity and accuracy, with an extremely low rate of misclassification. The area under the receiver operating characteristic curve (AUC) was 0.958 ± 0.011, which further demonstrated the model accuracy. Thirteen classes were detected with an accuracy of 100% based on a confusion matrix. Nevertheless, the relatively low detection rates for the two species were likely a result of the limited number of wild-caught biological samples available. The proposed model can help establish the population densities of mosquito vectors in remote areas to predict disease outbreaks in advance.

Efficient Real-Time Human Detection Using Unmanned Aerial Vehicles Optical Imagery

ABSTRACT Unmanned Aerial Vehicles (UAVs) are promising technologies within many different application scenarios including human detection in search and rescue and surveillance use cases, which have received considerable attention worldwide. However, adverse conditions, such as varying altitude, overhead camera placement, changing illumination and moving platform, impose challenges for high-performance yet cost-efficient human detection. To overcome these challenges, we propose a novel combination of dilated convolutions with Path Aggregation Network (PAN) as a new deep neural network-based human detection algorithm in real time. Furthermore, we establish a comprehensive human detection dataset with varying backgrounds, illuminations, and contrast and train the proposed machine-learning model on the collected dataset. Our approach achieves both high precision (88.0% mean Average Precision (mAP)) and real time (67.0 Frames Per Second (FPS)) on a commercial off-the-shelf PC platform. In terms of accuracy, the result is comparable to the standard You Only Look Once v3 (YOLOv3). However, the speed is twice as that of the standard YOLOv3. YOLOv4 is slightly more accurate (89.8%) than our approach. However, it is slower (38.0 versus 67.0 FPS) and has more Billion Floating-Point Operations (BFLOPS). The proposed algorithm has also trained with the VisDrone2019 dataset and compared with seven studies using this dataset. The results have further validated the effectiveness of the proposed approach. Moreover, the algorithm has been evaluated on an embedded system (Jetson AGX Xavier), which demonstrates the usefulness of this method on power-constrained devices. The proposed algorithm is fast, memory efficient, and computationally less expensive to achieve high detection performance. It is expected to contribute significantly to the wider use of UAV applications including search and rescue missions to locate missing people, and surveillance particularly for applications running on resource-constrained platforms, like smartphones or tablets. This proposed system is now being used in aerial drone system of Police of Scotland to help them locate and find missing and vulnerable people. The results of the project were broadcasted by BBC Scotland.

Multitarget Real‐Time Tracking Algorithm for UAV IoT

Unmanned aerial vehicles (UAVs) have increased the convenience of urban life. Representing the recent rapid development of drone technology, UAVs have been widely used in fifth‐generation (5G) cellular networks and the Internet of Things (IoT), such as drone aerial photography, express drone delivery, and drone traffic supervision. However, owing to low altitude and low speed, drones can only limitedly monitor and detect small target objects, resulting in frequent intrusion and collision. Traditional methods of monitoring the safety of drones are mostly expensive and difficult to implement. In smart city construction, a large number of smart IoT cameras connected to 5G networks are installed in the city. Captured drone images are transmitted to the cloud via a high‐speed and low‐latency 5G network, and machine learning algorithms are used for target detection and tracking. In this study, we propose a method for real‐time tracking of drone targets by using the existing monitoring network to obtain drone images in real time and employing deep learning methods by which drones in urban environments can be guided. To achieve real‐time tracking of UAV targets, we employed the tracking‐by‐detection mode in machine learning, with the network‐modified YOLOv3 (you only look once v3) as the target detector and Deep SORT as the target tracking correlation algorithm. We established a drone tracking dataset that contains four types of drones and 2800 pictures in different environments. The tracking model we trained achieved 94.4% tracking accuracy in real‐time UAV target tracking and a tracking speed of 54 FPS. These results comprehensively demonstrate that our tracking model achieves high‐precision real‐time UAV target tracking at a reduced cost.

Study on Local Optical Flow Method Based on YOLOv3 in Human Behavior Recognition

In the process of human behavior recognition, the traditional dense optical flow method has too many pixels and too much overhead, which limits the running speed. This paper proposed a method combing YOLOv3 (You Only Look Once v3) and local optical flow method. Based on the dense optical flow method, the optical flow modulus of the area where the human target is detected is calculated to reduce the amount of computation and save the cost in terms of time. And then, a threshold value is set to complete the human behavior identification. Through design algorithm, experimental verification and other steps, the walking, running and falling state of human body in real life indoor sports video was identified. Experimental results show that this algorithm is more advantageous for jogging behavior recognition.

Development and Optimization of a Deep-Learning-Based Egg-Collecting Robot

HighlightsAn egg-collecting robot embedded with deep-learning-based computer vision algorithms was constructed.The retrained deep-learning detector achieved >93% performance for detecting and locating eggs in images.The kernel sizes of 65 × 65 pixels for erosion and dilation in image processing assisted in extracting geometry features of eggs with the least remaining noises.The robot with soft grouting sponge on the gripper had 92% to 94% success in picking white and brown eggs.Abstract. Manual collection of floor eggs in cage-free hen housing systems is time-consuming and laborious. The objectives of this study were to (1) develop a robot to automatically collect floor eggs and (2) optimize the performance of recognizing and picking eggs with the robot. The robot consisted of a deep-learning-based egg detector, a robot arm, a two-finger gripper, and a hand-mounted camera. The deep-learning model, You Only Look Once (YOLO) V3, was embedded in the vision system to detect and locate eggs on a simulated litter floor in real-time. Image processing algorithms (e.g., cropping, erosion, etc.) were implemented for the detection and provided the robot with centroid coordinates, orientation, and axis lengths of the detected eggs, so that the gripper could be manipulated with an appropriate angle and opening width to grasp the detected eggs. For optimization, the YOLO V3 model was retrained with a dataset of floor eggs and achieved >93% performance in detecting and locating eggs. The kernel size of 65 × 65 pixels for erosion and dilation in image processing assisted in extracting the geometry features of eggs with the least remaining noises. Among the tested materials, soft grouting sponge attached to the gripper had the highest success rates for egg picking. The robot achieved 92% to 94% success in picking white and brown eggs. In sum, the developed egg-collecting robot can be relied on for picking floor eggs to assist in precision management of cage-free hen housing systems. Keywords: Floor egg, Image processing, Laying hen, Robot arm, Soft gripper, YOLO V3.

Key Parts of Transmission Line Detection Using Improved YOLO v3

Unmanned Aerial Vehicle (UAV) inspection has become one of main methods for current transmission line inspection, but there are still some shortcomings such as slow detection speed, low efficiency, and inability for low light environment. To address these issues, this paper proposes a deep learning detection model based on You Only Look Once (YOLO) v3. On the one hand, the neural network structure is simplified, that is the three feature maps of YOLO v3 are pruned into two to meet specific detection requirements. Meanwhile, the K-means++ clustering method is used to calculate the anchor value of the data set to improve the detection accuracy. On the other hand, 1000 sets of power tower and insulator data sets are collected, which are inverted and scaled to expand the data set, and are fully optimized by adding different illumination and viewing angles. The experimental results show that this model using improved YOLO v3 can effectively improve the detection accuracy by 6.0%, flops by 8.4%, and the detection speed by about 6.0%.

An implementation methodology for Neural Network on a Low-end FPGA Board

Artificial Intelligence(AI) has achieved unprecedented success in various fields that include image, speech, or even video recognition. Most systems are implemented on power-hungry devices like CPU, GPU, or even TPU to process data due to the models' high computation and storage complexity. CPU platforms do weak in computation capacity, while energy budgets and expense of GPU and TPU are often not affordable to edge computing in the industrial business. Recently, the FPGA-based Neural Network (NN) accelerator has been a trendy topic in the research field. It is regarded as a promising solution to suppress GPU in both speed and energy efficiency with its specifically designed architecture. Our work performs on a low-end FPGA board, a more desirable platform in meeting the restrictions of energy efficiency and computational resource on an autonomous driving car. We propose a methodology that integrates a NN model into the board using HLS description in this paper. The whole design consists of algorithm-level downscaling and hardware optimization. The former emphasizes the model downscale through model pruning and binarization, which balance the model size and accuracy. The latter applies various HLS design techniques on each NN component, like loop unrolling, inter- /intra- level pipelining, and so on, to speed-up the application running on the target board. In the case study of tiny YOLO (You Only Look Once) v3, the model running on PYNQ-Z1 presents up to 22x acceleration comparing with the PYNQ's ARM CPU. Energy efficiency also achieves 3x better than Xeon E5-2667. To verify the flexibility of our methodology, we extend our work to the BinaryConnect and DoReFaNet. It is worth mentioning that the BinaryConnect even achieves around 100x acceleration comparing with it purely running on the PYNQ-Z1 ARM core.

Real Time Vehicle Detection, Tracking, and Inter-vehicle Distance Estimation based on Stereovision and Deep Learning using YOLOv3

In this paper, we propose a robust real-time vehicle tracking and inter-vehicle distance estimation algorithm based on stereovision. Traffic images are captured by a stereoscopic system installed on the road, and then we detect moving vehicles with the YOLO V3 Deep Neural Network algorithm. Thus, the real-time video goes through an algorithm for stereoscopy-based measurement in order to estimate distance between detected vehicles. However, detecting the real-time objects have always been a challenging task because of occlusion, scale, illumination etc. Thus, many convolutional neural network models based on object detection were developed in recent years. But they cannot be used for real-time object analysis because of slow speed of recognition. The model which is performing excellent currently is the unified object detection model which is You Only Look Once (YOLO). But in our experiment, we have found that despite of having a very good detection precision, YOLO still has some limitations. YOLO processes every image separately even in a continuous video or frames. Because of this much important identification can be lost. So, after the vehicle detection and tracking, inter-vehicle distance estimation is done.

Intelligent co-detection of cyclists and motorcyclists based on an improved deep learning method

Vulnerable road user (VRU) collision avoidance is a challenge task in developing an advanced driving assistant system (ADAS). This is because motorcyclists are the most VRUs, while most existing ADAS systems are not capable of distinguishing motorcyclists from cyclists. In order to address this problem, this study proposes a new detection method based on a center and scale prediction-proposal region (CSP-PR) improved deep learning model. This new method attempts to identify the individual difference and similar appearance between cyclists and motorcyclists from different perspectives; by doing so, the cyclists and motorcyclists can be co-detected from each other to enable effective VRU collision avoidance. For this purpose, a shared salient region extraction method is first introduced based on the CSP-PR; then multiple-instance target candidate regions are generated based on the redundancy strategy; last, a multi-task learning method is proposed to train an ResNet50-deep neural network to reduce the risk of overfitting and improve the co-detection rate of the cyclists and motorcyclists. In order to verify the effectiveness of the proposed CSP-PR-ResNet50 method, a local motorcyclist database and the Tsinghua-Daimler cyclist database were combined to form an experimental dataset. The experimental results demonstrate that the proposed method is able to effectively identify the cyclists and the motorcyclists. Compared with popular algorithms, including the faster region-based convolutional neural networks, YOLOv3 (You Only Look Once v3) and single shot multibox detector, the average precision (AP) of the cyclist detection produced by the proposed method is improved by 3.69%, 17.74% and 31.22%, respectively; and the AP of the motorcyclists detection is improved by 4.29%, 15.5% and 32.57%, respectively.

Application of a Model that Combines the YOLOv3 Object Detection Algorithm and Canny Edge Detection Algorithm to Detect Highway Accidents

This study proposed a model for highway accident detection that combines the You Only Look Once v3 (YOLOv3) object detection algorithm and Canny edge detection algorithm. It not only detects whether an accident has occurred in front of a vehicle, but further performs a preliminary classification of the accident to determine its severity. First, this study established a dataset consisting of around 4500 images mainly taken from the angle of view of dashcams from an open-source online platform. The dataset was named the Highway Dashcam Car Accident for Classification System (HDCA-CS) and was developed with the aim of conforming to the setting of this study. The HDCA-CS not only considers weather conditions (rainy days, foggy days, nighttime settings, and other low-visibility conditions), but also various types of accidents, thus increasing the diversity of the dataset. In addition, we proposed two types of accidents—accidents involving damaged cars and accidents involving overturned cars—and developed three different design methods for comparing vehicles involved in accidents involving damaged cars. Canny edge detection algorithm processed single high-resolution images of accidents were also added to compensate for the low volume of accident data, thereby addressing the problem of data imbalance for training purposes. Lastly, the results showed that the proposed model achieved a mean average precision (mAP) of 62.60% when applied to the HDCA-CS testing dataset. When comparing the proposed model with a benchmark model, two abovementioned accident types were combined to allow the proposed model to produce binary classification outputs (i.e., non-occurrence and occurrence of an accident). The HDCA-CS was then applied to the two models, and testing was conducted using single high-resolution images. At 76.42%, the mAP of the proposed model outperformed the benchmark model’s 75.18%; and if we were to apply the proposed model to only test scenarios in which an accident has occurred, its performance would be even better relative to the benchmark. Therefore, our findings demonstrate that our proposed model is superior to other existing models.

Real-Time Pattern-Recognition of GPR Images with YOLO v3 Implemented by Tensorflow.

Artificial intelligence (AI) is widely used in pattern recognition and positioning. In most of the geological exploration applications, it needs to locate and identify underground objects according to electromagnetic wave characteristics from the ground-penetrating radar (GPR) images. Currently, a few robust AI approach can detect targets by real-time with high precision or automation for GPR images recognition. This paper proposes an approach that can be used to identify parabolic targets with different sizes and underground soil or concrete structure voids based on you only look once (YOLO) v3. With the TensorFlow 1.13.0 developed by Google, we construct YOLO v3 neural network to realize real-time pattern recognition of GPR images. We propose the specific coding method for the GPR image samples in Yolo V3 to improve the prediction accuracy of bounding boxes. At the same time, K-means algorithm is also applied to select anchor boxes to improve the accuracy of positioning hyperbolic vertex. For some instances electromagnetic-vacillated signals may occur, which refers to multiple parabolic electromagnetic waves formed by strong conductive objects among soils or overlapping waveforms. This paper deals with the vacillating signal similarity intersection over union (IoU) (V-IoU) methods. Experimental result shows that the V-IoU combined with non-maximum suppression (NMS) can accurately frame targets in GPR image and reduce the misidentified boxes as well. Compared with the single shot multi-box detector (SSD), YOLO v2, and Faster-RCNN, the V-IoU YOLO v3 shows its superior performance even when implemented by CPU. It can meet the real-time output requirements by an average 12 fps detected speed. In summary, this paper proposes a simple and high-precision real-time pattern recognition method for GPR imagery, and promoted the application of artificial intelligence or deep learning in the field of the geophysical science.

Automatic recognition of dairy cow mastitis from thermal images by a deep learning detector

Mastitis is a disease that affects dairy cow health, and the timely detection of mastitis can improve the efficiency of mastitis treatment and reduce economic losses in the milk industry. To improve the detection speed and achieve automatic recognition of dairy cow mastitis, this study proposed a deep learning network EFMYOLOv3 (Enhanced Fusion MobileNetV3 You Only Look Once v3) based on the bilateral filtering enhancement of thermal images. EFMYOLOv3 is used to automatically detect dairy cow eyes and udders and is applied to the detection of mastitis in dairy cows based on thermal infrared images. We proposed a bilateral filtering image enhancement algorithm based on gray histograms to enhance image details to compensate for weak thermal image details and enhance the contrast between the foreground and background. We chose the lightweight MobileNetV3 as the backbone of YOLOv3. Based on the location attention mechanism, we used the multiscale enhanced fusion feature pyramid network structure as the feature extraction module. The feature map used for prediction was designed with the appropriate resolution and powerful multilayer semantic features to improve the accuracy of target detection. We replaced the standard convolutions in the base layer with depthwise separable convolutions to reduce the number of learning parameters. To verify the effectiveness of the target detection algorithm, the accuracy, recall, average frame rate, average accuracy and other indicators were compared with the SSD (single shot multibox detector) and YOLOv3 (You Only Look Once v3) algorithms. The test results revealed that the average frame rate of the EFMYOLOv3 algorithm is 99 frames per second (fps), and the average accuracy is 96.8%, which means that the key parts of the cow can be detected quickly and accurately. The temperature difference between the eyes and the udders was obtained by the target detection algorithm, and the mastitis detection of dairy cows was performed and compared with the somatic cell count (SCC). The results showed that the accuracy of the mastitis classification algorithm is 83.33%, and the sensitivity and specificity are 92.31% and 76.47%, respectively. This method realized accurate positioning of key parts of dairy cows and can be used for the automatic recognition of dairy cow mastitis.

Improved YOLO v3 network-based object detection for blind zones of heavy trucks

Object detection for blind zones is critical to ensuring the driving safety of heavy trucks. We propose a scheme to realize object detection in the blind zones of heavy trucks based on the improved you-only-look-once (YOLO) v3 network. First, according to the actual detection requirements, the targets are determined to establish a new data set of persons, cars, and fallen pedestrians, with a focus on small and medium objects. Subsequently, the network structure is optimized, and the features are enhanced by combining the shallow and deep convolution information of the Darknet platform. In this way, the feature propagation can be effectively enhanced, feature reuse can be promoted, and the network performance for small object detection can be improved. Furthermore, new anchors are obtained by clustering the data set using the K-means technique to improve the accuracy of the detection frame positioning. In the test stage, detection is performed using the trained model. The test results demonstrate that the proposed improved YOLO v3 network is superior to the original YOLO v3 model in terms of the blind zone detection and can satisfy the accuracy and real-time requirements with an accuracy of 94% and runtime of 13.792 ms / frame. Moreover, the mean average precision value for the improved model is 87.82%, which is 2.79% higher than that of the original YOLO v3 network.

A Deep Learning Approach for Weed Detection in Lettuce Crops Using Multispectral Images

Weed management is one of the most important aspects of crop productivity; knowing the amount and the locations of weeds has been a problem that experts have faced for several decades. This paper presents three methods for weed estimation based on deep learning image processing in lettuce crops, and we compared them to visual estimations by experts. One method is based on support vector machines (SVM) using histograms of oriented gradients (HOG) as feature descriptor. The second method was based in YOLOV3 (you only look once V3), taking advantage of its robust architecture for object detection, and the third one was based on Mask R-CNN (region based convolutional neural network) in order to get an instance segmentation for each individual. These methods were complemented with a NDVI index (normalized difference vegetation index) as a background subtractor for removing non photosynthetic objects. According to chosen metrics, the machine and deep learning methods had F1-scores of 88%, 94%, and 94% respectively, regarding to crop detection. Subsequently, detected crops were turned into a binary mask and mixed with the NDVI background subtractor in order to detect weed in an indirect way. Once the weed image was obtained, the coverage percentage of weed was calculated by classical image processing methods. Finally, these performances were compared with the estimations of a set from weed experts through a Bland–Altman plot, intraclass correlation coefficients (ICCs) and Dunn’s test to obtain statistical measurements between every estimation (machine-human); we found that these methods improve accuracy on weed coverage estimation and minimize subjectivity in human-estimated data.

MOVING SHIP DETECTION AND MOVEMENT PREDICTION IN REMOTE SENSING VIDEOS

Abstract. With the rapid development of remote sensing technology, it is possible to obtain continuous video data from outer space successfully. It is of great significance in military and civilian fields to detect moving objects from the remote sensing image sequence and predict their movements. In recent years, this issue has attracted more and more attention. However, researches on moving object detection and movement prediction in high-resolution remote sensing videos are still in its infancy, which is worthy of further study. In this paper, we propose a ship detection and movement prediction method based on You-Only-Look-Once (YOLO) v3 and Simple Online and Realtime Tracking (SORT). Original YOLO v3 is improved by multi-frame training to fully utilize the information of continuous frames in a fusion way. The simple and practical multiple object tracking algorithm SORT is used to recognize multiple targets detected by multi-frame YOLO v3 model and obtain their coordinates. These coordinates are fitted by the least square method to get the trajectories of multiple targets. We take the derivative of each trajectory to obtain the real-time movement direction and velocity of the detected ships. Experiments are performed on multi-spectral remote sensing images selected on Google Earth, as well as real multi-spectral remote sensing videos captured by Jilin-1 satellite. Experimental results validate the effectiveness of our method for moving ship detection and movement prediction. It shows a feasible way for efficient interpretation and information extraction of new remote sensing video data.

Improve YOLOv3 using dilated spatial pyramid module for multi-scale object detection

Effectively and efficiently recognizing multi-scale objects is one of the key challenges of utilizing deep convolutional neural network to the object detection field. YOLOv3 (You only look once v3) is the state-of-the-art object detector with good performance in both aspects of accuracy and speed; however, the scale variation is still the challenging problem which needs to be improved. Considering that the detection performances of multi-scale objects are related to the receptive fields of the network, in this work, we propose a novel dilated spatial pyramid module to integrate multi-scale information to effectively deal with scale variation problem. Firstly, the input of dilated spatial pyramid is fed into multiple parallel branches with different dilation rates to generate feature maps with different receptive fields. Then, the input of dilated spatial pyramid and outputs of different branches are concatenated to integrate multi-scale information. Moreover, dilated spatial pyramid is integrated with YOLOv3 in front of the first detection header to present dilated spatial pyramid-You only look once model. Experiment results on PASCAL VOC2007 demonstrate that dilated spatial pyramid-You only look once model outperforms other state-of-the-art methods in mean average precision, while it still keeps a satisfying real-time detection speed. For 416 × 416 input, dilated spatial pyramid-You only look once model achieves 82.2% mean average precision at 56 frames per second, 3.9% higher than YOLOv3 with only slight speed drops.