Neural Networks For Object Detection Research Articles

Evaluation of different deep convolutional neural networks for detection of broadleaf weed seedlings in wheat.

In-field weed detection in wheat (Triticum aestivum L.) is challenging due to the occurrence of weeds in close proximity with the crop. The objective of this research was to evaluate the feasibility of using deep convolutional neural networks for detecting broadleaf weed seedlings growing in wheat. The object detection neural networks, including CenterNet, Faster R-CNN, TridenNet, VFNet, and You Only Look Once Version 3 (YOLOv3) were insufficient for weed detection in wheat because the recall never exceeded 0.58 in the testing dataset. The image classification neural networks including AlexNet, DenseNet, ResNet, and VGGNet were trained with small (5500 negative and 5500 positive images) or large training datasets (11 000 negative and 11 000 positive images) and three training image sizes (200 × 200, 300 × 300, and 400 × 400 pixels). For the small training dataset, increasing image sizes decreased the F1 scores of AlexNet and VGGNet but generally increased the F1 scores of DenseNet and ResNet. For the large training dataset, no obvious difference was detected between the training image sizes since all neural networks exhibited remarkable classification accuracies with high F1 scores (≥0.96). All image classification neural networks exhibited high F1 scores (≥0.99) when trained with the large training dataset and the training images of 200 × 200 pixels. CenterNet, Faster R-CNN, TridentNet, VFNet, and YOLOv3 were insufficient, while AlexNet, DenseNet, ResNet, and VGGNet trained with a large training dataset were highly effective for detection of broadleaf weed seedlings in wheat. © 2021 Society of Chemical Industry.

Evaluation of Deep Learning-Based Neural Network Methods for Cloud Detection and Segmentation

This paper presents a systematic approach for accurate short-time cloud coverage prediction based on a machine learning (ML) approach. Based on a newly built omnidirectional ground-based sky camera system, local training and evaluation data sets were created. These were used to train several state-of-the-art deep neural networks for object detection and segmentation. For this purpose, the camera-generated a full hemispherical image every 30 min over two months in daylight conditions with a fish-eye lens. From this data set, a subset of images was selected for training and evaluation according to various criteria. Deep neural networks, based on the two-stage R-CNN architecture, were trained and compared with a U-net segmentation approach implemented by CloudSegNet. All chosen deep networks were then evaluated and compared according to the local situation.

A New Method Based on Deep Convolutional Neural Networks for Object Detection and Classification

Accurate object detection and classification has a broad application in industrial tasks, such as fabric defect and invoice detection. Previous state-of-the-art methods such as SSD and Faster-RCNN usually need to carefully adjust anchor box related hyper parameters and have poor performance in special fields with large object size/ratio variations and complex background texture. In this study, we proposed a new accurate, robust, and anchor-free method to handle automatic object detection and classification problems. First, we used the feature pyramid network (FPN), to merge the feature maps of different scales of features extracted from a convolutional neural network (CNN), which allowed easy and robust multi-scale feature fusion. Second, we built two subnets to generate candidate region proposals from the FPN outputs. followed by another CNN that determined the categories of the proposed regions from the two subnets.

Integrating object proposal with attention networks for video saliency detection

Video saliency detection is an active research issue in both information science and visual psychology. In this paper, we propose an efficient video saliency-detection model, based on integrating object-proposal with attention networks, for efficiently capturing salient objects and human attention areas in the dynamic scenes of videos. In our algorithm, visual object features are first exploited from individual video frame, using real-time neural networks for object detection. Then, the spatial position information of each frame is used to screen out the large background in the video, so as to reduce the influence of background noises. Finally, the results, with backgrounds removed, are further refined by spreading the visual clues through an adaptive weighting scheme into the later layers of a convolutional neural network. Experimental results, conducted on widespread and commonly used databases for video saliency detection, verify that our proposed framework outperforms existing deep models.

Surrogate Object Detection Explainer (SODEx) with YOLOv4 and LIME

Due to impressive performance, deep neural networks for object detection in images have become a prevalent choice. Given the complexity of the neural network models used, users of these algorithms are typically given no hint as to how the objects were found. It remains, for example, unclear whether an object is detected based on what it looks like or based on the context in which it is located. We have developed an algorithm, Surrogate Object Detection Explainer (SODEx), that can explain any object detection algorithm using any classification explainer. We evaluate SODEx qualitatively and quantitatively by detecting objects in the COCO dataset with YOLOv4 and explaining these detections with LIME. This empirical evaluation does not only demonstrate the value of explainable object detection, it also provides valuable insights into how YOLOv4 detects objects.

Rotational multipyramid network with bounding‐box transformation for object detection

The study proposes a rotational multipyramid network (RoMP Net) with bounding-box transformation for object detection. The RoMP Net is a single-stage object detection neural network featuring three characteristics. First, the network uses a rotational bounding box to minimize the effect of background images when extracting features of objects. Bounding-box transformation was proposed to compensate for the limitation of the rotational bounding boxes, which have relatively low prediction accuracy for objects with a high aspect ratio. Second, the RoMP Net introduces a multi-scale and multi-level feature pyramid network to extract distinct and semantic features efficiently. This network architecture ensures high prediction accuracy and robustness regardless of the size and complexity of objects. Third, hyperparameters in the bounding boxes are automatically determined through an unsupervised clustering method. This optimization method is also critical in improving accuracy. The performance of the proposed network and preprocessing methods are validated through image-sets comprising critical components in power transmission facilities, which have a variety of sizes and aspect ratios. This case study demonstrates the effectiveness and robustness of the three key characteristics in the RoMP Net. Furthermore, the RoMP Net outperforms other state-of-the-art deep neural networks in prediction accuracy and robustness for object detection. Specifically, the mean average precision of the RoMP Net in the validation image-sets shows that it has the highest prediction accuracy, whereas its values in the test image-sets confirm the network's robustness. The fast yet accurate RoMP Net will expand the range of object detection through deep neural networks.

High precision localization of pulmonary nodules on chest CT utilizing axial slice number labels

BackgroundReidentification of prior nodules for temporal comparison is an important but time-consuming step in lung cancer screening. We develop and evaluate an automated nodule detector that utilizes the axial-slice number of nodules found in radiology reports to generate high precision nodule predictions.Methods888 CTs from Lung Nodule Analysis were used to train a 2-dimensional (2D) object detection neural network. A pipeline of 2D object detection, 3D unsupervised clustering, false positive reduction, and axial-slice numbers were used to generate nodule candidates. 47 CTs from the National Lung Cancer Screening Trial (NLST) were used for model evaluation.ResultsOur nodule detector achieved a precision of 0.962 at a recall of 0.573 on the NLST test set for any nodule. When adjusting for unintended nodule predictions, we achieved a precision of 0.931 at a recall 0.561, which corresponds to 0.06 false positives per CT. Error analysis revealed better detection of nodules with soft tissue attenuation compared to ground glass and undeterminable attenuation. Nodule margins, size, location, and patient demographics did not differ between correct and incorrect predictions.ConclusionsUtilization of axial-slice numbers from radiology reports allowed for development of a lung nodule detector with a low false positive rate compared to prior feature-engineering and machine learning approaches. This high precision nodule detector can reduce time spent on reidentification of prior nodules during lung cancer screening and can rapidly develop new institutional datasets to explore novel applications of computer vision in lung cancer imaging.

Yolo-tiny-MSC: A tiny neural network for object detection

Object detection is an important and active area of computer vision. However, the expensive computational cost and memory requirement are big challenges for deploying object detection networks in resource constraints embedded devices. To address this problem, we designed the multi-stage cascade module to reduce the computational cost of object detection networks. The proposed structure is used to build a compact and efficient object detection neural network. Experiments show that our multi-stage cascade module could significantly reduce the computational and parameter budgets meanwhile achieve outstanding object detection accuracy.

A dual neural network for object detection in UAV images

Computer vision analysis in Unmanned Aerial Vehicle (UAV) represents a major trend in the future development. Object detection in UAV images is one of the important techonlogies to analyze scene information. However, the UAV image includes small and dense targets, which easily leads to errors of missed detection. In this paper, we propose a dual neural network review method, which quickly screens the missed targets in the one-stage detection by classifying the secondary features of the suspected target regions, so as to achieve high quality detection of small targets. Firstly, the one-stage detector recognizes UAV images, and the result with confidence greater than or equal to the threshold is detected as the target. The result less than the threshold are considered as suspected areas containing missed targets. Secondly, the feature map of UAV image is extracted by VGG backbone. The feature map and the location information of the suspected areas are combined to secondary identification. Then, the features of the dual network are late fusion, and the re-identified results guide the initial confidence addition. After the addition, regions with confidence greater than the threshold are considered as targets. Finally, we synthesize the targets of initial detection and secondary identification to obtain the final detection results. Experimental results show that our method achieves breakthrough performance on VisDrone, UAVDT and MS COCO datasets.

Deep Consensus Network: Aggregating predictions to improve object detection in microscopy images

Detection of cells and particles in microscopy images is a common and challenging task. In recent years, detection approaches in computer vision achieved remarkable improvements by leveraging deep learning. Microscopy images pose challenges like small and clustered objects, low signal to noise, and complex shape and appearance, for which current approaches still struggle. We introduce Deep Consensus Network, a new deep neural network for object detection in microscopy images based on object centroids. Our network is trainable end-to-end and comprises a Feature Pyramid Network-based feature extractor, a Centroid Proposal Network, and a layer for ensembling detection hypotheses over all image scales and anchors. We suggest an anchor regularization scheme that favours prior anchors over regressed locations. We also propose a novel loss function based on Normalized Mutual Information to cope with strong class imbalance, which we derive within a Bayesian framework. In addition, we introduce an improved algorithm for Non-Maximum Suppression which significantly reduces the algorithmic complexity. Experiments on synthetic data are performed to provide insights into the properties of the proposed loss function and its robustness. We also applied our method to challenging data from the TUPAC16 mitosis detection challenge and the Particle Tracking Challenge, and achieved results competitive or better than state-of-the-art.

Low bit‐based convolutional neural network for one‐class object detection

Low-performance systems such as mobile and embedded devices require an efficient deep neural network for object detection. In this letter, we propose a very efficient network made by both quantisation and model compression for detecting one class. First, our proposed network uses 1-bit weights to reduce the kernel parameter size and 8-bit activations to increase the speed. Second, we optimise the model size and computational power by compressing the maximum number of channels of the network. Therefore, compared to Darknet19, our proposed network infers 35 times faster on the CPU and saves over 7000 times memory. For fair evaluations, we built one-class object detection databases to detect subtitles of various videos and a specific class from the Pascal VOC database. We verify, compared to Darknet19 and Tiny you only look once (YOLO), that the proposed optimised network does not degrade in object detection accuracy with the efficient and applicable parameter sizes and computational complexity.

Fully Automated DCNN-Based Thermal Images Annotation Using Neural Network Pretrained on RGB Data.

One of the biggest challenges of training deep neural network is the need for massive data annotation. To train the neural network for object detection, millions of annotated training images are required. However, currently, there are no large-scale thermal image datasets that could be used to train the state of the art neural networks, while voluminous RGB image datasets are available. This paper presents a method that allows to create hundreds of thousands of annotated thermal images using the RGB pre-trained object detector. A dataset created in this way can be used to train object detectors with improved performance. The main gain of this work is the novel method for fully automatic thermal image labeling. The proposed system uses the RGB camera, thermal camera, 3D LiDAR, and the pre-trained neural network that detects objects in the RGB domain. Using this setup, it is possible to run the fully automated process that annotates the thermal images and creates the automatically annotated thermal training dataset. As the result, we created a dataset containing hundreds of thousands of annotated objects. This approach allows to train deep learning models with similar performance as the common human-annotation-based methods do. This paper also proposes several improvements to fine-tune the results with minimal human intervention. Finally, the evaluation of the proposed solution shows that the method gives significantly better results than training the neural network with standard small-scale hand-annotated thermal image datasets.

Robust Real-Time Traffic Surveillance with Deep Learning.

Real-time vehicle monitoring in highways, roads, and streets may provide useful data both for infrastructure planning and for traffic management in general. Even though it is a classic research area in computer vision, advances in neural networks for object detection and classification, especially in the last years, made this area even more appealing due to the effectiveness of these methods. This study presents TrafficSensor, a system that employs deep learning techniques for automatic vehicle tracking and classification on highways using a calibrated and fixed camera. A new traffic image dataset was created to train the models, which includes real traffic images in poor lightning or weather conditions and low-resolution images. The proposed system consists mainly of two modules, first one responsible of vehicle detection and classification and a second one for vehicle tracking. For the first module, several neural models were tested and objectively compared, and finally, the YOLOv3 and YOLOv4-based network trained on the new traffic dataset were selected. The second module combines a simple spatial association algorithm with a more sophisticated KLT (Kanade–Lucas–Tomasi) tracker to follow the vehicles on the road. Several experiments have been conducted on challenging traffic videos in order to validate the system with real data. Experimental results show that the proposed system is able to successfully detect, track, and classify vehicles traveling on a highway on real time.

An improved semi-supervised transfer learning method for infrared object detection neural network

An improved semi-supervised transfer learning method for infrared object detection neural network

Faster R-CNN classification for the recognition of glaucoma

Glaucoma is an optic neuropathy characterized by progressive degeneration of retinal ganglion cells. The early identification of Glaucoma is extremely important as it is detrimental to one’s blindness. In this paper, we present the identification of glaucoma using faster R-CNN which is one of the most well-known object detection neural networks. The proposed method uses artificial intelligence and enhanced deep learning to detect Glaucoma. Faster R-CNN comprises two modules, the region proposal network (RPN), in which the region of object is distinguished on the picture, and a network that enables to classify the objects in the proposed region. We have accomplished the finest output by applying a transfer learning scheme with ResNet50 and VGG16. Using ResNet we have detected Glaucoma with up to 96% accuracy. The test results obtained by making use of two unique publicly available data sets DRISHTI_GS and ORIGA with 751 images demonstrate that this arrangement can be a significant alternative for the computer design aid framework for the large-scale screening programs of glaucoma detection.

Object detection neural network improves Fourier ptychography reconstruction.

High resolution microscopy is heavily dependent on superb optical elements and superresolution microscopy even more so. Correcting unavoidable optical aberrations during post-processing is an elegant method to reduce the optical system's complexity. A prime method that promises superresolution, aberration correction, and quantitative phase imaging is Fourier ptychography. This microscopy technique combines many images of the sample, recorded at differing illumination angles akin to computed tomography and uses error minimisation between the recorded images with those generated by a forward model. The more precise knowledge of those illumination angles is available for the image formation forward model, the better the result. Therefore, illumination estimation from the raw data is an important step and supports correct phase recovery and aberration correction. Here, we derive how illumination estimation can be cast as an object detection problem that permits the use of a fast convolutional neural network (CNN) for this task. We find that faster-RCNN delivers highly robust results and outperforms classical approaches by far with an up to 3-fold reduction in estimation errors. Intriguingly, we find that conventionally beneficial smoothing and filtering of raw data is counterproductive in this type of application. We present a detailed analysis of the network's performance and provide all our developed software openly.

Automated mapping of cultural heritage in Norway from airborne lidar data using faster R-CNN

The existing cultural heritage mapping in Norway is incomplete. Some selected areas are mapped well, while the majority of areas only contain chance discoveries, often with bad positional accuracy. The goal of this research was to develop automated tools for improving the cultural heritage mapping in Norway, thus enabling detailed mapping of large areas within realistic budgets and time frames. The focus was on three types of cultural heritage that occur frequently in many types of Norwegian landscape: grave mounds, pitfall traps in deer hunting systems and charcoal kilns.A recent development in deep neural networks for object detection in natural images is the region-proposing convolutional neural network (R-CNN), which may also be used for cultural heritage detection in local relief model (LRM) visualizations of airborne laser scanning (ALS) data. Python code for ‘Simple Faster R-CNN’ was downloaded from Github.On 737 test images (16.6 km2) not seen during training, 87 % of the true cultural heritage objects were correctly identified, while 24 % of the predicted cultural heritage locations were false. However, all test images were small (150 m × 150 m) and contained at least one cultural heritage object, meaning that the false positive rate may be higher for an entire landscape. In Larvik municipality, Vestfold and Telemark County, on a 67 km2 area not seen during training, the R-CNN correctly identified 38 % of the true grave mounds, with 89 % false positives. On a 937 km2 area covering Øvre Eiker municipality, Viken County, the R-CNN correctly identified 90 % of the charcoal kilns, with 38 % false positives.In conclusion, we have demonstrated that Faster R-CNN is well suited for semi-automatic detection of cultural heritage objects such as charcoal kilns, grave mounds and pitfall traps in high resolution airborne lidar data. However, it is desirable to reduce the false positive rate in order to limit the amount of visual inspection needed when the method is applied to large areas for detailed archaeological mapping.

LRCN-RetailNet: A recurrent neural network architecture for accurate people counting

Measuring and analyzing the flow of customers in retail stores is essential for a retailer to better comprehend customers’ behavior and support decision-making. Nevertheless, not much attention has been given to the development of novel technologies for automatic people counting. We introduce LRCN-RetailNet: a recurrent neural network architecture capable of learning a non-linear regression model and accurately predicting the people count from videos captured by low-cost surveillance cameras. The input video format follows the recently proposed RGBP image format, which is comprised of color and people (foreground) information. Our architecture is capable of considering two relevant aspects: spatial features extracted through convolutional layers from the RGBP images; and the temporal coherence of the problem, which is exploited by recurrent layers. We show that, through a supervised learning approach, the trained models are capable of predicting the people count with high accuracy. Additionally, we present and demonstrate that a straightforward modification of the methodology is effective to exclude salespeople from the people count. Comprehensive experiments were conducted to validate, evaluate and compare the proposed architecture. Results corroborated that LRCN-RetailNet remarkably outperforms both the previous RetailNet architecture, which was limited to evaluating a single image per iteration; and two state-of-the-art neural networks for object detection. Finally, computational performance experiments confirmed that the entire methodology is effective to estimate people count in real-time.

SYNTHETIC VISION SYSTEM CALIBRATION FOR CONFORM PROJECTION ON THE PILOT’S HEAD-UP DISPLAY

Abstract. Situational awareness of the crew is critical for the safety of the air flight. Head-up display allows providing all required flight information in front of the pilot over the cockpit view visible through the cockpit’s front window. This device has been created for solving the problem of informational overload during piloting of an aircraft. While computer graphics such as scales and digital terrain model can be easily presented on the such display, errors in the Head-up display alignment for correct presenting of sensor data pose challenges. The main problem arises from the parallax between the pilot’s eyes and the position of the camera. This paper is focused on the development of an online calibration algorithm for conform projection of the 3D terrain and runway models on the pilot’s head-up display. The aim of our algorithm is to align the objects visible through the cockpit glass with their projections on the Head-up display. To improve the projection accuracy, we use an additional optical sensor installed on the aircraft. We combine classical photogrammetric techniques with modern deep learning approaches. Specifically, we use an object detection neural network model to find the runway area and align runway projection with its actual location. Secondly, we re-project the sensor’s image onto the 3D model of the terrain to eliminate errors caused by the parallax. We developed an environment simulator to evaluate our algorithm. Using the simulator we prepared a large training dataset. The dataset includes 2000 images of video sequences representing aircraft’s motion during takeoff, landing and taxi. The results of the evaluation are encouraging and demonstrate both qualitatively and quantitatively that the proposed algorithm is capable of precise alignment of the 3D models projected on a Head-up display.

A MATCH-MOVING METHOD COMBINING AI AND SFM ALGORITHMS IN HISTORICAL FILM FOOTAGE

Abstract. Searching for suitable material for photogrammetry is a key part in the documentation of Cultural Heritage. Photogrammetry can be used to produce a metrically certified 3D model. Material contained in historical film footage archives is especially useful for documentation when the heritage has been lost. In this research an innovative match-moving method is proposed that aims to exploit Artificial Intelligence and SfM algorithms to identify the frames extracted from a film footage in which the lost monument appears and that are suitable to be processed with photogrammetry for its 3D reconstruction. First of all the identification and tracking of the heritage in the videos was performed training an object detection Neural Network. Then the frames detected were automatically extracted with the coordinates of the bounding boxes that contain the monument. The camera motions were identified by selecting only the shots taken from multiple points of view of the same scene and analysing the evolution of the bounding boxes position over time. A further check of the material was necessary to select only sequences and to eliminate single frames and images from different historic periods. After this process, only the correct frames were automatically selected and processed with photogrammetry and the quality of the obtained 3D model was assessed. The method experimented in this research represents a powerful tool in the field of Cultural Heritage because it makes the selection of suitable material for photogrammetry automatic. Moreover it offers important insights that could be extended to other sectors.