Performing Object Detection Research Articles

From Dataset Creation to Defect Detection: A Proposed Procedure for a Custom CNN Approach for Polishing Applications on Low-Performance PCs

This study focuses on training a custom, small Convolutional Neural Network (CNN) using a limited dataset through data augmentation that is aimed at developing weights for subsequent fine-tuning on specific defects, namely improperly polished aluminum surfaces. The objective is to adapt the network for use in computationally restricted environments. The methodology involves using two computers—a low-performance PC for network creation and initial testing and a more powerful PC for network training using the Darknet framework—after which the network is transferred back to the initial low-performance PC. The results demonstrate that the custom lightweight network suited for a low-performance PC effectively performs object detection under the described conditions. These findings suggest that using tailored lightweight networks for recognizing specific types of defects is feasible and warrants further investigation to enhance the industrial defect detection processes in limited computational settings. This approach highlights the potential for deploying AI-driven quality control in environments with constrained hardware capabilities.

Real-Time Object Detection and Tracking Based on Embedded Edge Devices for Local Dynamic Map Generation

This paper proposes a camera system designed for local dynamic map (LDM) generation, capable of simultaneously performing object detection, tracking, and 3D position estimation. This paper focuses on improving existing approaches to better suit our application, rather than proposing novel methods. We modified the detection head of YOLOv4 to enhance the detection performance for small objects and to predict fiducial points for 3D position estimation. The modified detector, compared to YOLOv4, shows an improvement of approximately 5% mAP on the Visdrone2019 dataset and around 3% mAP on our database. We also proposed a tracker based on DeepSORT. Unlike DeepSORT, which applies a feature extraction network for each detected object, the proposed tracker applies a feature extraction network once for the entire image. To increase the resolution of feature maps, the tracker integrates the feature aggregation network (FAN) structure into the DeepSORT network. The difference in multiple objects tracking accuracy (MOTA) between the proposed tracker and DeepSORT is minimal at 0.3%. However, the proposed tracker has a consistent computational load, regardless of the number of detected objects, because it extracts a feature map once for the entire image. This characteristic makes it suitable for embedded edge devices. The proposed methods have been implemented on a system on chip (SoC), Qualcomm QCS605, using network pruning and quantization. This enables the entire process to be executed at 10 Hz on this edge device.

Enhanced encoder–decoder architecture for visual perception multitasking of autonomous driving

Visual perception plays a vital role in autonomous driving systems, demanding high accuracy and real-time inference speed to ensure safety. In this paper, we propose a multi-task framework that simultaneously performs object detection, drivable area segmentation, and lane line identification, addressing the requirements of accurate and efficient visual perception. Our approach utilizes a shared-encoder architecture with three separate decoders, targeting each specific task. We investigate three configurations for the shared encoder: a Convolutional Neural Network (CNN), a Polyp Vision Transformer (PVT), and a hybrid CNN+PVT model. Through extensive experimentation and comparative analysis on the challenging BD100K dataset, we evaluate the performance of these shared-encoder models and provide valuable insights into their strengths and weaknesses. Our research contributes to the advancement of multi-task visual perception for autonomous driving systems by achieving competitive results in terms of accuracy and efficiency. The source code is publicly available on GitHub to facilitate further research in this domain.

Edge Computing Driven Low-Light Image Dynamic Enhancement for Object Detection

We focus on performing object detection in images taken under low-light conditions, which is critical task and often occurs in mobile multimedia computing environment. Unlike former methods to obtain enhanced images before detection with variant kinds of manually designed filters, we propose an edge computing and multi-task driven framework to complete tasks of image enhancement and object detection with fast response. The proposed framework consists of two stages, namely cloud-based enhancement stage and edge-based detection stage. In cloud-based enhancement stage, we establish connection between mobile users and cloud servers to input rescaled and small-size illumination parts of lowlight images, where enhancement subnetworks are dynamically combined to output several enhanced illumination parts and corresponding weights based on low-light context of input images. During edge-based detection stage, cloud-computed weights offers informativeness information on extracted feature maps to enhance their representation abilities, which results in accurate predictions on labels and positions for objects. By applying the proposed framework in cloud computing system, experimental results show it significantly improves detection performance in mobile multimedia and low-light environment.

Lightweight and Energy-Efficient Deep Learning Accelerator for Real-Time Object Detection on Edge Devices.

Tiny machine learning (TinyML) has become an emerging field according to the rapid growth in the area of the internet of things (IoT). However, most deep learning algorithms are too complex, require a lot of memory to store data, and consume an enormous amount of energy for calculation/data movement; therefore, the algorithms are not suitable for IoT devices such as various sensors and imaging systems. Furthermore, typical hardware accelerators cannot be embedded in these resource-constrained edge devices, and they are difficult to drive real-time inference processing as well. To perform the real-time processing on these battery-operated devices, deep learning models should be compact and hardware-optimized, and hardware accelerator designs also have to be lightweight and consume extremely low energy. Therefore, we present an optimized network model through model simplification and compression for the hardware to be implemented, and propose a hardware architecture for a lightweight and energy-efficient deep learning accelerator. The experimental results demonstrate that our optimized model successfully performs object detection, and the proposed hardware design achieves 1.25× and 4.27× smaller logic and BRAM size, respectively, and its energy consumption is approximately 10.37× lower than previous similar works with 43.95 fps as a real-time process under an operating frequency of 100 MHz on a Xilinx ZC702 FPGA.

Improved Ship Object Detection in Low-Illumination Environments Using RetinaMFANet

Video-based ship object detection has long been a popular research issue that has received attention in the water transportation industry. However, in low-illumination environments, such as at night or in fog, the water environment has a complex variety of light sources, video surveillance images are often accompanied by noise, and information on the details of objects in images is worsened. These problems cause high rates of false detection and missed detection when performing object detection for ships in low-illumination environments. Thus, this paper takes the detection of ship objects in low-illumination environments at night as the research object. The technical difficulties faced by object detection algorithms in low-illumination environments are analyzed, and a dataset of ship images is constructed by collecting images of ships (in the Nanjing section of Yangtze River in China) in low-illumination environments. In view of the outstanding performance of the RetinaNet model in general object detection, a new multiscale feature fusion network structure for a feature extraction module is proposed based on the same network architecture, in such a way that the extraction of more potential feature information from low-illumination images can be realized. In line with the feature detection network, the regression and classification detection network for anchor boxes is improved by means of the attention mechanism, guiding the network structure in the detection of object features. Moreover, the design and optimization of the augmentation of multiple random images and prior bounding boxes in the training process are also carried out. Finally, on the basis of experimental validation analysis, the optimized detection model was able to improve ship detection accuracy by 3.7% with a limited decrease in FPS (frames per second), and has better results in application.

FLNet: A Near-shore Ship Detection Method Based on Image Enhancement Technology

In the past few years, Synthetic Aperture Radar (SAR) has been widely used to detect marine ships due to its ability to work in various weather conditions. However, due to the imaging mechanism of SAR, there is a lot of background information and noise information similar to ships in the images, which seriously affects the performance of ship detection models. To solve the above problems, this paper proposes a new ship detection model called Feature enhancement and Land burial Net (FLNet), which blends traditional image processing methods with object detection approaches based on deep learning. We first design a SAR image threshold segmentation method, Salient Otsu (S-Otsu), according to the difference between the object and the noise background. To better eliminate noise in SAR images, we further combine image processing methods such as Lee filtering. These constitute a Feature Enhancement Module (FEM) that mitigates the impact of noise data on the overall performance of a ship detection model. To alleviate the influence of land information on ship detection, we design a Land Burial Module (LBM) according to the morphological differences between ships and land areas. Finally, these two modules are added to You Only Look Once V5 (YOLO V5) to form our FLNet. Experimental results on the SAR Ship Detection Dataset (SSDD) dataset show that FLNet comparison with YOLO V5 accuracy when performing object detection is improved by 7% and recall rate by 6.5%.

Instance Segmentation for Feature Recognition on Noncooperative Resident Space Objects

Active debris removal and unmanned on-orbit servicing missions have gained interest in the last few years, along with the possibility to perform them through the use of an autonomous chasing spacecraft. In this work, new resources are proposed to aid the implementation of guidance, navigation, and control algorithms for satellites devoted to the inspection of noncooperative targets before any proximity operation is initiated. In particular, the use of convolutional neural networks (CNN) performing object detection and instance segmentation is proposed, and its effectiveness in recognizing the components and parts of the target satellite is evaluated. Yet, no reliable training images dataset of this kind exists to date. A tailored and publicly available software has been developed to overcome this limitation by generating synthetic images. Computer-aided design models of existing satellites are loaded on a three-dimensional animation software and used to programmatically render images of the objects from different points of view and in different lighting conditions, together with the necessary ground truth labels and masks for each image. The results show how a relatively low number of iterations is sufficient for a CNN trained on such datasets to reach a mean average precision value in line with state-of-the-art performances achieved by CNN in common datasets. An assessment of the performance of the neural network when trained on different conditions is provided. To conclude, the method is tested on real images from the Mission Extension Vehicle-1 on-orbit servicing mission, showing that using only artificially generated images to train the model does not compromise the learning process.

IS-Count: Large-Scale Object Counting from Satellite Images with Covariate-Based Importance Sampling

Object detection in high-resolution satellite imagery is emerging as a scalable alternative to on-the-ground survey data collection in many environmental and socioeconomic monitoring applications. However, performing object detection over large geographies can still be prohibitively expensive due to the high cost of purchasing imagery and compute. Inspired by traditional survey data collection strategies, we propose an approach to estimate object count statistics over large geographies through sampling. Given a cost budget, our method selects a small number of representative areas by sampling from a learnable proposal distribution. Using importance sampling, we are able to accurately estimate object counts after processing only a small fraction of the images compared to an exhaustive approach. We show empirically that the proposed framework achieves strong performance on estimating the number of buildings in the United States and Africa, cars in Kenya, brick kilns in Bangladesh, and swimming pools in the U.S., while requiring as few as 0.01% of satellite images compared to an exhaustive approach.

A_CenterNet: Object as a Point by Attention

In this paper, in view of the insufficiency of the CenterNet detector’s inability to achieve high real-time performance with high accuracy when performing object detection, we designed a new detector called A_CenterNet. In the detector, we use our newly designed lightweight Hourglass-256 model, and we also use the feature map fusion method we designed, as well as our improved attention mechanism. Through the experimental results on multiple datasets, it can be known that the A_CenterNet proposed in this paper has a competitive advantage compared with some existing classic detectors. A_CenterNet achieves the best speed-accuracy trade-off on the MS COCO dataset, with 44.6 AP at 36 FPS. Compared with CenterNet, A_CenterNet greatly improves the detection speed without loss of detection accuracy.

Comparative study of 3D object detection frameworks based on LiDAR data and sensor fusion techniques

Estimating and understanding the surroundings of the vehicle precisely forms the basic and crucial step for the autonomous vehicle. The perception system plays a significant role in providing an accurate interpretation of a vehicle’s environment in real-time. Generally, the perception system involves various subsystems such as localization, obstacle (static and dynamic) detection, and avoidance, mapping systems, and others. For perceiving the environment, these vehicles will be equipped with various exteroceptive (both passive and active) sensors in particular cameras, Radars, LiDARs, and others. These systems are equipped with deep learning techniques that transform the huge amount of data from the sensors into semantic information on which the object detection and localization tasks are performed. For numerous driving tasks, to provide accurate results, the location and depth information of a particular object is necessary. 3D object detection methods, by utilizing the additional pose data from the sensors such as LiDARs, stereo cameras, provides information on the size and location of the object. Based on recent research, 3D object detection frameworks performing object detection and localization on LiDAR data and sensor fusion techniques show significant improvement in their performance. In this work, a comparative study of the effect of using LiDAR data for object detection frameworks and the performance improvement seen by using sensor fusion techniques are performed. Along with discussing various state-of-the-art methods in both the cases, performing experimental analysis, and providing future research directions.

PSNet: LiDAR and Camera Registration Using Parallel Subnetworks

The working environment of autonomous driving and robot navigation is so complex and dynamic that a single type of sensor is insufficient for performing object detection. Thus, in many perception schemes, the LiDAR-camera fusion strategy is preferred. However, the performance of a LiDAR-camera fusion heavily relies on a set of accurately calibrated extrinsic parameters. We propose PSNet, an end-to-end convolutional neural network (CNN) for calibration; this is the first calibration network to use parallel subnetworks to obtain multiresolution features and fuse them adaptively to encourage robustness against different initial error ranges. The method has three key characteristics: (i) Addition of a downsampling block to improve suitability for sparse projected depth maps; (ii) Connection of the high-to-low resolution convolution streams in parallel to obtain multiresolution features that are spatially more precise and contain richer semantic information; (iii) Fusion of multiresolution streams by the multiscale feature aggregation module. The network corrects errors from initial calibration to the ground truth online, rather than directly obtaining the accurate parameters. We evaluated our model on the KITTI datasets and it outperformed other CNN-based methods. In addition, extensive experiments evaluating the model with untrained and unfamiliar datasets demonstrated that our method exhibited good generalization ability.

Enhanced the moving object detection and object tracking for traffic surveillance using RBF-FDLNN and CBF algorithm

For traffic surveillance systems (TSS), Moving object detection and tracking is the key technology. On account of moving object-orientation variation, changing weather conditions, moving objects’ appearance, and non-target objects in the background, it's very hard to detection some objects in a traffic sequence of the video frame. Though many methods were introduced to detect and track moving objects in a traffic environment, they could not attain a desirable output. Hence, this work proposes an enhanced system for automatic moving object detection (MOD) and object tracking (OT) using RBF-FDLNN and CFR algorithms. The noises existent in a video frame are initially filtered using a GESFA filter. Next, the conversion of the RGB frame to HSL is done, and the background regions are eradicated from the frame using IGMM. After background removal (BGR), the Motion Estimation (ME) of object and OT are performed using GCS and CFR algorithm, respectively. Subsequently, the required features in the frames are extracted and fed into the RBF-FDLNN classifier for performing object detection (OD). The performance of the proposed method RBF-FDLNN classifier is better than other existing methods in the given video frame. Lastly, the outcomes are proffered to corroborate the proposed method’s effectiveness.

Automatic extraction of rockfall source based on terrain analysis map using support vector machine

Disaster prevention inspections, without overlooking the sources of falling rock, are essential for establishing efficient slope management and countermeasures. This study presents an automatic extraction method for rockfall sources, using remote sensing and artificial intelligence technology, to reduce overlooking during the inspection cycle and improve slope management efficiency. Current inspections have some factors that lead to object overlooking, one of them being the use of maps with low expression accuracy. Furthermore, biased criteria for the interpretation of inspection points in a desk study can play a part. Therefore, improving map accuracy and using quantified interpretation methods will improve the current inspection significantly. In such cases, the use of remote sensing technology is an effective measure. The utilization of airborne laser surveying and terrain analysis methods is effective for accurately acquiring the slope surface and topography. Airborne laser surveying is a system that takes measurements with multiple sensors mounted on an airplane, and the measurement data are represented by a collection of multiple points with three-dimensional coordinates. Furthermore, the terrain analysis method, which converts the survey data to two-dimensional raster images, extracts the necessary information, such as the ridge, valley, and elevation, of the slope. In this study, two-dimensional continuous wavelet transforms, used as a terrain analysis method suitable for rockfall extraction, are adopted to create a wavelet analysis map from survey data. Furthermore, to automatically extract the inspection points, the classification technology in artificial intelligence (AI) is applied to the terrain analysis map to extract the rockfall source in a desk study. A support vector machine (SVM) is a type of AI model that classifies based on training data and works by determining the best possible separation between the closest observations belonging to different classes. By applying this classification method, the rockfall source was extracted by performing object detection on the map. First, the entire map was divided into smaller patch images. Next, each patch image is classified as a rockfall source using the trained SVM. Finally, the area corresponding to the patch image, classified as the rockfall source, was drawn on the entire map. In this study, the performance of these integrated systems was verified in an area with a falling rock hazard. In the training process, wavelet analysis maps that reflect inventory data based on past inspection results were used. The extraction performance was evaluated by comparing the verification results with the inventory data and interpretation results based on the map features. Consequently, all learned rockfall source points were extracted, obtaining high-precision readability. Furthermore, the extraction performance of the same tendency inside the inventory range was shown in the range outside the inventory, acquiring a high versatility. Accordingly, we discuss the possibilities and issues for automatic extraction of the desk survey using the proposed method.

Automated image identification, detection and fruit counting of top-view pineapple crown using machine learning

Automated fruit identification or recognition using image processing is a key element in precision agriculture for performing object detection in large crop plots. Automation of fruit recognition for the captured top-view of RGB based images using an unmanned aerial vehicle (UAV) is a challenge. Image analysis demonstrated the difficulty of processing the captured image under variant illumination in natural environment and with textured objects of non-ideal geometric shapes. However, this is subjected to certain consideration settings and image-processing algorithms. The study presents an automatic method for identifying and recognising the pineapple’s crown images in the designated plot using image processing and further counts the detected images using machine learning classifiers namely artificial neural network (ANN), support vector machine (SVM), random forest (RF), naive Bayes (NB), decision trees (DT) and k-nearest neighbours (KNN). The high spatial-resolution aerial images were pre-processed and segmented, and its extracted features were analysed according to shape, colour and texture for recognising the pineapple crown before classifying it as fruit or non-fruit. Feature fusion using one-way analysis of variance (ANOVA) was incorporated in this study to optimise the performance of machine learning classifier. The algorithm was quantitatively analysed and validated for performance via accuracy, specificity, sensitivity and precision. The detection for the pineapple’s crown images with ANN-GDX classification has demonstrated best performance fruit counting with accuracy of 94.4% and has thus demonstrated clear potential application of an effective RGB images analysis for the pineapple industry.

IoT Based Road Traffic Control System for Bangladesh

The existing traffic administration policy is not worthy enough to tackle the density of movement in Bangladesh. This study proposes an advanced Internet of Things (IoT) based road traffic administration system to resolve the problem. All the smart lamp posts of road crossings handle four factors, i.e., number of cars, activation time, waiting time, and emergency signal of each lane. This research uses an automatic video processing method to count the number of cars on the road. In order to process the video mask, R-CNN is used, which is a combination of the faster R-CNN that performs object detection (class + bounding box), and Fully Convolutional Network (FCN) results into a pixel border. Modern statistical methods are also used, such as multiple regression analysis, cluster analysis, and factor analysis. For handling emergency traffic situations, a new activation function was proposed and named the RT activation function. Factor analysis with principal component analysis (PCA) allowed in reducing the number of variables from elevens to five. The linear regression explains 90.2% of the variance in the data. This research considers R, R-square, adjusted R-square with 0.950, 0.902, and 0.409, values respectively. The results analysis ensures that the performance of the proposed schema is good enough to apply in the road of Bangladesh.

RAVIP: Real-Time AI Vision Platform for Heterogeneous Multi-Channel Video Stream

Object detection techniques based on deep learning such as YOLO have high detection performance and precision in a single channel video stream. In order to expand to multiple channel object detection in real-time, however, high-performance hardware is required. In this paper, we propose a novel back-end server framework, a real-time AI vision platform (RAVIP), which can extend the object detection function from single channel to simultaneous multi-channels, which can work well even in low-end server hardware. RAVIP assembles appropriate component modules from the RODEM (real-time object detection module) Base to create perchannel instances for each channel, enabling efficient parallelization of object detection instances on limited hardware resources through continuous monitoring with respect to resource utilization. Through practical experiments, RAVIP shows that it is possible to optimize CPU, GPU, and memory utilization while performing object detection service in a multi-channel situation. In addition, it has been proven that RAVIP can provide object detection services with 25 FPS for all 16 channels at the same time.

Real-Time Weakly Supervised Object Detection Using Center-of-Features Localization

We propose a high-speed convolutional neural network approach for weakly supervised localization (WSL) and weakly supervised object detection (WSOD). The proposed method, called center-of-features localization (COFL), performs localization of objects in a visual scene by combining both multi-label classification and regression for the number of instances of each class object. A modified Xception network architecture is used as the main feature extractor, and a classification-plus-regression loss function is used to perform the detection task. The method does not require bounding box annotations but only image labels and counts of the objects of each class in the image. This combination can produce a clear localization of objects in the scene through a masking technique between class activation maps (CAMs) and regression activation maps (RAMs). The proposed method was trained and tested on the PASCAL VOC2007 and VOC2012 datasets; it attained a mean average precision (mAP) of 47.0% and a correct localization CorLoc of 64.1% on PASCAL VOC2007 and a mAP of 42.3% and a CorLoc of 65.5% on PASCAL VOC2012 while performing object detection at a speed of ~50 fps. These results demonstrate that the network can perform object detection accurately in real-time using only image labels and object counts, which are inexpensive to annotate compared with the bounding box annotations typically employed in fully supervised object detection methods. The network far outperforms other weakly supervised methods and some fully supervised methods in terms of processing time while achieving fair accuracy.

Automatic Faults Detection of Photovoltaic Farms: solAIr, a Deep Learning-Based System for Thermal Images

Renewable energy sources will represent the only alternative to limit fossil fuel usage and pollution. For this reason, photovoltaic (PV) power plants represent one of the main systems adopted to produce clean energy. Monitoring the state of health of a system is fundamental. However, these techniques are time demanding, cause stops to the energy generation, and often require laboratory instrumentation, thus being not cost-effective for frequent inspections. Moreover, PV plants are often located in inaccessible places, making any intervention dangerous. In this paper, we propose solAIr, an artificial intelligence system based on deep learning for anomaly cells detection in photovoltaic images obtained from unmanned aerial vehicles equipped with a thermal infrared sensor. The proposed anomaly cells detection system is based on the mask region-based convolutional neural network (Mask R-CNN) architecture, adopted because it simultaneously performs object detection and instance segmentation, making it useful for the automated inspection task. The proposed system is trained and evaluated on the photovoltaic thermal images dataset, a publicly available dataset collected for this work. Furthermore, the performances of three state-of-art deep neural networks, (DNNs) including UNet, FPNet and LinkNet, are compared and evaluated. Results show the effectiveness and the suitability of the proposed approach in terms of intersection over union (IoU) and the Dice coefficient.

A loss-balanced multi-task model for simultaneous detection and segmentation

Scene understanding comes in many flavors, two of the most popular being object detection and semantic segmentation, which act as two important aspects for scene understanding, and are applied to many areas, such as autonomous driving and intelligent surveillance. Although much progress has already been made, the two tasks of object detection and semantic segmentation are often investigated independently. In practice, scene understanding is complicated, and comprises many sub-tasks, so that research of learning multiple tasks simultaneously with a single model is feasible. With the interrelated goals of these two tasks, there is a strong motivation to improve the object detection accuracy with the help of semantic segmentation, and vice versa. In this paper, we propose a loss-balanced multi-task model for simultaneous object detection and semantic segmentation. We explore multi-task learning with sharing parameters based on deep learning to realize improved object detection and segmentation, and propose a single-stage deep architecture based on multi-task learning, jointly performing object detection and semantic segmentation to boost each other. With no more computation load in the inference compared with the baselines of SSD and FCN, we show that these two tasks, object detection and semantic segmentation, benefit from each other. Experimental results on Pascal VOC and COCO show that our method improves much in object detection and semantic segmentation compared with the corresponding baselines of both tasks.