Neural Network Based Object Detection Research Articles

Ontology based autonomous robot task processing framework.

In recent years, the perceptual capabilities of robots have been significantly enhanced. However, the task execution of the robots still lacks adaptive capabilities in unstructured and dynamic environments. In this paper, we propose an ontology based autonomous robot task processing framework (ARTProF), to improve the robot's adaptability within unstructured and dynamic environments. ARTProF unifies ontological knowledge representation, reasoning, and autonomous task planning and execution into a single framework. The interface between the knowledge base and neural network-based object detection is first introduced in ARTProF to improve the robot's perception capabilities. A knowledge-driven manipulation operator based on Robot Operating System (ROS) is then designed to facilitate the interaction between the knowledge base and the robot's primitive actions. Additionally, an operation similarity model is proposed to endow the robot with the ability to generalize to novel objects. Finally, a dynamic task planning algorithm, leveraging ontological knowledge, equips the robot with adaptability to execute tasks in unstructured and dynamic environments. Experimental results on real-world scenarios and simulations demonstrate the effectiveness and efficiency of the proposed ARTProF framework. In future work, we will focus on refining the ARTProF framework by integrating neurosymbolic inference.

Automatic Characterization of WEDM Single Craters Through AI Based Object Detection

Wire electrical discharge machining (WEDM) is a process that removes material from conductive workpieces by using sequential electrical discharges. The morphology of the craters formed by these discharges is influenced by various process parameters and affects the quality and efficiency of the machining. To understand and optimize the WEDM process, it is essential to identify and characterize single craters from microscopy images. However, manual labeling of craters is tedious and prone to errors. This paper presents a novel approach to detect and segment single craters using state-of-the-art computer vision techniques. The YOLOv8 model, a convolutional neural network-based object detection technique, is fine-tuned on a custom dataset of WEDM craters to locate and enclose them with tight bounding boxes. The segment anything model, a vision transformer-based instance segmentation technique, is applied to the cropped images of individual craters to delineate their shape and size. Geometric analysis of the segmented craters reveals significant variations in their contour and area depending on the energy setting, while the wire diameter has minimal influence.

RIOD:Reinforced Image-based Object Detection for Unruly Weather Conditions

Deep Neural Network (DNN) object detectors have proved their efficiency in the detection and classification of objects in normal weather. However, these models suffer a lot during bad weather conditions (foggy, rain, haze, night, etc.). This study presents a new scheme to reduce the aforementioned issue by attenuating the noise in the input image before feeding it to any kind of neural network-based object detector. In this study, the image optimization function transforms subpar-quality images due to bad weather into pictures with the optimal possible quality by estimating the proper illumination and transmission function. These optimized images showed improved object detection rates in the YOLOv4 and YOLOv5 models. This improvement in object detection was also noticed in the case of video input. This scheme was tested with images/videos from various weather conditions, and the results showed an encouraging improvement in detection rates.

Hypernuclear event detection in the nuclear emulsion with Monte Carlo simulation and machine learning

This study developed a novel method for detecting hypernuclear events recorded in nuclear emulsion sheets using machine learning techniques. The artificial neural network-based object detection model was trained on surrogate images created through Monte Carlo simulations and image-style transformations using generative adversarial networks. The performance of the proposed model was evaluated using α-decay events obtained from the J-PARC E07 emulsion data. The model achieved approximately twice the detection efficiency of conventional image processing and reduced the time spent on manual visual inspection by approximately 1/17. The established method was successfully applied to the detection of hypernuclear events. This approach is a state-of-the-art tool for discovering rare events recorded in nuclear emulsion sheets without any real data for training.

Feature disentanglement in one-stage object detection

In this paper, an enhanced disentanglement module is proposed to address feature misalignment caused by inherently irreconcilable conflicts between classification and regression tasks in Convolutional Neural Network-based object detectors. The proposed method disentangles features in the feature pyramid network (FPN) at the neck of the architecture. In addition, a response alignment strategy is proposed to reduce inconsistent responses and suppress inferior predictions. Extensive experiments are performed on the MS COCO and PASCAL VOC datasets with different backbones, confirming that the proposed method improves performance significantly. The proposed method exhibits two main advantages over existing solutions—features are disentangled at the neck instead of at the head, enabling comprehensive resolution of feature misalignment, and independent outputs of the two tasks after feature disentanglement are avoided, thereby preventing response inconsistencies.

Towards deep learning based smart farming for intelligent weeds management in crops.

Deep learning (DL) is a core constituent for building an object detection system and provides a variety of algorithms to be used in a variety of applications. In agriculture, weed management is one of the major concerns, weed detection systems could be of great help to improve production. In this work, we have proposed a DL-based weed detection model that can efficiently be used for effective weed management in crops. Our proposed model uses Convolutional Neural Network based object detection system You Only Look Once (YOLO) for training and prediction. The collected dataset contains RGB images of four different weed species named Grass, Creeping Thistle, Bindweed, and California poppy. This dataset is manipulated by applying LAB (Lightness A and B) and HSV (Hue, Saturation, Value) image transformation techniques and then trained on four YOLO models (v3, v3-tiny, v4, v4-tiny). The effects of image transformation are analyzed, and it is deduced that the model performance is not much affected by this transformation. Inferencing results obtained by making a comparison of correctly predicted weeds are quite promising, among all models implemented in this work, the YOLOv4 model has achieved the highest accuracy. It has correctly predicted 98.88% weeds with an average loss of 1.8 and 73.1% mean average precision value. In the future, we plan to integrate this model in a variable rate sprayer for precise weed management in real time.

3D positioning and autofocus of the particle field based on the depth-from-defocus method and the deep networks

Accurate three-dimensional positioning of particles is a critical task in microscopic particle research, with one of the main challenges being the measurement of particle depths. In this paper, we propose a method for detecting particle depths from their blurred images using the depth-from-defocus technique and a deep neural network-based object detection framework called you-only-look-once. Our method provides simultaneous lateral position information for the particles and has been tested and evaluated on various samples, including synthetic particles, polystyrene particles, blood cells, and plankton, even in a noise-filled environment. We achieved autofocus for target particles in different depths using generative adversarial networks, obtaining clear-focused images. Our algorithm can process a single multi-target image in 0.008 s, allowing real-time application. Our proposed method provides new opportunities for particle field research.

A new YOLO-based method for social distancing from real-time videos.

The coronavirus disease (COVID-19) is primarily disseminated through physical contact. As a precaution, it is recommended that indoor spaces have a limited number of people and at least one meter apart. This study proposes a real-time method for monitoring physical distancing compliance in indoor spaces using computer vision and deep learning techniques. The proposed method utilizes YOLO (You Only Look Once), a popular convolutional neural network-based object detection model, pre-trained on the Microsoft COCO (Common Objects in Context) dataset to detect persons and estimate their physical distance in real time. The effectiveness of the proposed method was assessed using metrics including accuracy rate, frame per second (FPS), and mean average precision (mAP). The results show that the YOLO v3 model had the most remarkable accuracy (87.07%) and mAP (89.91%). On the other hand, the highest fps rate of up to 18.71 was achieved by the YOLO v5s model. The results demonstrate the potential of the proposed method for effectively monitoring physical distancing compliance in indoor spaces, providing valuable insights for future use in other public health scenarios.

Animal Movement Prediction based on the Corroboration of Object Detection and Pose Estimation

With the development of automated driving, driving safety has become a hot topic in the field of automated driving. However, the existing methods mainly define living objects as obstacle handling and mostly focus on the safety of pedestrians, thus ignoring the safety of animal movement in automatic driving. In this paper, we propose an animal movement prediction model with the corroboration of object detection and pose estimation, which aims, to help the autopilot perception system judge animal pose more accurately. First of all, we preprocess the datasets, use a Convolutional Neural Network based object detection method to detect animal images, and then use the current mainstream 2D animal pose estimation method to obtain the key points of animal images, and divide the resulting data set into three categories: running, walking and waiting. After preprocessing, we use Resnet and MLP models to classify the pose. For Resnet model, we will evaluate the accuracy of the model on the test set during training and compare the accuracy of the last model to determine the model with the highest training accuracy. The final model has achieved an accuracy of up to 75% for running evaluation. For MLP model, we tend to use the rank of 13 specific bone points’ angles as the standard of pose classification. Three types of test sets will be put into three trained MLPRegressor model for training to get the classification we want. Finally, the accuracy of all three poses is more than 80%. Finally, we choose the model with the highest accuracy as our classification method. This work is expected to apply animal pose estimation and animal and pose classification to automated driving to improve driving safety.

Differential evolution based dual adversarial camouflage: Fooling human eyes and object detectors

Deep neural network-based object detectors are vulnerable to adversarial examples. Among existing works to fool object detectors, the camouflage-based method is more often adopted due to its adaptation to multi-view scenarios and non-planar objects. However, most of them can still be easily observed by human eyes, which limits their application in the real world. To fool human eyes and object detectors simultaneously, we propose a differential evolution based dual adversarial camouflage method. Specifically, we try to obtain the camouflage texture by the two-stage training, which can be wrapped over the surface of the object. In the first stage, we optimize the global texture to minimize the discrepancy between the rendered object and the scene background, making human eyes difficult to distinguish. In the second stage, we design three loss functions to optimize the local texture, which is selected from the global texture, making object detectors ineffective. In addition, we introduce the differential evolution algorithm to search for the near-optimal areas of the object to attack, improving the adversarial performance under certain attack area limitations. Experimental results show that our proposed method can obtain a good trade-off between fooling human eyes and object detectors under multiple specific scenes and objects.

A Framework to Estimate the Key Point Within an Object Based on a Deep Learning Object Detection

Automatic identification of key points within objects is crucial in various application domains. This paper presents a novel framework for accurately estimating the key point within an object by leveraging deep neural network-based object detection. The proposed framework is built upon a training dataset annotated with four non-overlapping bounding boxes, one of which shares a coordinate with the key point. These bounding boxes collectively cover the entire object, enabling automatic annotation if region annotations around the key point exist. The trained object detector is then utilized to generate detection results, which are subsequently post-processed to estimate the key point. To validate the effectiveness of the framework, experiments were conducted using two distinct datasets: cross-sectional images of a parawood log and pupil images. The experimental results demonstrate that our proposed framework surpasses previously proposed approaches in terms of precision, recall, F1-score, and other domain-specific metrics. The improvement in performance can be attributed to the unique annotation strategy and the fusion of object detection and key point estimation within a unified deep learning framework. The contribution of this study lies in introducing a novel framework for closely estimating key points within objects based on deep neural network-based object detection. By leveraging annotated training data and post-processing techniques, our approach achieves superior performance compared to existing methods. This work fills a critical gap in the field by integrating object detection and key point estimation, which has received limited attention in previous research. Our framework provides valuable insights and advancements in key point estimation techniques, offering potential applications in precise object analysis and understanding. Doi: 10.28991/HIJ-2023-04-01-08 Full Text: PDF

Towards automatic verification of the critical view of the myopectineal orifice with artificial intelligence.

Visualization of key anatomical landmarks is required during surgical Trans Abdominal Pre Peritoneal repair (TAPP)of inguinal hernia. The Critical View of the MyoPectineal Orifice (CVMPO) was proposed to ensure correct dissection. An artificial intelligence (AI) system that automatically validates the presence of key and marks during the procedure is a critical step towards automatic dissection quality assessment and video-based competency evaluation. The aim of this study was to develop an AI system that automatically recognizes the TAPP key CVMPO landmarks in hernia repair videos. Surgical videos of 160TAPP procedures were used in this single-center study. A deep neural network-based object detector was developed to automatically recognize the pubic symphysis, direct hernia orifice, Cooper's ligament, the iliac vein, triangle of Doom, deep inguinal ring, and iliopsoas muscle. The system was trained using 130 videos, annotated and verified by two board-certified surgeons. Performance was evaluated in 30 videos of new patients excluded from the training data. Performance was validated in 2 ways: first, single-image validation where the AI model detected landmarks in a single laparoscopic image (mean average precision (MAP) of 51.2%). The second validation is video evaluation where the model detected landmarks throughout the myopectineal orifice visual inspection phase (mean accuracy and F-score of 77.1 and 75.4% respectively). Annotation objectivity was assessed between 2 surgeons in video evaluation, showing a high agreement of 88.3%. This study establishes the first AI-based automated recognition of critical structures in TAPP surgical videos, and a major step towards automatic CVMPO validation with AI. Strong performance was achieved in the video evaluation. The high inter-rater agreement confirms annotation quality and task objectivity.

Camouflaged Adversarial Patch Attack on Object Detector

Adversarial attacks have received great attentions for their capacity to distract state-of-the-art neural networks by modifying objects in physical domain. Patch-based attack especially have got much attention for its optimization effectiveness and feasible adaptation to any objects to attack neural network-based object detectors. However, despite their strong attack performance, generated patches are strongly perceptible for humans, violating the fundamental assumption of adversarial examples. In this paper, we propose a camouflaged adversarial patch optimization method using military camouflage assessment metrics for naturalistic patch attacks. We also investigate camouflaged attack loss functions, applications of various camouflaged patches on army tank images, and validate the proposed approach with extensive experiments attacking Yolov5 detection model. Our methods produce more natural and realistic looking camouflaged patches while achieving competitive performance.

A Spatial Cross-Scale Attention Network and Global Average Accuracy Loss for SAR Ship Detection

A neural network-based object detection algorithm has the advantages of high accuracy and end-to-end processing, and it has been widely used in synthetic aperture radar (SAR) ship detection. However, the multi-scale variation of ship targets, the complex background of near-shore scenes, and the dense arrangement of some ships make it difficult to improve detection accuracy. To solve the above problem, in this paper, a spatial cross-scale attention network (SCSA-Net) for SAR image ship detection is proposed, which includes a novel spatial cross-scale attention (SCSA) module for eliminating the interference of land background. The SCSA module uses the features at each scale output from the backbone to calculate where the network needs attention in space and enhances the features of the feature pyramid network (FPN) output to eliminate interference from noise, and land complex backgrounds. In addition, this paper analyzes the reasons for the “score shift” problem caused by average precision loss (AP loss) and proposes the global average precision loss (GAP loss) to solve the “score shift” problem. GAP loss enables the network to distinguish positive samples and negative samples faster than focal loss and AP loss, and achieve higher accuracy. Finally, we validate and illustrate the effectiveness of the proposed method by performing it on SAR Ship Detection Dataset (SSDD), SAR-ship-dataset, and High-Resolution SAR Images Dataset (HRSID). The experimental results show that the proposed method can significantly reduce the interference of background noise on the ship detection results, improve the detection accuracy, and achieve superior results to the existing methods.

Intelligent Deep Convolutional Neural Network Based Object Detection Model for Visually Challenged People

Artificial Intelligence (AI) and Computer Vision (CV) advancements have led to many useful methodologies in recent years, particularly to help visually-challenged people. Object detection includes a variety of challenges, for example, handling multiple class images, images that get augmented when captured by a camera and so on. The test images include all these variants as well. These detection models alert them about their surroundings when they want to walk independently. This study compares four CNN-based pre-trained models: Residual Network (ResNet-50), Inception v3, Dense Convolutional Network (DenseNet-121), and SqueezeNet, predominantly used in image recognition applications. Based on the analysis performed on these test images, the study infers that Inception V3 outperformed other pre-trained models in terms of accuracy and speed. To further improve the performance of the Inception v3 model, the thermal exchange optimization (TEO) algorithm is applied to tune the hyperparameters (number of epochs, batch size, and learning rate) showing the novelty of the work. Better accuracy was achieved owing to the inclusion of an auxiliary classifier as a regularizer, hyperparameter optimizer, and factorization approach. Additionally, Inception V3 can handle images of different sizes. This makes Inception V3 the optimum model for assisting visually challenged people in real-world communication when integrated with Internet of Things (IoT)-based devices.

Convolutional neural network based object detection system for video surveillance application

SummaryVideo surveillance is emerging as a promising solution for the humans to lead a peaceful and independent life in their homes. The recognition and localization of moving objects plays a central role in the video surveillance. The manual surveillance is time consuming and tedious. Therefore, novel object detection via optimized deep learning model is developed in this work that supports the video surveillance application. In the initial phase, proposed angle and distance based Local Binary Pattern (LBP) features are extracted. Subsequently, these extracted features are subjected to object detection phase, where optimized Convolutional Neural Network (CNN) will expose the information about the detected object. Further, the learning quality of CNN is decided by the weight parameter, which is responsible to distinguish the objects with high accuracy. Therefore, a hybrid optimization concept referred as Sealion Leader Update with Particles (SLUP) is introduced in this research work to fine‐tune the weight of CNN. Finally, a comparative analysis is made between the proposed and the extant approaches in terms of “positive, negative, and other measures.”

FEA-Swin: Foreground Enhancement Attention Swin Transformer Network for Accurate UAV-Based Dense Object Detection.

UAV-based object detection has recently attracted a lot of attention due to its diverse applications. Most of the existing convolution neural network based object detection models can perform well in common object detection cases. However, due to the fact that objects in UAV images are spatially distributed in a very dense manner, these methods have limited performance for UAV-based object detection. In this paper, we propose a novel transformer-based object detection model to improve the accuracy of object detection in UAV images. To detect dense objects competently, an advanced foreground enhancement attention Swin Transformer (FEA-Swin) framework is designed by integrating context information into the original backbone of a Swin Transformer. Moreover, to avoid the loss of information of small objects, an improved weighted bidirectional feature pyramid network (BiFPN) is presented by designing the skip connection operation. The proposed method aggregates feature maps from four stages and keeps abundant information of small objects. Specifically, to balance the detection accuracy and efficiency, we introduce an efficient neck of the BiFPN network by removing a redundant network layer. Experimental results on both public datasets and a self-made dataset demonstrate the performance of our method compared to the state-of-the-art methods in terms of detection accuracy.

A Deep Learning Approach for LiDAR Resolution-Agnostic Object Detection

Existing neural network-based object detection approaches process LiDAR point clouds trained from one kind of LiDAR sensor. In the case of a different point cloud input, the trained network performs with less efficiency, especially when the given point cloud has low resolution. In this paper, we propose a new object detection approach, which is more resilient to variations in point cloud resolution. Firstly, layers from the point cloud are randomly discarded during the training phase in order to increase the variability of the data processed by the network. Secondly, the obstacles are described as Gaussian functions, grouping multiple parameters into a single representation. A Bhattacharyya distance is used as a loss function. This approach is tested on a LiDAR-based network and on an architecture using camera and LiDAR sensors. The networks are trained exclusively on the KITTI dataset and tested on Pandaset and the nuScenes Mini dataset. Experiments show that our method improves the performance of the tested networks on low-resolution point clouds without decreasing the ability to process high-resolution data.

Misleading attention and classification: An adversarial attack to fool object detection models in the real world

Object detection is a hot topic in computer vision (CV), and it has many applications in various security fields. However, many works have demonstrated that neural network-based object detection is vulnerable to adversarial attacks. In this paper, we study adversarial attacks on object detectors in the real world and propose a new adversarial attack called Misleading Attention and Classification Attack (MACA), which can generate adversarial patches to mislead the object detectors. Specifically, we propose a new scheme to generate adversarial patches to fool the object detector. Our scheme restricts the noise of the adversarial patches and aims to ensure that the generated adversarial patches are visually similar to natural images. The attack simulates the complex external physical environment and the 3D transformations of non-rigid objects to increase the robustness of adversarial patches. We attack the up-to-date object detectors (e.g., Yolo-V5), and we prove that our technique has strong transferability among different detectors. Extensive experiments show that it is feasible to transfer the digital adversarial patches to the real world while maintaining the transferability of adversarial patches among different models and the success rate of adversarial attacks.

At-scale evaluation of weight clustering to enable energy-efficient object detection

Accelerators implementing Deep Neural Networks (DNNs) for image-based object detection operate on large volumes of data due to fetching images and neural network parameters, especially if they need to process video streams, hence with high power dissipation and bandwidth requirements to fetch all those data. While some solutions exist to mitigate power and bandwidth demands for data fetching, they are often assessed in the context of limited evaluations with a scale much smaller than that of the target application, which challenges finding the best tradeoff in practice.This paper sets up the infrastructure to assess at-scale a key power and bandwidth optimization – weight clustering – for You Only Look Once v3 (YOLOv3), a neural network-based object detection system, using videos of real driving conditions. Our assessment shows that accelerators such as systolic arrays with an Output Stationary architecture turn out to be a highly effective solution combined with weight clustering. In particular, applying weight clustering independently per neural network layer, and using between 32 (5-bit) and 256 (8-bit) weights allows achieving an accuracy close to that of the original YOLOv3 weights (32-bit weights). Such bit-count reduction of the weights allows shaving bandwidth requirements down to 30%–40% of the original requirements, and reduces energy consumption down to 45%. This is based on the fact that (i) energy due to multiply-and-accumulate operations is much smaller than DRAM data fetching, and (ii) designing accelerators appropriately may make that most of the data fetched corresponds to neural network weights, where clustering can be applied. Overall, our at-scale assessment provides key results to architect camera-based object detection accelerators by putting together a real-life application (YOLOv3), and real driving videos, in a unified setup so that trends observed are reliable.