Neural Network Based Object Detection Research Articles

Модель та метод навчання детектора об’єктів на аерозображенні з оптимізацією робастності та кількості обчислень

The subject of research is Neural network-based object detectors, which are widely used for video image analysis. An increasing number of tasks now demand data processing directly at the source, which limits the available computational resources. However, the vulnerability of neural networks to noise, adversarial attacks, and weight error injections significantly diminishes their robustness and overall effectiveness. The relevant task is to develop models that provide both computational efficiency and robustness against perturbations. This paper investigates a model and method for enhancing the robustness of neural network detectors under limited resources. The objective is to design a model that allocates resources optimally while maintaining stability. To achieve this, the study employs techniques such as dynamic neural networks, robustness optimization, and resilience strategies. The following results were obtained. A detector with a feature extractor based on ViT-S/16, modified with gate modules for dynamic examination was developed. The model was trained on the RSOD dataset and meta-learned on the adaptation results to various perturbations. The model's resistance to random bit inversions in weights (10 % of weights) and to adversarial attacks with perturbation amplitudes up to 3/255 (L∞ norm) was tested. Conclusion. The proposed detector model incorporating dynamic examination and optimized robustness, reduced floating-point operations by over 20 % without loss of accuracy. A novel method for training the detector was developed, combining the RetinaNet loss function with the loss function of gate blocks and applying meta-learning on the adaptation results for various types of synthetic perturbations. Testing demonstrated an increase in accuracy by 11.9 % under the influence of error injection and by 13.2 % under the influence of adversarial attacks.

Deep learning-based ultrasonographic classification of canine chronic kidney disease.

In veterinary medicine, attempts to apply artificial intelligence (AI) to ultrasonography have rarely been reported, and few studies have investigated the value of AI in ultrasonographic diagnosis. This study aimed to develop a deep learning-based model for classifying the status of canine chronic kidney disease (CKD) using renal ultrasonographic images and assess its diagnostic performance in comparison with that of veterinary imaging specialists, thereby verifying its clinical utility. In this study, 883 ultrasonograms were obtained from 198 dogs, including those diagnosed with CKD according to the International Renal Interest Society (IRIS) guidelines and healthy dogs. After preprocessing and labeling each image with its corresponding IRIS stage, the renal regions were extracted and classified based on the IRIS stage using the convolutional neural network-based object detection algorithm You Only Look Once. The training scenarios consisted of multi-class classification, categorization of images into IRIS stages, and four binary classifications based on specific IRIS stages. To prevent model overfitting, we balanced the dataset, implemented early stopping, used lightweight models, and applied dropout techniques. Model performance was assessed using accuracy, recall, precision, F1 score, and receiver operating characteristic curve and compared with the diagnostic accuracy of four specialists. Inter- and intra-observer variabilities among specialists were also evaluated. The developed model exhibited a low accuracy of 0.46 in multi-class classification. However, a significant performance improvement was observed in binary classifications, with the model designed to distinguish stage 3 or higher showing the highest accuracy of 0.85. In this classification, recall, precision, and F1 score values were all 0.85, and the area under the curve was 0.89. Compared with radiologists, whose accuracy ranged from 0.48 to 0.62 in this experimental scenario, the AI model exhibited superiority. Intra-observer reliability among radiologists was substantial, whereas inter-observer variability showed a moderate level of agreement. This study developed a deep-learning framework capable of reliably classifying CKD IRIS stages 3 and 4 in dogs using ultrasonograms. The developed framework demonstrated higher accuracy than veterinary imaging specialists and provided more objective and consistent interpretations. Therefore, deep-learning-based ultrasound diagnostics are potentially valuable tools for diagnosing CKD in dogs.

Ontology based autonomous robot task processing framework.

In recent years, the perceptual capabilities of robots have been significantly enhanced. However, the task execution of the robots still lacks adaptive capabilities in unstructured and dynamic environments. In this paper, we propose an ontology based autonomous robot task processing framework (ARTProF), to improve the robot's adaptability within unstructured and dynamic environments. ARTProF unifies ontological knowledge representation, reasoning, and autonomous task planning and execution into a single framework. The interface between the knowledge base and neural network-based object detection is first introduced in ARTProF to improve the robot's perception capabilities. A knowledge-driven manipulation operator based on Robot Operating System (ROS) is then designed to facilitate the interaction between the knowledge base and the robot's primitive actions. Additionally, an operation similarity model is proposed to endow the robot with the ability to generalize to novel objects. Finally, a dynamic task planning algorithm, leveraging ontological knowledge, equips the robot with adaptability to execute tasks in unstructured and dynamic environments. Experimental results on real-world scenarios and simulations demonstrate the effectiveness and efficiency of the proposed ARTProF framework. In future work, we will focus on refining the ARTProF framework by integrating neurosymbolic inference.

Automatic Characterization of WEDM Single Craters Through AI Based Object Detection

Wire electrical discharge machining (WEDM) is a process that removes material from conductive workpieces by using sequential electrical discharges. The morphology of the craters formed by these discharges is influenced by various process parameters and affects the quality and efficiency of the machining. To understand and optimize the WEDM process, it is essential to identify and characterize single craters from microscopy images. However, manual labeling of craters is tedious and prone to errors. This paper presents a novel approach to detect and segment single craters using state-of-the-art computer vision techniques. The YOLOv8 model, a convolutional neural network-based object detection technique, is fine-tuned on a custom dataset of WEDM craters to locate and enclose them with tight bounding boxes. The segment anything model, a vision transformer-based instance segmentation technique, is applied to the cropped images of individual craters to delineate their shape and size. Geometric analysis of the segmented craters reveals significant variations in their contour and area depending on the energy setting, while the wire diameter has minimal influence.

RIOD:Reinforced Image-based Object Detection for Unruly Weather Conditions

Deep Neural Network (DNN) object detectors have proved their efficiency in the detection and classification of objects in normal weather. However, these models suffer a lot during bad weather conditions (foggy, rain, haze, night, etc.). This study presents a new scheme to reduce the aforementioned issue by attenuating the noise in the input image before feeding it to any kind of neural network-based object detector. In this study, the image optimization function transforms subpar-quality images due to bad weather into pictures with the optimal possible quality by estimating the proper illumination and transmission function. These optimized images showed improved object detection rates in the YOLOv4 and YOLOv5 models. This improvement in object detection was also noticed in the case of video input. This scheme was tested with images/videos from various weather conditions, and the results showed an encouraging improvement in detection rates.

Hypernuclear event detection in the nuclear emulsion with Monte Carlo simulation and machine learning

This study developed a novel method for detecting hypernuclear events recorded in nuclear emulsion sheets using machine learning techniques. The artificial neural network-based object detection model was trained on surrogate images created through Monte Carlo simulations and image-style transformations using generative adversarial networks. The performance of the proposed model was evaluated using α-decay events obtained from the J-PARC E07 emulsion data. The model achieved approximately twice the detection efficiency of conventional image processing and reduced the time spent on manual visual inspection by approximately 1/17. The established method was successfully applied to the detection of hypernuclear events. This approach is a state-of-the-art tool for discovering rare events recorded in nuclear emulsion sheets without any real data for training.

Feature disentanglement in one-stage object detection

In this paper, an enhanced disentanglement module is proposed to address feature misalignment caused by inherently irreconcilable conflicts between classification and regression tasks in Convolutional Neural Network-based object detectors. The proposed method disentangles features in the feature pyramid network (FPN) at the neck of the architecture. In addition, a response alignment strategy is proposed to reduce inconsistent responses and suppress inferior predictions. Extensive experiments are performed on the MS COCO and PASCAL VOC datasets with different backbones, confirming that the proposed method improves performance significantly. The proposed method exhibits two main advantages over existing solutions—features are disentangled at the neck instead of at the head, enabling comprehensive resolution of feature misalignment, and independent outputs of the two tasks after feature disentanglement are avoided, thereby preventing response inconsistencies.

Towards deep learning based smart farming for intelligent weeds management in crops.

Deep learning (DL) is a core constituent for building an object detection system and provides a variety of algorithms to be used in a variety of applications. In agriculture, weed management is one of the major concerns, weed detection systems could be of great help to improve production. In this work, we have proposed a DL-based weed detection model that can efficiently be used for effective weed management in crops. Our proposed model uses Convolutional Neural Network based object detection system You Only Look Once (YOLO) for training and prediction. The collected dataset contains RGB images of four different weed species named Grass, Creeping Thistle, Bindweed, and California poppy. This dataset is manipulated by applying LAB (Lightness A and B) and HSV (Hue, Saturation, Value) image transformation techniques and then trained on four YOLO models (v3, v3-tiny, v4, v4-tiny). The effects of image transformation are analyzed, and it is deduced that the model performance is not much affected by this transformation. Inferencing results obtained by making a comparison of correctly predicted weeds are quite promising, among all models implemented in this work, the YOLOv4 model has achieved the highest accuracy. It has correctly predicted 98.88% weeds with an average loss of 1.8 and 73.1% mean average precision value. In the future, we plan to integrate this model in a variable rate sprayer for precise weed management in real time.

3D positioning and autofocus of the particle field based on the depth-from-defocus method and the deep networks

Accurate three-dimensional positioning of particles is a critical task in microscopic particle research, with one of the main challenges being the measurement of particle depths. In this paper, we propose a method for detecting particle depths from their blurred images using the depth-from-defocus technique and a deep neural network-based object detection framework called you-only-look-once. Our method provides simultaneous lateral position information for the particles and has been tested and evaluated on various samples, including synthetic particles, polystyrene particles, blood cells, and plankton, even in a noise-filled environment. We achieved autofocus for target particles in different depths using generative adversarial networks, obtaining clear-focused images. Our algorithm can process a single multi-target image in 0.008 s, allowing real-time application. Our proposed method provides new opportunities for particle field research.

A new YOLO-based method for social distancing from real-time videos.

The coronavirus disease (COVID-19) is primarily disseminated through physical contact. As a precaution, it is recommended that indoor spaces have a limited number of people and at least one meter apart. This study proposes a real-time method for monitoring physical distancing compliance in indoor spaces using computer vision and deep learning techniques. The proposed method utilizes YOLO (You Only Look Once), a popular convolutional neural network-based object detection model, pre-trained on the Microsoft COCO (Common Objects in Context) dataset to detect persons and estimate their physical distance in real time. The effectiveness of the proposed method was assessed using metrics including accuracy rate, frame per second (FPS), and mean average precision (mAP). The results show that the YOLO v3 model had the most remarkable accuracy (87.07%) and mAP (89.91%). On the other hand, the highest fps rate of up to 18.71 was achieved by the YOLO v5s model. The results demonstrate the potential of the proposed method for effectively monitoring physical distancing compliance in indoor spaces, providing valuable insights for future use in other public health scenarios.

Animal Movement Prediction based on the Corroboration of Object Detection and Pose Estimation

With the development of automated driving, driving safety has become a hot topic in the field of automated driving. However, the existing methods mainly define living objects as obstacle handling and mostly focus on the safety of pedestrians, thus ignoring the safety of animal movement in automatic driving. In this paper, we propose an animal movement prediction model with the corroboration of object detection and pose estimation, which aims, to help the autopilot perception system judge animal pose more accurately. First of all, we preprocess the datasets, use a Convolutional Neural Network based object detection method to detect animal images, and then use the current mainstream 2D animal pose estimation method to obtain the key points of animal images, and divide the resulting data set into three categories: running, walking and waiting. After preprocessing, we use Resnet and MLP models to classify the pose. For Resnet model, we will evaluate the accuracy of the model on the test set during training and compare the accuracy of the last model to determine the model with the highest training accuracy. The final model has achieved an accuracy of up to 75% for running evaluation. For MLP model, we tend to use the rank of 13 specific bone points’ angles as the standard of pose classification. Three types of test sets will be put into three trained MLPRegressor model for training to get the classification we want. Finally, the accuracy of all three poses is more than 80%. Finally, we choose the model with the highest accuracy as our classification method. This work is expected to apply animal pose estimation and animal and pose classification to automated driving to improve driving safety.

Differential evolution based dual adversarial camouflage: Fooling human eyes and object detectors

Deep neural network-based object detectors are vulnerable to adversarial examples. Among existing works to fool object detectors, the camouflage-based method is more often adopted due to its adaptation to multi-view scenarios and non-planar objects. However, most of them can still be easily observed by human eyes, which limits their application in the real world. To fool human eyes and object detectors simultaneously, we propose a differential evolution based dual adversarial camouflage method. Specifically, we try to obtain the camouflage texture by the two-stage training, which can be wrapped over the surface of the object. In the first stage, we optimize the global texture to minimize the discrepancy between the rendered object and the scene background, making human eyes difficult to distinguish. In the second stage, we design three loss functions to optimize the local texture, which is selected from the global texture, making object detectors ineffective. In addition, we introduce the differential evolution algorithm to search for the near-optimal areas of the object to attack, improving the adversarial performance under certain attack area limitations. Experimental results show that our proposed method can obtain a good trade-off between fooling human eyes and object detectors under multiple specific scenes and objects.

A Framework to Estimate the Key Point Within an Object Based on a Deep Learning Object Detection

Automatic identification of key points within objects is crucial in various application domains. This paper presents a novel framework for accurately estimating the key point within an object by leveraging deep neural network-based object detection. The proposed framework is built upon a training dataset annotated with four non-overlapping bounding boxes, one of which shares a coordinate with the key point. These bounding boxes collectively cover the entire object, enabling automatic annotation if region annotations around the key point exist. The trained object detector is then utilized to generate detection results, which are subsequently post-processed to estimate the key point. To validate the effectiveness of the framework, experiments were conducted using two distinct datasets: cross-sectional images of a parawood log and pupil images. The experimental results demonstrate that our proposed framework surpasses previously proposed approaches in terms of precision, recall, F1-score, and other domain-specific metrics. The improvement in performance can be attributed to the unique annotation strategy and the fusion of object detection and key point estimation within a unified deep learning framework. The contribution of this study lies in introducing a novel framework for closely estimating key points within objects based on deep neural network-based object detection. By leveraging annotated training data and post-processing techniques, our approach achieves superior performance compared to existing methods. This work fills a critical gap in the field by integrating object detection and key point estimation, which has received limited attention in previous research. Our framework provides valuable insights and advancements in key point estimation techniques, offering potential applications in precise object analysis and understanding. Doi: 10.28991/HIJ-2023-04-01-08 Full Text: PDF

Towards automatic verification of the critical view of the myopectineal orifice with artificial intelligence.

Visualization of key anatomical landmarks is required during surgical Trans Abdominal Pre Peritoneal repair (TAPP)of inguinal hernia. The Critical View of the MyoPectineal Orifice (CVMPO) was proposed to ensure correct dissection. An artificial intelligence (AI) system that automatically validates the presence of key and marks during the procedure is a critical step towards automatic dissection quality assessment and video-based competency evaluation. The aim of this study was to develop an AI system that automatically recognizes the TAPP key CVMPO landmarks in hernia repair videos. Surgical videos of 160TAPP procedures were used in this single-center study. A deep neural network-based object detector was developed to automatically recognize the pubic symphysis, direct hernia orifice, Cooper's ligament, the iliac vein, triangle of Doom, deep inguinal ring, and iliopsoas muscle. The system was trained using 130 videos, annotated and verified by two board-certified surgeons. Performance was evaluated in 30 videos of new patients excluded from the training data. Performance was validated in 2 ways: first, single-image validation where the AI model detected landmarks in a single laparoscopic image (mean average precision (MAP) of 51.2%). The second validation is video evaluation where the model detected landmarks throughout the myopectineal orifice visual inspection phase (mean accuracy and F-score of 77.1 and 75.4% respectively). Annotation objectivity was assessed between 2 surgeons in video evaluation, showing a high agreement of 88.3%. This study establishes the first AI-based automated recognition of critical structures in TAPP surgical videos, and a major step towards automatic CVMPO validation with AI. Strong performance was achieved in the video evaluation. The high inter-rater agreement confirms annotation quality and task objectivity.

Camouflaged Adversarial Patch Attack on Object Detector

Adversarial attacks have received great attentions for their capacity to distract state-of-the-art neural networks by modifying objects in physical domain. Patch-based attack especially have got much attention for its optimization effectiveness and feasible adaptation to any objects to attack neural network-based object detectors. However, despite their strong attack performance, generated patches are strongly perceptible for humans, violating the fundamental assumption of adversarial examples. In this paper, we propose a camouflaged adversarial patch optimization method using military camouflage assessment metrics for naturalistic patch attacks. We also investigate camouflaged attack loss functions, applications of various camouflaged patches on army tank images, and validate the proposed approach with extensive experiments attacking Yolov5 detection model. Our methods produce more natural and realistic looking camouflaged patches while achieving competitive performance.

A Spatial Cross-Scale Attention Network and Global Average Accuracy Loss for SAR Ship Detection

A neural network-based object detection algorithm has the advantages of high accuracy and end-to-end processing, and it has been widely used in synthetic aperture radar (SAR) ship detection. However, the multi-scale variation of ship targets, the complex background of near-shore scenes, and the dense arrangement of some ships make it difficult to improve detection accuracy. To solve the above problem, in this paper, a spatial cross-scale attention network (SCSA-Net) for SAR image ship detection is proposed, which includes a novel spatial cross-scale attention (SCSA) module for eliminating the interference of land background. The SCSA module uses the features at each scale output from the backbone to calculate where the network needs attention in space and enhances the features of the feature pyramid network (FPN) output to eliminate interference from noise, and land complex backgrounds. In addition, this paper analyzes the reasons for the “score shift” problem caused by average precision loss (AP loss) and proposes the global average precision loss (GAP loss) to solve the “score shift” problem. GAP loss enables the network to distinguish positive samples and negative samples faster than focal loss and AP loss, and achieve higher accuracy. Finally, we validate and illustrate the effectiveness of the proposed method by performing it on SAR Ship Detection Dataset (SSDD), SAR-ship-dataset, and High-Resolution SAR Images Dataset (HRSID). The experimental results show that the proposed method can significantly reduce the interference of background noise on the ship detection results, improve the detection accuracy, and achieve superior results to the existing methods.

Intelligent Deep Convolutional Neural Network Based Object Detection Model for Visually Challenged People

Artificial Intelligence (AI) and Computer Vision (CV) advancements have led to many useful methodologies in recent years, particularly to help visually-challenged people. Object detection includes a variety of challenges, for example, handling multiple class images, images that get augmented when captured by a camera and so on. The test images include all these variants as well. These detection models alert them about their surroundings when they want to walk independently. This study compares four CNN-based pre-trained models: Residual Network (ResNet-50), Inception v3, Dense Convolutional Network (DenseNet-121), and SqueezeNet, predominantly used in image recognition applications. Based on the analysis performed on these test images, the study infers that Inception V3 outperformed other pre-trained models in terms of accuracy and speed. To further improve the performance of the Inception v3 model, the thermal exchange optimization (TEO) algorithm is applied to tune the hyperparameters (number of epochs, batch size, and learning rate) showing the novelty of the work. Better accuracy was achieved owing to the inclusion of an auxiliary classifier as a regularizer, hyperparameter optimizer, and factorization approach. Additionally, Inception V3 can handle images of different sizes. This makes Inception V3 the optimum model for assisting visually challenged people in real-world communication when integrated with Internet of Things (IoT)-based devices.

Convolutional neural network based object detection system for video surveillance application

SummaryVideo surveillance is emerging as a promising solution for the humans to lead a peaceful and independent life in their homes. The recognition and localization of moving objects plays a central role in the video surveillance. The manual surveillance is time consuming and tedious. Therefore, novel object detection via optimized deep learning model is developed in this work that supports the video surveillance application. In the initial phase, proposed angle and distance based Local Binary Pattern (LBP) features are extracted. Subsequently, these extracted features are subjected to object detection phase, where optimized Convolutional Neural Network (CNN) will expose the information about the detected object. Further, the learning quality of CNN is decided by the weight parameter, which is responsible to distinguish the objects with high accuracy. Therefore, a hybrid optimization concept referred as Sealion Leader Update with Particles (SLUP) is introduced in this research work to fine‐tune the weight of CNN. Finally, a comparative analysis is made between the proposed and the extant approaches in terms of “positive, negative, and other measures.”

FEA-Swin: Foreground Enhancement Attention Swin Transformer Network for Accurate UAV-Based Dense Object Detection.

UAV-based object detection has recently attracted a lot of attention due to its diverse applications. Most of the existing convolution neural network based object detection models can perform well in common object detection cases. However, due to the fact that objects in UAV images are spatially distributed in a very dense manner, these methods have limited performance for UAV-based object detection. In this paper, we propose a novel transformer-based object detection model to improve the accuracy of object detection in UAV images. To detect dense objects competently, an advanced foreground enhancement attention Swin Transformer (FEA-Swin) framework is designed by integrating context information into the original backbone of a Swin Transformer. Moreover, to avoid the loss of information of small objects, an improved weighted bidirectional feature pyramid network (BiFPN) is presented by designing the skip connection operation. The proposed method aggregates feature maps from four stages and keeps abundant information of small objects. Specifically, to balance the detection accuracy and efficiency, we introduce an efficient neck of the BiFPN network by removing a redundant network layer. Experimental results on both public datasets and a self-made dataset demonstrate the performance of our method compared to the state-of-the-art methods in terms of detection accuracy.

A Deep Learning Approach for LiDAR Resolution-Agnostic Object Detection

Existing neural network-based object detection approaches process LiDAR point clouds trained from one kind of LiDAR sensor. In the case of a different point cloud input, the trained network performs with less efficiency, especially when the given point cloud has low resolution. In this paper, we propose a new object detection approach, which is more resilient to variations in point cloud resolution. Firstly, layers from the point cloud are randomly discarded during the training phase in order to increase the variability of the data processed by the network. Secondly, the obstacles are described as Gaussian functions, grouping multiple parameters into a single representation. A Bhattacharyya distance is used as a loss function. This approach is tested on a LiDAR-based network and on an architecture using camera and LiDAR sensors. The networks are trained exclusively on the KITTI dataset and tested on Pandaset and the nuScenes Mini dataset. Experiments show that our method improves the performance of the tested networks on low-resolution point clouds without decreasing the ability to process high-resolution data.