Image Object Detection Research Articles

Deep convolutional neural networks for ship detection using refined DOTA and TGRS-HRRSD high-resolution image datasets

Automated identification of ships in satellite imagery is of utmost significance in many military and civilian applications, including monitoring maritime traffic and maintaining maritime security. However, ship detection from remotely sensed imagery is a challenging task because of their varying sizes, orientations, types, and various interferences. In recent years, deep learning algorithms have become a powerful tool in image processing applications, including image classification and object detection due to their strong feature representation capabilities. In this study, a refined ship dataset, containing a total of 13,735 instances and representing different ship types, was generated using state-of-the-art datasets for the ship detection task. In a comparative framework, the performance of five deep learning algorithms (Faster R-CNN, YOLOv8, CenterNet, EfficientDet, and SSD) in ship detection was investigated using the created dataset. The performance of the deep learning models was compared using MS COCO accuracy metrics. In addition, the generalization capabilities of the models were tested on an independent image acquired by Göktürk-1 having a mucilage effect that undermines the spectral discrimination. Results revealed that the YOLOv8 model had the greatest ship detection performance (AP@0.50 = 96.33 %), followed by CenterNet (AP@0.50 = 88.80 %), Faster R-CNN (AP@0.50 = 85.30 %), EfficientDet (AP@0.50 = 78.91 %) and SSD (AP@0.5 = 75.98 %), and obtained the highest accuracies in terms of other COCO metrics. In addition, generalization capabilities on the test image confirmed the superiority of the YOLOv8 model with AP@0.50 score of 63.93 %. Overall, the results of this study validated the effectiveness of the YOLOv8 model in identifying ship targets in remotely sensed images.

Improved YOLOX Remote Sensing Image Object Detection Algorithm

Remote sensing image object detection is one of the core tasks of remote sensing image processing. In recent years, with the development of deep learning, great progress has been made in object detection in remote sensing. However, the problems of dense small targets, complex backgrounds and poor target positioning accuracy in remote sensing images make the detection of remote sensing targets still difficult. In order to solve these problems, this research proposes a remote sensing image object detection algorithm based on improved YOLOX-S. Firstly, the Efficient Channel Attention (ECA) module is introduced to improve the network's ability to extract features in the image and suppress useless information such as background; Secondly, the loss function is optimized to improve the regression accuracy of the target bounding box. We evaluate the effectiveness of our algorithm on the NWPU VHR-10 remote sensing image dataset, the experimental results show that the detection accuracy of the algorithm can reach 95.5%, without increasing the amount of parameters. It is significantly improved compared with that of the original YOLOX-S network, and the detection performance is much better than that of some other mainstream remote sensing image detection methods. Besides, our method also shows good generalization detection performance in experiments on aircraft images in the RSOD dataset.

Shape-Guided Detection: A joint network combining object detection and underwater image enhancement together

Most of the existing underwater image object detection methods involve pre-processing, such as using underwater image enhancement, to improve the accuracy of object detection. However, pre-processing methods are designed to improve the subjective perception of the human eye, which does not necessarily improve the object detection performance and consumes a large amount of computational resources. Therefore, in this paper, we creatively combine these two tasks and propose a Shape-Guided Detection network (SGD) to simultaneously optimize underwater image enhancement and object detection. In the SGD network, we innovatively incorporate the prior shape features as a learnable module embedded in it to fully explore the shape characteristics and structural details of the target object. To ensure that the prior knowledge can be effectively fused into the global network structure, we design a Shape Prior Enhancement module, which aims to realize the deep integration of the prior information with the local details. In order to optimize the stability of model training and enhance its convergence performance, a dual strategy of explicit and implicit constraints is ingeniously proposed in our method. We conduct extensive experiments on public datasets and the results show that the combination of our method with different detectors significantly improves the performance. The object detection performance reaches up to 0.491 mAP for optical images and 0.576 mAP for sonar images, and improves the preprocessing speed by 0.1 s.

Spatial attention for human-centric visual understanding: An Information Bottleneck method

The selective visual attention mechanism in the Human Visual System (HVS) restricts the amount of information that reaches human visual awareness, allowing the brain to perceive high-fidelity natural scenes in real-time with limited computational cost. This selectivity acts as an “Information Bottleneck (IB)” that balances information compression and predictive accuracy. However, such information constraints are rarely explored in the attention mechanism for deep neural networks (DNNs). This paper introduces an IB-inspired spatial attention module for DNNs, which generates an attention map by minimizing the mutual information (MI) between the attentive content and the input while maximizing that between the attentive content and the output. We develop this IB-inspired attention mechanism based on a novel graphical model and explore various implementations of the framework. We show that our approach can yield attention maps that neatly highlight the regions of interest while suppressing the backgrounds, and are interpretable for the decision-making of the DNNs. To validate the effectiveness of the proposed IB-inspired attention mechanism, we apply it to various computer vision tasks including image classification, fine-grained recognition, cross-domain classification, semantic segmentation, and object detection. Extensive experiments demonstrate that it bootstraps standard DNN structures quantitatively and qualitatively for these tasks.

Multimodal Social Media Fake News Detection Based on 1D-CCNet Attention Mechanism

Due to the explosive rise of multimodal content in online social communities, cross-modal learning is crucial for accurate fake news detection. However, current multimodal fake news detection techniques face challenges in extracting features from multiple modalities and fusing cross-modal information, failing to fully exploit the correlations and complementarities between different modalities. To address these issues, this paper proposes a fake news detection model based on a one-dimensional CCNet (1D-CCNet) attention mechanism, named BTCM. This method first utilizes BERT and BLIP-2 encoders to extract text and image features. Then, it employs the proposed 1D-CCNet attention mechanism module to process the input text and image sequences, enhancing the important aspects of the bimodal features. Meanwhile, this paper uses the pre-trained BLIP-2 model for object detection in images, generating image descriptions and augmenting text data to enhance the dataset. This operation aims to further strengthen the correlations between different modalities. Finally, this paper proposes a heterogeneous cross-feature fusion method (HCFFM) to integrate image and text features. Comparative experiments were conducted on three public datasets: Twitter, Weibo, and Gossipcop. The results show that the proposed model achieved excellent performance.

Estimating Power, Performance, and Area for On-Sensor Deployment of AR/VR Workloads Using an Analytical Framework

Augmented Reality and Virtual Reality have emerged as the next frontier of intelligent image sensors and computer systems. In these systems, 3D die stacking stands out as a compelling solution, enabling in situ processing capability of the sensory data for tasks such as image classification and object detection at low power, low latency, and a small form factor. These intelligent 3D CMOS Image Sensor (CIS) systems present a wide design space, encompassing multiple domains (e.g., computer vision algorithms, circuit design, system architecture, and semiconductor technology, including 3D stacking) that have not been explored in-depth so far. This article aims to fill this gap. We first present an analytical evaluation framework, STAR-3DSim, dedicated to rapid pre-RTL evaluation of 3D-CIS systems capturing the entire stack from the pixel layer to the on-sensor processor layer. With STAR-3DSim, we then propose several knobs for PPA (power, performance, area) improvement of the Deep Neural Network (DNN) accelerator that can provide up to 53%, 41%, and 63% reduction in energy, latency, and area, respectively, across a broad set of relevant AR/VR workloads. Last, we present full-system evaluation results by taking image sensing, cross-tier data transfer, and off-sensor communication into consideration.

A novel target detection method with dual‐domain multi‐frequency feature in side‐scan sonar images

AbstractSide‐scan sonar (SSS) detection is a key method in underwater environmental security and subsea resource development. However, many detection approaches primarily concentrate on tracking the evolution path of optical image object detection tasks when using acoustic images, resulting in complex structures and limited versatility. To tackle this issue, we introduce a pioneering dual‐domain multi‐frequency network (D2MFNet) meticulously crafted to harness the distinct characteristics of SSS image detection. In D2MFNet, a novel method for optimizing and improving the detection sensitivity in different frequency ranges called multi‐frequency combined attention mechanism (MFCAM) is proposed. This mechanism amplifies the relevance of dual‐domain features across different channels and spaces. Moreover, we introduce a dual‐domain feature pyramid network (D2FPN) significantly augments the depth and breadth of feature information in underwater small datasets. The methods offer plug‐and‐play functionality with substantial performance enhancements. Extensive experiments are conducted to validate the efficacy of the proposed techniques, and the results showcase their state‐of‐the‐art performance. MFCAM improves the mAP by 16.9% in the KLSG dataset and 15.5% in the SCTD dataset. The mAP of D2FPN was improved by 8.4% in the KLSG dataset and by 9.8% in the SCTD dataset. The code and models will be publicly available at https://dagshub.com/estrellaww00/D2MFNet.

A novel multiscale cGAN approach for enhanced salient object detection in single haze images

In computer vision, image dehazing is a low-level task that employs algorithms to analyze and remove haze from images, resulting in haze-free visuals. The aim of Salient Object Detection (SOD) is to locate the most visually prominent areas in images. However, most SOD techniques applied to visible images struggle in complex scenarios characterized by similarities between the foreground and background, cluttered backgrounds, adverse weather conditions, and low lighting. Identifying objects in hazy images is challenging due to the degradation of visibility caused by atmospheric conditions, leading to diminished visibility and reduced contrast. This paper introduces an innovative approach called Dehaze-SOD, a unique integrated model that addresses two vital tasks: dehazing and salient object detection. The key novelty of Dehaze-SOD lies in its dual functionality, seamlessly integrating dehazing and salient object identification into a unified framework. This is achieved using a conditional Generative Adversarial Network (cGAN) comprising two distinct subnetworks: one for image dehazing and another for salient object detection. The first module, designed with residual blocks, Dark Channel Prior (DCP), total variation, and the multiscale Retinex algorithm, processes the input hazy images. The second module employs an enhanced EfficientNet architecture with added attention mechanisms and pixel-wise refinement to further improve the dehazing process. The outputs from these subnetworks are combined to produce dehazed images, which are then fed into our proposed encoder–decoder framework for salient object detection. The cGAN is trained with two modules working together: the generator aims to produce haze-free images, whereas the discriminator distinguishes between the generated haze-free images and real haze-free images. Dehaze-SOD demonstrates superior performance compared to state-of-the-art dehazing methods in terms of color fidelity, visibility enhancement, and haze removal. The proposed method effectively produces high-quality, haze-free images from various hazy inputs and accurately detects salient objects within them. This makes Dehaze-SOD a promising tool for improving salient object detection in challenging hazy conditions. The effectiveness of our approach has been validated using benchmark evaluation metrics such as mean absolute error (MAE), peak signal-to-noise ratio (PSNR), and structural similarity index measure (SSIM).

DOUNet: Dynamic Optimization and Update Network for Oriented Object Detection

Object detection can accurately identify and locate targets in images, serving basic industries such as agricultural monitoring and urban planning. However, targets in remote sensing images have random rotation angles, which hinders the accuracy of remote sensing image object detection algorithms. In addition, due to the long-tailed distribution of detected objects in remote sensing images, the network finds it difficult to adapt to imbalanced datasets. In this article, we designed and proposed the Dynamic Optimization and Update network (DOUNet). By introducing adaptive rotation convolution to replace 2D convolution in the Region Proposal Network (RPN), the features of rotating targets are effectively extracted. To address the issues caused by imbalanced data, we have designed a long-tail data detection module to collect features of tail categories and guide the network to output more balanced detection results. Various experiments have shown that after two stages of feature learning and classifier learning, our designed network can achieve optimal performance and perform better in detecting imbalanced data.

Evolutionary computation-based self-supervised learning for image processing: a big data-driven approach to feature extraction and fusion for multispectral object detection

The image object recognition and detection technology are widely used in many scenarios. In recent years, big data has become increasingly abundant, and big data-driven artificial intelligence models have attracted more and more attention. Evolutionary computation has also provided a powerful driving force for the optimization and improvement of deep learning models. In this paper, we propose an image object detection method based on self-supervised and data-driven learning. Differ from other methods, our approach stands out due to its innovative use of multispectral data fusion and evolutionary computation for model optimization. Specifically, our method uniquely combines visible light images and infrared images to detect and identify image targets. Firstly, we utilize a self-supervised learning method and the AutoEncoder model to perform high-dimensional feature extraction on the two types of images. Secondly, we fuse the extracted features from the visible light and infrared images to detect and identify objects. Thirdly, we introduce a model parameter optimization method using evolutionary learning algorithms to enhance model performance. Validation on public datasets shows that our method achieves comparable or superior performance to existing methods.

Stage-by-Stage Adaptive Alignment Mechanism for Object Detection in Aerial Images

Stage-by-Stage Adaptive Alignment Mechanism for Object Detection in Aerial Images

Object Detection in Satellite Images with Hierarchical Collective Hunter Algorithm Optimized Ensemble Classifier

Object detection in remote sensing images plays an essential role in computer vision task, which significantly identifies the position and class of the located objects with arbitrary orientation information. Recently, object detection plays an important role in various applications such as Surveillance and security in airports, organizations, and so on. However, target tracking is a very challenging task in natural images to attain appropriate detection results together with they obtain certain issues such as limited availability of high-resolute images, background complexity, and so on. To avoid these limitations, developed a Hierarchical collective Hunter algorithm with a Deep convolutional Neural network and a Bidirectional long short-term classifier (2HC-DBTM) model, which improved the interaction range of feature extraction by reducing the background complexity. Additional attributes mechanism such as the Hierarchical Collective Hunter (HCH) algorithm and hybrid attention mechanism are integrated with the proposed model for improve the ability and performance with minimum computational complexity and achieves better detection accuracy. The experimental results obtained in the evaluation of the model are achieved with high robustness and efficiency when compared with other conventional methods. The achieved performance evaluation of the model for evaluation metric F1-score is 97.71%, precision is 97.16%, recall is 98.27% and accuracy is 98.05%.

UAV image object detection based on self-attention guidance and global feature fusion

Unmanned aerial vehicle (UAV) image object detection has garnered considerable attentions in fields such as Intelligent transportation, urban management and agricultural monitoring. However, it suffers from key challenges of the deficiency in multi-scale feature extraction and the inaccuracy when processing complex scenes and small-sized targets in practical applications. To address this challenge, we propose a novel UAV image object detection network based on self-attention guidance and global feature fusion, named SGGF-Net. First, in order to optimizing feature extraction in global perspective and enhancing target localization precision, the global feature extraction module (GFEM) is introduced by exploiting the self-attention mechanism to capture and integrate long-range dependencies within images. Second, a normal distribution-based prior assigner (NDPA) is developed by measuring the resemblance between ground truth and the priors, which improves the precision of target position matching and thus handle the problem of inaccurate localization of small targets. Furthermore, we design an attention-guided ROI pooling module (ARPM) via a deep fusion strategy of multilevel features for optimizing the integration of multi-scale features and improving the quality of feature representation. Finally, experimental results demonstrate the effectiveness of the proposed SGGF-Net approach.

Advancing Nighttime Object Detection through Image Enhancement and Domain Adaptation

Due to the lack of annotations for nighttime low-light images, object detection in low-light images has always been a challenging problem. Achieving high-precision results at night is also an issue. Additionally, we aim to use a single nighttime dataset to complete the knowledge distillation task while improving the detection accuracy of object detection models under nighttime low-light conditions and reducing the computational cost of the model, especially for small targets and objects contaminated by special nighttime lighting. This paper proposes a Nighttime Unsupervised Domain Adaptation Network (NUDN) based on knowledge distillation to address these issues. To improve the detection accuracy of nighttime images, high-confidence bounding box predictions from the teacher and region proposals from the student are first fused, allowing the teacher to perform better in subsequent training, thus generating a combination of high-confidence and low-confidence pseudo-labels. This combination of feature information is used to guide model training, enabling the model to extract feature information similar to that of source images in nighttime low-light images. Nighttime images and pseudo-labels undergo random size transformations before being used as input for the student, enhancing the model’s generalization across different scales. To address the scarcity of nighttime datasets, we propose a nighttime-specific augmentation pipeline called LightImg. This pipeline enhances nighttime features, transforming them into daytime features and reducing issues such as backlighting, uneven illumination, and dim nighttime light, enabling cross-domain research using existing nighttime datasets. Our experimental results show that NUDN can significantly improve nighttime low-light object detection accuracy on the SHIFT and ExDark datasets. We conduct extensive experiments and ablation studies to demonstrate the effectiveness and efficiency of our work.

AID-YOLO: An Efficient and Lightweight Network Method for Small Target Detector in Aerial Images

The progress of object detection technology is crucial for obtaining extensive scene information from aerial perspectives based on computer vision. However, aerial image detection presents many challenges, such as large image background sizes, small object sizes, and dense distributions. This research addresses the specific challenges relating to small object detection in aerial images and proposes an improved YOLOv8s-based detector named Aerial Images Detector-YOLO(AID-YOLO). Specifically, this study adopts the General Efficient Layer Aggregation Network (GELAN) from YOLOv9 as a reference and designs a four-branch skip-layer connection and split operation module Re-parameterization-Net with Cross-Stage Partial CSP and Efficient Layer Aggregation Networks (RepNCSPELAN4) to achieve a lightweight network while capturing richer feature information. To fuse multi-scale features and focus more on the target detection regions, a new multi-channel feature extraction module named Convolutional Block Attention Module with Two Convolutions Efficient Layer Aggregation Net-works (C2FCBAM) is designed in the neck part of the network. In addition, to reduce the sensitivity to position bias of small objects, a new function, Normalized Weighted Distance Complete Intersection over Union (NWD-CIoU_Loss) weight adaptive loss function, was designed in this study. We evaluate the proposed AID-YOLO method through ablation experiments and comparisons with other advanced models on the VEDAI (512, 1024) and DOTAv1.0 datasets. The results show that compared to the Yolov8s baseline model, AID-YOLO improves the mAP@0.5 metric by 7.36% on the VEDAI dataset. Simultaneously, the parameters are reduced by 31.7%, achieving a good balance between accuracy and parameter quantity. The Average Precision (AP) for small objects has improved by 8.9% compared to the baseline model (YOLOv8s), making it one of the top performers among all compared models. Furthermore, the FPS metric is also well-suited for real-time detection in aerial image scenarios. The AID-YOLO method also demonstrates excellent performance on infrared images in the VEDAI1024 (IR) dataset, with a 2.9% improvement in the mAP@0.5 metric. We further validate the superior detection and generalization performance of AID-YOLO in multi-modal and multi-task scenarios through comparisons with other methods on different resolution images, SODA-A and the DOTAv1.0 datasets. In summary, the results of this study confirm that the AID-YOLO method significantly improves model detection performance while maintaining a reduced number of parameters, making it applicable to practical engineering tasks in aerial image object detection.

Improved Architecture and Training Strategies of YOLOv7 for Remote Sensing Image Object Detection

The technology for object detection in remote sensing images finds extensive applications in production and people’s lives, and improving the accuracy of image detection is a pressing need. With that goal, this paper proposes a range of improvements, rooted in the widely used YOLOv7 algorithm, after analyzing the requirements and difficulties in the detection of remote sensing images. Specifically, we strategically remove some standard convolution and pooling modules from the bottom of the network, adopting stride-free convolution to minimize the loss of information for small objects in the transmission. Simultaneously, we introduce a new, more efficient attention mechanism module for feature extraction, significantly enhancing the network’s semantic extraction capabilities. Furthermore, by adding multiple cross-layer connections in the network, we more effectively utilize the feature information of each layer in the backbone network, thereby enhancing the network’s overall feature extraction capability. During the training phase, we introduce an auxiliary network to intensify the training of the underlying network and adopt a new activation function and a more efficient loss function to ensure more effective gradient feedback, thereby elevating the network performance. In the experimental results, our improved network achieves impressive mAP scores of 91.2% and 80.8% on the DIOR and DOTA version 1.0 remote sensing datasets, respectively. These represent notable improvements of 4.5% and 7.0% over the original YOLOv7 network, significantly enhancing the efficiency of detecting small objects in particular.

SpQuant-SNN: ultra-low precision membrane potential with sparse activations unlock the potential of on-device spiking neural networks applications.

Spiking neural networks (SNNs) have received increasing attention due to their high biological plausibility and energy efficiency. The binary spike-based information propagation enables efficient sparse computation in event-based and static computer vision applications. However, the weight precision and especially the membrane potential precision remain as high-precision values (e.g., 32 bits) in state-of-the-art SNN algorithms. Each neuron in an SNN stores the membrane potential over time and typically updates its value in every time step. Such frequent read/write operations of high-precision membrane potential incur storage and memory access overhead in SNNs, which undermines the SNNs' compatibility with resource-constrained hardware. To resolve this inefficiency, prior works have explored the time step reduction and low-precision representation of membrane potential at a limited scale and reported significant accuracy drops. Furthermore, while recent advances in on-device AI present pruning and quantization optimization with different architectures and datasets, simultaneous pruning with quantization is highly under-explored in SNNs. In this work, we present SpQuant-SNN, a fully-quantized spiking neural network with ultra-low precision weights, membrane potential, and high spatial-channel sparsity, enabling the end-to-end low precision with significantly reduced operations on SNN. First, we propose an integer-only quantization scheme for the membrane potential with a stacked surrogate gradient function, a simple-yet-effective method that enables the smooth learning process of quantized SNN training. Second, we implement spatial-channel pruning with membrane potential prior, toward reducing the layer-wise computational complexity, and floating-point operations (FLOPs) in SNNs. Finally, to further improve the accuracy of low-precision and sparse SNN, we propose a self-adaptive learnable potential threshold for SNN training. Equipped with high biological adaptiveness, minimal computations, and memory utilization, SpQuant-SNN achieves state-of-the-art performance across multiple SNN models for both event-based and static image datasets, including both image classification and object detection tasks. The proposed SpQuant-SNN achieved up to 13× memory reduction and >4.7× FLOPs reduction with < 1.8% accuracy degradation for both classification and object detection tasks, compared to the SOTA baseline.

Drones in Action: A Comprehensive Analysis of Drone-Based Monitoring Technologies

Unmanned aerial vehicles (UAVs), commonly referred to as drones, are extensively employed in various real-time applications, including remote sensing, disaster management and recovery, logistics, military operations, search and rescue, law enforcement, and crowd monitoring and control, owing to their affordability, rapid processing capabilities, and high-resolution imagery. Additionally, drones mitigate risks associated with terrorism, disease spread, temperature fluctuations, crop pests, and criminal activities. Consequently, this paper thoroughly analyzes UAV-based surveillance systems, exploring the opportunities, challenges, techniques, and future trends of drone technology. It covers common image preprocessing methods for drones and highlights notable one- and two-stage deep learning algorithms used for object detection in drone-captured images. The paper also offers a valuable compilation of online datasets containing drone-acquired photographs for researchers. Furthermore, it compares recent UAV-based imaging applications, detailing their purposes, descriptions, findings, and limitations. Lastly, the paper addresses potential future research directions and challenges related to drone usage

Underwater image object detection based on multi-scale feature fusion

Underwater image object detection based on multi-scale feature fusion

Computer vision that can ‘see’ in the dark

Insufficient lighting environment has raised challenges for night shift workers’ safety monitoring. Thus, we have developed a computer vision-based algorithm recognizing 11 actions based on action recognition in dark (ARID) dataset. A hybrid model of integrating convolutional neural network (CNN) into YOLOv7 has been proposed. YOLOv7 is an algorithm designed for real-time object detection in image or video, for fast and accurate detection in applications such as autonomous vehicles and surveillance systems. In this work, video in dark environment has first been enhanced using CNN algorithm before feeding into YOLOv7 network for activity recognition. Adaptive gamma intensity correction (GIC) has been integrated to further improving the overall result. The proposed model has been evaluated over different enhancement modes. The proposed model is able to handle dark video frames with 74.95% Top-1 accuracy with fast processing speed of 93.99 ms/frame on a 4 GB RTX 3050 graphical processing unit (GPU) and 17.59 ms/frame on 16 GB Tesla T4 GPU. The base size of the proposed model is tiny, only 74.8 MB, but with 36.54 M of total parameters indicating that it has more capacity to learn more meaningful information with limited hardware resources.