Image Object Detection Research Articles

Efficient Small Object Detection You Only Look Once: A Small Object Detection Algorithm for Aerial Images

Aerial images have distinct characteristics, such as varying target scales, complex backgrounds, severe occlusion, small targets, and dense distribution. As a result, object detection in aerial images faces challenges like difficulty in extracting small target information and poor integration of spatial and semantic data. Moreover, existing object detection algorithms have a large number of parameters, posing a challenge for deployment on drones with limited hardware resources. We propose an efficient small-object YOLO detection model (ESOD-YOLO) based on YOLOv8n for Unmanned Aerial Vehicle (UAV) object detection. Firstly, we propose that the Reparameterized Multi-scale Inverted Blocks (RepNIBMS) module is implemented to replace the C2f module of the Yolov8n backbone extraction network to enhance the information extraction capability of small objects. Secondly, a cross-level multi-scale feature fusion structure, wave feature pyramid network (WFPN), is designed to enhance the model’s capacity to integrate spatial and semantic information. Meanwhile, a small-object detection head is incorporated to augment the model’s ability to identify small objects. Finally, a tri-focal loss function is proposed to address the issue of imbalanced samples in aerial images in a straightforward and effective manner. In the VisDrone2019 test set, when the input size is uniformly 640 × 640 pixels, the parameters of ESOD-YOLO are 4.46 M, and the average mean accuracy of detection reaches 29.3%, which is 3.6% higher than the baseline method YOLOv8n. Compared with other detection methods, it also achieves higher detection accuracy with lower parameters.

Review of Convolutional Neural Network

Over the past decade, the domain of deep learning has emerged as a swiftly expanding area of interest among researchers globally. It offers substantial benefits over traditional shallow networks, particularly in the realms of feature extraction and model fitting. Deep learning excels at uncovering intricate, distributed features from raw data, which exhibit robust generalization capabilities. It has triumphantly addressed challenges that were once deemed intractable within the field of artificial intelligence. With the exponential growth in the volume of training data and a marked increase in computational power, deep learning has achieved remarkable strides and found applications across a spectrum of domains, including but not limited to object detection, computer vision, natural language processing, speech recognition, and semantic parsing, thereby accelerating the evolution of artificial intelligence. Deep learning encompasses a series of non-linear transformations organized hierarchically, with deep neural networks representing the predominant form of contemporary deep learning methodologies. Inspired by the neural connections found in the animal visual cortex, these networks are characterized by local connectivity, shared weights, and pooling operations. Such attributes not only simplify the network model and curtail the number of trainable parameters but also engender invariance to shifts, distortions, and scaling, thereby endowing the model with enhanced robustness and fault tolerance. This makes the training and optimization of the network structure more manageable. This paper commences with a retrospective on the evolution of convolutional neural networks (CNNs). Subsequently, it delineates the architectures of neuron models and multilayer perceptron. It proceeds to dissect the CNN architecture, which typically encompasses a sequence of convolutional and pooling layers, culminating in fully connected layers, each serving distinct functions. The paper also elaborates on various enhanced algorithms for CNNs, such as the “Network in Network” and “Spatial Transformer Networks,” while also introducing supervised and unsupervised learning methodologies pertinent to CNNs and highlighting several prevalent open-source tools. Further, the paper scrutinizes the application of CNNs in various fields, including image classification, facial recognition, audio retrieval, electrocardiogram analysis, and object detection. It posits the amalgamation of CNNs with recurrent neural networks as a potential alternative for training datasets. The research culminates in the design of CNN structures with varying parameters and depths, supported by experimental analysis that elucidates the interplay among these parameters and the ramifications of their configuration. Ultimately, the paper synthesizes the merits and lingering challenges associated with CNNs and their practical applications.

ENHANCING CNN PERFORMANCE WITH RIGHT-NEIGHBOR DEVIATION (RND) POOLING: A FOCUS ON MULTI-CLASS BRAIN TUMOR CLASSIFICATION IN MRI IMAGES

In recent years, the field of image processing and computer vision has been deeply transformed by the advent of Convolutional Neural Network (CNN). These deep learning models, inspired by biological processes, have demonstrated remarkable success in tasks such as image classification, object detection, and semantic segmentation. A critical component of this success is feature extraction, where the pooling layer within CNN serves as a nonlinear down-sampling technique. This layer reduces the spatial size of the convolved feature map through operations like average or maximum pooling, significantly impacting the model’s performance in complex tasks. This research introduces a novel pooling algorithm, Right-Neighbor Deviation (RND), designed to enhance the performance of CNN. The RND pooling method aims to capture detailed spatial features and hierarchical representations more effectively, thereby improving the accuracy and resilience of deep learning models. A dataset of multi-class brain tumor classification, incorporating MRI images from the BR35H dataset, is utilized to explore the potential benefits of RND pooling compared to traditional methods. The effectiveness of the proposed method is evaluated using performance metrics including accuracy, precision, recall, and F1 score, achieving generalized values of 98.86%, 98.84%, 98.80%, and 98.80%, respectively. The results highlight the potential of RND pooling to significantly advance the state of the art in brain tumor classification using CNNs, potentially leading to improved diagnosis and treatment outcomes.

Establishing effective learning bridge cross multi-scale feature maps for object detection and semantic segmentation

ABSTRACT In the field of image object detection and semantic segmentation, improving the accuracy of object identification and segmentation is a primary goal. To achieve this, leveraging the potential of multi-scale information through feature map refinement and fusion has been widely recognized. However, existing feature fusion methods either design more complex feature pyramid networks, replace existing detectors, or incrementally introduce feature fusion modules, overlooking the effective approach of enhancing spatial information in deep feature maps. We propose a novel pluggable feature fusion paradigm termed ‘Effective Learning Bridge’. Our research introduces an efficient and adaptive learning mechanism that builds learning bridges between feature maps at different scales within the feature pyramid, thereby enhancing the spatial information of objects in deep feature maps. This mechanism is specifically designed for multi-scale feature maps and can be seamlessly integrated into any network incorporating feature maps. By altering the model’s backpropagation path, we successfully improve learning efficiency, which in turn enhances the accuracy of object detection and segmentation. Our proposed paradigm and method were extensively evaluated through experiments on SIMD, HRSID, and WHDLD datasets and benchmark models. The results unequivocally demonstrate the effectiveness of our approach in significantly improving the accuracy of object detection and semantic segmentation, as well as the overall learning efficiency of the model.

Image quality and object detection performance

Image quality and object detection performance

Low power RTL implementation of Tiny-YOLO-v2 DCNN hardware accelerator on Virtex-7 FPGA

Deep Convolution Neural Networks (DCNNs) are widely used in real-time applications, including image classification, speech recognition, and object detection. However, there are challenges for real-time applications on portable devices like mobile phones or embedded systems. Most object detection models are optimized for desktop configurations, requiring fast GPUs. Shallower networks with fewer computational complexities have been proposed for real-time detection, but ultimately compromise detection accuracy. Performance and complexity trade-offs are significant for computationally complex deep networks. This work aims to implement the CNN-based object detection model Tiny-Yolo-v2 on a Field Programmable Gate Array (FPGA) using Register Transfer Logic (RTL) as a native language. Hardware implementation is synthesized on The AMD Virtex 7 FPGA VC709 Connectivity Kit using VHDL code on Vivado 2020.1. This is the first, up to the authors' knowledge, RTL implementation of the Tiny-Yolo-v2 object identification algorithm on FPGA. The power consumed by the CNN layers equals 7.09W at a frequency of 100MHz.

Object Detection in Historical Images: Transfer Learning and Pseudo Labelling

The automatic analysis of images in the historical sciences often requires the identification of objects. Object identification is a well researched problem for modern photographs, however, for historical material annotations are often necessary. We present a solution for finding objects without manual work. The method consists of a style transfer of images from the COCO dataset into the domain using CycleGAN and training with items obtained through pseudo labelling on the original and the additional transferred COCO images. Different strategies to assemble the dataset are compared. The best method obtains a F1 score of 0.58 for 15 object types without any labelling.

Detection and Identification of Hazardous Hidden Objects in Images: A Comprehensive Review

Detection and Identification of Hazardous Hidden Objects in Images: A Comprehensive Review

Brief Study on Convolutional Neural Networks

Convolutional Neural Networks (CNNs) have revolutionized the field of computer vision and pattern recognition, offering state-of-the-art performance in tasks such as image classification, object detection, and segmentation. This study explores the architecture, training strategies, and applications of CNNs, focusing on their ability to automatically learn spatial hierarchies of features through layers of convolutional filters. The research evaluates various CNN architectures, including Lent, Alex Net, and ResNet, highlighting their improvements in accuracy and efficiency. Additionally, the study investigates advanced techniques such as data augmentation, transfer learning, and regularization methods (e. g, dropout and batch normalization), which enhance the model's generalization capabilities. Through empirical experiments, we demonstrate the effectiveness of CNNs in real-world scenarios, including medical image analysis, autonomous driving, and facial recognition. The study concludes with a discussion of the future trends of CNNs, particularly in the context of deep learning optimization, hardware acceleration, and integration with emerging technologies like edge computing and quantum machine learning.

An Efficient UAV Image Object Detection Algorithm Based on Global Attention and Multi-Scale Feature Fusion

An Efficient UAV Image Object Detection Algorithm Based on Global Attention and Multi-Scale Feature Fusion

Data science basis and influencing factors for the evaluation of environmental safety perception in Macau parishes

In the context of rapid urbanization, accurately identifying the visual factors that influence environmental safety perception is crucial for improving urban transportation environments and enhancing pedestrian safety. With the increase in urban population density and traffic flow, optimizing urban environmental design to elevate residents’ sense of safety has become a key issue in urban planning and management. However, the existing studies face numerous challenges in conducting large-scale quantitative analysis of environmental safety perception in complex scenarios, such as difficulties in data acquisition and limitations in analytical methods. This study addresses these challenges by applying image semantic segmentation and object detection techniques to extract key visual elements from street view images, combined with manual scoring and deep learning methods, to construct a road safety perception dataset. Using a LightGBM model and the SHAP interpretation framework, in this study, we identify the critical visual factors influencing environmental safety perception. An empirical study was conducted in Macau, a modern city where Eastern and Western cultures intersect, and tourism thrives. The findings reveal that: ① The overall environmental safety perception in the eight parishes and surrounding roads of Macau is relatively high, with significant regional differences in safety perception scores around Macau’s parish roads; ② The proportions of buildings, sidewalks, roads, and trees in images are the four primary factors influencing environmental safety perception; ③ The proportions and quantities of visual elements interact with each other, and their reasonable distribution helps form clear spatial visibility and creates conducive activity spaces, thereby enhancing the perception of environmental safety. Through empirical analysis, this study uncovers the mechanisms by which visual elements in urban street scenes affect environmental safety perception, providing scientific evidence for urban planning and transportation environment improvement. The research holds theoretical significance and offers practical references for urban design and management, demonstrating broad application value.

A generic shared attention mechanism for various backbone neural networks

The self-attention mechanism is crucial for enhancing various backbone neural networks’ performance. However, current methods add self-attention modules (SAMs) to each network layer without fully utilizing their potential, resulting in suboptimal performance and higher parameter consumption as network depth increases. In this paper, we reveal an inherent phenomenon: SAMs produce highly correlated attention maps across layers, with an average Pearson correlation coefficient of 0.85. Inspired by this inherent observation, we propose Dense-and-Implicit Attention (DIA), which shares SAMs across layers and uses a long short-term memory module to calibrate and connect these correlated attention maps, improving parameter efficiency. This approach aligns with the neural network’s dynamic system perspective. Extensive experiments show DIA consistently enhances various backbones like ResNet, Transformer, and UNet in tasks such as image classification, object detection, and image generation with diffusion models. Our analysis indicates that DIA’s effectiveness stems from its dense inter-layer information connections, absent in conventional mechanisms, stabilizing training and providing regularization effects. This paper’s insights advance our understanding of attention mechanisms, optimizing them and paving the way for future developments across diverse neural networks.

Image Enhancement Guided Object Detection in Visually Degraded Scenes.

Object detection accuracy degrades seriously in visually degraded scenes. A natural solution is to first enhance the degraded image and then perform object detection. However, it is suboptimal and does not necessarily lead to the improvement of object detection due to the separation of the image enhancement and object detection tasks. To solve this problem, we propose an image enhancement guided object detection method, which refines the detection network with an additional enhancement branch in an end-to-end way. Specifically, the enhancement branch and detection branch are organized in a parallel way, and a feature guided module is designed to connect the two branches, which optimizes the shallow feature of the input image in the detection branch to be as consistent as possible with that of the enhanced image. As the enhancement branch is frozen during training, such a design plays a role in using the features of enhanced images to guide the learning of object detection branch, so as to make the learned detection branch being aware of both image quality and object detection. When testing, the enhancement branch and feature guided module are removed, and so no additional computation cost is introduced for detection. Extensive experimental results, on underwater, hazy, and low-light object detection datasets, demonstrate that the proposed method can improve the detection performance of popular detection networks (YOLO v3, Faster R-CNN, DetectoRS) significantly in visually degraded scenes.

Deep convolutional neural networks for ship detection using refined DOTA and TGRS-HRRSD high-resolution image datasets

Automated identification of ships in satellite imagery is of utmost significance in many military and civilian applications, including monitoring maritime traffic and maintaining maritime security. However, ship detection from remotely sensed imagery is a challenging task because of their varying sizes, orientations, types, and various interferences. In recent years, deep learning algorithms have become a powerful tool in image processing applications, including image classification and object detection due to their strong feature representation capabilities. In this study, a refined ship dataset, containing a total of 13,735 instances and representing different ship types, was generated using state-of-the-art datasets for the ship detection task. In a comparative framework, the performance of five deep learning algorithms (Faster R-CNN, YOLOv8, CenterNet, EfficientDet, and SSD) in ship detection was investigated using the created dataset. The performance of the deep learning models was compared using MS COCO accuracy metrics. In addition, the generalization capabilities of the models were tested on an independent image acquired by Göktürk-1 having a mucilage effect that undermines the spectral discrimination. Results revealed that the YOLOv8 model had the greatest ship detection performance (AP@0.50 = 96.33 %), followed by CenterNet (AP@0.50 = 88.80 %), Faster R-CNN (AP@0.50 = 85.30 %), EfficientDet (AP@0.50 = 78.91 %) and SSD (AP@0.5 = 75.98 %), and obtained the highest accuracies in terms of other COCO metrics. In addition, generalization capabilities on the test image confirmed the superiority of the YOLOv8 model with AP@0.50 score of 63.93 %. Overall, the results of this study validated the effectiveness of the YOLOv8 model in identifying ship targets in remotely sensed images.

Shape-Guided Detection: A joint network combining object detection and underwater image enhancement together

Most of the existing underwater image object detection methods involve pre-processing, such as using underwater image enhancement, to improve the accuracy of object detection. However, pre-processing methods are designed to improve the subjective perception of the human eye, which does not necessarily improve the object detection performance and consumes a large amount of computational resources. Therefore, in this paper, we creatively combine these two tasks and propose a Shape-Guided Detection network (SGD) to simultaneously optimize underwater image enhancement and object detection. In the SGD network, we innovatively incorporate the prior shape features as a learnable module embedded in it to fully explore the shape characteristics and structural details of the target object. To ensure that the prior knowledge can be effectively fused into the global network structure, we design a Shape Prior Enhancement module, which aims to realize the deep integration of the prior information with the local details. In order to optimize the stability of model training and enhance its convergence performance, a dual strategy of explicit and implicit constraints is ingeniously proposed in our method. We conduct extensive experiments on public datasets and the results show that the combination of our method with different detectors significantly improves the performance. The object detection performance reaches up to 0.491 mAP for optical images and 0.576 mAP for sonar images, and improves the preprocessing speed by 0.1 s.

Spatial attention for human-centric visual understanding: An Information Bottleneck method

The selective visual attention mechanism in the Human Visual System (HVS) restricts the amount of information that reaches human visual awareness, allowing the brain to perceive high-fidelity natural scenes in real-time with limited computational cost. This selectivity acts as an “Information Bottleneck (IB)” that balances information compression and predictive accuracy. However, such information constraints are rarely explored in the attention mechanism for deep neural networks (DNNs). This paper introduces an IB-inspired spatial attention module for DNNs, which generates an attention map by minimizing the mutual information (MI) between the attentive content and the input while maximizing that between the attentive content and the output. We develop this IB-inspired attention mechanism based on a novel graphical model and explore various implementations of the framework. We show that our approach can yield attention maps that neatly highlight the regions of interest while suppressing the backgrounds, and are interpretable for the decision-making of the DNNs. To validate the effectiveness of the proposed IB-inspired attention mechanism, we apply it to various computer vision tasks including image classification, fine-grained recognition, cross-domain classification, semantic segmentation, and object detection. Extensive experiments demonstrate that it bootstraps standard DNN structures quantitatively and qualitatively for these tasks.

Multimodal Social Media Fake News Detection Based on 1D-CCNet Attention Mechanism

Due to the explosive rise of multimodal content in online social communities, cross-modal learning is crucial for accurate fake news detection. However, current multimodal fake news detection techniques face challenges in extracting features from multiple modalities and fusing cross-modal information, failing to fully exploit the correlations and complementarities between different modalities. To address these issues, this paper proposes a fake news detection model based on a one-dimensional CCNet (1D-CCNet) attention mechanism, named BTCM. This method first utilizes BERT and BLIP-2 encoders to extract text and image features. Then, it employs the proposed 1D-CCNet attention mechanism module to process the input text and image sequences, enhancing the important aspects of the bimodal features. Meanwhile, this paper uses the pre-trained BLIP-2 model for object detection in images, generating image descriptions and augmenting text data to enhance the dataset. This operation aims to further strengthen the correlations between different modalities. Finally, this paper proposes a heterogeneous cross-feature fusion method (HCFFM) to integrate image and text features. Comparative experiments were conducted on three public datasets: Twitter, Weibo, and Gossipcop. The results show that the proposed model achieved excellent performance.

Estimating Power, Performance, and Area for On-Sensor Deployment of AR/VR Workloads Using an Analytical Framework

Augmented Reality and Virtual Reality have emerged as the next frontier of intelligent image sensors and computer systems. In these systems, 3D die stacking stands out as a compelling solution, enabling in situ processing capability of the sensory data for tasks such as image classification and object detection at low power, low latency, and a small form factor. These intelligent 3D CMOS Image Sensor (CIS) systems present a wide design space, encompassing multiple domains (e.g., computer vision algorithms, circuit design, system architecture, and semiconductor technology, including 3D stacking) that have not been explored in-depth so far. This article aims to fill this gap. We first present an analytical evaluation framework, STAR-3DSim, dedicated to rapid pre-RTL evaluation of 3D-CIS systems capturing the entire stack from the pixel layer to the on-sensor processor layer. With STAR-3DSim, we then propose several knobs for PPA (power, performance, area) improvement of the Deep Neural Network (DNN) accelerator that can provide up to 53%, 41%, and 63% reduction in energy, latency, and area, respectively, across a broad set of relevant AR/VR workloads. Last, we present full-system evaluation results by taking image sensing, cross-tier data transfer, and off-sensor communication into consideration.

A novel target detection method with dual‐domain multi‐frequency feature in side‐scan sonar images

AbstractSide‐scan sonar (SSS) detection is a key method in underwater environmental security and subsea resource development. However, many detection approaches primarily concentrate on tracking the evolution path of optical image object detection tasks when using acoustic images, resulting in complex structures and limited versatility. To tackle this issue, we introduce a pioneering dual‐domain multi‐frequency network (D2MFNet) meticulously crafted to harness the distinct characteristics of SSS image detection. In D2MFNet, a novel method for optimizing and improving the detection sensitivity in different frequency ranges called multi‐frequency combined attention mechanism (MFCAM) is proposed. This mechanism amplifies the relevance of dual‐domain features across different channels and spaces. Moreover, we introduce a dual‐domain feature pyramid network (D2FPN) significantly augments the depth and breadth of feature information in underwater small datasets. The methods offer plug‐and‐play functionality with substantial performance enhancements. Extensive experiments are conducted to validate the efficacy of the proposed techniques, and the results showcase their state‐of‐the‐art performance. MFCAM improves the mAP by 16.9% in the KLSG dataset and 15.5% in the SCTD dataset. The mAP of D2FPN was improved by 8.4% in the KLSG dataset and by 9.8% in the SCTD dataset. The code and models will be publicly available at https://dagshub.com/estrellaww00/D2MFNet.

A novel multiscale cGAN approach for enhanced salient object detection in single haze images

In computer vision, image dehazing is a low-level task that employs algorithms to analyze and remove haze from images, resulting in haze-free visuals. The aim of Salient Object Detection (SOD) is to locate the most visually prominent areas in images. However, most SOD techniques applied to visible images struggle in complex scenarios characterized by similarities between the foreground and background, cluttered backgrounds, adverse weather conditions, and low lighting. Identifying objects in hazy images is challenging due to the degradation of visibility caused by atmospheric conditions, leading to diminished visibility and reduced contrast. This paper introduces an innovative approach called Dehaze-SOD, a unique integrated model that addresses two vital tasks: dehazing and salient object detection. The key novelty of Dehaze-SOD lies in its dual functionality, seamlessly integrating dehazing and salient object identification into a unified framework. This is achieved using a conditional Generative Adversarial Network (cGAN) comprising two distinct subnetworks: one for image dehazing and another for salient object detection. The first module, designed with residual blocks, Dark Channel Prior (DCP), total variation, and the multiscale Retinex algorithm, processes the input hazy images. The second module employs an enhanced EfficientNet architecture with added attention mechanisms and pixel-wise refinement to further improve the dehazing process. The outputs from these subnetworks are combined to produce dehazed images, which are then fed into our proposed encoder–decoder framework for salient object detection. The cGAN is trained with two modules working together: the generator aims to produce haze-free images, whereas the discriminator distinguishes between the generated haze-free images and real haze-free images. Dehaze-SOD demonstrates superior performance compared to state-of-the-art dehazing methods in terms of color fidelity, visibility enhancement, and haze removal. The proposed method effectively produces high-quality, haze-free images from various hazy inputs and accurately detects salient objects within them. This makes Dehaze-SOD a promising tool for improving salient object detection in challenging hazy conditions. The effectiveness of our approach has been validated using benchmark evaluation metrics such as mean absolute error (MAE), peak signal-to-noise ratio (PSNR), and structural similarity index measure (SSIM).