Image Object Detection Research Articles

Infrared remote sensing ship image object detection model based on YOLO In multiple environments

Infrared remote sensing ship image object detection model based on YOLO In multiple environments

An Evaluation of Image Slicing and YOLO Architectures for Object Detection in UAV Images

Object detection in aerial images poses significant challenges due to the high dimensions of the images, requiring efficient handling and resizing to fit object detection models. The image-slicing approach for object detection in aerial images can increase detection accuracy by eliminating pixel loss in high-resolution image data. However, determining the proper dimensions to slice is essential for the integrity of the objects and their learning by the model. This study presents an evaluation of the image-slicing approach for alternative sizes of images to optimize efficiency. For this purpose, a dataset of high-resolution images collected with Unmanned Aerial Vehicles (UAV) has been used. The experiments evaluated using alternative YOLO architectures like YOLOv7, YOLOv8, and YOLOv9 show that the image dimensions significantly change the performance results. According to the experiments, the best mAP@05 accuracy was obtained by slicing 1280×1280 for YOLOv7 producing 88.2. Results show that edge-related objects are better preserved as the overlap and slicing sizes increase, resulting in improved model performance.

A novel knowledge distillation framework for enhancing small object detection in blurry environments with unmanned aerial vehicle-assisted images

Deep learning-based object detectors excel on mobile devices but often struggle with blurry images that are common in real-world scenarios, like unmanned aerial vehicle (UAV)-assisted images. Current models are designed for sharp images, leading to potential detection failures in blurry images. Using image deblurring before object detection is an option, but it demands significant computing power and relies heavily on the accuracy of the deblurring algorithms. Another common issue is the suitable dataset for the specific problem. To address the aforementioned issues, we develop a UAV-assisted small object detection dataset and propose a novel knowledge distillation method for object detection in blurry images in complex environments. Following this, we employ a technique known as self-supervised knowledge distillation, where we introduce a deblurring subnet module with the help of two attention modules, where both networks are trained in a fully-supervised manner. Based on the experiment results, our proposed model achieves an improvement of 4.3% accuracy in the VisDrone synthetic motion blur dataset and 4.6% in detecting objects within synthetic blurry images in our developed small object detection dataset (SOD-Dataset), as well as competitive results compared with other state-of-the-art methods. Meanwhile, ablation experiments and a visualization analysis validate the contributions of each component of the model.

Computer vision methods for side scan sonar imagery

Abstract Side-scan sonar data presents a significant challenge to analysts due to the size of the material that requires processing. Automatic recognition systems can help in the interpretation and filtration of the data, reducing the time needed for analysis. Deep learning architectures have been employed for computer vision tasks in the underwater domain. The objective of this work is to review current deep learning methods for automatic image classification, object detection, semantic segmentation, and instance segmentation through statistical analysis of published studies. Further, this work includes an inventory of publicly available side-scan sonar data followed by a discussion on augmentation methods applied to side-scan sonar imagery. The study discusses current challenges in the automatic underwater target detection field, highlights main gaps within the domain and identifies potential research directions. Our proposal includes the creation of a standardised benchmark and the use of a common set of metrics allowing for consistent evaluation of future research works.

Small aircraft detection in infrared aerial imagery based on deep neural network

Detection of aerial target is an important part of infrared image processing. Both neural network method and traditional method can be used in infrared object detection. Neural network method has many advantages such as high accuracy and good portability compared with traditional object detection method. Since the features extracted by neural network method can change over detection target, automatic feature extraction makes neural network based detection method more effective. In recent years deep learning method has been also found wide use for object detection in images. In this paper, an object detection model based on the deep learning network YOLO is constructed for solving the infrared aircraft detection problem. We construct the dataset used for training and testing with recognized features being iteratively learned. The task of infrared otject detection is sensitive to model size and detection speed. There is a requirement of using quantization method to reduce the storage space and the computation complexity. We propose a quantized model with appropriate accuracy for infrared object detection task. To solve the detection task for multiple extremely small aircrafts, model adjustment and quantization are used in proposed model and it gets a better performance. Experimental results on the constructed dataset show that the storage space for weight after quantization shrinks to a quarter, and there is no precision loss for extremely small aircrafts compared to the original model. The optimized YOLO-based deep learning model is effective to detect the small aircraft target in infrared aerial imagery.

CCW-YOLO: A Modified YOLOv5s Network for Pedestrian Detection in Complex Traffic Scenes

In traffic scenes, pedestrian target detection faces significant issues of misdetection and omission due to factors such as crowd density and obstacle occlusion. To address these challenges and enhance detection accuracy, we propose an improved CCW-YOLO algorithm. The algorithm first introduces a lightweight convolutional layer using GhostConv and incorporates an enhanced C2f module to improve the network’s detection performance. Additionally, it integrates the Coordinate Attention module to better capture key points of the targets. Next, the bounding box loss function CIoU loss at the output of YOLOv5 is replaced with WiseIoU loss to enhance adaptability to various detection scenarios, thereby further improving accuracy. Finally, we develop a pedestrian count detection system using PyQt5 to enhance human–computer interaction. Experimental results on the INRIA public dataset showed that our algorithm achieved a detection accuracy of 98.4%, representing a 10.1% improvement over the original YOLOv5s algorithm. This advancement significantly enhances the detection of small objects in images and effectively addresses misdetection and omission issues in complex environments. These findings have important practical implications for ensuring traffic safety and optimizing traffic flow.

Influence Function Based Second-Order Channel Pruning: Evaluating True Loss Changes for Pruning is Possible Without Retraining.

A challenge of channel pruning is designing efficient and effective criteria to select channels to prune. A widely used criterion is minimal performance degeneration, e.g., loss changes before and after pruning being the smallest. To accurately evaluate the truth performance degeneration requires retraining the survived weights to convergence, which is prohibitively slow. Hence existing pruning methods settle to use previous weights (without retraining) to evaluate the performance degeneration. However, we observe that the loss changes differ significantly with and without retraining. It motivates us to develop a technique to evaluate true loss changes without retraining, using which to select channels to prune with more reliability and confidence. We first derive a closed-form estimator of the true loss change per mask change, using influence functions without retraining. Influence function is a classic technique from robust statistics that reveals the impacts of a training sample on the model's prediction and is repurposed by us to assess impacts on true loss changes. We then show how to assess the importance of all channels simultaneously and develop a novel global channel pruning algorithm accordingly. We conduct extensive experiments to verify the effectiveness of the proposed algorithm, which significantly outperforms the competing channel pruning methods on both image classification and object detection tasks. To the best of our knowledge, we are the first that shows evaluating true loss changes or pruning without retraining is possible. This finding will open up opportunities for a series of new paradigms to emerge that differ from existing pruning methods.

Revolutionizing agricultural efficiency with advanced coconut harvesting automation

<span style="vertical-align: inherit;"><span style="vertical-align: inherit;">The precision coconut harvesting system aims to develop an efficient system for accurately detecting coconuts in agricultural landscapes using advanced image processing techniques. Coconut cultivation is vital to many tropical economies and precise monitoring is essential for optimizing yield and resource utilization. Traditional methods of coconut detection are labor-intensive and time-consuming. The proposed computer vision-based approach automates and enhances coconut detection by analyzing high-resolution images of coconut plantations. Pre-processing techniques improve image quality and object detection algorithms such as convolutional neural networks (CNNs) identify coconut clusters. Challenges like lighting variations and background clutter are addressed using feature extraction and pattern recognition. A user-friendly interface visualizes detection results, aiding farmers in timely decision-making. Extensive testing on diverse datasets evaluates system effectiveness. This model aims to advance precision agriculture, enhancing productivity and informing coconut farmers' decision-making processes. Using a CNN model, the accuracy of coconut detection based on its ripeness was 98.8%.</span></span>

Perceptual Video Coding for Machines via Satisfied Machine Ratio Modeling.

Video Coding for Machines (VCM) aims to compress visual signals for machine analysis. However, existing methods only consider a few machines, neglecting the majority. Moreover, the machine's perceptual characteristics are not leveraged effectively, resulting in suboptimal compression efficiency. To overcome these limitations, this paper introduces Satisfied Machine Ratio (SMR), a metric that statistically evaluates the perceptual quality of compressed images and videos for machines by aggregating satisfaction scores from them. Each score is derived from machine perceptual differences between original and compressed images. Targeting image classification and object detection tasks, we build two representative machine libraries for SMR annotation and create a large-scale SMR dataset to facilitate SMR studies. We then propose an SMR prediction model based on the correlation between deep feature differences and SMR. Furthermore, we introduce an auxiliary task to increase the prediction accuracy by predicting the SMR difference between two images in different quality. Extensive experiments demonstrate that SMR models significantly improve compression performance for machines and exhibit robust generalizability on unseen machines, codecs, datasets, and frame types.

Pixel Distillation: Cost-Flexible Distillation Across Image Sizes and Heterogeneous Networks.

Previous knowledge distillation (KD) methods mostly focus on compressing network architectures, which is not thorough enough in deployment as some costs like transmission bandwidth and imaging equipment are related to the image size. Therefore, we propose Pixel Distillation that extends knowledge distillation into the input level while simultaneously breaking architecture constraints. Such a scheme can achieve flexible cost control for deployment, as it allows the system to adjust both network architecture and image quality according to the overall requirement of resources. Specifically, we first propose an input spatial representation distillation (ISRD) mechanism to transfer spatial knowledge from large images to student's input module, which can facilitate stable knowledge transfer between CNN and ViT. Then, a Teacher-Assistant-Student (TAS) framework is further established to disentangle pixel distillation into the model compression stage and input compression stage, which significantly reduces the overall complexity of pixel distillation and the difficulty of distilling intermediate knowledge. Finally, we adapt pixel distillation to object detection via an aligned feature for preservation (AFP) strategy for TAS, which aligns output dimensions of detectors at each stage by manipulating features and anchors of the assistant. Comprehensive experiments on image classification and object detection demonstrate the effectiveness of our method.

OOD-CV-v2 : An Extended Benchmark for Robustness to Out-of-Distribution Shifts of Individual Nuisances in Natural Images.

Enhancing the robustness of vision algorithms in real-world scenarios is challenging. One reason is that existing robustness benchmarks are limited, as they either rely on synthetic data or ignore the effects of individual nuisance factors. We introduce OOD-CV-v2, a benchmark dataset that includes out-of-distribution examples of 10 object categories in terms of pose, shape, texture, context and the weather conditions, and enables benchmarking of models for image classification, object detection, and 3D pose estimation. In addition to this novel dataset, we contribute extensive experiments using popular baseline methods, which reveal that: 1) Some nuisance factors have a much stronger negative effect on the performance compared to others, also depending on the vision task. 2) Current approaches to enhance robustness have only marginal effects, and can even reduce robustness. 3) We do not observe significant differences between convolutional and transformer architectures. We believe our dataset provides a rich test bed to study robustness and will help push forward research in this area.

Efficient Masked Autoencoders With Self-Consistency.

Inspired by the masked language modeling (MLM) in natural language processing tasks, the masked image modeling (MIM) has been recognized as a strong self-supervised pre-training method in computer vision. However, the high random mask ratio of MIM results in two serious problems: 1) the inadequate data utilization of images within each iteration brings prolonged pre-training, and 2) the high inconsistency of predictions results in unreliable generations, i.e., the prediction of the identical patch may be inconsistent in different mask rounds, leading to divergent semantics in the ultimately generated outcomes. To tackle these problems, we propose the efficient masked autoencoders with self-consistency (EMAE) to improve the pre-training efficiency and increase the consistency of MIM. In particular, we present a parallel mask strategy that divides the image into K non-overlapping parts, each of which is generated by a random mask with the same mask ratio. Then the MIM task is conducted parallelly on all parts in an iteration and the model minimizes the loss between the predictions and the masked patches. Besides, we design the self-consistency learning to further maintain the consistency of predictions of overlapping masked patches among parts. Overall, our method is able to exploit the data more efficiently and obtains reliable representations. Experiments on ImageNet show that EMAE achieves the best performance on ViT-Large with only 13% of MAE pre-training time using NVIDIA A100 GPUs. After pre-training on diverse datasets, EMAE consistently obtains state-of-the-art transfer ability on a variety of downstream tasks, such as image classification, object detection, and semantic segmentation.

End to end polysemantic cooperative mixed task trainer for UAV target detection

With the rapid advancement and application of Unmanned Aerial Vehicles (UAVs), target detection in urban scenes has made significant progress. Achieving precise 3D reconstruction from oblique imagery is essential for accurate urban object detection in UAV images. However, challenges persist due to low detection accuracy caused by subtle target features, complex backgrounds, and the prevalence of small targets. To address these issues, we introduce the Polysemantic Cooperative Detection Transformer (Pc-DETR), a novel end-to-end UAV image target detection network. Our primary innovation, the Polysemantic Transformer (PoT) Backbone, enhances visual representation by leveraging contextual information to guide a dynamic attention matrix. This matrix, formed through convolutions, captures both static and dynamic features, resulting in superior detection. Additionally, we propose the Polysemantic Cooperative Mixed-Task Training scheme, which employs multiple auxiliary heads for diverse label assignments, boosting the encoder’s learning capacity. This approach customizes queries and optimizes training efficiency without increasing inference costs. Comparative experiments show that Pc-DETR achieves a 3% improvement in detection accuracy over the current state-of-the-art MFEFNet, setting a new benchmark in UAV image detection and advancing methodologies for intelligent UAV surveillance systems.

Green building interior design based on digital image processing and thermal environment simulation

With the increasingly severe global climate change and energy crisis, this study aims to explore effective strategies for green building interior design, especially in terms of thermal environment control, through digital image processing and thermal environment simulation. The study simulates indoor scenes based on image processing technology, using methods such as image acquisition, localization, and object detection to improve the accuracy of scene reconstruction, and conducts in-depth analysis of detection effects, laying the foundation for subsequent thermal environment simulations. In the design of the thermal energy environment control system, the calculation of indoor cooling load was studied and compared with existing thermal environment models to determine the impact of user activities on the indoor thermal environment. By establishing a thermal energy environment control model and implementing precise environmental regulation for different usage scenarios, indoor comfort and energy efficiency have been significantly improved. The experimental results show that the use of digital image processing simulation technology can effectively improve the accuracy and flexibility of indoor thermal environment control in green buildings, optimize energy efficiency, emphasize the importance of thermal factors in interior design, and provide theoretical basis and practical guidance for future building design practices.

Depth-Wise Convolutions in Vision Transformers for efficient training on small datasets

The Vision Transformer (ViT) leverages the Transformer’s encoder to capture global information by dividing images into patches and achieves superior performance across various computer vision tasks. However, the self-attention mechanism of ViT captures the global context from the outset, overlooking the inherent relationships between neighboring pixels in images or videos. Transformers mainly focus on global information while ignoring the fine-grained local details. Consequently, ViT lacks inductive bias during image or video dataset training. In contrast, convolutional neural networks (CNNs), with their reliance on local filters, possess an inherent inductive bias, making them more efficient and quicker to converge than ViT with less data. In this paper, we present a lightweight Depth-Wise Convolution module as a shortcut in ViT models, bypassing entire Transformer blocks to ensure the models capture both local and global information with minimal overhead. Additionally, we introduce two architecture variants, allowing the Depth-Wise Convolution modules to be applied to multiple Transformer blocks for parameter savings, and incorporating independent parallel Depth-Wise Convolution modules with different kernels to enhance the acquisition of local information. The proposed approach significantly boosts the performance of ViT models on image classification, object detection, and instance segmentation by a large margin, especially on small datasets, as evaluated on CIFAR-10, CIFAR-100, Tiny-ImageNet and ImageNet for image classification, and COCO for object detection and instance segmentation. The source code can be accessed at https://github.com/ZTX-100/Efficient_ViT_with_DW.

Multi-scale feature fusion algorithm for underwater remote object detection in forward-looking sonar images

Multi-scale feature fusion algorithm for underwater remote object detection in forward-looking sonar images

SIGKD: A Structured Instance Graph Distillation Method for Efficient Object Detection in Remote Sensing Images

In the realm of remote sensing image object detection compression distillation, establishing an efficient method for instance feature knowledge transfer between teacher and student models holds paramount importance. To this end, this paper introduces an innovative deep structured instance graph distillation method that endeavors to delve into the underlying information between instance features, thereby optimizing detection performance. Specifically, our proposed method incorporates feature instances and their relations into a graph-based structure (SIG). In this graph, feature instances serve as the nodes, while the relations between them serve as the edges. This structure enables us to capture both the individual significance of each feature instance and their collective influence within the context. Furthermore, in the experiment, we found that the index of some dense and small-target objects did not improve much because the edge assembly generated by a large number of background feature nodes in the SIG module inhibited the loss. To address the perennial imbalance between foreground and background features, we introduce an adaptive background feature mining strategy. Through carefully calibrated weights, this strategy effectively extracts and integrates background information, thereby minimizing noise interference in detection results and augmenting the expressive capacity of foreground features. We achieved state-of-the-art results on both the challenging DIOR and DOTA datasets, with the two-stage Oriented RCNN-based student Resnet18 model achieving a 73.23 mAP on the DOTA benchmark, close to the teacher Resnet101’s 76.16. In addition, on the DIOR dataset, the student Resnet18 based on the two-stage Faster RCNN achieved 70.13 mAP, higher than the baseline 66.31, and the student Resnet50 achieved 72.28, higher than the teacher’s 72.25.

An Improved DETR Based on Angle Denoising and Oriented Boxes Refinement for Remote Sensing Object Detection

Remote sensing image object detection presents significant challenges, due to the difficulty in accurately predicting the rotational angles of ground-oriented objects, coupled with issues such as the false and missed detection caused by insufficient object information. Moreover, traditional convolutional neural networks are inherently limited in their capacity to capture global contextual information. To address these challenges, a DETR-based remote sensing image object detection model is designed for oriented objects. Except for the backbone, transformer encoders and decoders, scenario query guiding modules, oriented boxes refinement modules, auxiliary multiple detectors, and oriented boxes denoising modules are also designed and included in our network. The scenario query guiding module is proposed that implicitly guides the decoder to focus more on object classification information specific to that scene when inferring. The multiple deformable attention mechanism is improved to the oriented one and utilized into the oriented boxes refinement module which repeatedly corrects the oriented boxes, enhancing the network’s ability to predict the oriented boxes precisely. At the same time, the improved auxiliary multiple detectors and oriented boxes denoising module are applied only for the training process to enhance the learning ability of the encoder and decoder for oriented objects. The ablation experiments proved the effectiveness of the designed modules. The detection accuracy of our network on DOTAv1.0 (76.77%) and HRCS2016 (97.01%) is improved compared with the state-of-the-art methods, which are especially significantly higher than DETR detection algorithms.

A multimodal framework for pepper diseases and pests detection

Pepper diseases and pests typically exhibit small target proportions, diverse shapes and sizes, complex imaging backgrounds, and similarities with the background. Existing detection methods perform poorly in identifying targets of different sizes and shapes within the same scene, and they lack adequate noise suppression capabilities. To address the practical needs of detecting pepper diseases and pests in complex scenarios, we have constructed the first multimodal pepper diseases and pests object detection dataset (PDD). This dataset includes a wide variety of diseases and pests images, along with detailed natural language descriptions of their attributes. Locating the described targets in complex scenes with similar disease symptoms and leaf occlusion presents a significant challenge. To tackle this issue, we propose the PepperNet model for object detection in pepper diseases and pests images using natural language descriptions. This model decomposes complex multimodal features of language and images into explicit attribute features and employs fine-grained multimodal attribute contrast learning strategies. This approach effectively distinguishes subtle local differences between similar objects, achieving fine-grained mapping from language to vision in complex scenarios. Our detection results show a mAP@0.5 of 91.93% and a detection speed of 121.8 frames per second. Visualizations indicate that the model maintains high robustness under varying noise levels and occlusion conditions, demonstrating superior performance and stability across diverse complex scenarios.

BG-YOLO: A Bidirectional-Guided Method for Underwater Object Detection.

Degraded underwater images decrease the accuracy of underwater object detection. Existing research uses image enhancement methods to improve the visual quality of images, which may not be beneficial in underwater image detection and lead to serious degradation in detector performance. To alleviate this problem, we proposed a bidirectional guided method for underwater object detection, referred to as BG-YOLO. In the proposed method, a network is organized by constructing an image enhancement branch and an object detection branch in a parallel manner. The image enhancement branch consists of a cascade of an image enhancement subnet and object detection subnet. The object detection branch only consists of a detection subnet. A feature-guided module connects the shallow convolution layers of the two branches. When training the image enhancement branch, the object detection subnet in the enhancement branch guides the image enhancement subnet to be optimized towards the direction that is most conducive to the detection task. The shallow feature map of the trained image enhancement branch is output to the feature-guided module, constraining the optimization of the object detection branch through consistency loss and prompting the object detection branch to learn more detailed information about the objects. This enhances the detection performance. During the detection tasks, only the object detection branch is reserved so that no additional computational cost is introduced. Extensive experiments demonstrate that the proposed method significantly improves the detection performance of the YOLOv5s object detection network (the mAP is increased by up to 2.9%) and maintains the same inference speed as YOLOv5s (132 fps).