Object Localization Task Research Articles

Weakly Supervised Underwater Object Real-time Detection Based on High-resolution Attention Class Activation Mapping and Category Hierarchy

Recently, deep learning-based underwater object detection technology has achieved remarkable success. However, the accuracy and completeness of dataset instance annotation are crucial for its success. The quality of underwater images is low, severe objects clustering, and occlusion, acquiring object's annotations demands substantial time and labor costs, while mis annotation and missed annotation can also degrade model performance and limit their application in practical scenarios. To address this issue, this paper presents a novel weakly supervised underwater object real-time detection method, which is divided into two subtasks: weakly supervised object localization and real-time object detection. In the weakly supervised object localization task, we design a novel category hierarchy structure network that integrates the high-resolution attention-class activation mapping algorithm to obtain high-quality object class activation maps, weaken background interference, and obtain more complete object regions. The parameterized spatial loss module is devised to enable the model to escape from local optimal solutions, thus accurately and efficiently obtaining object pseudo-detection annotation boxes. For the real-time object detection task, the single-stage detector YOLOv7 is selected as the basic detection model, and an object perception loss function is designed based on the class activation map to jointly supervise the training process. A method for filtering noisy pseudo-supervision information is proposed to enhance the pseudo-supervision information involved in training. Ablation experiments and multi-method comparison experiments were conducted on the URPC and RUOD datasets, and the results verify the effectiveness of the proposed strategy, and our model exhibits significant advantages in detection performance and detection efficiency compared to current mainstream and advanced models.

INTEGRATION OF RESUNET AND YOLO ALGORITHMS INTO A UNIFIED MODEL FOR OBJECTS DETECTION

<p>Automatic extraction of footprints of buildings from orthophotos is a challenge in the field of remote sensing data processing. The combination of image classification and object localization tasks in this research aims to develop a new model based on ResUNet and the YOLO algorithm. By applying the proposed model and publicly available data, a high level of building extraction success of 89% is achieved. Although there is potential to improve the results by introducing other types of data, the integration of these models represents a significant step towards improving the technology of automatic extraction of buildings from orthophotos.</p>

PCSformer: Pair-wise Cross-scale Sub-prototypes mining with CNN-transformers for weakly supervised semantic segmentation

Generating initial seeds is an important step in weakly supervised semantic segmentation (WSSS). Our approach concentrates on generating and refining initial seeds. The convolutional neural networks (CNNs)–based initial seeds focus only on the most discriminative regions and lack global information about the target. The Vision Transformer (ViT)–based approach can capture long-range feature dependencies due to the unique advantage of the self-attention mechanism. Still, we find that it suffers from distractor object leakage and background leakage problems. Based on these observations, we propose PCSformer, which improves the model’s ability to extract features through a Pair-wise Cross-scale (PC) strategy and solves the problem of distractor object leakage by further extracting potential target features through Sub-Prototypes (SP) mining. In addition, the proposed Conflict Self-Elimination (CSE) module further alleviates the background leakage problem. We validate our approach on the widely adopted Pascal VOC 2012 and MS COCO 2014, and extensive experiments demonstrate our superior performance. Furthermore, our method proves to be competitive for WSSS in medical images and challenging scenarios involving deformable and cluttered scenes. Additionally, we extend the PCSformer to weakly supervised object localization tasks, further highlighting its scalability and versatility. The code is available at https://github.com/ChunmengLiu1/PCSformer.

Weakly Supervised Open-Vocabulary Object Detection

Despite weakly supervised object detection (WSOD) being a promising step toward evading strong instance-level annotations, its capability is confined to closed-set categories within a single training dataset. In this paper, we propose a novel weakly supervised open-vocabulary object detection framework, namely WSOVOD, to extend traditional WSOD to detect novel concepts and utilize diverse datasets with only image-level annotations. To achieve this, we explore three vital strategies, including dataset-level feature adaptation, image-level salient object localization, and region-level vision-language alignment. First, we perform data-aware feature extraction to produce an input-conditional coefficient, which is leveraged into dataset attribute prototypes to identify dataset bias and help achieve cross-dataset generalization. Second, a customized location-oriented weakly supervised region proposal network is proposed to utilize high-level semantic layouts from the category-agnostic segment anything model to distinguish object boundaries. Lastly, we introduce a proposal-concept synchronized multiple-instance network, i.e., object mining and refinement with visual-semantic alignment, to discover objects matched to the text embeddings of concepts. Extensive experiments on Pascal VOC and MS COCO demonstrate that the proposed WSOVOD achieves new state-of-the-art compared with previous WSOD methods in both close-set object localization and detection tasks. Meanwhile, WSOVOD enables cross-dataset and open-vocabulary learning to achieve on-par or even better performance than well-established fully-supervised open-vocabulary object detection (FSOVOD).

Module of Axis-based Nexus Attention for weakly supervised object localization

Weakly supervised object localization tasks remain challenging to identify and segment an entire object rather than only discriminative parts of the object. To tackle this problem, corruption-based approaches have been devised, which involve the training of non-discriminative regions by corrupting (e.g., erasing) the input images or intermediate feature maps. However, this approach requires an additional hyperparameter, the corrupting threshold, to determine the degree of corruption and can unfavorably disrupt training. It also tends to localize object regions coarsely. In this paper, we propose a novel approach, Module of Axis-based Nexus Attention (MoANA), which helps to adaptively activate less discriminative regions along with the class-discriminative regions without an additional hyperparameter, and elaborately localizes an entire object. Specifically, MoANA consists of three mechanisms (1) triple-view attentions representation, (2) attentions expansion, and (3) features calibration mechanism. Unlike other attention-based methods that train a coarse attention map with the same values across elements in feature maps, MoANA trains fine-grained values in an attention map by assigning different attention values to each element. We validated MoANA by comparing it with various methods. We also analyzed the effect of each component in MoANA and visualized attention maps to provide insights into the calibration.

Data Generation Using Simulation Technology to Improve Perception Mechanism of Autonomous Vehicles

Recent advancements in computer graphics technology allow more realistic rendering of car driving environments. They have enabled self-driving car simulators such as DeepGTA-V and CARLA (Car Learning to Act) to generate large amounts of synthetic data that can complement the existing real-world dataset in training autonomous car perception. Furthermore, since self-driving car simulators allow full control of the environments, they can generate dangerous driving scenarios that the real-world dataset lacks such as bad weather and accident scenarios. In this paper, we will demonstrate the effectiveness of combining data gathered from the real-world with data generated in simulated world to train perception system on object detection and localization task. We will also propose a multi-level deep learning perception framework that aims to emulate a human learning experience in which series of tasks from simple to more difficult ones are learnt in a certain domain. The autonomous car perceptron can learn from easy-to-drive scenarios to more challenging ones customized by simulation software.

Object Identification Using Deep Learning

Human visual system is very fast that he can easily detect and identify the objects using this visual system it can easily identify multiple objects and detect the obstacle. This time there is a very large amount of data, a faster CPU and good algorithm that can easily train computers to detect and classify multiple objects with high accuracy. It uses an algorithm to detect and identify the objects that is “You Only Look Once” (YOLO) algorithm. This project is based on a deep learning approach to solve the problem of object detection and recognition. This network is trained on the most challenging dataset known as (PASCAL VOC), in which object detection is conducted annually manner. The resulting system is very fast and accurate. Object recognition is a collection of related tasks for identifying objects in digital photograph. R-CNN is a family of techniques for addressing object localization and recognition tasks. Propose a new object detection method which improves the existing model at every stage of object detection and identification, use COCO dataset model in this lot of categories of images are stored.

Sonified Distance in Sensory Substitution Does Not Always Improve Localization: Comparison With a 2-D and 3-D Handheld Device

Early visual to auditory substitution devices encode 2-D monocular images into sounds while more recent devices use distance information from 3-D sensors. This study assesses whether the addition of sound-encoded distance in recent systems helps to convey the “where” information. This is important to the design of new sensory substitution devices. We conducted experiments for object localization and navigation tasks with a handheld visual to audio substitution system. It comprises 2-D and 3-D modes. Both encode in real-time the position of objects in images captured by a camera. The 3-D mode encodes in addition the distance between the system and the object. Experiments have been conducted with 16 blindfolded sighted participants. For the localization, participants were quicker to understand the scene with the 3-D mode that encodes distances. On the other hand, with the 2-D only mode, they were able to compensate for the lack of distance encoding after a small training. For the navigation, participants were as good with the 2-D only mode than with the 3-D mode encoding distance.

Identifying and Localizing Multiple Objects Using Artificial Ventral and Dorsal Cortical Visual Pathways.

In our previous study (Han & Sereno, 2022a), we found that two artificial cortical visual pathways trained for either identity or space actively retain information about both identity and space independently and differently. We also found that this independently and differently retained information about identity and space in two separate pathways may be necessary to accurately and optimally recognize and localize objects. One limitation of our previous study was that there was only one object in each visual image, whereas in reality, there may be multiple objects in a scene. In this study, we find we are able to generalize our findings to object recognition and localization tasks where multiple objects are present in each visual image. We constrain the binding problem by training the identity network pathway to report the identities of objects in a given order according to the relative spatial relationships between the objects, given that most visual cortical areas including high-level ventral steam areas retain spatial information. Under these conditions, we find that the artificial neural networks with two pathways for identity and space have better performance in multiple-objects recognition and localization tasks (higher average testing accuracy, lower testing accuracy variance, less training time) than the artificial neural networks with a single pathway. We also find that the required number of training samples and the required training time increase quickly, and potentially exponentially, when the number of objects in each image increases, and we suggest that binding information from multiple objects simultaneously within any network (cortical area) induces conflict or competition and may be part of the reason why our brain has limited attentional and visual working memory capacities.

Boundary distribution estimation for precise object detection

<abstract><p>In the field of state-of-the-art object detection, the task of object localization is typically accomplished through a dedicated subnet that emphasizes bounding box regression. This subnet traditionally predicts the object's position by regressing the box's center position and scaling factors. Despite the widespread adoption of this approach, we have observed that the localization results often suffer from defects, leading to unsatisfactory detector performance. In this paper, we address the shortcomings of previous methods through theoretical analysis and experimental verification and present an innovative solution for precise object detection. Instead of solely focusing on the object's center and size, our approach enhances the accuracy of bounding box localization by refining the box edges based on the estimated distribution at the object's boundary. Experimental results demonstrate the potential and generalizability of our proposed method.</p></abstract>

Adaptively Attentional Feature Fusion Oriented to Multiscale Object Detection in Remote Sensing Images

Currently, reliable and accurate object detection in high-resolution remote sensing images still faces significant challenges such as color, aspect ratio, complex background, and scale variations. Even the detection results obtained based on the latest convolutional neural network (CNN) methods are not satisfactory. To obtain more accurate detection results from large-scale remote sensing images, we proposed a multiscale object detection algorithm oriented to adaptively attentional feature fusion based on the YOLOX algorithm. Firstly, a multiscale attentional feature fusion (MSAFF) structure was added to the YOLOX network to increase the perceptual field and aggregate the contextual information to obtain a feature map with richer semantic information. Secondly, an adaptively spatial feature fusion (ASFF) structure was introduced to process the fused feature maps, to which spatial feature weights were assigned at different levels to enhance the feature representation of remote sensing objects and reduce feature loss. Finally, the aligned convolutional network was used in the object classification and localization task to achieve a more accurate localization of densely arranged objects with arbitrary orientation. The algorithm proposed in this paper was extensively experimented on the PASCAL VOC and the DIOR datasets, and the average accuracy reached 89.2% and 75.3%, respectively. Compared with some current mainstream two-stage and one-stage object detection algorithms, the experimental results demonstrated that our method performed well in both accuracy and speed. At the same time, it reduced the leakage rate of remote sensing objects to a certain extent.

Learning Consistency From High-Confidence Pseudo-Labels for Weakly Supervised Object Localization

Weakly supervised object localization (WSOL) tasks aim to classify and locate a single object under the supervision of only image-level labels. Pseudo-supervised learning methods have been shown to be effective for WSOL, which divided WSOL tasks into two decoupled subtasks: classification and localization. The decoupled framework has been proven to be effective in improving the performance of the localization subtask, but the predicted localizations are not robust enough due to the noise of pseudo-labels. Based on the assumption that the localization model should have similar predictions on different versions of the same image, we propose an additional refinement stage to learn more consistent localization. Specifically, in the refinement stage, we propose a simple and effective method for evaluating the confidence of pseudo-labels based on classification discrimination, and by learning consistency from high-confidence pseudo-labels, we further refine the localization model to get better localization performance. Besides, in the initialization stage, we propose a mask-based pseudo-label generator to initialize the localization model. We conduct experiments on two benchmark datasets: CUB-200-2011 and ImageNet-1k. Experimental results show that our two-stage approach achieves 94.01% <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">GT-Konwn</i> localization accuracy on the CUB-200-2011 testing dataset, and 65.23% <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">GT-Konwn</i> localization accuracy on the ImageNet-1k validation dataset. Moreover, when directly applied to the pseudo-supervised localization model, our refinement stage could achieve 94.05% and 67.13% <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">GT-Konwn</i> localization accuracy on CUB-200-2011 and ImageNet-1k datasets, respectively, which outperforms the corresponding pseudo-supervised localization model with 3.34% and 2.34% accuracy.

SPA2Net: Structure-Preserved Attention Activated Network for Weakly Supervised Object Localization.

By exploring the localizable representations in deep CNN, weakly supervised object localization (WSOL) methods could determine the position of the object in each image just trained by the classification task. However, the partial activation problem caused by the discriminant function makes the network unable to locate objects accurately. To alleviate this problem, we propose Structure-Preserved Attention Activated Network (SPA2Net), a simple and effective one-stage WSOL framework to explore the ability of structure preservation of deep features. Different from traditional WSOL approaches, we decouple the object localization task from the classification branch to reduce their mutual influence by involving a localization branch which is online refined by a self-supervised structural-preserved localization mask. Specifically, we employ the high-order self-correlation as structural prior to enhance the perception of spatial interaction within convolutional features. By succinctly combining the structural prior with spatial attention, activations by SPA2Net will spread from part to the whole object during training. To avoid the structure-missing issue caused by the classification network, we furthermore utilize the restricted activation loss (RAL) to distinguish the difference between foreground and background in the channel dimension. In conjunction with the self-supervised localization branch, SPA2Net can directly predict the class-irrelevant localization map while prompting the network to pay more attention to the target region for accurate localization. Extensive experiments on two publicly available benchmarks, including CUB-200-2011 and ILSVRC, show that our SPA2Net achieves substantial and consistent performance gains compared with baseline approaches. The code and models are available at https://github.com/MsterDC/SPA2Net.

Transformation of primary sensory cortical representations from layer 4 to layer 2

Sensory input arrives from thalamus in cortical layer (L) 4, which outputs predominantly to superficial layers. L4 to L2 thus constitutes one of the earliest cortical feedforward networks. Despite extensive study, the transformation performed by this network remains poorly understood. We use two-photon calcium imaging to record neural activity in L2-4 of primary vibrissal somatosensory cortex (vS1) as mice perform an object localization task with two whiskers. Touch responses sparsen and become more reliable from L4 to L2, with nearly half of the superficial touch response confined to ~1 % of excitatory neurons. These highly responsive neurons have broad receptive fields and can more accurately decode stimulus features. They participate disproportionately in ensembles, small subnetworks with elevated pairwise correlations. Thus, from L4 to L2, cortex transitions from distributed probabilistic coding to sparse and robust ensemble-based coding, resulting in more efficient and accurate representations.

Deep Learning Approaches for Automatic Localization in Medical Images.

Recent revolutionary advances in deep learning (DL) have fueled several breakthrough achievements in various complicated computer vision tasks. The remarkable successes and achievements started in 2012 when deep learning neural networks (DNNs) outperformed the shallow machine learning models on a number of significant benchmarks. Significant advances were made in computer vision by conducting very complex image interpretation tasks with outstanding accuracy. These achievements have shown great promise in a wide variety of fields, especially in medical image analysis by creating opportunities to diagnose and treat diseases earlier. In recent years, the application of the DNN for object localization has gained the attention of researchers due to its success over conventional methods, especially in object localization. As this has become a very broad and rapidly growing field, this study presents a short review of DNN implementation for medical images and validates its efficacy on benchmarks. This study presents the first review that focuses on object localization using the DNN in medical images. The key aim of this study was to summarize the recent studies based on the DNN for medical image localization and to highlight the research gaps that can provide worthwhile ideas to shape future research related to object localization tasks. It starts with an overview on the importance of medical image analysis and existing technology in this space. The discussion then proceeds to the dominant DNN utilized in the current literature. Finally, we conclude by discussing the challenges associated with the application of the DNN for medical image localization which can drive further studies in identifying potential future developments in the relevant field of study.

Comparative study of 3D object detection frameworks based on LiDAR data and sensor fusion techniques

Estimating and understanding the surroundings of the vehicle precisely forms the basic and crucial step for the autonomous vehicle. The perception system plays a significant role in providing an accurate interpretation of a vehicle’s environment in real-time. Generally, the perception system involves various subsystems such as localization, obstacle (static and dynamic) detection, and avoidance, mapping systems, and others. For perceiving the environment, these vehicles will be equipped with various exteroceptive (both passive and active) sensors in particular cameras, Radars, LiDARs, and others. These systems are equipped with deep learning techniques that transform the huge amount of data from the sensors into semantic information on which the object detection and localization tasks are performed. For numerous driving tasks, to provide accurate results, the location and depth information of a particular object is necessary. 3D object detection methods, by utilizing the additional pose data from the sensors such as LiDARs, stereo cameras, provides information on the size and location of the object. Based on recent research, 3D object detection frameworks performing object detection and localization on LiDAR data and sensor fusion techniques show significant improvement in their performance. In this work, a comparative study of the effect of using LiDAR data for object detection frameworks and the performance improvement seen by using sensor fusion techniques are performed. Along with discussing various state-of-the-art methods in both the cases, performing experimental analysis, and providing future research directions.

Weakly Supervised Learning for Object Localization Based on an Attention Mechanism

Recently, deep learning has been successfully applied to object detection and localization tasks in images. When setting up deep learning frameworks for supervised training with large datasets, strongly labeling the objects facilitates good performance; however, the complexity of the image scene and large size of the dataset make this a laborious task. Hence, it is of paramount importance that the expensive work associated with the tasks involving strong labeling, such as bounding box annotation, is reduced. In this paper, we propose a method to perform object localization tasks without bounding box annotation in the training process by means of employing a two-path activation-map-based classifier framework. In particular, we develop an activation-map-based framework to judicially control the attention map in the perception branch by adding a two-feature extractor so that better attention weights can be distributed to induce improved performance. The experimental results indicate that our method surpasses the performance of the existing deep learning models based on weakly supervised object localization. The experimental results show that the proposed method achieves the best performance, with 75.21% Top-1 classification accuracy and 55.15% Top-1 localization accuracy on the CUB-200-2011 dataset.

Contrastive and consistent feature learning for weakly supervised object localization and semantic segmentation

Weakly supervised learning attempts to construct predictive models by learning with weak supervision. In this paper, we concentrate on weakly supervised object localization and semantic segmentation tasks. Existing methods are limited to focusing on narrow discriminative parts or overextending the activations to less discriminative regions even on backgrounds. To mitigate these problems, we regard the background as an important cue that guides the feature activation to cover the entire object to the right extent, and propose two novel objective functions: 1) contrastive attention loss and 2) foreground consistency loss. Contrastive attention loss draws the foreground feature and its dropped version close together and pushes the dropped foreground feature away from the background feature. Foreground consistency loss favors agreement between layers and provides early layers with a sense of objectness. Using both losses leads to balanced improvements over localization and segmentation accuracy by boosting activations on less discriminative regions but restraining the activation in the target object extent. For better optimizing the above losses, we use the non-local attention blocks to replace channel-pooled attention leading to enhanced attention maps considering the spatial similarity. Finally, our method achieves state-of-the-art localization performance on CUB-200-2011, ImageNet, and OpenImages benchmarks regarding top-1 localization accuracy, MaxBoxAccV2, and PxAP. We also demonstrate the effectiveness of our method in improving segmentation performance measured by mIoU on the PASCAL VOC dataset.

Point Cloud Inversion: A Novel Approach for the Localization of Trees in Forests from TLS Data

Tree localization in point clouds of forest scenes is critical in the forest inventory. Most of the existing methods proposed for TLS forest data are based on model fitting or point-wise features which are time-consuming, sensitive to data incompleteness and complex tree structures. Furthermore, these methods often require lots of preprocessing such as ground filtering and noise removal. The fast and easy-to-use top-based methods that are widely applied in processing ALS point clouds are not applicable in localizing trees in TLS point clouds due to the data incompleteness and complex canopy structures. The objective of this study is to make the top-based methods applicable to TLS forest point clouds. To this end, a novel point cloud transformation is presented, which enhances the visual salience of tree instances and makes the top-based methods adapting to TLS forest scenes. The input for the proposed method is the raw point clouds and no other pre-processing steps are needed. The new method is tested on an international benchmark and the experimental results demonstrate its necessity and effectiveness. Finally, the proposed method has the potential to benefit other object localization tasks in different scenes based on detailed analysis and tests.

Vision-based robotic grasping from object localization, object pose estimation to grasp estimation for parallel grippers: a review

This paper presents a comprehensive survey on vision-based robotic grasping. We conclude three key tasks during vision-based robotic grasping, which are object localization, object pose estimation and grasp estimation. In detail, the object localization task contains object localization without classification, object detection and object instance segmentation. This task provides the regions of the target object in the input data. The object pose estimation task mainly refers to estimating the 6D object pose and includes correspondence-based methods, template-based methods and voting-based methods, which affords the generation of grasp poses for known objects. The grasp estimation task includes 2D planar grasp methods and 6DoF grasp methods, where the former is constrained to grasp from one direction. These three tasks could accomplish the robotic grasping with different combinations. Lots of object pose estimation methods need not object localization, and they conduct object localization and object pose estimation jointly. Lots of grasp estimation methods need not object localization and object pose estimation, and they conduct grasp estimation in an end-to-end manner. Both traditional methods and latest deep learning-based methods based on the RGB-D image inputs are reviewed elaborately in this survey. Related datasets and comparisons between state-of-the-art methods are summarized as well. In addition, challenges about vision-based robotic grasping and future directions in addressing these challenges are also pointed out.