Object Localization Research Articles

MonoFG: Monocular 3D Object Detection with Knowledge Distillation for Human-Centric Autonomous Driving Systems

Monocular 3D object detection is essential for identifying objects in road images, thus offering valuable environmental perception data that are crucial for human-centric autonomous driving systems. However, due to the inherent limitations of camera imaging, obtaining precise depth information from images alone is challenging, which hampers the accuracy of within-scene object localization. In this paper, we introduce a monocular 3D object detection method called MonoFG that uses knowledge distillation with separated foreground and background components to improve the accuracy of object localization. First, detached foreground and background distillation processes can strategically leverage the distinct positional information acquired from each location to optimize the produced global distillation effects. This step serves as the foundation for the subsequent feature and response distillation process, which focuses on the distilled foreground and background rather than isolated object distillation. Second, triple attention mechanism-based feature distillation intensifies the feature imitation and feature representation capabilities of the student network. Spatial and channel attention mechanisms encourage the student network to capture crucial pixels and channels from the teacher network, whereas a self-attention mechanism globally transfers the learned relationships between pixels. Third, localization error-based response distillation facilitates a clearer transfer of positional information from the teacher network to the student network. Only when the positioning ability of the teacher network exceeds that of the student network can knowledge be comprehensively distilled across both the foreground and background. Therefore, the distillation process is constrained to specific content, which is delineated by positioning errors that serve as the boundaries. Finally, experiments conducted on the KITTI benchmark dataset demonstrate that our method outperforms many well-known baseline methods in several representative evaluation tasks (e.g., 3D object detection and bird's-eye view (BEV) detection) involving human-centric autonomous driving systems.

CBCT Guidance for Removing Foreign Object from the Jawbone: A Case Study

AbstractForeign objects lodged in the mandibular bone often result from trauma or dental procedures. Accurate localization of these foreign objects is crucial for guided surgical removal, especially when patients present with persistent pain. Conventional imaging modalities like periapical and panoramic radiographs may not provide sufficient detail for precise localization. Cone beam computed tomography (CBCT) has emerged as a valuable tool for guided surgical interventions owing to its superior imaging capabilities. We present two cases of foreign objects lodged in the mandible, where patients complained of prolonged pain, soreness, and numbness in the right mandible extending to the head and behind the ear. Both patients had undergone previous right mandibular tooth extractions and received treatment from a neurologist without resolution of symptoms. Both cases showed the control improvement in complaints, and no paresthesia or postoperative complications were found. CBCT-guided surgical removal was performed in both cases, revealing a metal specimen measuring 6 × 3 × 1 mm3 in the first patient and a remaining root measuring 5 × 3 × 2 mm3 in the second patient. Diagnosing foreign objects in the mandible poses challenges due to their varied size, composition, and proximity to vital structures. CBCT offers superior imaging resolution, enabling precise localization and assessment of anatomical relationships, such as the distance to the inferior alveolar nerve and surrounding boundaries. CBCT emerges as the preferred imaging modality for diagnosing and guiding the surgical removal of foreign objects in the mandible. Its advantages include accurate localization, low radiation exposure, and cost-effectiveness. Compared with CT scans, CBCT also offers faster scanning times, making it a valuable tool in clinical practice for managing such cases

Study of conveyor belt deviation detection based on improved YOLOv8 algorithm

Conveyor belt deviation is a commmon and severe type of fault in belt conveyor systems, often resulting in significant economic losses and potential environment pollution. Traditional detection methods have obvious limitations in fault localization precision and analysis accuracy, unable to meet the demands of efficient and real-time fault detection in complex industrial scenarios. To address these issues, this paper proposes an improved detection algorithm based on YOLOv8, aiming to achieve efficient and accurate detection during the operation of the belt. Firstly, an Enhanced Squeeze-and-Excitation (ESE) module is incorporated into C2f to boost feature extraction for rollers and belts. Secondly, the construction of the BiFPN_DoubleAttention module in the neck network enhances bidirectional feature fusion and attention mechanism, thereby improving multi-scale object localization accuracy under complex environments. Then, a Multi-Head Self-Attention (MHSA) mechanism is introduced in the head network, better capturing positional features of small roller targets and belt areas in various environments, thus enhancing detection performance. Finally, extensive experiments are conducted on a self-built dataset, achieving an accuracy of 98.1%, mAP0.5 of 99.0%, and a detection speed of 46 frames per second (FPS). This method effectively handles variations and disturbances in the conveyor belt transportation environment, meeting real-time diagnostic needs for belt misalignment in the industry.

Using spatial video and deep learning for automated mapping of ground-level context in relief camps

BackgroundThe creation of relief camps following a disaster, conflict or other form of externality often generates additional health problems. The density of people in a highly stressed environment with questionable safe food and water access presents the potential for infectious disease outbreaks. These camps are also not static data events but rather fluctuate in size, composition, and level and quality of service provision. While contextualized geospatial data collection and mapping are vital for understanding the nature of these camps, various challenges, including a lack of data at the required spatial or temporal granularity, as well as the issue of sustainability, can act as major impediments. Here, we present the first steps toward a deep learning-based solution for dynamic mapping using spatial video (SV).MethodsWe trained a convolutional neural network (CNN) model on a SV dataset collected from Goma, Democratic Republic of Congo (DRC) to identify relief camps from video imagery. We developed a spatial filtering approach to tackle the challenges associated with spatially tagging objects such as the accuracy of global positioning system and positioning of camera. The spatial filtering approach generates smooth surfaces of detection, which can further be used to capture changes in microenvironments by applying techniques such as raster math.ResultsThe initial results suggest that our model can detect temporary physical dwellings from SV imagery with a high level of precision, recall, and object localization. The spatial filtering approach helps to identify areas with higher concentrations of camps and the web-based tool helps to explore these areas. The longitudinal analysis based on applying raster math on the detection surfaces revealed locations, which had a considerable change in the distribution of tents over space and time.ConclusionsThe results lay the groundwork for automated mapping of spatial features from imagery data. We anticipate that this work is the building block for a future combination of SV, object identification and automatic mapping that could provide sustainable data generation possibilities for challenging environments such as relief camps or other informal settlements.

Knowledge-Guided Causal Intervention for Weakly-Supervised Object Localization

Knowledge-Guided Causal Intervention for Weakly-Supervised Object Localization

Object Localization in Highly Cluttered Environments Using Neural Network Learning on Microwave Scattering Data

ABSTRACTBifocusing‐based microwave imaging is a promising direct imaging method for object localization in various applications. However, the method has the limitation that it only works well in environments with a homogenous background. In this paper, we present a direct microwave imaging method using a neural network (NN) model that can localize a small object even in highly cluttered environments with strong scatterers. The NN model is trained on some data sets obtained through multistatic measurements in a nonhomogeneous environment. To verify the approach, we prepared an experimental testbed equipped with a water tank containing 3 strong scatterers and 16 antennas to obtain 920 MHz multistatic scattering data. This experiment shows good localization performance, with an average localization error of approximately 4 mm (1/9 of a wavelength) over the entire experimental area, even in a strong scattering background.

MSSA: Multi-Representation Semantics-Augmented Set Abstraction for 3D Object Detection

Accurate recognition and localization of 3D objects is a fundamental research problem in 3D computer vision. Benefiting from transformation-free point cloud processing and flexible receptive fields, point-based methods have become accurate in 3D point cloud modeling, but still fall behind voxel-based competitors in 3D detection. We observe that the set abstraction module, commonly utilized by point-based methods for downsampling points, tends to retain excessive irrelevant background information, thus hindering the effective learning of features for object detection tasks. To address this issue, we propose MSSA, a Multi-representation Semantics-augmented Set Abstraction for 3D object detection. Specifically, we first design a backbone network to encode different representation features of point clouds, which extracts point-wise features through PointNet to preserve fine-grained geometric structure features, and adopts VoxelNet to extract voxel features and BEV features to enhance the semantic features of key points. Second, to efficiently fuse different representation features of keypoints, we propose a Point feature-guided Voxel feature and BEV feature fusion (PVB-Fusion) module to adaptively fuse multi-representation features and remove noise. At last, a novel Multi-representation Semantic-guided Farthest Point Sampling (MS-FPS) algorithm is designed to help set abstraction modules progressively downsample point clouds, thereby improving instance recall and detection performance with more important foreground points. We evaluate MSSA on the widely used KITTI dataset and the more challenging nuScenes dataset. Experimental results show that compared to PointRCNN, our method improves the AP of “moderate” level for three classes of objects by 7.02%, 6.76%, and 5.44%, respectively. Compared to the advanced point-voxel-based method PV-RCNN, our method improves the AP of “moderate” level by 1.23%, 2.84%, and 0.55% for the three classes, respectively.

Detection and interpretation of central type structures within the territory of southeastern Transbaikalia for prediction of ore-forming systems

Extremely little attention is paid to the issues of detecting and interpreting of central type structures (CTS) when conducting remote structural-geological and structural-geomorphological studies. At the same time, in the 70–80s of the 20th century, the important role of CTSs in the localization of deposits and ore fields was proven. The position of these structures must necessarily be taken into account when solving problems of searching for and predicting mineral resources in the context of metallogenic analysis and reconstruction of the geological history of development of the studied areas. The almost absence of results of mass detecting and interpreting of CTSs can be explained by the still poorly developed methodology for identifying and analyzing this type of structure. In the present study for the territory of southeastern Transbaikalia, based on modern geoinformation technologies, the use of remote sensing data (radar topographic survey) of high resolution, the creation of a digital elevation model and the application of an integrated structural-spatial analysis, an author’s approach to detection and interpretation of the CTSs is presented, including in connection with the localization of ore objects of various geological-industrial (geological-genetic) types within the framework of the concept of the formation of mineral systems. A statistical analysis of the CTSs identified in the area was carried out, which made it possible to establish a smooth increase in the number of structures with a decrease in their diameter. It is shown that the spatial maxima of ore mineralization extent within the territory are concentrated on the periphery of large CTSs and in their immediate vicinity. Most of the known large ore objects are confined to the internal areas of structures less than 10 km in diameter. Based on the approach of constructing model sections, it was possible to reconstruct the deep position of magma chambers associated with the identified CTSs, and, thereby, to determine the probable sources of metal-bearing fluids. A close spatial relationship between the identified magma chambers and deep faults has been established. To determine the most favorable sites for the deposition of ore mineralization, based on structural-spatial criteria, which include not only structural elements of the CTSs, but also segments of known fault structures, weight of evidence models of the territory have been created. The accuracy of the complex model is 89%. Thus, in accordance with the concept of mineral systems, the sources, migration pathways and sites of the most probable deposition of ore mineralization have been reconstructed.

Multi-layer feature fusion and attention enhancement for fine-grained vehicle recognition research

Abstract Vehicle recognition technology is widely applied in automatic parking, traffic restrictions, and public security investigations, playing a significant role in the construction of intelligent transportation systems. Fine-grained vehicle recognition seeks to surpass conventional vehicle recognition by concentrating on more detailed sub-classifications. This task is more challenging due to the subtle inter-class differences and significant intra-class variations. Localization-classification subnetworks represent an efficacious approach frequently employed for this task, but previous research has typically relied on CNN deep feature maps for object localization, which suffer from the low resolution, leading to poor localization accuracy. The multi-layer feature fusion localization method proposed by us fuses the high-resolution feature map of the shallow layer of CNN with the deep feature map, and makes full use of the rich spatial information of the shallow feature map to achieve more precise object localization. In addition, traditional methods acquire local attention information through the design of complex models, frequently resulting in regional redundancy or information omission. To address this, we introduce an attention module that adaptively enhances the expressiveness of global features and generates global attention features. These global attention features are then integrated with object-level features and local attention cues to achieve a more comprehensive attention enhancement. Lastly, we devise a multi-branch model and employ the aforementioned object localization and attention enhancement methods for end-to-end training to make the multiple branches collaborate seamlessly to adequately extract fine-grained features. Extensive experiments conducted on the Stanford Cars dataset and the self-built Cars-126 dataset have demonstrated the effectiveness of our method, achieving a leading position among existing methods with 97.7% classification accuracy on the Stanford Cars dataset.

A Real-Time Semantic Map Production System for Indoor Robot Navigation.

Although grid maps help mobile robots navigate in indoor environments, some lack semantic information that would allow the robot to perform advanced autonomous tasks. In this paper, a semantic map production system is proposed to facilitate indoor mobile robot navigation tasks. The developed system is based on the employment of LiDAR technology and a vision-based system to obtain a semantic map with rich information, and it has been validated using the robot operating system (ROS) and you only look once (YOLO) v3 object detection model in simulation experiments conducted in indoor environments, adopting low-cost, -size, and -memory computers for increased accessibility. The obtained results are efficient in terms of object recognition accuracy, object localization error, and semantic map production precision, with an average map construction accuracy of 78.86%.

Seal Pipeline: Enhancing Dynamic Object Detection and Tracking for Autonomous Unmanned Surface Vehicles in Maritime Environments

This study addresses the dynamic object detection problem for Unmanned Surface Vehicles (USVs) in marine environments, which is complicated by boat tilting and camera illumination sensitivity. A novel pipeline named “Seal” is proposed to enhance detection accuracy and reliability. The approach begins with an innovative preprocessing stage that integrates data from the Inertial Measurement Unit (IMU) with LiDAR sensors to correct tilt-induced distortions in LiDAR point cloud data and reduce ripple effects around objects. The adjusted data are grouped using clustering algorithms and bounding boxes for precise object localization. Additionally, a specialized Kalman filter tailored for maritime environments mitigates object discontinuities between successive frames and addresses data sparsity caused by boat tilting. The methodology was evaluated using the VRX simulator, with experiments conducted on the Volga River using real USVs. The preprocessing effectiveness was assessed using the Root Mean Square Error (RMSE) and tracking accuracy was evaluated through detection rate metrics. The results demonstrate a 25% to 30% improvement in detection accuracy and show that the pipeline can aid industry even with sparse object representation across different frames. This study highlights the potential of integrating sensor fusion with specialized tracking for accurate dynamic object detection in maritime settings, establishing a new benchmark for USV navigation systems’ accuracy and reliability.

Evaluating the relationship between high-frequency hearing loss and front-back localization in the elderly

Sound localization is an essential ability for normal hearing listeners, aiding in the localization of objects and the detection of potential dangers, as well as enabling the extraction of target speech information in a noisy environment. However, high-frequency hearing loss, commonly observed in the elderly population, may lead to a decline in spatial localization ability. The present work evaluated the ability of horizontal localization of 60 elderly individuals in a psychoacoustic experiment and investigated the relationship between localization impairment and hearing loss in different frequency ranges. Results indicate that high-frequency hearing loss above 2 kHz has a minor impact on left-right localization, but significantly impairs front-back localization. The physical and psychoacoustic explanations are provided for the experimental observations.

Weakly Supervised Underwater Object Real-time Detection Based on High-resolution Attention Class Activation Mapping and Category Hierarchy

Recently, deep learning-based underwater object detection technology has achieved remarkable success. However, the accuracy and completeness of dataset instance annotation are crucial for its success. The quality of underwater images is low, severe objects clustering, and occlusion, acquiring object's annotations demands substantial time and labor costs, while mis annotation and missed annotation can also degrade model performance and limit their application in practical scenarios. To address this issue, this paper presents a novel weakly supervised underwater object real-time detection method, which is divided into two subtasks: weakly supervised object localization and real-time object detection. In the weakly supervised object localization task, we design a novel category hierarchy structure network that integrates the high-resolution attention-class activation mapping algorithm to obtain high-quality object class activation maps, weaken background interference, and obtain more complete object regions. The parameterized spatial loss module is devised to enable the model to escape from local optimal solutions, thus accurately and efficiently obtaining object pseudo-detection annotation boxes. For the real-time object detection task, the single-stage detector YOLOv7 is selected as the basic detection model, and an object perception loss function is designed based on the class activation map to jointly supervise the training process. A method for filtering noisy pseudo-supervision information is proposed to enhance the pseudo-supervision information involved in training. Ablation experiments and multi-method comparison experiments were conducted on the URPC and RUOD datasets, and the results verify the effectiveness of the proposed strategy, and our model exhibits significant advantages in detection performance and detection efficiency compared to current mainstream and advanced models.

A semantic segmentation network based on CNN and Swin transformer for landslide detection

Abstract To address the issues of poor object localization, difficulty in object recognition, and inadequate segmentation performance of CNN-based semantic segmentation methods in landslide detection, a semantic segmentation model based on the Swin transformer and ResUNet architectures for landslide detection in remote sensing images is proposed in this paper. This method combines the global feature capabilities of the Swin transformer with the local feature extraction abilities of CNN and integrates our proposed RCCT module for landslide segmentation in remote sensing images. We apply this model to the identification and extraction of landslide disasters in Menyuan County and the Sanjiangyuan region of Qinghai Province, China, achieving a promising F1 Score of 81.91. Furthermore, we conducted a comprehensive evaluation of our proposed model on the BiJie Landslide dataset in comparison to four recently developed methods for landslide recognition. The experimental findings indicate the superior performance and effectiveness of our model over the comparative approaches.

One-Click-Based Perception for Interactive Image Segmentation.

Existing deep learning-based interactive image segmentation methods have significantly reduced the user's interaction burden with simple click interactions. However, they still require excessive numbers of clicks to continuously correct the segmentation for satisfactory results. This article explores how to harvest accurate segmentation of interested targets while minimizing the user interaction cost. To achieve the above goal, we propose a one-click-based interactive segmentation approach in this work. For this particularly challenging problem in the interactive segmentation task, we build a top-down framework dividing the original problem into a one-click-based coarse localization followed by a fine segmentation. A two-stage interactive object localization network is first designed, which aims to completely enclose the target of interest based on the supervision of object integrity (OI). Click centrality (CC) is also utilized to overcome the overlapping problem between objects. This coarse localization helps to reduce the search space and increase the focus of the click at a higher resolution. A principled multilayer segmentation network is then designed by a progressive layer-by-layer structure, which aims to accurately perceive the target with extremely limited prior guidance. A diffusion module is also designed to enhance the information flow between layers. Besides, the proposed model can be naturally extended to multiobject segmentation task. Our method achieves the state-of-the-art performance under one-click interaction on several benchmarks.

Tag-Array-Based UHF Passive RFID Tag Attitude Identification of Tracking Methods.

Attitude information is as important as position information in describing and localizing objects. Based on this, this paper proposes a method for object attitude sensing utilizing ultra-high frequency passive RFID technology. This method adopts a double tag array strategy, which effectively enhances the spatial freedom and eliminates phase ambiguity by leveraging the phase difference information between the two tags. Additionally, we delve into the issue of the phase shift caused by coupling interference between the two tags. To effectively compensate for this coupling effect, a series of experiments were conducted to thoroughly examine the specific impact of coupling effects between tags, and based on these findings, a coupling model between tags was established. This model was then integrated into the original phase model to correct for the effects of phase shift, significantly improving the sensing accuracy. Furthermore, we considered the influence of the object rotation angle on phase changes to construct an accurate object attitude recognition and tracking model. To reduce random errors during phase measurement, we employed a polynomial regression method to fit the measured tag phase information, further enhancing the precision of the sensing model. Compared to traditional positioning modes, the dual-tag array strategy essentially increases the number of virtual antennas available for positioning, providing the system with more refined directional discrimination capabilities. The experimental results demonstrated that incorporating the effects of inter-tag coupling interference and rotation angle into the phase model significantly improved the recognition accuracy for both object localization and attitude angle determination. Specifically, the average error of object positioning was reduced to 12.3 cm, while the average error of attitude angle recognition was reduced to 8.28°, making the method suitable for various practical application scenarios requiring attitude recognition.

Zero-shot autonomous robot manipulation via natural language

Smart manufacturing revolutionizes advanced automation by leveraging deep learning and robotics technology. Nevertheless, deep learning implementation in manufacturing faces several challenges in preparing datasets, training a model, and adapting to various applications. Furthermore, programming languages are required to develop deep learning models and communicate with robots, which is not straightforward to quickly and precisely reflect human intentions in manufacturing processes. To alleviate the issue, this study proposes an autonomous zero-shot robot manipulation framework via natural language. Specifically, the Segmentation Anything Model (SAM), a promptable image segmentation model, first segments objects in images captured by an RGB-D camera, which are passed to Generative Pre-trained Transformers (GPTs), a Large Language Model (LLM), to identify and localize a designated object based on human commands given in natural language. Afterwards, the Robot Operating System (ROS) manipulates a robot to a localized position. With no additional programming or specialized skills in object localization and robot manipulation, the framework demonstrates an autonomous robot automation approach through intuitive communication between humans and machines through natural language.

Consistency-based semi-supervised learning for oriented object detection

Semi-Supervised Object Detection (SSOD) has emerged as a potent framework that leverages unlabeled data to reduce annotation costs while enhancing model performance. Nevertheless, existing SSOD methods primarily concentrate on the conventional localization of horizontal objects, overlooking the common arbitrary-oriented objects in aerial scenes. Extending semi-supervised learning to oriented object detection still encounters two major challenges: (1) There is an inconsistency between classification and localization in oriented object detectors, which means that pseudo-labels selected solely based on classification scores cannot properly represent the localization quality. (2) The static and monotonic pseudo-label selection process fails to dynamically filter pseudo-labels for different tasks and categories throughout the iteration process, inevitably introducing accumulated noise and bias into the supervisory signals for the student model. To mitigate this problems, we proposed a novel consistency-based semi-supervised oriented object detection framework, named CSLO-Det. Specifically, Task Alignment Learning (TAL) is proposed to improve the consistency in terms of both features and learning objectives, thereby facilitating a more robust selection of pseudo-labels. Noise-Resistant Pseudo-label Mining (NR-PM) is introduced to separately and dynamically exploits positives for the classification and localization task. Moreover, diverging from the conventional semi-supervised learning frameworks that utilize pseudo-boxes, our method applies pixel-level dense supervision during unsupervised training, which improves detection performance. Comprehensive experiment results reveal that the proposed CSLO-Det attains state-of-the-art performance under multiple semi-supervised datasets.

3D Visual Grounding-Audio: 3D scene object detection based on audio

3D Visual Grounding (3DVG) is a prevalent multi-modal information fusion task capable of accurately localizing target objects referenced in natural language descriptions within a point cloud scene. Nevertheless, the stringent demands for input and output devices present substantial hurdles for the application and integration of 3DVG in fields like remote robotic control and telemedicine. To address this challenge, we introduce several innovative approaches. Firstly, we have initiated a novel multi-modal task, termed 3D Visual Grounding-Audio (3DVG-Audio), which is based on the fusion of audio and point cloud. To the best of our knowledge, this represents the first instance of an Audio-Point Cloud multi-modal task. 3DVG-Audio achieves precise localization of audio-mentioned objects within the point cloud by utilizing the point cloud in conjunction with the corresponding audio input. Secondly, building upon the ScanRefer, we have developed a novel dataset named 3DVG-AudioSet, specifically designed for the training and evaluation of the 3DVG-Audio method. Finally, we have crafted a tailored loss function to further enhance the performance of 3DVG-Audio and introduced a method named AP-Refer, which serves as a benchmark for the task. Extensive experimental results demonstrate the potential for deep integration of audio and point cloud to tackle complex real-world challenges. AP-Refer has successfully addressed the 3DVG-Audio, circumventing the limitations of conventional 3DVG methods, and exhibits significant application potential.

YOLO V3 AND CCN FOR THE TRACKING AND CLASSIFICATION OF AERIAL OBJECT AND DRONES

The goal of this study is to give headways in aeronautical article ID that will help with making recognitions that are both more exact and more precise. Specifically, we revamp the meaning of the article recognition anchor enclose request to remember turns for expansion to level and width, and besides, we make it conceivable to have erratic four corner point structures. Furthermore, the consideration of new anchor boxes gives the model additional adaptability to address protests that are focused at a pivot of turn that gives a 45-degree point. By accomplishing these results, we can make an organization that considers negligible tradeoffs about speed and unwavering quality, while likewise giving more exact restrictions. The latest ways to deal with PC vision and article acknowledgment are for the most part dependent on brain organizations and different advances that utilize profound learning. This powerful field of study is utilized in various applications, including military and observation, aeronautical photography, independent driving, and airborne perception. To precisely locate the location of an item, contemporary object identification techniques make use of bounding boxes that are drawn over the object and have a rectangular form (horizontal and vertical). These orthogonal bounding boxes do not consider the posture of the object, which leads to a decrease in the amount of object localization and restricts subsequent tasks such as object comprehension and tracking. We have used the DOTA dataset to present all of the results, demonstrating the value of flexible object boundaries, particularly with rotated and non-rectangular objects. We have also achieved an accuracy of 98.47% for the detection and classification of aerial objects, with forty percent of the data being used for training and the remaining twenty percent being used for testing. There was a minimum of 2.8 seconds of processing time required for the whole program to be executed to categorize all of the aerial items that were parked on the base.