Location Information Of Objects Research Articles

A spatial map: a propitious choice for constraining the binding problem.

Many studies have shown that the human visual system has two major functionally distinct cortical visual pathways: a ventral pathway, thought to be important for object recognition, and a dorsal pathway, thought to be important for spatial cognition. According to our and others previous studies, artificial neural networks with two segregated pathways can determine objects' identities and locations more accurately and efficiently than one-pathway artificial neural networks. In addition, we showed that these two segregated artificial cortical visual pathways can each process identity and spatial information of visual objects independently and differently. However, when using such networks to process multiple objects' identities and locations, a binding problem arises because the networks may not associate each object's identity with its location correctly. In a previous study, we constrained the binding problem by training the artificial identity pathway to retain relative location information of objects. This design uses a location map to constrain the binding problem. One limitation of that study was that we only considered two attributes of our objects (identity and location) and only one possible map (location) for binding. However, typically the brain needs to process and bind many attributes of an object, and any of these attributes could be used to constrain the binding problem. In our current study, using visual objects with multiple attributes (identity, luminance, orientation, and location) that need to be recognized, we tried to find the best map (among an identity map, a luminance map, an orientation map, or a location map) to constrain the binding problem. We found that in our experimental simulations, when visual attributes are independent of each other, a location map is always a better choice than the other kinds of maps examined for constraining the binding problem. Our findings agree with previous neurophysiological findings that show that the organization or map in many visual cortical areas is primarily retinotopic or spatial.

Structural Transformer with Region Strip Attention for Video Object Segmentation

Memory-based methods in semi-supervised video object segmentation (VOS) achieve competitive performance by performing feature similarity between the current frame and memory frames. However, this operation involves two challenges: 1) instances of occlusion caused by interaction, and 2) interference from similar objects or clutters in the background. In this work, we propose a Structural Transformer with Region Strip Attention (STRSA) approach to address these challenges. Specifically, we build a Structural Transformer (ST) architecture to decompose the feature similarity between the long-term memory frames and the current frame into two aspects: a time–space part and an object significance part. This allows us to investigate the spatio-temporal relationship of pixels and capture the salient features of the objects. Therefore, the differences between pixels and the specificity of objects are fully exploited. In addition, we leverage the object location information from the long-term memory masks and the strip pooling to design a Region Strip Attention (RSA) module, which boosts the attention for the foreground regions and suppresses the background clutters. Extensive experiments on DAVIS, YouTube-VOS, and MOSE benchmarks prove that our method achieves satisfactory results and outperforms the retrained baseline model.

Feature alignment via mutual mapping for few-shot fine-grained visual classification

Few-shot fine-grained visual classification aims to identify fine-grained concepts with very few samples, which is widely used in many fields, such as the classification of different species of birds in biological research, and the identification of car models in traffic monitoring. Compared with the common few-shot classification task, it encounters difficulties due to significant variations within each class and small gaps between different categories. To address such problems, previous studies primarily project support samples into the space of query samples and employ metric learning to classify query images into their respective categories. However, we observe that such methods are not effective in resolving inter-class variations. To overcome this limitation, we propose a new feature alignment method based on mutual mapping, which simultaneously considers the discriminative features of new samples and classes. Specifically, besides projecting support samples into the space of query samples for reducing intra-class variations, we also project query samples into the space of support samples to increase inter-class variations. Furthermore, a direct position self-reconstruction module is proposed to utilize the location information of objects and obtain more discriminative features. Extensive experiments on four fine-grained benchmarks demonstrate that our approach is competitive when compared with other state-of-the-art methods, in both 1-shot and 5-shot settings. In the case of 5-shot, our method achieved the best performance on all four datasets, with 92.11%, 85.31%, 96.09%, and 94.64% accuracies on CUB-200-2011, Stanford Dogs, Stanford Cars, and Aircaft, respectively.

CLIM: Contrastive Language-Image Mosaic for Region Representation

Detecting objects accurately from a large or open vocabulary necessitates the vision-language alignment on region representations. However, learning such a region-text alignment by obtaining high-quality box annotations with text labels or descriptions is expensive and infeasible. In contrast, collecting image-text pairs is simpler but lacks precise object location information to associate regions with texts. In this paper, we propose a novel approach called Contrastive Language-Image Mosaic (CLIM), which leverages large-scale image-text pairs effectively for aligning region and text representations. CLIM combines multiple images into a mosaicked image and treats each image as a ‘pseudo region’. The feature of each pseudo region is extracted and trained to be similar to the corresponding text embedding while dissimilar from others by a contrastive loss, enabling the model to learn the region-text alignment without costly box annotations. As a generally applicable approach, CLIM consistently improves different open-vocabulary object detection methods that use caption supervision. Furthermore, CLIM can effectively enhance the region representation of vision-language models, thus providing stronger backbones for open-vocabulary object detectors. Our experimental results demonstrate that CLIM improves different baseline open-vocabulary object detectors by a large margin on both OV-COCO and OV-LVIS benchmarks. The code is available at https://github.com/wusize/CLIM.

A Literature Review of Exploratory Analysis of Geolocational Data

Geospatial data is a key part of the digital age. It contains geographic coordinates and location information of people, vehicles, objects and natural phenomena. The rapid growth in its use is due to smartphones, GPS, social media and various location-based applications. This has led to a new paradigm shift in how data is used across industries. This article looks at the multifaceted role of geolocation data in today's society and examines how important it is in the development and change of various industries in today's world. Geospatial data plays a key role in the digital age and the birth of the Internet of Things. In addition to smooth navigation, it also helps industries better allocate resources, improve decision-making and increase overall efficiency. This report explores the growing importance of spatial data and its critical role in urban planning, transportation, environmental monitoring and surveillance, marketing, and public safety

Grounding spatial relations in text-only language models

This paper shows that text-only Language Models (LM) can learn to ground spatial relations like left of or below if they are provided with explicit location information of objects and they are properly trained to leverage those locations. We perform experiments on a verbalized version of the Visual Spatial Reasoning (VSR) dataset, where images are coupled with textual statements which contain real or fake spatial relations between two objects of the image. We verbalize the images using an off-the-shelf object detector, adding location tokens to every object label to represent their bounding boxes in textual form. Given the small size of VSR, we do not observe any improvement when using locations, but pretraining the LM over a synthetic dataset automatically derived by us improves results significantly when using location tokens. We thus show that locations allow LMs to ground spatial relations, with our text-only LMs outperforming Vision-and-Language Models and setting the new state-of-the-art for the VSR dataset. Our analysis show that our text-only LMs can generalize beyond the relations seen in the synthetic dataset to some extent, learning also more useful information than that encoded in the spatial rules we used to create the synthetic dataset itself.

An Adaptive Multi-Content Complementary Network for Salient Object Detection

Deep learning methods for salient object detection (SOD) have been studied actively and promisingly. However, the existing methods mainly focus on the decoding process and ignore the differences in the contributions of different encoder blocks. To address this problem for SOD, we propose an adaptive multi-content complementary network (PASNet) for salient object detection which aims to exploit the valuable contextual information in the encoder fully. Unlike existing CNN-based methods, we adopt the pyramidal visual transformer (PVTv2) as the backbone network to learn global and local representations with its self-attention mechanism. Then, we follow the coarse-to-fine strategy and introduce two novel modules, including an advanced semantic fusion module (ASFM) and a self-refinement module (SRM). Among these, the ASFM takes local branches and adjacent branches as inputs and collects semantic and location information of salient objects from high-level features to generate an initial coarse saliency map. The coarse saliency map serves as the location guidance for low-level features, and the SRM is applied to capture detailed information disguised in low-level features. We expand the location information with high-level semantics from top to bottom across the salient region, which is effectively fused with detailed information through feature modulation. The model effectively suppresses noises in the features and significantly improves their expressive capabilities. To verify the effectiveness of our PASNet, we conducted extensive experiments on five challenging datasets, and the results show that the proposed model is superior to some of the current state-of-the-art methods under different evaluation metrics.

Box-supervised dynamical instance segmentation for in-field cotton

Crop growing information is important for precision agriculture. Instance segmentation can be employed to extract not only the location and quantity information of objects from images, but also the individual phenotype information of each target. However, when it is applied on real in-field images, complex environment increases the cost of annotating. Besides, it is hard for instance segmentation network with complicated structure to be deployed in real application scenarios. To cost-efficiently extract data used for precision management from field images, this paper proposes a novel model based on dense prediction and conditionally parameterized convolution termed Box-supervised dynamical regression instance segmentation net(BDRISNet) for segmenting individuals from image in a simple way, using box-level supervision. By formulating segmentation into dense regression from each pixel to the location of its instance indicators, mask features can be generated without redundant convolution branch. The compact construction of proposed network achieves efficient inference for segmentation. Robustness and accuracy of proposed method are evaluated on an in-field cotton boll image dataset we provide. Comparison experiments with other state-of-the-arts demonstrate that our method has an average improvement of 3.13%, 5.55%, 7.08% in AP, AP50 and AP75. Furthermore, predictions of proposed method have desirable and balanced recall and precision.

TSDTVOS: Target-guided spatiotemporal dual-stream transformers for video object segmentation

Video object segmentation automatically separates the interested objects from the background across a video sequence and was an active research area in recent years. The crucial challenge lies in investigating an effective architecture to fully exploit spatiotemporal correlation in a given video sequence for achieving accurate segmentation results. In this paper, we propose a novel semi-supervised Transformer-based framework called Target-guided Spatiotemporal Dual-stream Transformers (TSDT) with two separate streams to enable effective spatiotemporal context propagation. Technically, the temporal stream is used to aggregate rich temporal cues from past frames, while the spatial stream is trained to encode object location and appearance information stored in the current frame. To compress and integrate temporal features, a target guidance block (TGB) is designed to retrieve target information in the past video flow under the guidance of the current frame. The experimental results on video object segmentation benchmarks demonstrate the feasibility and effectiveness of the proposed framework. Codes and trained models are available at https://github.com/zhouweii234/TSDTVOS.

Apple object detection based on improved YOLOX

To enhance the working efficiency of apple picking systems by improving the accuracy of object detection in natural scenes, an improved apple detection network based on YOLOX-s is proposed. The self-attention residual module is added to the last layer of the improved YOLOX-s, which is used to add global feature information for small apples. An additional object detection head is added in YOLOX-s to strengthen the feature information of dense objects. Extra convolutional branches are added into the spatial pyramid pooling structure to enhance feature fusion and compensate for lost object location information. The loss function of the network is replaced by the α-CIoU loss, which is used to improve the bbox regression accuracy by up-weighting the loss and gradient of the high IoU prediction box. Experiment results show that the mAP50 value and the recall rate of the improved network reached 91.8% and 97%, respectively. Therefore, the improved network is superior to the existing detection networks in detection accuracy. Meanwhile, the detection time increased slightly, but it still meets the requirements of real-time detection of the picking system.

Triple critical feature capture network: A triple critical feature capture network for weakly supervised object detection

AbstractWeakly supervised object detection (WSOD) is becoming increasingly important for computer vision tasks, as it alleviates the burden of manual annotation. Most WSOD techniques rely on multiple instance learning (MIL), which tends to localise the discriminative parts of salient objects instead of the whole object. In addition, network training is often supervised using simple image‐level annotations, without including object quantities or location information. However, this can lead to ambiguous differentiation of object instances, both in terms of location and semantics. To address these issues, propose an end‐to‐end triple critical feature capture network (TCFCNet) for WSOD is proposed. Specifically, a multi‐task branch, which can perform fully supervised classification and regression task, was integrated with a PCL in an end‐to‐end network for refining object locations in an online method. A cyclic parametric dropblock module (CPDM) was then designed to help the detector focus on the contextual information by using cyclic masking techniques to maximise the removal of the discriminative components of an object instance to alleviate the part domination problem. Finally, a feature decoupling module (FDM) is proposed to further reduce the ambiguous distinction of object instances by adaptively constructing robust critical features that adapt to multi‐task branch for classification and regression tasks, which contains a feature enhancement module and task‐specific polarisation functions. Comprehensive experiments are carried out on the challenging Pascal VOC 2007 and VOC 2012 datasets. The proposed method achieves a 54.6% mAP and a 44.3% mAP on the Pascal VOC 2007 and VOC 2012 datasets respectively, showed that our method outperformed existing mainstream techniques by a considerable margin.

ALFPN: Adaptive Learning Feature Pyramid Network for Small Object Detection

Object detection has become a crucial technology in intelligent vision systems, enabling automatic detection of target objects. While most detectors perform well on open datasets, they often struggle with small-scale objects. This is due to the traditional top-down feature fusion methods that weaken the semantic and location information of small objects, leading to poor classification performance. To address this issue, we propose a novel feature pyramid network, the adaptive learnable feature pyramid network (ALFPN). Our approach features an adaptive feature inspection that incorporates learnable fusion coefficients in the fusion of different levels of feature layers, aiding the network in learning features with less noise. In addition, we construct a context-aligned supervisor that adjusts the feature maps fused at different levels to avoid scaling-related offset effects. Our experiments demonstrate that our method achieves state-of-the-art results and is highly robust for the small object detection on the TT-100K, PASCAL VOC, and COCO datasets. These findings indicate that a model’s ability to extract discriminant features is positively correlated with its performance in detecting small objects.

Independent spatiotemporal effects of spatial attention and background clutter on human object location representations

Spatial attention helps us to efficiently localize objects in cluttered environments. However, the processing stage at which spatial attention modulates object location representations remains unclear. Here we investigated this question identifying processing stages in time and space in an EEG and fMRI experiment respectively. As both object location representations and attentional effects have been shown to depend on the background on which objects appear, we included object background as an experimental factor. During the experiments, human participants viewed images of objects appearing in different locations on blank or cluttered backgrounds while either performing a task on fixation or on the periphery to direct their covert spatial attention away or towards the objects. We used multivariate classification to assess object location information. Consistent across the EEG and fMRI experiment, we show that spatial attention modulated location representations during late processing stages (>150 ms, in middle and high ventral visual stream areas) independent of background condition. Our results clarify the processing stage at which attention modulates object location representations in the ventral visual stream and show that attentional modulation is a cognitive process separate from recurrent processes related to the processing of objects on cluttered backgrounds.

A novel algorithm for small object detection based on YOLOv4.

Small object detection is one of the difficulties in the development of computer vision, especially in the case of complex image backgrounds, and the accuracy of small object detection still needs to be improved. In this article, we present a small object detection network based on YOLOv4, which solves some obstacles that hinder the performance of traditional methods in small object detection tasks in complex road environments, such as few effective features, the influence of image noise, and occlusion by large objects, and improves the detection of small objects in complex background situations such as drone aerial survey images. The improved network architecture reduces the computation and GPU memory consumption of the network by including the cross-stage partial network (CSPNet) structure into the spatial pyramid pool (SPP) structure in the YOLOv4 network and convolutional layers after concatenation operation. Secondly, the accuracy of the model on the small object detection task is improved by adding a more suitable small object detection head and removing one used for large object detection. Then, a new branch is added to extract feature information at a shallow location in the backbone part, and the feature information extracted from this branch is fused in the neck part to enrich the small object location information extracted by the model; when fusing feature information from different levels in the backbone, the fusion weight of useful information is increased by adding a weighting mechanism to improve detection performance at each scale. Finally, a coordinated attention (CA) module is embedded at a suitable location in the neck part, which enables the model to focus on spatial location relationships and inter-channel relationships and enhances feature representation capability. The proposed model has been tested to detect 10 different target objects in aerial images from drones and five different road traffic signal signs in images taken from vehicles in a complex road environment. The detection speed of the model meets the criteria of real-time detection, the model has better performance in terms of accuracy compared to the existing state-of-the-art detection models, and the model has only 44M parameters. On the drone aerial photography dataset, the average accuracy of YOLOv4 and YOLOv5L is 42.79% and 42.10%, respectively, while our model achieves an average accuracy (mAP) of 52.76%; on the urban road traffic light dataset, the proposed model achieves an average accuracy of 96.98%, which is also better than YOLOv4 (95.32%), YOLOv5L (94.79%) and other advanced models. The current work provides an efficient method for small object detection in complex road environments, which can be extended to scenarios involving small object detection, such as drone cruising and autonomous driving.

Hybrid Attention Asynchronous Cascade Network for Salient Object Detection

The highlighted area or object is defined as the salient region or salient object. For salient object detection, the main challenges are still the clarity of the boundary information of the salient object and the positioning accuracy of the salient object in the complex background, such as noise and occlusion. To remedy these issues, it is proposed that the asynchronous cascade saliency detection algorithm based on a deep network, which is embedded in an encoder–decoder architecture. Moreover, the lightweight hybrid attention module is designed to obtain the explicit boundaries of salient regions. In order to effectively improve location information of salient objects, this paper adopts a bi-directional asynchronous cascade fusion strategy, which generates prediction maps with higher accuracy. The experimental results on five benchmark datasets show that the proposed network HACNet is on a par with the state of the art for image saliency datasets.

MAD-YOLO: A quantitative detection algorithm for dense small-scale marine benthos

Marine biological resources are abundant, and the reasonable development, research and protection of marine biological resources are of great significance to marine ecological health and economic development. At present, underwater object quantitative detection plays a very important role in marine biological science research, marine species richness survey, and rare species conservation. However, the problems of a large amount of noise in the underwater environment, small object scale, dense biological distribution, and occlusion all increase the detection difficulty. In this paper, a detection algorithm MAD-YOLO (Multiscale Feature Extraction and Attention Feature Fusion Reinforced YOLO for Marine Benthos Detection) is proposed, which is based on improved YOLOv5 is proposed to solve the above problems. To improve the adaptability of the network to the underwater environment, VOVDarkNet is designed as the feature extraction backbone. It uses the intermediate features with different receptive fields to reinforce the ability to extract feature. AFC-PAN is proposed as the feature fusion network so that the network can learn correct feature information and location information of objects at various scales, improving the network's ability to perceive small objects. Label assignment strategy SimOTA and decoupled head are introduced to help the model better handles occlusion and dense distribution problems. Experiments show the MAD-YOLO algorithm increases mAP0.5:0.95 on the URPC2020 dataset from 49.8% to 53.4% compared to the original YOLOv5. Moreover, the advantages of the model are visualized and analyzed by the method of controlling variables in the experimental part. The experiments show that MAD-YOLO is suitable for detecting blurred, dense, and small-scale objects. The model performs well in marine benthos detection tasks and can effectively promote marine life science research and marine engineering implementation. The source code is publicly available at https://github.com/JoeNan1/MAD-YOLO.

Learning Salient Feature for Salient Object Detection Without Labels.

Supervised salient object detection (SOD) methods achieve state-of-the-art performance by relying on human-annotated saliency maps, while unsupervised methods attempt to achieve SOD by not using any annotations. In unsupervised SOD, how to obtain saliency in a completely unsupervised manner is a huge challenge. Existing unsupervised methods usually gain saliency by introducing other handcrafted feature-based saliency methods. In general, the location information of salient objects is included in the feature maps. If the features belonging to salient objects are called salient features and the features that do not belong to salient objects, such as background, are called nonsalient features, by dividing the feature maps into salient features and nonsalient features in an unsupervised way, then the object at the location of the salient feature is the salient object. Based on the above motivation, a novel method called learning salient feature (LSF) is proposed, which achieves unsupervised SOD by LSF from the data itself. This method takes enhancing salient feature and suppressing nonsalient features as the objective. Furthermore, a salient object localization method is proposed to roughly locate objects where the salient feature is located, so as to obtain the salient activation map. Usually, the object in the salient activation map is incomplete and contains a lot of noise. To address this issue, a saliency map update strategy is introduced to gradually remove noise and strengthen boundaries. The visualization of images and their salient activation maps show that our method can effectively learn salient visual objects. Experiments show that we achieve superior unsupervised performance on a series of datasets.

TCRNet: A Trifurcated Cascaded Refinement Network for Salient Object Detection

Due to the rapid development of deep learning, salient object detection has achieved promising results in recent years. However, how to construct a powerful saliency detection network to generate saliency maps with completely highlighted salient objects and effectively suppressed background noise still remains an open and challenging problem. In this paper, we propose a novel trifurcated cascaded refinement network (TCRNet) to explore multi-level feature fusion and global information representation. Specifically, all the side-output features from the backbone network are divided into three branches (i.e., global information, deep information and shallow information) according to feature similarities, and then the global contexts, semantic knowledge and fined details respectively generated by the three branches are integrated in a cascaded refinement way. This refinement strategy aims to guide the feature learning of important regions and suppress background noises. Moreover, we propose a cross-level feature fusion (CFF) module to fuse multi-level similar features within a same branch by exploring their interrelationship, thus capturing the important semantic information and structure details. To this end, the complementarity of similar features is utilized to refine feature at each level by sharing an encoded spatial weight map. Besides, we design a context-aware feature learning (CFL) module to capture global contexts in a single layer by taking account of both single-feature representation and multi-feature interaction, thus providing important location information of salient objects. Extensive experiments on five benchmarks are conducted to demonstrate that the proposed network TCRNet performs favorably against 20 state-of-the-art salient object detection methods on five benchmark datasets, indicating the effectiveness and superiority of our method.

Deep learning-based foreign object detection method for aviation runways

Abstract Airport runway foreign object detection systems can quickly and accurately detect and identify foreign runway objects, which is significant for ensuring airport flight safety. Because of the drawbacks shown by the algorithm, the paper proposes to combine a new system scheme based on deep learning to obtain multiple feature information for identifying foreign objects on airport runways and improve the recognition accuracy of foreign object detection. This paper designs and constructs a dataset for accomplishing airport runway foreign object detection based on the data distribution and attribute semantics of actual airport runway foreign object scenarios and the technical features of deep learning, designs FOD detection and multi-attribute recognition networks, further design algorithms, and perform validation. The results show that the deep learning technology can accomplish all tasks of the airport runway foreign object detection system, which has not only good robustness to different environments but also has practical value for multi-tasking, and the localization task can accurately obtain the location information of foreign objects and improve the recognition accuracy of foreign object detection. Therefore, the deep learning-based airport runway foreign object recognition system designed in this paper is effective and can improve the accuracy of foreign object recognition.

WSRC: Weakly Supervised Faster RCNN Toward Accurate Traffic Object Detection

Weakly supervised object detection has attracted increasing interest in recent years as driven by data annotation bottleneck in deep network training. On the other hand, the performance of weakly supervised models still lags far behind due to the missing of object location information during training. This paper presents a Weakly Supervised Faster RCNN (WSRC) that aims to train accurate traffic object detectors without using any box annotations, addressing the phenomenon that the traffic object detector trained by traditional method cannot be applied in the real traffic scene due to the lack of data annotation. The WSRC designs an innovative proposal evaluation network (PEN) that is trained by using a large amount of proposals that are scored according to their Intersection-over-Union (IoU) overlap with pseudo ground-truth boxes. Novel loss functions are designed to teach the PEN and WSRC to identify good proposals and regress to accurate object boxes under the guidance of loss dynamics and proposal scores. The proposed WSRC achieves superior detection performance over the datasets Pascal VOC2007, MS COCO and Cityscape, demonstrating its effectiveness while short of annotation boxes.