Representation Of Object Features Research Articles

Direction-aware multi-branch attention and Gaussian label assignment for remote sensing aggregative object detection

ABSTRACT A large number of objects of different scales appear aggregative distribution in remote sensing images, which brings two challenges to remote sensing object detection. The first challenge arises from the presence of objects with large scale differences within the same scene, which complicates the detector’s ability to balance the detection performance between large and small objects. The second challenge is that the distribution of dense objects on remote sensing images with complex backgrounds can reduce the feature representation capability of the network, leading to false detections and missed detections of objects. To address the two challenges present in remote sensing object detection, we propose a new object detection network DA-GLANet (Direction-Aware Multi-Branch Attention and Gaussian Label Assignment Balance Net). For the first challenge, we designed a multi-scale feature fusion module to fully utilize semantic information at different scales, thereby improving the detection accuracy of multi-scale objects. Subsequently, to tackle the second challenge, we proposed a multi-branch attention module with direction-aware capabilities. This module can enhance the network’s feature representation of objects of interest by generating attention maps that are both direction-aware and position-sensitive. In addition, we propose a label assignment strategy based on Gaussian distribution to address the imbalance of positive and negative labels caused by the complexity of remote sensing images. These methods work synergistically to improve aggregative object detection accuracy. We have conducted extensive experiments on the DOTA dataset, Levir dataset, and WHU buildings change detection dataset to demonstrate the superiority of our method. On the DOTA dataset, our method improves mAP by 0.4%, 0.3%, and 0.2% compared to the top three methods. On the Levir dataset, our method improves mAP by 2.5%, 2.2%, and 1.6% compared to the top three methods.

Multimodal audio-visual robot fusing 3D CNN and CRNN for player behavior recognition and prediction in basketball matches.

Intelligent robots play a crucial role in enhancing efficiency, reducing costs, and improving safety in the logistics industry. However, traditional path planning methods often struggle to adapt to dynamic environments, leading to issues such as collisions and conflicts. This study aims to address the challenges of path planning and control for logistics robots in complex environments. The proposed method integrates information from different perception modalities to achieve more accurate path planning and obstacle avoidance control, thereby enhancing the autonomy and reliability of logistics robots. Firstly, a 3D convolutional neural network (CNN) is employed to learn the feature representation of objects in the environment for object recognition. Next, long short-term memory (LSTM) is used to model spatio-temporal features and predict the behavior and trajectory of dynamic obstacles. This enables the robot to accurately predict the future position of obstacles in complex environments, reducing collision risks. Finally, the Dijkstra algorithm is applied for path planning and control decisions to ensure the robot selects the optimal path in various scenarios. Experimental results demonstrate the effectiveness of the proposed method in terms of path planning accuracy and obstacle avoidance performance. The method outperforms traditional approaches, showing significant improvements in both aspects. The intelligent path planning and control scheme presented in this paper enhances the practicality of logistics robots in complex environments, thereby promoting efficiency and safety in the logistics industry.

CA-YOLOv5: A YOLO model for apple detection in the natural environment

Improving the effectiveness of harvesting robots requires quick and accurate apple detection in natural environments. The colour and shape features of apples are corrupted due to the reflected light and the incomplete coverage of the fruit bag, bringing difficulties to apple detection. To address this issue, the Coordinate Attention You Only Look Once version 5 (CA-YOLOv5) is designed to simultaneously detect bagged and unbagged apples in the natural environment. Firstly, 1525 apple images are collected from apple orchards to build a dataset. Secondly, to solve the reflected light problem, all C3 modules in the Backbone are substituted for Coordinate Attention modules which can improve the feature representation of objects. Finally, to solve the incomplete bagging problem, the Path Aggregation Network in the Neck is replaced by a Bidirectional Feature Pyramid Network which can better fuse the features of various sizes. The CA-YOLOv5 network reaches 82.7%, 89.8%, 48.6%, and 87.0% for recall, mAP@0.5, mAP@0.5:0.95, and F1 score, respectively, which is 2.3%,1.2%,1.9%, and 2.9% higher than the YOLOv5. The results reveal that CA-YOLOv5 has much superior detection performance than the original YOLOv5, and it can serve as a technical benchmark for the development of automatic orchard-picking robots.

Dynamic neural reconstructions of attended object location and features using EEG.

Attention allows us to select relevant and ignore irrelevant information from our complex environments. What happens when attention shifts from one item to another? To answer this question, it is critical to have tools that accurately recover neural representations of both feature and location information with high temporal resolution. In the present study, we used human electroencephalography (EEG) and machine learning to explore how neural representations of object features and locations update across dynamic shifts of attention. We demonstrate that EEG can be used to create simultaneous time courses of neural representations of attended features (time point-by-time point inverted encoding model reconstructions) and attended location (time point-by-time point decoding) during both stable periods and across dynamic shifts of attention. Each trial presented two oriented gratings that flickered at the same frequency but had different orientations; participants were cued to attend one of them and on half of trials received a shift cue midtrial. We trained models on a stable period from Hold attention trials and then reconstructed/decoded the attended orientation/location at each time point on Shift attention trials. Our results showed that both feature reconstruction and location decoding dynamically track the shift of attention and that there may be time points during the shifting of attention when 1) feature and location representations become uncoupled and 2) both the previously attended and currently attended orientations are represented with roughly equal strength. The results offer insight into our understanding of attentional shifts, and the noninvasive techniques developed in the present study lend themselves well to a wide variety of future applications.NEW & NOTEWORTHY We used human EEG and machine learning to reconstruct neural response profiles during dynamic shifts of attention. Specifically, we demonstrated that we could simultaneously read out both location and feature information from an attended item in a multistimulus display. Moreover, we examined how that readout evolves over time during the dynamic process of attentional shifts. These results provide insight into our understanding of attention, and this technique carries substantial potential for versatile extensions and applications.

Health insurance fraud detection by using an attributed heterogeneous information network with a hierarchical attention mechanism

BackgroundWith the rapid growth of healthcare services, health insurance fraud detection has become an important measure to ensure efficient use of public funds. Traditional fraud detection methods have tended to focus on the attributes of a single visit and have ignored the behavioural relationships of multiple visits by patients.MethodsWe propose a health insurance fraud detection model based on a multilevel attention mechanism that we call MHAMFD. Specifically, we use an attributed heterogeneous information network (AHIN) to model different types of objects and their rich attributes and interactions in a healthcare scenario. MHAMFD selects appropriate neighbour nodes based on the behavioural relationships at different levels of a patient’s visit. We also designed a hierarchical attention mechanism to aggregate complex semantic information from the interweaving of different levels of behavioural relationships of patients. This increases the feature representation of objects and makes the model interpretable by identifying the main factors of fraud.ResultsExperimental results using real datasets showed that MHAMFD detected health insurance fraud with better accuracy than existing methods.ConclusionsExperiment suggests that the behavioral relationships between patients’ multiple visits can also be of great help to detect health care fraud. Subsequent research fraud detection methods can also take into account the different behavioral relationships between patients.

Text Semantic Fusion Relation Graph Reasoning for Few-Shot Object Detection on Remote Sensing Images

Most object detection methods based on remote sensing images are generally dependent on a large amount of high-quality labeled training data. However, due to the slow acquisition cycle of remote sensing images and the difficulty in labeling, many types of data samples are scarce. This makes few-shot object detection an urgent and necessary research problem. In this paper, we introduce a remote sensing few-shot object detection method based on text semantic fusion relation graph reasoning (TSF-RGR), which learns various types of relationships from common sense knowledge in an end-to-end manner, thereby empowering the detector to reason over all classes. Specifically, based on the region proposals provided by the basic detection network, we first build a corpus containing a large number of text language descriptions, such as object attributes and relations, which are used to encode the corresponding common sense embeddings for each region. Then, graph structures are constructed between regions to propagate and learn key spatial and semantic relationships. Finally, a joint relation reasoning module is proposed to actively enhance the reliability and robustness of few-shot object feature representation by focusing on the degree of influence of different relations. Our TSF-RGR is lightweight and easy to expand, and it can incorporate any form of common sense information. Sufficient experiments show that the text information is introduced to deliver excellent performance gains for the baseline model. Compared with other few-shot detectors, the proposed method achieves state-of-the-art performance for different shot settings and obtains highly competitive results on two benchmark datasets (NWPU VHR-10 and DIOR).

High-resolution satellite video single object tracking based on thicksiam framework

ABSTRACT High-resolution satellite videos realize the short-dated gaze observation of the designated area on the ground, and its emergence has improved the temporal resolution of remote sensing data to the second level. Single object tracking (SOT) task in satellite video has attracted considerable attention. However, it faces challenges such as complex background, poor object feature representation, and lack of publicly available datasets. To cope with these challenges, a ThickSiam framework consisting of a Thickened Residual Block Siamese Network (TRBS-Net) for extracting robust semantic features to obtain the initial tracking results and a Remoulded Kalman Filter (RKF) module for simultaneously correcting the trajectory and size of the targets is designed in this work. The results of TRBS-Net and RKF modules are combined by an N-frame-convergence mechanism to achieve accurate tracking results. Ablation experiments are implemented on our annotated dataset to evaluate the performance of the proposed ThickSiam framework and other 19 state-of-the-art trackers. The comparison results show that our ThickSiam tracker obtains a precision value of 0.991 and a success value of 0.755 while running at 56.849 FPS implemented on one NVIDIA GTX1070Ti GPU.

Cross-Layer Triple-Branch Parallel Fusion Network for Small Object Detection in UAV Images

Detection of small objects in Unmanned Aerial Vehicle (UAV) images, which contain a large number of objects with extremely small pixels, has been a major challenge over the last few decades. Owing to the lack of sufficiently detailed representation of object features, conventional detectors based on deep learning are unsatisfactory when detecting small objects. Meanwhile, the flight altitude and the shooting angle of the drone always changing, which leads to an uneven dispersion of the multiple objects and a varying density of small objects. In response to the issues mentioned above, a novel lightweight Cross-layer Triple-branch Parallel Fusion Network (CTPFNet) is proposed to improve the real-time detection accuracy of small objects under UAV images. Firstly, a novel downsampling structure called the Inverse Residual Pooling Cascade (IRPC) module is proposed to obtain richer feature information about small objects. Secondly, we construct an improved global feature extraction structure Efficient Layer Aggregation Networks-Transformer (ELAN-Trans) to enhance the association between global features. Then, we design the Hybrid Dilated Depthwise Separable-Spatial Pyramid Pooling-Fast (HDDS-SPPF) to capture more contextual information by using the dilated convolution operations. To enhance the cross-scale transfer fusion of low-level original features containing more detailed localization information and deep-level deep features containing more semantic information, we propose a CTPF module embedded into the neck region for the secondary feature reuse of adjacent feature maps with varying resolutions. Extensive experiments on the public VisDrone2021-DET dataset show that the proposed model achieves significant performance gains with fewer parameters.

Scale Enhancement Pyramid Network for Small Object Detection from UAV Images.

Object detection is challenging in large-scale images captured by unmanned aerial vehicles (UAVs), especially when detecting small objects with significant scale variation. Most solutions employ the fusion of different scale features by building multi-scale feature pyramids to ensure that the detail and semantic information are abundant. Although feature fusion benefits object detection, it still requires the long-range dependencies information necessary for small objects with significant scale variation detection. We propose a simple yet effective scale enhancement pyramid network (SEPNet) to address these problems. A SEPNet consists of a context enhancement module (CEM) and feature alignment module (FAM). Technically, the CEM combines multi-scale atrous convolution and multi-branch grouped convolution to model global relationships. Additionally, it enhances object feature representation, preventing features with lost spatial information from flowing into the feature pyramid network (FPN). The FAM adaptively learns offsets of pixels to preserve feature consistency. The FAM aims to adjust the location of sampling points in the convolutional kernel, effectively alleviating information conflict caused by the fusion of adjacent features. Results indicate that the SEPNet achieves an AP score of 18.9% on VisDrone, which is 7.1% higher than the AP score of state-of-the-art detectors RetinaNet achieves an AP score of 81.5% on PASCAL VOC.

Action planning modulates the representation of object features in human fronto-parietal and occipital cortex.

The visual cortex has been extensively studied to investigate its role in object recognition but to a lesser degree to determine how action planning influences the representation of objects' features. We used functional MRI and pattern classification methods to determine if during action planning, object features (orientation and location) could be decoded in an action‐dependent way. Sixteen human participants used their right dominant hand to perform movements (Align or Open reach) towards one of two 3D‐real oriented objects that were simultaneously presented and placed on either side of a fixation cross. While both movements required aiming towards target location, Align but not Open reach movements required participants to precisely adjust hand orientation. Therefore, we hypothesized that if the representation of object features is modulated by the upcoming action, pre‐movement activity pattern would allow more accurate dissociation between object features in Align than Open reach tasks. We found such dissociation in the anterior and posterior parietal cortex, as well as in the dorsal premotor cortex, suggesting that visuomotor processing is modulated by the upcoming task. The early visual cortex showed significant decoding accuracy for the dissociation between object features in the Align but not Open reach task. However, there was no significant difference between the decoding accuracy in the two tasks. These results demonstrate that movement‐specific preparatory signals modulate object representation in the frontal and parietal cortex, and to a lesser extent in the early visual cortex, likely through feedback functional connections.

CG-SSD: Corner guided single stage 3D object detection from LiDAR point cloud

Detecting accurate 3D bounding boxes of the object from point clouds is a major task in autonomous driving perception. At present, the anchor-based or anchor-free models that use LiDAR point clouds for 3D object detection use the center assigner strategy to infer the 3D bounding boxes. However, in the real-world scene, due to the occlusions and the effective detection range of the LiDAR system, only part of the object surface can be covered by the collected point clouds, and there are no measured 3D points corresponding to the physical object center. Obtaining the object by aggregating the incomplete surface point clouds will bring a loss of accuracy in direction and dimension estimation. To address this problem, we propose a corner-guided anchor-free single-stage 3D object detection model (CG-SSD). Firstly, the point clouds within a single frame are assigned to regular 3D grids. 3D sparse convolution backbone network composed of residual layers and sub-manifold sparse convolutional layers are used to construct bird’s eye view (BEV) features for further deeper feature mining by a lite U-shaped network; Secondly, a novel corner-guided auxiliary module (CGAM) with adaptive corner classification algorithm is proposed to incorporate corner supervision signals into the neural network. CGAM is explicitly designed and trained to estimate locations of partially visible and invisible corners to obtain a more accurate object feature representation, especially for small or partial occluded objects; Finally, the deep features from both the backbone networks and CGAM module are concatenated and fed into the head module to predict the classification and 3D bounding boxes of the objects in the scene. The experiments demonstrate that CG-SSD achieves the state-of-art performance on the ONCE benchmark for supervised 3D object detection using single frame point cloud data, with 62.77% mAP. Additionally, the experiments on ONCE and Waymo Open Dataset show that CGAM can be extended to most anchor-based models which use the BEV feature to detect objects, as a plug-in and bring +1.17%∼+14.23% AP improvement. The code is available at https://github.com/mrqrs/CG-SSD.

Attention-Guided Multitask Convolutional Neural Network for Power Line Parts Detection

Power line parts detection refers to the inspection of key parts on transmission lines against the complex background in aerial images and identifying whether exist anomalies that cause transmission failure. Obviously, this process plays a pivotal role in ensuring the safety of power transmission. Most of the existing methods are based on deep convolutional neural networks. However, the complexity and variability of the aerial image background and the problem of unmanned aerial vehicles (UAVs) shooting perspective and distance pose a challenge for previous works. This study aims to improve the detection accuracy of the model and propose an attention-guided multitask convolutional neural network (AGMNet). First, to enhance the feature representation of objects in aerial images, we construct spatial region attention blocks that are suitable for object detection. It can be inserted into any existing convolutional backbone network. Due to its efficient feature tensor computation method, the network can obtain competitive results with less computational memory. Second, we introduce a multitask framework that creatively considers the identification of rust levels and abnormal conditions of power line components, which has not been considered in previous works. Finally, we incorporate the refinable region proposal network (RPN) structure and multiscale training strategy to improve the robustness of the network. The experimental results on the testing datasets show that the proposed AGMNet can recognize the power parts (dampers and suspension clamps) with a mean average precision (mAP) of 95.3% and simultaneously identify their rust levels with an mAP of 75.4% and abnormal conditions with an mAP of 92.7%.

Realtime Single-Shot Refinement Neural Network With Adaptive Receptive Field for 3D Object Detection From LiDAR Point Cloud

Object detection plays an important role in autonomous driving systems. LiDAR is widely used in autonomous driving vehicles and robots as a sensor for environmental perception. Recently, the development of computational power and deep learning technology makes it possible to classify and locate objects from LiDAR point cloud in a single end-to-end learnable network. However, objects are sparsely distributed in large point cloud field, and are always been partly scanned by LiDAR, which pose a challenge for accurate and rapid object positioning and classification from the raw point cloud. In this paper, we introduce a new single-shot refinement neural network for fast and accurate 3D object detection from the raw LiDAR point cloud. Firstly, we exploit self-attention mechanism in main object detection branch to enhance object feature representation. Secondly, we apply deformable convolution for learning adaptive receptive fields to fully capture the features of rotating and partially visible objects. Thirdly, an object refinement branch is introduced to produce a finer regression of objects upon the primary estimation from the main detection branch. All proposed modules have been proven to effectively improve the accuracy of object detection. Our method is evaluated on KITTI 3D detection benchmark and achieves state-of-the-art results while maintains real-time efficiency. Furthermore, real-time test in autonomous driving vehicle demonstrates that our method is robust to 16 channels LiDAR and can meet the demands of accuracy, efficiency, and visibility of object detection in various urban scenarios.

Object Detection in High Resolution Remote Sensing Imagery Based on Convolutional Neural Networks With Suitable Object Scale Features

Object detection in high spatial resolution remote sensing images (HSRIs) is an important part of image information automatic extraction, analysis, and understanding. The region of interest (ROI) scale of object detection and the object feature representation are two vital factors in HSRI object detection. With respect to these two issues, this article presents a novel HSRI object detection method based on convolutional neural networks (CNNs) with suitable object scale features. First, the suitable ROI scale of object detection is obtained by compiling statistics for the scale range of objects in HSRIs. Then, a CNN framework for object detection in HSRIs is designed using a suitable ROI scale of object detection. The object features obtained using a CNN have good universality and robustness. Finally, a CNN framework with a suitable ROI scale of object detection is trained and tested. Using the WHU-RSONE data set, the proposed method is compared with the faster region-based CNN (Faster-RCNN) framework. The experimental results show that the proposed method outperforms the Faster-RCNN framework and provides good object detection results in HSRIs.

Deep Learning for Generic Object Detection: A Survey

Object detection, one of the most fundamental and challenging problems in computer vision, seeks to locate object instances from a large number of predefined categories in natural images. Deep learning techniques have emerged as a powerful strategy for learning feature representations directly from data and have led to remarkable breakthroughs in the field of generic object detection. Given this period of rapid evolution, the goal of this paper is to provide a comprehensive survey of the recent achievements in this field brought about by deep learning techniques. More than 300 research contributions are included in this survey, covering many aspects of generic object detection: detection frameworks, object feature representation, object proposal generation, context modeling, training strategies, and evaluation metrics. We finish the survey by identifying promising directions for future research.

Multisensory stimuli enhance 3-year-old children’s executive function: A three-dimensional object version of the standard Dimensional Change Card Sort

A three-dimensional object version of the standard Dimensional Change Card Sort (DCCS) was developed to examine the influence of multisensory stimuli on 3-year-old children’s executive function. Whereas the developmental phenomenon marking 3-year-olds’ difficulties with rule use in the standard DCCS can be attributed to several cognitive factors, we examined the possibility that better encoding of object features could facilitate children’s rule-switching behavior. We examined whether 3-year-olds might be able to capitalize on cues available to multiple senses to create a more robust representation of object features that would enable them to overcome previous difficulties with rule switching in the standard DCCS. Participants were randomly assigned to the standard two-dimensional DCCS or the three-dimensional object version that was designed to match the rabbit and boat images used in the card version. The 3-year-olds who completed the object version outperformed those who completed the standard card version, succeeding in switching rules more accurately when provided with visual, auditory–verbal labeling, and tactile information of object features. Notably, more children achieved perfect accuracy and fewer children achieved floor-level performance in the object version than in the card version. We attribute 3-year-olds’ success in the object version to greater cognitive control made possible by the enhanced encoding of the stimulus properties through multisensory input and enhanced cognitive processing of ecologically valid three-dimensional objects.

Cash-Out User Detection Based on Attributed Heterogeneous Information Network with a Hierarchical Attention Mechanism

As one of the major frauds in financial services, cash-out fraud is that users pursue cash gains with illegal or insincere means. Conventional solutions for the cash-out user detection are to perform subtle feature engineering for each user and then apply a classifier, such as GDBT and Neural Network. However, users in financial services have rich interaction relations, which are seldom fully exploited by conventional solutions. In this paper, with the real datasets in Ant Credit Pay of Ant Financial Services Group, we first study the cashout user detection problem and propose a novel hierarchical attention mechanism based cash-out user detection model, called HACUD. Specifically, we model different types of objects and their rich attributes and interaction relations in the scenario of credit payment service with an Attributed Heterogeneous Information Network (AHIN). The HACUD model enhances feature representation of objects through meta-path based neighbors exploiting different aspects of structure information in AHIN. Furthermore, a hierarchical attention mechanism is elaborately designed to model user’s preferences towards attributes and meta-paths. Experimental results on two real datasets show that the HACUD outperforms the state-of-the-art methods.

Modality-correlation-aware sparse representation for RGB-infrared object tracking

To intelligently analyze and understand video content, a key step is to accurately perceive the motion of the interested objects in videos. To this end, the task of object tracking, which aims to determine the position and status of the interested object in consecutive video frames, is very important, and has received great research interest in the last decade. Although numerous algorithms have been proposed for object tracking in RGB videos, most of them may fail to track the object when the information from the RGB video is not reliable (e.g. in dim environment or large illumination change). To address this issue, with the popularity of dual-camera systems for capturing RGB and infrared videos, this paper presents a feature representation and fusion model to combine the feature representation of the object in RGB and infrared modalities for object tracking. Specifically, this proposed model is able to (1) perform feature representation of objects in different modalities by employing the robustness of sparse representation, and (2) combine the representation by exploiting the modality correlation. Extensive experiments demonstrate the effectiveness of the proposed method.

Visual Tracking via Dynamic Graph Learning.

Existing visual tracking methods usually localize a target object with a bounding box, in which the performance of the foreground object trackers or detectors is often affected by the inclusion of background clutter. To handle this problem, we learn a patch-based graph representation for visual tracking. The tracked object is modeled by with a graph by taking a set of non-overlapping image patches as nodes, in which the weight of each node indicates how likely it belongs to the foreground and edges are weighted for indicating the appearance compatibility of two neighboring nodes. This graph is dynamically learned and applied in object tracking and model updating. During the tracking process, the proposed algorithm performs three main steps in each frame. First, the graph is initialized by assigning binary weights of some image patches to indicate the object and background patches according to the predicted bounding box. Second, the graph is optimized to refine the patch weights by using a novel alternating direction method of multipliers. Third, the object feature representation is updated by imposing the weights of patches on the extracted image features. The object location is predicted by maximizing the classification score in the structured support vector machine. Extensive experiments show that the proposed tracking algorithm performs well against the state-of-the-art methods on large-scale benchmark datasets.

Deep Residual Network Predicts Cortical Representation and Organization of Visual Features for Rapid Categorization

The brain represents visual objects with topographic cortical patterns. To address how distributed visual representations enable object categorization, we established predictive encoding models based on a deep residual network, and trained them to predict cortical responses to natural movies. Using this predictive model, we mapped human cortical representations to 64,000 visual objects from 80 categories with high throughput and accuracy. Such representations covered both the ventral and dorsal pathways, reflected multiple levels of object features, and preserved semantic relationships between categories. In the entire visual cortex, object representations were organized into three clusters of categories: biological objects, non-biological objects, and background scenes. In a finer scale specific to each cluster, object representations revealed sub-clusters for further categorization. Such hierarchical clustering of category representations was mostly contributed by cortical representations of object features from middle to high levels. In summary, this study demonstrates a useful computational strategy to characterize the cortical organization and representations of visual features for rapid categorization.