Neighboring Features Research Articles

The Graph Neural Network Detector Based on Neighbor Feature Alignment Mechanism in LIDAR Point Clouds

Three-dimensional (3D) object detection has a vital effect on the environmental awareness task of autonomous driving scenarios. At present, the accuracy of 3D object detection has significant improvement potential. In addition, a 3D point cloud is not uniformly distributed on a regular grid because of its disorder, dispersion, and sparseness. The strategy of the convolution neural networks (CNNs) for 3D point cloud feature extraction has the limitations of potential information loss and empty operation. Therefore, we propose a graph neural network (GNN) detector based on neighbor feature alignment mechanism for 3D object detection in LiDAR point clouds. This method exploits the structural information of graphs, and it aggregates the neighbor and edge features to update the state of vertices during the iteration process. This method enables the reduction of the offset error of the vertices, and ensures the invariance of the point cloud in the spatial domain. For experiments performed on the KITTI public benchmark, the results demonstrate that the proposed method achieves competitive experimental results.

APP-Net: Auxiliary-Point-Based Push and Pull Operations for Efficient Point Cloud Recognition.

Aggregating neighbor features is essential for point cloud neural network. In the existing work, each point in the cloud may inevitably be selected as the neighbors of multiple aggregation centers, as all centers will gather neighbor features from the whole point cloud independently. Thus, each point has to participate in the calculation repeatedly, generating redundant duplicates in the memory, leading to intensive computation costs and memory consumption. Meanwhile, to pursue higher accuracy, previous methods often rely on a complex local aggregator to extract fine geometric representation, further slowing down the processing pipeline. To address these issues, we propose a new local aggregator of linear complexity for point cloud analysis, coined as APP. Specifically, we introduce an auxiliary container as an anchor to exchange features between the source point and the aggregating center. Each source point pushes its feature to only one auxiliary container, and each center point pulls features from only one auxiliary container. This avoids the re-computation issue of each source point. To facilitate the learning of the local structure of point cloud, we use an online normal estimation module to provide explainable geometric information to enhance our APP modeling capability. Our built network is more efficient than all the previous baselines with a clear margin while still consuming a lower memory. Experiments on classification and semantic segmentation demonstrate that APP-Net reaches comparable accuracies to other networks. In the classification task, it can process more than 10,000 samples per second with less than 10GB of memory on a single GPU. We will release the code at https://github.com/MCG-NJU/ APP-Net.

Automated polyp segmentation based on a multi-distance feature dissimilarity-guided fully convolutional network.

Colorectal malignancies often arise from adenomatous polyps, which typically begin as solitary, asymptomatic growths before progressing to malignancy. Colonoscopy is widely recognized as a highly efficacious clinical polyp detection method, offering valuable visual data that facilitates precise identification and subsequent removal of these tumors. Nevertheless, accurately segmenting individual polyps poses a considerable difficulty because polyps exhibit intricate and changeable characteristics, including shape, size, color, quantity and growth context during different stages. The presence of similar contextual structures around polyps significantly hampers the performance of commonly used convolutional neural network (CNN)-based automatic detection models to accurately capture valid polyp features, and these large receptive field CNN models often overlook the details of small polyps, which leads to the occurrence of false detections and missed detections. To tackle these challenges, we introduce a novel approach for automatic polyp segmentation, known as the multi-distance feature dissimilarity-guided fully convolutional network. This approach comprises three essential components, i.e., an encoder-decoder, a multi-distance difference (MDD) module and a hybrid loss (HL) module. Specifically, the MDD module primarily employs a multi-layer feature subtraction (MLFS) strategy to propagate features from the encoder to the decoder, which focuses on extracting information differences between neighboring layers' features at short distances, and both short and long-distance feature differences across layers. Drawing inspiration from pyramids, the MDD module effectively acquires discriminative features from neighboring layers or across layers in a continuous manner, which helps to strengthen feature complementary across different layers. The HL module is responsible for supervising the feature maps extracted at each layer of the network to improve prediction accuracy. Our experimental results on four challenge datasets demonstrate that the proposed approach exhibits superior automatic polyp performance in terms of the six evaluation criteria compared to five current state-of-the-art approaches.

Attentive Neighborhood Feature Augmentation for Semi-supervised Learning

Attentive Neighborhood Feature Augmentation for Semi-supervised Learning

PM2.5 hourly concentration prediction based on graph capsule networks

<abstract> <p>In this paper, we use a graph capsule network to capture the spatial dependence of air quality data and meteorological data among cities, then use an LSTM network to model the temporal dependence of air pollution levels in specific cities and finally implement PM2.5 concentration prediction. We propose a graph-capsule-LSTM model based on a graph-capsule network and an LSTM network. The model uses a graph capsule network to model the neighboring feature information of the target city and then combines the local data of the target city to form the final feature vector. The feature mapping on the time axis is then used to obtain the temporal feature sequences of the target nodes, which are fed into the LSTM network for learning and prediction. Experiments show that the method achieves better results than the latest baseline model in the PM2.5 prediction task. While demonstrating that the capsule network outperforms the convolutional network, it also shows that this capsule network is very competent for the task of PM2.5 prediction.</p> </abstract>

Neighborhood Contrastive Transformer for Change Captioning

Change captioning is to describe the semantic change between a pair of similar images in natural language. It is more challenging than general image captioning, because it requires capturing fine-grained change information while being immune to irrelevant viewpoint changes, and solving syntax ambiguity in change descriptions. In this paper, we propose a neighborhood contrastive transformer to improve the model's perceiving ability for various changes under different scenes and cognition ability for complex syntax structure. Concretely, we first design a neighboring feature aggregating to integrate neighboring context into each feature, which helps quickly locate the inconspicuous changes under the guidance of conspicuous referents. Then, we devise a common feature distilling to compare two images at neighborhood level and extract common properties from each image, so as to learn effective contrastive information between them. Finally, we introduce the explicit dependencies between words to calibrate the transformer decoder, which helps better understand complex syntax structure during training. Extensive experimental results demonstrate that the proposed method achieves the state-of-the-art performance on three public datasets with different change scenarios. The code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/tuyunbin/NCT</uri> .

A Neighborhood Re-Ranking Model With Relation Constraint for Knowledge Graph Completion

Knowledge graph completion (KGC) aims to predict missing links based on observed triples. However, current KGC models are still limited by the following two aspects. (1) the entity semantics is implicitly learned by neural network and merely depends on existing facts, which mostly suffers from less additional specific knowledge. Although previous studies have noticed that entity type information can effectively improve KGC task, most of them rely on labeled type-specific data. (2) the recent graph-based models mainly concentrate on Graph Neural Network (GNN) to update source entity representation, regardless of the separate role that neighborhood information plays and may mix noisy neighbor features for target prediction. To address the above two issues, we propose a neighborhood re-ranking model with relation constraint for KGC task. We suggest that both relation constraint and structured information located in triples can boost the model performance. More importantly, we automatically generate explicit constraints as additional type feature to enrich entity representation instead of depending on human annotated labels. Meanwhile, we construct a neighborhood completion module to re-rank candidate entities for full use of the neighbor structure rather than traditional GNN updating manner. Extensive experiments on seven benchmarks demonstrate that our model achieves the competitive results in comparison to the recent advanced baselines.

A Remote Sensing Method for Crop Mapping Based on Multiscale Neighborhood Feature Extraction

Obtaining accurate and timely crop mapping is essential for refined agricultural refinement and food security. Due to the spectral similarity between different crops, the influence of image resolution, the boundary blur and spatial inconsistency that often occur in remotely sensed crop mapping, remotely sensed crop mapping still faces great challenges. In this article, we propose to extend a neighborhood window centered on the target pixel to enhance the receptive field of our model and extract the spatial and spectral features of different neighborhood sizes through a multiscale network. In addition, we also designed a coordinate convolutional module and a convolutional block attention module to further enhance the spatial information and spectral features in the neighborhoods. Our experimental results show that this method allowed us to obtain accuracy scores of 0.9481, 0.9115, 0.9307 and 0.8729 for OA, kappa coefficient, F1 score and IOU, respectively, which were better than those obtained using other methods (Resnet-18, MLP and RFC). The comparison of the experimental results obtained from different neighborhood window sizes shows that the spatial inconsistency and boundary blurring in crop mapping could be effectively reduced by extending the neighborhood windows. It was also shown in the ablation experiments that the coordinate convolutional and convolutional block attention modules played active roles in the network. Therefore, the method proposed in this article could provide reliable technical support for remotely sensed crop mapping.

GSNNet: Group semantic-guided neighbor interaction network for co-salient object detection

The critical challenge for co-salient object detection (CoSOD) is to extract common saliency information from a group of relevant images. Most of the existing CoSOD methods do not fully explore the semantic commonality of co-salient objects, which can be strong guidance for collaborative feature learning, and do not take full advantage of rich hierarchical features of different layers, resulting in inferior performance. To this end, we propose a Group Semantic-guided Neighbor interaction network (GSNNet) for co-salient object detection. Specifically, the proposed network contains the group semantic module (GSM), neighbor interaction module (NIM), and feature enhancement module (FEM). The network first learns semantic consensus from a group of relevant images by the GSM, which uses the reverse guidance strategy and the group-wise combination strategy to distill the group semantic cues from the forward and complementary features. With the powerful guidance of group semantic, the NIM is employed to conduct the neighbor feature interaction of adjacent layers to excavate the contextual information and enhance feature representation. Then the FEM is adopted to refine the critical cues by the attention mechanism, which enhances the compactness of feature representation. The proposed GSNNet is evaluated on three challenging CoSOD benchmark datasets using four widely-used metrics, which demonstrates that our proposed method is superior to the other twelve cutting-edge methods for co-salient object detection.

Group-based bi-directional recurrent wavelet neural network for efficient video super-resolution (VSR)

Video super-resolution (VSR) is an important technology for enhancing the quality of video frames. The recurrent neural network (RNN)-based approach is suitable for sequential data because it can use accumulated temporal information. However, since existing methods only tend to capture slow and symmetrical motion with low frame rate, there are still limitations to restore the missing details for more dynamic motion. Most of the previous methods using spatial information treat different types of the spatial features identically. It leads to lack of obtaining meaningful information and enhancing the fine details. We propose a group-based bi-directional recurrent wavelet neural network (GBR-WNN) to exploit spatio-temporal information effectively. The proposed group-based bi-directional RNN (GBR) framework is built on the well-structured process with the group of pictures (GOP). In a GOP, we resolves the low-resolution (LR) frames from border frames to center target frame. Because super-resolved features in a GOP are cumulative, neighboring features are improved progressively and asymmetrical motion can be dealt with. Also, we propose a temporal wavelet attention (TWA) adopting attention module for both spatial and temporal features simultaneously based on discrete wavelet transform. Experiments show that the proposed scheme achieves superior performance compared with state-of-the-art methods.

An improved GraphSAGE to detect power system anomaly based on time-neighbor feature

With the development of smart grids, more and more information appears in power systems. Therefore, information security has attracted more attention, since malicious attacks aimed at power systems are increasing and causing heavy losses. However, the current research on data identification of power systems has not considered hidden false data. Therefore, a time-neighbor-based GraphSAGE anomaly detection model is constructed by this paper, which can detect the features of the target node type from the neighbor features and its own time series features. More importantly, it can effectively detect hidden anomalies in the data, and provide more secure data for power system operation. First, in order to detect the authenticity of the data, the similarities between the features of the target node and the neighbor nodes are calculated, and the similarity of the features of the target node and the neighbor nodes varies with time. Second, the GraphSAGE introduced the neighbor information of the target node, and used the similarity to supervise the training of the GraphSAGE. Finally, the model this paper constructed is verified on the public datasets and compared with the traditional anomaly detection method. The results show that the time-neighbor-based GraphSAGE anomaly detection can effectively detect abnormal data, the Recall and Auc indicators increased by 0.2 and 0.3 respectively using this anomaly detection method.

A Multiscale Multi-Feature Deep Learning Model for Airborne Point-Cloud Semantic Segmentation

In point-cloud scenes, semantic segmentation is the basis for achieving an understanding of a 3D scene. The disorderly and irregular nature of 3D point clouds makes it impossible for traditional convolutional neural networks to be applied directly, and most deep learning point-cloud models often suffer from an inadequate utilization of spatial information and of other related point-cloud features. Therefore, to facilitate the capture of spatial point neighborhood information and obtain better performance in point-cloud analysis for point-cloud semantic segmentation, a multiscale, multi-feature PointNet (MSMF-PointNet) deep learning point-cloud model is proposed in this paper. MSMF-PointNet is based on the classical point-cloud model PointNet, and two small feature-extraction networks called Mini-PointNets are added to operate in parallel with the modified PointNet; these additional networks extract multiscale, multi-neighborhood features for classification. In this paper, we use the spherical neighborhood method to obtain the local neighborhood features of the point cloud, and then we adjust the radius of the spherical neighborhood to obtain the multiscale point-cloud features. The obtained multiscale neighborhood feature point set is used as the input of the network. In this paper, a cross-sectional comparison analysis is conducted on the Vaihingen urban test dataset from the single-scale and single-feature perspectives.

Dark Spot Detection from SAR Images Based on Superpixel Deeper Graph Convolutional Network

Synthetic Aperture Radar (SAR) is the primary equipment used to detect oil slicks on the ocean’s surface. On SAR images, oil spill regions, as well as other places impacted by atmospheric and oceanic phenomena such as rain cells, upwellings, and internal waves, appear as dark spots. Dark spot detection is typically the initial stage in the identification of oil spills. Because the identified dark spots are oil slick candidates, the quality of dark spot segmentation will eventually impact the accuracy of oil slick identification. Although certain sophisticated deep learning approaches employing pixels as primary processing units work well in remote sensing image semantic segmentation, finding some dark patches with weak boundaries and small regions from noisy SAR images remains a significant difficulty. In light of the foregoing, this paper proposes a dark spot detection method based on superpixels and deeper graph convolutional networks (SGDCNs), with superpixels serving as processing units. The contours of dark spots can be better detected after superpixel segmentation, and the noise in the SAR image can also be smoothed. Furthermore, features derived from superpixel regions are more robust than those derived from fixed pixel neighborhoods. Using the support vector machine recursive feature elimination (SVM-RFE) feature selection algorithm, we obtain an excellent subset of superpixel features for segmentation to reduce the learning task difficulty. After that, the SAR images are transformed into graphs with superpixels as nodes, which are fed into the deeper graph convolutional neural network for node classification. SGDCN leverages a differentiable aggregation function to aggregate the node and neighbor features to form more advanced features. To validate our method, we manually annotated six typical large-scale SAR images covering the Baltic Sea and constructed a dark spot detection dataset. The experimental results demonstrate that our proposed SGDCN is robust and effective compared with several competitive baselines. This dataset has been made publicly available along with this paper.

M[formula omitted]RNet: Multi-modal and multi-scale refined network for RGB-D salient object detection

Salient object detection is a fundamental topic in computer vision, which has promising application prospects. The previous methods based on RGB-D may potentially suffer from the incompatibility of multi-modal feature fusion and the insufficiency of multi-scale feature aggregation. To tackle these two dilemmas, we propose a novel multi-modal and multi-scale refined network (M2RNet). Specifically, three essential components are presented in this network. The nested dual attention module (NDAM) explicitly exploits the combined features of RGB and depth flows. The adjacent interactive aggregation module (AIAM) gradually integrates the neighbor features of high, middle and low levels. The joint hybrid optimization loss (JHOL) makes the predictions have a prominent outline. Extensive experiments quantitatively and qualitatively demonstrate that our method outperforms other state-of-the-art approaches.

An Embedding Model for Knowledge Graph Completion Based on Graph Sub-Hop Convolutional Network

The research on knowledge graph completion based on representation learning is increasingly dependent on the node structural feature in the graph. However, a large number of nodes have few immediate neighbors, resulting in the node features unable to be fully expressed. Hence, multi-hop structure features are crucial to the representation learning of nodes. GCN (Graph Convolutional Network) is a graph embedding model that can introduce the multi-hop structure. However, the multi-hop information transmitted between GCN layers suffers a lot of losses. This would lead to the insufficient mining of the node structure features and semantic feature association among entities, further reducing the efficiency of graph knowledge completion. A gate-controlled graph sub-hop convolutional network model for knowledge graph completion is proposed to fill these research gaps. Firstly, a graph sub-hop convolutional network based on matrix representation is designed, which can transmit multi-hop neighbor features directly to the encoded node vector to avoid a large loss of features during multi-hop transmission. On this basis, the implicit multi-hop relations are explicitly embedded into the model based on the TransE. In the process of each hop convolution, aiming at the accumulation of noise redundancy caused by the increase of the receptive field, a sub-hop gate mechanism strategy is proposed to filter information. Finally, the linear model is used to decode the encoded nodes and then complete the knowledge graph. We carried out experimental comparison and analysis on WN18RR, FB15k-237, UMLS, and KINSHIP datasets. The results show that the embedding method based on the sub-hop structural information fusion can greatly improve the results of link prediction.

PU-WGCN: Point Cloud Upsampling Using Weighted Graph Convolutional Networks

Point clouds are sparse and unevenly distributed, which makes upsampling a challenging task. The current upsampling algorithm encounters the problem that neighboring nodes are similar in terms of specific features, which tends to produce hole overfilling and boundary blurring. The local feature variability of the point cloud is small, and the aggregated neighborhood feature operation treats all neighboring nodes equally. These two reasons make the local node features too similar. We designed the graph feature enhancement module to reduce the similarity between different nodes as a solution to the problem. In addition, we calculate the feature similarity between neighboring nodes based on both spatial information and features of the point cloud, which is used as the boundary weight of the point cloud graph to solve the problem of boundary blurring. We fuse the graph feature enhancement module with the boundary information weighting module to form the weighted graph convolutional networks (WGCN). Finally, we combine the WGCN module with the upsampling module to form a point cloud upsampling network named PU-WGCN. Compared with other upsampling networks, the experimental results show that PU-WGCN can solve the problems of hole overfilling and boundary blurring and improve the upsampling accuracy.

A transfer-learning fault diagnosis method considering nearest neighbor feature constraints

Aiming at the problem of low diagnostic accuracy of fault diagnosis models due to changes in actual operating conditions, a novel fault diagnosis method based on transfer learning considering nearest neighbor feature constraints is proposed. First, nearest neighbor samples are considered to measure data features. In addition, a nearest neighbor feature constraint strategy is designed to improve the feature extraction performance of the network. Second, a multiple-alignment strategy of nearest neighbor samples is proposed to enhance the domain adaptation performance of the network model utilizing multiple alignments. Then, a loss function dynamic weight strategy is used to improve the convergence of the loss function during model training. Finally, the experimental verification is carried out on the public data set of the Western Reserve University and the private data set. The experimental results show that the proposed method exhibits superior transfer performance with reliability and stability compared to the existing methods.

End-to-End Learning to Grasp via Sampling From Object Point Clouds

The ability to grasp objects is an essential skill that enables many robotic manipulation tasks. Recent works have studied point cloud-based methods for object grasping by starting from simulated datasets and have shown promising performance in real-world scenarios. Nevertheless, many of them still rely on ad-hoc geometric heuristics to generate grasp candidates, which fail to generalize to objects with significantly different shapes with respect to those observed during training. Several approaches exploit complex multi-stage learning strategies and local neighborhood feature extraction while ignoring semantic global information. Furthermore, they are inefficient in terms of number of training samples and time required for inference. In this letter, we propose an end-to-end learning solution to generate 6-DOF parallel-jaw grasps starting from the 3D partial view of the object. Our Learning to Grasp (L2G) method gathers information from the input point cloud through a new procedure that combines a differentiable sampling strategy to identify the visible contact points, with a feature encoder that leverages local and global cues. Overall, L2G is guided by a multi-task objective that generates a diverse set of grasps by optimizing contact point sampling, grasp regression, and grasp classification. With a thorough experimental analysis, we show the effectiveness of L2G as well as its robustness and generalization abilities.

Javanese Character Recognition Based on K-Nearest Neighbor and Linear Binary Pattern Features

Javanese script (Hanacaraka) is one of the cultures owned by Indonesia. Javanese script is found in temples, inscriptions, cultural and prehistoric sites, ancient Javanese manuscripts, Gulden series banknotes, street signage, and palace documents. Javanese script has a form with an article, and the use of reading above the script is a factor that affects the character detection process. Punctuation marks, clothing, Swara script, vowels, and consonants are parts of the script that are often found in Javanetest scripts. Preserving Javanese script in the digital era, of course, must use technology that can support the digitization of Javanese script through the script detection process. The concept of script image is the image of Javanese script in ancient manuscripts. The process of character detection using certain techniques can be carried out to extract characters so that they can be read. Detection of Javanese characters can be found by finding a testing image. Here, we had been used 10 words images consisting of 3 to 5 syllables with the vowel aiu. Dataset process by Linear Binary Pattern (LBP) feature extraction, which is used to characterize images and describe image textures locally. LBP has been used in r=4 and preprocessing is also done by thresholding with d=0.3. This process can be done using the K-Nearest Neighbor algorithm. In 10 datasets of Javanese script words, an average accuracy value of 90.5% was obtained. The accuracy value of 100% is the highest and 50% is the lowest.

PV-RCNN++: semantical point-voxel feature interaction for 3D object detection

Large imbalance often exists between the foreground points (i.e., objects) and the background points in outdoor LiDAR point clouds. It hinders cutting-edge detectors from focusing on informative areas to produce accurate 3D object detection results. This paper proposes a novel object detection network by semantical point-voxel feature interaction, dubbed PV-RCNN++. Unlike most of existing methods, PV-RCNN++ explores the semantic information to enhance the quality of object detection. First, a semantic segmentation module is proposed to retain more discriminative foreground keypoints. Such a module will guide our PV-RCNN++ to integrate more object-related point-wise and voxel-wise features in the pivotal areas. Then, to make points and voxels interact efficiently, we utilize voxel query based on Manhattan distance to quickly sample voxel-wise features around keypoints. Such the voxel query will reduce the time complexity from O(N) to O(K), compared to the ball query. Further, to avoid being stuck in learning only local features, an attention-based residual PointNet module is designed to expand the receptive field to adaptively aggregate the neighboring voxel-wise features into keypoints. Extensive experiments on the KITTI dataset show that PV-RCNN++ achieves 81.60\(\%\), 40.18\(\%\), 68.21\(\%\) 3D mAP on Car, Pedestrian, and Cyclist, achieving comparable or even better performance to the state-of-the-arts.