Local Aggregation Research Articles

GLCANet: Global–Local Context Aggregation Network for Cropland Segmentation from Multi-Source Remote Sensing Images

Cropland is a fundamental basis for agricultural development and a prerequisite for ensuring food security. The segmentation and extraction of croplands using remote sensing images are important measures and prerequisites for detecting and protecting farmland. This study addresses the challenges of diverse image sources, multi-scale representations of cropland, and the confusion of features between croplands and other land types in large-area remote sensing image information extraction. To this end, a multi-source self-annotated dataset was developed using satellite images from GaoFen-2, GaoFen-7, and WorldView, which was integrated with public datasets GID and LoveDA to create the CRMS dataset. A novel semantic segmentation network, the Global–Local Context Aggregation Network (GLCANet), was proposed. This method integrates the Bilateral Feature Encoder (BFE) of CNNs and Transformers with a global–local information mining module (GLM) to enhance global context extraction and improve cropland separability. It also employs a multi-scale progressive upsampling structure (MPUS) to refine the accuracy of diverse arable land representations from multi-source imagery. To tackle the issue of inconsistent features within the cropland class, a loss function based on hard sample mining and multi-scale features was constructed. The experimental results demonstrate that GLCANet improves OA and mIoU by 3.2% and 2.6%, respectively, compared to the existing advanced networks on the CRMS dataset. Additionally, the proposed method also demonstrated high precision and practicality in segmenting large-area croplands in Chongzhou City, Sichuan Province, China.

Geological remote sensing interpretation via a local-to-global sensitive feature fusion network

Interpreting surface geological elements (such as rocks, minerals, soils, and water bodies) is the main task of geological survey, which plays a crucial role in geological environment remote sensing (GERS). However, the characteristics of geological elements, including high variabilities, various morphology, complicated boundaries and imbalanced class distribution, make it still a challenge for deep learning methods to interpret GERS images. Considering the correlations of geological elements as the regionalized variables in geostatistics, the sensitive features of GERS interpretation mainly include three aspects: tonal, textural and structural characteristics within a singular-class elements, spatial and spectral correlations of adjacent elements, and their global tectonic or spatial distribution. Thus, to simulate the manual interpretation process of geologists from local to global and promote GERS interpretation performance, we propose a local-to-global multi-scale feature fusion network (LGMSFNet). A geological object context represents the intra-class semantic dependencies of pixel sets with the same class. And a local feature aggregation module models the channel and spatial association. Then discriminative features are integrated by a global feature fusion module. For the model optimization, we focus on hard examples during the training process to achieve the balanced optimization of various categories. Two research areas that include large-scale rocks, soils and water exposed on the surface are selected. Massive experiments demonstrate the superiority of the LGMSFNet in GERS interpretation.

FedFAIM: A Model Performance-Based Fair Incentive Mechanism for Federated Learning

Federated Learning (FL) has emerged as a privacy-preserving distributed machine learning paradigm. To motivate data owners to contribute towards FL, research on FL incentive mechanisms is gaining great interest. Existing monetary incentive mechanisms generally share the same FL model with all participants regardless of their contributions. Such an assumption can be unfair towards participants who contributed more and promote undesirable free-riding, especially when the final model is of great utility value to participants. In this paper, we propose a Fairness-Aware Incentive Mechanism for federated learning (FedFAIM) to address such problem. It satisfies two types of fairness notion: 1) aggregation fairness, which determines aggregation results according to data quality; 2) reward fairness, which assigns each participant a unique model with performance reflecting his contribution. Aggregation fairness is achieved through efficient gradient aggregation which examines local gradient quality and aggregates them based on data quality. Reward fairness is achieved through an efficient Shapley value-based contribution assessment method and a novel reward allocation method based on reputation and distribution of local and global gradients. We further prove reward fairness is theoretically guaranteed. Extensive experiments show that FedFAIM provides stronger incentives than similar non-monetary FL incentive mechanisms while achieving a high level of fairness.

A Survey on Graph Neural Networks and Graph Transformers in Computer Vision: A Task-Oriented Perspective.

Graph Neural Networks (GNNs) have gained momentum in graph representation learning and boosted the state of the art in a variety of areas, such as data mining (e.g., social network analysis and recommender systems), computer vision (e.g., object detection and point cloud learning), and natural language processing (e.g., relation extraction and sequence learning), to name a few. With the emergence of Transformers in natural language processing and computer vision, graph Transformers embed a graph structure into the Transformer architecture to overcome the limitations of local neighborhood aggregation while avoiding strict structural inductive biases. In this paper, we present a comprehensive review of GNNs and graph Transformers in computer vision from a task-oriented perspective. Specifically, we divide their applications in computer vision into five categories according to the modality of input data, i.e., 2D natural images, videos, 3D data, vision + language, and medical images. In each category, we further divide the applications according to a set of vision tasks. Such a task-oriented taxonomy allows us to examine how each task is tackled by different GNN-based approaches and how well these approaches perform. Based on the necessary preliminaries, we provide the definitions and challenges of the tasks, in-depth coverage of the representative approaches, as well as discussions regarding insights, limitations, and future directions.

Influences of the stress ratio and local micro mineral aggregates on small fatigue crack propagation in the shale containing bedding planes

Due to the existence of bedding planes in shale oil reservoirs, the complexity of crack networks created by conventional hydraulic fracturing (HF) technology is limited, resulting in low oil production. In this paper, a fatigue loading method was proposed to increase the complexity of the cracking network. The propagation behaviors of small cracks were investigated and compared under both monotonic and fatigue loading conditions by using SEM in-situ three-point bending tests. In addition, the influences of local micro mineral aggregates and stress ratios (R = 0.1, 0.3 and 0.5) on the propagation behavior of small fatigue cracks (SFCs) in shale were quantitatively evaluated. The SEM in-situ fatigue tests demonstrated the effectiveness of fatigue loading in enabling SFC to propagate through the bedding plane and increase crack network complexity from a micro perspective. A fitting exponential formula of the stress intensity factor range threshold ΔKth and stress ratio R was obtained. This study assists us in deeply understanding the influence of micro mineral aggregates and stress ratio on the SFC propagation mechanism and its implications for HF design in shale oil reservoirs.

Reliable federated learning based on delayed gradient aggregation for intelligent connected vehicles

As an organic combination of the Internet of Vehicles and intelligent vehicles, Intelligent Connected Vehicles (ICVs) have very high research and application value. Traditional data application methods require the local aggregation of sensitive user data, which poses a threat to user data privacy. Federated learning (FL) is a promising machine learning method that leverages distributed, personalized datasets to enhance performance while preserving user privacy. However, in mobile environments, unreliable client data can degrade the global model, reducing accuracy. Additionally, the mobility of ICVs can destabilize the training process, prolonging model updates and diminishing aggregation accuracy. To address these challenges, this paper proposes a dynamic asynchronous aggregation method that improves both reliability and training efficiency in FL for mobile networks. Therefore, it becomes crucial to find reliable aggregation of mobile device participation in FL tasks. To this end, we propose a reliable FL scheme, which only selects reliable mobile devices to participate in model aggregation to improve the generalization ability of the model. In addition, we design a dynamic asynchronous aggregation method based on reputation scores without affecting the model. Reduce model training time without compromising performance. Through experimental analysis, it is proved that this method can improve the reliability and effectiveness of FL tasks in mobile networks.

MGCNet: Multi-granularity consensus network for remote sensing image correspondence pruning

Correspondence pruning aims to remove false correspondences (outliers) from an initial putative correspondence set. This process holds significant importance and serves as a fundamental step in various applications within the fields of remote sensing and photogrammetry. The presence of noise, illumination changes, and small overlaps in remote sensing images frequently result in a substantial number of outliers within the initial set, thereby rendering the correspondence pruning notably challenging. Although the spatial consensus of correspondences has been widely used to determine the correctness of each correspondence, achieving uniform consensus can be challenging due to the uneven distribution of correspondences. Existing works have mainly focused on either local or global consensus, with a very small perspective or large perspective, respectively. They often ignore the moderate perspective between local and global consensus, called group consensus, which serves as a buffering organization from local to global consensus, hence leading to insufficient correspondence consensus aggregation. To address this issue, we propose a multi-granularity consensus network (MGCNet) to achieve consensus across regions of different scales, which leverages local, group, and global consensus to accomplish robust and accurate correspondence pruning. Specifically, we introduce a GroupGCN module that randomly divides the initial correspondences into several groups and then focuses on group consensus and acts as a buffer organization from local to global consensus. Additionally, we propose a Multi-level Local Feature Aggregation Module that adapts to the size of the local neighborhood to capture local consensus and a Multi-order Global Feature Module to enhance the richness of the global consensus. Experimental results demonstrate that MGCNet outperforms state-of-the-art methods on various tasks, highlighting the superiority and great generalization of our method. In particular, we achieve 3.95% and 8.5% mAP5° improvement without RANSAC on the YFCC100M dataset in known and unknown scenes for pose estimation, compared to the second-best models (MSA-LFC and CLNet). Source code: https://github.com/1211193023/MGCNet.

Learning to zoom: Exploiting mixed-scale contextual information for object detection

With the development of deep neural networks, object detection has made a significant leap. However, apart from intrinsic intra and inter class differences, the objects in real-world scenarios may encounter diverse scales, blurred appearance or even severe occlusion. Inspired by the human behavior of looking at blurred images, i.e., zooming in/out, we propose a Mixed-Scale Network (dubbed for MSNet) to tackle these issues. Specifically, MSNet employs the zoom strategy to exploit mixed-scale contextual information, which fully unleashes the representation ability of deep neural networks. Firstly, the global feature aggregation module and global feature enhancement module aims at aggregating mixed-scale features from the global perspective, which learns the transform offset of pixels to align the higher-res features contextually. Moreover, the local feature aggregation module enriches the instance-level feature by adaptively aligning the features of different receptive fields. Extensive experiments on MS COCO demonstrated the effectiveness of MSNet, yielding significant improvements of 0.7–2.8 points in APbox over baselines when paired with different detectors, backbones, and schedules. In addition, for pixel-level prediction tasks including semantic segmentation and instance segmentation, the proposed method gains consistent improvements of 0.6–2.7 points in mIoU/APmask over baselines, which substantiates the effectiveness of MSNet further.

Thangka image captioning model with Salient Attention and Local Interaction Aggregator

Thangka image captioning aims to automatically generate accurate and complete sentences that describe the main content of Thangka images. However, existing methods fall short in capturing the features of the core deity regions and the surrounding background details of Thangka images, and they significantly lack an understanding of local actions and interactions within the images. To address these issues, this paper proposes a Thangka image captioning model based on Salient Attention and Local Interaction Aggregator (SALIA). The model is designed with a Dual-Branch Salient Attention Module (DBSA) to accurately capture the expressions, decorations of the deity, and descriptive background elements, and it introduces a Local Interaction Aggregator (LIA) to achieve detailed analysis of the characters’ actions, facial expressions, and the complex interactions with surrounding elements in Thangka images. Experimental results show that SALIA outperforms other state-of-the-art methods in both qualitative and quantitative evaluations of Thangka image captioning, achieving BLEU4: 94.0%, ROUGE_L: 95.0%, and CIDEr: 909.8% on the D-Thangka dataset, and BLEU4: 22.2% and ROUGE_L: 47.2% on the Flickr8k dataset.

Lightweight Neural Network for Centroid Detection of Weak, Small Infrared Targets via Background Matching in Complex Scenes

In applications such as aerial object interception and ballistic estimation, it is crucial to precisely detect the centroid position of the target rather than to merely identify the position of the target bounding box or segment all pixels belonging to the target. Due to the typically long distances between targets and imaging devices in such scenarios, targets often exhibit a low contrast and appear as dim, obscure shapes in infrared images, which represents a challenge for human observation. To rapidly and accurately detect small targets, this paper proposes a lightweight, end-to-end detection network for small infrared targets. Unlike existing methods, the input of this network is five consecutive images after background matching. This design significantly improves the network’s ability to extract target motion features and effectively reduces the interference of static backgrounds. The network mainly consists of a local feature aggregation module (LFAM), which uses multiple-sized convolution kernels to capture multi-scale features in parallel and integrates multiple spatial attention mechanisms to achieve accurate feature fusion and effective background suppression, thereby enhancing the ability to detect small targets. To improve the accuracy of predicted target centroids, a centroid correction algorithm is designed. In summary, this paper presents a lightweight centroid detection network based on background matching for weak, small infrared targets. The experimental results show that, compared to directly inputting a sequence of images into the neural network, inputting a sequence of images processed by background matching can increase the detection rate by 9.88%. Using the centroid correction algorithm proposed in this paper can therefore improve the centroid localization accuracy by 0.0134.

Evolution of Frustrated Coordination in Eutectic Electrolyte Driven by Ligand Asymmetry toward High‐Performance Zinc Batteries

AbstractEutectic electrolytes hold promise for aqueous zinc metal batteries in sustainable energy storage chemistries, yet improvement from perspective of molecule configurational engineering are ambiguous. Herein, we propose design strategy of increasing asymmetric molecular geometry in organic ligands to regulate frustrated coordination and disordered structure for eutectic electrolytes toward enhanced zinc metal batteries. The introduced asymmetry in eutectic component gives rise to relatively weak coordination strength and configurational disorder interaction among cation‐anion‐ligand, leading to suppressed local aggregation, steady eutectic phase and improved Zn2+ diffusion kinetics. Such highly frustrated coordination state also enables disruption of hydrogen bonding network and reinforcement of anion participation, which results in confined side reactions, decreased water activity and the formation of inorganic‐enriched solid electrolyte interphase. In comparison to highly symmetric ligands, asymmetric ligand‐involved eutectic electrolytes with configurational disorder deliver high Coulombic efficiency of 99.4 %, stabilized Zn plating/stripping of 5000 h and impressive rate capability even under harsh conditions such as small N/P, low temperature. The rationale in this work advances the deep understanding of asymmetric molecular engineering in eutectic electrolytes and showcases suitability of frustrated coordination to achieve high‐performance zinc metal batteries.

AIFormer: Adaptive Interaction Transformer for 3D Point Cloud Understanding

Recently, significant advancements have been made in 3D point cloud analysis by leveraging transformer architecture in 3D space. However, it remains challenging to effectively implement local and global learning within irregular and sparse structures of 3D point clouds. This paper presents the Adaptive Interaction Transformer (AIFormer), a novel hierarchical transformer architecture designed to enhance 3D point cloud analysis by fusing local and global features through the adaptive interaction of features. Specifically, AIFormer mainly consists of several stacked AIFormer Blocks. Each AIFormer module employs the Local Relation Aggregation Module and the Global Context Aggregation Module, respectively, to extract local details of relationships within the reference point and long-range dependencies between reference points. Then, the local and global features are fused using the Adaptive Interaction Module for adaptive interaction to optimize the point representation. Additionally, the AIFormer Block further designs geometric relation functions and contextual relative semantic encoding to enhance local and global feature extraction capabilities, respectively. Extensive experiments on three popular 3D point cloud datasets verify that AIFormer achieves state-of-the-art or comparable performances. Our comprehensive ablation study further validates the effectiveness and soundness of the AIFormer design.

Synchronization-Inspired Interpretable Neural Networks.

Synchronization is a ubiquitous phenomenon in nature that enables the orderly presentation of information. In the human brain, for instance, functional modules such as the visual, motor, and language cortices form through neuronal synchronization. Inspired by biological brains and previous neuroscience studies, we propose an interpretable neural network incorporating a synchronization mechanism. The basic idea is to constrain each neuron, such as a convolution filter, to capture a single semantic pattern while synchronizing similar neurons to facilitate the formation of interpretable functional modules. Specifically, we regularize the activation map of a neuron to surround its focus position of the activated pattern in a sample. Moreover, neurons locally interact with each other, and similar ones are synchronized together during the training phase adaptively. Such local aggregation preserves the globally distributed representation nature of the neural network model, enabling a reasonably interpretable representation. To analyze the neuron interpretability comprehensively, we introduce a series of novel evaluation metrics from multiple aspects. Qualitative and quantitative experiments demonstrate that the proposed method outperforms many state-of-the-art algorithms in terms of interpretability. The resulting synchronized functional modules show module consistency across data and semantic specificity within modules.

Two for tau: Automated model discovery reveals two-stage tau aggregation dynamics in Alzheimer’s disease

Alzheimer’s disease is a neurodegenerative disorder characterized by the presence of amyloid-β plaques and the accumulation of misfolded tau proteins and neurofibrillary tangles in the brain. A thorough understanding of the local accumulation of tau is critical to develop effective therapeutic strategies. Tau pathology has traditionally been described using reaction–diffusion models, which succeed in capturing the global spread, but fail to accurately describe the local aggregation dynamics. Current mathematical models enforce a single-peak behavior in tau aggregation, which does not align well with clinical observations. Here we identify a more accurate description of tau aggregation that reflects the complex patterns observed in patients. We propose an innovative approach that uses constitutive neural networks to autonomously discover bell-shaped aggregation functions with multiple peaks from clinical positron emission tomography (PET) data of misfolded tau protein. Our method reveals previously overlooked two-stage aggregation dynamics by uncovering a two-term ordinary differential equation that links the local accumulation rate to the tau concentration. When trained on data from amyloid-β positive and negative subjects, the neural network clearly distinguishes between both groups and uncovers a more subtle relationship between amyloid-β and tau than previously postulated. In line with the amyloid-tau dual pathway hypothesis, our results show that the presence of toxic amyloid-β influences the accumulation of tau, particularly in the earlier disease stages. We expect that our approach to autonomously discover the accumulation dynamics of pathological proteins will improve simulations of tau dynamics in Alzheimer’s disease and provide new insights into disease progression.Significance StatementIn Alzheimer’s disease, understanding the local dynamics of tau protein aggregation is crucial for developing effective treatments. Traditional models for tau protein dynamics use reaction-diffusion models that fail to accurately capture these local patterns. Our study introduces a novel approach that leverages constitutive neural networks to autonomously discover the complex, multi-peak aggregation dynamics from clinical PET data. This method reveals a previously overlooked two-stage tau accumulation process and a nuanced relationship between amyloid-β and tau. By distinguishing between amyloid-β positive and negative subjects, our model supports the amyloid-tau dual pathway hypothesis and offers novel insights into tau protein aggregation that have the potent to advance our understanding of Alzheimer’s disease progression.

Automatic Apple Detection and Counting with AD-YOLO and MR-SORT.

In the production management of agriculture, accurate fruit counting plays a vital role in the orchard yield estimation and appropriate production decisions. Although recent tracking-by-detection algorithms have emerged as a promising fruit-counting method, they still cannot completely avoid fruit occlusion and light variations in complex orchard environments, and it is difficult to realize automatic and accurate apple counting. In this paper, a video-based multiple-object tracking method, MR-SORT (Multiple Rematching SORT), is proposed based on the improved YOLOv8 and BoT-SORT. First, we propose the AD-YOLO model, which aims to reduce the number of incorrect detections during object tracking. In the YOLOv8s backbone network, an Omni-dimensional Dynamic Convolution (ODConv) module is used to extract local feature information and enhance the model's ability better; a Global Attention Mechanism (GAM) is introduced to improve the detection ability of a foreground object (apple) in the whole image; a Soft Spatial Pyramid Pooling Layer (SSPPL) is designed to reduce the feature information dispersion and increase the sensory field of the network. Then, the improved BoT-SORT algorithm is proposed by fusing the verification mechanism, SURF feature descriptors, and the Vector of Local Aggregate Descriptors (VLAD) algorithm, which can match apples more accurately in adjacent video frames and reduce the probability of ID switching in the tracking process. The results show that the mAP metrics of the proposed AD-YOLO model are 3.1% higher than those of the YOLOv8 model, reaching 96.4%. The improved tracking algorithm has 297 fewer ID switches, which is 35.6% less than the original algorithm. The multiple-object tracking accuracy of the improved algorithm reached 85.6%, and the average counting error was reduced to 0.07. The coefficient of determination R2 between the ground truth and the predicted value reached 0.98. The above metrics show that our method can give more accurate counting results for apples and even other types of fruit.

Evolution of Frustrated Coordination in Eutectic Electrolyte Driven by Ligand Asymmetry toward High-Performance Zinc Batteries.

Eutectic electrolytes hold promise for aqueous zinc metal batteries in sustainable energy storage chemistries, yet improvement from perspective of molecule configurational engineering are ambiguous. Herein, we propose design strategy of increasing asymmetric molecular geometry in organic ligands to regulate frustrated coordination and disordered structure for eutectic electrolytes toward enhanced zinc metal batteries. The introduced asymmetry in eutectic component gives rise to relatively weak coordination strength and configurational disorder interaction among cation-anion-ligand, leading to suppressed local aggregation, steady eutectic phase and improved Zn2+diffusion kinetics. Such highly frustrated coordination state also enables disruption of hydrogen bonding network and reinforcement of anion participation, which results in confined side reactions, decreased water activity and the formation of inorganic-enriched solid electrolyte interphase. In comparison to highly symmetric ligands, asymmetric ligand-involved eutectic electrolytes with configurational disorderdeliver high Coulombic efficiency of 99.4 %, stabilized Zn plating/stripping of 5000 h and impressive rate capability even under harsh conditions such as small N/P, low temperature. The rationale in this work advances the deep understanding of asymmetric molecular engineering in eutectic electrolytes and showcases suitability of frustrated coordination to achieve high-performance zinc metal batteries.

PillarVTP: vehicle trajectory prediction method based on local point cloud aggregation and receptive field expansion

PillarVTP: vehicle trajectory prediction method based on local point cloud aggregation and receptive field expansion

Flow of temporal network properties under local aggregation and time shuffling: a tool for characterizing, comparing and classifying temporal networks

Abstract Although many tools have been developed and employed to characterize temporal networks (TNs), the issue of how to compare them remains largely open. It depends indeed on what features are considered as relevant, and on the way the differences in these features are quantified. In this paper, we propose to characterize TNs through their behavior under general transformations that are local in time: (i) a local time shuffling, which destroys correlations at time scales smaller than a given scale b, while preserving large time scales, and (ii) a local temporal aggregation on time windows of length n. By varying b and n, we obtain a flow of TNs, and flows of observable values, which encode the phenomenology of the TN on multiple time scales. We use a symbolic approach to summarize these flows into labels (strings of characters) describing their trends. These labels can then be used to compare TNs, validate models, or identify groups of networks with similar labels. Our procedure can be applied to any TN and with an arbitrary set of observables, and we illustrate it on an ensemble of data sets describing face-to-face interactions in various contexts, including both empirical and synthetic data.

The impact of particle size and concentration on the characteristics of solid–fluid two-phase flow in vertical pipe under pulsating flow

Vertical pipe hydraulic conveying is currently considered the most promising method for deep-sea mining. The conveying velocity at the inlet of the vertical pipe is often close to a pulsating form due to the influence of the flexible pipe and the marine environment. This study employs a combined Computational Fluid Dynamics (CFD) and Discrete Element Method (DEM) approach to investigate the impact of various particle sizes and concentrations on the flow behavior of solid–fluid two-phase flow in a vertical pipe under pulsating flow. The study reveals that fluid velocities exhibit periodic fluctuations within the vertical pipe under pulsating inlet flow. Moreover, the periodic phenomenon of local increases in particle velocity and concentration becomes more pronounced as particle size and concentration increase. Larger particle sizes increase particle inertia, reducing the ability to follow the fluid and leading to more localized particle aggregation. Therefore, the flow pattern within the vertical pipe transitions from homogeneous flow to plug flow as particle size increases. Variations in fluid–solid interaction forces and particle collision forces are explored to explain this phenomenon. When the particle size exceeds a critical threshold, the collision force surpasses the fluid–solid interaction force, leading to the transition.

Contraband detection algorithm for X‐ray security inspection images based on global semantic enhancement

AbstractAiming at the problem of low detection accuracy caused by overlapping occlusion and noise disturbance, a contraband detection algorithm for X‐ray security inspection images based on global semantic enhancement is proposed to achieve accurate contraband target detection by enhancing global semantic information. First, the disturbance suppression module is used to weaken the noise disturbance at different positions by local suppression and aggregate finer detail information. Then, the parallel cascade search module is used to capture long‐range dependencies and strengthen the representation of global semantic information, which helps the model identify contraband under overlapping occlusion. Finally, the contribution of different features is adaptively adjusted through the feature‐weighted fusion module, which promotes the effective fusion of multi‐scale features and improves the accuracy of model detection. The method in this article has been extensively evaluated and experimented on three mainstream benchmark datasets: SIXray, OPIXray, and PIDray. The mAPs of three datasets reach 93.5%, 91.9%, and 85.9%, respectively. The experimental results fully demonstrate that the method in this article has better performance compared with the latest method, which can meet the practical application requirements of real‐time target detection.