Aggregate Characteristics Research Articles

ADT: Person re-identification based on efficient attention mechanism and single-channel dual-channel fusion with transformer features aggregation

In person re-identification(Re-ID) tasks, using only convolutional neural network(CNNs) makes it challenging to fully exploit the global key information in person images, while using only the Transformer structure fails to adequately capture the multi-scale local detail information in person images. Therefore, to address the more challenging person Re-ID tasks, effectively integrating CNN and Transformer into an algorithmic framework capable of fusing multi-dimensional features has become one of the current focal points of research. Due to the ability of the attention mechanism to suppress the extraction of non-salient features by convolutional neural networks, and the capability of the dual-channel multi-scale mechanism to facilitate the extraction of finer-grained deep information by the Transformer, we propose a network Based on Efficient Attention Mechanism and Single-channel Dual-channel Fusion with Transformer Features Aggregation (ADT). First, we introduce the Convolutional Block Attention Module(CBAM) and Spatial Reconstruction Unit (SRU) into the CNN network to construct a Multi-dimensional Attention Features Re-fusion Network(MCF). By weighting the local features extracted in the convolutional network, attention to person detail features in the model is enhanced. Then, we propose the Single and Dual-channel Integrated Transformer Features Aggregator(SDT) and overlap it in MCF. This integration is designed to more effectively aggregate the multi-scale high-dimensional global features from each layer of the CNN network. Experimental results on the Market1501, DukeMTMC, MSMT17, and CUHK03 datasets demonstrate that our model significantly enriches features representation compared to most methods.

GS–CDNet: a remote sensing image cloud detection method with geographic spatial data integration

ABSTRACT In optical remote sensing images, clouds exhibit irregular scales and boundaries that vary with elevation across diverse geographical locations. To accurately capture the diverse visual patterns of clouds, we propose a cloud image segmentation approach named GS-CDNet (Geographic Spatial Data-Cloud Detection Network), which is based on the integration of geospatial data with multifaceted self-attention feature extraction, multi-scale feature aggregation, and boundary clarification techniques.Firstly, we utilize geographical coordinates from optical remote sensing images to extract a raster DEM (Digital Elevation Model) from SRTM3. This process creates a dataset consisting of elevation images, longitude, and latitude maps as geospatial data, enhancing the model’s capability in spatial positioning for cloud detection. Secondly, the proposed method consists of three interconnected modules within the cloud detection network: the Interleaved Self-Attention module(ISAM) utilizes a variety of self-attention mechanisms in an interleaved manner to extract multi-scale feature information.The Bidirectional Multi-Scale Feature Fusion Module(BIMFM) is responsible for integrating features, enabling a more comprehensive contextual understanding. The Boundary Extraction Module(BEM) utilizes a residual structure to generate a boundary cloud mask, effectively addressing the common issue of boundary blurring in multi-scale cloud masks. Finally, we compared and evaluated GS-CDNet with other cloud detection methods and conducted an ablation study on the key components of the method. The validation of generalization performance demonstrates the exceptional performance of the proposed model in cloud mask generation. Geospatial data and the different modules of the method play a significant role in the model.

A flexible deep learning framework for liver tumor diagnosis using variable multi-phase contrast-enhanced CT scans

BackgroundLiver cancer is a significant cause of cancer-related mortality worldwide and requires tailored treatment strategies for different types. However, preoperative accurate diagnosis of the type presents a challenge. This study aims to develop an automatic diagnostic model based on multi-phase contrast-enhanced CT (CECT) images to distinguish between hepatocellular carcinoma (HCC), intrahepatic cholangiocarcinoma (ICC), and normal individuals.MethodsWe designed a Hierarchical Long Short-Term Memory (H-LSTM) model, whose core components consist of a shared image feature extractor across phases, an internal LSTM for each phase, and an external LSTM across phases. The internal LSTM aggregates features from different layers of 2D CECT images, while the external LSTM aggregates features across different phases. H-LSTM can handle incomplete phases and varying numbers of CECT image layers, making it suitable for real-world decision support scenarios. Additionally, we applied phase augmentation techniques to process multi-phase CECT images, improving the model’s robustness.ResultsThe H-LSTM model achieved an overall average AUROC of 0.93 (0.90, 1.00) on the test dataset, with AUROC for HCC classification reaching 0.97 (0.93, 1.00) and for ICC classification reaching 0.90 (0.78, 1.00). Comprehensive validation in scenarios with incomplete phases was performed, with the H-LSTM model consistently achieving AUROC values over 0.9.ConclusionThe proposed H-LSTM model can be employed for classification tasks involving incomplete phases of CECT images in real-world scenarios, demonstrating high performance. This highlights the potential of AI-assisted systems in achieving accurate diagnosis and treatment of liver cancer. H-LSTM offers an effective solution for processing multi-phase data and provides practical value for clinical diagnostics.

Aggregation Behavior of CHR-bis(BODIPY) Bichromophores in THF-water Mixtures: Effect of Linking Positions and Aryl-spacer Substituents.

Aggregation-caused quenching effect (ACQ) greatly limits the practical use of many organic luminophores in biomedicine, optics and electronics. The comparative analysis of aggregation characteristics of CHR-bis(BODIPY) bichromophores 1-6 with R = H, Ph, MeOPh and various linking positions (α,α-; α,β-; β,β- and β',β'-) in THF-water mixtures with different water fractions or dye concentrations is first presented in this article. Both the linking style 1-4 and the arylation of the spacer with phenyl (Ph-) 5 or methoxyphenyl (MeOPh-) 6 substituents strongly affect the formation of luminophore aggregated forms in binary THF-water mixtures. The α,α-and β,β-isomers (1 and 3) form non-fluorescent H-type aggregates in THF-water mixtures with fw > 70%. The α,β-; β',β'-isomers (2, 4) and the MeOPh-substituted β,β-bichromophore 6 are characterized by predominant formation fluorescent aggregates. All bichromophores are characterized by the presence of residual amounts of non-aggregated forms in binary mixtures with maximum water content. The results are useful for controlling the aggregation behavior and spectral characteristics of CHR-bis(BODIPY) bichromophores in aqueous-organic media, which is important in the development of biomarkers and PDT agents.

High-modulus engineered cementitious composites: Design mechanism and performance characterization

Engineered cementitious composites features with high tensile performance while relatively weak compressive stiffness. This study endeavors to address the inherent trade-off between tensile properties and elastic modulus in conventional Engineered Cementitious Composites (ECC). Utilizing the distinctive characteristics of iron ore aggregates, known for their relatively stiff nature, smooth surface and rounded shape, an Iron Sand-based ECC (IS-ECC) emphasizing both high elastic modulus and ductility is formulated. Guided by micromechanical design theory and multiscale homogenization model, this study systematically explores the impacts of aggregate types, water-to-binder (w/b) ratios, sand-to-binder (s/b) ratios, and sand particle sizes on ECC properties. Compared with Quartz Sand-based ECC (QS-ECC), IS-ECC exhibits notably enhanced matrix fluidity and elastic modulus, reduced matrix toughness, and more robust strain-hardening behavior. The proposed three-level multiscale homogenization model accurately predicts the elastic modulus of ECC and provides insights into the underlying mechanism contributing to the enhanced elastic modulus of IS-ECC. With a resulting high elastic modulus of 33.3–48.6 GPa and superior tensile properties, IS-ECC holds promise for widespread applications in structural engineering.

Which to select?: Analysis of speaker representation with graph attention networks.

Although the recent state-of-the-art systems show almost perfect performance, analysis of speaker embeddings has been lacking thus far. An in-depth analysis of speaker representation will be performed by looking into which features are selected. To this end, various intermediate representations of the trained model are observed using graph attentive feature aggregation, which includes a graph attention layer and graph pooling layer followed by a readout operation. To do so, the TIMIT dataset, which has comparably restricted conditions (e.g., the region and phoneme) is used after pre-training the model on the VoxCeleb dataset and then freezing the weight parameters. Through extensive experiments, there is a consistent trend in speaker representation in that the models learn to exploit sequence and phoneme information despite no supervision in that direction. The results shed light to help understand speaker embedding, which is yet considered to be a black box.

Exploiting Neighbor Effect: Conv-Agnostic GNN Framework for Graphs With Heterophily.

Due to the homophily assumption in graph convolution networks (GCNs), a common consensus in the graph node classification task is that graph neural networks (GNNs) perform well on homophilic graphs but may fail on heterophilic graphs with many interclass edges. However, the previous interclass edges' perspective and related homo-ratio metrics cannot well explain the GNNs' performance under some heterophilic datasets, which implies that not all the interclass edges are harmful to GNNs. In this work, we propose a new metric based on the von Neumann entropy to reexamine the heterophily problem of GNNs and investigate the feature aggregation of interclass edges from an entire neighbor identifiable perspective. Moreover, we propose a simple yet effective Conv-Agnostic GNN framework (CAGNNs) to enhance the performance of most GNNs on the heterophily datasets by learning the neighbor effect for each node. Specifically, we first decouple the feature of each node into the discriminative feature for downstream tasks and the aggregation feature for graph convolution (GC). Then, we propose a shared mixer module to adaptively evaluate the neighbor effect of each node to incorporate the neighbor information. The proposed framework can be regarded as a plug-in component and is compatible with most GNNs. The experimental results over nine well-known benchmark datasets indicate that our framework can significantly improve performance, especially for the heterophily graphs. The average performance gain is 9.81%, 25.81%, and 20.61% compared with graph isomorphism network (GIN), graph attention network (GAT), and GCN, respectively. Extensive ablation studies and robustness analysis further verify the effectiveness, robustness, and interpretability of our framework. Code is available at https://github.com/JC-202/CAGNN.

Learning Constrained Dynamic Correlations in Spatiotemporal Graphs for Motion Prediction.

Human motion prediction is challenging due to the complex spatiotemporal feature modeling. Among all methods, graph convolution networks (GCNs) are extensively utilized because of their superiority in explicit connection modeling. Within a GCN, the graph correlation adjacency matrix drives feature aggregation, and thus, is the key to extracting predictive motion features. State-of-the-art methods decompose the spatiotemporal correlation into spatial correlations for each frame and temporal correlations for each joint. Directly parameterizing these correlations introduces redundant parameters to represent common relations shared by all frames and all joints. Besides, the spatiotemporal graph adjacency matrix is the same for different motion samples, and thus, cannot reflect samplewise correspondence variances. To overcome these two bottlenecks, we propose dynamic spatiotemporal decompose GC (DSTD-GC), which only takes 28.6% parameters of the state-of-the-art GC. The key of DSTD-GC is constrained dynamic correlation modeling, which explicitly parameterizes the common static constraints as a spatial/temporal vanilla adjacency matrix shared by all frames/joints and dynamically extracts correspondence variances for each frame/joint with an adjustment modeling function. For each sample, the common constrained adjacency matrices are fixed to represent generic motion patterns, while the extracted variances complete the matrices with specific pattern adjustments. Meanwhile, we mathematically reformulate GCs on spatiotemporal graphs into a unified form and find that DSTD-GC relaxes certain constraints of other GC, which contributes to a better representation capability. Moreover, by combining DSTD-GC with prior knowledge like body connection and temporal context, we propose a powerful spatiotemporal GCN called DSTD-GCN. On the Human3.6M, Carnegie Mellon University (CMU) Mocap, and 3D Poses in the Wild (3DPW) datasets, DSTD-GCN outperforms state-of-the-art methods by 3.9%-8.7% in prediction accuracy with 55.0%-96.9% fewer parameters. Codes are available at https://github.com/Jaakk0F/DSTD-GCN.

Engineering and physicochemical characterization of a novel, stable, symmetric bispecific antibody with dual target-binding using a common light chain.

Bispecific antibodies (BsAbs) have emerged as a major class of antibody therapeutics owing to their substantial potential in disease treatment. While several BsAbs have been successfully approved in recent years, ongoing development efforts continue to focus on optimizing various BsAbs tailored to particular antigens and action mechanisms, aiming to achieve favorable physicochemical properties. BsAbs generally encounter challenges due to their unfavorable physicochemical characteristics and poor manufacturing efficiencies, highlighting the need for optimization to achieve reliable productivity and developability. Herein, we describe the development of a novel symmetric BsAb, REGULGENT™ (N-term/C-term), comprising two Fab domains, using a common light chain. The heavy chain fragment encoded two antigen-binding determinants in one chain. The design and production of REGULGENT™ (N-term/C-term) are simple owing to the use of the same light chain, which does not induce heavy and light chain mispairing, frequently observed with the asymmetric BsAb format. REGULGENT™ (N-term/C-term) exhibited high expression and low aggregation characteristics during cell culture and stress treatment under low pH conditions. Differential scanning calorimetric data indicated that REGULGENT™ molecules had high conformational stability, similar to that of stabilized monoclonal antibodies. Surface plasmon resonance data showed that REGULGENT™ (N-term/C-term) could bind to two antigens simultaneously and exhibited a high affinity for two antigens. In summary, the symmetric BsAb format of REGULGENT™ confers its desirable IgG-like physicochemical properties, thus making it an excellent candidate for commercial development. The findings demonstrate a novel BsAb with substantial development potential for clinical applications.

Long‐term conservation tillage practices affect total carbon and contribute to the formation of soil aggregates in the semi‐arid North China Plain

AbstractThe soil aggregates characteristics and organic carbon (SOC) distribution under various tillage practices have been widely studied, but the changes in inorganic carbon (SIC) and their effect on soil aggregate formation are not clear. This study evaluated how long‐term tillage affects total carbon (both organic and inorganic) and contributes to forming aggregates in a fluvo‐aquic soil derived from limestone alluvial fans in the North China Plain. The four tillage practices that were used in a maize (Zea mays)/wheat (Triticum aestivum) double‐cropping system were moldboard plough without residues (CK), moldboard plough with residue retention (CT), rotary tillage with residue retention (RT) and no tillage with residue retention (NT). CT treatment, compared with NT treatment, decreased the SIC concentration at 0–300 mm and SOC concentration at 0–50 mm but increased the SOC concentration at 100–300 mm. Thus, there were no differences in the total carbon storage in the 0–300 mm layer because of the changes in SOC and SIC concentrations among four tillage practices. NT and RT practices increased the large macro‐aggregates (LM>2mm) and the mean weight diameter (MWD) only within the 0–50 mm layer, while the CT practice increased the LM>2mm and the MWD within the 100–250 mm layer. Correlation analysis revealed a significant link between LM>2mm and SOC, while small macroaggregates (SM0.25–2mm) were closely correlated with SIC. SOC and SIC had a relay cementation effect on the formation of large aggregates under long‐term tillage, and therefore, the effects of these two types of carbon should be comprehensively considered in the study of carbon sequestration and aggregate formation in calcareous soil.

A Task-Guided, Implicitly-Searched and Meta-Initialized Deep Model for Image Fusion.

Image fusion plays a key role in a variety of multi-sensor-based vision systems, especially for enhancing visual quality and/or extracting aggregated features for perception. However, most existing methods just consider image fusion as an individual task, thus ignoring its underlying relationship with these downstream vision problems. Furthermore, designing proper fusion architectures often requires huge engineering labor. It also lacks mechanisms to improve the flexibility and generalization ability of current fusion approaches. To mitigate these issues, we establish a Task-guided, Implicit-searched and Meta-initialized (TIM) deep model to address the image fusion problem in a challenging real-world scenario. Specifically, we first propose a constrained strategy to incorporate information from downstream tasks to guide the unsupervised learning process of image fusion. Within this framework, we then design an implicit search scheme to automatically discover compact architectures for our fusion model with high efficiency. In addition, a pretext meta initialization technique is introduced to leverage divergence fusion data to support fast adaptation for different kinds of image fusion tasks. Qualitative and quantitative experimental results on different categories of image fusion problems and related downstream tasks (e.g., visual enhancement and semantic understanding) substantiate the flexibility and effectiveness of our TIM.

Multi-contrast image super-resolution with deformable attention and neighborhood-based feature aggregation (DANCE): Applications in anatomic and metabolic MRI

Multi-contrast magnetic resonance imaging (MRI) reflects information about human tissues from different perspectives and has wide clinical applications. By utilizing the auxiliary information from reference images (Refs) in the easy-to-obtain modality, multi-contrast MRI super-resolution (SR) methods can synthesize high-resolution (HR) images from their low-resolution (LR) counterparts in the hard-to-obtain modality. In this study, we systematically discussed the potential impacts caused by cross-modal misalignments between LRs and Refs and, based on this discussion, proposed a novel deep-learning-based method with Deformable Attention and Neighborhood-based feature aggregation to be Computationally Efficient (DANCE) and insensitive to misalignments. Our method has been evaluated in two public MRI datasets, i.e., IXI and FastMRI, and an in-house MR metabolic imaging dataset with amide proton transfer weighted (APTW) images. Experimental results reveal that our method consistently outperforms baselines in various scenarios, with significant superiority observed in the misaligned group of IXI dataset and the prospective study of the clinical dataset. The robustness study proves that our method is insensitive to misalignments, maintaining an average PSNR of 30.67 dB when faced with a maximum range of ±9°and ±9 pixels of rotation and translation on Refs. Given our method’s desirable comprehensive performance, good robustness, and moderate computational complexity, it possesses substantial potential for clinical applications.

Aggregation Characteristics of Biobased Anionic Surfactant, Hydroxy Alkane Sulfonate in Aqueous CaCl2 Solutions: Vesicle and Supported Bilayer Formation.

Vesicles are known to spontaneously adsorb onto solid-liquid interfaces and to form supported bilayers in aqueous solution. Cationic surfactants have typically been used to generate supported bilayers because solid surfaces in water are often negatively charged. The present study investigated the aggregation behavior of an anionic surfactant, hydroxy alkane sulfonate having a C18 alkyl chain (C18HAS) in aqueous CaCl2 solutions. These assessments were performed by acquiring data related to equilibrium surface tension, the solubilization of an oil-soluble dye, UV-visible transmittance, pyrene fluorescence and dynamic light scattering together with freeze-fracture transmission electron microscopy observations. The results suggest that C18HAS can form vesicles in aqueous CaCl2 solutions under certain surfactant concentrations. Specifically, this aggregation behavior is greatly affected by C18HAS/CaCl2 molar ratio. At the C18HAS/CaCl2 molar ratio is less than an equivalence point (that is, less than 2:1), phase separation occurs with the formation of a vesicle above solubility limit of the C18HAS Ca salt. On the other hand, in the case that the C18HAS/CaCl2 molar ratio is above an equivalence point (that is, above 2:1), the Na salt of C18HAS forms micelles above the critical micelle concentration (cmc), causing solubilization of vesicles. Analyses by high-speed atomic force microscopy demonstrated that the C18HAS vesicles can spontaneously form a supported bilayer on a negatively charged mica surfaces, similar to the behavior of cationic surfactant vesicles, even though C18HAS is an anionic surfactant. These results suggest that C18HAS could serve as a detergent component but also as a surface modifier when the C18HAS/CaCl2 molar ratio is optimized.

Peritoneal dialysis promotes microbial-driven biosynthesis pathways of sesquiterpenes and triterpenes compounds in end-stage renal disease patients

The concept of the gut-kidney axis is gaining significant attention due to the close relationship between gut microbiota and kidney disease. Peritoneal dialysis is recognized as a crucial renal replacement therapy for end-stage renal disease (ESRD). The alterations in gut microbiota and related mechanisms after receiving this dialysis method are not fully understood. This study conducted shotgun metagenomic sequencing on fecal samples from 11 end-stage renal disease patients who did not receive dialysis (ESRD_N) and 7 patients who received peritoneal dialysis (ESRD_P). After quality control and correlation analysis of the data, our study is aimed at exploring the impact of peritoneal dialysis on the gut microbiota and health of ESRD patients. Our research findings indicate that the complexity and aggregation characteristics of gut microbiota interactions increase in ESRD_P. In addition, the gut microbiota drives the biosynthesis pathways of sesquiterpenes and triterpenes in ESRD_P patients, which may contribute to blood purification and improve circulation. Therefore, our research will lay the foundation for the prevention and treatment of ESRD.

VQGNet: An Unsupervised Defect Detection Approach for Complex Textured Steel Surfaces

Defect detection on steel surfaces with complex textures is a critical and challenging task in the industry. The limited number of defect samples and the complexity of the annotation process pose significant challenges. Moreover, performing defect segmentation based on accurate identification further increases the task’s difficulty. To address this issue, we propose VQGNet, an unsupervised algorithm that can precisely recognize and segment defects simultaneously. A feature fusion method based on aggregated attention and a classification-aided module is proposed to segment defects by integrating different features in the original images and the anomaly maps, which direct the attention to the anomalous information instead of the irregular complex texture. The anomaly maps are generated more confidently using strategies for multi-scale feature fusion and neighbor feature aggregation. Moreover, an anomaly generation method suitable for grayscale images is introduced to facilitate the model’s learning on the anomalous samples. The refined anomaly maps and fused features are both input into the classification-aided module for the final classification and segmentation. VQGNet achieves state-of-the-art (SOTA) performance on the industrial steel dataset, with an I-AUROC of 99.6%, I-F1 of 98.8%, P-AUROC of 97.0%, and P-F1 of 80.3%. Additionally, ViT-Query demonstrates robust generalization capabilities in generating anomaly maps based on the Kolektor Surface-Defect Dataset 2.

BCL-Former: Localized Transformer Fusion with Balanced Constraint for polyp image segmentation

Polyp segmentation remains challenging for two reasons: (a) the size and shape of colon polyps are variable and diverse; (b) the distinction between polyps and mucosa is not obvious. To solve the above two challenging problems and enhance the generalization ability of segmentation method, we propose the Localized Transformer Fusion with Balanced Constraint (BCL-Former) for Polyp Segmentation. In BCL-Former, the Strip Local Enhancement module (SLE module) is proposed to capture the enhanced local features. The Progressive Feature Fusion module (PFF module) is presented to make the feature aggregation smoother and eliminate the difference between high-level and low-level features. Moreover, the Tversky-based Appropriate Constrained Loss (TacLoss) is proposed to achieve the balance and constraint between True Positives and False Negatives, improving the ability to generalize across datasets. Extensive experiments are conducted on four benchmark datasets. Results show that our proposed method achieves state-of-the-art performance in both segmentation precision and generalization ability. Also, the proposed method is 5%–8% faster than the benchmark method in training and inference. The code is available at: https://github.com/sjc-lbj/BCL-Former.

Incremental RPN: Hierarchical Region Proposal Network for Apple Leaf Disease Detection in Natural Environments.

Apple leaf diseases can seriously affect apple production and quality, and accurately detecting them can improve the efficiency of disease monitoring. Owing to the complex natural growth environment, apple leaf lesions may be easily confused with background noise, leading to poor performance. In this study, a cascaded Incremental Region Proposal Network (Inc-RPN) is proposed to accurately detect apple leaf diseases in natural environments. The proposed Inc-RPN has a two-layer RPN architecture, where the precursor RPN is leveraged to generate diseased leaf proposals, and the successor RPN focuses on extracting target disease spots based on diseased leaf proposals. In the successor RPN, a low-level feature aggregation module is designed to fully utilize the bridged features and preserve the semantic information of the target disease spots. An incremental module is also leveraged to extract aggregated diseased leaf features and target disease spot features. Finally, a novel position anchor generator is designed to generate anchors based on diseased leaf proposals. The experimental results show that the proposed Inc-RPN performs very well on the FALD_CED and Apple Leaf Disease datasets, showing that it can accurately perform apple leaf disease detection tasks.

3D Point Cloud Semantic Segmentation based on Multi-scale Dense Nested Networks

Aiming at the problem that the relationship between geometric features and semantic features is ignored in the point cloud data downsampling process, which leads to inaccurate segmentation of object boundaries and structural details, this paper proposes a 3D point cloud semantic segmentation network based on multi-scale dense nested type. Firstly, a dense nested network architecture is constructed by nesting multiple multi-scale feature fusion modules to fuse multi-scale features of different directions between encoder-decoder paths, so as to effectively propagate local ge-ometric context information and enhance the ability of cross-scale information interaction. Secondly, a local feature aggregation unit is constructed in the multi-scale feature fusion module, which strengthens the structural awareness within the local point set based on graph convolution and attention mechanism, and promotes the complementarity of local geometric features and abstract semantic information. Then, the cross-layer multi-loss supervision module is combined to further optimize the multi-scale feature propagation, which makes the network training more stable and improves the point cloud segmentation accuracy. Finally, this paper verified the proposed network on the S3DIS dataset. The experimental results show that the proposed network has the mean intersection over Union of 71.2% and the overall accuracy of 88.7%, which is 1.2 and 0.7 percentage points higher than RandLA-Net, respectively, which proves that the proposed network can effectively improve the accuracy of 3D point cloud semantic segmentation.

A unified feature-motion consistency framework for robust image matching

Establishing reliable feature matches between a pair of images in various scenarios is a long-standing open problem in photogrammetry. Attention-based detector-free matching with coarse-to-fine architecture has been a typical pipeline to build matches, but the cross-attention module with global receptive field may compromise the structural local consistency by introducing irrelevant regions (outliers). Motion field can maintain structural local consistency under the assumption that matches for adjacent features should be spatially proximate. However, motion field can only estimate local displacements between consecutive images and struggle with long-range displacements estimation in large-scale variation scenarios without spatial correlation priors. Moreover, large-scale variations may also disrupt the geometric consistency with the application of mutual nearest neighbor criterion in patch-level matching, making it difficult to recover accurate matches. In this paper, we propose a unified feature-motion consistency framework for robust image matching (MOMA), to maintain structural consistency at both global and local granularity in scale-discrepancy scenarios. MOMA devises a motion consistency-guided dependency range strategy (MDR) in cross attention, aggregating highly relevant regions within the motion consensus-restricted neighborhood to favor true matchable regions. Meanwhile, a unified framework with hierarchical attention structure is established to couple local motion field with global feature correspondence. The motion field provides local consistency constraints in feature aggregation, while feature correspondence provides spatial context prior to improve motion field estimation. To alleviate geometric inconsistency caused by hard nearest neighbor criterion, we propose an adaptive neighbor search (soft) strategy to address scale discrepancy. Extensive experiments on three datasets demonstrate that our method outperforms solid baselines, with AUC improvements of 4.73/4.02/3.34 in two-view pose estimation task at thresholds of 5°/10°/20° on Megadepth test, and 5.94% increase of accuracy at threshold of 1px in homography task on HPatches datasets. Furthermore, in the downstream tasks such as 3D mapping, the 3D models reconstructed using our method on the self-collected SYSU UAV datasets exhibit significant improvement in structural completeness and detail richness, manifesting its high applicability in wide downstream tasks. The code is publicly available at https://github.com/BunnyanChou/MOMA.

MAMCL: Multi-attributes Masking Contrastive Learning for explainable seismic facies analysis

Seismic facies analysis is crucial in hydrocarbon exploration and development. Traditional machine learning approaches typically require manual selection of attributes and lack interpretability analysis. We propose an interpretable framework, multi-attribute masking contrastive learning (MAMCL), designed to adaptively select, explore and aggregate seismic attributes for seismic facies analysis. The MAMCL framework includes a depthwise CNN module for feature extraction and an iTransformer module for feature aggregation. Based on the assumption that different attributes computed on the same seismic sample imply common information associated with the same geologic facies, we formulate an unsupervised strategy of contrastive learning to pre-train the MAMCL framework for refining the attributes. This pre-training method encourages the network to extract and integrate highly correlated attribute features by enhancing the expression of commonalities within the same sample, and implicitly increase the distance between features of different categories by differentiating the expressions of different samples. Ultimately, these refined features only need to be input into a simple clustering algorithm, such as K-Means, to achieve seismic facies classification. MAMCL requires no labels or manual selection of attributes and can utilize the self-attention mechanism of iTransformer to compute adaptive attribute weights, facilitating interpretability analysis. We applied MAMCL framework to both unlogged turbidite channel systems in Canterbury Basin, New Zealand, and logged Chengdao area in Bohai Bay Basin, China, achieving reliable classification results and providing interpretability analysis.