Inherent Structure Information Research Articles

Seismic Data Reconstruction Via A Hybrid CNN-Low-Rank Method

Convolutional Neural Networks (CNNs) have shown promising results in seismic data reconstruction due to their exceptional ability to extract local features. However, incomplete seismic data may impede CNNs’ ability to discern meaningful features, leading to a degradation of reconstruction performance. Therefore, we propose a hybrid CNN-Low-Rank (CNN-LR) method which leverages Low-Rank (LR) techniques to recover data structure information, followed by detail refinement using a plug-and-play (PnP) trained CNN within an iterative framework. This approach utilizes LR techniques to restore inherent structural information, compensating for the absence of local information, thereby enabling CNNs to infer missing traces more accurately. Specifically, we first train a network to learn the mapping from low-rank recovered data patches with randomly selected rank parameters to complete data patches. Next, we incorporate the PnP trained CNN into an iterative framework, where low-rank recovery and network refinement are interleaved within each iteration. This creates a continuous reconstruction process until a termination condition is met. Performance evaluations of our proposed algorithm, conducted on synthetic and field datasets with a high missing trace ratio, demonstrate its superiority over the traditional low-rank method (multi-channel singular spectrum analysis, MSSA) and CNN-based method (U-net). Moreover, our approach shows strong performance in reconstructing large contiguous gaps and handling higher-dimensional seismic data (including 3D and 5D), broadening its applicability and robustness in practical seismic data reconstruction tasks.

Stereo digital image correlation using binocular super-resolution

Abstract The spatial resolution and measurement accuracy of the digital image correlation (DIC) method are constrained by camera resolution. This limitation is primarily determined by hardware costs. However, in current stereo DIC measurements, only the gray level or its gradient from two images is used for integer-pixel matching and sub-pixel optimization. It implicitly treats the two images from different viewpoints as independent entities before correlating them. However, the inherent structural information has not been fully utilized. This previously overlooked structural information provides a novel approach to enhancing the accuracy of DIC by leveraging the inherent correlations between the stereo image pairs. The realization of binocular super-resolution typically requires a relatively small parallax. Moreover, the DIC method can achieve image window pairing with small parallax through pre-matching. This implies that binocular super-resolution and stereo-DIC can complement each other by sharing information. In this paper, the DIC method is employed for whole-pixel image matching, while the binocular super-resolution method, based on deep learning, is applied to process the matched image pairs. Building on previous experiments, extensive datasets containing diverse experimental scenes and various speckle patterns were compiled and utilized. Furthermore, the DIC method can establish training datasets with minimal parallax through integer-pixel matching, thereby achieving highly effective super-resolution results.Experimental results demonstrate that super-resolution images with a higher signal-to-noise ratio can be obtained. Additionally, it effectively provides more image details, which enhance the calculation accuracy and resolution of DIC.

A low-rank support tensor machine for multi-classification

In recent decades, there has been an increasing demand for effectively handling high-dimensional multi-channel tensor data. Due to the inability to utilize internal structural information, Support Vector Machine (SVM) and its variations struggle to classify flattened tensor data, consequently resulting in the ‘curse of dimensionality’ issue. Furthermore, most of these methods can not directly apply to multiclass datasets. To overcome these challenges, we have developed a novel classification method called Multiclass Low-Rank Support Tensor Machine (MLRSTM). Our method is inspired by the well-established low-rank tensor hypothesis, which suggests a correlation between each channel of the feature tensor. Specifically, MLRSTM adopts the hinge loss function and introduces a convex approximation of tensor rank, the order-d Tensor Nuclear Norm (order-d TNN), in the regularization term. By leveraging the order-d TNN, MLRSTM effectively exploits the inherent structural information in tensor data to enhance generalization performance and avoid the curse of dimensionality. Moreover, we develop the Alternating Direction Method of Multipliers (ADMM) algorithm to optimize the convex problem inherent in training MLRSTM. Finally, comprehensive experiments validate the excellent performance of MLRSTM in tensor multi-classification tasks, showcasing its potential and efficacy in handling high-dimensional multi-channel tensor data.

Diversity matters: Cross-head mutual mean-teaching for semi-supervised medical image segmentation

Semi-supervised medical image segmentation (SSMIS) has witnessed substantial advancements by leveraging limited labeled data and abundant unlabeled data. Nevertheless, existing state-of-the-art (SOTA) methods encounter challenges in accurately predicting labels for the unlabeled data, giving rise to disruptive noise during training and susceptibility to erroneous information overfitting. Moreover, applying perturbations to inaccurate predictions further impedes consistent learning. To address these concerns, we propose a novel cross-head mutual mean-teaching network (CMMT-Net) incorporated weak-strong data augmentations, thereby benefiting both co-training and consistency learning. More concretely, our CMMT-Net extends the cross-head co-training paradigm by introducing two auxiliary mean teacher models, which yield more accurate predictions and provide supplementary supervision. The predictions derived from weakly augmented samples generated by one mean teacher are leveraged to guide the training of another student with strongly augmented samples. Furthermore, two distinct yet synergistic data perturbations at the pixel and region levels are introduced. We propose mutual virtual adversarial training (MVAT) to smooth the decision boundary and enhance feature representations, and a cross-set CutMix strategy to generate more diverse training samples for capturing inherent structural data information. Notably, CMMT-Net simultaneously implements data, feature, and network perturbations, amplifying model diversity and generalization performance. Experimental results on three publicly available datasets indicate that our approach yields remarkable improvements over previous SOTA methods across various semi-supervised scenarios. The code is available at https://github.com/Leesoon1984/CMMT-Net.

Predictive Modeling with Temporal Graphical Representation on Electronic Health Records.

Deep learning-based predictive models, leveraging Electronic Health Records (EHR), are receiving increasing attention in healthcare. An effective representation of a patient's EHR should hierarchically encompass both the temporal relationships between historical visits and medical events, and the inherent structural information within these elements. Existing patient representation methods can be roughly categorized into sequential representation and graphical representation. The sequential representation methods focus only on the temporal relationships among longitudinal visits. On the other hand, the graphical representation approaches, while adept at extracting the graph-structured relationships between various medical events, fall short in effectively integrate temporal information. To capture both types of information, we model a patient's EHR as a novel temporal heterogeneous graph. This graph includes historical visits nodes and medical events nodes. It propagates structured information from medical event nodes to visit nodes and utilizes time-aware visit nodes to capture changes in the patient's health status. Furthermore, we introduce a novel temporal graph transformer (TRANS) that integrates temporal edge features, global positional encoding, and local structural encoding into heterogeneous graph convolution, capturing both temporal and structural information. We validate the effectiveness of TRANS through extensive experiments on three real-world datasets. The results show that our proposed approach achieves state-of-the-art performance.

Deep fair clustering with multi-level decorrelation

Fair clustering aims to prevent sensitive attributes (e.g., race or gender) from dominating the clustering process. However, real-world datasets, often characterized by low quality and high dimensionality, restrict existing fair clustering methods from achieving satisfactory outcomes. Typically, these sensitive attributes are intricately intertwined with other attributes in high-dimensional continuous data, forming backgrounds or entities within the data. The integration results in a significant correlation of features and samples across different clusters, thereby hindering the model's ability to explore the intrinsic structure. To address these issues, we propose a novel fair clustering method that incorporates multi-level decorrelation constraints. Our goal is to extract inherent fair structural information under the interference of sensitive attributes, enhancing both the validity and fairness of the model. Specifically, we introduce a new cluster-wise similarity metric based on the partition correlation coefficient, which facilitates cluster-level decorrelation and captures cluster-discriminative properties. Furthermore, by incorporating softmax-formulated decorrelation at the sample-level and feature-level, we concurrently explore representations that favor fairness. These three components are seamlessly integrated into our clustering framework, yielding a more robust and confident data partition. Experiments conducted on six commonly-used datasets demonstrate the effectiveness of our proposed method.

Tensor Completion for Alzheimer's Disease Prediction From Diffusion Tensor Imaging.

Alzheimer's disease (AD) is a slowly progressive neurodegenerative disorder with insidious onset. Accurate prediction of the disease progression has received increasing attention. Cognitive scores that reflect patients' cognitive status have become important criteria for predicting AD. Most existing methods consider the relationship between neuroimages and cognitive scores to improve prediction results. However, the inherent structure information in interrelated cognitive scores is rarely considered. In this article, we propose a relation-aware tensor completion multitask learning method (RATC-MTL), in which the cognitive scores are represented as a third-order tensor to preserve the global structure information in clinical scores. We combine both tensor completion and linear regression into a unified framework, which allows us to capture both inter and intra modes correlations in cognitive tensor with a low-rank constraint, as well as incorporate the relationship between biological features and cognitive status by imposing a regression model on multiple cognitive scores. Compared to the single-task and state-of-the-art multi-task algorithms, our proposed method obtains the best results for predicting cognitive scores in terms of four commonly used metrics. Furthermore, the overall performance of our method in classifying AD progress is also the best. Our results demonstrate the effectiveness of the proposed framework in fully exploring the global structure information in cognitive scores. This study introduces a novel concept of leveraging tensor completion to assist in disease diagnoses, potentially offering a solution to the issue of data scarcity encountered in prolonged monitoring scenarios.

A study of extractive summarization of long documents incorporating local topic and hierarchical information

In recent years, the transformer-based language models have achieved remarkable success in the field of extractive text summarization. However, there are still some limitations in this kind of research. First, the transformer language model usually regards the text as a linear sequence, ignoring the inherent hierarchical structure information of the text. Second, for long text data, traditional extractive models often focus on global topic information, which poses challenges in how they capturing and integrating local contextual information within topic segments. To address these issues, we propose a long text extractive summarization model that employs a local topic information extraction module and a text hierarchical extraction module to capture the local topic information and document's hierarchical structure information of the original text. Our approach enhances the ability to determine whether a sentence belongs to the summary. In this experiment, ROUGE score is used as the experimental evaluation index, and evaluates the model on three large public datasets. Through experimental validation, the model demonstrates superior performance in terms of ROUGE-1, ROUGE-2, and ROUGE-L scores compared to current mainstream summarization models, affirming the effectiveness of incorporating local topic information and document hierarchical structure into the model.

Low-tubal-rank tensor completion via local and nonlocal knowledge

Many previous studies have indicated that exploiting prior information about the underlying tensor data is a sound approach for completing missing or damaged entries. This prior information can be divided into three commonly used categories: low-rankness, local piecewise smoothness, and nonlocal self-similarity (NSS) priors. Although existing methods based on these priors have gained considerable attention, the majority of studies utilize only one or two of these priors, leading to inadequate extraction of structural information from tensors. To address this limitation and comprehensively depict the inherent structural information of underlying tensor data, this article develops a novel tensor completion framework that can simultaneously utilize the three abovementioned priors within a plug-and-play framework. More precisely, we adopt the tensor correlated total variation (t-CTV) norm as a robust representation for capturing the combined effects of low-rankness and local piecewise smoothness priors, eliminating the need for a trade-off parameter in the process; meanwhile, we introduce an advanced denoiser to explore the NSS prior. Furthermore, to address the presented optimization model, we design an alternating direction method of multipliers (ADMM) algorithm and innovatively provide its corresponding global convergence guarantees. Extensive numerical experiments on real tensor data, including color, medical and hyperspectral images, demonstrate that the proposed method surpasses various advanced approaches in terms of both quality metrics and visual effects.

SEMANTIC ENRICHMENT OF 3D POINT CLOUDS USING 2D IMAGE SEGMENTATION

Abstract. 3D point cloud segmentation is computationally intensive due to the lack of inherent structural information and the unstructured nature of the point cloud data, which hinders the identification and connection of neighboring points. Understanding the structure of the point cloud data plays a crucial role in obtaining a meaningful and accurate representation of the underlying 3D environment. In this paper, we propose an algorithm that builds on existing state-of-the-art techniques of 2D image segmentation and point cloud registration to enrich point clouds with semantic information. DeepLab2 with ResNet50 as backbone architecture trained on the COCO dataset is used for indoor scene semantic segmentation into several classes like wall, floor, ceiling, doors, and windows. Semantic information from 2D images is propagated along with other input data, i.e., RGB images, depth images, and sensor information to generate 3D point clouds with semantic information. Iterative Closest Point (ICP) algorithm is used for the pair-wise registration of consecutive point clouds and finally, optimization is applied using the pose graph optimization on the whole set of point clouds to generate the combined point cloud of the whole scene. 3D point cloud of the whole scene contains pseudo-color information which denotes the semantic class to which each point belongs. The proposed methodology use an off-the-shelf 2D semantic segmentation deep learning model to semantically segment 3D point clouds collected using handheld mobile LiDAR sensor. We demonstrate a comparison of the accuracy achieved compared to a manually segmented point cloud on an in-house dataset as well as a 2D3DS benchmark dataset.

Incorporating Structural Information into Legal Case Retrieval

Legal case retrieval has received increasing attention in recent years. However, compared to ad hoc retrieval tasks, legal case retrieval has its unique challenges. First, case documents are rather lengthy and contain complex legal structures. Therefore, it is difficult for most existing dense retrieval models to encode an entire document and capture its inherent complex structure information. Most existing methods simply truncate part of the document content to meet the input length limit of PLMs, which will lead to information loss. Additionally, the definition of relevance in the legal domain differs from that in the general domain. Previous semantic-based or lexical-based methods fail to provide a comprehensive understanding of the relevance of legal cases. In this article, we propose a Structured Legal case Retrieval (SLR) framework, which incorporates internal and external structural information to address the above two challenges. Specifically, to avoid the truncation of long legal documents, the internal structural information, which is the organization pattern of legal documents, can be utilized to split a case document into segments. By dividing the document-level semantic matching task into segment-level subtasks, SLR can separately process segments using different methods based on the characteristic of each segment. In this way, the key elements of a case document can be highlighted without losing other content information. Second, toward a better understanding of relevance in the legal domain, we investigate the connections between criminal charges appearing in large-scale case corpus to generate a chargewise relation graph. Then, the similarity between criminal charges can be pre-computed as the external structural information to enhance the recognition of relevant cases. Finally, a learning-to-rank algorithm integrates the features collected from internal and external structures to output the final retrieval results. Experimental results on public legal case retrieval benchmarks demonstrate the superior effectiveness of SLR over existing state-of-the-art baselines, including traditional bag-of-words and neural-based methods. Furthermore, we conduct a case study to visualize how the proposed model focuses on key elements and improves retrieval performance.

Global and cross-modal feature aggregation for multi-omics data classification and application on drug response prediction

With rapid development of single-cell multi-modal sequencing technologies, more and more multi-omics data come into being and provide a unique opportunity for the identification of distinct cell types at the single-cell level. Therefore, it is important to integrate different modalities which are with high-dimensional features for boosting final multi-omics data classification performance. However, existing multi-omics data classification methods mainly focus on exploiting the complementary information of different modalities, while ignoring the learning confidence and cross-modal sample relationship during information fusion. In this paper, we propose a multi-omics data classification network via global and cross-modal feature aggregation, referred to as GCFANet. On one hand, considering that a large number of feature dimensions in different modalities could not contribute to final classification performance but disturb the discriminability of different samples, we propose a feature confidence learning mechanism to suppress some redundant features, as well as enhancing the expression of discriminative feature dimensions in each modality. On the other hand, in order to capture the inherent sample structure information implied in each modality, we design a graph convolutional network branch to learn the corresponding structure preserved feature representation. Then the modal-specific feature representations are concatenated and input to a transformer induced global and cross-modal feature aggregation module for learning consensus feature representation from different modalities. In addition, the consensus feature representation used for final classification is enhanced via a view-specific consistency preserved contrastive learning strategy. Extensive experiments on four multi-omics datasets are conducted to demonstrate the efficacy of the proposed GCFANet.

Auto-weighted multi-view clustering with the use of an augmented view

Multi-view clustering is a powerful technique that leverages both consensus and complementary information from multiple perspectives to achieve impressive results. However, existing methods have not fully explored the inherent structural information contained in multi-view data, such as inadequately harnessing the consistency among views or neglecting to explore their high-order correlations. To address these limitations, we propose a novel auto-weighted multi-view clustering (AWMVC) method with the use of an augmented view. The main motivation of AWMVC is that each view naturally embodies consensus information, and concatenating the views maximizes the consistency properties. AWMVC first connects multiple views via feature concatenation to form a newly augmented view. Then, we stack the self-representation matrices of the original views and the newly augmented view, i.e., the concatenated view into a 3rd-order tensor, subject to a low-rank tensor constraint. Additionally, this approach automatically assigns weights to each view and constructs a more reliable affinity matrix. Our method unifies the feature concatenation process, high-order information, and a weighted strategy into one model, and is efficiently optimized through an iterative algorithm. Experimental results on benchmark datasets demonstrate that AWMVC outperforms state-of-the-art methods, highlighting the effectiveness of our approach.

Integrative Bayesian tensor regression for imaging genetics applications.

Identifying biomarkers for Alzheimer's disease with a goal of early detection is a fundamental problem in clinical research. Both medical imaging and genetics have contributed informative biomarkers in literature. To further improve the performance, recently, there is an increasing interest in developing analytic approaches that combine data across modalities such as imaging and genetics. However, there are limited methods in literature that are able to systematically combine high-dimensional voxel-level imaging and genetic data for accurate prediction of clinical outcomes of interest. Existing prediction models that integrate imaging and genetic features often use region level imaging summaries, and they typically do not consider the spatial configurations of the voxels in the image or incorporate the dependence between genes that may compromise prediction ability. We propose a novel integrative Bayesian scalar-on-image regression model for predicting cognitive outcomes based on high-dimensional spatially distributed voxel-level imaging data, along with correlated transcriptomic features. We account for the spatial dependencies in the imaging voxels via a tensor approach that also enables massive dimension reduction to address the curse of dimensionality, and models the dependencies between the transcriptomic features via a Graph-Laplacian prior. We implement this approach via an efficient Markov chain Monte Carlo (MCMC) computation strategy. We apply the proposed method to the analysis of longitudinal ADNI data for predicting cognitive scores at different visits by integrating voxel-level cortical thickness measurements derived from T1w-MRI scans and transcriptomics data. We illustrate that the proposed imaging transcriptomics approach has significant improvements in prediction compared to prediction using a subset of features from only one modality (imaging or genetics), as well as when using imaging and transcriptomics features but ignoring the inherent dependencies between the features. Our analysis is one of the first to conclusively demonstrate the advantages of prediction based on combining voxel-level cortical thickness measurements along with transcriptomics features, while accounting for inherent structural information.

MIGTNet: Metapath Instance-based Graph Transformation Network for heterogeneous graph embedding

Many real-world graphs are heterogeneous with various types of entities and relations. These heterogeneous graphs commonly inherent various structural and semantic information of sequences of node types, called metapaths; and many current heterogeneous graph embedding models learn not only the node features but also the metapaths. However, all of them only focus on the metapath instances (i.e., the sequences of nodes following the schema defined by a metapath) and ignore the relations between them when learning about the metapaths. To overcome this limitation, we propose a new model called the Metapath Instance-based Graph Transformation Network (MIGTNet) for heterogeneous graph embedding, which uses both metapath instances and relations between them. MIGTNet constructs a metapath instance-based graph, where a node represents a metapath instance and a link represents a relation between metapath instances, and inputs it to a hierarchical graph attention network to obtain meaningful node embeddings. Extensive experiments using real datasets show that MIGTNet outperforms state-of-the-art heterogeneous graph embedding models in node classification and node clustering.

Multi-Scale Feature Fusion and Structure-Preserving Network for Face Super-Resolution

Deep convolutional neural networks have demonstrated significant performance improvements in face super-resolution tasks. However, many deep learning-based approaches tend to overlook the inherent structural information and feature correlation across different scales in face images, making the accurate recovery of face structure in low-resolution cases challenging. To address this, this paper proposes a method that fuses multi-scale features while preserving the facial structure. It introduces a novel multi-scale residual block (MSRB) to reconstruct key facial parts and structures from spatial and channel dimensions, and utilizes pyramid attention (PA) to exploit non-local self-similarity, improving the details of the reconstructed face. Feature Enhancement Modules (FEM) are employed in the upscale stage to refine and enhance current features using multi-scale features from previous stages. The experimental results on CelebA, Helen and LFW datasets provide evidence that our method achieves superior quantitative metrics compared to the baseline, the Peak Signal-to-Noise Ratio (PSNR) outperforms the baseline by 0.282 dB, 0.343 dB, and 0.336 dB. Furthermore, our method demonstrates improved visual performance on two additional no-reference datasets, Widerface and Webface.

MT-FANet: A Morphology and Topology-Based Feature Alignment Network for SAR Ship Rotation Detection

In recent years, ship target detection in synthetic aperture radar (SAR) images has significantly progressed due to the rapid development of deep learning (DL). However, since only the spatial feature information of ship targets is utilized, the current DL-based SAR ship detection approaches cannot achieve a satisfactory performance, especially in the case of multiscale, rotations, or complex backgrounds. To address these issues, in this paper, a novel deep-learning network for SAR ship rotation detection, called a morphology and topology-based feature alignment network, is proposed which can better exploit the morphological features and inherent topological structure information. This network consists of the following three main steps: First, deformable convolution is introduced to improve the representational ability for irregularly shaped ship targets, and subsequently, a morphology and topology feature pyramid network is developed to extract inherent topological structure information. Second, based on the aforementioned features, a rotation alignment feature head is devised for fine-grained processing as well as aligning and distinguishing the features; to enable regression prediction of rotated bounding boxes; and to adopt a parameter-sharing mechanism to improve detection efficiency. Therefore, utilizing morphological and inherent topological structural information enables a superior detection performance to be achieved. Finally, we evaluate the effectiveness of the proposed method using the rotated ship detection dataset in SAR images (RSDD-SAR). Our method outperforms other DL-based algorithms with fewer parameters. The overall average precision is 90.84% and recall is 92.21%. In inshore and offshore scenarios, our method performs well for the detection of multi-scale and rotation-varying ship targets, with its average precision reaching 66.87% and 95.72%, respectively.

Synwmd: Syntax-aware word Mover’s distance for sentence similarity evaluation

Word Mover’s Distance (WMD) computes the distance between words and models text similarity with the moving cost between words in two text sequences. Yet, it does not offer good performance in sentence similarity evaluation since it does not incorporate word importance and fails to take inherent contextual and structural information in a sentence into account. An improved WMD method using the syntactic parse tree, called Syntax-aware Word Mover’s Distance (SynWMD), is proposed to address these two shortcomings in this work. First, a weighted graph is built upon the word co-occurrence statistics extracted from the syntactic parse trees of sentences. The importance of each word is inferred from graph connectivities. Second, the local syntactic parsing structure of words is considered in computing the distance between words. To demonstrate the effectiveness of the proposed SynWMD, we conduct experiments on 6 textual semantic similarity (STS) datasets and 4 sentence classification datasets. Experimental results show that SynWMD achieves state-of-the-art performance on STS tasks. It also outperforms other WMD-based methods on sentence classification tasks.

Preserving bilateral view structural information for subspace clustering

Subspace clustering algorithms have been found successful in various applications that involve two-dimensional data, i.e., each example of the data is a matrix. However, most of the existing methods transform the matrix-type examples to vectors in a pre-processing step, which omits and severely damages the inherent structural information of such data. In this paper, we propose a novel subspace clustering method for two-dimensional data, which is capable of extracting the most representative structural information from the data to recover the underlying grouping relationships of the data. The structural features are extracted from two views of the data and the numbers of feature spaces in both views are automatically determined by optimization. Extensive experiments confirm the effectiveness of the proposed method.

Cross-SRN: Structure-Preserving Super-Resolution Network With Cross Convolution

It is challenging to restore low-resolution (LR) images to super-resolution (SR) images with correct and clear details. Existing deep learning works almost neglect the inherent structural information of images, which acts as an important role for visual perception of SR results. In this paper, we design a hierarchical feature exploitation network to probe and preserve structural information in a multi-scale feature fusion manner. First, we propose a cross convolution upon traditional edge detectors to localize and represent edge features. Then, cross convolution blocks (CCBs) are designed with feature normalization and channel attention to consider the inherent correlations of features. Finally, we leverage multi-scale feature fusion group (MFFG) to embed the cross convolution blocks and develop the relations of structural features in different scales hierarchically, invoking a lightweight structure-preserving network named as Cross-SRN. Experimental results demonstrate the Cross-SRN achieves competitive or superior restoration performances against the state-of-the-art methods with accurate and clear structural details. Moreover, we set a criterion to select images with rich structural textures. The proposed Cross-SRN outperforms the state-of-the-art methods on the selected benchmark, which demonstrates that our network has a significant advantage in preserving edges.