Hierarchical Information Research Articles

Deep Autoencoder-like non-negative matrix factorization with graph regularized for link prediction in dynamic networks

Link prediction in dynamic(temporal) networks refers to predicting future edges by analyzing the available network information. Among the existing temporal link prediction approaches, non-negative matrix factorization(NMF) is a kind of competitive algorithm and has attracted extensive attention. However, traditional NMF-based prediction methods are shallow methods and cannot fully mine the dynamic network, which may lead to a decrease in performance of algorithms. To overcome these shortcomings, inspired by deep Autoencoder, we propose two novel deep Autoencoder-like NMF with graph regularized prediction methods for dynamic networks. By fusing encoder component with deep structure into deep NMF model, our algorithms can sufficiently exploit the complex hierarchical information hidden in dynamic networks. To further extract the abundant information hidden in dynamic networks, graph regularization and PageRank are utilized to exploit the local and global topology information of each snapshot, respectively. By jointly optimizing them in deep Autoencoder-like NMF model, our model is able to preserve the local and global information hidden in dynamic networks, simultaneously. Moreover, an effective alternating iterative method with convergence guarantee is developed for minimizing the established model. Finally, we test our proposed prediction methods on several synthetic and real world datasets to demonstrate that our approaches outperform the state-of-the-art prediction approaches.

Contrastive diagnostic embedding (CDE) model for automated coding – A case study using emergency department encounters

BackgroundBilling codes are utilized for medical reimbursement, clinical quality metric valuation and for epidemiologic purposes to report and follow disease trends and outcomes. The current paradigm of manual coding can be expensive, time-consuming, and subject to human error. Though automation of the billing codes has been widely reported in the literature via rule-based and supervised approaches, existing strategies lack generalizability and robustness towards large and constantly changing ICD hierarchical structure. MethodWe propose a weakly supervised training strategy by leveraging contrastive learning, contrastive diagnosis embedding (CDE) to capture the fine semantic variations between the diagnosis codes. The approach consists of a two-phase contrastive training for generating the semantic embedding space adapted to incorporate hierarchical information of ICD-10 vocabulary and a weakly supervised retrieval scheme. Core strength of the proposed method is that it puts no limit on the 70 K ICD-10 codes set and can handle all rare codes for coding the diagnosis. ResultsOur CDE model outperformed string-based partial matching and ClinicalBERT embedding on three test cases (a retrospective testset, a prospective testset, and external testset) and produced an accurate prediction of rare and newly introduced diagnosis codes. A detailed ablation study showed the importance of each phase of the proposed multi-phase training. Each successive phase of training – ICD-10 group sensitive training (phase 1.1), ICD-10 subgroup sensitive training (phase 1.2), free-text diagnosis description-based training (phase 2) – improved performance beyond the previous phase of training. The model also outperformed existing supervised models like CAML and PLM-ICD and produced satisfactory performance on the rare codes. ConclusionCompared to the existing rule-based and supervised models, the proposed weakly supervised contrastive learning overcomes the limitations in terms of generalization capability and increases the robustness of the automated billing. Such a model will allow flexibility through accurate billing code automation for practice convergence and gains efficiencies in a value-based care payment environment.

A Method to Enable Automatic Extraction of Cost and Quantity Data from Hierarchical Construction Information Documents to Enable Rapid Digital Comparison and Analysis

Context: Despite the effort put into developing standards for structuring construction costs and the strong interest in the field, most construction companies still perform the process of data gathering and processing manually. This provokes inconsistencies, different criteria when classifying, misclassifications, and the process becomes very time-consuming, particularly in large projects. Additionally, the lack of standardization makes cost estimation and comparison tasks very difficult. Objective: The aim of this work was to create a method to extract and organize construction cost and quantity data into a consistent format and structure to enable rapid and reliable digital comparison of the content. Methods: The approach consisted of a two-step method: firstly, the system implemented data mining to review the input document and determine how it was structured based on the position, format, sequence, and content of descriptive and quantitative data. Secondly, the extracted data were processed and classified with a combination of data science and experts’ knowledge to fit a common format. Results: A large variety of information coming from real historical projects was successfully extracted and processed into a common format with 97.5% accuracy using a subset of 5770 assets located on 18 different files, building a solid base for analysis and comparison. Conclusions: A robust and accurate method was developed for extracting hierarchical project cost data to a common machine-readable format to enable rapid and reliable comparison and benchmarking.

Kernel Attention Transformer for Histopathology Whole Slide Image Analysis and Assistant Cancer Diagnosis.

Transformer has been widely used in histopathology whole slide image analysis. However, the design of token-wise self-attention and positional embedding strategy in the common Transformer limits its effectiveness and efficiency when applied to gigapixel histopathology images. In this paper, we propose a novel kernel attention Transformer (KAT) for histopathology WSI analysis and assistant cancer diagnosis. The information transmission in KAT is achieved by cross-attention between the patch features and a set of kernels related to the spatial relationship of the patches on the whole slide images. Compared to the common Transformer structure, KAT can extract the hierarchical context information of the local regions of the WSI and provide diversified diagnosis information. Meanwhile, the kernel-based cross-attention paradigm significantly reduces the computational amount. The proposed method was evaluated on three large-scale datasets and was compared with 8 state-of-the-art methods. The experimental results have demonstrated the proposed KAT is effective and efficient in the task of histopathology WSI analysis and is superior to the state-of-the-art methods.

Sequence based local-global information fusion framework for vertebrae detection under pathological and FOV variation challenges.

Automated vertebrae detection (identification and localization) aims to identify vertebrae and locate their centroids in medical images, which is a critical step of spinal computer-aided systems. However, due to unpredictable field-of-view and various pathology cases, the image content is diverse and the vertebral morphology can be abnormal in a variety of ways, which challenges the designed systems. In this paper, we propose an effective sequence-based framework robust to various tough cases for accurate vertebrae identification and localization. Our method consists of three sub-modules: (1) Local Feature Extraction (LFE) module designs a shape-compatible category-balanced sampler to collect patches to train a convolution neural network, which extracts representative local features and generates score maps. (2) Discriminative Sequential Image Description (DSID) module proposes a node screening strategy for reliable vertebral feature sequence construction based on feature maps and score maps. This effectively prevents false positives and false negatives in light-weighted dense prediction schemes and fuses local features into a hierarchical discriminative description of given images. (3) Spinal Pattern Exploitation (SPE) module designs an end-balanced relative position learning scheme to fuse hierarchical local-global information for comprehensively exploiting spinal patterns to overcome the FOV and pathological variation challenges in vertebrae detection. Extensive experiments on a challenging dataset consisting of 450 spinal MRIs show that the identification rate of FSDF reaches 0.974 ±0.025 and the localization error is only 4.742 ±2.928 pixels, which demonstrates the effectiveness of our method with pathological and field-of-view variations and its superiority over other state-of-the-art methods.

Towards efficient and accurate CO2 sequestration optimization: Integrating hierarchical spatio-temporal information into deep-learning-based surrogate models

Optimizing CO2 storage in brine water is a critical task that requires process-level methods based on geological modeling and numerical simulation. However, these methods can be computationally expensive, especially when simulating larger grid scales and longer simulation times. To address this challenge, we proposed a novel approach that utilizes advanced deep learning techniques to develop a U-LSTM-net neural network capable of incorporating spatio-temporal information at varying hierarchies. The method employs an encoder-decoder architecture that extracts, memorizes, and integrates localized and holistic information from changing receptive fields. Multi-task learning algorithms enable simultaneous learning and predicting multiple flow fields. Additionally, we employed transfer learning to transfer learned knowledge and experience to new tasks and a fine-tuning algorithm to improve the retention of domain-general knowledge. We demonstrated the effectiveness of our model on synthetic models and an adjusted 3D Brugge mode. The U-LSTM-net effectively captures and integrates spatio-temporal information, exhibiting powerful memory capacity. Balancing algorithms improve multiple tasks' learning efficiency and avoid the dominance of tasks with higher loss values. The fine-tuning algorithm reduces overfitting and preserves pre-trained knowledge better, thereby reducing the demand for extensive information. Furthermore, when transferred to the 3D authentic problem, the surrogate model exhibits impressive predictive capabilities for various attributes across different layers. This study integrated multiple learning methods and established a precise and practical learning workflow for accelerating the prediction of the CO2 sequestration dynamic.

Mechanically Ultra-Robust Elastomers Integrating Self-Healing and Recycling Properties Enable Information Encryption and Hierarchical Decryption.

Developing high-performance elastomers with distinctive features opens up new vistas and exciting possibilities for information encryption but remains a daunting challenge. To surmount this difficulty, an unprecedented synthetic approach, "modular molecular engineering", was proposed to develop tailor-made advanced elastomers. The customized hydrophobic poly(urea-urethane) (HPUU-R) elastomer perfectly integrated ultrahigh tensile strength (∼75.3 MPa), extraordinary toughness (∼292.5 MJ m-3), satisfactory room-temperature healing, high transparency, puncture-, scratch-, and water-resistance; and miraculously, its 0.20 g film could lift objects over 100 000 times its weight without rupture. Intriguingly, we unexpectedly discovered that the elastomers fluoresce brightly at the optimal excitation wavelength attributed to the "clusterization-triggered emission". Based on the gradient hydrophobicity and fluorescent properties of HPUU-R, a hierarchical information encryption/decryption mode was innovatively established. Using high-performance HPUU-R as a double encryption platform makes the information highly stable and persistent, thus providing a stronger guarantee for the encrypted information. More attractively, given the impressive recyclability and self-healing of HPUU-R, information encryption can be realized by using recycled elastomers, injecting new vitality into green and sustainable development.

HimGNN: a novel hierarchical molecular graph representation learning framework for property prediction.

Accurate prediction of molecular properties is an important topic in drug discovery. Recent works have developed various representation schemes for molecular structures to capture different chemical information in molecules. The atom and motif can be viewed as hierarchical molecular structures that are widely used for learning molecular representations to predict chemical properties. Previous works have attempted to exploit both atom and motif to address the problem of information loss in single representation learning for various tasks. To further fuse such hierarchical information, the correspondence between learned chemical features from different molecular structures should be considered. Herein, we propose a novel framework for molecular property prediction, called hierarchical molecular graph neural networks (HimGNN). HimGNN learns hierarchical topology representations by applying graph neural networks on atom- and motif-based graphs. In order to boost the representational power of the motif feature, we design a Transformer-based local augmentation module to enrich motif features by introducing heterogeneous atom information in motif representation learning. Besides, we focus on the molecular hierarchical relationship and propose a simple yet effective rescaling module, called contextual self-rescaling, that adaptively recalibrates molecular representations by explicitly modelling interdependencies between atom and motif features. Extensive computational experiments demonstrate that HimGNN can achieve promising performances over state-of-the-art baselines on both classification and regression tasks in molecular property prediction.

Neural Architecture Search for GNN-Based Graph Classification

Graph classification is an important problem with applications across many domains, for which graph neural networks (GNNs) have been state-of-the-art (SOTA) methods. In the literature, to adopt GNNs for the graph classification task, there are two groups of methods: global pooling and hierarchical pooling. The global pooling methods obtain the graph representation vectors by globally pooling all of the node embeddings together at the end of several GNN layers, whereas the hierarchical pooling methods provide one extra pooling operation between the GNN layers to extract hierarchical information and improve the graph representations. Both global and hierarchical pooling methods are effective in different scenarios. Due to highly diverse applications, it is challenging to design data-specific pooling methods with human expertise. To address this problem, we propose PAS (Pooling Architecture Search) to design adaptive pooling architectures by using the neural architecture search (NAS). To enable the search space design, we propose a unified pooling framework consisting of four modules: Aggregation, Pooling, Readout, and Merge. Two variants, PAS-G and PAS-NE, are provided to design the pooling operations in different scales. A set of candidate operations is designed in the search space using this framework. Then, existing human-designed pooling methods, including global and hierarchical ones, can be incorporated. To enable efficient search, a coarsening strategy is developed to continuously relax the search space, and then a differentiable search method can be adopted. We conduct extensive experiments on six real-world datasets, including the large-scale datasets MR and ogbg-molhiv. Experimental results in this article demonstrate the effectiveness and efficiency of the proposed PAS in designing the pooling architectures for graph classification. The Top-1 performance on two Open Graph Benchmark (OGB) datasets 1 further indicates the utility of PAS when facing diverse realistic data. The implementation of PAS is available at: https://github.com/AutoML-Research/PAS.

Research on hierarchical network security situation awareness data fusion method in big data environment

ABSTRACT With the increase in the number of network users, data information has become more abundant, and query speed and coverage have greatly improved. In this era, the problem of information leakage is relatively serious, and some malicious software has invaded the network system of user devices, increasing the threat to network security. In order to improve the analysis ability of network security situation awareness, this study designs a hierarchical network security situation awareness data fusion method under the big data environment. On the basis of data fusion technology and network security situation awareness technology, the hierarchical network security situation information is collected. Then, its features are extracted, and the network security situation awareness process is constructed based on RBF neural. By using this process, hierarchical network security situation awareness data can be obtained. The data is first sorted out and analyzed, and then the data is filtered. Finally, the matrix three decomposition data fusion algorithm is selected as the blueprint of the data fusion method. It can complete the hierarchical network security situation awareness data fusion. Experimental results show that the proposed method has high matching accuracy, little influence on the data fusion time due to the increase of data volume and low energy consumption in the fusion process.

LGM-GNN: A Local and Global Aware Memory-Based Graph Neural Network for Fraud Detection

Graphs have been widely adopted to accomplish fraud detection tasks because of their inherently favorable structure to capture the intricate features in many complicated scenarios, especially in some modern e-commerce situations that have various relation attributes like transactions. These works tend to utilize the direct aggregate information about neighbor nodes of the target node or the aggregation of the neighbor information after the conditional filter and mostly use local information but ignore global information. However, in some cases, local abnormal points are detected in the global view, so global information is very important for fraud detection. In this paper, we propose a local and global aware memory-based graph neural network for fraud detection (LGM-GNN). It first obtains the preliminary node embedding through relation-aware embedding and then interactively aggregates the local and global memory network to fuse and utilize the local and global information. Finally, the node embeddings of different levels are aggregated through the hierarchical information aggregator. Extensive experiments show our proposed LGM-GNN outperforms other SOAT methods on two real-world fraud detection datasets.

Shared and individual representation learning with Feature Diversity for Deep MultiView Clustering

Due to the remarkable representation ability of Nonnegative matrix factorization (NMF), its multiview variants have become a crucial kind of multiview representation learning methods. However, the majority of existing methods fail to effectively utilize complex higher-level hierarchical information, shared information, and individual information of multiview data. To address these challenges, we propose the Shared and individual representation learning with feature Diversity for Deep Multiview Clustering (SiDDMVC). Specifically, this method decomposes data of each view through nonnegative matrix factorization in a layer-wise fashion to exploit complex higher-level hierarchical information. The outputs of NMF are then divided into shared and individual representations with shared geometrical constraint and feature diversity constraint. Experimental results on six real-world datasets have shown that SiDDMVC not only achieves superior performance through the use of graph regularization and two diversity constraints, but also surpasses several state-of-the-art multiview clustering methods.

Reconstructing controllable faces from brain activity with hierarchical multiview representations

Reconstructing visual experience from brain responses measured by functional magnetic resonance imaging (fMRI) is a challenging yet important research topic in brain decoding, especially it has proved more difficult to decode visually similar stimuli, such as faces. Although face attributes are known as the key to face recognition, most existing methods generally ignore how to decode facial attributes more precisely in perceived face reconstruction, which often leads to indistinguishable reconstructed faces. To solve this problem, we propose a novel neural decoding framework called VSPnet (voxel2style2pixel) by establishing hierarchical encoding and decoding networks with disentangled latent representations as media, so that to recover visual stimuli more elaborately. And we design a hierarchical visual encoder (named HVE) to pre-extract features containing both high-level semantic knowledge and low-level visual details from stimuli. The proposed VSPnet consists of two networks: Multi-branch cognitive encoder and style-based image generator. The encoder network is constructed by multiple linear regression branches to map brain signals to the latent space provided by the pre-extracted visual features and obtain representations containing hierarchical information consistent to the corresponding stimuli. We make the generator network inspired by StyleGAN to untangle the complexity of fMRI representations and generate images. And the HVE network is composed of a standard feature pyramid over a ResNet backbone. Extensive experimental results on the latest public datasets have demonstrated the reconstruction accuracy of our proposed method outperforms the state-of-the-art approaches and the identifiability of different reconstructed faces has been greatly improved. In particular, we achieve feature editing for several facial attributes in fMRI domain based on the multiview (i.e., visual stimuli and evoked fMRI) latent representations.

FURSformer: Semantic Segmentation Network for Remote Sensing Images with Fused Heterogeneous Features

Semantic segmentation of remote sensing images poses a formidable challenge within this domain. Our investigation commences with a pilot study aimed at scrutinizing the advantages and disadvantages of employing a Transformer architecture and a CNN architecture in remote sensing imagery (RSI). Our objective is to substantiate the indispensability of both local and global information for RSI analysis. In this research article, we harness the potential of the Transformer model to establish global contextual understanding while incorporating an additional convolution module for localized perception. Nonetheless, a direct fusion of these heterogeneous information sources often yields subpar outcomes. To address this limitation, we propose an innovative hierarchical fusion feature information module that this model can fuse Transformer and CNN features using an ensemble-to-set approach, thereby enhancing information compatibility. Our proposed model, named FURSformer, amalgamates the strengths of the Transformer architecture and CNN. The experimental results clearly demonstrate the effectiveness of this approach. Notably, our model achieved an outstanding accuracy of 90.78% mAccuracy on the DLRSD dataset.

MOHP-EC: A Multiobjective Hierarchical Prediction Framework for Urban Rail Transit Passenger Flow

For sophisticated management, advertisement placement, and epidemic prevention control of urban rail transit (URT), accurate and real-time predictions of passenger flows at different levels are of great importance. Unlike traditional prediction tasks, hierarchical prediction (HP) requires that the hierarchical constraints be satisfied as much as possible (the sum of the predicted passenger flow of child nodes should nearly equal the parent node) to achieve realistic predictions. This article proposes a multiobjective HP (MOHP) framework with an error compensation (EC) mechanism for predicting URT passenger flow with a hierarchical structure. Three components are included: the initial prediction module, the EC module, and the hierarchical coordination module. In the initial prediction module, the initial passenger flow prediction of each layer is carried out. The EC model is developed based on proportional-integral-derivative control to compensate for the initial predicted value of every layer. As a final step, a trainable HP model is constructed based on deep learning to coordinate the prediction values of each layer. As examples, we construct three scenarios of passenger flow hierarchy based on the URT system in Wuxi, China. The constructed prediction framework is used to conduct experimental analyses. As a result, the MOHP-EC prediction framework could satisfy the hierarchical constraints and use the passenger flow hierarchical information to reduce prediction errors. The mean absolute error was reduced by 35%, and the root-mean-square error was reduced by 39%.

SLOTH: Structured Learning and Task-Based Optimization for Time Series Forecasting on Hierarchies

Multivariate time series forecasting with hierarchical structure is widely used in real-world applications, e.g., sales predictions for the geographical hierarchy formed by cities, states, and countries. The hierarchical time series (HTS) forecasting includes two sub-tasks, i.e., forecasting and reconciliation. In the previous works, hierarchical information is only integrated in the reconciliation step to maintain coherency, but not in forecasting step for accuracy improvement. In this paper, we propose two novel tree-based feature integration mechanisms, i.e., top-down convolution and bottom-up attention to leverage the information of the hierarchical structure to improve the forecasting performance. Moreover, unlike most previous reconciliation methods which either rely on strong assumptions or focus on coherent constraints only, we utilize deep neural optimization networks, which not only achieve coherency without any assumptions, but also allow more flexible and realistic constraints to achieve task-based targets, e.g., lower under-estimation penalty and meaningful decision-making loss to facilitate the subsequent downstream tasks. Experiments on real-world datasets demonstrate that our tree-based feature integration mechanism achieves superior performances on hierarchical forecasting tasks compared to the state-of-the-art methods, and our neural optimization networks can be applied to real-world tasks effectively without any additional effort under coherence and task-based constraints.

Yet Another Traffic Classifier: A Masked Autoencoder Based Traffic Transformer with Multi-Level Flow Representation

Traffic classification is a critical task in network security and management. Recent research has demonstrated the effectiveness of the deep learning-based traffic classification method. However, the following limitations remain: (1) the traffic representation is simply generated from raw packet bytes, resulting in the absence of important information; (2) the model structure of directly applying deep learning algorithms does not take traffic characteristics into account; and (3) scenario-specific classifier training usually requires a labor-intensive and time-consuming process to label data. In this paper, we introduce a masked autoencoder (MAE) based traffic transformer with multi-level flow representation to tackle these problems. To model raw traffic data, we design a formatted traffic representation matrix with hierarchical flow information. After that, we develop an efficient Traffic Transformer, in which packet-level and flow-level attention mechanisms implement more efficient feature extraction with lower complexity. At last, we utilize the MAE paradigm to pre-train our classifier with a large amount of unlabeled data, and perform fine-tuning with a few labeled data for a series of traffic classification tasks. Experiment findings reveal that our method outperforms state-of-the-art methods on five real-world traffic datasets by a large margin. The code is available at https://github.com/NSSL-SJTU/YaTC.

Hierarchical Text Classification as Sub-hierarchy Sequence Generation

Hierarchical text classification (HTC) is essential for various real applications. However, HTC models are challenging to develop because they often require processing a large volume of documents and labels with hierarchical taxonomy. Recent HTC models based on deep learning have attempted to incorporate hierarchy information into a model structure. Consequently, these models are challenging to implement when the model parameters increase for a large-scale hierarchy because the model structure depends on the hierarchy size. To solve this problem, we formulate HTC as a sub-hierarchy sequence generation to incorporate hierarchy information into a target label sequence instead of the model structure. Subsequently, we propose the Hierarchy DECoder (HiDEC), which decodes a text sequence into a sub-hierarchy sequence using recursive hierarchy decoding, classifying all parents at the same level into children at once. In addition, HiDEC is trained to use hierarchical path information from a root to each leaf in a sub-hierarchy composed of the labels of a target document via an attention mechanism and hierarchy-aware masking. HiDEC achieved state-of-the-art performance with significantly fewer model parameters than existing models on benchmark datasets, such as RCV1-v2, NYT, and EURLEX57K.

Hierarchical quantum information splitting of an arbitrary m-qudit state with multiparty

Hierarchical quantum information splitting of an arbitrary m-qudit state with multiparty

Towards a Holistic Understanding of Mathematical Questions with Contrastive Pre-training

Understanding mathematical questions effectively is a crucial task, which can benefit many applications, such as difficulty estimation. Researchers have drawn much attention to designing pre-training models for question representations due to the scarcity of human annotations (e.g., labeling difficulty). However, unlike general free-format texts (e.g., user comments), mathematical questions are generally designed with explicit purposes and mathematical logic, and usually consist of more complex content, such as formulas, and related mathematical knowledge (e.g., Function). Therefore, the problem of holistically representing mathematical questions remains underexplored. To this end, in this paper, we propose a novel contrastive pre-training approach for mathematical question representations, namely QuesCo, which attempts to bring questions with more similar purposes closer. Specifically, we first design two-level question augmentations, including content-level and structure-level, which generate literally diverse question pairs with similar purposes. Then, to fully exploit hierarchical information of knowledge concepts, we propose a knowledge hierarchy-aware rank strategy (KHAR), which ranks the similarities between questions in a fine-grained manner. Next, we adopt a ranking contrastive learning task to optimize our model based on the augmented and ranked questions. We conduct extensive experiments on two real-world mathematical datasets. The experimental results demonstrate the effectiveness of our model.