Maximum Mutual Information Research Articles

SENSE: An unsupervised semantic learning model for cross-platform vulnerability search

Binary Similarity Analysis (BSA) emerges as a vital approach for identifying homologous vulnerabilities. However, it is constrained by semantic incompleteness, structural differences, and false positives arising from variations in compilation environments. In this paper, we propose a novel Unsupervised Semantic Learning Model named SENSE for cross-platform vulnerability search. The model comprises two main components: semantic learner and graph learner. The semantic learner is pre-trained with a mask language task on a well-normalized binary corpus, enabling it to capture contextual semantic relations and generate block embedding that effectively encode the semantic features. In the graph learner, a gated graph neural network with a self-gating layer is adopted to eliminate redundant features and an adversarial loss is incorporated to enhance the robustness of function embedding across different compiler environments. Finally, SENSE is trained in an unsupervised manner using a batch-wise sampling strategy along with maximum mutual information loss. This encourages semantically similar functions to exhibit tighter embedding representations, thereby reducing false positives and improving search efficiency. Through extensive experiments, we have demonstrated that SENSE outperforms state-of-the-art methods in terms of binary search accuracy. Our results also reveal that SENSE is capable of generating robust function embedding that mitigate the differences arising from diverse architecture and optimization options.

Improved Network and Training Scheme for Cross-Trial Surface Electromyography (sEMG)-Based Gesture Recognition.

To enhance the performance of surface electromyography (sEMG)-based gesture recognition, we propose a novel network-agnostic two-stage training scheme, called sEMGPoseMIM, that produces trial-invariant representations to be aligned with corresponding hand movements via cross-modal knowledge distillation. In the first stage, an sEMG encoder is trained via cross-trial mutual information maximization using the sEMG sequences sampled from the same time step but different trials in a contrastive learning manner. In the second stage, the learned sEMG encoder is fine-tuned with the supervision of gesture and hand movements in a knowledge-distillation manner. In addition, we propose a novel network called sEMGXCM as the sEMG encoder. Comprehensive experiments on seven sparse multichannel sEMG databases are conducted to demonstrate the effectiveness of the training scheme sEMGPoseMIM and the network sEMGXCM, which achieves an average improvement of +1.3% on the sparse multichannel sEMG databases compared to the existing methods. Furthermore, the comparison between training sEMGXCM and other existing networks from scratch shows that sEMGXCM outperforms the others by an average of +1.5%.

Improved conversational recommender system based on dialog context

Abstract Conversational recommender system (CRS) needs to be seamlessly integrated between the two modules of recommendation and dialog, aiming to recommend high-quality items to users through multiple rounds of interactive dialogs. Items can typically refer to goods, movies, news, etc. Through this form of interactive dialog, users can express their preferences in real time, and the system can fully understand the user’s thoughts and recommend corresponding items. Although mainstream dialog recommendation systems have improved the performance to some extent, there are still some key issues, such as insufficient consideration of the entity’s order in the dialog, the different contributions of items in the dialog history, and the low diversity of generated responses. To address these shortcomings, we propose an improved dialog context model based on time-series features. Firstly, we augment the semantic representation of words and items using two external knowledge graphs and align the semantic space using mutual information maximization techniques. Secondly, we add a retrieval model to the dialog recommendation system to provide auxiliary information for generating replies. We then utilize a deep timing network to serialize the dialog content and more accurately learn the feature relationship between users and items for recommendation. In this paper, the dialog recommendation system is divided into two components, and different evaluation indicators are used to evaluate the performance of the dialog component and the recommendation component. Experimental results on widely used benchmarks show that the proposed method is effective.

Multimodal Mutual Information Maximization: A Novel Approach for Unsupervised Deep Cross-Modal Hashing.

In this article, we adopt the maximizing mutual information (MI) approach to tackle the problem of unsupervised learning of binary hash codes for efficient cross-modal retrieval. We proposed a novel method, dubbed cross-modal info-max hashing (CMIMH). First, to learn informative representations that can preserve both intramodal and intermodal similarities, we leverage the recent advances in estimating variational lower bound of MI to maximizing the MI between the binary representations and input features and between binary representations of different modalities. By jointly maximizing these MIs under the assumption that the binary representations are modeled by multivariate Bernoulli distributions, we can learn binary representations, which can preserve both intramodal and intermodal similarities, effectively in a mini-batch manner with gradient descent. Furthermore, we find out that trying to minimize the modality gap by learning similar binary representations for the same instance from different modalities could result in less informative representations. Hence, balancing between reducing the modality gap and losing modality-private information is important for the cross-modal retrieval tasks. Quantitative evaluations on standard benchmark datasets demonstrate that the proposed method consistently outperforms other state-of-the-art cross-modal retrieval methods.

Learning Deep Generative Clustering via Mutual Information Maximization.

Deep clustering refers to joint representation learning and clustering using deep neural networks. Existing methods can be mainly categorized into two types: discriminative and generative methods. The former learns representations for clustering with discriminative mechanisms directly, and the latter estimate the latent distribution of each cluster for generating data points and then infers cluster assignments. Although generative methods have the advantage of estimating the latent distributions of clusters, their performances still significantly fall behind discriminative methods. In this work, we argue that this performance gap might be partly due to the overlap of data distribution of different clusters. In fact, there is little guarantee of generative methods to separate the distributions of different clusters in the data space. To tackle these problems, we theoretically prove that mutual information maximization promotes the separation of different clusters in the data space, which provides a theoretical justification for deep generative clustering with mutual information maximization. Our theoretical analysis directly leads to a model which integrates a hierarchical generative adversarial network and mutual information maximization. Moreover, we further propose three techniques and empirically show their effects to stabilize and enhance the model. The proposed approach notably outperforms other generative models for deep clustering on public benchmarks.

Multiscale Fuzzy Entropy-Based Feature Selection

In practice, it is common that there will be the same decision results under different scale conditions. Therefore, knowledge representation based on a single scale feature framework is far from meeting the needs of practical applications. Based on this, multi-scale data has received extensive attention. Feature selection is an important application of fuzzy multigranularity data analysis model. The existing multi-scale fuzzy granulation-based feature selection methods remove redundant or irrelevant features by selecting the optimal scale. However, this will lose the information corresponding to the remaining scale fuzzy granules, which will affect the classification results or learning tasks. Inspired by this, multi-scale fuzzy entropy is defined to fuse the granule information at different scales, and applied to feature selection. First, the feature with maximum multi-scale fuzzy mutual information is first selected. Then, the most significant features are gradually selected by evaluation metric that simultaneously considers the redundancy, relevance and complementary. A multi-scale fuzzy entropy-based feature selection (MFEFS) algorithm by means of this evaluation index is further designed. Finally, the proposed method is compared with some state-of-the-art methods. The experimental results show that the proposed algorithm has higher reduction efficiency than the comparison algorithms.

Decoding Visual Neural Representations by Multimodal Learning of Brain-Visual-Linguistic Features.

Decoding human visual neural representations is a challenging task with great scientific significance in revealing vision-processing mechanisms and developing brain-like intelligent machines. Most existing methods are difficult to generalize to novel categories that have no corresponding neural data for training. The two main reasons are 1) the under-exploitation of the multimodal semantic knowledge underlying the neural data and 2) the small number of paired (stimuli-responses) training data. To overcome these limitations, this paper presents a generic neural decoding method called BraVL that uses multimodal learning of brain-visual-linguistic features. We focus on modeling the relationships between brain, visual and linguistic features via multimodal deep generative models. Specifically, we leverage the mixture-of-product-of-experts formulation to infer a latent code that enables a coherent joint generation of all three modalities. To learn a more consistent joint representation and improve the data efficiency in the case of limited brain activity data, we exploit both intra- and inter-modality mutual information maximization regularization terms. In particular, our BraVL model can be trained under various semi-supervised scenarios to incorporate the visual and textual features obtained from the extra categories. Finally, we construct three trimodal matching datasets, and the extensive experiments lead to some interesting conclusions and cognitive insights: 1) decoding novel visual categories from human brain activity is practically possible with good accuracy; 2) decoding models using the combination of visual and linguistic features perform much better than those using either of them alone; 3) visual perception may be accompanied by linguistic influences to represent the semantics of visual stimuli.

Semi-Supervised Deep Learning-Based Multi-component Spectral Calibration Modeling for UV-vis and Near-Infrared Spectroscopy without Information Loss.

Spectral analysis is an important method for characterizing and identifying chemical species. However, quantitative spectral analysis of multiple chemical properties in the real world has always been a challenging problem due to the strong correlation, massive noise, and serious information overlapping of the spectral features. Here, we present a new semi-supervised spectral calibration method based on information lossless decoupling of spectral features named NICEM. To realize the separation and extraction of key latent features, the method uses the flow-based model non-linear independent component estimation (NICE) to learn the sample distribution. The spectral data information is transformed into independent latent variables obeying Gaussian distribution by the reversible structure of deep network without information loss, so as to find the essential properties and realize the feature nonlinear decomposition. Moreover, the association between the input latent feature variables and attributes is evaluated by the maximum mutual information coefficient to eliminate the adverse effects of irrelevant information in the latent variable space and mine key information. Since the latent variables are independent in each dimension, the NICEM method is easier to establish an accurate semi-supervised multi-component calibration model even for high overlapping and complex spectral data. The applicability of the proposed spectral modeling method is demonstrated by using three ultraviolet-visible and near-infrared spectral data sets with 15 physical and chemical properties including diesel fuels, corn, and multi-metal ions solution. Results show that the proposed NICEM method has the highest determination coefficient (R2) and significantly improves extrapolation compared with the seven state-of-the-art methods. The proposed method is intuitive because it obviates complex feature engineering and prior knowledge and is a promising spectral calibration tool for quantitative analysis in other spectroscopy applications.

Accurate non-invasive grading of nonfunctional pancreatic neuroendocrine tumors with a CT derived radiomics signature

PurposeThe purpose of this study was to develop a radiomics-signature using computed tomography (CT) data for the preoperative prediction of grade of nonfunctional pancreatic neuroendocrine tumors (NF-PNETs). Materials and methodsA retrospective study was performed on patients undergoing resection for NF-PNETs between 2010 and 2019. A total of 2436 radiomic features were extracted from arterial and venous phases of pancreas-protocol CT examinations. Radiomic features that were associated with final pathologic grade observed in the surgical specimens were subjected to joint mutual information maximization for hierarchical feature selection and the development of the radiomic-signature. Youden-index was used to identify optimal cutoff for determining tumor grade. A random forest prediction model was trained and validated internally. The performance of this tool in predicting tumor grade was compared to that of EUS-FNA sampling that was used as the standard of reference. ResultsA total of 270 patients were included and a fusion radiomic-signature based on 10 selected features was developed using the development cohort (n = 201). There were 149 men and 121 women with a mean age of 59.4 ± 12.3 (standard deviation) years (range: 23.3–85.0 years). Upon internal validation in a new set of 69 patients, a strong discrimination was observed with an area under the curve (AUC) of 0.80 (95% confidence interval [CI]: 0.71–0.90) with corresponding sensitivity and specificity of 87.5% (95% CI: 79.7–95.3) and 73.3% (95% CI: 62.9–83.8) respectively. Of the study population, 143 patients (52.9%) underwent EUS-FNA. Biopsies were non-diagnostic in 26 patients (18.2%) and could not be graded due to insufficient sample in 42 patients (29.4%). In the cohort of 75 patients (52.4%) in whom biopsies were graded the radiomic-signature demonstrated not different AUC as compared to EUS-FNA (AUC: 0.69 vs. 0.67; P = 0.723), however greater sensitivity (i.e., ability to accurately identify G2/3 lesion was observed (80.8% vs. 42.3%; P < 0.001). ConclusionNon-invasive assessment of tumor grade in patients with PNETs using the proposed radiomic-signature demonstrated high accuracy. Prospective validation and optimization could overcome the commonly experienced diagnostic uncertainty in the assessment of tumor grade in patients with PNETs and could facilitate clinical decision-making.

Mutual information measure of visual perception based on noisy spiking neural networks.

Note that images of low-illumination are weak aperiodic signals, while mutual information can be used as an effective measure for the shared information between the input stimulus and the output response of nonlinear systems, thus it is possible to develop novel visual perception algorithm based on the principle of aperiodic stochastic resonance within the frame of information theory. To confirm this, we reveal this phenomenon using the integrate-and-fire neural networks of neurons with noisy binary random signal as input first. And then, we propose an improved visual perception algorithm with the image mutual information as assessment index. The numerical experiences show that the target image can be picked up with more easiness by the maximal mutual information than by the minimum of natural image quality evaluation (NIQE), which is one of the most frequently used indexes. Moreover, the advantage of choosing quantile as spike threshold has also been confirmed. The improvement of this research should provide large convenience for potential applications including video tracking in environments of low illumination.

Radiomic analysis for early differentiation of lung cancer recurrence from fibrosis in patients treated with lung stereotactic ablative radiotherapy.

The development of radiation-induced fibrosis after SABR can obscure follow-up images and delay detection of a local recurrence in early-stage lung cancer patients. The objective of this study was to develop a radiomics model for computer-assisted detection of local recurrence and fibrosis for an earlier timepoint (<1 year) after the SABR treatment.&#xD;&#xD;This retrospective clinical study included CT images (n=107) of 66 patients treated with SABR. A z-score normalization technique was used for radiomic feature standardization across scanner protocols. The training set for the radiomics model consisted of CT images (66 patients; 22 recurrences and 44 fibrosis) obtained at 24 months (median) follow-up. The test set included CT-images of 41 patients acquired at 5-12 months follow-up. Combinations of four widely used machine learning techniques (support vector machines (SVMs), gradient boosting, random forests (RF), and logistic regression (LR)) and feature selection methods (Relief feature scoring, maximum relevance minimum redundancy (MRMR), mutual information maximization (MIM), forward feature selection, and LASSO) were investigated. Pyradiomics was used to extract 106 radiomic features from the CT-images for feature selection and classification. &#xD;&#xD;An RF + LASSO model scored the highest in terms of AUC (0.87) and obtained a sensitivity of 75% and a specificity of 88% in identifying a local recurrence in the test set. In the training set, 86% accuracy was achieved using five-fold cross-validation. Delong's test indicated that AUC achieved by the RF+LASSO is significantly better than 11 other machine learning models presented here. The top three radiomic features: Interquartile Range (first order), Cluster Prominence (GLCM), and Autocorrelation (GLCM), were revealed as differentiating a recurrence from fibrosis with this model.&#xD;&#xD;The radiomics model selected, out of multiple machine learning and feature selection algorithms, was able to differentiate a recurrence from fibrosis in earlier follow-up CT-images with a high specificity rate and satisfactory sensitivity performance. &#xD.

Covert transmission performance for visible light communications under optical intensity and covert constraints

This paper investigates the fundamental performance limit for an indoor covert visible light communication (VLC) system consisting of a transmitter, a receiver, and a warden. In the system, the input signal of VLC is constrained by nonnegativity, peak optical intensity constraint, average optical intensity constraint and covertness constraint. We derive the optimal distribution of the input signal by using the variational method and the entropy power inequality. The allowable maximum peak optical intensity is obtained by substituting the optimal input signal into the covertness constraint. Then, by substituting the maximum peak optical intensity into the lower bound of the maximum mutual information, we derive an upper bound of the information bits that can be covertly transmitted. The asymptotic analysis shows that the upper bound on the transmittable covert information bits tends to a constant when the number of channel uses tends to infinity. These conclusions are also confirmed by numerical results.

Hic-KGQA: Improving multi-hop question answering over knowledge graph via hypergraph and inference chain

Question answering over knowledge graph (KGQA) aims at answering natural language questions posed over knowledge graphs (KGs). Moreover, multi-hop KGQA requires reasoning across multiple triplets in KGs to get to the answer. Unfortunately, KGs often lack complete information and contain many missing links, which poses huge challenges for multi-hop KGQA. To address this, recent several approaches have introduced KG embedding techniques, which have shown good performance on the multi-hop KGQA task. However, these methods ignore the semantic correlations between paths and questions, and the reasoning process is not easily explained. Furthermore, traditional KG embedding methods consider only low-order pairwise relations and ignore the higher-order relations among entities, leading to a sub-optimal embedding result. To address these problems, we propose Hic-KGQA, a novel hypergraph and inference chain-based model for multi-hop KGQA. Specifically, Hic-KGQA first generates pre-trained entity embeddings with multiple semantics via a hypergraph-based KGC module (HKM). Then, an inference chain modeling module (ICMM) is designed to learn the importance of different inference chains for the question and encode the highest-ranked inference chain into an embedding representation. Finally, two scoring networks are used to evaluate the correlation between the candidate answers and the questions from the perspective of both the triplet facts and the reasoning process to obtain more accurate answers. Furthermore, the mutual information maximization (MIM) is innovatively implemented to capture richer common path features from similar cases to alleviate the missing path problem caused by KG incompleteness. Experiments show that Hic-KGQA significantly outperforms existing state-of-the-art methods and is explainable.

ADscreen: A speech processing-based screening system for automatic identification of patients with Alzheimer's disease and related dementia

Alzheimer's disease and related dementias (ADRD) present a looming public health crisis, affecting roughly 5 million people and 11 % of older adults in the United States. Despite nationwide efforts for timely diagnosis of patients with ADRD, >50 % of them are not diagnosed and unaware of their disease. To address this challenge, we developed ADscreen, an innovative speech-processing based ADRD screening algorithm for the protective identification of patients with ADRD. ADscreen consists of five major components: (i) noise reduction for reducing background noises from the audio-recorded patient speech, (ii) modeling the patient's ability in phonetic motor planning using acoustic parameters of the patient's voice, (iii) modeling the patient's ability in semantic and syntactic levels of language organization using linguistic parameters of the patient speech, (iv) extracting vocal and semantic psycholinguistic cues from the patient speech, and (v) building and evaluating the screening algorithm. To identify important speech parameters (features) associated with ADRD, we used the Joint Mutual Information Maximization (JMIM), an effective feature selection method for high dimensional, small sample size datasets. Modeling the relationship between speech parameters and the outcome variable (presence/absence of ADRD) was conducted using three different machine learning (ML) architectures with the capability of joining informative acoustic and linguistic with contextual word embedding vectors obtained from the DistilBERT (Bidirectional Encoder Representations from Transformers). We evaluated the performance of the ADscreen on an audio-recorded patients' speech (verbal description) for the Cookie-Theft picture description task, which is publicly available in the dementia databank. The joint fusion of acoustic and linguistic parameters with contextual word embedding vectors of DistilBERT achieved F1-score = 84.64 (standard deviation [std] = ±3.58) and AUC-ROC = 92.53 (std = ±3.34) for training dataset, and F1-score = 89.55 and AUC-ROC = 93.89 for the test dataset. In summary, ADscreen has a strong potential to be integrated with clinical workflow to address the need for an ADRD screening tool so that patients with cognitive impairment can receive appropriate and timely care.

Fault detection of a reciprocating plunger pump with fault-free data based on an unsupervised feature encoder and minimum covariance determinant

It is well known that complex mechanical equipment has many differnt the failure modes, and monitoring data for fault conditions are scarce. Therefore, research on fault detection for reciprocating plunger pumps with fault-free data is significant for theory and application. Due to the lack of prior knowledge of faults, it is always a great challenge for researchers to extract fault features from signals. In this paper, an innovative fault detection method for a reciprocating plunger pump is proposed based on an unsupervised feature encoder (UFE) and minimum covariance determination (MCD). Firstly, a criterion based on mutual information maximization for local and global features is proposed for unsupervised feature extraction. In addition, an unsupervised training strategy based on negative sampling is proposed to train the encoder, so that the model can converge quickly and improve the stability of model training. A fault detection algorithm for a reciprocating plunger pump is proposed based on a UFE and MCD. Finally, the effectiveness and superiority of the proposed method are verified with the measured data for a reciprocating plunger pump. The results show that the proposed method can accurately detect the faults in the reciprocating plunger pump with a detection accuracy of more than 98%. Compared with other methods, the proposed fault detection algorithm has better applicability and accuracy for fault detection with fault-free data.

Amer: A New Attribute-Missing Network Embedding Approach.

Network embedding which aims to learn a low dimensional representation of nodes is a powerful technique for network analysis. While network embedding for networks with complete attributes has been widely investigated, in many real-world applications the attributes of partial nodes are unobserved (i.e., missing) due to privacy concern or resource limit. Very recently, several network embedding methods have been proposed for attribute-missing networks. They first complete the missing attributes and then use the complemented network to learn network embedding. The parameters of these two processes cannot be adjusted by each other, resulting in compromised results. To address this problem, we propose a unified model in which the process of completing missing attributes and the process of learning embedding are not separated but closely intertwined. Being specific, completing missing attributes is under the guidance of learning network representation via mutual information maximization, and the complemented attributes directly enter network representation module which will generate further feedback for completing missing attributes. We further impose attribute-structure relationship constraint for completing missing attributes by designing a new generative adversarial networks (GANs) model. To the best of our knowledge, this is the first unified model for attribute-missing network embedding. Empirical results on real-world datasets show the superiority of our new method over other state-of-the-art methods on four network analysis tasks, including node classification, node clustering, link prediction, and network visualization.

SSMI: Semantic Similarity and Mutual Information Maximization Based Enhancement for Chinese NER

The Chinese NER task consists of two steps, first determining entity boundaries and then labeling them. Some previous work incorporating related words from pre-trained vocabulary into character-based models has been demonstrated to be effective. However, the number of words that characters can match in the vocabulary is large, and their meanings vary widely. It is unreasonable to concatenate all the matched words into the character's representation without making semantic distinctions. This is because words with different semantics also have distinct vectors by the distributed representation. Moreover, mutual information maximization (MIM) provides a unified way to characterize the correction between different granularity of embeddings, we find it can be used to enhance the features in our task. Consequently, this paper introduces a novel Chinese NER model named SSMI based on semantic similarity and MIM. We first match all the potential word boundaries of the input characters from the pre-trained vocabulary and employ BERT to segment the input sentence to get the segmentation containing these characters. After computing their cosine similarity, we obtain the word boundary with the highest similarity and the word group with similarity score larger than a specific threshold. Then, we concatenate the most relevant word boundaries with character vectors. We further calculate the mutual information maximization of group, character and sentence, respectively. Finally, we feed the result from the above steps to our novel network. The results on four Chinese public NER datasets show that our SSMI achieves state-of-the-art performance.

Fair Representation Learning for Recommendation: A Mutual Information Perspective

Recommender systems have been widely used in recent years. By exploiting historical user-item interactions, recommender systems can model personalized potential interests of users and have been widely applied to a wide range of scenarios. Despite their impressive performance, most of them may be subject to unwanted biases related to sensitive attributes (e.g., race and gender), leading to unfairness. An intuitive idea to alleviate this problem is to ensure that there is no mutual information between recommendation results and sensitive attributes. However, keeping independence conditions solely achieves fairness improvement while causing an obvious degradation of recommendation accuracy, which is not a desired result. To this end, in this paper, we re-define recommendation fairness with a novel two-fold mutual information objective. In concerned details, we define fairness as mutual information minimization between embeddings and sensitive information, and mutual information maximization between embeddings and non-sensitive information. Then, a flexible Fair Mutual Information (FairMI) framework is designed to achieve this goal. FairMI first employs a sensitive attribute encoder to capture sensitive information in the data. Then, based on results from the sensitive attribute encoder, an interest encoder is developed to generate sensitive-free embeddings, which are expected to contain rich non-sensitive information of input data. Moreover, we propose novel mutual information (upper/lower) bounds with contrastive information estimation for model optimization. Extensive experiments over two real-world datasets demonstrate the effectiveness of our proposed FairMI in reducing unfairness and improving recommendation accuracy simultaneously.

Study on the influence of high and low temperature environment on the energy consumption of battery electric vehicles

In order to study the influence of high and low temperature environments on the energy consumption of battery electric vehicles, this paper takes Tianjin as an example to collect the actual road data of 8 battery electric passenger cars for eight months. It determines the correlation between each characteristic parameter and energy consumption based on the maximum mutual information coefficient of short stroke segments, of which the average driving speed correlation is the highest, which is 0.785. Given the two factors that have a greater impact on energy consumption in high and low temperature environments: driving resistance and air conditioning on motion and idle energy consumption analysis, the study shows that in terms of driving resistance, high temperature and low temperature energy consumption are reduced and increased by 5.4% and 13.3% respectively relative to normal temperature; The energy consumption of high temperature and low temperature air conditioner switching on increased by 17.12% and 47.48% respectively compared with the air conditioner off.

Continuous-time graph directed information maximization for temporal network representation

Given a temporal network that is composed of a stream of events that evolves continuously over time, like social posts on Reddit, how can we spot interesting events like anomalous posts? In general, it is critical to learn useful information from event time, attributes, and network structure, which means that the anomalous post can be detected by over-frequent posts, impolite comments, and incorrect choices of subreddits. In this paper, we propose a new learning objective, named Continuous-time Graph Directed Information Maximization (CGDIM), to learn informative node presentations for temporal networks. The proposed CGDIM is based on the popular mutual information maximization (InfoMax) method to learn node features that are shared across temporal neighborhoods. Specifically, to capture the time causal relations among edges with continuous time, we utilize directed information rather than mutual information as the measure. The designed learning objective is to maximize the directed information with the direction from the inputs of temporal neighborhoods to the node representations. By setting a source variable that only contains useful information on each node, we prove that directed information is more effective than mutual information for preserving information of the source variables. To show the effectiveness of the CGDIM, we train several popular backbone models for temporal network representation with the CGDIM and test the performance on temporal node/edge classification. Experiment results on four real-world temporal network datasets show that the proposed CGDIM can improve the Average-Precision scores of the backbone models by 3%-8%.