Mutual Information Approach Research Articles

Integrated Anomaly Detection and Early Warning System for Forest Fires in the Odisha Region

The present study aims to develop a random forest algorithm-based classifier to predict the occurrence of fire events using observed meteorological parameters a day in advance. We considered the skin temperature, the air temperature close to the surface, the humidity close to the surface level, and soil moisture as important meteorological factors influencing forest fire occurrence. Twenty additional parameters were derived based on these four parameters that account for the energy exchanged in sensible and latent forms and the change in parameters in recent trends. We used the mutual information approach to identify critical meteorological parameters that carry significant information about fire occurrence the next day. The top nine parameters were then fed as input to the random forest algorithm to predict fire/no fire the next day. The weighted data sampling and SMOTE techniques were employed to address the class imbalance in the fire data class. Both techniques correctly classified fire incidents well, given the meteorological input from the previous days. This study also showed that as the class imbalance increases to 1:9, the performance based on the precision, recall, F1 score, and accuracy are maximum, showing the model’s ability to perform with class imbalance. Both techniques helped the random forest algorithm forecast fire instances as the data sample size increased.

An efficient data fusion model based on Bayesian model averaging for robust water quality prediction using deep learning strategies

Accurate monitoring of dissolved oxygen (DO) levels is critical for stakeholders to effectively safeguard water resources and aquatic ecosystem health. This research presents an innovative data fusion framework based on Bayesian model averaging (BMA) by the combination of several neuroscience models (deep learning methodologies) including multilayer perceptron neural network (MLPNN), recurrent neural network (RNN), convolutional neural network (CNN), gated recurrent unit (GRU), long short-term memory (LSTM), and seasonal autoregressive integrated moving average with exogenous variables (SARIMAX). BMA has the capacity to greatly enhance the results attained by standalone approaches. In this study, two feature selection methods such as mutual information (MI) and recursive feature elimination (RFE) applied to select effective predictors and investigate the importance of each input parameters. The techniques were evaluated based on four different metrics and, finally, to demonstrate the usefulness and effectiveness of the newly implemented strategy, it was applied proficiently at the USGS stations, 01427510 and 02336152 within the USA. The findings from analyzing data based on two stations confirmed that the suggested approach (BMA) outperformed other methods such as MLPNN, RNN, CNN, LSTM, and SARIMAX when it came to predicting the levels of DO on a daily basis. In terms of RMSE, MAE and R2, BMA yielded 0.272 mg/L, 0.216 mg/L, and 0.975 using MI technique and 0.320 mg/L, 0.261 mg/L, and 0.965 using RFE method at 01427510 USGS station, respectively. Similarly, based on RMSE, MAE and R2, BMA produced DO prediction by RMSE = 0.352 mg/L, 0.264 mg/L, and 0.968 by MI approach and RMSE = 0.378 mg/L, 0.282 mg/L, and 0.963 via RFE process at 02336152 USGS station, respectively. After analyzing different combinations of input that obtained from feature selection paradigms, it was observed that the variables with the greatest impact on daily dissolved oxygen levels are the dissolved oxygen levels in the previous time period and the water temperature. On the other hand, the pH and turbidity have minimal influence on the daily DO. Finally, the results of this study confirmed that the BMA tool can be efficiently applied to predict DO concentration based on deep learning model outputs.

Forecasting sovereign CDS spreads with a regime‐switching combination method

AbstractWith the growing importance of the sovereign credit default swap (CDS) market, accurate forecasting of sovereign CDS spreads has gained significant attention. In view of the complex volatility in the series of sovereign CDS spreads, this study presents a novel combination forecasting framework, which introduces time‐varying weights to effectively combine diverse individual models. To identify optimal subsets of models, a mutual information approach is employed, while the regime‐switching method is utilized to integrate the selected models. The proposed method's efficacy is validated using data from 65 countries. Empirical findings underscore the superiority of the proposed approach over benchmark models in terms of both horizontal and directional prediction accuracy, particularly when the sovereign CDS data exhibits a balanced distribution between high and low volatility regimes.

Robust phase unwrapping algorithm for gear interferometry based on mutual information and adaptive spin-inpainting approaches

An enhanced robust phase unwrapping algorithm for gear tooth flank measurement by laser interferometry is proposed. The method integrates wrapped phase mutual information estimation with an adaptive spin-inpainting strategy, resulting in improved quality phase maps to guide the phase unwrapping process. Mutual information estimation is employed to investigate potential phase relationships among interference fringes in the wrapped phase map, forming the foundation for the suppression and recovery of abnormal fringes. The adaptive spin-inpainting method, which adjusts in response to fringe density, plays a crucial role in efficiently addressing the restoration of abnormal fringes and obtaining the optimized quality map. With the enhanced new quality map, the approach yields optimized results, enabling a concurrent assessment of unwrapping process efficiency and quality. Several simulations and experiments were conducted to validate the effectiveness of the proposed method. Comparative analyses against commonly employed methods consistently demonstrate the superior performance of the proposed algorithm.

Examining performance calibration in smart power system electricity metering based on environmental perception attention LSTM-network

The operating environment greatly influences the accuracy of power metering devices, resulting in variations and inconsistencies in measurement results across different working situations. A calibration model for power metering devices is proposed in this study, considering a range of environmental circumstances. The first step involves investigating the environmental conditions that impact the accuracy of power metering devices. The mutual information approach is utilized to identify environmental disturbances affecting device accuracy. A machine learning-driven symmetry attention Long Short-Term Memory (LSTM) network addresses measurement errors, capitalizing on the network’s symmetry data knowledge. Ultimately, the efficacy of the suggested approach is substantiated through the utilization of performance indicators, namely, Mean Absolute Percentage Error (MAPE), Root Mean Square Error (RMSE), and Mean Absolute Error (MAE). The results show that the proposed method can effectively reduce the errors of the power measurement device in all quarters, and the error reduction effect is over 10% in the spring, which is better than other models, demonstrating exemplary performance in correcting the calibration errors of the power measurement device.

Multimodal Mutual Information Maximization: A Novel Approach for Unsupervised Deep Cross-Modal Hashing.

In this article, we adopt the maximizing mutual information (MI) approach to tackle the problem of unsupervised learning of binary hash codes for efficient cross-modal retrieval. We proposed a novel method, dubbed cross-modal info-max hashing (CMIMH). First, to learn informative representations that can preserve both intramodal and intermodal similarities, we leverage the recent advances in estimating variational lower bound of MI to maximizing the MI between the binary representations and input features and between binary representations of different modalities. By jointly maximizing these MIs under the assumption that the binary representations are modeled by multivariate Bernoulli distributions, we can learn binary representations, which can preserve both intramodal and intermodal similarities, effectively in a mini-batch manner with gradient descent. Furthermore, we find out that trying to minimize the modality gap by learning similar binary representations for the same instance from different modalities could result in less informative representations. Hence, balancing between reducing the modality gap and losing modality-private information is important for the cross-modal retrieval tasks. Quantitative evaluations on standard benchmark datasets demonstrate that the proposed method consistently outperforms other state-of-the-art cross-modal retrieval methods.

The reliability of the trade dependence network in the tungsten industry chain based on percolation

The unbalanced distribution of globe mineral resources can be addressed partly by the worldwide commerce in those resources. To identify significant international trade linkages and assess the reliability of the global tungsten trade, this paper suggests a modified point mutual information (PMI) and percolation approach. To build a directional network of trade dependence between countries (regions) of representative products at each stage of the tungsten industry chain, we calculate the relationships among trade dependence between countries using countries as nodes, trade dependence intensity as connected edges, and dependence intensity as the ability to cope with risk. The percolation process is replicated using various scenarios (risk disruptions) to filter the network's connected edges. The findings indicate that the downstream and overall networks of the tungsten industry chain are more robust to nonsystemic risks than the upstream and midstream trading networks. Furthermore, the countries with significant total imports and exports are near those with high critical link scores. Additionally, a few countries play an essential role in the tungsten industry chain. The critical countries are upstream tungsten-producing countries, including China, Canada, Russia, and Portugal. At the same time, the United States, Germany, and Japan, which rely on a certain level of in-depth processing technology, are critical downstream countries. Finally, increasing the maximum critical link is crucial for enhancing network reliability. Additionally, it is vital to concentrate on the policy directives of countries with high critical link scores because doing so will help the country create relevant policies and plans.

Beyond linear neural envelope tracking: a mutual information approach

Objective. The human brain tracks the temporal envelope of speech, which contains essential cues for speech understanding. Linear models are the most common tool to study neural envelope tracking. However, information on how speech is processed can be lost since nonlinear relations are precluded. Analysis based on mutual information (MI), on the other hand, can detect both linear and nonlinear relations and is gradually becoming more popular in the field of neural envelope tracking. Yet, several different approaches to calculating MI are applied with no consensus on which approach to use. Furthermore, the added value of nonlinear techniques remains a subject of debate in the field. The present paper aims to resolve these open questions. Approach. We analyzed electroencephalography (EEG) data of participants listening to continuous speech and applied MI analyses and linear models. Main results. Comparing the different MI approaches, we conclude that results are most reliable and robust using the Gaussian copula approach, which first transforms the data to standard Gaussians. With this approach, the MI analysis is a valid technique for studying neural envelope tracking. Like linear models, it allows spatial and temporal interpretations of speech processing, peak latency analyses, and applications to multiple EEG channels combined. In a final analysis, we tested whether nonlinear components were present in the neural response to the envelope by first removing all linear components in the data. We robustly detected nonlinear components on the single-subject level using the MI analysis. Significance. We demonstrate that the human brain processes speech in a nonlinear way. Unlike linear models, the MI analysis detects such nonlinear relations, proving its added value to neural envelope tracking. In addition, the MI analysis retains spatial and temporal characteristics of speech processing, an advantage lost when using more complex (nonlinear) deep neural networks.

Sentiment analysis of vegan related tweets using mutual information for feature selection.

Nowadays, people get increasingly attached to social media to connect with other people, to study, and to work. The presented article uses Twitter posts to better understand public opinion regarding the vegan (plant-based) diet that has traditionally been portrayed negatively on social media. However, in recent years, studies on health benefits, COVID-19, and global warming have increased the awareness of plant-based diets. The study employs a dataset derived from a collection of vegan-related tweets and uses a sentiment analysis technique for identifying the emotions represented in them. The purpose of sentiment analysis is to determine whether a piece of text (tweet in our case) conveys a negative or positive viewpoint. We use the mutual information approach to perform feature selection in this study. We chose this method because it is suitable for mining the complicated features from vegan tweets and extracting users' feelings and emotions. The results revealed that the vegan diet is becoming more popular and is currently framed more positively than in previous years. However, the emotions of fear were mostly strong throughout the period, which is in sharp contrast to other types of emotions. Our findings place new information in the public domain, which has significant implications. The article provides evidence that the vegan trend is growing and new insights into the key emotions associated with this growth from 2010 to 2022. By gaining a deeper understanding of the public perception of veganism, medical experts can create appropriate health programs and encourage more people to stick to a healthy vegan diet. These results can be used to devise appropriate government action plans to promote healthy veganism and reduce the associated emotion of fear.

Feedforward Mutual-Information Anomaly Detection: Application to Autonomous Vehicles

Abstract This paper describes a mutual-information (MI)-based approach that exploits a dynamics model to quantify and detect anomalies for applications such as autonomous vehicles. First, the MI is utilized to quantify the level of uncertainty associated with the driving behaviors of a vehicle. The MI approach handles novel anomalies without the need for data-intensive training; and the metric readily applies to multivariate datasets for improved robustness compared to, e.g., monitoring vehicle tracking error. Second, to further improve the response time of anomaly detection, current and past measurements are combined with a predictive component that utilizes the vehicle dynamics model. This approach compensates for the lag in the anomaly detection process compared to strictly using current and past measurements. Finally, three different MI-based strategies are described and compared experimentally: anomaly detection using MI with (1) current and past measurements (reaction), (2) current and future information (prediction), and (3) a combination of past and future information (reaction–prediction) with three different time windows. The experiments demonstrate quantification and detection of anomalies in three driving situations: (1) veering off the road, (2) driving on the wrong side of the road, and (3) swerving within a lane. Results show that by anticipating the movements of the vehicle, the quality and response time of the anomaly detection are more favorable for decision-making while not raising false alarms compared to just using current and past measurements.

Pervasive conditional selection of driver mutations and modular epistasis networks in cancer.

Cancer driver mutations often display mutual exclusion or co-occurrence, underscoring the key role of epistasis in carcinogenesis. However, estimating the magnitude of epistasis and quantifying its effect on tumor evolution remains a challenge. We develop a method (Coselens) to quantify conditional selection on the excess of nonsynonymous substitutions in cancer genes. Coselens infers the number of drivers per gene in different partitions of a cancer genomics dataset using covariance-based mutation models and determines whether coding mutations in a gene affect selection for drivers in any other gene. Using Coselens, we identify 296 conditionally selected gene pairs across 16 cancer types in the TCGA dataset. Conditional selection affects 25%-50% of driver substitutions in tumors with >2 drivers. Conditionally co-selected genes form modular networks, whose structures challenge the traditional interpretation of within-pathway mutual exclusivity and across-pathway synergy, suggesting a more complex scenario where gene-specific across-pathway epistasis shapes differentiated cancer subtypes.

All-Digital LoS MIMO With Low-Precision Analog-to-Digital Conversion

Line-of-sight (LoS) multi-input multi-output (MIMO) systems exhibit attractive scaling properties with increase in carrier frequency: for a fixed form factor and range, the spatial degrees of freedom increase quadratically for 2D arrays, in addition to the typically linear increase in available bandwidth. In this paper, we investigate whether modern all-digital baseband signal processing architectures can be devised for such regimes, given the difficulty of analog-to-digital conversion for large bandwidths. We propose low-precision quantizer designs and accompanying spatial demultiplexing algorithms, considering <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$2 \times 2$ </tex-math></inline-formula> LoS MIMO with QPSK for analytical insight, and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$4 \times 4$ </tex-math></inline-formula> MIMO with QPSK and 16QAM for performance evaluation. Unlike prior work, channel state information is utilized only at the receiver (i.e., transmit precoding is not employed). We investigate quantizers with regular structure whose high-SNR mutual information approaches that of an unquantized system. We prove that amplitude-phase quantization is necessary to attain this benchmark; phase-only quantization falls short. We show that quantizers based on maximizing per-antenna output entropy perform better than standard Minimum Mean Squared Quantization Error (MMSQE) quantization. For spatial demultiplexing with severely quantized observations, we introduce the novel concept of virtual quantization which, combined with linear detection, provides reliable demodulation at significantly reduced complexity compared to maximum likelihood detection.

Exploring Non-Linear Dependencies in Atmospheric Data with Mutual Information

Relations between atmospheric variables are often non-linear, which complicates research efforts to explore and understand multivariable datasets. We describe a mutual information approach to screen for the most significant associations in this setting. This method robustly detects linear and non-linear dependencies after minor data quality checking. Confounding factors and seasonal cycles can be taken into account without predefined models. We present two case studies of this method. The first one illustrates deseasonalization of a simple time series, with results identical to the classical method. The second one explores associations in a larger dataset of many variables, some of them lognormal (trace gas concentrations) or circular (wind direction). The examples use our Python package ‘ennemi’.

A new approach to analyze the independence of statistical tests of randomness

One of the fundamental aspects when working with batteries of statistic tests is that they should be as efficient as possible, i.e. that they should check the properties and do so in a reasonable computational time. This assumes that there are no tests that are checking the same properties, i.e. that they are not correlated. One of the most commonly used measures to verify the interrelation between variables is the Pearson’s correlation. In this case, linear dependencies are checked, but it may be interesting to verify other types of non-linear relationships between variables. For this purpose, mutual information has recently been proposed, which measures how much information, on average, one random variable provides to another. In this work we analyze some well-known batteries by using correlation analysis and mutual information approaches.

Information sharing among cryptocurrencies: Evidence from mutual information and approximate entropy during COVID-19

In this paper, we use mutual information approach to investigate the information sharing between cryptocurrencies during the COVID-19 crisis. We also use the approximate entropy to study their dynamics before COVID-19 and during the pandemic. Results from the mutual information measure indicate a rise in information sharing and ordering in the cryptocurrency markets in the pandemic period, while the evidence from the approximate entropy estimates indicates a rise in randomness during the COVID-19 period. Our results provide new insights on the information sharing of cryptocurrencies and their reaction to shocks such as the COVID-19 pandemic.

Application of Rough and Fuzzy Set Theory for Prediction of Stochastic Wind Speed Data Using Long Short-Term Memory

Despite the great significance of precisely forecasting the wind speed for development of the new and clean energy technology and stable grid operators, the stochasticity of wind speed makes the prediction a complex and challenging task. For improving the security and economic performance of power grids, accurate short-term wind power forecasting is crucial. In this paper, a deep learning model (Long Short-term Memory (LSTM)) has been proposed for wind speed prediction. Knowing that wind speed time series is nonlinear stochastic, the mutual information (MI) approach was used to find the best subset from the data by maximizing the joint MI between subset and target output. To enhance the accuracy and reduce input characteristics and data uncertainties, rough set and interval type-2 fuzzy set theory are combined in the proposed deep learning model. Wind speed data from an international airport station in the southern coast of Iran Bandar-Abbas City was used as the original input dataset for the optimized deep learning model. Based on the statistical results, the rough set LSTM (RST-LSTM) model showed better prediction accuracy than fuzzy and original LSTM, as well as traditional neural networks, with the lowest error for training and testing datasets in different time horizons. The suggested model can support the optimization of the control approach and the smooth procedure of power system. The results confirm the superior capabilities of deep learning techniques for wind speed forecasting, which could also inspire new applications in meteorology assessment.

Modelling transitions in sealed surface cover fraction with Quantitative State Cellular Automata

Cellular Automata (CA) applications simulating urban processes generally employ discrete land-use classes to characterise the physical environment. Yet there is an increasing demand for urban land cover models simulating quantitative change at the sub-cell level. The proposed Quantitative State Cellular Automata model (QCA) addresses this issue by relaxing part of the CA definition and considering real-valued quantitative cell states reflecting a physically meaningful measure. QCA entails two components of change: transition potential and quantity of change. The potential component addresses the likelihood of any change occurring in a cell, whereas the quantity component estimates the magnitude of change. The QCA concept is illustrated for Sealed Surface Density (SSD) transitions in Brussels and part of Flanders (Belgium). A Mutual Information (MI) approach is used to define the neighbourhood interaction framework. The QCA model is respectively calibrated and validated using Landsat-derived 1987–2001 and 2001–2013 SSD change on 30 m resolution. The results show that QCA successfully emulates spatial patterns of urban development, and significantly outperforms a random model in terms of quantitative and spatial distribution of SSD change. Further improvements can be achieved by explicitly integrating socio-economic information in the proposed workflow.

Trivial and nontrivial error sources account for misidentification of protein partners in mutual information approaches

The problem of finding the correct set of partners for a given pair of interacting protein families based on multi-sequence alignments (MSAs) has received great attention over the years. Recently, the native contacts of two interacting proteins were shown to store the strongest mutual information (MI) signal to discriminate MSA concatenations with the largest fraction of correct pairings. Although that signal might be of practical relevance in the search for an effective heuristic to solve the problem, the number of MSA concatenations with near-native MI is large, imposing severe limitations. Here, a Genetic Algorithm that explores possible MSA concatenations according to a MI maximization criteria is shown to find degenerate solutions with two error sources, arising from mismatches among (i) similar and (ii) non-similar sequences. If mistakes made among similar sequences are disregarded, type-(i) solutions are found to resolve correct pairings at best true positive (TP) rates of 70%—far above the very same estimates in type-(ii) solutions. A machine learning classification algorithm helps to show further that differences between optimized solutions based on TP rates are not artificial and may have biological meaning associated with the three-dimensional distribution of the MI signal. Type-(i) solutions may therefore correspond to reliable results for predictive purposes, found here to be more likely obtained via MI maximization across protein systems having a minimum critical number of amino acid contacts on their interaction surfaces (N > 200).

Automatically Building Financial Sentiment Lexicons While Accounting for Negation

Financial investors make trades based on available information. Previous research has proved that microblogs are a useful source for supporting stock market decisions. However, the financial domain lacks specific sentiment lexicons that could be utilized to extract the sentiment from these microblogs. In this research, we investigate automatic approaches that can be used to build financial sentiment lexicons. We introduce weighted versions of the Pointwise Mutual Information approaches to build sentiment lexicons automatically. Furthermore, existing sentiment lexicons often neglect negation while building the sentiment lexicons. In this research, we also propose two methods (Negated Word and Flip Sentiment) to extend the sentiment building approaches to take into account negation when constructing a sentiment lexicon. We build the financial sentiment lexicons by leveraging 200,000 messages from StockTwits. We evaluate the constructed financial sentiment lexicons in two different sentiment classification tasks (unsupervised and supervised). In addition, the created financial sentiment lexicons are compared with each other and with other existing sentiment lexicons. The best performing financial sentiment lexicon is built by combining our Weighted Normalized Pointwise Mutual Information approach with the Negated Word approach. It outperforms all the other sentiment lexicons in the two sentiment classification tasks. In the unsupervised sentiment classification task, it has, on average, a balanced accuracy of 69.4%, and in the supervised setting, a balanced accuracy of 75.1%. Moreover, the various sentiment classification tasks confirm that the sentiment lexicons could be improved by taking into account negation while building the sentiment lexicons. The improvement could be made by using one of the proposed methods to incorporate negation in the sentiment lexicon construction process.

A Novel Mutual Information and Partial Least Squares Approach for Quality-Related and Quality-Unrelated Fault Detection

Partial least squares (PLS) and linear regression methods are widely utilized for quality-related fault detection in industrial processes. Standard PLS decomposes the process variables into principal and residual parts. However, as the principal part still contains many components unrelated to quality, if these components were not removed it could cause many false alarms. Besides, although these components do not affect product quality, they have a great impact on process safety and information about other faults. Removing and discarding these components will lead to a reduction in the detection rate of faults, unrelated to quality. To overcome the drawbacks of Standard PLS, a novel method, MI-PLS (mutual information PLS), is proposed in this paper. The proposed MI-PLS algorithm utilizes mutual information to divide the process variables into selected and residual components, and then uses singular value decomposition (SVD) to further decompose the selected part into quality-related and quality-unrelated components, subsequently constructing quality-related monitoring statistics. To ensure that there is no information loss and that the proposed MI-PLS can be used in quality-related and quality-unrelated fault detection, a principal component analysis (PCA) model is performed on the residual component to obtain its score matrix, which is combined with the quality-unrelated part to obtain the total quality-unrelated monitoring statistics. Finally, the proposed method is applied on a numerical example and Tennessee Eastman process. The proposed MI-PLS has a lower computational load and more robust performance compared with T-PLS and PCR.