Minimum Mutual Information Research Articles

Markov Decision Process Based Design of SWIPT Systems: Non-Linear EH Circuits, Memory, and Impedance Mismatch

In this paper, we study simultaneous wireless information and power transfer (SWIPT) systems employing practical non-linear energy harvester (EH) circuits. Since the voltage across the reactive elements of realistic EH circuits cannot drop or rise instantaneously, EHs have memory which we model with a Markov decision process (MDP). Moreover, since an analytical model that accurately models all non-linear effects and the unavoidable impedance mismatch of EHs is not tractable, we propose a learning based model for the EH circuit. We optimize the input signal distribution for maximization of the harvested power under a constraint on the minimum mutual information between transmitter (TX) and information receiver (IR). We distinguish the cases where the MDP state is known and not known at TX and IR. When the MDP state is known, the formulated optimization problem for the harvested power is convex. In contrast, if TX and IR do not know the MDP state, the resulting optimization problem is non-convex and solved via alternating optimization, which is shown to yield a limit point of the problem. Our simulation results reveal that the rate-power region of the considered SWIPT system depends on the symbol duration, the EH input power level, the EH impedance mismatch, and the type of EH circuit. In particular, a shorter symbol duration enables higher bit rates at the expense of a significant decrease in the average harvested power. Furthermore, whereas half-wave rectifiers outperform full-wave rectifiers in the low and medium input power regimes, full-wave rectifiers are preferable if the input power at the EH is high.

Privacy-Aware Time-Series Data Sharing With Deep Reinforcement Learning

Internet of things (IoT) devices are becoming increasingly popular thanks to many new services and applications they offer. However, in addition to their many benefits, they raise privacy concerns since they share fine-grained time-series user data with untrusted third parties. In this work, we study the privacy-utility trade-off (PUT) in time-series data sharing. Existing approaches to PUT mainly focus on a single data point; however, temporal correlations in time-series data introduce new challenges. Methods that preserve the privacy for the current time may leak significant amount of information at the trace level as the adversary can exploit temporal correlations in a trace. We consider sharing the distorted version of a user's true data sequence with an untrusted third party. We measure the privacy leakage by the mutual information between the user's true data sequence and shared version. We consider both the instantaneous and average distortion between the two sequences, under a given distortion measure, as the utility loss metric. To tackle the history-dependent mutual information minimization, we reformulate the problem as a Markov decision process (MDP), and solve it using asynchronous actor-critic deep reinforcement learning (RL). We evaluate the performance of the proposed solution in location trace privacy on both synthetic and GeoLife GPS trajectory datasets. For the latter, we show the validity of our solution by testing the privacy of the released location trajectory against an adversary network.

BER Performance Improvement of Alamouti MIMO-STBC Decoder Using Mutual Information Method

Bit error rate (BER) performance of multiple input multiple output (MIMO) space time block code (STBC) decoder depends on the channel estimator quality, where the quality of classical least square (LS) estimator depends mainly on the number of pilot symbols. The aims of this paper are to implement a new STBC decoder, measure the BER performance and make comparison with conventional one, without increasing the number of pilot symbols. Although the new strategy is more complex than a conventional decoder, it gives the opportunity to join estimation and decoding at the same time. Minimum mutual information (MMI)- independent component analysis (ICA) algorithm has been used for extracting source signals. MIMO-STBC systems were taken as a cases study with two transmitters. The simulation results show that MMI-ICA algorithm provides superior BER performance as compared with conventional maximum ratio combiner (MRC). The important point, which denoted during simulation, the performance of ICA algorithm with high frame length provides better BER performance than the conventional one.

Spatial-spectral hyperspectral image classification based on information measurement and CNN

In order to construct virtual land environment for virtual test, we propose a construction method of virtual land environment using multi-satellite remote sensing data, the key step of which is accurate recognition of ground object. In this paper, a method of ground object recognition based on hyperspectral image (HSI) was proposed, i.e., a HSI classification method based on information measure and convolutional neural networks (CNN) combined with spatial-spectral information. Firstly, the most important three spectra of the hyperspectral image was selected based on information measure. Specifically, the entropy and color-matching functions were applied to determine the candidate spectra sets from all the spectra of the hyperspectral image. Then three spectra with the largest amount of information were selected through the minimum mutual information. Through the above two steps, the dimensionality reduction for hyperspectral images was effectively achieved. Based on the three selected spectra, the CNN network input combined with the spatial-spectral information was designed. Two input strategies were designed: (1) The patch surrounding the pixel to be classified was directly intercepted from the grayscale images of the three selected spectra. (2) In order to highlight the effect of the spectrum of the pixel to be classified, all the spectral components of this pixel were superimposed on the patch obtained by the previous strategy. As a result, a new patch with more prominent spectral components of the pixel to be classified was obtained. Using the two public hyperspectral datasets, Salinas and Pavia Center, the experiments of on both parameter selection and classification performance were performed to verify that the proposed methods had better classification performance.

Correlation measures and distillable entanglement in AdS/CFT

Recent developments have exposed close connections between quantum information and holography. In this paper, we explore the geometrical interpretations of the recently introduced $Q$-correlation and $R$-correlation, $E_Q$ and $E_R$. We find that $E_Q$ admits a natural geometric interpretation via the surface-state correspondence: it is a minimal mutual information between a surface region $A$ and a cross-section of $A$'s entanglement wedge with $B$. We note a strict trade-off between this minimal mutual information and the symmetric side-channel assisted distillable entanglement from the environment $E$ to $A$, $I^{ss}(E\rangle A)$. We also show that the $R$-correlation, $E_R$, coincides holographically with the entanglement wedge cross-section. This further elucidates the intricate relationship between entanglement, correlations, and geometry in holographic field theories.

Optimized Antenna Selection and Adaptive Power Allocation for Physical Layer Network Coding with Selective Soft-Message-Forward Cooperation

To improve the systematic bit error rate (BER) performance and power efficiency, an optimized antenna selection and adaptive power allocation scheme is proposed for physical layer network coding (PNC) with selective soft-message-forward (SSMF) cooperation. In this scheme, two adaptive power allocation (APA) schemes are proposed for the multiple access (MA) and broadcast (BC) stages, respectively. In the MA stage, the APA scheme aims to maximize the Euclidean distance of network coding symbols received by relay node. In the second BC stage, the relay node takes the maximization of the minimum mutual information of two source nodes as objective function. Then it uses the narrow range method (NRM) to select the optimal antenna set and equally allocates power to them. To further reduce the systematic complexity, an antenna set pre-selection is proposed. It minimizes the antenna number in the initial antenna set of the NRM based on the deletion standard which does not reduce the mutual information in the BC stage. Simulation results indicate that our APA schemes surpass the existing equal power allocation (EPA) scheme with a binary phase-shift keying modulation. Compared with current EPA scheme, the proposed APA schemes for the BC, MA and global stages obtain about 1 dB, 2 dB and 3.5 dB performance gains at BER of 10−3 with two antennas in the relay node, respectively.

Novel data-driven short-frequency mutual information entropy threshold filtering and its application to bearing fault diagnosis

A rotating mechanical vibration signal is time-varying and non-stationary because of the effect of defects on the rotating components and working conditions. The commonly used short-time Fourier transform (STFT) divides the time-domain signal into several segments by a sliding window. However, the STFT cannot optimize the time and frequency resolution. Considerable prior knowledge and much expertise are needed to further analyze these complex vibration signals. Therefore, a novel processing method comprising a fault vibration signal and named short-frequency mutual information entropy threshold filtering is proposed to filter and reconstruct the non-stationary vibration signal for diagnosing the impulse fault features. In this method, the improved STFT with an adaptive optimal window length is used to transform the vibration signal into the time-frequency domain. Then, a Meyer wavelet filter bank is used to divide the frequency spectrum into several sub-bands in each moment. The minimum mutual information entropy between the sub-bands and the original frequency is used to remove the partial sub-bands and filter the useful features from each frequency spectrum. Then, the frequency spectra are rearranged and an inverse STFT is used to recover the original signal. The envelope spectrum method is used to extract the fault features from the recovered signal. This method is completely determined by the characteristics of the vibration signal data. Finally, the fault vibration signal from our testing rig and wind turbine gearbox are used to verify the performance of the proposed method. The results show that this method can efficiently detect impact fault features in the vibration signal.

Assessing Synergy/Redundancy of Baroreflex and Non-Baroreflex Components of the Cardiac Control during Sleep.

Cardiovascular regulation and autonomic function change across sleep stages and compared to wake. Little information is present in literature about cardiac control during sleep especially in relation to new information-theoretic quantities such as synergy and redundancy. In the present work we compute synergy and redundancy of baroreflex and non-baroreflex components of the cardiac control according to two information-theoretic approaches, namely predictive information decomposition (PID) and minimal mutual information (MMI) methods. We applied a bivariate approach to heart period (HP) and systolic arterial pressure (SAP) beat-to-beat variability series during sleep in a healthy subject. PID approach computes the net balance between synergy and redundancy, while MMI calculates the two quantities as separate entities. Results suggested that: i) redundancy was dominant over synergy during NREM phases; ii) redundancy increased during NREM phase; iii) synergy did not change across the sleep stages. We interpret this result as a consequence of the vagal enhancement, slowing and deepening of respiration during NREM phases. These preliminary findings support the potential of assessing redundancy/synergy of baroreflex-related and unrelated regulations during sleep to improve our knowledge about physiological mechanisms.

Learning Domain-Independent Deep Representations by Mutual Information Minimization.

Domain transfer learning aims to learn common data representations from a source domain and a target domain so that the source domain data can help the classification of the target domain. Conventional transfer representation learning imposes the distributions of source and target domain representations to be similar, which heavily relies on the characterization of the distributions of domains and the distribution matching criteria. In this paper, we proposed a novel framework for domain transfer representation learning. Our motive is to make the learned representations of data points independent from the domains which they belong to. In other words, from an optimal cross-domain representation of a data point, it is difficult to tell which domain it is from. In this way, the learned representations can be generalized to different domains. To measure the dependency between the representations and the corresponding domain which the data points belong to, we propose to use the mutual information between the representations and the domain-belonging indicators. By minimizing such mutual information, we learn the representations which are independent from domains. We build a classwise deep convolutional network model as a representation model and maximize the margin of each data point of the corresponding class, which is defined over the intraclass and interclass neighborhood. To learn the parameters of the model, we construct a unified minimization problem where the margins are maximized while the representation-domain mutual information is minimized. In this way, we learn representations which are not only discriminate but also independent from domains. An iterative algorithm based on the Adam optimization method is proposed to solve the minimization to learn the classwise deep model parameters and the cross-domain representations simultaneously. Extensive experiments over benchmark datasets show its effectiveness and advantage over existing domain transfer learning methods.

Multi-scale fusion algorithm of intensity and polarization-difference images based on edge information enhancement

To better integrate complementary and redundant information from different source images, improve the edge information, and facilitate the target detection. A multi-scale fusion algorithm of intensity and polarization-difference (PD) images based on edge information enhancement is proposed. Firstly, intensity images are obtained by the polarization information analysis method. PD images are obtained by the adaptive polarization-difference imaging approach based on the principle of minimum mutual information. Secondly, guided filter, affine transformations and Block-Matching and 3D filtering are embedded in visibility enhancement to improve the intensity and PD images. Thirdly, the two images are decomposed into high-frequency and low-frequency images by the dual-tree complex wavelet transform (DT-CWT). The high-frequency and low-frequency images are fused by the fusion rules based on edge detection and the regional variance matching degree respectively. Finally, the fusion image is obtained by the inverse DT-CWT. Experimental results demonstrate that fusion images of the proposed algorithm are significantly improved in information entropy, average gradient, and spatial frequency. Compared with the existing methods, it can achieve a better edge enhancement for images in a turbid medium.

Rényi Resolvability and Its Applications to the Wiretap Channel

The conventional channel resolvability problem refers to the determination of the minimum rate required for an input process so that the output distribution approximates a target distribution in either the total variation distance or the relative entropy. In contrast to previous works, in this paper, we use the (normalized or unnormalized) Rényi divergence (with the Rényi parameter in[0, 2] U {∞}) to measure the level of approximation. We also provide asymptotic expressions for normalized Rényi divergence when the Rényi parameter is larger than or equal to 1 as well as (lower and upper) bounds for the case when the same parameter is smaller than 1. We characterize the Rényi resolvability, which is defined as the minimum rate required to ensure that the Rényi divergence vanishes asymptotically. The Rényi resolvabilities are the same for both the normalized and unnormalized divergence cases. In addition, when the Rényi parameter smaller than 1, consistent with the traditional case where the Rényi parameter is equal to 1, the Rényi resolvability equals the minimum mutual information over all input distributions that induce the target output distribution. When the Rényi parameter is larger than 1 the Rényi resolvability is, in general, larger than the mutual information. The optimal Rényi divergence is proven to vanish at least exponentially fast for both of these two cases, as long as the code rate is larger than the Rényi resolvability. The optimal exponential rate of decay for i.i.d. random codes is also characterized exactly. We apply these results to the wiretap channel, and completely characterize the optimal tradeoff between the rates of the secret and non-secret messages when the leakage measure is given by the (unnormalized) Rényi divergence. This tradeoff differs from the conventional setting when the leakage is measured by the traditional mutual information.

Performance comparison of MeRMaId-ICA and Np-ICA in composite abdominal electrocardiogram separation

Blind source separation strives to disintegrate a multivariate composite non-invasive abdominal electrocardiogram signal into independent non-Gaussian signals such as maternal and foetal electrocardiograms. This paper proffers two separation algorithms especially for foetal electrocardiogram (FECG) extraction as it has a great role in diagnosing the foetal heart deformities namely minimum Renyi's mutual information called MeRMaId algorithm and non-parametric independent component analysis (Np-ICA) algorithm. Both the algorithms are experimentally evaluated on synthetic and real abdominal data. Performance juxtaposition of these two algorithms is done by scrutinising the signal to noise ratio at assorted noise levels and signal to interference ratio at assorted amplitude levels, and computing correlation coefficient (ρ) of the original and the estimated maternal and foetal electrocardiograms.

Paced Breathing Increases the Redundancy of Cardiorespiratory Control in Healthy Individuals and Chronic Heart Failure Patients.

Synergy and redundancy are concepts that suggest, respectively, adaptability and fault tolerance of systems with complex behavior. This study computes redundancy/synergy in bivariate systems formed by a target X and a driver Y according to the predictive information decomposition approach and partial information decomposition framework based on the minimal mutual information principle. The two approaches assess the redundancy/synergy of past of X and Y in reducing the uncertainty of the current state of X. The methods were applied to evaluate the interactions between heart and respiration in healthy young subjects (n = 19) during controlled breathing at 10, 15 and 20 breaths/minute and in two groups of chronic heart failure patients during paced respiration at 6 (n = 9) and 15 (n = 20) breaths/minutes from spontaneous beat-to-beat fluctuations of heart period and respiratory signal. Both methods suggested that slowing respiratory rate below the spontaneous frequency increases redundancy of cardiorespiratory control in both healthy and pathological groups, thus possibly improving fault tolerance of the cardiorespiratory control. The two methods provide markers complementary to respiratory sinus arrhythmia and the strength of the linear coupling between heart period variability and respiration in describing the physiology of the cardiorespiratory reflex suitable to be exploited in various pathophysiological settings.

A Novel Feature Selection Scheme and a Diversified-Input SVM-Based Classifier for Sensor Fault Classification

The efficiency of a binary support vector machine- (SVM-) based classifier depends on the combination and the number of input features extracted from raw signals. Sometimes, a combination of individual good features does not perform well in discriminating a class due to a high level of relevance to a second class also. Moreover, an increase in the dimensions of an input vector also degrades the performance of a classifier in most cases. To get efficient results, it is needed to input a combination of the lowest possible number of discriminating features to a classifier. In this paper, we propose a framework to improve the performance of an SVM-based classifier for sensor fault classification in two ways: firstly, by selecting the best combination of features for a target class from a feature pool and, secondly, by minimizing the dimensionality of input vectors. To obtain the best combination of features, we propose a novel feature selection algorithm that selects m out of M features having the maximum mutual information (or relevance) with a target class and the minimum mutual information with nontarget classes. This technique ensures to select the features sensitive to the target class exclusively. Furthermore, we propose a diversified-input SVM (DI-SVM) model for multiclass classification problems to achieve our second objective which is to reduce the dimensions of the input vector. In this model, the number of SVM-based classifiers is the same as the number of classes in the dataset. However, each classifier is fed with a unique combination of features selected by a feature selection scheme for a target class. The efficiency of the proposed feature selection algorithm is shown by comparing the results obtained from experiments performed with and without feature selection. Furthermore, the experimental results in terms of accuracy, receiver operating characteristics (ROC), and the area under the ROC curve (AUC-ROC) show that the proposed DI-SVM model outperforms the conventional model of SVM, the neural network, and the k-nearest neighbor algorithm for sensor fault detection and classification.

Information‐Theoretic Approach to Bidirectional Scaling

AbstractWe present an information‐theoretic approach for integration of multiresolution data into multiscale simulations. This general framework is used to upscale and downscale equations of fluid flow in heterogeneous porous media. Fine‐scale information can comprise observational data and/or simulation results related to both system states and system parameters. It is aggregated into its coarse‐scale representation by setting a probabilistic equivalence between the two scales, with parameters that are determined via minimization of observable error and mutual information across scales. The same quantities facilitate the use of coarse‐scale data to constrain compatible fine‐scale distributions.

The Role of Entropy in Estimating Financial Network Default Impact.

Agents in financial networks can simultaneously be both creditors and debtors, creating the possibility that a default may cause a subsequent default cascade. Resolution of unpayable debts in these situations will have a distributional impact. Using a relative entropy-based measure of the distributional impact of the subsequent default resolution process, it is argued that minimum mutual information estimation of unknown cells in the matrix of funds originally owed by the network participants to each other does not introduce systematic biases when estimating that impact.

Exact Partial Information Decompositions for Gaussian Systems Based on Dependency Constraints.

The Partial Information Decomposition, introduced by Williams P. L. et al. (2010), provides a theoretical framework to characterize and quantify the structure of multivariate information sharing. A new method () has recently been proposed by James R. G. et al. (2017) for computing a two-predictor partial information decomposition over discrete spaces. A lattice of maximum entropy probability models is constructed based on marginal dependency constraints, and the unique information that a particular predictor has about the target is defined as the minimum increase in joint predictor-target mutual information when that particular predictor-target marginal dependency is constrained. Here, we apply the approach to Gaussian systems, for which the marginally constrained maximum entropy models are Gaussian graphical models. Closed form solutions for the PID are derived for both univariate and multivariate Gaussian systems. Numerical and graphical illustrations are provided, together with practical and theoretical comparisons of the PID with the minimum mutual information partial information decomposition (), which was discussed by Barrett A. B. (2015). The results obtained using appear to be more intuitive than those given with other methods, such as , in which the redundant and unique information components are constrained to depend only on the predictor-target marginal distributions. In particular, it is proved that the method generally produces larger estimates of redundancy and synergy than does the method. In discussion of the practical examples, the PIDs are complemented by the use of tests of deviance for the comparison of Gaussian graphical models.

One-Class Classifiers Based on Entropic Spanning Graphs.

One-class classifiers offer valuable tools to assess the presence of outliers in data. In this paper, we propose a design methodology for one-class classifiers based on entropic spanning graphs. Our approach also takes into account the possibility to process nonnumeric data by means of an embedding procedure. The spanning graph is learned on the embedded input data, and the outcoming partition of vertices defines the classifier. The final partition is derived by exploiting a criterion based on mutual information minimization. Here, we compute the mutual information by using a convenient formulation provided in terms of the -Jensen difference. Once training is completed, in order to associate a confidence level with the classifier decision, a graph-based fuzzy model is constructed. The fuzzification process is based only on topological information of the vertices of the entropic spanning graph. As such, the proposed one-class classifier is suitable also for data characterized by complex geometric structures. We provide experiments on well-known benchmarks containing both feature vectors and labeled graphs. In addition, we apply the method to the protein solubility recognition problem by considering several representations for the input samples. Experimental results demonstrate the effectiveness and versatility of the proposed method with respect to other state-of-the-art approaches.

Electroencephalogram-Artifact Extraction Enhancement Based on Artificial Intelligence Technique

Blind source separation (BSS) is an important technique used to recover isolated independent sources signals from mixtures. This paper proposes two blind artificial intelligent separation algorithms based on hybridization between artificial intelligent techniques with classical blind source separation algorithms to enhance the separation process. Speedy genetic algorithm SGA directly guesses the optimal coefficients of the separating matrix based on candidate initial from classical BSS algorithms also the separation criteria based on minimization of mutual information between the separating independent components. The proposed algorithms are tested by real Electroencephalogram (EEG) data, the experimental results indicate that the algorithms can quickly and effectively get optimum solution to linear blind source separation compared to classical BSS techniques, the proposed works are described by high accuracy and robustness.

Power allocation for uplink SC‐FDMA systems with arbitrary input distribution

Existing power allocation schemes for single-carrier frequency division multiple access (SC-FDMA) systems are based on the assumption of Gaussian inputs. However, in practical systems, the inputs belong to a set of finite symbol alphabets. In this reported work the power allocation problem in the uplink SC-FDMA system with arbitrarily distributed finite power inputs is considered. An optimal solution to the problem is found by using the relationship between minimum mean-square error and mutual information. Simulation results show that solution allocates power in accordance to the modulation scheme employed and achieves significant power saving compared to schemes that assume Gaussian inputs.