Function Of Mutual Information Research Articles

Statistical reliability of a relative standard deviation of chromatographic peak area estimated by a chemometric tool based on the FUMI theory

In pharmaceutical analysis using a high-performance liquid chromatography (HPLC) system, repeatability assessment is significant to obtain reliable and precise quantitative results. The purpose of the present study is to experimentally show the statistical reliability of a relative standard deviation (RSD) of peak area estimated by a chemometric tool based on probability theory, called the function of mutual information (FUMI) theory, which stochastically provided an RSD of peak area and SD of baseline areas with width k (s(k)) from noises and a signal on a single chromatogram. An ultra-high-performance liquid chromatography with ultraviolet detection (UHPLC-UV) for determining ergosterol was applied as an example of the repeatability assessment. In addition, the statistical reliability of an RSD of peak area in the UHPLC-UV system was certified according to a chi-square (χ2) distribution. The 712 values of s(k) were experimentally obtained from a data series of 1001 points in the noise regions of 712 chromatograms. The histogram of χ2 of s(k) was well-fitted to the χ2 distribution curve (freedom degree, ν = 50), indicating that the statistical reliability of an RSD of the peak area in the UHPLC-UV estimated by the FUMI theory (n = 1) was equivalent to that estimated by 50 runs of chromatographic measurements.

Jaccard matrix for nonlinear filter statistics

We propose the Jaccard matrix (JM) and the Jaccard cell (JC), define them as the extended concepts of the Jaccard index, and theoretically and numerically analyze them. The data on the Euclidean plane can derive the JM as a sparse matrix. We show the JC inherits the feature of similarity of the Jaccard index as the exponential function of mutual information. We theoretically and numerically confirm that the local correlation coefficient of the data on the Euclidean plane relates the JC to the mutual information. Although one could potentially select an arbitrary cell size of the grid to make the JM, the knowledge we can obtain from the matrix decreases if the cell size is too big or too small to distinguish the data clusters appropriately. Therefore, the JM needs a computational procedure to determine the cell size within the appropriate scale. Maximizing the variance of the JCs supports determining the unique cell size, which value locates in the middle range of the parabolic function of the cell-size parameter. The JM could derive an index extracting nonlinear correlation of the data. The maximized standard deviation of the JCs as such an index is a decreasing function of the noise scale of the data under the constraint conditions. The ability to determine the homogeneous rectangular grid pattern of the JM might be a significant feature for finding nonlinear correlation. We would summarize this study as that of a nonlinear filter working as an efficient component of explainable AI and statistics.

Chemometric evaluations of repeatability and detection limit in high-performance liquid chromatography with electrochemical detection

The function of mutual information (FUMI) theory proposes that it is possible to obtain a relative standard deviation (RSD) of the peak area of an analyte from baseline noises and a signal on a single chromatogram when the analyte concentrations are proportional to their peak areas. In this study, we demonstrate that the FUMI theory using noise parametrization by the difference method is applicable for the evaluations of repeatability and detection limit (DL) in high-performance liquid chromatography with electrochemical detection (HPLC-ECD). HPLC-ECD for determining vincristine (VCR) was taken as an example, and VCR was detected on a glassy carbon surface at +0.8 V vs. Ag/AgCl. In the comparisons of precision profiles (plots of RSD of peak area against concentrations of an analyte), the stochastically estimated RSD (N = 1) by the FUMI theory fell within the 95% confidence intervals of statistically estimated RSD (N = 6) by repetitive measurements. Furthermore, the DL obtained by the FUMI theory (3.3×SD) was almost the same value as that by the signal-to-noise ratio (S/N) = 3. Moreover, we introduced a manual method for signal-noise resolution to obtain noise parameters from a chromatogram with low-pass Bessel filter processing. In conclusion, our results show that the FUMI theory using noise parametrization by the difference method can be applied to the evaluations of repeatability and DL in HPLC-ECD without and with a low-pass Bessel filter.

Ergotropy from quantum and classical correlations

It is an established fact that quantum coherences have thermodynamic value. The natural question arises, whether other genuine quantum properties such as entanglement can also be exploited to extract thermodynamic work. In the present analysis, we show that the ergotropy can be expressed as a function of the quantum mutual information, which demonstrates the contributions to the extractable work from classical and quantum correlations. More specifically, we analyze bipartite quantum systems with locally thermal states, such that the only contribution to the ergotropy originates in the correlations. Our findings are illustrated for a two-qubit system collectively coupled to a thermal bath.

A simple method for daily inspections of gas chromatography-mass spectrometry systems with an instrumental detection limit as an indicator

The purpose of this paper is to propose a simple method for daily inspections of a gas chromatography-mass spectrometry (GC–MS) system with an instrumental detection limit (IDL) as an indicator. A definition of DLs by ISO is 3.3σ where σ denotes the standard deviation (SD) of blank measurements. Estimation of σ is carried out according to the function of mutual information (FUMI) theory and actually with commercial software (TOCO19). An IDL which is a combination of a signal area, width and noise level is concluded to be a good indicator for daily inspections compared with each of its constituents. Methyl stearate is used as a standard material for the daily inspection of a programmed-temperature GC–MS system. A short chromatogram of 1800 data points (1.5 min) containing a target signal and background noise is fit for the IDL prediction by TOCO19. The relative SDs (RSDs) based on the theoretically estimated σ are shown to coincide with statistical results from repeated measurements within 95% confidence intervals. Column temperature is observed to affect IDLs through background fluctuation and then temperature-IDL relationship is examined in a range from 170 to 270 °C. Actual daily inspections over a month are demonstrated.

Optical MIMO communication with unequal power allocation to channels

Multiple input multiple output (MIMO) approach in fiber optical communication has emerged as an effective proposition to address the ever increasing demand for information exchange. In the ergodic case, the multiple channels, associated with multiple modes or cores or both in the optical fiber, is modeled by the Jacobi ensemble of random matrices. A key quantity for assessing the performance of MIMO systems is the mutual information (MI). We focus here on the case of an arbitrary transmission covariance matrix and derive exact determinant based results for the moment generating function (MGF) of mutual information (MI), and thereby address the scenario of unequal power per excited mode. The MGF is used to obtain Gaussian- and Weibull-distribution based approximations for the probability density function (PDF), cumulative distribution function (CDF) or, equivalently, the outage probability, and also the survival function (SF) or reliability function. Moreover, a numerical Fourier inversion approach is implemented to obtain the PDF, CDF, and SF directly from the MGF. The MGF is further used to investigate the ergodic capacity, which is the first moment (mean) of the mutual information. The analytical results are found to be in excellent agreement with Monte Carlo simulations. Our study goes beyond the earlier investigations where covariance matrix proportional to identity matrix has been considered which corresponds to equal power allocation per excited mode.

Chemometric evaluation of repeatability of internal standard methods in high-performance liquid chromatography with a Japanese pharmacopoeia assay for indomethacin as an example

The purpose of this study is to elucidate uncertainty structures of internal standard (IS) methods as compared with absolute calibration methods in liquid chromatography. A quantitative test of indomethacin with butyl 4-hydroxybenzoate as an IS in high-performance liquid chromatography with ultra-violet detection is taken here as an example. The repeatability is evaluated by both a usual statistical method of repetition and a theoretical approach, called the function of mutual information (FUMI) theory. The latter predicts the precision from noise and signals of instrumental output. Plots of relative standard deviations (RSDs) of measurements against analyte amounts, called precision profiles, are compared between the IS methods for indomethacin and their corresponding absolute calibration methods over a wide range of amount. Sample injection errors are observed to be effectively eliminated at high amounts by the IS methods, but at low amounts where background random noise dominates over the other error, the superiority of the IS methods is overshadowed and the precision of both the methods is almost comparable. The smallest possible amount of IS material without spoiling the integrity of analysis is estimated from the precision profiles.

Learning Invariant Representations using Inverse Contrastive Loss

Learning invariant representations is a critical first step in a number of machine learning tasks. A common approach is given by the so-called information bottleneck principle in which an application dependent function of mutual information is carefully chosen and optimized. Unfortunately, in practice, these functions are not suitable for optimization purposes since these losses are agnostic of the metric structure of the parameters of the model. In our paper, we introduce a class of losses for learning representations that are invariant to some extraneous variable of interest by inverting the class of contrastive losses, i.e., inverse contrastive loss (ICL). We show that if the extraneous variable is binary, then optimizing ICL is equivalent to optimizing a regularized MMD divergence. More generally, we also show that if we are provided a metric on the sample space, our formulation of ICL can be decomposed into a sum of convex functions of the given distance metric. Our experimental results indicate that models obtained by optimizing ICL achieve significantly better invariance to the extraneous variable for a fixed desired level of accuracy. In a variety of experimental settings, we show applicability of ICL for learning invariant representations for both continuous and discrete protected/extraneous variables. The project page with code is available at https://github.com/adityakumarakash/ICL

A Harmony Search-Based Wrapper-Filter Feature Selection Approach for Microstructural Image Classification

Besides chemical composition, microstructure plays a key role to control the properties of engineering materials. A strong correlation exists between microstructure and many mechanical and physical properties of a metal. It has the utmost importance to understand the microstructure and distinguish the microstructure accurately for the appropriate selection of engineering materials in product fabrication. Computer vision and machine learning play a major role to extract the feature and predict the most probable class of a 7-class microstructural image with a high degree of accuracy. Features contain information about the image, and the classification function is defined in terms of features. Feature selection plays an important role in the classification problem to improve the classification accuracy and also to reduce the computational time by eliminating redundant or non-influential features. The current research aims at classifying microstructure image datasets by an improved wrapper-filter based feature selection method using texture-based feature descriptor. Before applying the feature selection method, a feature descriptor, called rotational local tetra pattern (RLTrP), is applied to extract the features from the input images. Then, an ensemble of three filter methods is developed by considering the union of the top-n features selected by Chi-square, Fisher score, and Gini impurity-based filter methods. The objective of this ensemble is to combine all possible important features selected by three filter methods which will be used to create an initial population of the wrapper-based meta-heuristic feature selection algorithm called, harmony search (HS). The novelty of this HS method lies in the objective function, which is defined as a function of Pearson correlation coefficient and mutual information to calculate the fitness value. The proposed method not only optimizes features with reduced dimension but also improves the performance of classification accuracy of the 7-class microstructural images. Moreover, the proposed HS model has also been compared with some standard optimization algorithms like whale optimization algorithm, particle swarm optimization, and Grey wolf optimization on the present dataset, and in every case, the HS method ensures better agreement between feature selection and classification accuracy than the other methods.

Characterizing Mutual Information of Multistream MIMO-SVD Systems in Heterogeneous Random Networks

The main focus of this paper is on multistream multiantenna (multiple-input multiple-output) communications in large-scale heterogeneous random networks. Previous work in this area mainly studies the outage probability from the perspective of a given stream, which is inaccurate as it generally overlooks possible interference-driven cross stream signal-to-interference correlation in each communication link. To tackle this issue, in this paper, we focus on a link-level performance evaluation by characterizing the complementary cumulative distribution function (CCDF) of random mutual information (RMI) under heterogeneous interference. We focus on the scenarios that no coordination among transmitters is persuaded, and the receivers do not attempt to cancel out part of interuser interference. Assume that channel state information is known at both the transmitter and receiver of each communication pair, Rayleigh fading, that the singular value decomposition (SVD) is applied to construct pre-coding/post-coding filters for single-user detection, and that transmitted signals of antennas are i.i.d. Gaussian. We first provide accurate expression for the achievable data rate (ADR) of each link, and show that, regardless of the interference power, when transceivers are equipped with equal number of antennas it grows sublinearly with the number of antennas. We then approximate RMI via Gaussian distribution that its mean is the ADR and its variance is derived based on network's parameters. It is seen that the Gaussian approximation is credible even when two streams of data are transmitted. This paper provides various insights regarding the impact of system parameters, including density of transmitters, path-loss exponent, and the number of antennas, on the ADR and CCDF of RMI. We finally spot tremendous benefits of the SVD scheme over other prominent multiplexing systems, including maximum ratio combining, zero-forcing, and diagonal BLAST superstructure with minimum mean square error receivers.

On the Exact Distribution of Mutual Information of Two-User MIMO MAC Based on Quotient Distribution of Wishart Matrices

We propose an exact calculation of the probability density function (PDF) and cumulative distribution function (CDF) of mutual information (MI) for a two-user multiple-input multiple-output (MIMO) multiple access channel (MAC) network over block Rayleigh fading channels. This scenario can be found in the uplink channel of MIMO non-orthogonal multiple access (NOMA) system, a promising multiple access technique for 5G networks. So far, the PDF and CDF have been numerically evaluated since MI depends on the quotient of two Wishart matrices, and no closed form for this quotient was available. We derive exact results for the PDF and CDF of extreme (the smallest/the largest) eigenvalues. Based on the results of quotient ensemble, the exact calculation for PDF and CDF of mutual information is presented via Laplace transform approach and by direct integration of joint PDF of quotient ensemble’s eigenvalues. Furthermore, our derivations also provide the parameters to apply the Gaussian approximation method, which is comparatively easier to implement. We show that approximation matches the exact results remarkably well for outage probability, i.e., CDF, above 10%. However, the approximation could also be used for 1% outage probability with a relatively small error. We apply the derived expressions to investigate the effects of adding antennas in the receiver and its ability to decode the weak user signal. By supposing no channel knowledge at transmitters and successive decoding at receiver, the capacity of the weak user increases and its outage probability decreases with the increment of extra antennas at the receiver end.

Theoretical repeatability assessment without repetitive measurements in gradient high-performance liquid chromatography

This paper puts forward a time and material-saving method for evaluating the repeatability of area measurements in gradient HPLC with UV detection (HPLC-UV), based on the function of mutual information (FUMI) theory which can theoretically provide the measurement standard deviation (SD) and detection limits through the stochastic properties of baseline noise with no recourse to repetitive measurements of real samples. The chromatographic determination of terbinafine hydrochloride and enalapril maleate is taken as an example. The best choice of the number of noise data points, inevitable for the theoretical evaluation, is shown to be 512 data points (10.24s at 50 point/s sampling rate of an A/D converter). Coupled with the relative SD (RSD) of sample injection variability in the instrument used, the theoretical evaluation is proved to give identical values of area measurement RSDs to those estimated by the usual repetitive method (n=6) over a wide concentration range of the analytes within the 95% confidence intervals of the latter RSD. The FUMI theory is not a statistical one, but the “statistical” reliability of its SD estimates (n=1) is observed to be as high as that attained by thirty-one measurements of the same samples (n=31).

Scheduling of the Activations in Iterative Detection, Decoding, and Channel Estimation for MIMO-OFDM

An iterative receiver for a multiple-input-multiple-output (MIMO) orthogonal frequency-division multiplexing (OFDM) system is considered to jointly decode the transmitted bits and estimate the channel state. The receiver consists of the a posteriori probability (APP) algorithm, the repeat-accumulate (RA) decoder, and the least-squares (LS) channel estimator. An obvious problem, with more than two blocks in an iterative receiver, is to find the optimal activation schedule of the different blocks. This paper proposes to use extrinsic information transfer (EXIT) charts to characterize the behavior of the receiver blocks and find out the optimal activation schedule for them. A semi-analytical expression of the EXIT function is derived for the decision directed LS channel estimator. An algorithm is proposed to generate the EXIT function of the APP algorithm as a function of channel estimate mutual information (MI). Surface fitting is used to get closed form expressions for the EXIT functions of the APP algorithm and the RA decoder. Trellis search based algorithms are shown to find the convergence with the lowest possible complexity using the EXIT charts. With the proposed concept, the activation scheduling can be adapted to prevailing channel circumstances and unnecessary iterations will be avoided.

On the Distribution of MIMO Mutual Information: An In-Depth Painlevé-Based Characterization

This paper builds upon our recent work which computed the moment generating function of the MIMO mutual information exactly in terms of a Painlev\'{e} V differential equation. By exploiting this key analytical tool, we provide an in-depth characterization of the mutual information distribution for sufficiently large (but finite) antenna numbers. In particular, we derive systematic closed-form expansions for the high order cumulants. These results yield considerable new insight, such as providing a technical explanation as to why the well known Gaussian approximation is quite robust to large SNR for the case of unequal antenna arrays, whilst it deviates strongly for equal antenna arrays. In addition, by drawing upon our high order cumulant expansions, we employ the Edgeworth expansion technique to propose a refined Gaussian approximation which is shown to give a very accurate closed-form characterization of the mutual information distribution, both around the mean and for moderate deviations into the tails (where the Gaussian approximation fails remarkably). For stronger deviations where the Edgeworth expansion becomes unwieldy, we employ the saddle point method and asymptotic integration tools to establish new analytical characterizations which are shown to be very simple and accurate. Based on these results we also recover key well established properties of the tail distribution, including the diversity-multiplexing-tradeoff.

Development of HPLC with High-sensitive and Precise Electrochemical Detection Enabling Dynamic Analysis of Compounds of Biological Importance

HPLC with electrochemical detection (HPLC-ECD) is an attractive method with sensitivity and selectivity for the determination of redox compounds. However, an improper system makeup or operation is apt not to show such the intrinsic characteristics of the analytical results by HPLC-ECD in regards to biological sample assays. In this review, HPLC-ECD enabling high-sensitive and precise analysis of compounds of biological importance was developed using the following chemometric strategies: spectrum analysis of chromatographic baseline noise, standard deviation (S.D.) of area measurements in baseline noise from stochastic aspects, and optimization of HPLC conditions and method validations in HPLC-ECD using the prediction of precision based on the FUMI (Function of Mutual Information) theory. When HPLC-ECD was established using a capillary column (0.2 mm i.d.), catechins were determined at attomole levels and the present HPLC-ECD was applied to the determination of concentration profiles of catechins in human plasma after green tea ingestion. Moreover, two HPLC-ECD systems for determining acids and bases were developed by the means of the voltammetric reduction of quinone and the oxidation of trolox, respectively. Thus, the application of HPLC-ECD methods has been remarkably expanded through the development of novel ECD for the determination of acids and bases which are less active electrochemically. The present methods for determining acids and bases were applied to the pharmacokinetic studies of free fatty acids and theophylline, respectively. In conclusion, it was shown the present HPLC-ECD methods have been successfully applied to biomedical and pharmaceutical analyses.

Calculating Outage Probability of Block Fading Channels Based on Moment Generating Functions

Block fading channels present a realistic model in many communication scenarios. We propose a method by which the outage probability of block fading channels can be evaluated in a straighforward and computationally efficient manner by utilizing the moment generating function of a single block's mutual information. The flexibility of moment generating functions helps obtain outage probabilities in various cases such as wideband noise jamming and multi-access interference. The effect of discrete input alphabets on outage is also investigated and turns out to be vital especially under interference.

Picomole Level Determination of Cholesterol by HPLC with Electrochemical Detection Using Boron‐doped Diamond Electrode after Performance Assessment Based on the FUMI Theory

AbstractFor the performance assessment of an electrochemical flow cell using a boron‐doped diamond (BDD) electrode in high‐performance liquid chromatography with electrochemical detection (HPLC‐ECD) to determine cholesterol, the standard deviation (SD) of area measurements in chromatographic baseline noise (AMCBN) was evaluated based on the function of mutual information (FUMI) theory without repetitive chromatographic measurements. After the selection of suitable components for the electrochemical flow cell in HPLC‐ECD, the detection limit of cholesterol was 0.04 pmol at a detection potential of +2.2 V vs. an Ag/AgCl electrode. The present HPLC‐ECD was applied to determine cholesterol in various meats.

A simple information-theoretic analysis of clustered sensor networks with decentralized detection

In this letter, we present a simple information-theoretic framework to analyze clustered sensor networks with hierarchical multi-level majority-like fusion and decentralized detection. The sensor nodes observe a binary phenomenon and transmit their own data to an access point (AP), possibly through intermediate fusion centers (FCs). We investigate the impact of uniform and non-uniform clustering on the system performance, evaluated in terms of mutual information between the true phenomenon status and its estimate at the AP. Being the overall system binary-input binary-output (BIBO), it will be shown that the probability of decision error (Pe) is a specific function of the input-output mutual information (I). In other words, the network operational point lies over a specific Pe - I curve and depends on the network characteristics (e.g., topology, observation and communication noise levels, etc.).

Capacity of MIMO Rayleigh Fading Channels in the Presence of Interference and Receive Correlation

In cellular systems, multiple users share the same radio spectrum, which leads to cochannel interference. Cochannel interference ultimately limits the quality of service offered to the users. At the moment, general capacity analysis for multiple-input-multiple-output (MIMO) correlated Rayleigh fading channels in the presence of correlated Rayleigh cochannel interference is not available in the literature, although its counterpart without cochannel interference has been thoroughly investigated. The difficulty arises from the quadratic form of the interference term. In this paper, based on the theory of matrix variate distribution, we investigate the capacity of MIMO Rayleigh fading channels with Rayleigh cochannel interference. We assume that perfect channel information is available to the receiver, but the transmitter has no channel information. We focus on the case when signals for the desired user and cochannel interferers are correlated at receiving antennas. We first derive an exact closed-form expression of the moment-generating function (MGF) of mutual information of MIMO channels with interference. Then, we further obtain an exact expression of the MIMO ergodic capacity with the MGF formula of mutual information. As a double check, these analytical results are validated by Monte Carlo simulations, and as an illustration of the mathematical formalism, numerical examples for particular cases of interest are plotted and discussed.

Selection of the Optimal Solvent Grade for the Mobile Phase in HPLC with Electrochemical Detection Based on FIMI Theory

The optimum conditions of the mobile phase for HPLC with electrochemical detection (HPLC-ECD) were selected from among solvents of different grades with the standard deviation (SD) of area measurements based on FUMI (function of mutual information) theory as a criterion. In HPLC-ECD for determining baicalin and baicalein using a phosphoric acid-methanol-water mixture (0.5:60:40, v/v/v) as a mobile phase, the effects of three grades of phosphoric acid, of five grades of methanol, and of seven grades of water on the measurement precision were examined. The mixture, if characterized by the minimum SD of measurements, was selected as the optimal mobile phase. This selection method saves considerable amounts of chemicals and experimental time, and would be a useful exploration technique for the routine check and troubleshooting of HPLC-ECD.