Independence Criterion Research Articles

Predicting autism spectrum disorder from associative genetic markers of phenotypic groups using machine learning

Machine learning is a discipline of artificial intelligence, geared towards the development of various critical applications. Due to its high precision, it is widely adopted in the process of extracting useful hidden patterns and valuable insights from complex data structures. Data extracted from the real-time environment might contain some irrelevant information. The presence of noise in the data degrades the model performance. Gene expression is an important source, carries the genetic information of species. Gene expression pattern reveals the significant relationship between genes associated with several diseases. But due to irregular molecular interactions and reactions occurs during the transcription process, the gene expressions are minimally affected. It causes a detrimental effect on the identification of biological markers of the diseases. To address this problem, a novel gene selection strategy is proposed to identify the candidate gene biomarkers from the genomic data. Signal to Noise ratio with logistic sigmoid function, Hilbert–Schmidt Independence Criterion Lasso, and regularized genetic algorithm amalgamation finds the optimal features. The proposed system is tested with the microarray gene expression dataset of autism spectrum disorder (ASD), accessed from gene expression omnibus repository. FAM104B, CCNDBP1, H1F0, ZER1 are identified as the candidate biomarkers of ASD. The methodical performance evaluation of the proposed model is examined with widely used machine learning algorithms. The proposed methodology enhanced the prediction rate of ASD and attained an accuracy of 97.62%, outperformed existing methods. Also, this system could act as a significant tool to assist the medical practitioners for accurate ASD diagnosis.

New HSIC-based tests for independence between two stationary multivariate time series

This paper proposes some novel one-sided omnibus tests for independence between two multivariate stationary time series. These new tests apply the Hilbert-Schmidt independence criterion (HSIC) to test the independence between the innovations of both time series. Under regular conditions, the limiting null distributions of our HSIC-based tests are established. Next, our HSIC-based tests are shown to be consistent. Moreover, a residual bootstrap method is used to obtain the critical values for our HSIC-based tests, and its validity is justified. Compared with the existing cross-correlation-based tests for linear dependence, our tests examine the general (including both linear and non-linear) dependence to give investigators more complete information on the causal relationship between two multivariate time series. The merits of our tests are illustrated by some simulation results and a real example.

Data mining techniques for analyzing healthcare conditions of urban space-person lung using meta-heuristic optimized neural networks

Urban computing is one of the effective fields that have ability to collect the large volume of data, integrate and analyze the data in urban space. The urban space faces several issues such as traffic congestion, more energy consumption, air pollution and so on. Among the several problems, air pollution is one of the major issues because it creates several health issues. So, this paper introduces the meta-heuristic optimized neural network to analyze patient health to predict different diseases. Initially, patient data are collected, normalized by applying a min–max normalization process. Then different features are extracted and Hilbert–Schmidt Independence Criterion based features are selected. Further patient's health condition is analyzed and classified into a normal and abnormal person. The classification process is done by applying the harmony optimized modular neural network. Here the system efficiency is evaluated using simulation results, which ensures maximum accuracy of 98.9% -ELT-COPD and 98% -NIH clinical dataset.

Improved metabolomic data-based prediction of depressive symptoms using nonlinear machine learning with feature selection

To solve major limitations in algorithms for the metabolite-based prediction of psychiatric phenotypes, a novel prediction model for depressive symptoms based on nonlinear feature selection machine learning, the Hilbert–Schmidt independence criterion least absolute shrinkage and selection operator (HSIC Lasso) algorithm, was developed and applied to a metabolomic dataset with the largest sample size to date. In total, 897 population-based subjects were recruited from the communities affected by the Great East Japan Earthquake; 306 metabolite features (37 metabolites identified by nuclear magnetic resonance measurements and 269 characterized metabolites based on the intensities from mass spectrometry) were utilized to build prediction models for depressive symptoms as evaluated by the Center for Epidemiologic Studies-Depression Scale (CES-D). The nested fivefold cross-validation was used for developing and evaluating the prediction models. The HSIC Lasso-based prediction model showed better predictive power than the other prediction models, including Lasso, support vector machine, partial least squares, random forest, and neural network. l-leucine, 3-hydroxyisobutyrate, and gamma-linolenyl carnitine frequently contributed to the prediction. We have demonstrated that the HSIC Lasso-based prediction model integrating nonlinear feature selection showed improved predictive power for depressive symptoms based on metabolome data as well as on risk metabolites based on nonlinear statistics in the Japanese population. Further studies should use HSIC Lasso-based prediction models with different ethnicities to investigate the generality of each risk metabolite for predicting depressive symptoms.

Linear independence of values of G-functions, II: outside the disk of convergence

Given any non-polynomial G-function $$F(z)=\sum _{k=0}^\infty A_kz^k$$ of radius of convergence R and in the kernel a G-operator $$L_F$$ , we consider the G-functions $$F_n^{[s]}(z)=\sum _{k=0}^\infty \frac{A_k}{(k+n)^s}z^k$$ for every integers $$s\ge 0$$ and $$n\ge 1$$ . These functions can be analytically continued to a domain $${\mathcal {D}}_F$$ star-shaped at 0 and containing the disk $$\{\vert z\vert <R\}$$ . Fix any $$\alpha \in {\mathcal {D}}_F \cap \overline{{\mathbb {Q}}}^*$$ , not a singularity of $$L_F$$ , and any number field $${\mathbb {K}}$$ containing $$\alpha $$ and the $$A_k$$ ’s. Let $$\Phi _{\alpha , S}$$ be the $${\mathbb {K}}$$ -vector space spanned by the values $$F_n^{[s]}(\alpha )$$ , $$n\ge 1$$ and $$0\le s\le S$$ . We prove that $$u_{{\mathbb {K}},F}\log (S)\le \dim _{\mathbb {K}}(\Phi _{\alpha , S })\le v_FS$$ for any S, for some constants $$u_{{\mathbb {K}},F}>0$$ and $$v_F>0$$ . This appears to be the first general Diophantine result for values of G-functions evaluated outside their disk of convergence. This theorem encompasses a previous result of the authors in [Linear independence of values of G-functions, J. Europ. Math. Soc. 22(5), 1531–1576 (2020)], where $$\alpha \in \overline{{\mathbb {Q}}}^*$$ was assumed to be such that $$\vert \alpha \vert <R$$ . Its proof relies on an explicit construction of a Pade approximation problem adapted to certain non-holomorphic functions associated to F, and it is quite different of that in the above mentioned paper. It makes use of results of Andre, Chudnovsky and Katz on G-operators, of a linear independence criterion a la Siegel over number fields, and of a far reaching generalization of Shidlovsky’s lemma built upon the approach of Bertrand–Beukers and Bertrand.

Development of a Patient-Reported Outcomes Measurement Information System (PROMIS\xae) short form for measuring physical function in geriatric rehabilitation patients

PurposeTo develop and test the validity of a Patient-Reported Outcomes Measurement Information System (PROMIS®) short form for measuring physical function of geriatric rehabilitation patients.MethodsExperts selected items from the Dutch-Flemish PROMIS v1.2 Physical Function (PROMIS-PF) item bank and proposed new items to develop the PROMIS-PF short form for geriatric rehabilitation (PROMIS-PF-GR). Patients evaluated its content validity. Structural validity was assessed by evaluating unidimensionality (confirmatory, exploratory, and bi-factor analyses [criterion: Omega H > 0.80 and ECV > 0.60]), local independence (criterion: residual correlation < 0.20) ,and monotonicity (criterion: Hi-coefficient ≥ 0.30). Measurement invariance was assessed by evaluating Differential Item Functioning (DIF) between geriatric rehabilitation patients and people from the general population using ordinal logistic regression. Internal consistency was assessed by calculating Cronbach’s alpha (criterion: alpha ≥ 0.70).ResultsExperts selected 24 items from the PROMIS-PF item bank and proposed one new item which was not included in the short form. Patients considered the 24 items relevant and containing essential information. The PROMIS-PF-GR’s psychometric properties were evaluated in 207 patients (mean age ± SD, 80.0 ± 8.3 year; 58% female). The 24 items were found to be sufficiently unidimensional (Omega H = 0.82, ECV = 0.70), locally independent (98.7% item pairs), and monotone (all ≥ 0.32). Five items were flagged for DIF, but their impact on the total score was negligible. Cronbach’s alpha was 0.94.ConclusionThe PROMIS-PF-GR was developed from the PROMIS-PF and has good content validity, structural validity, measurement invariance, and internal consistency in Dutch geriatric rehabilitation patients. We recommend to confirm the content validity of the PROMIS-PF-GR in other countries.

Independence Promoted Graph Disentangled Networks

We address the problem of disentangled representation learning with independent latent factors in graph convolutional networks (GCNs). The current methods usually learn node representation by describing its neighborhood as a perceptual whole in a holistic manner while ignoring the entanglement of the latent factors. However, a real-world graph is formed by the complex interaction of many latent factors (e.g., the same hobby, education or work in social network). While little effort has been made toward exploring the disentangled representation in GCNs. In this paper, we propose a novel Independence Promoted Graph Disentangled Networks (IPGDN) to learn disentangled node representation while enhancing the independence among node representations. In particular, we firstly present disentangled representation learning by neighborhood routing mechanism, and then employ the Hilbert-Schmidt Independence Criterion (HSIC) to enforce independence between the latent representations, which is effectively integrated into a graph convolutional framework as a regularizer at the output layer. Experimental studies on real-world graphs validate our model and demonstrate that our algorithms outperform the state-of-the-arts by a wide margin in different network applications, including semi-supervised graph classification, graph clustering and graph visualization.

Side Information Dependence as a Regularizer for Analyzing Human Brain Conditions across Cognitive Experiments

The increasing of public neuroimaging datasets opens a door to analyzing homogeneous human brain conditions across datasets by transfer learning (TL). However, neuroimaging data are high-dimensional, noisy, and with small sample sizes. It is challenging to learn a robust model for data across different cognitive experiments and subjects. A recent TL approach minimizes domain dependence to learn common cross-domain features, via the Hilbert-Schmidt Independence Criterion (HSIC). Inspired by this approach and the multi-source TL theory, we propose a Side Information Dependence Regularization (SIDeR) learning framework for TL in brain condition decoding. Specifically, SIDeR simultaneously minimizes the empirical risk and the statistical dependence on the domain side information, to reduce the theoretical generalization error bound. We construct 17 brain decoding TL tasks using public neuroimaging data for evaluation. Comprehensive experiments validate the superiority of SIDeR over ten competing methods, particularly an average improvement of 15.6% on the TL tasks with multi-source experiments.

Unsupervised Nonlinear Feature Selection from High-Dimensional Signed Networks

With the rapid development of social media services in recent years, relational data are explosively growing. The signed network, which consists of a mixture of positive and negative links, is an effective way to represent the friendly and hostile relations among nodes, which can represent users or items. Because the features associated with a node of a signed network are usually incomplete, noisy, unlabeled, and high-dimensional, feature selection is an important procedure to eliminate irrelevant features. However, existing network-based feature selection methods are linear methods, which means they can only select features that having the linear dependency on the output values. Moreover, in many social data, most nodes are unlabeled; therefore, selecting features in an unsupervised manner is generally preferred. To this end, in this paper, we propose a nonlinear unsupervised feature selection method for signed networks, called SignedLasso. This method can select a small number of important features with nonlinear associations between inputs and output from a high-dimensional data. More specifically, we formulate unsupervised feature selection as a nonlinear feature selection problem with the Hilbert-Schmidt Independence Criterion Lasso (HSIC Lasso), which can find a small number of features in a nonlinear manner. Then, we propose the use of a deep learning-based node embedding to represent node similarity without label information and incorporate the node embedding into the HSIC Lasso. Through experiments on two real world datasets, we show that the proposed algorithm is superior to existing linear unsupervised feature selection methods.

The HSIC Bottleneck: Deep Learning without Back-Propagation

We introduce the HSIC (Hilbert-Schmidt independence criterion) bottleneck for training deep neural networks. The HSIC bottleneck is an alternative to the conventional cross-entropy loss and backpropagation that has a number of distinct advantages. It mitigates exploding and vanishing gradients, resulting in the ability to learn very deep networks without skip connections. There is no requirement for symmetric feedback or update locking. We find that the HSIC bottleneck provides performance on MNIST/FashionMNIST/CIFAR10 classification comparable to backpropagation with a cross-entropy target, even when the system is not encouraged to make the output resemble the classification labels. Appending a single layer trained with SGD (without backpropagation) to reformat the information further improves performance.

Copula versions of distance multivariance and dHSIC via the distributional transform – a general approach to construct invariant dependence measures

ABSTRACTThe multivariate Hilbert-Schmidt Independence Criterion (dHSIC) and distance multivariance allow to measure and test independence of an arbitrary number of random vectors with arbitrary dimensions. Here, we define versions which only depend on an underlying copula. The approach is based on the distributional transform, yielding dependence measures which always feature a natural invariance with respect to scalings and translations. Moreover, it requires no distributional assumptions, i.e., the distributions can be of pure type or any mixture of discrete and continuous distributions and (in our setting) no existence of moments is required. Empirical estimators and tests, which are consistent against all alternatives, are provided based on a Monte Carlo distributional transform. In particular, it is shown that the new estimators inherit the exact limiting distributional properties of the original estimators. Examples illustrate that tests based on the new measures can be more powerful than tests based on other copula dependence measures.

A Suppression Method of Concentration Background Noise by Transductive Transfer Learning for a Metal Oxide Semiconductor-Based Electronic Nose.

Signal drift caused by sensors or environmental changes, which can be regarded as data distribution changes over time, is related to transductive transfer learning, and the data in the target domain is not labeled. We propose a method that learns a subspace with maximum independence of the concentration features (MICF) according to the Hilbert-Schmidt Independence Criterion (HSIC), which reduces the inter-concentration discrepancy of distributions. Then, we use Iterative Fisher Linear Discriminant (IFLD) to extract the signal features by reducing the divergence within classes and increasing the divergence among classes, which helps to prevent inconsistent ratios of different types of samples among the domains. The effectiveness of MICF and IFLD was verified by three sets of experiments using sensors in real world conditions, along with experiments conducted in the authors’ laboratory. The proposed method achieved an accuracy of 76.17%, which was better than any of the existing methods that publish their data on a publicly available dataset (the Gas Sensor Drift Dataset). It was found that the MICF-IFLD was simple and effective, reduced interferences, and deftly managed tasks of transfer classification.

Generation of Dynamical S-Boxes for Block Ciphers via Extended Logistic Map

In this work, we present a simple algorithm to design n × n-bits substitution boxes (S-boxes) based on chaotic time series of the logistic map for different carrying capacities. The use of different carrying capacities in the chaotic map leads to low computational complexity, which is desirable to get high-speed communication systems. We generate a main sequence by means of two auxiliary sequences with uniform distribution via the logistic map for different carrying capacities. The elements of the main sequence are useful for generating the elements of an S-box. The auxiliary sequences are generated by considering lag time chaotic series; this helps to hide the chaotic map used. The U-shape distribution of logistic chaotic map is also avoided, in contrast with common chaos-based schemes without considering lag time chaotic series, and uncorrelated S-box elements are obtained. The proposed algorithm guarantees the generation of strong S-boxes that fulfill the following criteria: bijection, nonlinearity, strict avalanche criterion, output bits independence criterion, criterion of equiprobable input/output XOR distribution, and maximum expected linear probability. Finally, an application premised on polyalphabetic ciphers principle is developed to obtain a uniform distribution of the plaintext via dynamical S-boxes.

Domain adaptive partial least squares regression

In practical applications, the problem of training- and test-samples from different distributions is often encountered, such as instruments or external environmental factors change when measuring the data. Therefore, a multivariate calibration model established, based on the training set needs to be adaptive to meet the requirements of test samples from different domains. Extracting domain adaptive latent variables is an effective method to address such issues, but related studies about making multivariate calibration models adaptive to different domains in unsupervised methods are rarely reported. In this paper, a domain adaptive partial least squares regression is proposed, which uses the Hilbert-Schmidt independence criterion to evaluate the independence of the extracted latent variables and domain labels. In both the original space and reproducing kernel Hilbert space, the proposed method can obtain a closed form of the domain adaptive projection vector, which ensures the high efficiency of the model calculation. The validity of the proposed method is verified by several simulation data sets and real near-infrared spectral data sets. The experimental results show that the proposed method has good universality for unsupervised regression calibration transfer.

Linear independence of values of $G$-functions

Given any non-polynomial G -function F(z)=\sum_{k=0}^\infty A_kz^k of radius of convergence R , we consider the G -functions F_n^{[s]}(z)=\sum_{k=0}^\infty \frac{A_k}{(k+n)^s}z^k for any integers s\ge 0 and n\ge 1 . For any fixed algebraic number \alpha such that 0 &lt; \vert \alpha\vert &lt; R and any number field \mathbb K containing \alpha and the A_k 's, we define \Phi_{\alpha, S} as the \mathbb K -vector space generated by the values F_n^{[s]}(\alpha) , n\ge 1 and 0\le s\le S . We prove that u_{\mathbb K,F}\log(S)\le \dim_\mathbb K(\Phi_{\alpha, S })\le v_FS for any S , with effective constants u_{\mathbb K,F} &gt; 0 and v_F &gt; 0 , and that the family \big(F_n^{[s]}(\alpha)\big)_{1\le n \le v_F, s\ge 0} contains infinitely many irrational numbers. This theorem applies in particular when F is a hypergeometric series with rational parameters or a multiple polylogarithm, and it encompasses a previous result by the second author and Marcovecchio in the case of polylogarithms. The proof relies on an explicit construction of Padé-type approximants. It makes use of results of André, Chudnovsky and Katz on G -operators, of a new linear independence criterion à la Nesterenko over number fields, of singularity analysis as well as of the saddle point method.

Regression and independence based variable importance measure

Evaluating the importance of input (predictor) variables is of interest in many applications of statistical models. However, nonlinearity and correlation among variables make it difficult to measure variable importance accurately. In this work, a novel variable importance measure, called regression and independence based variable importance (RIVI), is proposed. RIVI is designed by integrating Gaussian process regression (GPR) and Hilbert-Schmidt independence criterion (HSIC) so that it is applicable to nonlinear systems. The results of two numerical examples demonstrate that RIVI is superior to several conventional measures including the Pearson correlation coefficient, PLS-β, PLS-VIP, Lasso, HSIC, and permutation importance with random forest in the variable identification accuracy.

A novel method of dynamic S-box design based on combined chaotic map and fitness function

This paper introduces a method of generating a strong S-box by using combined chaotic system and fitness function. Our method presents an algorithm which has two stages: static S-box generation and dynamic S-box optimization. Chaotic sequences generation, sequences discretization are involved in the first stage. Moreover, the proposed static S-box is converted into a dynamic S-box with the help of fitness function. The optimal S-box satisfies the following criteria: bijectivity, nonlinearity, strict avalanche criteria, output bits independence criterion, equiprobable inputs/outputs XOR distribution. Extensive experiments demonstrate that the proposed algorithm has excellent performance in cryptography and attacks resistance.

Construction of Nonlinear Component of Block Cipher by Action of Modular Group PSL(2, Z) on Projective Line PL(GF(28))

Substitution box (S-Box) has a prominent significance being the fundamental nonlinear component of block cipher which fulfils confusion, one of the properties proposed by Claude Shannon in 1949. In this paper, we proposed an S-Box by using the action of modular group PSL $\left ({2,\mathbb {Z} }\right)$ on projective line PL $\left ({F_{257} }\right)$ over Galois field GF $\left ({2^{8} }\right)$ . In the first step we obtained elements of GF $\left ({2^{8} }\right)$ by using powers of $\alpha $ , where $\alpha $ is the primitive root of irreducible polynomial $p\left ({x }\right)$ of order 8 over field $\mathbb {Z}_{2}$ , then applied the generators of PSL $\left ({2,\mathbb {Z} }\right)$ and followed steps to get rid of infinity from output. In the final step of proposed scheme, one of the permutations of $S_{16}$ is applied which enhanced the possible number of S-Boxes obtained by any single specific irreducible polynomial $p(x)$ over field $\mathbb {Z}_{2}$ of order 8. We analyzed performance of the proposed $8\times 8$ S-Box under cryptographic properties such as strict avalanche criterion, bit independence criterion, nonlinearity, differential approximation probability, linear approximation probability; and compared obtained results with a number of renowned S-Boxes. Lastly, we performed statistical analysis (which comprises of contrast analysis, homogeneity analysis, energy analysis, correlation analysis, entropy analysis and mean of absolute deviation analysis) on our proposed S-Box and obtained results have been compared with adequate number of S-Boxes.

An Efficient Generation and Security Analysis of Substitution Box Using Fingerprint Patterns

Information and its security have attracted the research community in recent years with increasing usage of mobile applications. Mobile devices have different security options in data transmission such as reading some biometric values. The keystone of the modern block and stream ciphers is the use of a substitution box (s-box) that obscures correlation between plaintext and ciphertext. In this study, we proposed a novel s-box generation algorithm by using the fingerprint pattern of the person who transfers information to the target. We generated several s-boxes by using bifurcation and ridge ending features of the fingerprint. Proposed s-boxes are compared with several known s-boxes over nonlinearity, bijectiveness, strict avalanche criterion, bit independence criterion, linear probability, and differential probability. Along with these properties, we analyzed confidence interval and randomness properties of new s-boxes as well. Also, the execution time of the proposed s-box generation algorithm is calculated and examined. The results of the cryptographic properties have shown that the proposed s-boxes by using ridge ending of the fingerprint performs better. The performances analysis show that the proposed s-box has satisfactory results according to the results of chaotic-based s-boxes. On the other hand, the fingerprint s-boxes are much better than the existing biometric s-boxes according to the s-box security metrics. The results have shown that the execution time of the proposed s-box generation algorithm is more minimum than the existing biometric s-box generation algorithms. Resulting from applying fingerprint biometric data to generate an s-box, such a successful algorithm is promising to be used in mobile devices.

A Power Associative Loop Structure for the Construction of Non-Linear Components of Block Cipher

In the symmetric key cryptography, the purpose of the substitution box is to generate confusion and hence improve the security of the whole cryptosystem. For this reason, many algebraic substitution boxes are constructed depending upon the associative algebras of Galois fields. In this paper, the power associative loop is used for the construction of substitution boxes. This novel structure comprises excellent features which include, the inverse of zero element, non-associativity and fewer constraints as compared to the cyclic group and Galois field. In comparison to existing substitution boxes, the substitution boxes based upon power associative loop are comparatively easy and the above-mentioned properties offer the number of structures to construct highly nonlinear substitution boxes. To obtain the number of substitution boxes, we further applied the symmetric group of order 16 on the proposed substitution box. The evaluation of proposed boxes with different algebraic and statistical analyses like nonlinearity test, strict avalanche criterion, bit independence criterion, linear approximation probability and differential approximation probability indicate the strength of proposed substitution boxes. Majority logic criterion results depict that proposed substitution boxes have better cryptographic strength to apply in different techniques of secure communication.