Dimensional Subspace Research Articles

Heart Disease Prediction using Hybrid Machine Learning Algorithms

Heart ailment is one of the most acknowledged and lethal illnesses within the world, and many human beings lose their lives from this disease every year. Early detection of this disease is essential to store people’s lives. Machine Learning (ML), an synthetic intelligence technology, is one of the most handy, quickest, and low-cost ways to discover sickness. In this look at, we purpose to attain an ML model that can expect heart sickness with the highest possible performance the use of the Cleveland coronary heart disease dataset. The functions in the dataset used to train the version and the selection of the ML algorithm have a significant effect at the performance of the model. To keep away from overfitting (because of the curse of dimensionality) due to the huge quantity of functions inside the Cleveland dataset, the dataset became decreased to a lower dimensional subspace the usage of the Jellyfish optimization set of rules. The Jellyfish algorithm has a excessive convergence speed and is flexible to locate the exceptional functions.

Hierarchical Bayesian optimization based convolutional neural network for chest X-ray disease classification

Pneumonia is an infection that affects the lungs, caused by bacteria or viruses inhaled through the air, leading to respiratory problems. The previous researches on this subject have limitations of high dimensional feature subspace and overfitting which minimize the classifier performance. In this research, hierarchical Bayesian optimization based convolutional neural network (HBO-CNN) method is proposed to effectively classify chest X-ray diseases. The proposed HBO algorithm optimizes hyperparameters of CNN which minimizes the overfitting issue and enhances the performance of classification. The hybrid Mexican axolotl optimization (MAO) and tuna swarm optimization (TSO) based feature selection method is used for selecting relevant features for classification that minimizes the high dimensional features. The ResNet 50 method is used for feature extraction to extract hierarchical features from the pre-processed images to differentiate the classes. The proposed HBO-CNN technique is estimated with performance metrics of accuracy, precision, recall, and F1-score. The proposed method attains the highest accuracy 97.95%, precision 92.00%, recall 89.00% and F1-score 92.00%, as opposed to the conventional methods, deep convolutional neural network (DCNN).

Online Lewis Weight Sampling

The seminal work of Cohen and Peng [10] (STOC 2015) introduced Lewis weight sampling to the theoretical computer science community, which yields fast row sampling algorithms for approximating \(d\) -dimensional subspaces of \(\ell_{p}\) up to \((1+ \varepsilon)\) relative error. Prior works have extended this important primitive to other settings, such as the online coreset and sliding window models [4] (FOCS 2020). However, these results are only for \(p\in\{1,2\}\) , and results for \(p=1\) require a suboptimal \(\tilde{O}(d^{2}/\varepsilon^{2})\) samples. In this work, we design the first nearly optimal \(\ell_{p}\) subspace embeddings for all \(p\in(0,\infty)\) in the online coreset and sliding window models. In both models, our algorithms store \(\tilde{O}(d/\varepsilon^{2})\) rows for \(p\in(0,2)\) and \(\tilde{O}(d^{p/2}/\varepsilon^{2})\) rows for \(p\in(2,\infty)\) . This answers a substantial generalization of the main open question of [4], gives the first results for all \(p\notin\{1,2\}\) , and achieves nearly optimal sample complexities for all \(p\) . Towards our result, we give the first analysis of “one-shot” Lewis weight sampling of sampling rows proportionally to their Lewis weights, which gives a sample complexity of \(\tilde{O}(d^{p/2}/\varepsilon^{2})\) rows for \(p>2\) . Previously, such a sampling scheme was only known to have a sample complexity of \(\tilde{O}(d^{p/2}/\varepsilon^{5})\) [10], whereas a bound of \(\tilde{O}(d^{p/2}/\varepsilon^{2})\) is known if a more sophisticated recursive sampling algorithm is used [20, 32]. Note that the recursive sampling strategy cannot be implemented in an online setting, thus necessitating an analysis of one-shot Lewis weight sampling. Perhaps surprisingly, our analysis crucially uses a novel connection to online numerical linear algebra, even for offline Lewis weight sampling . As an application, we obtain the first online coreset algorithms for \((1+\varepsilon)\) approximation of important generalized linear models, such as logistic regression and \(p\) -probit regression. Our upper bounds are parameterized by a complexity parameter \(\mu\) introduced by [31], and we also provide the first lower bounds showing that a linear dependence on \(\mu\) is necessary.

Low-Complexity 2D DOA Estimation via L-Shaped Array for Underwater Hexapod Robot

This paper takes underwater hexapod robot target grasping in an extremely shallow water environment as the research goal and carries out the research on a high-precision and low-complexity method of target positioning. We address the above problem of estimating the two-dimensional (2D) directions of arrival (DOAs) of targets, using an L-shaped ultrasonic array. Based on the above considerations, low-complexity 2D multiple signal classification (MUSIC) based on sparse signal recovery (SSR) is proposed to enhance the super-resolution capability and DOA estimation accuracy. In the first step, subspace dimension is determined based on space distance. Then, a mixed-norm method is exploited to construct the projection subspace and new noise subspace. Finally, the orthogonality between the noise and signal subspaces is used to estimate DOAs. Via a numerical simulations analysis, we illustrate that the proposed technique can enhance the accuracy of DOA estimation while also being robust against coherent sources and limited snapshots.

Doubling the rate: improved error bounds for orthogonal projection with application to interpolation

Convergence rates for L2 approximation in a Hilbert space H are a central theme in numerical analysis. The present work is inspired by Schaback (Math Comp, 1999), who showed, in the context of best pointwise approximation for radial basis function interpolation, that the convergence rate for sufficiently smooth functions can be doubled, compared to the best rate for functions in the “native space” H. Motivated by this, we obtain a general result for H-orthogonal projection onto a finite dimensional subspace of H: namely, that any known L2 convergence rate for all functions in H translates into a doubled L2 convergence rate for functions in a smoother normed space B, along with a similarly improved error bound in the H-norm, provided that L2, H and B are suitably related. As a special case we improve the known L2 and H-norm convergence rates for kernel interpolation in reproducing kernel Hilbert spaces, with particular attention to a recent study (Kaarnioja, Kazashi, Kuo, Nobile, Sloan, Numer. Math., 2022) of periodic kernel-based interpolation at lattice points applied to parametric partial differential equations. A second application is to radial basis function interpolation for general conditionally positive definite basis functions, where again the L2 convergence rate is doubled, and the convergence rate in the native space norm is similarly improved, for all functions in a smoother normed space B.

Accurate channel estimation of on-grid partially coherent compressive phase retrieval for mmWave massive MIMO systems

Abstract Channel estimation of sparse signal is a challenging task for millimeter-wave massive multi-input multi-output systems. Due to the hardware imperfections and use of large carrier frequency oscillator in hybrid precoding architectures leads to random phase drifts in the received pilots. Channel estimation with random phase drifts exploits partial coherence, where pilots share phase distortion within a frame but differ across different frames. To achieve the reliable and efficient communication, the support vector initialization needs to be optimized effectively. In this article, the singular value decomposition-based on-grid partially coherent compressive phase retrieval (SVD-PC-CPR) algorithm is proposed. This algorithm optimizes the initial support vector by finding the best dimensional subspace of the k sparsity elements by forming the singular vectors. The channel vectors are estimated by applying the gradient descent method through iterative refinements. This algorithm utilizes SVD for identifying the dominant paths from the phases of the measurements by selecting basis vectors. Based on the simulation results, the proposed algorithm achieves better normalized mean squared error, a multi-user sum-rate gain of 19 bits/s/Hz, and a 48 % improvement in estimation accuracy over the conventional PC-CPR algorithm at 20 dB signal-to-noise ratio.

Accelerated design of solid bio-based foams for plastics substitutes.

Biobased substitutes for plastics are a future necessity. However, the design of substitute materials with similar or improved properties is a known challenge. Here we show an example case of optimizing the mechanical properties of a fully biobased methylcellulose-fiber composite material. We tackle the process-structure-property paradigm using Bayesian optimization with Gaussian process regression to map the processed material composition to the final mechanical properties of new bio-based solid foams. We exploited the fast-to-measure rheological properties of the liquid biofiber suspensions processed into foams to show how these collapse to an auxiliary sub-space of low dimensionality for design. The optimal compositions for methylcellulose-fiber foams shown here correspond to two distinct cases: high methylcellulose content for the formation of strong closed-cell foams, and high fiber contents with approximately equal amounts of methylcellulose for the formation of methylcellulose-bound fiber networks. The novel approach is transferable to other biobased foam compositions with different fibers and additives. This new approach allows the rational design of bio-based plastics replacements by encompassing desired final material properties, descriptors of materials processed, and knowledge of the process.

Direct localization of wideband sources using distributed arrays: A subspace focusing and dimension reduction approach

Direct localization of wideband sources using distributed arrays: A subspace focusing and dimension reduction approach

Detection of Energetic Low Dimensional Subspaces in Spatio-Temporal Space in Turbulent Pipe Flow

AbstractLow dimensional subspaces are extracted out of highly complex turbulent pipe flow at $$Re_{\tau }=181$$ R e τ = 181 using a Characteristic Dynamic Mode Decomposition (CDMD). Having lower degrees of freedom, the subspaces provide a more clear basis to detect events which meet our understanding of large-scale coherent structures. To this end, a temporal sequence of state vectors from direct numerical simulations are rotated in space-time such that persistent dynamical modes on a hyper-surface are found travelling along its normal in space-time, which serves as the new time-like coordinate. The main flow features are captured with a minimal number of modes on a moving frame of reference whose velocity matches that of the most energetic scale. Reconstruction of the candidate modes in physical space gives the low rank model of the flow. The structures living in this subspace have long lifetimes, posses wide range of length-scales and travel at group velocities close to that of the moving frame of reference. The modes within this subspace are highly aligned, but are separated from the remaining modes by larger angles. We are able to capture the essential features of the flow like the spectral energy distribution and Reynolds stresses with a subspace consisting of about 10 modes. The remaining modes are collected in two further subspaces, which distinguish themselves by their axial length scale and degree of isotropy.

The index of equidimensional flag manifolds

AbstractIn this paper, we consider the flag manifold of orthogonal subspaces of equal dimension that carries an action of the cyclic group of order . We provide a complete calculation of the associated Fadell–Husseini index. This may be thought of as an odd primary version of the computations of Baralić, Blagojevic, Karasev, and Vucic, for the Grassmann manifold . These results have geometric consequences for ‐fold orthogonal shadows of a convex body.

Structural properties of Krylov subspaces, Krylov solvability, and applications to unbounded self-adjoint operators

Structural properties of Krylov subspaces, Krylov solvability, and applications to unbounded self-adjoint operators

Invariants in divided power algebras

Let k be an algebraically closed field of characteristic p>0, let G=GLn be the general linear group over k, let g=gln be its Lie algebra and let Ds be subalgebra of the divided power algebra of g⁎ spanned by the divided power monomials with exponents <ps. We give a basis for the G-invariants in Ds up to degree n and show that these are also the g-invariants.We define a certain natural restriction property and show that it doesn't hold when s>1. If s=1, then D1 is isomorphic to the truncated coordinate ring of g of dimension pdim⁡(g) and we conjecture that the restriction property holds and show that this leads to a conjectural spanning set for the invariants (in all degrees).We give similar results for the divided power algebras of several matrices and of vectors and covectors, and show that in the second case the restriction property doesn't hold.We also give the dimensions of the filtration subspaces of degree ≤n of the centre of the hyperalgebra of the Frobenius kernel Gs.

High-Dimensional Bayesian Optimization Using Both Random and Supervised Embeddings

Bayesian optimization (BO) is one of the most powerful strategies to solve computationally expensive-to-evaluate blackbox optimization problems. However, BO methods are conventionally used for optimization problems of small dimension because of the curse of dimensionality. In this paper, a high-dimensionnal optimization method incorporating linear embedding subspaces of small dimension is proposed to efficiently perform the optimization. An adaptive learning strategy for these linear embeddings is carried out in conjunction with the optimization. The resulting BO method, named efficient global optimization coupled with random and supervised embedding (EGORSE), combines in an adaptive way both random and supervised linear embeddings. EGORSE has been compared to state-of-the-art algorithms and tested on academic examples with a number of design variables ranging from 10 to 600. The obtained results show the high potential of EGORSE to solve high-dimensional blackbox optimization problems, in terms of both CPU time and the limited number of calls to the expensive blackbox simulation.

Unsupervised Feature Selection via Controllable Adaptive Graph Learning and Discriminative Feature Learning.

Unsupervised feature selection is challenging in machine learning, pattern recognition, and data mining. The crucial difficulty is to learn a moderate subspace that preserves the intrinsic structure and to find uncorrelated or independent features simultaneously. The most common solution is first to project the original data into a lower dimensional space and then force them to preserve the similar intrinsic structure under linear uncorrelation constraint. However, there are three shortcomings. First, the final graph generated by the iterative learning process differs significantly from the initial graph in which the original intrinsic structure is embedded. Second, it requires prior knowledge about a moderate dimension of subspace. Third, it is inefficient when dealing with high-dimensional datasets. The first shortcoming, which is longstanding and undiscovered, makes the previous methods fail to achieve their expected results. The last two ones increase the difficulty of applying in different fields. Therefore, two unsupervised feature selection methods are proposed based on controllable adaptive graph learning and uncorrelated/independent feature learning (CAG-U and CAG-I) to address the abovementioned issues. In the proposed methods, the final graph that preserves intrinsic structure can be adaptively learned while the difference between the two graphs can be well controlled. Besides, relatively uncorrelated/independent features can be selected using a discrete projection matrix. The experimental results on 12 datasets in different fields show the superiority of CAG-U and CAG-I.

An auxiliary correction model implemented by the Correction Property of intermediate layer against adversarial examples

In powerful adversarial attacks against deep neural networks (DNN), the generated adversarial example will mislead the DNN-implemented classifier by destroying the features of the last layer. To enhance the robustness of the classifier, in our paper, a Feature Analysis and Conditional Matching prediction distribution (FACM) model is proposed to utilize the features of intermediate layers to correct the misclassification. Specifically, we first prove that the intermediate layers of the classifier still retain effective features for the original category when the classifier is subjected to adversarial attacks, which is defined as the Correction Property in our paper. According to this, we propose the FACM model consisting of Feature Analysis (FA) correction module, Conditional Matching Prediction Distribution (CMPD) correction module and decision module. Specifically, the FA correction module is comprised of fully connected layers, which takes the features of the intermediate layers as the input to correct the misclassification of the classifier. The CMPD correction module is based on a conditional autoencoder, which not only uses the features of intermediate layers as the condition to accelerate convergence but also mitigates the negative effect of adversarial examples, trained with the Kullback–Leibler loss to match prediction distribution. Through the empirically verified Diversity Property among the individual correction modules, the decision module is proposed to integrate the proposed correction modules to enhance the DNN-implemented classifier’s robustness by reducing the dimensionality of adversarial subspace. That is, the input perturbed in certain directions (i.e., dimensions) that lead to misclassifications for the classifier can be correctly classified by the proposed correction modules. The extended experiments demonstrate our FACM model outperforms the existing methods against adversarial attacks, especially optimization-based white-box attacks and query-based black-box attacks.

Light field images super-resolution method based on hybrid low-dimensional spatial–angular interaction feature and linear complementation epipolar feature

Light-field (LF) cameras record 4D information about the intensity and angle of light, but limited by the resolution of the sensor, LF images require a trade-off between spatial and angular resolution, which results in generally low spatial resolution for LF images. We work on spatial super-resolution reconstruction (SR) of LF images, but due to the high dimensionality of 4D LF, it is difficult to process it directly, so we decompose 4D LF into low-dimensional sub-spaces. Since the 4D features of interest to different low-dimensional sub-spaces are different, we design a hybrid network structure—HSAESR, which processes multiple low-dimensional sub-spaces individually to adequately model the texture information in the 4D LF. HSAESR is designed to reconstruct Macro Pixel Images (MacPI) and Extremely Planar Images (EPI) respectively, and the final reconstruction results are weighted and fused. For MacPI, a CNN-based feature interaction module is designed using a novel asymptotic ’grid’ interaction approach to fully utilize the correlation between the 2D spatial dimension and the 2D angular dimension sub-spaces in 4D LF. For EPI, a Transformer-based epipolar feature linear complementation module is designed learning global and non-global multi-directional epipolar feature correlations, which learns feature linear correlation factor (α) and linear bias weight (β) through the correlation of non-global epipolar features, and corrects and complements the global epipolar correlation features through feature linear complementation. Extensive experiments are conducted on five datasets, the results show that HSAESR further improves the performance of LF images spatial SR, the reconstruction results are better than the comparison methods.

Environmental sound classification using raw-audio based ensemble framework

Environmental sound classification (ESC) aims to automatically recognize audio recordings from the underlying environment, such as "urban park" or "city centre". Most of the existing methods for ESC use hand-crafted time-frequency features such as log-mel spectrogram to represent audio recordings. However, the hand-crafted features rely on transformations that are defined beforehand and do not consider the variability in the environment due to differences in recording conditions or recording devices. To overcome this, we present an alternative representation framework by leveraging a pre-trained convolutional neural network, SoundNet, trained on a large-scale audio dataset to represent raw audio recordings. We observe that the representations obtained from the intermediate layers of SoundNet lie in low-dimensional subspace. However, the dimensionality of the low-dimensional subspace is not known. To address this, an automatic compact dictionary learning framework is utilized that gives the dimensionality of the underlying subspace. The low-dimensional embeddings are then aggregated in a late-fusion manner in the ensemble framework to incorporate hierarchical information learned at various intermediate layers of SoundNet. We perform experimental evaluation on publicly available DCASE 2017 and 2018 ASC datasets. The proposed ensemble framework improves performance between 1 and 4 percentage points compared to that of existing time-frequency representations.

Understanding overfitting in random forest for probability estimation: a visualization and simulation study

BackgroundRandom forests have become popular for clinical risk prediction modeling. In a case study on predicting ovarian malignancy, we observed training AUCs close to 1. Although this suggests overfitting, performance was competitive on test data. We aimed to understand the behavior of random forests for probability estimation by (1) visualizing data space in three real-world case studies and (2) a simulation study.MethodsFor the case studies, multinomial risk estimates were visualized using heatmaps in a 2-dimensional subspace. The simulation study included 48 logistic data-generating mechanisms (DGM), varying the predictor distribution, the number of predictors, the correlation between predictors, the true AUC, and the strength of true predictors. For each DGM, 1000 training datasets of size 200 or 4000 with binary outcomes were simulated, and random forest models were trained with minimum node size 2 or 20 using the ranger R package, resulting in 192 scenarios in total. Model performance was evaluated on large test datasets (N = 100,000).ResultsThe visualizations suggested that the model learned “spikes of probability” around events in the training set. A cluster of events created a bigger peak or plateau (signal), isolated events local peaks (noise). In the simulation study, median training AUCs were between 0.97 and 1 unless there were 4 binary predictors or 16 binary predictors with a minimum node size of 20. The median discrimination loss, i.e., the difference between the median test AUC and the true AUC, was 0.025 (range 0.00 to 0.13). Median training AUCs had Spearman correlations of around 0.70 with discrimination loss. Median test AUCs were higher with higher events per variable, higher minimum node size, and binary predictors. Median training calibration slopes were always above 1 and were not correlated with median test slopes across scenarios (Spearman correlation − 0.11). Median test slopes were higher with higher true AUC, higher minimum node size, and higher sample size.ConclusionsRandom forests learn local probability peaks that often yield near perfect training AUCs without strongly affecting AUCs on test data. When the aim is probability estimation, the simulation results go against the common recommendation to use fully grown trees in random forest models.

The maximum principle for lumped-distributed control systems.

This paper concerns the optimal control of lumped-distributed systems, an examplar of which is a mechanical system with rotating components connected by flexible rods. The underlying mathematical model is a controlled semilinear evolution equation, in which nonlinear terms involve a projection of the full state onto a finite dimensional subspace. We derive necessary conditions of optimality in the form of a maximum principle, for a problem formulation which involves pathwise and end-point constraints on the lumped components of the state variable. Perturbational methods are employed in the derivation, which allow for non-smooth data. A key step in the analysis is the reduction of the optimization problem with an infinite dimensional state space, to one with finite dimensional aspects. The computational implications of this reduction will be explored in future work. Necessary conditions for problems with state constraints and end-point constraints can only be achieved under rather strict compatibility hypotheses on the way the dynamics interact with the state constraint and end-point constraint sets, due the infinite dimensionality of the state space. This paper identifies a class of infinite dimensional problems, of engineering significance, for which necessary conditions of optimality can be derived, in the absence of any additional compatibility hypotheses.

Minimal cover of high-dimensional chaotic attractors by embedded recurrent patterns

We propose a general method for constructing a minimal cover of high-dimensional chaotic attractors by embedded unstable recurrent patterns. By minimal cover we mean a subset of available patterns such that the approximation of chaotic dynamics by a minimal cover with a predefined proximity threshold is as good as the approximation by the full available set. The proximity measure, based on the concept of a directed Hausdorff distance, can be chosen with considerable freedom and adapted to the properties of a given chaotic system. In the context of a spatiotemporally chaotic attractor of the Kuramoto–Sivashinsky system on a periodic domain, we demonstrate that the minimal cover can be faithfully constructed even when the proximity measure is defined within a subspace of dimension much smaller than the dimension of space containing the attractor. We discuss how the minimal cover can be used to provide a reduced description of the attractor structure and the dynamics on it.