Cauchy Matrix Structure Underlying the BKP Equation and its Dimensional Reductions

This paper examines the existence of efficiently implementable approximations of a general real linear dimensionality reduction (LDR) operator. The specific focus is on approximating a given LDR operator with a partial circulant structured matrix (a matrix whose rows are related by circular shifts) as these constructions allow for low-memory footprint and computationally efficient implementations. Our main contributions are theoretical: we quantify how well general matrices may be approximated (in a Frobenius sense) by partial circulant structured matrices, and also consider a variation of this problem where the aim is only to accurately approximate the action of a given LDR operator on a restricted set of inputs. For the latter setting, we also propose a sparsity-regularized alternating minimization based algorithm for learning partial circulant approximations from data, and provide experimental evidence demonstrating the potential efficacy of this approach on real-world data.

In most pattern recognition applications, the object of interest is represented by a very high dimensional data-vector. High dimensionality of modeling vectors poses serious challenges related to the efficiency of retrieval, analysis and classifying the pattern of interest. The Curse of Dimension is a general reference to these challenges and commonly addressed by Dimension Reduction (DR) techniques. The most commonly used DR schemes are data-dependent like Principal Component Analysis (PCA). However, we may expect over-fitting and biasness of the adaptive models to the training sets as consequences of low sample density ratio to dimension. Therefore, data-independent DR schemes such as Random Projections (RP) are more desirable. In this paper, we investigate and test the performance of differently constructed overcomplete Hadamard-based mxn (m<<n) sub-matrices using Walsh-Paley (WP) matrices as a DR scheme for Gait-based Gender Classification (GBGC). In particular, we shall demonstrate that these Hadamard-based RPs perform as well as, if not better, PCA and Gaussian-based RPs. Moreover, we shall show that Walsh-Paley Structured Matrices (WPSM) perform better than Walsh-Paley Random Matrices (WPRM).

ABSTRACTGaussian graphical models are widely used to infer and visualize networks of dependencies between continuous variables. However, inferring the graph is difficult when the sample size is small compared to the number of variables. To reduce the number of parameters to estimate in the model, we propose a nonasymptotic model selection procedure supported by strong theoretical guarantees based on an oracle type inequality and a minimax lower bound. The covariance matrix of the model is approximated by a block-diagonal matrix. The structure of this matrix is detected by thresholding the sample covariance matrix, where the threshold is selected using the slope heuristic. Based on the block-diagonal structure of the covariance matrix, the estimation problem is divided into several independent problems: subsequently, the network of dependencies between variables is inferred using the graphical lasso algorithm in each block. The performance of the procedure is illustrated on simulated data. An application to a real gene expression dataset with a limited sample size is also presented: the dimension reduction allows attention to be objectively focused on interactions among smaller subsets of genes, leading to a more parsimonious and interpretable modular network. Supplementary materials for this article are available online.

Nearly all existing dimension reduction methods on 2D matrix-valued image predictors are unsupervised or supervised without preserving matrix structure, which can result in loss of the structure-specific relation between the response and predictors. In this paper, we propose a kernel-based solution for supervised dimension reduction which preserves the matrix structure of the reduced predictors. This approach is computationally efficient and offers a unified framework to handle image predictors. We illustrate the method using both simulations and applications.

With the increasing design dimensionality, it is more difficult to solve Multidisciplinary design optimization (MDO) problems. To reduce the dimensionality of MDO problems, many MDO decomposition strategies have been developed. However, those strategies consider the design problem as a black-box function. In practice, the designers usually have certain knowledge of their problem. In this paper, a method leveraging causal graph and qualitative analysis is developed to reduce the dimensionality of the MDO problem by systematically modeling and incorporating knowledge of the design problem. Causal graph is employed to show the input-output relationships between variables. Qualitative analysis using design structure matrix (DSM) is carried out to automatically find the variables that can be determined without optimization. According to the weight of variables, the MDO problem is divided into two sub-problems, the optimization problem with respect to important variables, and the one with less important variables. The novel method is performed to solve an aircraft concept design problem and the results show that the new dimension reduction and decomposition method can significantly improve optimization efficiency.

Previous works have demonstrated that dimensionality reduction algorithms that combine sparse subspace learning (SSL) and discriminant information of sample data can improve the classification performance for some pattern recognition problems. However, most of these approaches introduce within-class sparse reconstruction matrix and within-class discriminative structure simultaneously, which may affect each other. To address this problem, in this paper, we propose a new dimensionality reduction algorithm called sparse discriminant preserving projections (SDPP).Different from the existing methods, SDPP uses between-class scatter as global discriminant information and integrates it with sparse reconstructive structure of samples in each class to establish the objective function. Since SDPP only takes into account the within-class sparsity reconstructive relationship and the between-class scatter information, it not only well preserves the sparse reconstruction relationship, but also improves classification performance. Experiments on face image databases demonstrate the superiority of the proposed algorithm.

Due to the spectral complexity and high dimensionality of hyperspectral images (HSIs), the processing of HSIs is susceptible to the curse of dimensionality. In addition, the classification results of ground truth are not ideal. To overcome the problem of the curse of dimensionality and improve classification accuracy, an improved spatial–spectral weight manifold embedding (ISS-WME) algorithm, which is based on hyperspectral data with their own manifold structure and local neighbors, is proposed in this study. The manifold structure was constructed using the structural weight matrix and the distance weight matrix. The structural weight matrix was composed of within-class and between-class coefficient representation matrices. These matrices were obtained by using the collaborative representation method. Furthermore, the distance weight matrix integrated the spatial and spectral information of HSIs. The ISS-WME algorithm describes the whole structure of the data by the weight matrix constructed by combining the within-class and between-class matrices and the spatial–spectral information of HSIs, and the nearest neighbor samples of the data are retained without changing when embedding to the low-dimensional space. To verify the classification effect of the ISS-WME algorithm, three classical data sets, namely Indian Pines, Pavia University, and Salinas scene, were subjected to experiments for this paper. Six methods of dimensionality reduction (DR) were used for comparison experiments using different classifiers such as k-nearest neighbor (KNN) and support vector machine (SVM). The experimental results show that the ISS-WME algorithm can represent the HSI structure better than other methods, and effectively improves the classification accuracy of HSIs.

This paper studies if the accuracies of mortality models (LC model vs. 4-parametric model) are aggravated if a mortality structure changes due to the impact of COVID-19. LC model (LCM) uses dimension reduction for fitting to the log mortality matrix so that the performance of the dimension reduction method may not be good when the matrix structure changes. On the other hand, 4-parametric factor model (4-PFM) is designed to use factors for fitting to log mortality data by age groups so that it would be less affected by the change of the mortality structure. In fact, the forecast accuracies of LCM are better than those of 4-PFM when life-tables are used whereas those of 4-PFM are better when the mortality structure changes. Thus this result shows that 4-PFM is more reliable in performance to the structural changes of the mortality. To support the accuracy changes of LCM the functional aspect is explained by computing eigenvalues produced by singular vector decomposition

Financial data (e.g., intraday share prices) are recorded almost continuously and thus take the form of a series of curves over the trading days. Those sequentially collected curves can be viewed as functional time series. When we have a large number of highly correlated shares, their intraday prices can be viewed as high-dimensional functional time series (HDFTS). In this paper, we propose a new approach to forecasting multiple financial functional time series that are highly correlated. The difficulty of forecasting high-dimensional functional time series lies in the “curse of dimensionality.” What complicates this problem is modeling the autocorrelation in the price curves and the comovement of multiple share prices simultaneously. To address these issues, we apply a matrix factor model to reduce the dimension. The matrix structure is maintained, as information contains in rows and columns of a matrix are interrelated. An application to the constituent stocks in the Dow Jones index shows that our approach can improve both dimension reduction and forecasting results when compared with various existing methods.

Data are often passed through rectangular matrix transformations in adaptive beamforming and direction-of-arrival estimation to reduce data dimension and lower computational load. The authors show that the transformation defines a structured model for the full dimension data covariance matrix. Signal processing applied to the reduced dimension data is equivalent to processing the original data while constraining the covariance matrix to have the specified structure. A procedure is given for designing the structure of the covariance matrix to minimize the average error between the true covariance and the structured model. Simulations indicate that this design procedure is very effective and that improved resolution can be obtained with reduced-dimension direction-of-arrival estimates.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

This work proposes a blind adaptive reduced-rank scheme and constrained constant-modulus (CCM) adaptive algorithms for interference suppression in wireless communications systems. The proposed scheme and algorithms are based on a two-stage processing framework that consists of a transformation matrix that performs dimensionality reduction followed by a reduced-rank estimator. The complex structure of the transformation matrix of existing methods motivates the development of a blind adaptive reduced-rank constrained (BARC) scheme along with a low-complexity reduced-rank decomposition. The proposed BARC scheme and a reduced-rank decomposition based on the concept of joint interpolation, switched decimation and reduced-rank estimation subject to a set of constraints are then detailed. The proposed set of constraints ensures that the multipath components of the channel are combined prior to dimensionality reduction. We develop low-complexity joint interpolation and decimation techniques, stochastic gradient, and recursive least squares reduced-rank estimation algorithms. A model-order selection algorithm for adjusting the length of the estimators is devised along with techniques for determining the required number of switching branches to attain a predefined performance. An analysis of the convergence properties and issues of the proposed optimization and algorithms is carried out, and the key features of the optimization problem are discussed. We consider the application of the proposed algorithms to interference suppression in DS-CDMA systems. The results show that the proposed algorithms outperform the best known reduced-rank schemes, while requiring lower complexity.

The dimension reduction is necessary steps for face recognition based on subspace analysis. The proposed method employs class information for structure of similarity matrix when implement of 2DLPP. A subspace which preserves local neighbor structure and centralizes same class samples of training images is got. Moreover, it has available computation efficiency and accuracy because it belongs to the methods based on images which avoid the matrix singularity problem. The performance of the proposed method is evaluated and compared with other popular subspace analysis method based on ORL database. The experiment results show that it has more accurate recognition than previous methods.

This paper is motivated by the problem of optimal allocation of trials in multi‐environment crop variety testing with a large number of varieties. Optimizing the allocation of trials results in the minimization of a design criterion with a Kronecker product structure in the information matrix. We consider the Kronecker–Bayesian linear criterion, which generalizes this design problem and has the form of the trace of the inverse of a sum of two Kronecker products. We derive a new general formula for the inverse of the sum of two Kronecker products, and we use this result to rewrite the Kronecker–Bayesian criterion in the form of the compound Bayes risk criterion, which can be recognized as a sum of Bayesian linear criteria with the same moment matrix. Based on the convexity and differentiability of the Kronecker–Bayesian linear criterion, we establish optimality conditions for approximate designs. We also propose a dimension reduction approach that provides highly efficient approximations for optimal designs. The proposed method allows for the preselection of an upper bound on the efficiency loss, which is independent of the true optimal design. Optimal or highly efficient designs can be computed under any kind of additional linear constraints, such as cost constraints. We apply our results to the problem of optimizing the allocation of trials in multi‐environment crop variety testing, and we illustrate the behavior of the optimal designs by real data examples. Finally, we consider further applications of the general formula for computing the inverse of the sum of two Kronecker products in control theory or multivariate time series analysis.

The wider use of wearable devices for electroencephalogram (EEG) data capturing provides a very useful way for the monitoring and self-management of human health. However, the large volumes of data with high dimensions cause computational complexity in EEG data processing and pose a great challenge to the use of wearable EEG devices in healthcare. This paper proposes a new approach to extract the structural information of EEG data and tackle the curse of dimensionality of the EEG data. A set of methods for dimensionality reduction (DR)-like linear discriminant analysis (LDA) and their improved methods have been developed for EEG processing in the literature. However, the existing LDA-related methods suffer from the singularity problem or expensive computational cost, and none of existing methods take into consideration the structure of the projection matrix, which is crucial for the extraction of the structural information of the EEG data. In this paper, a new method called a regularized matrix discriminant analysis (R-MDA) is proposed for EEG feature representation and DR. In the R-MDA, the EEG data are represented as a data matrix, and projection vectors are reshaped to be a set of projection matrices stacking together. By reformulating the LDA as a least-square formulation and imposing specified constraint on each projection matrix, the new R-MDA has been constructed to effectively reduce EEG dimensions and capturing the structural information of the EEG data. Experimental results demonstrate that this new R-MDA outperforms the existing LDA-related methods, including achieving improved accuracy with significant DR of the EEG data. This offers an effective way to enable wearable EEG devices be applicable in human-centered health monitoring.

Dimensionality reduction has attracted increasing attention, because high-dimensional data have arisen naturally in numerous domains in recent years. As one popular dimensionality reduction method, nonnegative matrix factorization (NMF), whose goal is to learn parts-based representations, has been widely studied and applied to various applications. In contrast to the previous approaches, this paper proposes a novel semisupervised NMF learning framework, called robust structured NMF, that learns a robust discriminative representation by leveraging the block-diagonal structure and the -norm (especially when ) loss function. Specifically, the problems of noise and outliers are well addressed by the -norm ( ) loss function, while the discriminative representations of both the labeled and unlabeled data are simultaneously learned by explicitly exploring the block-diagonal structure. The proposed problem is formulated as an optimization problem with a well-defined objective function solved by the proposed iterative algorithm. The convergence of the proposed optimization algorithm is analyzed both theoretically and empirically. In addition, we also discuss the relationships between the proposed method and some previous methods. Extensive experiments on both the synthetic and real-world data sets are conducted, and the experimental results demonstrate the effectiveness of the proposed method in comparison to the state-of-the-art methods.