Low-rank Feature Research Articles

FPGA-oriented lightweight multi-modal free-space detection network

For autonomous vehicles, free-space detection is an essential part of visual perception. With the development of multi-modal convolutional neural networks (CNNs) in recent years, the performance of driving scene semantic segmentation algorithms has been dramatically improved. Therefore most free-space detection algorithms are developed based on multiple sensors. However, multi-modal CNNs have high data throughput and contain a large number of computationally intensive convolution calculations, limiting their feasibility for real-time applications. Field Programmable Gate Arrays (FPGAs) provide a unique combination of flexibility, performance, and low power for these problems to accommodate multi-modal data and the computational acceleration of different compression algorithms. Network lightweight methods offer great assurance for facilitating the deployment of CNNs on such resource-constrained devices. In this paper, we propose a network lightweight method for a multi-modal free-space detection algorithm. We first propose an FPGA-friendly multi-modal free-space detection lightweight network. It comprises operators that FPGA prefers and achieves a 95.54 % MaxF score on the test set of KITTI-Road free-space detection tasks and 81 ms runtime when running on 700 W GPU devices. Then we present a pruning approach for this network to reduce the number of parameters in case the complete model exceeds the FPGA chip memory. The pruning is in two parts. For the feature extractors, we propose a data-dependent filter pruner according to the principle that the low-rank feature map contains less information. To not compromise the integrity of the multi-modal information, the pruner is independent for each modality. For the segmentation decoder, we apply a channel pruning approach to remove redundant parameters. Finally, we implement our designs on an FPGA board using 8-bit quantisation, and the accelerator achieves outstanding performance. A real-time application of scene segmentation on KITTI-Road is used to evaluate our algorithm, and the model achieves a 94.39 % MaxF score and minimum 14 ms runtime on 20W FPGA devices.

Low-field NMR inversion based on low-rank and sparsity restraint of relaxation spectra

In this paper, we proposed a novel method for low-field nuclear magnetic resonance (NMR) inversion based on low-rank and sparsity restraint (LRSR) of relaxation spectra, with which high quality construction is made possible for one- and two-dimensional low-field and low signal to noise ratio NMR data. In this method, the low-rank and sparsity restraints are introduced into the objective function instead of the smoothing term. The low-rank features in relaxation spectra are extracted to ensure the local characteristics and morphology of spectra. The sparsity and residual term are contributed to the resolution and precision of spectra, with the elimination of the redundant relaxation components. Optimization process of the objective function is designed with alternating direction method of multiples, in which the objective function is decomposed into three subproblems to be independently solved. The optimum solution can be obtained by alternating iteration and updating process. At first, numerical simulations are conducted on synthetic echo data with different signal-to-noise ratios, to optimize the desirable regularization parameters and verify the feasibility and effectiveness of proposed method. Then, NMR experiments on solutions and artificial sandstone samples are conducted and analyzed, which validates the robustness and reliability of the proposed method. The results from simulations and experiments have demonstrated that the suggested method has unique advantages for improving the resolution of relaxation spectra and enhancing the ability of fluid quantitative identification.

Robust discriminative projection with dynamic graph regularization for feature extraction and classification

Learning an efficient discriminative projection to extract meaningful features from high-dimensional data is a challenging task in machine learning. However, most of the existing discriminative projection literatures have the following two issues: (1) the performance of projection would largely degenerate when handling the data corrupted by complex outlier, e.g., noisy and outlier mixed data; and (2) fixed graph in graph regularization can be negatively affected by outliers or noisy features. To deal with above issues, we design a new feature extraction method called robust discriminative projection with dynamic graph regularization (RDPDG) to handle above problems. Specifically, RDPDG simulates complex outlier features, i.e., low rank and sparse structured outlier features, using two matrices constrained by the nuclear and adaptive loss norm. RDPDG is able to boost the performance of classification, which primarily due to two key factors: (1) learns reliable projection through a unified framework that integrates discriminant analysis, reconstituted data and manifold learning; and (2) can exploit the local intrinsic structure among data effectively by dynamic graph regularization. Furthermore, an alternating iteration algorithm is designed to solve the proposed nonconvex model, which seamlessly connects the update process of all components in the RDPDG framework, offering a powerful discriminative capability for projection. Comprehensive experiments on a synthetic and several real-world data sets demonstrate that the RDPDG outperforms the precious related methods for classification.

Low-rank matrix factorization with nonconvex regularization and bilinear decomposition

Low-rank matrix factorization (LRMF) is a powerful approach that can recover useful information with low-rank features from corrupted data and has been broadly applied in various scenarios such as computer vision and statistics. Existing convex relaxation-based LRMF models' performance and solutions are often limited by unsatisfied recovery accuracy and prolonged execution time due to excessive relaxation bias and heavy computational burden. In this paper, we first explore one class of parameterized non-convex penalty function and generalize it as a non-convex relaxation substitution term for the original rank function in the LRMF, which can effectively enhance the recovery capability for low-rank data. In order to reduce the computational burden of the nuclear-norm-based optimization scheme and make LRMF applicable to large-scale data. We propose to perform bilinear decomposition on the low-rank matrix and derive the bilinear factorization formula under the parameterized non-convex alternative. With the above considering, a low-rank matrix recovery model via bilinear decomposition and non-convex regularization, denoted as NCBF, is formulated. Further, the optimal solution algorithm for the NCBF model is developed in the framework of the block coordinate descent, combining simultaneously three strategies of decoupling, proximal gradient descent, and block prox-linear. Also, the convergence condition for the proposed algorithm has been given. We perform extensive experiments using synthetic data and real-world image datasets. The synthetic data test results demonstrate the dual superiority of the bilinear decomposition scheme in terms of operation time and estimation accuracy. Application of denoising to both generic and CT image datasets, experimental results show that NCBF outperforms several existing methods under both subjective and objective metrics evaluations.

LJDA-Net: A Low-Rank Joint Domain Adaptation Network for Industrial Sample Enhancement

The small sample problem is a challenge for data-driven industrial fault diagnosis. The available sample enhancement works have limited application in real industry, due to the difficulty in achieving the expected efficiency and accuracy performance of modeling. To address this issue, this paper proposes a low-rank joint domain adaptation network (LJDA-Net). First, we design the low-rank feature extraction subnet to effectively alleviate the limitation of feature redundancy and improve model training efficiency while ensuring feature accuracy. Second, we design the adaptively joint distribution alignment subnet, in which the internal multi-source domains and each source-target domain are jointly aligned, meanwhile, the correlation of each source-target domain is embedded in the alignment as the adaptive term, thus the performance of sample distribution alignment can be improved. By doing so, fault samples in other working conditions can be employed to enhance the small samples in current working conditions. Finally, benchmark simulated experiments and actual application experiments are conducted to evaluate the proposed method. All the results demonstrate that our LJDA-Net performs favorably against the state-of-the-art methods.

Information-decision searching algorithm: Theory and applications for solving engineering optimization problems

The nature of the real-world problem is multi-modal and multidimensional. This paper proposes a novel metaheuristic algorithm based on social behaviors of people acquiring favorable information, which is the society-based metaheuristic optimization mechanism, called the Information-Decision Search Algorithm(IDSE), aiming to provide a new optimization technology for solving real-world optimization problems. This optimization technology proposes special searching mechanisms of delivery behavior, approaching behavior, inheritance behavior, mutation behavior, interaction, and learning behavior, establishing corresponding mathematical models to develop an efficient optimization framework for solving constrained optimization. The performance of the proposed algorithm and 10 state-of-the-art optimizers is evaluated on 46 benchmarks, including convergence, solution accuracy, robustness, diversity, significance, and the dimensional-scalability on CEC 2017 benchmarks (50 Dim and 100 Dim). The statistical results suggest, with the dimensionality of the problem variable increasing, the computing efficiency of the proposed optimization technology keeps on the highest level at all times. The low-rank feature for IDSE on 46 benchmarks emphasizes the selective priority in solving the same optimization problem. In addition, IDSE also considers 7 real-world engineering problems. The comparison results suggest that IDSE is superior to competitive algorithms in improving solution accuracy and reducing optimization costs, indicating the significant performance for solving constraint optimization.

Low-Rank Approximation for Multiscale PDEs

Historically, analysis for multiscale PDEs is largely unified while numerical schemes tend to be equation-specific. In this paper, we propose a unified framework for computing multiscale problems through random sampling. This is achieved by incorporating randomized SVD solvers and manifold learning techniques to numerically reconstruct the low-rank features of multiscale PDEs. We use multiscale radiative transfer equation and elliptic equation with rough media to showcase the application of this framework.

SNIFF: A Scalable Network Inference Framework for Measuring End-to-End Performance

For Internet measurement, a key challenge is how to measure end-to-end performance accurately and effectively. To achieve this goal, several recent studies suggest applying advanced matrix completion techniques to recover the complete matrix using a small percentage of samples. While the low-rank features of matrices are generally utilized in existing studies, important features, which are unique in our domain context and can be used to significantly reduce path sampling overhead with guaranteed recovery accuracy, have not been fully explored. To fill the gap, we propose a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">scalable network inference framework</i> , namely, SNIFF, to monitor the end-to-end performance of a set of paths over time. In our framework, we extensively investigate real datasets and identify key features in measurement data, including the low-rank feature of the path-time matrix, the smoothness of performance over a short period, and the pair-wise linear correlation of paths. Based on these features, we develop three non-uniform sampling strategies that increase the sampling rate of important paths while reducing the rate if two paths are highly correlated. Moreover, we formulate two types of optimization problems for matrix completion that take into account important characteristics of data and we develop efficient algorithms to solve the corresponding problems. To evaluate the proposed framework, we conduct extensive numerical experiments for measuring end-to-end delay and throughput performance. The results confirm the feasibility and advantages of the proposed framework and algorithms.

Hyperspectral Image Restoration via Spatial-Spectral Residual Total Variation Regularized Low-Rank Tensor Decomposition

To eliminate the mixed noise in hyperspectral images (HSIs), three-dimensional total variation (3DTV) regularization has been proven as an efficient tool. However, 3DTV regularization is prone to losing image details in restoration. To resolve this issue, we proposed a novel TV, named spatial domain spectral residual total variation (SSRTV). Considering that there is much residual texture information in spectral variation image, SSRTV first calculates the difference between the pixel values of adjacent bands and then calculates a 2DTV for the residual image. Experimental results demonstrated that the SSRTV regularization term is powerful at changing the structures of noises in an original HSI, thus allowing low-rank techniques to get rid of mixed noises more efficiently without treating them as low-rank features. The global low-rankness and spatial–spectral correlation of HSI is exploited by low-rank Tucker decomposition (LRTD). Moreover, it was demonstrated that the l2,1 norm is more effective to deal with sparse noise, especially the sample-specific noise such as stripes or deadlines. The augmented Lagrange multiplier (ALM) algorithm was adopted to solve the proposed model. Finally, experimental results with simulated and real data illustrated the validity of the proposed method. The proposed method outperformed state-of-the-art TV-regularized low-rank matrix/tensor decomposition methods in terms of quantitative metrics and visual inspection.

Identification of Room Acoustic Impulse Responses via Kronecker Product Decompositions

The identification of room acoustic impulse responses represents a challenging problem in the framework of many important applications related to the acoustic environment, like echo cancellation, noise reduction, and microphone arrays, among others. In this context, the main issues are related to the long length of such impulse responses and their time-variant nature. These raise significant difficulties in terms of the convergence rate, computational complexity, and accuracy of the solution. Recently, a decomposition-based approach was developed for the identification of low-rank systems, which can also be applied (to some extent) for the identification of acoustic impulse responses. This approach exploits the nearest Kronecker product decomposition of the impulse response and solves a high-dimension system identification problem using a combination of low-dimension solutions (provided by shorter filters), thus gaining in terms of both performance and complexity. Nevertheless, it does not consider the intrinsic nature of the room acoustic impulse responses, which contain specific components (e.g., early reflections and late reverberation) that can be very different in nature. In this paper, we propose an improved decomposition-based method (via the Kronecker product) that takes into account these specific components and processes them separately, in order to better exploit their important low-rank features. Following this approach, an iterative Wiener filter is firstly developed, followed by a recursive least-squares (RLS) algorithm designed in the same framework. Both solutions outperform the conventional benchmarks, i.e., the conventional Wiener filter and the RLS algorithm, respectively. Moreover, they achieve superior performances as compared to the recently developed versions based on the nearest Kronecker product decomposition, also owning lower computational complexities than their previous counterparts. Simulations are performed in the framework of acoustic echo cancellation and the obtained results support the performance features of the proposed algorithms.

Deep low-rank feature learning and encoding for cross-age face recognition

Cross-age face recognition (CAFR) is a challenging task, due to significant intra-personal variations. Furthermore, the training and testing data may contain random noise components. To address these issues, this paper proposes a deep low-rank feature learning and encoding method. Firstly, our method employs manifold learning in the low-rank optimization, which preserves the global and local structure of the data samples, while learning the clean low-rank features. Secondly, we encode the low-rank features using our locality-constrained feature encoding method, which learns an age-insensitive codebook from training data, and enables the intra-class samples to share the same local bases in a codebook. In the testing stage, the gallery and probe features are encoded by the learned codebook, which represents the images of the same identity by similar codewords for recognition. Furthermore, the periocular region of human faces is investigated for CAFR. Extensive experiments on five datasets demonstrate the effectiveness of our method.

A geometric calibration approach for an industrial cone-beam CT system based on a low-rank phantom

Precise alignment of the system scan geometry is crucial to ensure the reconstructed image quality in a cone-beam computed tomography (CT) system. A calibration method that depends on the local features of a multi-sphere phantom is probably affected by local image variations. In addition, multiple projections with circular scanning are usually required by this type of method to derive misaligned parameters. In contrast to previous works, this paper proposes a method that depends on the global symmetric low-rank feature of a novel phantom, which can accurately represent the system geometrical misalignment. All the misaligned parameters of the cone-beam CT system can be estimated from a single perspective direction without circular scanning. Meanwhile, since the global low-rank feature of the phantom is utilized, the proposed method is robust to noise. We study the performance of the proposed method through simulations and real experiments. The experimental results demonstrate that the proposed method can estimate the system geometric parameters accurately, leading to superior image quality, with the modulation transfer function of our method (0.22) higher than that of the competing method (0.13) at the spatial resolution of 10 lp mm−1.

Clinical drug response prediction from preclinical cancer cell lines by logistic matrix factorization approach.

Predicting tumor drug response using cancer cell line drug response values for a large number of anti-cancer drugs is a significant challenge in personalized medicine. Predicting patient response to drugs from data obtained from preclinical models is made easier by the availability of different knowledge on cell lines and drugs. This paper proposes the TCLMF method, a predictive model for predicting drug response in tumor samples that was trained on preclinical samples and is based on the logistic matrix factorization approach. The TCLMF model is designed based on gene expression profiles, tissue type information, the chemical structure of drugs and drug sensitivity (IC 50) data from cancer cell lines. We use preclinical data from the Genomics of Drug Sensitivity in Cancer dataset (GDSC) to train the proposed drug response model, which we then use to predict drug sensitivity of samples from the Cancer Genome Atlas (TCGA) dataset. The TCLMF approach focuses on identifying successful features of cell lines and drugs in order to calculate the probability of the tumor samples being sensitive to drugs. The closest cell line neighbours for each tumor sample are calculated using a description of similarity between tumor samples and cell lines in this study. The drug response for a new tumor is then calculated by averaging the low-rank features obtained from its neighboring cell lines. We compare the results of the TCLMF model with the results of the previously proposed methods using two databases and two approaches to test the model's performance. In the first approach, 12 drugs with enough known clinical drug response, considered in previous methods, are studied. For 7 drugs out of 12, the TCLMF can significantly distinguish between patients that are resistance to these drugs and the patients that are sensitive to them. These approaches are converted to classification models using a threshold in the second approach, and the results are compared. The results demonstrate that the TCLMF method provides accurate predictions across the results of the other algorithms. Finally, we accurately classify tumor tissue type using the latent vectors obtained from TCLMF's logistic matrix factorization process. These findings demonstrate that the TCLMF approach produces effective latent vectors for tumor samples. The source code of the TCLMF method is available in https://github.com/emdadi/TCLMF.

Data-Driven Modeling of Geometry-Adaptive Steady Heat Convection Based on Convolutional Neural Networks

Heat convection is one of the main mechanisms of heat transfer, and it involves both heat conduction and heat transportation by fluid flow; as a result, it usually requires numerical simulation for solving heat convection problems. Although the derivation of governing equations is not difficult, the solution process can be complicated and usually requires numerical discretization and iteration of differential equations. In this paper, based on neural networks, we developed a data-driven model for an extremely fast prediction of steady-state heat convection of a hot object with an arbitrary complex geometry in a two-dimensional space. According to the governing equations, the steady-state heat convection is dominated by convection and thermal diffusion terms; thus the distribution of the physical fields would exhibit stronger correlations between adjacent points. Therefore, the proposed neural network model uses convolutional neural network (CNN) layers as the encoder and deconvolutional neural network (DCNN) layers as the decoder. Compared with a fully connected (FC) network model, the CNN-based model is good for capturing and reconstructing the spatial relationships of low-rank feature spaces, such as edge intersections, parallelism, and symmetry. Furthermore, we applied the signed distance function (SDF) as the network input for representing the problem geometry, which contains more information compared with a binary image. For displaying the strong learning and generalization ability of the proposed network model, the training dataset only contains hot objects with simple geometries: triangles, quadrilaterals, pentagons, hexagons, and dodecagons, while the testing cases use arbitrary and complex geometries. According to the study, the trained network model can accurately predict the velocity and temperature field of the problems with complex geometries, which has never been seen by the network model during the model training; and the prediction speed is two orders faster than the CFD. The ability of accurate and extremely fast prediction of the network model suggests the potential of applying reduced-order network models to the applications of real-time control and fast optimization in the future.

Locality Sensitive Hash Aggregated Nonlinear Neighborhood Matrix Factorization for Online Sparse Big Data Analysis

Matrix factorization (MF) can extract the low-rank features and integrate the information of the data manifold distribution from high-dimensional data, which can consider the nonlinear neighborhood information. Thus, MF has drawn wide attention for low-rank analysis of sparse big data, e.g., Collaborative Filtering (CF) Recommender Systems, Social Networks, and Quality of Service. However, the following two problems exist: (1) huge computational overhead for the construction of the Graph Similarity Matrix (GSM) and (2) huge memory overhead for the intermediate GSM. Therefore, GSM-based MF, e.g., kernel MF, graph regularized MF, and so on, cannot be directly applied to the low-rank analysis of sparse big data on cloud and edge platforms. To solve this intractable problem for sparse big data analysis, we propose Locality Sensitive Hashing (LSH) aggregated MF (LSH-MF), which can solve the following problems: (1) The proposed probabilistic projection strategy of LSH-MF can avoid the construction of the GSM. Furthermore, LSH-MF can satisfy the requirement for the accurate projection of sparse big data. (2) To run LSH-MF for fine-grained parallelization and online learning on GPUs, we also propose CULSH-MF, which works on CUDA parallelization. Experimental results show that CULSH-MF can not only reduce the computational time and memory overhead but also obtain higher accuracy. Compared with deep learning models, CULSH-MF can not only save training time but also achieve the same accuracy performance.

Alzheimer Disease Diagnosis from fMRI Images Based on Latent Low Rank Features and Support Vector Machine (SVM)

Introduction: In recent years, resting-state functional magnetic resonance imaging (rsfMRI) has been increasingly used as a noninvasive and practical method in different areas of neuroscience and psychology for recognizing brain’s mechanism as well as diagnosing neurological diseases. In this work, we use rs-fMRI data for diagnosing Alzheimer's disease. Materials and Methods: To do that, by using the rs-fMRI of a patient, we computed the time series of some anatomical regions and then applied the Latent Low Rank Representation method to extract suitable features. Next, based on the extracted features, we apply a Support Vector Machine (SVM) classifier to determine whether the patient belongs to a healthy category, mild stage of the disease or Alzheimer's stage. Results: The obtained classification accuracy for the proposed method is more than 97.5%. Conclusion: We performed different experiments on a database of rs-fMRI data containing the images of 43 healthy subjects, 36 mild cognitive impairment patients and 32 Alzheimer’s patients and the obtained results demonstrated that the best performance is achieved when the SVM with Gaussian kernel and the features of only 7 regions were used.

False Data Injection Attacks Detection in Smart Grid: A Structural Sparse Matrix Separation Method

Smart grid has become the trend of future power system owing to its efficient allocation of resources, real-time monitoring and decision-making ability. However, due to the deep integration of power physical system and information system, smart grid is under serious threat such as malicious attacks and so on. The false data injection attack (FDIA), which can circumvent the traditional bad data detection technology, is a practical significant yet challenging problem for the operation of smart grid. In this paper, a novel attack detection method based on the matrix separation theory was proposed, which can detect the FDIA efficiently by exploring the low-rank feature of the unattacked measurement matrix and the structural sparsity feature of the attack matrix. Furthermore, a structural sparse matrix separation algorithm is proposed to improve the attack identification accuracy. In order to verify the effectiveness of the proposed method, we carried out experiments under three types of attack. Compared with the existing methods, the proposed method has significantly improved the performance of FDIA detection. At the same time, it also has good robustness for noise and good scalability in large networks.

Learnable low-rank latent dictionary for subspace clustering

Recently, Self-Expressive-based Subspace Clustering (SESC) has been widely applied in pattern clustering and machine learning as it aims to learn a representation that can faithfully reflect the correlation between data points. However, most existing SESC methods directly use the original data as the dictionary, which miss the intrinsic structure (e.g., low-rank and nonlinear) of the real-word data. To address this problem, we propose a novel Projection Low-Rank Subspace Clustering (PLRSC) method by integrating feature extraction and subspace clustering into a unified framework. In particular, PLRSC learns a projection transformation to extract the low-dimensional features and utilizes a low-rank regularizer to ensure the informative and important structures of the extracted features. The extracted low-rank features effectively enhance the self-expressive property of the dictionary. Furthermore, we extend PLRSC to a nonlinear version (i.e., NPLRSC) by integrating a nonlinear activator into the projection transformation. NPLRSC cannot only effectively extract features but also guarantee the data structure of the extracted features. The corresponding optimization problem is solved by the Alternating Direction Method (ADM), and we also prove that the algorithm converges to a stationary point. Experimental results on the real-world datasets validate the superior of our model over the existing subspace clustering methods.

Robust Image Representation via Low Rank Locality Preserving Projection

Locality preserving projection (LPP) is a dimensionality reduction algorithm preserving the neighhorhood graph structure of data. However, the conventional LPP is sensitive to outliers existing in data. This article proposes a novel low-rank LPP model called LR-LPP. In this new model, original data are decomposed into the clean intrinsic component and noise component. Then the projective matrix is learned based on the clean intrinsic component which is encoded in low-rank features. The noise component is constrained by the ℓ 1 -norm which is more robust to outliers. Finally, LR-LPP model is extended to LR-FLPP in which low-dimensional feature is measured by F-norm. LR-FLPP will reduce aggregated error and weaken the effect of outliers, which will make the proposed LR-FLPP even more robust for outliers. The experimental results on public image databases demonstrate the effectiveness of the proposed LR-LPP and LR-FLPP.

A New Noisy Random Forest Based Method for Feature Selection

Abstract Feature selection is an essential pre-processing step in data mining. It aims at identifying the highly predictive feature subset out of a large set of candidate features. Several approaches for feature selection have been proposed in the literature. Random Forests (RF) are among the most used machine learning algorithms not just for their excellent prediction accuracy but also for their ability to select informative variables with their associated variable importance measures. Sometimes RF model over-fits on noisy features, which lead to choosing the noisy features as the informative variables and eliminating the significant ones. Whereas, eliminating and preventing those noisy features first, the low ranked features may become more important. In this study we propose a new variant of RF that provides unbiased variable selection where a noisy feature trick is used to address this problem. First, we add a noisy feature to a dataset. Second, the noisy feature is used as a stopping criterion. If the noisy feature is selected as the best splitting feature, then we stop the creation process because at this level, the model starts to over-fit on the noisy features. Finally, the best subset of features is selected out of the best-ranked feature regarding the Geni impurity of this new variant of RF. To test the validity and the effectiveness of the proposed method, we compare it with RF variable importance measure using eleven benchmarking datasets.