Matrix Completion Algorithm Research Articles

Maritime piracy risk assessment and policy implications: A two-step approach

Maritime piracy is a serious threat to the safety of cargo, ships, and crews, which can cause enormous losses to stakeholders, highlighting the significance of piracy risk assessment and prevention to the maritime industry. In this study, we propose a two-step analytical framework based on a Random Forest (RF) model, Generative Adversarial Nets (GANs), and Matrix Completion (MC) algorithm to assess the risks of successful piracy attacks. We consider different decision-makers in each step, namely, pirates first select which ships to attack and then ship operators determine the probability of a (un)successful attack. We propose different influencing factors for each step and, in the meantime, solving the problems of incomplete and imbalanced data. A case study in Southeast Asia is then conducted based on the proposed approach. The results show that, in mild wind weather, the likelihood of a piracy attack on a ship with a DWT not exceeding 50,000 tons is up to 90 %. Further, if the attack occurs between 0 and 6 a.m., the probability of success is over 90 %. These results provide more specific information for ship operators and local authorities to develop efficient anti-piracy strategies and policies.

Highly accurate millimeter wave channel estimation in massive MIMO system

AbstractAccurate channel state information (CSI) is extremely crucial to realize high‐accurate hybrid precoding, in millimeter wave communication systems. In order to improve the CSI estimation performance, several traditional channel estimators have been developed by exploiting the sparse or low‐rank property, whilst they bring huge channel training overhead and computational complexity. In this work, based on jointly sparse and low‐rank property of massive multiple input multiple output (MIMO) channel, one non‐convex mmWave channel estimation problem is formulated and a novel scheme to acquire one accurate CSI estimation result with greatly reduced training overhead is proposed. Specifically, the non‐convex problem is reformulated as two simple sub‐problems, by exploiting the alternating direction method of multipliers (ADMM) technique. Based on the low‐rank characteristic, one fast gradient descent matrix completion algorithm is developed to accurately solve the first sub‐problem. On this basis, the compressed sensing (CS) technique to acquire the accurate CSI estimation matrix is further utilized. Numerical simulation validates that the method could achieve the much higher channel estimation accuracy, yet only incurs the lower overhead compared with the traditional scheme.

A WTC suppression algorithm for weather radar based on weighted schatten p-norm minimization

ABSTRACT The Matrix Completion (MC) method can effectively suppress the wind turbine clutter (WTC) of the weather radar, however its recovery accuracy suffers from the low signal-to-noise ratio (SNR) of the real measured weather signal. Moreover, the singular value distribution of the observation matrix constructed by the measured radar data has no sharp cut-off characteristic, which results the rank can not be well approximated and also reduces the recovery accuracy. In this paper, a non-convex MC algorithm based on weighted Schatten p-norm is proposed for the WTC suppression in Doppler frequency domain. Experiments are carried out on the measured signal data, and the results demonstrate that compared with the commonly used nuclear norm minimization (NNM) model, the weighted Schatten p-norm minimization (WSNM) model can further improve the recovery accuracy of weather signal and realizes the superior estimation performance of spectral moment.

Identifying circRNA-miRNA interaction based on multi-biological interaction fusion.

CircRNA is a new type of non-coding RNA with a closed loop structure. More and more biological experiments show that circRNA plays important roles in many diseases by regulating the target genes of miRNA. Therefore, correct identification of the potential interaction between circRNA and miRNA not only helps to understand the mechanism of the disease, but also contributes to the diagnosis, treatment, and prognosis of the disease. In this study, we propose a model (IIMCCMA) by using network embedding and matrix completion to predict the potential interaction of circRNA-miRNA. Firstly, the corresponding adjacency matrix is constructed based on the experimentally verified circRNA-miRNA interaction, circRNA-cancer interaction, and miRNA-cancer interaction. Then, the Gaussian kernel function and the cosine function are used to calculate the circRNA Gaussian interaction profile kernel similarity, circRNA functional similarity, miRNA Gaussian interaction profile kernel similarity, and miRNA functional similarity. In order to reduce the influence of noise and redundant information in known interactions, this model uses network embedding to extract the potential feature vectors of circRNA and miRNA, respectively. Finally, an improved inductive matrix completion algorithm based on the feature vectors of circRNA and miRNA is used to identify potential interactions between circRNAs and miRNAs. The 10-fold cross-validation experiment is utilized to prove the predictive ability of the IIMCCMA. The experimental results show that the AUC value and AUPR value of the IIMCCMA model are higher than other state-of-the-art algorithms. In addition, case studies show that the IIMCCMA model can correctly identify the potential interactions between circRNAs and miRNAs.

A two-phase rank-based algorithm for low-rank matrix completion

Matrix completion aims to recover an unknown low-rank matrix from a small subset of its entries. In many applications, the rank of the unknown target matrix is known in advance and this information can be useful in the completion process. In this paper, first, we revisit a recently proposed rank-based heuristic for “known-rank” matrix completion and establish a condition under which the generated sequence is quasi-Fejér convergent to the solution set. Then, by including an acceleration mechanism similar to Nesterov’s acceleration, we obtain a new heuristic. Even though the convergence of this new heuristic cannot be granted in general, it turns out that it can be very useful as a warm-start phase (phase one), providing a suitable estimate for the regularization parameter and a good starting point to an accelerated proximal gradient algorithm (phase two) aimed to solve a nuclear-norm regularized problem. Numerical experiments with both synthetic and real data show that the resulting two-phase rank-based algorithm can recover low-rank matrices, with relatively high precision, faster than other well-established matrix completion algorithms.

A reweighted matrix completion algorithm for sparse inverse synthetic aperture radar imaging

Sparse inverse synthetic aperture radar (ISAR) imaging can be generally achieved by compressed sensing (CS) methods because sparse sampling disables the approach of conventional range Doppler. However, the CS-based methods have to convert the matrix into a vector when the random sampling is adopted, which results in huge memory usage and high computational complexity. Note that matrix completion (MC) is suitable for matrix operations, which can recover random sparse data directly based on the low-rank property of the echo matrix. In order to improve the performance of the conventional MC approaches, a novel reweighted matrix completion method for sparse ISAR imaging is proposed in this paper. To avoid using the same singular value thresholding of the traditional MC method to achieve a low-rank solution, the reweighted scheme for singular values is applied to restrain the rank. Furthermore, the weights of the current iteration are updated with their singular values obtained in the former iteration, rather than using a fixed value. Then, the alternating direction method of multipliers is utilised to alternatively optimise the proposed model, which has an improved computational efficiency. Once the full data is recovered, the ISAR image can be achieved via the conventional 2D inverse fast Fourier transform. Experimental results based on measured data validate the proposed approach that can result in improving imaging performance within several seconds, which is tens of times faster than some reported low-rank-based methods.

Robust Matrix Completion With Column Outliers.

Matrix completion, in essence, involves recovering a low-rank matrix from a subset of its entries. Most existing methods for matrix completion neglect two significant issues. First, in several practical applications, such as collaborative filtering, some samples may be corrupted completely. However, most of the robust algorithms consider only the condition that a few components of each column have been arbitrarily contaminated. Second, many real data are not static in nature. Nevertheless, the conventional batch-based matrix completion methods cannot efficiently deal with the out-of-sample, that is, the vector completion problem. In this article, we first provide a novel robust matrix completion model and then develop an efficient optimization method that only requires conducting one time singular value decomposition for a thin matrix per iteration. Furthermore, by exploiting the essence of online matrix completion algorithms, we develop a vector completion model which can help users predict the missing values of out of sample. Numerical comparisons with traditional batch-based and online matrix completion algorithms demonstrate the benefits of the proposed method on streaming data corrupted by column outliers. Moreover, we show that our model can be used to detect outliers from incomplete information. This advantage is validated via numerous experimental results on synthetic and real-world data.

Matrix completion methods for the total electron content video reconstruction

The total electron content (TEC) maps can be used to estimate the signal delay of GPS due to the ionospheric electron content between a receiver and satellite. This delay can result in GPS positioning error. Thus, it is important to monitor the TEC maps. The observed TEC maps have big patches of missingness in the ocean and scattered small areas of missingness on the land. In this paper we propose several extensions of existing matrix completion algorithms to achieve TEC map reconstruction, accounting for spatial smoothness and temporal consistency while preserving important structures of the TEC maps. We call the proposed method video imputation with softImpute, temporal smoothing and auxiliary data (VISTA). Numerical simulations that mimic patterns of real data are given. We show that our proposed method achieves better reconstructed TEC maps, as compared to existing methods in literature. Our proposed computational algorithm is general and can be readily applied for other problems besides TEC map reconstruction.

A novel initialization method of fixed point continuation for recommendation systems

In recent years, the problem of matrix completion based on rank minimization has received widespread attention in machine learning. The tightest convex relaxation of this problem is the linearly constrained nuclear norm minimization. Fixed point continuation (FPC), as a representative nuclear norm relaxation matrix completion algorithm, has been proven to perform well in theories and experiments. However, the traditional FPC algorithm initializes the matrix to be completed by zero, which does not make full use of the shrinkage characteristics of the singular value shrinkage operator on the matrix elements and the known field data information, then will lead to slow convergence and poor accuracy. Aiming at this problem, this paper analyzes the shrinkage properties of matrix elements in the iterative process of the FPC algorithm. Combined with the known rating information in the recommendation systems, a new initialization method of overestimation based on FPC is proposed, and it is applied to the rating prediction in the recommendation systems. The experimental results show that the initialization method proposed in this paper greatly improves the algorithm efficiency and prediction accuracy.

Toward Efficient Direct Dynamics Studies of Chemical Reactions: A Novel Matrix Completion Algorithm.

This paper describes the development and testing of a polynomial variety-based matrix completion (PVMC) algorithm. Our goal is to reduce computational effort associated with reaction rate coefficient calculations using variational transition state theory with multidimensional tunneling (VTST-MT). The algorithm recovers eigenvalues of quantum mechanical Hessians constituting the minimum energy path (MEP) of a reaction using only a small sample of the information, by leveraging underlying properties of these eigenvalues. In addition to the low-rank property that constitutes the basis for most matrix completion (MC) algorithms, this work introduces a polynomial constraint in the objective function. This enables us to sample matrix columns unlike most conventional MC methods that can only sample elements, which makes PVMC readily compatible with quantum chemistry calculations as sampling a single column requires one Hessian calculation. For various types of reactions─SN2, hydrogen atom transfer, metal-ligand cooperative catalysis, and enzyme chemistry─we demonstrate that PVMC on average requires only six to seven Hessian calculations to accurately predict both quantum and variational effects.

SEMCM: A Self-Expressive Matrix Completion Model for Anti-cancer Drug Sensitivity Prediction

Background: Genomic data sets generated by several recent large scale high-throughput screening efforts pose a complex computational challenge for anticancer drug sensitivity prediction. Objective: We aimed to design an algorithm model that would predict missing elements in incomplete matrices and could be applicable to drug response prediction programs. Method: We developed a novel self-expressive matrix completion model to improve the predictive performance of drug response prediction problems. The model is based on the idea of subspace clustering and as a convex problem, it can be solved by alternating direction method of multipliers. The original incomplete matrix can be filled through model training and parameters updated iteratively. Results: We applied SEMCM to Genomics of Drug Sensitivity in Cancer (GDSC) and Cancer Cell Line Encyclopedia (CCLE) datasets to predict unknown response values. A large number of experiments have proved that the algorithm has good prediction results and stability, which are better than several existing advanced drug sensitivity prediction and matrix completion algorithms. Without modeling mutation information, SEMCM could correctly predict cell line-drug associations for mutated cell lines and wild cell lines. SEMCM can also be used for drug repositioning. The newly predicted drug responses of GDSC dataset suggest that TI-73 was sensitive to Erlotinib. Moreover, the sensitivity of A172 and NCIH1437 to Paclitaxel was roughly the same. Conclusion: We report an efficient anticancer drug sensitivity prediction algorithm which is opensource and can predict the unknown responses of cancer cell lines to drugs. Experimental results prove that our method can not only improve the prediction accuracy but also can be applied to drug repositioning.

2D-DOA Estimation in Switching UCA Using Deep Learning-Based Covariance Matrix Completion.

In this paper, we study the two-dimensional direction of arrival (2D-DOA) estimation problem in a switching uniform circular array (SUCA), which means performing 2D-DOA estimation with a reduction in the number of radio frequency (RF) chains. We propose a covariance matrix completion algorithm for 2D-DOA estimation in a SUCA. The proposed algorithm estimates the complete covariance matrix of a fully sampled UCA (FUCA) from the sample covariance matrix of the SUCA through a neural network. Afterwards, the MUSIC algorithm is performed for 2D-DOA estimation with the completed covariance matrix. We conduct Monte Carlo simulations to evaluate the performance of the proposed algorithm in various scenarios; the performance of 2D-DOA estimation in the SUCA gradually approaches that in the FUCA as the SNR or the number of snapshots increases, which means that the advantages of a FUCA can be preserved with fewer RF chains. In addition, the proposed algorithm is able to implement underdetermined 2D-DOA estimation.

A matrix completion algorithm for efficient calculation of quantum and variational effects in chemical reactions.

This work examines the viability of matrix completion methods as cost-effective alternatives to full nuclear Hessians for calculating quantum and variational effects in chemical reactions. The harmonic variety-based matrix completion (HVMC) algorithm, developed in a previous study [S. J. Quiton et al., J. Chem. Phys. 153, 054122 (2020)], exploits the low-rank character of the polynomial expansion of potential energy to recover vibrational frequencies (square roots of eigenvalues of nuclear Hessians) constituting the reaction path using a small sample of its entities. These frequencies are essential for calculating rate coefficients using variational transition state theory with multidimensional tunneling (VTST-MT). HVMC performance is examined for four SN2 reactions and five hydrogen transfer reactions, with each H-transfer reaction consisting of at least one vibrational mode strongly coupled to the reaction coordinate. HVMC is robust and captures zero-point energies, vibrational free energies, zero-curvature tunneling, and adiabatic ground state and free energy barriers as well as their positions on the reaction coordinate. For medium to large reactions involving H-transfer, with the sole exception of the most complex Ir catalysis system, less than 35% of total eigenvalue information is necessary for accurate recovery of key VTST-MT observables.

Fast and robust rank-one matrix completion via maximum correntropy criterion and half-quadratic optimization

Matrix completion refers to seeking a low-rank matrix to match an incomplete matrix and fill its missing entries, and is an important topic because many real-world data can be modeled as low-rank matrices. Most existing schemes rely on ℓ2-norm minimization and hence are not robust to outliers. Even the ℓ1-norm based robust matrix completion algorithms proposed in the literature have poor recovery performance for large gross errors, and require knowing the matrix rank, which is difficult to accurately determine in practice. In this paper, we devise a robust and fast rank-one matrix completion algorithm via combining the maximum correntropy criterion (MCC) and half-quadratic (HQ) optimization theory. The MCC, i.e., minimizing the Welsch cost function, can resist the gross errors but it is non-convex. While HQ optimization can transform the Welsch cost function into a quadratic form, making the resultant optimization problem easy to solve. Furthermore, an adaptive kernel width strategy is derived and there are no tunable parameters except for the termination conditions in our algorithm. Computer simulations using synthetic data and experimental results of real-world images demonstrate that the developed method achieves accurate recovery performance and high computational efficiency.

Low-rank traffic matrix completion with marginal information

Accurate spatio-temporal traffic data is crucial to intelligent transportation systems. Missing traffic data is an important problem to solve. Low-rank matrix completion provides an effective way to find the missing data. The completion aims to obtain a low-rank matrix that can approximate the known entries as far as possible. Meanwhile, some linear constraint marginal information of the matrix can also be observed in the real application. In this paper, we utilize such marginal information to largely improve the performance of common matrix completion algorithms and propose an alternating direction method of multipliers (ADMM) and conjugate gradient descent method (CGD) based SoftImpute alternative least square (ALS) algorithm. We analyze their convergence rates and prove that the model can always converge to a first-order stationary point. We also utilize ADMM and CGD to largely accelerate the subproblem and make its complexity of each iteration at the same level as the popular SoftImpute-ALS matrix completion algorithm. Furthermore, this algorithm can be used in distributed computation, suitable for large-scale problems. In the numerical experiments, we demonstrate its outstanding matrix completion performance and high speed in several traffic matrix datasets.

Sound source localization of non-synchronous measurements beamforming based on the truncated nuclear norm regularization

Due to the size and aperture limitations of the microphone array, poor resolution and limited working frequency range are challenging for acoustic imaging with beamforming methods. In this work, two fast iteration algorithms, alternating direction method of multipliers (ADMM) and accelerated proximal gradient line search method (APGL) based on truncated nuclear norm regularization (TNNR), are proposed to complete the cross-spectral matrix for non-synchronous measurements beamforming. Compared with current matrix completion algorithm based on nuclear norm minimization (NNM), TNNR is regarded as a better approximation to the rank function, which means it can approach the real solution better and restore the sound sources accurately at a low signal-to-noise ratio (SNR) environment. Among the TNNR-APGL and TNNR-ADMM, the TNNR-APGL algorithm has a lower matrix completion error and a shorter calculation time. The localization results of TNNR-ADMM method are more robust to the number of measurements. The experiment results show that TNNR methods greatly improve the resolution of conventional beamforming (CBF) and suppress the sidelobes under low SNR conditions, which indicates that the non-synchronous measurements beamforming based on TNNR methods has the potential to be applied in the industrial scene and low SNR environment.

Spatiotemporal Urban Inference and Prediction in Sparse Mobile CrowdSensing: a Graph Neural Network Approach

Mobile CrowdSensing (MCS) has recently become a promising data acquisition paradigm, which recruits a large number of users to collect data from the target sensing areas. Obviously, with the increase of sensing scale and the decrease of sensing granularity, traditional MCS cannot fully cover the required sensing areas especially the inaccessible areas. As a variant, Sparse MCS can utilize the spatiotemporal correlations in sensing data to infer the whole sensing map only by sensing a few subareas. However, in many real-world scenarios, such as traffic congestion prediction or parking occupancy detection, inferring the current unsensed data may not be the final goal. By comparison, it is more important to get the future information through the sparse sensed data. In this paper, we turn attention from inferring the current unsensed data to predicting the future unknown data and propose an urban inference and prediction framework in Sparse MCS. To deal with the sparse sensed data, we first present a bipartite-graph-based matrix completion algorithm with spatiotemporal constraints to accurately recover the current full map. Then, by exploiting spatiotemporal correlations based on the inferred full map, we present a Graph Convolutional Networks (GCN) with spatiotemporal attention to predict the future maps. Furthermore, we design a spatiotemporal iterative method to repeatedly update the spatiotemporal attentions and constraints, in order to connect the urban inference and prediction to improve the accuracy of the whole framework. Extensive experiments have been conducted on two types of typical urban sensing tasks with four real-world data sets, which verify the effectiveness of our proposed algorithms in improving the inference and prediction accuracy with the sparse sensed data.

Mc2g: An Efficient Algorithm for Matrix Completion With Social and Item Similarity Graphs

In this paper, we design and analyze MC2G (Matrix Completion with 2 Graphs), an algorithm that performs matrix completion in the presence of social and item similarity graphs. MC2G runs in quasilinear time and is parameter free. It is based on spectral clustering and local refinement steps. For the matrix completion problem which possesses additional block structures in its rows and columns, we derive the expected number of sampled entries required for MC2G to succeed, and further show that it matches an information-theoretic lower bound up to a constant factor for a wide range of parameters. We perform extensive experiments on both synthetic datasets and a semi-real dataset inspired by real graphs. The experimental results show that MC2G outperforms other state-of-the-art matrix completion algorithms.

Illinois-Type Methods for Noisy Euclidean Distance Realization

In this work, we introduce an iterative algorithm for the Euclidean distance matrix completion (EDMC) problem with noisy and incomplete distance measurements. The proposed method is based on semidefinite programming, utilizes a Pareto iterative approach, and performs a projection-free convex optimization over the spectrahedron to solve a level-set problem relevant to EDMC problems. The optimality trade-off between the trace of a positive semidefinite matrix and a loss function is pursued over Pareto optimal points with simple, derivative-free, costly efficient nonlinear equation root finding iterations called Illinois-type methods. We evaluate our approach numerically in a scenario where distance measurements are affected by multiplicative noise.

DLSLA 3-D SAR Imaging via Sparse Recovery Through Combination of Nuclear Norm and Low-Rank Matrix Factorization

Downward-looking sparse linear array 3-D synthetic aperture radar (DLSLA 3-D SAR) cross-track dimensional imaging always suffers from incomplete observation which does not satisfy the Nyquist sampling theorem and leads to the failure of conventional 3-D frequency-domain methods. Although several sparse reconstruction-based methods have been presented to solve this problem, the basis mismatch issue in sparse reconstruction theory will degrade the image reconstruction performance. To address this issue, this article proposes a novel 3-D imaging method for DLSLA 3-D SAR, which provides another idea for 3-D imaging through sparse recovery. It utilizes recovered full-sampled data to achieve cross-track dimensional imaging instead of using the under-sampled data directly as before. The Along-track-Height plane imaging is first finished by the range-Doppler (RD) algorithm and motion error compensation. Then, an advanced nuclear norm and low-rank matrix factorization (NU-LRMF)-based matrix completion (MC) algorithm and a vector reconstruction framework are built to achieve accurate recovery of full-sampled data. Finally, the cross-track dimensional imaging is completed with recovered full-sampled data by geometric correction and beamforming. Moreover, a fast two-stage iteration strategy for NU-LRMF (TS-NU-LRMF) is also presented to accelerate convergence. The robustness and effectiveness of the proposed 3-D imaging method are verified by several numerical simulations and comparative studies based on both the complex 3-D ship model and the simulated 3-D distributed scenario.