Alternating Least Squares Algorithm Research Articles

A comparative study of various recommendation algorithms based on E-commerce big data

With the rapid development of the Internet and the concomitant exponential growth of information, we have entered an era characterized by information overload. The abundance of data has rendered it increasingly arduous for users to pinpoint specific information they require. However, various forms of recommendation algorithms proffer solutions to this challenge. These algorithms predict items or products that may pique users’ interest based on their historical behavior, preferences, and interests. As one of the current hot research fields, recommendation algorithms are extensively employed across E-commerce platforms, movie streaming services, and various other contexts to cater to the diverse needs of users. In this context, a multi-recommendation algorithms comparison platform is proposed, which includes a two-fold model: online evaluation and offline evaluation. Taking the data set of the Chinese Amazon online shopping mall as the experimental data, item-based collaborative filtering (Item-CF) algorithm, content-based (TF-IDF) algorithm, item2vec model, alternating least squares (ALS) algorithm and neural network algorithm are evaluated in the offline model. In the real-time recommendation part, model-based algorithm is used to achieve the users’ rating mechanism. And the metrics used for evaluation include: precision, recall, accuracy and performance. The experimental results show that the average performance of hybrid algorithms such as ALS algorithm and neural network algorithm is higher than that of other traditional algorithms, and the real-time recommendation system achieves the purpose of improving recommendation speed. By integrating various recommender algorithms into the multi-recommendation algorithms comparison platform, this platform automatically computes and presents various performance indicators based on the user-provided dataset. It aids E-commerce platforms in making informed decisions regarding algorithm selection.

A Clustering Model for Three-Way Asymmetric Proximities: Unveiling Origins and Destinations

In many real-world situations, the available data consist of a set of several asymmetric pairwise proximity matrices that collect directed exchanges between pairs of objects measured or observed in a number of occasions (three-way data). To unveil patterns of exchange, a clustering model is proposed that accounts for the systematic differences across occasions. Specifically, the goal is to identify the groups of objects that are primarily origins or destinations of the directed exchanges, and, together, to measure the extent to which these clusters differ across occasions. The model is based on two clustering structures for the objects, which are linked one-to-one and common to all occasions. The first structure assumes a standard partition of the objects to fit the average amounts of the exchanges, while the second one fits the imbalances using an “incomplete” partition of the objects, allowing some to remain unassigned. In addition, to account for the heterogeneity of the occasions, the amounts and directions of exchange between clusters are modeled by occasion-specific weights. An Alternating Least-Squares algorithm is provided. Results from artificial data and a real application on international student mobility show the capability of the model to identify origin and/or destination clusters with common behavior across occasions.

Blockwise acceleration of alternating least squares for canonical tensor decomposition

AbstractThe canonical polyadic (CP) decomposition of tensors is one of the most important tensor decompositions. While the well‐known alternating least squares (ALS) algorithm is often considered the workhorse algorithm for computing the CP decomposition, it is known to suffer from slow convergence in many cases and various algorithms have been proposed to accelerate it. In this article, we propose a new accelerated ALS algorithm that accelerates ALS in a blockwise manner using a simple momentum‐based extrapolation technique and a random perturbation technique. Specifically, our algorithm updates one factor matrix (i.e., block) at a time, as in ALS, with each update consisting of a minimization step that directly reduces the reconstruction error, an extrapolation step that moves the factor matrix along the previous update direction, and a random perturbation step for breaking convergence bottlenecks. Our extrapolation strategy takes a simpler form than the state‐of‐the‐art extrapolation strategies and is easier to implement. Our algorithm has negligible computational overheads relative to ALS and is simple to apply. Empirically, our proposed algorithm shows strong performance as compared to the state‐of‐the‐art acceleration techniques on both simulated and real tensors.

Algorithm 1026: Concurrent Alternating Least Squares for Multiple Simultaneous Canonical Polyadic Decompositions

Tensor decompositions, such as CANDECOMP/PARAFAC (CP), are widely used in a variety of applications, such as chemometrics, signal processing, and machine learning. A broadly used method for computing such decompositions relies on the Alternating Least Squares (ALS) algorithm. When the number of components is small, regardless of its implementation, ALS exhibits low arithmetic intensity, which severely hinders its performance and makes GPU offloading ineffective. We observe that, in practice, experts often have to compute multiple decompositions of the same tensor, each with a small number of components (typically fewer than 20), to ultimately find the best ones to use for the application at hand. In this article, we illustrate how multiple decompositions of the same tensor can be fused together at the algorithmic level to increase the arithmetic intensity. Therefore, it becomes possible to make efficient use of GPUs for further speedups; at the same time, the technique is compatible with many enhancements typically used in ALS, such as line search, extrapolation, and non-negativity constraints. We introduce the Concurrent ALS algorithm and library, which offers an interface to MATLAB, and a mechanism to effectively deal with the issue that decompositions complete at different times. Experimental results on artificial and real datasets demonstrate a shorter time to completion due to increased arithmetic intensity.

An alternating least-squares algorithm for approximate joint diagonalization of Hermitian matrices

Approximate joint diagonalization (AJD) of a set of target matrices is one of the approaches for blind identification, blind source separation (BSS), and blind source extraction. This approach is useful for applications with measurement noise and/or estimation errors. This paper presents an alternating least-squares (ALS) algorithm for solving AJD problem. The ALS algorithm consists of two processing phases. The first processing phase works for finding diagonal matrices by minimizing the indirect least-squares criterion. A mixing matrix is estimated by minimizing the constrained direct least-squares criterion in the second processing phase. Our ALS algorithm can circumvent numerically unstable behavior by improving the condition number of the estimated mixing matrix in the second processing phase. The proposed algorithm is shown through experimental results to have better estimation performance and faster convergence than existing ALS algorithms with lower computational requirement in the rectangular mixing case.

Improved ALS Recommendation Algorithm Based on Interest-Value Model

Abstract In the era of big data, all kinds of information around, how to effectively get the information others want and achieve recommendation is the original purpose of recommendation algorithm, traditional recommendation has collaborative filtering, content-based, model-based recommendation algorithm. Matrix factorization recommendation algorithm solves the problems of data sparsity and low space utilization brought by dimensional explosion. ALS(Alternating Least Squares) algorithm is one of the matrix factorization and the only one recommendation algorithm implemented in Spark, which is in line with the computation method of big data distributed computing. However, in the traditional ALS, there are still some shortcomings: the determination of the number of Implicit features does not give a clear definition, the information factor is lost after matrix factorization, which leads to the poor interpretability of the decomposed matrix, and the performance parameters of the ALS model are not optimal in terms of performance. Based on the above shortcomings, the IVM-ALS(ALS Based on Interest-Value Model) based on interest-value is proposed, an interest-value model is introduced, a solution to solve the Implicit feature number is introduced, and the feasibility of the feature matrix of the subdivided item is enhanced, so as to meet the actual requirements., And improve the algorithm performance than traditional ALS.

Drug–disease associations prediction via Multiple Kernel-based Dual Graph Regularized Least Squares

Predicting associations in drug–disease network provides effective information for the drug repositioning. Therefore, it is an important task to develop an effective drug–disease association prediction method. In this paper, we propose a model, called the Multiple Kernel-based Dual Graph Regularized Least Squares Model (MKDGRLS), to predict potential drug–disease associations. First, we calculate the multiple kernels of drug and disease spaces respectively. Moreover, we utilize multiple kernels and related Laplacian regularization terms to construct MKDGRLS. Finally, we solve the object function of MKDGRLS by the Alternating Least squares algorithm (ALSA), for predicting drug–disease associations. Simultaneously, we design a variant MKDGRLS-A, which is added one dimension as the bias for compensate the inexact solution of linear systems. Our proposed approach has better prediction performance than existing prediction tools under two types of cross validation on the three real drug–disease association network datasets. The results of the case studies of the two diseases prove that our model can effectively predict new associations. We also test MKDGRLS on six real-world networks and it achieves better performance than other methods. In conclusion, our research work can effectively discover potential drug–disease associations, and can provide effective help for related drug repositioning work.

Identifying potential association on gene-disease network via dual hypergraph regularized least squares

BackgroundIdentifying potential associations between genes and diseases via biomedical experiments must be the time-consuming and expensive research works. The computational technologies based on machine learning models have been widely utilized to explore genetic information related to complex diseases. Importantly, the gene-disease association detection can be defined as the link prediction problem in bipartite network. However, many existing methods do not utilize multiple sources of biological information; Additionally, they do not extract higher-order relationships among genes and diseases.ResultsIn this study, we propose a novel method called Dual Hypergraph Regularized Least Squares (DHRLS) with Centered Kernel Alignment-based Multiple Kernel Learning (CKA-MKL), in order to detect all potential gene-disease associations. First, we construct multiple kernels based on various biological data sources in gene and disease spaces respectively. After that, we use CAK-MKL to obtain the optimal kernels in the two spaces respectively. To specific, hypergraph can be employed to establish higher-order relationships. Finally, our DHRLS model is solved by the Alternating Least squares algorithm (ALSA), for predicting gene-disease associations.ConclusionComparing with many outstanding prediction tools, DHRLS achieves best performance on gene-disease associations network under two types of cross validation. To verify robustness, our proposed approach has excellent prediction performance on six real-world networks. Our research work can effectively discover potential disease-associated genes and provide guidance for the follow-up verification methods of complex diseases.

Identification of Drug–Target Interactions via Dual Laplacian Regularized Least Squares with Multiple Kernel Fusion

Detection of Drug–Target Interactions (DTIs) is the time-consuming and laborious experiment via biochemical approaches. Machine learning based methods have been widely used to mine meaningful information of drug research. In this study, we establish a novel computational method to predict DTIs via Dual Laplacian Regularized Least Squares model (DLapRLS) with Hilbert–Schmidt Independence Criterion-based Multiple Kernel Learning (HSIC-MKL). Multiple kernels are built from different information sources (drug and target spaces). Then, above corresponding kernels are integrated by HSIC-MKL. At last, DLapRLS model is trained by Alternating Least Squares Algorithm (ALSA) and employed to predict new DTIs. On four benchmark datasets, the results of our method are comparable and even better than existing models.

Using elastic net restricted kernel canonical correlation analysis for cross-language information retrieval

Kernel methods, which are a non-linear variant of linear methods, are used to increase the flexibility and allow to examine non-linear relationships by linear methods. The conventional solution of the restricted kernel canonical correlation analysis problem has a major drawback, it solves the problem in a reasonable time frame only for problems with few variables. We successfully overcame this limitation by implementing the method with the alternating least-squares algorithm. This allowed us to apply the method to cross-language information retrieval problem on a big dataset. We compared the results to other established methods and they were encouraging.

Quantized CP approximation and sparse tensor interpolation of function‐generated data

SummaryIn this article, we consider the iterative schemes to compute the canonical polyadic (CP) approximation of quantized data generated by a function discretized on a large uniform grid in an interval on the real line. This paper continues the research on the quantics‐tensor train (QTT) method (“O(d log N)‐quantics approximation of N‐d tensors in high‐dimensional numerical modeling” in Constructive Approximation, 2011) developed for the tensor train (TT) approximation of the quantized images of function related data. In the QTT approach, the target vector of length 2L is reshaped to a Lth‐order tensor with two entries in each mode (quantized representation) and then approximated by the QTT tensor including 2r2L parameters, where r is the maximal TT rank. In what follows, we consider the alternating least squares (ALS) iterative scheme to compute the rank‐r CP approximation of the quantized vectors, which requires only 2rL≪2L parameters for storage. In the earlier papers (“Tensors‐structured numerical methods in scientific computing: survey on recent advances” in Chemom Intell Lab Syst, 2012), such a representation was called QCan format, whereas in this paper, we abbreviate it as the QCP (quantized canonical polyadic) representation. We test the ALS algorithm to calculate the QCP approximation on various functions, and in all cases, we observed the exponential error decay in the QCP rank. The main idea for recovering a discretized function in the rank‐r QCP format using the reduced number of the functional samples, calculated only at O(2rL) grid points, is presented. The special version of the ALS scheme for solving the arising minimization problem is described. This approach can be viewed as the sparse QCP‐interpolation method that allows to recover all 2rL representation parameters of the rank‐r QCP tensor. Numerical examples show the efficiency of the QCP‐ALS‐type iteration and indicate the exponential convergence rate in r.

Joint Channel Parameter Estimation in Multi-Cell Massive MIMO System

In this paper, we consider the uplink channel parameter estimation problem in the presence of pilot contamination for massive multiple-input-multiple-output (MIMO) systems. We propose a parallel factor (PARAFAC)-based estimation scheme, which exploits the low-rank property of massive MIMO channels caused by the finite scattering in a physical environment. Specifically, we first parameterize the channel in terms of three parameters, i.e., fading coefficients, directions of arrival (DOAs), and delays; thereby, the channel is characterized via three equivalent PARAFAC models. Then, the proposed PARAFAC-based scheme is developed, which jointly estimates these three channel parameters using an alternating least squares (ALS) algorithm. Therein, we certify the identifiability of the three channel parameters of the PARAFAC models to mitigate the pilot contamination and state the convergence of the ALS algorithm, which guarantees that the three channel parameters can be uniquely determined with the proposed scheme. Moreover, to further reduce the computational complexity, two advanced schemes are proposed by antenna selection and reducing the estimation frequency of DOAs and delays, respectively. Simulation results show that the proposed schemes can achieve both low computational complexities and close to optimal Cramer–Rao Bound performance.

Matrix Polynomial Predictive Model: A New Approach to Accelerating the PARAFAC Decomposition

Alternating least squares (ALS) and its variations are the most commonly used algorithms for the PARAFAC decomposition of a tensor. However, it is still troubled for one how to accelerate the ALS algorithm with the reduced computational complexity. In this paper, a new acceleration method for the ALS with a matrix polynomial predictive model (MPPM) is proposed. In the MPPM, a matrix-valued function is first approximated by a matrix polynomial. It is shown that the future value of the function can be predicted by an FIR filter with the coefficients determined offline. By viewing each factor matrix of a tensor as a matrix-valued function, a new ALS algorithm, the ALS-MPPM algorithm, is then given. Analyses show that our ALS-MPPM algorithm is of low computational complexity and a close relation with the existing ALS algorithms. Moreover, to further accelerate the convergence of the proposed algorithm, a new technique called the multi-model (MM) prediction is also introduced. While the analytical results are verified by the numerical simulations, it is also shown that our ALS-MPPM outperforms the existing ALS-based algorithms in terms of the rate of convergence.

Constrained ALS-based tensor blind receivers for multi-user MIMO systems

Tensor factorizations has shown to be an efficient approach for symbols and/or channel estimation in multi-input multi-output (MIMO) systems, where the factor matrices of tensor that correspond to symbols, channel, code/diversity of signals, are often estimated by using alternating least squares (ALS) algorithm. Although the performance of tensor approaches strongly depend on the initializations of the factor matrices. However, due to the absence of a priori on channels, these initializations are done randomly in traditional ALS algorithm. This generally implies a slow convergence. Further, ALS does not take into account the potential orthogonal structure in the factor matrices, which can be exploited to improve the accuracy of factor matrices recovery. To address these insures, this paper proposes constrained ALS tensor blind receivers for multi-user MIMO systems. We show that the multi-user MIMO signals can be expressed as a third-order tensor model, where the matrices of users symbols, direction-of-arrival (DOA) and delay can be viewed as three factor matrices of the tensor model. Two constrained ALS blind algorithms that take into account the potential orthogonal and Vandermonde structures in the factor matrices, are proposed to learn the tensor model, where the users symbols, DOA and delay are joint estimated as three factor matrices. Besides provide the estimations for the factor matrices, the orthogonal and Vandermonde structures also give a better uniqueness results for the use of tensor model. Interestingly, these structures are the nature properties of the factor matrices in our system. This results in an efficient blind approach that has better performance and lower complexity compare with the traditional ALS.

Alternating Least Squares (ALS) blind source separation performance varied on dimension, salinity, and temperature water environment

This experiment conduct in mini semi-anechoic water test tank and towing tank at IHL. The purpose is to analyse performance from ALS algorithm in dimension, salinity and temperature water environment. For this purpose we varied water salinity and temperature in mini semi-anechoic water test tank, and for different dimension we took the experiment on two different size of test tank. The results showed that ALS have consistent result at salinity and temperature variety but at large dimension test tank 200 × 10 × 5.5 m without variation on the water medium indicates an anomaly in the results of the ALS BSS technique performance.

A seminorm regularized alternating least squares algorithm for canonical tensor decomposition

The regularization method could deal with the swamp effect of alternating least squares (ALS) algorithms for tensor decomposition. Usually, the regularization term is a norm of the difference between the solution and the current iterate. In this paper, we show that the norm could be weakened to a seminorm, so the selection of the regularization term could be more flexible. To overcome the swamp effect and avoid the drawback that the Hessian of the subproblem may get close to singular in the iterative process, we propose a seminorm regularized ALS algorithm for solving the canonical tensor decomposition. Moreover, in the new algorithm, we introduce a novel extrapolation in the update of each mode factor which makes an immediate impression on the update of subsequent ones. By assuming the boundness of the infinite sequence of iterates generated by the new algorithm, we establish the global convergence and the (weakly) linear convergence rate of the sequence of iterates Numerical experiments on synthetic and real-world problems illustrate that the new method is efficient and promising.

A real-time recommendation engine using lambda architecture

In a data science theory, the recommended methodology is one of the most popular theories and has been deployed in many real industries. However, one of the most challenging problems these days is how to recommend items with massively streaming data. Therefore, this paper aims to do a real-time recommendation engine using the Lambda architecture. The Apache Hadoop and Apache Spark frameworks were used in this research to process the MovieLens dataset comprised 100 K and 20 M ratings from the GroupLens research. Using alternating least squares (ALS) and k-means algorithms, the top K recommendation movies and the top K trending movies for each user were shown as results. Additionally, the mean squared error (MSE) and within cluster sum of squared error (WCSS) had been computed to evaluate the performance of the ALS and k-means algorithms, sequentially. The results showed that they are acceptable since the MSE and WCSS values are low when comparing to the size of data. However, they can still be improved by tuning some parameters.

HPC formulations of optimization algorithms for tensor completion

Abstract Tensor completion is a powerful tool used to estimate or recover missing values in multi-way data. It has seen great success in domains such as product recommendation and healthcare. Tensor completion is most often accomplished via low-rank sparse tensor factorization, a computationally expensive non-convex optimization problem which has only recently been studied in the context of parallel computing. In this work, we study three optimization algorithms that have been successfully applied to tensor completion: alternating least squares (ALS), stochastic gradient descent (SGD), and coordinate descent (CCD++). We explore opportunities for parallelism on shared- and distributed-memory systems and address challenges such as memory- and operation-efficiency, load balance, cache locality, and communication. Among our advancements are a communication-efficient CCD++ algorithm, an ALS algorithm rich in level-3 BLAS routines, and an SGD algorithm which combines stratification with asynchronous communication. Furthermore, we show that introducing randomization during ALS and CCD++ can accelerate convergence. We evaluate our parallel formulations on a variety of real datasets on a modern supercomputer and demonstrate speedups through 16384 cores. These improvements reduce time-to-solution from hours to seconds on real-world datasets. We show that after our optimizations, ALS is advantageous on parallel systems of small-to-moderate scale, while both ALS and CCD++ provide the lowest time-to-solution on large-scale distributed systems.

Low-complexity tensor-based blind receivers for MIMO systems

Abstract In the tensor-based MIMO receivers, the multidimensional MIMO signals first are expressed as a third-order tensor model, wherein the factor matrices of tensor model are corresponding time/frequency, symbols, code/diversity of signals. A algorithm then is used for fitting this tensor mode, in which the symbols are estimated as a independent factor matrix. Although the performance of tensor-based receivers strongly depends on the initializations of the factor matrices. However, due to the absence of a priori on channels, these initializations are done randomly in alternating least squares (ALS), a basic algorithm for fitting the tensor models. In order to avoid these random initializations, this paper proposes two algorithms for fitting the tensor models. The first one, called delta bilinear ALS (DBALS) algorithm, where we exploit the increment values between two iterations of the factor matrices, refine these predictions by using the enhanced line search and use these refined values to initialize for two factor matrices. The second one, called orthogonal DBALS algorithm that takes into account the potential orthogonal in factor matrix for the DBALS algorithm, to provide the initialization for this factor matrix. By this way, we avoid random initializations for three factor matrices of tensor model. The performance of proposed receivers is illustrated by means of simulation results and a comparison is made with traditional ALS algorithm and other receivers. Beside a performance improving, our receivers give a lower complexity due to avoid random initializations.

GINDCLUS: Generalized INDCLUS with External Information

A Generalized INDCLUS model, termed GINDCLUS, is presented for clustering three-way two-mode proximity data. In order to account for the heterogeneity of the data, both a partition of the subjects into homogeneous classes and a covering of the objects into groups are simultaneously determined. Furthermore, the availability of information which is external to the three-way data is exploited to better account for such heterogeneity: the weights of both classifications are linearly linked to external variables allowing for the identification of meaningful classes of subjects and groups of objects. The model is fitted in a least-squares framework, and an efficient Alternating Least-Squares algorithm is provided. An extensive simulation study and an application on benchmark data are also presented.