Alternating Least Squares Research Articles

A novel estimation procedure for robust CANDECOMP/PARAFAC model fitting

The parameter estimation in CANDECOMP/PARAFAC (CP) is carried out by alternating least squares (ALS) that yields least-squares solutions and provides consistent outcomes. At the same time it has several drawbacks, like sensitivity to the presence of outliers in the data, issues with the computational efficiency in terms of processing time and memory requirements, as well as susceptibility to degeneracy conditions. These weaknesses have been addressed, but there is no outlier-robust procedure that at the same time is highly computationally efficient, especially for large data sets. A novel procedure based on an integrated estimation algorithm is proposed. This is an alternative to ALS, which guards against outliers and is computationally efficient at the same time. The performance of the new method is demonstrated on an extensive simulation study and an empirical example.

Blockwise acceleration of alternating least squares for canonical tensor decomposition

AbstractThe canonical polyadic (CP) decomposition of tensors is one of the most important tensor decompositions. While the well‐known alternating least squares (ALS) algorithm is often considered the workhorse algorithm for computing the CP decomposition, it is known to suffer from slow convergence in many cases and various algorithms have been proposed to accelerate it. In this article, we propose a new accelerated ALS algorithm that accelerates ALS in a blockwise manner using a simple momentum‐based extrapolation technique and a random perturbation technique. Specifically, our algorithm updates one factor matrix (i.e., block) at a time, as in ALS, with each update consisting of a minimization step that directly reduces the reconstruction error, an extrapolation step that moves the factor matrix along the previous update direction, and a random perturbation step for breaking convergence bottlenecks. Our extrapolation strategy takes a simpler form than the state‐of‐the‐art extrapolation strategies and is easier to implement. Our algorithm has negligible computational overheads relative to ALS and is simple to apply. Empirically, our proposed algorithm shows strong performance as compared to the state‐of‐the‐art acceleration techniques on both simulated and real tensors.

CP decomposition for tensors via alternating least squares with QR decomposition

AbstractThe CP tensor decomposition is used in applications such as machine learning and signal processing to discover latent low‐rank structure in multidimensional data. Computing a CP decomposition via an alternating least squares (ALS) method reduces the problem to several linear least squares problems. The standard way to solve these linear least squares subproblems is to use the normal equations, which inherit special tensor structure that can be exploited for computational efficiency. However, the normal equations are sensitive to numerical ill‐conditioning, which can compromise the results of the decomposition. In this paper, we develop versions of the CP‐ALS algorithm using the QR decomposition and the singular value decomposition, which are more numerically stable than the normal equations, to solve the linear least squares problems. Our algorithms utilize the tensor structure of the CP‐ALS subproblems efficiently, have the same complexity as the standard CP‐ALS algorithm when the input is dense and the rank is small, and are shown via examples to produce more stable results when ill‐conditioning is present. Our MATLAB implementation achieves the same running time as the standard algorithm for small ranks, and we show that the new methods can obtain lower approximation error.

Downlink channel estimation of intelligent reflective surface aided MU-MIMO system

Obtaining channel status information (CSI) is one of the main problems of the intelligent reflective surface (IRS) assisted communication system. This is because IRS has only a large number of passive reflective components and cannot process signals. In this paper, we aim to estimate downlink channel of IRS assisted multi-user multiple-input multiple-output (MU-MIMO) communication system. In time division duplex (TDD) mode, the downlink channel is estimated according to the uplink pilot by using channel reciprocity. We propose a parallel factor analysis based on projection subspace (PSS-PARAFAC) channel estimation scheme. In the first step, the common sparsity among multiple users is used for signal subspace projection to reduce noise energy and complexity. The second step is to convert each user signal received of base station (BS) into the PARAFAC problem. The iterative method of alternating least squares (ALS) is used to solve this problem. From the analysis of simulation results, the channel estimation performance of the proposed algorithm is better.

Clofazimine pKa Determination by Potentiometry and Spectrophotometry: Reverse Cosolvent Dependence as an Indicator of the Presence of Dimers in Aqueous Solutions.

The weakly basic antibiotic and anti-inflammatory drug, clofazimine (CFZ), was first described in 1957. It has been used therapeutically, most notably in the treatment of leprosy. However, the compound is extremely insoluble in aqueous media, and, indeed, there is poor consensus about what its intrinsic solubility is since the reported values range from 0.04 to 11 ng/mL. To understand the speciation and solubilization of CFZ as a function of pH, it is of paramount importance to know the true aqueous pKa. However, there is also poor consensus about the value of the pKa (reported measured values range from 6.08 to 9.11). In the present study, we report the determination of the CFZ ionization constant using two independent techniques. A state-of-the-art potentiometric analysis was performed, drawing on titration data in methanol-water solutions (46-75 wt % MeOH) of CFZ, using the bias-reducing consensus of two different procedures of extrapolating the apparent psKa values to zero cosolvent to approximate the true aqueous pKa as 9.43 ± 0.12 (25 °C, I = 0.15 M reference ionic strength). In parallel, spectrophotometric UV/vis titration data were acquired (250-600 nm at different pH) in 10 mM HEPES buffer solutions containing up to 54 wt % MeOH. The alternating least squares (ALS) method was used in the analysis of the absorbance-pH spectra. Uncharacteristically, the cosolvent UV/vis data in our study showed reverse cosolvent dependence (apparent pKa values increased with increasing cosolvent) which could be explained by a dimerization of the free base. The analysis of UV/vis data obtained from 54 wt % MeOH-water solution containing 20 μM CFZ yielded the apparent pKa 9.51 ± 0.17 (I ≈ 0.005 M). To assess whether self-assembly of CFZ was energetically feasible, density functional theory (DFT) calculations were used to study the putative CFZ dimers in aqueous and methanol media. The DFT-optimized geometries and infrared spectra of CFZ dimers using water and methanol as solvents were calculated and analyzed. Based on the lack of negative frequencies in calculated infrared spectra, it was confirmed that optimized geometries correspond to the true energetic minima. Visual analysis of optimized structures indicates the presence of stacking interactions between two CFZ molecules. The protonation site (the imine nitrogen atom) was determined by 1H NMR spectroscopy.

General Purpose Recommendation System

A recommendation system is a class of information retrieval systems that provides personalized suggestions to users. Our approach is to generalize the process of creating a recommendation system so that it can be used effectively in multiple domains. This approach can be achieved by collaborative filtering by using the website’s clickstream analytics. Data related to user clicks can easily be extracted and this data's use is limited to analyse the user behaviour. We can use this extracted data to generate recommendations for the users and increase engagement on the website. The identifiers of users and items can be passed along with the user click rate to a supervised machine learning model of matrix factorization. We are using alternating least square (ALS) to predict the recommended items. The suggested system interfaces with the client system using a web API which will collect data and generate recommendations

Big data visual exploration as a recommendation problem

Big data visual exploration is believed to be considered as a recommendation problem. This proximity concerns essentially their purpose: it consists in selecting among huge amount of data those that are the most valuable according to specific criteria, to eventually present it to users. On the other hand, the recommendation systems are recently resolved mostly using neural networks (NNs). The present paper proposes three alternative solutions to improve the big data visual exploration based on recommendation using matrix factorisation (MF) namely: conventional, alternating least squares (ALS)-based and NN-based methods. It concerns generating the implicit data used to build recommendations, and providing the most valuable data patterns according to the user profiles. The first two solutions are developed using Apache Spark, while the third one was developed using TensorFlow2. A comparison based on results is done to show the most efficient one. The results show their applicability and effectiveness.

Algorithms for Structure Preserving Best Rank-one Approximations of Partially Symmetric Tensors

Based on the alternating least squares (ALS) technique and the singular value decomposition (SVD) technique, the algorithms for finding the structure preserving best rank-one approximations of even order paired symmetric tensors and k-mode symmetric tensors are presented. Then the convergence of the algorithms is discussed. Some numerical examples and applications are given to illustrate the feasibility and effectiveness of the proposed algorithms.

Tensor-Based Underdetermined Blind Identification of Instantaneous Mixtures

In order to explore the effective statistical information of observed signals to stack into a tensor, while reduce the computational complexity and speed up the convergence, a new tensor-based underdetermined blind identification method of instantaneous mixture is proposed in this brief. Firstly, the autocovariance matrix of each observed sub-block is calculated to construct the symmetric third-order tensor according to segmentation scheme. Secondly, the original tensor is reduced into a low-rank kernel tensor by truncated multi-linear singular value decomposition (MLSVD). Finally, an enhanced line search (ELS) technique is applied to speed up the convergence of alternating least squares (ALS) to identify the mixing matrix. The simulation results indicate that the presented approach provides a superior estimated performance compared to the existing methods in both real and complex scenarios.

Channel Estimation in RIS-Assisted MIMO Systems Operating Under Imperfections

The promising gains of reconfigurable intelligent surface (RIS)-assisted multiple-input multiple-output (MIMO) systems, in terms of extended coverage and enhanced capacity, are critically dependent on the accuracy of the channel state information. However, traditional channel estimation (CE) schemes are not applicable in RIS-assisted MIMO networks, since passive RISs typically lack the signal processing capabilities that are assumed by CE algorithms. This becomes problematic when physical imperfections or electronic impairments affect the RIS due to its exposition to different environmental effects or caused by hardware limitations from the circuitry. While these real-world effects are typically ignored in the literature, in this paper we propose efficient CE schemes for RIS-assisted MIMO systems taking different imperfections into account. Specifically, we propose two sets of tensor-based algorithms, based on the parallel factor analysis decomposition schemes. First, assuming a long-term model-where the RIS imperfections, modeled as unknown phase shifts, are static within the channel coherence time-we formulate an iterative alternating least squares (ALS)- based algorithm for the joint estimation of the unknown phase deviations and the communication channels. Then, we develop the short-term imperfection model, which allows both amplitude and phase RIS imperfections to be non-static with respect to the channel coherence time. We propose two iterative ALS-based and closed-form higher-order singular value decomposition-based algorithms for jointly estimating the channels and the unknown impairments. We also investigate the computational complexity and the identifiability of the proposed algorithms and study the effect of various imperfections on the CE quality. Simulation results show the effectiveness of our proposed tensor-based algorithms in terms of estimation accuracy and computational complexity.

Remote Sensing Image Reconstruction Based on Improved Adaptive Tensor Nuclear Norm Minimization

Incomplete remote sensing image observation data require proper processing to restore the original data. Nevertheless, the lack of information, due to noise and dead pixels, along with the quality of the acquired images that may be severely degraded, limits the accuracy of later processing. Therefore, this paper proposes a remote sensing image restoration scheme that employs the low-rank structure of the Bayesian probability tensor latent space and a new tensor completion method, which has strong robustness for the model selection process of the following remote sensing image data analysis. Considering the data space, we incorporate an adaptive nuclear norm regularization into the processing of latent factors that performs singular value decomposition on a more accurate scale. The proposed model adopts two algorithms: Augmented Lagrange Multiplier (ALM) and Alternating Least Squares (ALS). Several synthetic and real-data-based experiments illustrate the inherent ability of our method in restoring original information and preventing overfitting problems, even when a vast number of items are missing. In addition, the combination of adaptive nuclear norm regularization and tensor decomposition is stable, settling a high sensitivity of rank selection and low computational efficiency in traditional tensor reconstruction methods.

Food and Beverage Recommendation in EatAja Application Using the Alternating Least Square Method Recommender System

EatAja is a startup in Indonesia that provides a mobile application-based food and beverage ordering solution for restaurants. The EatAja application uses transaction data to recommend food and beverage menus to customers. Previous studies have developed recommender systems using the Apriori and Collaborative Filtering methods. However, there are shortcomings in the recommendation system using both methods, i.e., the lack of personalization factors and low scalability. The learning method with matrix factorization can overcome the problem. In this study, we improve the food and beverage product recommender system in the EatAja application using the Alternating Least Square (ALS) matrix factorization method on Apache Spark. We will compare the results of the recommender system using the ALS method with the Collaborative Filtering method. The comparison uses the Mean Absolute Error (MAE) evaluation method. The results showed that the MAE value decreased by 0.07 with the ALS Matrix factorization method.

On the Training Mode of Innovative and Entrepreneurial Talents in Higher Vocational Finance and Economics Professional Groups under the Background of “Big Wisdom and Cloud”

The arrival of the era of “big wisdom and cloud” has promoted the restructuring of the industrial system, and the development of the new economy has also brought new changes to the demand for financial and business professionals. With the change of the employment market, the training objectives of financial and business professionals are also changing day by day. Practical teaching is becoming more and more important in the training of financial and business professionals. “Employment” in people’s livelihood has been identified as the main goal of government work. As the main source of economic restructuring, “mass innovation and mass entrepreneurship” has successfully integrated the spirit of innovation into all fields of society. Entrepreneurship has attracted the attention of the whole society. Under the influence of policy factors and various factors during the entrepreneurship period, the source and connotation of China’s entrepreneurship policy have changed significantly. On the basis of summarizing the practical teaching modes of foreign finance and economics majors, this study introduces the training objectives, conditions, and curriculum settings of the practical teaching of finance and economics majors in higher vocational colleges. This paper designs a recommendation algorithm for innovation and entrepreneurship talent training model based on potential Dirichlet distribution (LDA) theme model and alternating least squares (ALS) collaborative filtering, which not only improves the accuracy of the recommendation results of talent training model but also improves the accuracy of the recommendation results of talent training model. On the premise of stable recommendation, the diversity of recommendation results has increased, which meets the needs of innovative and entrepreneurial talent training of higher vocational finance and economics majors under the background of “great wisdom and cloud” and provides a certain reference for the practical teaching reform of higher vocational finance and economics majors in China. The cultivation of compound talents is of great significance to the development of professionals in modern multimedia and advanced e-commerce industries.

Accelerated PARAFAC-Based Channel Estimation for Reconfigurable Intelligent Surface-Assisted MISO Systems

To achieve fast and accurate channel estimation of reconfigurable intelligent surface (RIS)-assisted multiple-input single-output (MISO) systems, we propose an accelerated bilinear alternating least squares algorithm (ABALS) based on parallel factor decomposition. Firstly, we build a tensor model of the received signal, and expand it to obtain the unfolded forms of the model. Secondly, we derive the expression of the estimation problem of two channels based on the unfolded forms to transform the problem into a cost function problem. Furthermore, we solve the cost function problem by introducing a simpler iterative optimization constraint and linear interpolation. Finally, we provide a strategy on the receiver design based on the feasibility conditions discussed in this paper, which can guarantee the uniqueness of the channel estimation problem. Simulation results show that the proposed algorithm can obtain a faster estimation speed and less iteration steps than the alternating least squares (ALS) algorithm, and the accuracy of the two algorithms is very close.

Speeding up Training of Linear Predictors for Multi-Antenna Frequency-Selective Channels via Meta-Learning

An efficient data-driven prediction strategy for multi-antenna frequency-selective channels must operate based on a small number of pilot symbols. This paper proposes novel channel-prediction algorithms that address this goal by integrating transfer and meta-learning with a reduced-rank parametrization of the channel. The proposed methods optimize linear predictors by utilizing data from previous frames, which are generally characterized by distinct propagation characteristics, in order to enable fast training on the time slots of the current frame. The proposed predictors rely on a novel long short-term decomposition (LSTD) of the linear prediction model that leverages the disaggregation of the channel into long-term space-time signatures and fading amplitudes. We first develop predictors for single-antenna frequency-flat channels based on transfer/meta-learned quadratic regularization. Then, we introduce transfer and meta-learning algorithms for LSTD-based prediction models that build on equilibrium propagation (EP) and alternating least squares (ALS). Numerical results under the 3GPP 5G standard channel model demonstrate the impact of transfer and meta-learning on reducing the number of pilots for channel prediction, as well as the merits of the proposed LSTD parametrization.

Tensor Rank Regularization with Bias Compensation for Millimeter Wave Channel Estimation

This paper presents a novel method of tensor rank regularization with bias compensation for channel estimation in a hybrid millimeter wave MIMO-OFDM system. Channel estimation is challenging due to the unknown number of multipath components that determines the channel rank. In general, finding the intrinsic rank of a tensor is a non-deterministic polynomial-time (NP) hard problem. However, by leveraging the sparse characteristics of millimeter wave channels, we propose a modified CANDECOMP/PARAFAC (CP) decomposition-based method that jointly estimates the tensor rank and channel component matrices. Our approach differs from most existing works that assume the number of channel paths is known and the proposed method is able to estimate channel parameters accurately without the prior knowledge of number of multipaths. The objective of this work is to estimate the tensor rank by a novel sparsity-promoting prior that is incorporated into a standard alternating least squares (ALS) function. We introduce a weighting parameter to control the impact of the previous estimate and the tensor rank estimation bias compensation in the regularized ALS. The channel information is then extracted from the estimated component matrices. Simulation results show that the proposed scheme outperforms the baseline l1 strategy in terms of accuracy and robustness. It also shows that this method significantly improves rank estimation success at the expense of slightly more iterations.

Algorithm 1026: Concurrent Alternating Least Squares for Multiple Simultaneous Canonical Polyadic Decompositions

Tensor decompositions, such as CANDECOMP/PARAFAC (CP), are widely used in a variety of applications, such as chemometrics, signal processing, and machine learning. A broadly used method for computing such decompositions relies on the Alternating Least Squares (ALS) algorithm. When the number of components is small, regardless of its implementation, ALS exhibits low arithmetic intensity, which severely hinders its performance and makes GPU offloading ineffective. We observe that, in practice, experts often have to compute multiple decompositions of the same tensor, each with a small number of components (typically fewer than 20), to ultimately find the best ones to use for the application at hand. In this article, we illustrate how multiple decompositions of the same tensor can be fused together at the algorithmic level to increase the arithmetic intensity. Therefore, it becomes possible to make efficient use of GPUs for further speedups; at the same time, the technique is compatible with many enhancements typically used in ALS, such as line search, extrapolation, and non-negativity constraints. We introduce the Concurrent ALS algorithm and library, which offers an interface to MATLAB, and a mechanism to effectively deal with the issue that decompositions complete at different times. Experimental results on artificial and real datasets demonstrate a shorter time to completion due to increased arithmetic intensity.

Spatio-Temporal Constraint-Based Low Rank Matrix Completion Approaches for Road Traffic Networks

Road traffic sensing is pivotal in all the imperative services offered by Intelligent Transportation Systems (ITS). Both static and probe based traffic sensing technologies aid in the acquisition of road traffic parameters, represented as spatio-temporal traffic data matrices. These matrices suffer from inevitable data losses, urging accurate matrix reconstruction. Many Low Rank Matrix Completion (LR-MC) approaches have been formulated in the literature, with varied optimization problems and solution methods. The subsequent approaches have exploited spatio-temporal constraints to further reduce the reconstruction error. In this paper, these formulations are modified to develop two new LR-MC approaches, the Augmented Lagrangian Sparsity Regularized Matrix Factorization (AL-SRMF) and the Constrained Low Rank (C-LR). AL-SRMF replaces the Alternating Least Squares (ALS) solution method of SRMF with an AL based method. Optimization in C-LR inherits its temporal constraint from the Temporal and Adaptive Spatial constrained Low Rank (TAS-LR) approach, while adopting its spatial constraint similar to SRMF. Spatial constraint in both C-LR and AL-SRMF is deduced by computing an adjacency matrix that leverages graph notion of the geographical road traffic network. For experimental validations of the proposed approaches, traffic matrices of Californian road network are obtained from PEMS database. Investigations reveal that for a signal integrity (SI) of 0.5, C-LR demonstrated a Normalized Mean Absolute Error (NMAE) of 3.4% and a Root Mean Square Error (RMSE) of 3.2. With a slight increase in computational time, the AL-SRMF rendered least values of NMAE and RMSE, 2.4% and 2.3 respectively, demonstrating significantly better performance than the state-of-the-art approaches.

Federated one-class collaborative filtering via privacy-aware non-sampling matrix factorization

In this paper, we study an emerging and important recommendation problem called federated one-class collaborative filtering (FOCCF). Specifically, we aim to build a recommendation model by exploiting each user’s one-class or implicit feedback in a distributed and privacy-aware manner rather than collecting and learning from data in a central server. For the studied problem, there are three important issues, i.e., recommendation accuracy, privacy and efficiency. As a response, we start from the state-of-the-art one-class collaborative filtering (OCCF) method, i.e., non-sampling matrix factorization (NSMF), and propose a novel federated recommendation framework called privacy-aware NSMF (P-NSMF). Our P-NSMF protects user privacy well without sacrificing recommendation accuracy and contains two variants, i.e., P-NSMF(ALS) and P-NSMF(BGD), which are based on alternating least squares (ALS) and batch gradient descent (BGD), respectively. Moreover, we design an improved strategy called group-wise concealing and adopt a secure aggregation technique in our framework for privacy and efficiency. We then analyze the security and complexity of our P-NSMF, and conduct extensive experiments on four public datasets. In particular, compared with the existing methods, our P-NSMF outperforms them in terms of recommendation accuracy, privacy and efficiency, which are important merits for real deployment.

A hybrid recommender system for recommending smartphones to prospective customers

Recommender Systems are a subclass of machine learning systems that employ sophisticated information filtering strategies to reduce the search time and suggest the most relevant items to any particular user. Hybrid recommender systems combine multiple recommendation strategies in different ways to benefit from their complementary advantages. Some hybrid recommender systems have combined collaborative filtering and content-based approaches to build systems that are more robust. In this paper, we propose a hybrid recommender system, which combines Alternating Least Squares (ALS) based collaborative filtering with deep learning to enhance recommendation performance as well as overcome the limitations associated with the collaborative filtering approach, especially concerning its “cold-start” problem. In essence, we use the outputs from ALS (collaborative filtering) to influence the recommendations from a Deep Neural Network (DNN), which combines characteristic, contextual, structural and sequential information, in a big data processing framework. We have conducted several experiments in testing the efficacy of the proposed hybrid architecture in recommending smartphones to prospective customers and compared its performance with other open-source recommenders. The results have shown that the proposed system has outperformed several existing hybrid recommender systems.