Outer-product Matrix Research Articles

Show-through removal with sparsity-based blind deconvolution.

When scanning a document printed on both sides by using an electronic scanner, the printed material on the back (front) side may be transmitted to the front (back) side. This phenomenon is called show-through. The problem to remove the show-through from scanned images is called the show-through removal in the literature. In this paper, we propose a new method of show-through removal based on the following principle. The proposed method uses two scanned images with the front side and with the back side as input images. The proposed method is based on Ahmed's Blind Image Deconvolution method discovered in 2013, which succeeded in formulating Blind Image Deconvolution as a nuclear norm minimization. Since the structure of show-through removal resembles that of Blind Image Deconvolution, we discovered that the show-through removal can be reformulated into a nuclear norm minimization in the space of outer product matrix constructed from an image vector and a point spread function vector of blurring. Using this key idea, we constructed the proposed method as follows. First, our cost function consists of the following three terms. The first term is the data term and the second term is the nuclear norm derived from the above reformulation. The third term is a regularization term to overcome the underdetermined nature of show-through removal problem and the existence of noise in the measured images. The regularization term consists of Total Variation imposed on the images. The resulting nuclear norm minimization problem is solved by using Accelerated Proximal Gradient method and Singular Value Projection with some problem-specific modifications, which converges fast and requires a simple implementation. We show results of simulation studies as well as results of real image experiments to demonstrate the performances of the proposed method.

Enhancing neural collaborative filtering using hybrid feature selection for recommendation.

The past decade has seen substantial growth in online transactions. Accordingly, many professionals and researchers utilize deep learning models to design and develop recommender systems to suit the needs of online personal services. These systems can model the interactions between users and items. However, existing approaches focus on either modeling global or local item correlation and rarely consider both cases, thus failing to represent user-item correlation very well. Therefore, this article proposes a deep collaborative recommendation system based on a convolutional neural network with an outer product matrix and a hybrid feature selection module to capture local and global higher-order interaction between users and items. Moreover, we incorporated the weights of generalized matrix factorization to optimize the overall network performance and prevent overfitting. Finally, we conducted extensive experiments on two real-world datasets with different sparsity to confirm that our proposed approach outperforms the baseline methods we have used in the experiment.

Efficient Adaptive Online Learning via Frequent Directions.

By employing time-varying proximal functions, adaptive subgradient methods (ADAGRAD) have improved the regret bound and been widely used in online learning and optimization. However, ADAGRAD with full matrix proximal functions (ADA-FULL) cannot handle large-scale problems due to the impractical O(d³) time and O(d²) space complexities, though it has better performance when gradients are correlated. In this paper, we propose two efficient variants of ADA-FULL via a matrix sketching technique called frequent directions (FD). The first variant named as ADA-FD directly utilizes FD to maintain and manipulate low-rank matrices, which reduces the space and time complexities to O(τd) and O(τ²d) respectively, where d is the dimensionality and τ << d is the sketching size. The second variant named as ADA-FFD further adopts a doubling trick to accelerate FD used in ADA-FD, which reduces the average time complexity to O(τd) while only doubles the space complexity of ADA-FD. Theoretical analysis reveals that the regret of ADA-FD and ADA-FFD is close to that of ADA-FULL as long as the outer product matrix of gradients is approximately low-rank. Experimental results demonstrate the efficiency and effectiveness of our algorithms.

Iron Ore Tailing Composition Estimation Using Fused Visible–Near Infrared and Thermal Infrared Spectra by Outer Product Analysis

Iron ore tailings are mainly composed of SiO2 and iron, whose content determines the potential reuse strategy of the tailings. Compared with the traditional wet chemistry approach, spectroscopy has proven its superior effectiveness in characterizing and predicting minerals, such as iron oxides, clay, and SiO2. This study aims to estimate the content of SiO2 and TFe in iron ore tailings based on visible–near infrared (VIS–NIR, 350–2500 nm) and thermal infrared (TIR, 8–14 μm) spectroscopy. The outer product analysis (OPA) method is used to combine VIS–NIR and TIR spectral domains, from which an outer product matrix of fusion data can be generated. The study area is the iron ore tailing dam from Waitoushan, which is one of the super-large iron deposits in the Anshan–Benxi iron cluster of northeastern China. The spectral analysis results demonstrated the following: (1) The reflectance feature at 1163–2499 nm in the VIS–NIR range correlates with TFe and the emissivity feature at 8–9.4 and 10.7–12 μm in the TIR range correlates with SiO2. (2) Compared with the original absorbance spectra, the correlation coefficients of fusion spectra improve from 0.66 to 0.87 for TFe and from 0.64 to 0.84 for SiO2. (3) The partial least squares regression, random forest (RF), and extreme learning machine exploiting particle swarm optimization modeling methods are established for SiO2 and TFe estimation. The prediction accuracy results indicate that the prediction model with OPA-fused spectra performs significantly better than with individual VIS–NIR and TIR spectra. The RF model with input-fused spectra provides the highest accuracy with the coefficients of determination of 0.95 and 0.91, root mean square errors of 0.97% and 0.96%, and ratios of performance to interquartile distance of 6.49 and 2.31 for SiO2 and TFe content estimation, respectively. These outcomes provide a theoretical basis and technical support for tailing composition estimation using spectroscopy.

Deep learning based recommender system using cross convolutional filters

With the recent development of online transactions, recommender systems have increasingly attracted attention in various domains. The recommender system supports the users’ decision making by recommending items that are more likely to be preferred. Many studies in the field of deep learning-based recommender systems have attempted to capture the complex interactions between users’ and items’ features for accurate recommendation. In this paper, we propose a recommender system based on the convolutional neural network using the outer product matrix of features and cross convolutional filters. The proposed method can deal with the various types of features and capture the meaningful higher-order interactions between users and items, giving greater weight to important features. Moreover, it can alleviate the overfitting problem since the proposed method includes the global average or max pooling instead of the fully connected layers in the structure. Experiments showed that the proposed method performs better than the existing methods, by capturing important interactions and alleviating the overfitting issue.

Computational Options for Standard Errors and Test Statistics with Incomplete Normal and Nonnormal Data in SEM

ABSTRACT This article provides an overview of different computational options for inference following normal theory maximum likelihood (ML) estimation in structural equation modeling (SEM) with incomplete normal and nonnormal data. Complete data are covered as a special case. These computational options include whether the information matrix is observed or expected, whether the observed information matrix is estimated numerically or using an analytic asymptotic approximation, and whether the information matrix and the outer product matrix of the score vector are evaluated at the saturated or at the structured estimates. A variety of different standard errors and robust test statistics become possible by varying these options. We review the asymptotic properties of these computational variations, and we show how to obtain them using lavaan in R. We hope that this article will encourage methodologists to study the impact of the available computational options on the performance of standard errors and test statistics in SEM.

Mixed construction of OOC for optical code division multiple access networks

Optical orthogonal codes play an important role in optical code division multiple access networks and optical wireless systems. In this study, by combining combinatorial design and meta-heuristic search algorithm, a mixed construction of ( v , k , 1 ) -OOCs is proposed. The mixed construction enjoys the benefits of both mathematical methods and search algorithms and can be used to produce OOCs with large and flexible parameters. Numerical results show that some of the OOCs obtained by this construction have more codewords in comparison with the construction method using outer-product matrix.

Statistical covariance-matching based blind channel estimation for zero-padding MIMO–OFDM systems

We propose a statistical covariance-matching based blind channel estimation scheme for zero-padding (ZP) multiple-input multiple-output (MIMO)–orthogonal frequency division multiplexing (OFDM) systems. By exploiting the block Toeplitz channel matrix structure, it is shown that the linear equations relating the entries of the received covariance matrix and the outer product of the MIMO channel matrix taps can be rearranged into a set of decoupled groups. The decoupled nature reduces computations, and more importantly guarantees unique recovery of the channel matrix outer product under a quite mild condition. Then the channel impulse response matrix is identified, up to a Hermitian matrix ambiguity, through an eigen-decomposition of the outer product matrix. Simulation results are used to evidence the advantages of the proposed method over a recently reported subspace algorithm applicable to the ZP-based MIMO–OFDM scheme.

OOC constructing algorithm based on outer-product matrix

Optical code division multiple access (OCDMA) has wide applications in the next optical access network. One of its key techniques is construction of address code. Aiming at the facts that most of constructing algorithm of optical orthogonal code (OOC) are based on inner-product representation of OOC, they can only construct certain parameters OOC, and the cardinality of (F, K, 1) OOC is small. It can not meet the requirement of optical networks. In this paper, an algorithm of constructing (F, K, λa, 1) OOC based on outer-product matrix is presented and simulated; some groups of (F, K, λa, 1) OOC are obtained. The results show that the algorithm can construct arbitrary code length, code weight and auto-correlation λa (F, K, λa, 1) OOC codes with cardinality closer to the Johnson upper bound, it can construct (F, K, λa, 1) OOC effectively. It is feasible.

Image processing of outer-product matrices – A new way to classify samples: Examples using visible/NIR/MIR spectral data

A chemometric analysis has been developed to emphasise the discrimination power of spectroscopic techniques such as near infrared, mid-infrared and visible spectroscopy. The combination of two spectral domains using outer product analysis (OPA) leads to the calculation of an outer product (OP) matrix. The representation of this matrix is called the “analytical fingerprint” of the samples and their classification is performed in the following steps. First, two different techniques are tested by subtracting the images one-by-one and the sum of all the elements of the resulting difference matrix gives a scalar, characteristic of the distance between the two images. Combining chemical analysis with image processing techniques provides an original approach to study butters and margarines in relation to their fat content. Best results were obtained with the OP matrix built from NIR and visible signals following the use of city-block distance and average linkage. Samples were arranged in four groups: 100%, 82–75%, 70–59% and 38–25% w/w fat. The cophenetic correlation coefficient (validity of the cluster information generated by the linkage function) associated with these spectral data has a value of 0.973. Similar results were obtained using Ward's algorithm which generated four groups and a cophenetic correlation coefficient equal to 0.959.

Outer-Product Matrix Representation of Optical Orthogonal Codes

In this letter, we introduce a whole new approach in defining and representing optical orthogonal codes (OOCs), namely, outer-product matrix representation. Instead of applying commonly used approaches based on inner product to construct OOC codes, we use the newly defined approach to obtain a more efficient algorithm in constructing and generating OOC codes. The outer-product matrix approach can obtain a family of OOC codes with a cardinality closer to the Johnson upper bound, when compared with the previously defined accelerated greedy algorithm using the inner-product approach. We believe the new look introduced in this letter on OOCs could help to devise new approaches in designing and generating OOC codes, using the rich literature in matrix algebra

Outer-product analysis (OPA) using PLS regression to study the retrogradation of starch

Outer-product analysis (OPA) is a method which makes it possible to emphasise co-evolutions of spectral regions in signals obtained for two different domains or even for the same domain. The calculation of the outer-product (OP) matrix, which links the two domains, corresponds to a mutual weighting of the two signals. Many statistical techniques – both univariate and multivariate – can be applied to these matrices to highlight these simultaneous variations. Applying partial least squares (PLS) regression to the OP matrix allows to direct the analysis and to visualise the simultaneous variations associated with a particular evolution of the samples. In this study, OPA is used to characterise the retrogradation of starch by time domain NMR, Fourier transform-MidInfraRed spectroscopy (MIR) and X-ray diffraction (XRD). During the process of retrogradation, the starch evolves from an amorphous state to a semi-crystalline state and this transformation affects these three signals. Three OP matrices are calculated corresponding to two-by-two combinations of the three signals. PLS regression is then performed to predict the retrogradation time. The B-coefficients obtained highlight the simultaneous variations in the pairs of signals as a result of retrogradation. The optimal regression model rank determined by “cross-validation” criterion applied to the B-coefficients vector indicated more components than the Durbin–Watson. Samples with a retrogradation time between 4 and 10 h (phase of the retrogradation process with visible changes in XRD) were characterised by the peak in the MIR spectrum at 998 cm −1.

Outer-product analysis (OPA) using PCA to study the influence of temperature on NIR spectra of water

Outer-product analysis (OPA) is a method which makes it possible to emphasise co-evolutions of spectral regions in signals acquired in two different domains or even for the same domain. The calculation of the outer-product (OP) matrix linking the two domains corresponds to a mutual weighting of the two signals. Various statistical techniques—both univariate and multivariate—can be applied to these matrices to bring out these simultaneous variations. Applying principal components analysis (PCA) to the OP matrix allows to visualise both the distribution of the individuals (scores plots) and the simultaneous variations in the signals related to this distribution of the individuals (loadings). In the case of complex data sets with many sources of variation in the samples, partial least squares regression (PLS) can be used to direct the analysis in order to visualise those simultaneous variations that are associated with a particular evolution of the samples. In this study, we applied OPA to a data set concerning the evolution of near infrared (NIR) spectra of water as a function of temperature. The different loadings profiles obtained by PCA are compared with the synchronous and asynchronous variable–variable spectra obtained using 2D-correlation spectroscopy (2DCOS), in order to study the resemblances and the differences between the two techniques. The outer-product may also be calculated for the transposed matrices. In this case, the PCA loadings may be compared with the synchronous and asynchronous sample–sample spectra obtained by 2DCOS.

Application of 2D correlation spectroscopy and outer product analysis to infrared spectra of sugar beets

The reference method for the measurement of the percentage of sugar in sugar beets is polarimetry. Infrared spectroscopy is now being more and more used in the food industry. Based on polarimetric reference values and infrared spectra, regression techniques, such as partial least square regression (PLS), may be used to develop mathematical models to predict the sugar content of beets. Before establishing the model, it is preferable to select informative parts of the infrared spectra in order to improve the model performance. Several different methods can be used to determine regions of interest in a spectrum, such as outer product analysis (OPA) and 2D correlation spectroscopy (2DCOS). Outer product analysis can be used to facilitate the interpretation of near infrared (NIR) and mid infrared (MIR) signals. In this method, the spectra acquired in the two domains are combined by calculating the outer product matrix of the two vector signals of each sample. It is then possible to perform statistical analyses, such as principal components analysis or partial least square regression, on the resulting data set of matrices in order to highlight relations between spectral features in the two domains. This can facilitate the attribution of NIR bands based on their relation to MIR peaks. Results of the 2D correlation spectroscopy will also be presented. This is another method that can be used to allow bands in the NIR spectrum to be resolved and assigned to characteristic absorbances in the MIR spectrum. The principle of this method is to detect regions in both spectra (NIR and MIR) that change simultaneously as sugar content varies.

Outer product analysis of electronic nose and visible spectra: application to the measurement of peach fruit characteristics

Visual aspect and aroma are among the most important features of fruit that determine consumer preferences. Electronic nose and spectroscopic techniques have shown positive results in evaluating some basic analytical parameters of fruit and global features such as the cultivar. In this paper, we illustrate and discuss a study aimed at evaluating the improvement derived by the fusion of visible spectra and electronic nose data. These experiments were performed on a population of yellow peaches belonging to two cultivars. Each sample was measured by visible optical spectroscopy and by electronic nose. In addition, a number of reference parameters were also measured by conventional destructive methodologies. Collected data were analysed individually and then fused together in order to classify the two cultivars and to estimate the reference parameters. Data fusion was performed building the outer product matrix for each measurement. The set of matrices was then successively unfolded and analysed by conventional chemometrics tools. Results were improved using outer products, for instance in classification average percentage errors of 25, 10, and 7 for electronic nose, spectra, and outer product, respectively was achieved. Regression analysis provides the evidence of a substantial orthogonal appearance of the datasets, which offer former hidden information on fruit classification.

Continuous symmetry measures: Finding the closest C2 -symmetric object or closest reflection-symmetric object using unit quaternions.

A closed-form solution is provided for the problem of finding the closest reflection-symmetric object, and its relation to the problems of finding the closest achiral object, the closest inversion-symmetric object, and the closest projection plane is discussed. The key to the solution is reducing this problem to the problem of finding the best (least squares) c2 rotation between two sets of points, in addition to solving the problem of finding the closest C2 -symmetric object. The solution is derived using the quaternion representation of rotation. It is shown that calculation of the best c2 rotation can be reduced to the diagonalization of the outer product matrix of the pair (between the two sets). ©1999 John Wiley & Sons, Inc. J Comput Chem 20: 772-780, 1999.

Adaptive linear interference suppression for packet DS‐CDMA

AbstractThe performance of adaptive linear interference suppression is studied in the context of packet DS‐CDMA. A multi‐cell system is assumed with stochastic arrivals and departures of asynchronous users, and additive Gaussian noise as the only channel impairment. Interference suppression is achieved with a tapped‐delay line filter, where the filter spans a single symbol interval. Adaptive algorithms considered include the stochastic gradient (LMS), exponentially‐weighted Least Squares (LS), block LS, and a reduced‐rank LS algorithm. The reduced‐rank LS algorithm first projects the received signal onto a signal sub‐space spanned by eigenvectors of the averaged outer product matrix of received vectors. The purpose of the projection is to eliminate low‐level background interference and noise. Both decision‐directed and blind algorithms, which do not require a training sequence, are compared. Computer simulation is used to obtain error rates as a function of traffic load, and algorithm and system parameters (including timing offset). Results indicate that the adaptive algorithms offer a significant increase in capacity (nearly a factor of two at moderate error rates), and are insensitive to variations in received power over the user population.

Alien attractors and memory annihilation of structured sets in Hopfield networks

This paper considers the encoding of structured sets into Hopfield associative memories. A structured set is a set of vectors with equal Hamming distance h from one another, and its centroid is an external vector that has distance h/2 from every vector of the set. Structured sets having centroids are not infrequent. When such a set is encoded into a noiseless Hopfield associative memory using a bipolar outer-product connection matrix, and the network operates with synchronous neuronal update, the memory of all encoded vectors is annihilated even for sets with as few as three vectors in dimension n>5 (four for n=5). In such self-annihilating structured sets, the centroid emerges as a stable attractor. We call it an alien attractor. For canonical structured sets, self-annihilation takes place only if h<n/2. Self-annihilation does not occur and alien attractors do not emerge in dimensions less than five.

Fast calculation of the Choi-Williams time-frequency distribution

The method presented allows faster calculation of any time-frequency distribution with a kernel that can be formulated in the time-lag plane. Specific examples are the Wigner and Choi-Williams distributions. The Choi-Williams distribution (CWD) uses an exponential kernel in the generalized class of bilinear time-frequency distributions to achieve a reduction in the cross-term components of the distribution. Matrix manipulations provide an intuitive approach and, when combined with parallel processing, improve the processing speed to allow real-time calculations of the CWD. The use of an outer product matrix with a weighting matrix is particularly useful when evaluating different weighting parameters. For any given signal, the outer product matrix needs to be calculated just once. The various weighting matrices can be stored and used with any signal when needed. Parallel processing architectures allow implementation of the algorithm with speeds that are appropriate for real-time, running window calculations. >

One-dimensional optical correlator based on outer product architecture

Optical correlators of 1D signals implemented with anamorphic optical systems are described. The outer product matrix of two input signals is calculated optically and the elements of the matrix are summed up along the lines to the diagonal direction so as to obtain the cross-correlation of the signals. An experimental verification of the concept is presented.