Multi-way Data Research Articles

Computation of Outer Inverse of Tensors Based on t‐Product

ABSTRACTTensor computations play an essential role in various fields of science and engineering, including multiway data analysis. In this study, we established a few basic properties of the range and null space of a tensor by using block circulant matrices and a discrete Fourier matrix. We then discuss the outer inverse of the tensors based on ‐product with a prescribed range and kernel of third‐order tensors. We address the relation of this outer inverse with other generalized inverses, such as the Moore–Penrose inverse, group inverse, and Drazin inverse. In addition, we present a few algorithms for computing the outer inverses of the tensors. In particular, a ‐QR decomposition based algorithm was developed to compute outer inverses. It is well known that the confidentiality of information transmitted through the virtual world grows exponentially, and color image and video security have become a significant concern when communicating over the internet. As an application, a ‐QR decomposition based algorithm was demonstrated for concealing secret color images and videos.

PARAFAC under non-negativity constraint is adapted to recover the underlying Beer-Lambert law of the excitation-emission fluorescence matrix measurements acquired from analyte-triggered semiconductor QDs photoluminescence modulation. When and why?

BackgroundAnalyte-triggered semiconductor quantum dots (QDs) modulation in the presence of non-consistently responsive fluorescent species represents a challenging analytical issue in concrete multi-way data handling. QDs with heterogeneous sizes and/or uneven distribution of functional moieties on their surfaces exhibit significant fluctuations in the fluorescent response components, known as chemical rank, across different excitation/emission modes. This phenomenon may lead to a substantial deviation from the proportionality prescribed by Beer-Lambert law. Nonetheless, even in the presence of such deviation, a multi-way model may be successfully selected after determining a proper chemical rank in a QDs system. ResultsWe show that in a valid PARAllel FACtor (PARAFAC) model under properly determined chemical rank, meaningfully resolved pure spectral profiles can be reached for each fluorescent responsive constituent in the original excitation-emission fluorescence matrix (EEFM) measurements. This was thoroughly illustrated by applying PARAFAC trilinear decomposition of a three-way data array of two distinct datasets acquired from semiconductor QDs sensing systems with low-rank trilinear assumption. The first dataset, presented here for the first time, comprises EEFM measurements of the ligand-driven quenching of thiomalic acid (TMA)-capped AgInS2 (AIS) QDs by vomitoxin. The second dataset, employed for illustrative purposes, comprises EEFM measurements of the quenching, via cation bridging, of glutathione (GSH)-capped CdTe QDs by Pb(II). The results of this study enabled the determination of vomitoxin at a ppb level in real samples of fish feeds, showcasing the efficacy of the PARAFAC model in resolving spectral signatures (loadings) and pure concentration profiles (scores). SignificancePARAFAC under a properly examined chemical rank can be easily adapted for retrieval the underlying Beer-Lambert law of the original EEFM measurements with a low-rank trilinear structure through the chemically meaningful information either when (i) no deviation of Beer-Lambert law was observed as deeply discussed in connection with the dataset acquired from vomitoxin-driven molecular sensing through TMA-capped AIS QDs, or when (ii) substantial deviations of the Beer-Lambert law are evident, as discussed in connection with the dataset collected from sensing ionic species through Pb(II) bridging of GSH-capped CdTe QDs.

G-CovSel: Covariance oriented variable clustering

Dimensionality reduction is an essential step in the processing of analytical chemistry data. When this reduction is carried out by variable selection, it can enable the identification of biochemical pathways. CovSel has been developed to meet this requirement, through a parsimonious selection of non-redundant variables. This article presents the g-CovSel method, which modifies the CovSel algorithm to produce highly complementary groups containing highly correlated variables. This modification requires the theoretical definition of the groups' construction and of the deflation of the data with respect to the selected groups. Two applications, on two extreme case studies, are presented. The first, based on near-infrared spectra related to four chemicals, demonstrates the relevance of the selected groups and the method's ability to handle highly correlated variables. The second, based on genomic data, demonstrates the method's ability to handle very highly multivariate data. Most of the groups formed can be interpreted from a functional point of view, making g-CovSel a tool of choice for biomarker identification in omics. Further work will be carried out to generalize g-CovSel to multi-block and multi-way data.

Probabilistic PARAFAC2.

The Parallel Factor Analysis 2 (PARAFAC2) is a multimodal factor analysis model suitable for analyzing multi-way data when one of the modes has incomparable observation units, for example, because of differences in signal sampling or batch sizes. A fully probabilistic treatment of the PARAFAC2 is desirable to improve robustness to noise and provide a principled approach for determining the number of factors, but challenging because direct model fitting requires that factor loadings be decomposed into a shared matrix specifying how the components are consistently co-expressed across samples and sample-specific orthogonality-constrained component profiles. We develop two probabilistic formulations of the PARAFAC2 model along with variational Bayesian procedures for inference: In the first approach, the mean values of the factor loadings are orthogonal leading to closed form variational updates, and in the second, the factor loadings themselves are orthogonal using a matrix Von Mises-Fisher distribution. We contrast our probabilistic formulations to the conventional direct fitting algorithm based on maximum likelihood on synthetic data and real fluorescence spectroscopy and gas chromatography-mass spectrometry data showing that the probabilistic formulations are more robust to noise and model order misspecification. The probabilistic PARAFAC2, thus, forms a promising framework for modeling multi-way data accounting for uncertainty.

MacroPARAFAC for handling rowwise and cellwise outliers in incomplete multiway data

Multiway data extend two-way matrices into higher-dimensional tensors, often explored through dimensional reduction techniques. In this paper, we study the Parallel Factor Analysis (PARAFAC) model for handling multiway data, representing it more compactly through a concise set of loading matrices and scores. We assume that the data may be incomplete and could contain both rowwise and cellwise outliers, signifying cases that deviate from the majority and outlying cells dispersed throughout the data array. To address these challenges, we present a novel algorithm designed to robustly estimate both loadings and scores. Additionally, we introduce an enhanced outlier map to distinguish various patterns of outlying behavior. Through simulations and the analysis of fluorescence Excitation-Emission Matrix (EEM) data, we demonstrate the robustness of our approach. Our results underscore the effectiveness of diagnostic tools in identifying and interpreting unusual patterns within the data.

Hypergraph regularized nonnegative triple decomposition for multiway data analysis

Tucker decomposition is widely used for image representation, data reconstruction, and machine learning tasks, but the calculation cost for updating the Tucker core is high. Bilevel form of triple decomposition (TriD) overcomes this issue by decomposing the Tucker core into three low-dimensional third-order factor tensors and plays an important role in the dimension reduction of data representation. TriD, on the other hand, is incapable of precisely encoding similarity relationships for tensor data with a complex manifold structure. To address this shortcoming, we take advantage of hypergraph learning and propose a novel hypergraph regularized nonnegative triple decomposition for multiway data analysis that employs the hypergraph to model the complex relationships among the raw data. Furthermore, we develop a multiplicative update algorithm to solve our optimization problem and theoretically prove its convergence. Finally, we perform extensive numerical tests on six real-world datasets, and the results show that our proposed algorithm outperforms some state-of-the-art methods.

Data-Driven Version of Multiway Soft Independent Modeling of Class Analogy (N-Way DD-SIMCA): Theory and Application.

One-class classification (OCC) is discussed in the framework of the measurement and processing of multiway data. Data-driven soft independent modeling of class analogy (DD-SIMCA) is applied in the following formats: (1) multiblock and (2) Tucker 3 N-way SIMCA, which are shown to be useful tools for solving classification tasks. A new decision rule for N-way DD-SIMCA is adopted based on the conventional two-way DD-SIMCA model. Multiblock SIMCA is shown to be useful for variable selection, and Tucker 3 SIMCA to select the optimal model complexity when applying multiway data decomposition and to assess the role of individual samples in the classification model. Both approaches, together with the two-way DD-SIMCA version applied to the unfolded data, are compared regarding the analysis of an experimental data set including genuine and adulterated blueberry extract samples. The latter were employed to produce matrix spectral-time data matrices per sample within a flow injection system, taking advantage of the spectral changes in the sample constituents as a function of the pH of the carrier phase. The need to employ the Tucker 3 model instead of a trilinear decomposition is supported by a discussion on the lack of the trilinearity property of the studied data.

Stable tensor neural networks for efficient deep learning.

Learning from complex, multidimensional data has become central to computational mathematics, and among the most successful high-dimensional function approximators are deep neural networks (DNNs). Training DNNs is posed as an optimization problem to learn network weights or parameters that well-approximate a mapping from input to target data. Multiway data or tensors arise naturally in myriad ways in deep learning, in particular as input data and as high-dimensional weights and features extracted by the network, with the latter often being a bottleneck in terms of speed and memory. In this work, we leverage tensor representations and processing to efficiently parameterize DNNs when learning from high-dimensional data. We propose tensor neural networks (t-NNs), a natural extension of traditional fully-connected networks, that can be trained efficiently in a reduced, yet more powerful parameter space. Our t-NNs are built upon matrix-mimetic tensor-tensor products, which retain algebraic properties of matrix multiplication while capturing high-dimensional correlations. Mimeticity enables t-NNs to inherit desirable properties of modern DNN architectures. We exemplify this by extending recent work on stable neural networks, which interpret DNNs as discretizations of differential equations, to our multidimensional framework. We provide empirical evidence of the parametric advantages of t-NNs on dimensionality reduction using autoencoders and classification using fully-connected and stable variants on benchmark imaging datasets MNIST and CIFAR-10.

A multiway spectral analysis model to monitor the kinetic chlorination reaction of caffeine and determine its content in commercial liquid and solid drinks.

A multiway data analysis model, namely parallel factor analysis (PARAFAC) was proposed to decompose a three-way array of second-order kinetic UV measurements, for the chlorination reaction of caffeine with NaOCl, into a set of the spectra, time, and concentration matrices. The multiway resolution provided the simultaneous estimation of spectral, kinetic, and quantitative analysis of caffeine. The ability of the PARAFAC tool was checked by analyzing the validation samples in the presence of interferences. The added recovery and relative standard deviations for caffeine in the spiked samples were calculated as 99.1%-99.5% and 0.52%-1.34% for Iced Coffee Black liquid coffee (ICB), 99.5%-103.0% and 0.42%-1.03% for Jacobs Monarch Gold 100% Instant Coffee (JMG) and 99.5%-101.4% and 0.11%-0.13% for Çaykur Black Filter (Süzen) Bag Tea (BTB). Caffeine in commercial drinks was analyzed using the concentration matrices of the PARAFAC application. The PARAFAC results were statistically compared to those obtained by the developed UPLC method.

Graph-Regularized Tensor Regression: A Domain-Aware Framework for Interpretable Modeling of Multiway Data on Graphs.

Modern data analytics applications are increasingly characterized by exceedingly large and multidimensional data sources. This represents a challenge for traditional machine learning models, as the number of model parameters needed to process such data grows exponentially with the data dimensions, an effect known as the curse of dimensionality. Recently, tensor decomposition (TD) techniques have shown promising results in reducing the computational costs associated with large-dimensional models while achieving comparable performance. However, such tensor models are often unable to incorporate the underlying domain knowledge when compressing high-dimensional models. To this end, we introduce a novel graph-regularized tensor regression (GRTR) framework, whereby domain knowledge about intramodal relations is incorporated into the model in the form of a graph Laplacian matrix. This is then used as a regularization tool to promote a physically meaningful structure within the model parameters. By virtue of tensor algebra, the proposed framework is shown to be fully interpretable, both coefficient-wise and dimension-wise. The GRTR model is validated in a multiway regression setting and compared against competing models and is shown to achieve improved performance at reduced computational costs. Detailed visualizations are provided to help readers gain an intuitive understanding of the employed tensor operations.

ALGORITHMIC SUBSAMPLING UNDER MULTIWAY CLUSTERING

This paper proposes a novel method of algorithmic subsampling (data sketching) for multiway cluster-dependent data. We establish a new uniform weak law of large numbers and a new central limit theorem for multiway algorithmic subsample means. We show that algorithmic subsampling allows for robustness against potential degeneracy, and even non-Gaussian degeneracy, of the asymptotic distribution under multiway clustering at the cost of efficiency and power loss due to algorithmic subsampling. Simulation studies support this novel result, and demonstrate that inference with algorithmic subsampling entails more accuracy than that without algorithmic subsampling. We derive the consistency and the asymptotic normality for multiway algorithmic subsampling generalized method of moments estimator and for multiway algorithmic subsampling M-estimator. We illustrate with an application to scanner data for the analysis of differentiated products markets.

Bayesian predictive modeling of multi-source multi-way data

A Bayesian approach to predict a continuous or binary outcome from data that are collected from multiple sources with a multi-way (i.e., multidimensional tensor) structure is described. As a motivating example, molecular data from multiple 'omics sources, each measured over multiple developmental time points, as predictors of early-life iron deficiency (ID) in a rhesus monkey model are considered. The method uses a linear model with a low-rank structure on the coefficients to capture multi-way dependence and model the variance of the coefficients separately across each source to infer their relative contributions. Conjugate priors facilitate an efficient Gibbs sampling algorithm for posterior inference, assuming a continuous outcome with normal errors or a binary outcome with a probit link. Simulations demonstrate that the model performs as expected in terms of misclassification rates and correlation of estimated coefficients with true coefficients, with large gains in performance by incorporating multi-way structure and modest gains when accounting for differing signal sizes across the different sources. Moreover, it provides robust classification of ID monkeys for the motivating application.

Graph-Regularized Non-Negative Tensor-Ring Decomposition for Multiway Representation Learning.

Tensor-ring (TR) decomposition is a powerful tool for exploiting the low-rank property of multiway data and has been demonstrated great potential in a variety of important applications. In this article, non-negative TR (NTR) decomposition and graph-regularized NTR (GNTR) decomposition are proposed. The former equips TR decomposition with the ability to learn the parts-based representation by imposing non-negativity on the core tensors, and the latter additionally introduces a graph regularization to the NTR model to capture manifold geometry information from tensor data. Both of the proposed models extend TR decomposition and can be served as powerful representation learning tools for non-negative multiway data. The optimization algorithms based on an accelerated proximal gradient are derived for NTR and GNTR. We also empirically justified that the proposed methods can provide more interpretable and physically meaningful representations. For example, they are able to extract parts-based components with meaningful color and line patterns from objects. Extensive experimental results demonstrated that the proposed methods have better performance than state-of-the-art tensor-based methods in clustering and classification tasks.

Zero-truncated Poisson regression for sparse multiway count data corrupted by false zeros

Abstract We propose a novel statistical inference methodology for multiway count data that is corrupted by false zeros that are indistinguishable from true zero counts. Our approach consists of zero-truncating the Poisson distribution to neglect all zero values. This simple truncated approach dispenses with the need to distinguish between true and false zero counts and reduces the amount of data to be processed. Inference is accomplished via tensor completion that imposes low-rank tensor structure on the Poisson parameter space. Our main result shows that an $N$-way rank-$R$ parametric tensor $\boldsymbol{\mathscr{M}}\in (0,\infty )^{I\times \cdots \times I}$ generating Poisson observations can be accurately estimated by zero-truncated Poisson regression from approximately $IR^2\log _2^2(I)$ non-zero counts under the nonnegative canonical polyadic decomposition. Our result also quantifies the error made by zero-truncating the Poisson distribution when the parameter is uniformly bounded from below. Therefore, under a low-rank multiparameter model, we propose an implementable approach guaranteed to achieve accurate regression in under-determined scenarios with substantial corruption by false zeros. Several numerical experiments are presented to explore the theoretical results.

A novel second-order calibration algorithm for processing fluorescence data with scattering: Three-direction resection ATLD

The light scattering is common during fluorescence measurements, such as first- and second-order Rayleigh and Raman scatterings, which are not conducive to the decomposition of the multi-way data because it destroys the multi-linear structure. This paper proposed two second-order calibration algorithms, including three-direction resection alternating trilinear decomposition (TDR-ATLD) and three-direction resection self-weight alternating trilinear decomposition (TDR-SWATLD), which are based on an improved trilinear decomposition model for directly decomposition of the three-way fluorescence data with scattering regions removed. Through the simulation data, the proposed algorithms were compared with other three algorithms (ATLD, Interpolation-ATLD, TDR-PARAFAC). The calculation results of three algorithms (TDR-PARAFAC, TDR-ATLD and TDR-SWATLD) are better at small or large scattering widths. Both TDR-ATLD and TDR-SWATLD are insensitive to the number of components and the performance of TDR-ATLD had little fluctuation with different noise levels among the five algorithms. Additionally, TDR-ATLD and TDR-SWATLD are better than other algorithms when dealing with collinear data. The proposed algorithms were also used in two sets of real data, compared with other two algorithms (Interpolation-ATLD and TDR-PARAFAC) for dealing with scattering, the proposed algorithms could deal with fluorescence data with scattering and provided satisfactory results and TDR-ATLD has faster convergence. In general, TDR-ATLD has greater potential and performs better in processing fluorescence data with scattering.

An upgrade of MVC2, a MATLAB graphical user interface for second-order multivariate calibration: Beyond trilinear models

In the present report, an upgrade of a MATLAB graphical user interface (GUI) toolbox for implementing second-order multivariate calibration models is presented, named Multivariate Calibration 2 (MVC2). Due to the advances in the chemometric field, this software has undergone numerous changes since its last release in 2009. The new freely available MVC2 incorporates novel features and models that make it a very practical tool for data processing. Overall, the current version presents different types of algorithms, designed to provide specific degrees of model flexibility for different multi-way data scenarios: in addition to trilinear decomposition (TLD) and residual bilinearization (RBL) models, which were previously included in the 2009 version, parallel factor analysis 2 (PARAFAC2) and multivariate curve resolution (MCR) methods are now available. All these chemometric methodologies are integrated in the software for quantitative purposes, and if needed, they permit to exploit the so-called analytical second-order advantage. The program offers practical tools for data loading and visualization, model development and validation, and prediction in unknown samples. The software output also includes a comprehensive report on the analytical figures of merit (AFOMs) applicable to second-order calibration models. Specifically, a tool was incorporated for the diagnosis of the remaining rotational ambiguity in the MCR solutions. In the present report, the new features of MVC2 are illustrated through simulated examples and real calibration datasets.

Does multi-way, long-range chromatin contact data advance 3D genome reconstruction?

BackgroundMethods for inferring the three-dimensional (3D) configuration of chromatin from conformation capture assays that provide strictly pairwise interactions, notably Hi-C, utilize the attendant contact matrix as input. More recent assays, in particular split-pool recognition of interactions by tag extension (SPRITE), capture multi-way interactions instead of solely pairwise contacts. These assays yield contacts that straddle appreciably greater genomic distances than Hi-C, in addition to instances of exceptionally high-order chromatin interaction. Such attributes are anticipated to be consequential with respect to 3D genome reconstruction, a task yet to be undertaken with multi-way contact data. However, performing such 3D reconstruction using distance-based reconstruction techniques requires framing multi-way contacts as (pairwise) distances. Comparing approaches for so doing, and assessing the resultant impact of long-range and multi-way contacts, are the objectives of this study.ResultsWe obtained 3D reconstructions via multi-dimensional scaling under a variety of weighting schemes for mapping SPRITE multi-way contacts to pairwise distances. Resultant configurations were compared following Procrustes alignment and relationships were assessed between associated Procrustes root mean square errors and key features such as the extent of multi-way and/or long-range contacts. We found that these features had surprisingly limited influence on 3D reconstruction, a finding we attribute to their influence being diminished by the preponderance of pairwise contacts.ConclusionDistance-based 3D genome reconstruction using SPRITE multi-way contact data is not appreciably affected by the weighting scheme used to convert multi-way interactions to pairwise distances.

Multiplex Transformed Tensor Decomposition for Multidimensional Image Recovery.

Low-rank tensor completion aims to recover the missing entries of multi-way data, which has become popular and vital in many fields such as signal processing and computer vision. It varies with different tensor decomposition frameworks. Compared with matrix SVD, recently emerging transform t-SVD can better characterize the low-rank structure of order-3 data. However, it suffers from rotation sensitivity, and dimensional limitation (i.e. only effective for order-3 tensors). To alleviate these deficiencies, we develop a novel multiplex transformed tensor decomposition (MTTD) framework, which can characterize the global low-rank structure along all modes for any order-N tensor. Based on MTTD, we propose a related multi-dimensional square model for low-rank tensor completion. Besides, a total variation term is also introduced to utilize the local piecewise smoothness of the tensor data. The classic alternating direction method of multipliers is used to solve the convex optimization problems. For performance testing, we choose three linear invertible transforms including FFT, DCT, and a group of unitary transform matrices for our proposed methods. The simulated and real-data experiments demonstrate the superior recovery accuracy and computational efficiency of our method compared with state-of-the-art ones.

EA-ADMM: noisy tensor PARAFAC decomposition based on element-wise average ADMM

Tensor decomposition is widely used to exploit the internal correlation in multi-way data analysis and process for communications and radar systems. As one of the main tensor decomposition methods, CANDECOMP/PARAFAC decomposition has advantages of uniqueness and interpretation properties which are significant in practical applications. However, traditional decomposition method is sensitive to both predefined rank and noise that results in inaccurate tensor decomposition. In this paper, we propose a improved algorithm called the Element-wise Average Alternating Direction Method of Multipliers by minimizing the sum of all factors’ trace norm and the noise variance. Our algorithm could overcome the dependence on predefined rank in traditional decomposition algorithms and alleviate the impact of noise. Moreover, this algorithm can be transferred to solve the problem of tensor completion conveniently. The simulation results show that our proposed algorithm could decompose the noisy tensor to the factors with above 90% similarity in various SNR and also interpolate the incomplete tensor with higher similar coefficient and lower relative reconstruction error when the missing rate is less than 0.5.

Multiview PCA: A Methodology of Feature Extraction and Dimension Reduction for High-Order Data.

Facing with rapidly increasing demands for analyzing high-order data or multiway data, feature-extracting methods become imperative for analysis and processing. The traditional feature-extracting methods, however, either need to overly vectorize the data and smash the original structure hidden in data, such as PCA and PCA-like methods, which is unfavorable to the data recovery, or cannot eliminate the redundant information very well, such as tucker decomposition (TD) and TD-like methods. To overcome these limitations, we propose a more flexible and more powerful tool, called the multiview principal components analysis (Multiview-PCA) in this article. By segmenting a random tensor into equal-sized subarrays called sections and maximizing variations caused by orthogonal projections of these sections, the Multiview-PCA finds principal components in a parsimonious and flexible way. In so doing, two new operations on tensors, the S -direction inner/outer product, are introduced to formulate tensor projection and recovery. With different segmentation ways characterized by section depth and direction, the Multiview-PCA can be implemented many times in different ways, which defines the sequential and global Multiview-PCA, respectively. These multiple Multiview-PCA take the PCA and PCA-like, and TD and TD-like as the special cases, which correspond to the deepest section depth and the shallowest section depth, respectively. We propose an adaptive depth and direction selection algorithm for the implementation of Multiview-PCA. The Multiview-PCA is then tested in terms of subspace recovery ability, compression ability, and feature extraction performance when applied to a set of artificial data, surveillance videos, and hyperspectral imaging data. All numerical results support the flexibility, effectiveness, and usefulness of Multiview-PCA.