Tensor Representation Research Articles

Tensor Representations for Action Recognition.

Human actions in video sequences are characterized by the complex interplay between spatial features and their temporal dynamics. In this paper, we propose novel tensor representations for compactly capturing such higher-order relationships between visual features for the task of action recognition. We propose two tensor-based feature representations, viz. (i) sequence compatibility kernel (SCK) and (ii) dynamics compatibility kernel (DCK). SCK builds on the spatio-temporal correlations between features, whereas DCK explicitly models the action dynamics of a sequence. We also explore generalization of SCK, coined SCK ⊕, that operates on subsequences to capture the local-global interplay of correlations, which can incorporate multi-modal inputs e.g., skeleton 3D body-joints and per-frame classifier scores obtained from deep learning models trained on videos. We introduce linearization of these kernels that lead to compact and fast descriptors. We provide experiments on (i) 3D skeleton action sequences, (ii) fine-grained video sequences, and (iii) standard non-fine-grained videos. As our final representations are tensors that capture higher-order relationships of features, they relate to co-occurrences for robust fine-grained recognition (Lin, 2017), (Koniusz, 2018). We use higher-order tensors and so-called Eigenvalue Power Normalization (EPN) which have been long speculated to perform spectral detection of higher-order occurrences (Koniusz, 2013), (Koniusz, 2017), thus detecting fine-grained relationships of features rather than merely count features in action sequences. We prove that a tensor of order r, built from Z* dimensional features, coupled with EPN indeed detects if at least one higher-order occurrence is 'projected' into one of its [Formula: see text] subspaces of dim. r represented by the tensor, thus forming a Tensor Power Normalization metric endowed with [Formula: see text] such 'detectors'.

Static coded illumination strategies for low-dose x-ray material decomposition.

Static coded aperture x-ray tomography was introduced recently where a static illumination pattern is used to interrogate an object with a low radiation dose, from which an accurate 3D reconstruction of the object can be attained computationally. Rather than continuously switching the pattern of illumination with each view angle, as traditionally done, static code computed tomography (CT) places a single pattern for all views. The advantages are many, including the feasibility of practical implementation. This paper generalizes this powerful framework to develop single-scan dual-energy coded aperture spectral tomography that enables material characterization at a significantly reduced exposure level. Two sensing strategies are explored: rapid kV switching with a single-static block/unblock coded aperture, and coded apertures with non-uniform thickness. Both systems rely on coded illumination with a plurality of x-ray spectra created by kV switching or 3D coded apertures. The structured x-ray illumination is projected through the objects of interest and measured with standard x-ray energy integrating detectors. Then, based on the tensor representation of projection data, we develop an algorithm to estimate a full set of synthesized measurements that can be used with standard reconstruction algorithms to accurately recover the object in each energy channel. Simulation and experimental results demonstrate the effectiveness of the proposed cost-effective solution to attain material characterization in low-dose dual-energy CT.

Note on the permutation group associated to E-polynomials

This is a continuation of our project which focuses on E-polynomials and the related combinatorics. A pair of groups appearing in the definition of E-polynomials yields the permutation group. In this paper, we determine the multi-matrix structures of the centralizer algebras of the tensor representations of this permutation group.

Logarithmic Schatten- p Norm Minimization for Tensorial Multi-View Subspace Clustering.

The low-rank tensor could characterize inner structure and explore high-order correlation among multi-view representations, which has been widely used in multi-view clustering. Existing approaches adopt the tensor nuclear norm (TNN) as a convex approximation of non-convex tensor rank function. However, TNN treats the different singular values equally and over-penalizes the main rank components, leading to sub-optimal tensor representation. In this paper, we devise a better surrogate of tensor rank, namely the tensor logarithmic Schatten- p norm ([Formula: see text]N), which fully considers the physical difference between singular values by the non-convex and non-linear penalty function. Further, a tensor logarithmic Schatten- p norm minimization ([Formula: see text]NM)-based multi-view subspace clustering ([Formula: see text]NM-MSC) model is proposed. Specially, the proposed [Formula: see text]NM can not only protect the larger singular values encoded with useful structural information, but also remove the smaller ones encoded with redundant information. Thus, the learned tensor representation with compact low-rank structure will well explore the complementary information and accurately characterize the high-order correlation among multi-views. The alternating direction method of multipliers (ADMM) is used to solve the non-convex multi-block [Formula: see text]NM-MSC model where the challenging [Formula: see text]NM problem is carefully handled. Importantly, the algorithm convergence analysis is mathematically established by showing that the sequence generated by the algorithm is of Cauchy and converges to a Karush-Kuhn-Tucker (KKT) point. Experimental results on nine benchmark databases reveal the superiority of the [Formula: see text]NM-MSC model.

An Unsupervised Laplacian Pyramid Network for Radiometrically Accurate Data Fusion of Hyperspectral and Multispectral Imagery

Improving the spatial resolution of hyperspectral images (HSIs) has traditionally been an important topic in the field of remote sensing. Many approaches have been proposed based on various theories, including component substitution, multiresolution analysis, spectral unmixing, Bayesian probability, and tensor representation. However, these methods have some common disadvantages such that their performance degrades dramatically as the up-scale ratio increases, and they have little concern for the per-pixel radiometric accuracy of the sharpened image. Moreover, many learning-based methods have been proposed through decades of innovations, but most of them require a large set of training pairs, which is unpractical for many real problems. To solve these problems, we propose a stable hyperspectral sharpening method based on the Laplacian pyramid and the generative convolutional neural network (CNN), which achieves superior radiometric accuracy of the sharpened data in different up-scale ratios based on one single input pair. First, with a low-resolution HSI (LR-HSI) and high-resolution multispectral image (HR-MSI) pair, the preliminary high-resolution HSI (HR-HSI) is calculated via linear regression. Then, the high-frequency details of the preliminary HR-HSI are estimated via the subtraction between it and the CNN-generated-blurry version. By injecting the details to the output of the generative CNN with the LR-HSI as input, the final HR-HSI is obtained. Nine different state-of-the-art sharpening methods are chosen as our baselines, and three different datasets with different scene content are tested. Furthermore, the target detection method, the adaptive coherence estimator (ACE), is conducted on the reconstructed HR-HSI to evaluate the per-pixel radiometric accuracy. The results demonstrate that the proposed method has the best and the most stable performance in terms of spectral and spatial accuracies.

Explicit Algebraic Stress Model for Three-Dimensional Turbulent Buoyant Flows Derived Using Tensor Representation

An explicit algebraic stress model (EASM) has been formulated for two-dimensional turbulent buoyant flows using a five-term tensor representation in a prior study. The derivation was based on partitioning the buoyant flux tensor into a two-dimensional and a three-dimensional component. The five-term basis was formed with the two-dimensional component of the buoyant flux tensor. As such, the derived EASM is limited to two-dimensional flows only. In this paper, a more general approach using a seven-term representation without partitioning the buoyant flux tensor is used to derive an EASM valid for two- and three-dimensional turbulent buoyant flows. Consequently, the basis tensors are formed with the fully three-dimensional buoyant flux tensor. The derived EASM has the two-dimensional flow as a special case. The matrices and the representation coefficients are further simplified using a four-term representation. When this four-term representation model is applied to calculate two-dimensional homogeneous buoyant flows, the results are essentially identical with those obtained previously using the two-dimensional component of the buoyant flux tensor. Therefore, the present approach leads to a more general EASM formulation that is equally valid for two- and three-dimensional turbulent buoyant flows.

A Comprehensive Tensor Framework for the Clustering of Hyperspectral Paper Data With an Application to Forensic Document Analysis

In forensic document analysis, the authenticity of a document must be properly checked in the context of suspected forgery. Hyperspectral Imaging (HSI) is a non-invasive way of detecting fraudulent papers in a multipage document. The occurrence of a forged paper in a multi-page document may have a substantial difference from rest of the papers in its age, type, color, texture, and so on. Each pixel in an HSI data can be used as the material fingerprint for the spatial point it corresponds to. Hence, hyperspectral data of paper samples made of the same substance have similar characteristics and can be grouped into a single cluster. Similarly, paper samples made of different substances have different spectral properties. This paper relies on this heuristic and proposes a tensor based clustering framework for hyperspectral paper data, with an application to detect the forged papers in multi-page documents. Information embedded in the hyperspectral patches of the papers to be clustered is arranged into individual lateral slices of a third-order tensor in this framework. Further, this work employs the self-expressiveness property of submodules and an objective function is formulated to extract self-expressive representation tensor with low multirank and f-diagonal structure. Objective function of the proposed method incorporates <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$l_{\frac {1}{2}}$ </tex-math></inline-formula> -induced Tensor Nuclear Norm (TNN) and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$l_{\frac {1}{2}}$ </tex-math></inline-formula> regularization to impart better low rankness and f-diagonal structure to the representation tensor. Experimental results of the proposed method were compared to the state-of-the-art subspace clustering approaches. The results demonstrate improved performance of the proposed method over the existing clustering algorithms.

FGANet: fNIRS-Guided Attention Network for Hybrid EEG-fNIRS Brain-Computer Interfaces.

Non-invasive brain-computer interfaces (BCIs) have been widely used for neural decoding, linking neural signals to control devices. Hybrid BCI systems using electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) have received significant attention for overcoming the limitations of EEG- and fNIRS-standalone BCI systems. However, most hybrid EEG-fNIRS BCI studies have focused on late fusion because of discrepancies in their temporal resolutions and recording locations. Despite the enhanced performance of hybrid BCIs, late fusion methods have difficulty in extracting correlated features in both EEG and fNIRS signals. Therefore, in this study, we proposed a deep learning-based early fusion structure, which combines two signals before the fully-connected layer, called the fNIRS-guided attention network (FGANet). First, 1D EEG and fNIRS signals were converted into 3D EEG and fNIRS tensors to spatially align EEG and fNIRS signals at the same time point. The proposed fNIRS-guided attention layer extracted a joint representation of EEG and fNIRS tensors based on neurovascular coupling, in which the spatially important regions were identified from fNIRS signals, and detailed neural patterns were extracted from EEG signals. Finally, the final prediction was obtained by weighting the sum of the prediction scores of the EEG and fNIRS-guided attention features to alleviate performance degradation owing to delayed fNIRS response. In the experimental results, the FGANet significantly outperformed the EEG-standalone network. Furthermore, the FGANet has 4.0% and 2.7% higher accuracy than the state-of-the-art algorithms in mental arithmetic and motor imagery tasks, respectively.

Geometrical and physical properties of W2-symmetric and -recurrent manifolds

The authors discuss mainly that the Riemannian manifold Mn admitting a unit preserving circle field ? in the present paper. A sufficient and necessary condition is given that Riemannian manifold Mn is an Einstein manifold by imposing some conditions on W2 curvature tensor. Further, this paper obtains the algebra representation of curvature tensors of a W2-recurrent Riemannian manifold Mn given by R???? = 1/d2 [d?d?R?? ? d?d?R?? + d?d?R?? ? d?d?R??].

ДВУХПАРАМЕТРИЧЕСКИЙ МЕТОД ОПРЕДЕЛЕНИЯ КОЭФФИЦИЕНТА ИНТЕНСИВНОСТИ НАПРЯЖЕНИЙ KI ТРЕЩИНОПОДОБНЫХ ДЕФЕКТОВ МЕТОДОМ ГОЛОГРАФИЧЕСКОЙ ИНТЕРФЕРОМЕТРИИ

A new method has been developed for finding the stress intensity coefficient and the nominal stress σox for a plate with a crack-like (elliptical) defect. As an experimental basis, the patterns of bands of absolute path difference obtained on the basis of the method of holographic interferometry are taken. Using Favre ratios and approximate decomposition of stress components for the plane case, the stress intensity coefficient and the nominal stress σox are determined. The difference of the proposed method is in a more accurate representation of the stress tensor in the vicinity of the vertex of a crack like defect. Such representation allows one to take the geometry of the defect into account. The values of the correction function calculated by the proposed method in the formula for theoretical determining of the intensity factor have been higher than those obtained by previously used methods. This indicates a possible underestimation of the magnitude of the intensity factor when using the previously proposed methods. In addition to using more precise formulas for the stress tensor, the approach implies considering the nominal stress and the intensity factor as independent parameters. Full consideration of the crack geometry and loading features is very difficult from a mathematical point of view; however, this property of the method allows partially compensating for the simplifications of the stress tensor analytic expressions. Also, this method makes it possible to determine the main stresses and the intensity of stresses in the vicinity of the vertex of the defect. The obtained formulas agree well with experimental results.

A Novel Tensorial Scheme for EEG-Based Person Identification

Biometrics have attracted growing research interests as the information security and safety gain increasing attention to date. As a kind of important biomedical signal, Electroen-cephalogram (EEG) contains valuable information about identity, emotionality, personality, etc. Thus, automatically distinguishing the identities based on EEG is beneficial to the development of biometrics, forensics and informatics. Although deep learning has absorbed much research attention for the issue of EEG-based person identification, the performance enhancement of this methodology seems to have hit a bottleneck recently. Hence, by rethinking the problems haunting this issue, we plan to reinvigorate the conventional method pipeline, and put forward a novel and effective tensorial scheme away from the deep learning mainstream. Specifically, the proposed tensorial scheme extracts the effective tensorial representation from multi-channel EEG at first; then, the scheme performs the designed tensorial learning to improve the discriminability of the feature space; finally, the scheme carries out the devised tensorial measurement in the learned metric space for classification. Experimental results have demonstrated the superiority of proposed scheme over the related advanced approaches by means of the challenging benchmark databases DEAP, SEED and DREAMER.

Constrained Tensor Representation Learning for Multi-View Semi-Supervised Subspace Clustering

Multi-view subspace clustering is an effective method to partition data into their corresponding categories. Nevertheless, existing multi-view subspace clustering approaches generally operate in a purely unsupervised manner, while ignoring the valuable weakly supervised information that can be readily obtained in many practical applications. In this paper, we consider the weakly supervised form of sample pair constraints, and devote to promoting the performance of multi-view subspace clustering with the aid of such prior knowledge. To achieve this goal, inspired by the intrinsic block diagonal structure of ideal low-rank representation (LRR), we propose a novel regularization to integrate must-link, cannot-link and normalization constraints into a unified formulation. The proposed regularization can be regarded as a general description for sample pairwise constraints, and thus provides a flexible framework for multi-view semi-supervised subspace clustering task. Furthermore, we devise a contrained tensor representation learning (CTRL) model that takes advantage of our proposed regularization to facilitate the learning of the desired representation tensor. An efficient optimization algorithm based on alternating direction minimization strategy is carefully designed to solve the proposed CTRL model. Extensive experiments on eight challenging real-world datasets are conducted, and the results validate the effectiveness of our designed pairwise constraints regularization, as well as the superiority of the proposed CTRL model.

Hyperspectral Unmixing Using Nonlocal Similarity-Regularized Low-Rank Tensor Factorization

Recently, methods based on nonnegative tensor factorization (NTF), which benefits from the tensor representation of hyperspectral imagery (HSI) without any information loss, have attracted increasing attention. However, most existing methods fail to explore the internal spatial structure of data, resulting in low unmixing performance. Moreover, when the algorithm is optimized, the solution is unstable. In this article, a regularizer based on nonlocal tensor similarity is proposed, which can not only fully preserve the global information of HSI but also mine the internal information of data in the spatial domain. HSI is regarded as a 3-D tensor and is directly subjected to endmember extraction and abundance estimation. To fully explore the structural characteristics of data, we simultaneously use the local smoothing and low tensor rank prior of the data to constrain the unmixing model. First, several 4-D tensor groups can be obtained after the nonlocal similarity structure of HSI is learned. Subsequently, a low tensor rank prior is applied to each 4-D tensor, which can fully simulate the nonlocal similarity in the image. In addition, total variation (TV) is also used to explore the local spatial relationship of data, which can generate a smooth abundance map through edge preservation. The optimization is solved by the ADMM algorithm. Experiments on synthetic and real data illustrate the superiority of the proposed method.

Deep High-Order Tensor Convolutional Sparse Coding for Hyperspectral Image Classification

Most hyperspectral image (HSI) data exist in the form of tensor; the tensor representation preserves the potential spatial–spectral structure information compared with the vector representation, which can help improve the classification performance of HSI. In this article, a deep high-order tensor convolutional sparse coding (CSC) model is proposed, which can be used to train deep high-order filters. Based on the deep high-order tensor CSC model, a deep feature extraction network (DHTCSCNet) is constructed, which is used for feature extraction of HSIs. By combining the spectral–spatial feature and the features extracted by the proposed DHTCSCNet at each layer, a combined feature that incorporates shallow, deep, spectral, and spatial features can be obtained. Then, the graph-based learning (GSL) methods are used to classify the combined feature. Experimental results show that the DHTCSCNet can obtain better classification performance compared with other HSI classification methods.

Non-Local Robust Quaternion Matrix Completion for Large-Scale Color Image and Video Inpainting.

The image nonlocal self-similarity (NSS) prior refers to the fact that a local patch often has many nonlocal similar patches to it across the image and has been widely applied in many recently proposed machining learning algorithms for image processing. However, there is no theoretical analysis on its working principle in the literature. In this paper, we discover a potential causality between NSS and low-rank property of color images, which is also available to grey images. A new patch group based NSS prior scheme is proposed to learn explicit NSS models of natural color images. The numerical low-rank property of patched matrices is also rigorously proved. The NSS-based QMC algorithm computes an optimal low-rank approximation to the high-rank color image, resulting in high PSNR and SSIM measures and particularly the better visual quality. A new tensor NSS-based QMC method is also presented to solve the color video inpainting problem based on quaternion tensor representation. The numerical experiments on color images and videos indicate the advantages of NSS-based QMC over the state-of-the-art methods.

Tensor-Based Sparse Recovery Space-Time Adaptive Processing for Large Size Data Clutter Suppression in Airborne Radar

Sparse recovery space-time adaptive processing (SR-STAP) can achieve an ideal clutter suppression with very few training samples, however, its application faces two challenges: (i) severe gird mismatch effect; (ii) large time-resources requirement. In practice, a coarse space-time grids will bring a serious mismatch between the true clutter points and the divided grids, which leads to a significant performance degradation of clutter suppression. Although the high-resolution mesh can effectively reduce the grid mismatch effect, its cost is huge computational load. Thus, it is meaningful to reduce the large-scale dictionary operation complexity while maintaining suboptimal clutter suppression performance for SR-STAP when applying in real airborne radar system. This paper proposed a tensor-based SR-STAP scheme aims at large-scale dictionary application. In the proposed framework, traditional vector-based operations are replaced by their corresponding low-complexity tensor representation. As a result, a large-scale matrix operation can be degraded into multiple small-scale matrix calculation, thus the huge computational loading can be saved in recovery. A comparison of tensor-based SR-STAP and traditional vector-based SR-STAP in large-scale dictionary application is also exhaustive discussed here. Based on this framework, a tensor-based sparse Bayesian learning and its fast matrix-realization form are developed. A series of carefully designed numerical simulation and measurement experiments indicate that the significant advantages of the tensor-based SR-STAP whether in performance or computation loading.

Structure-Guaranteed SAR Imagery via Spatially-Variant Morphology Regularization in ADMM Manner

Conventional sparsity-driven synthetic aperture radar (SAR) imagery proceeds via &#x2113;₁ regularization, or named by least absolute shrinkage and selection operator (LASSO). However, followed by the enhanced sparse feature, structures of the scenes or targets of interests in weak scattering are easily lost. Therefore, it becomes difficult to make use of the high-resolution SAR data, even high costs have been paid for the resolution. In this paper, a novel structure-guaranteed SAR (SG-SAR) imaging algorithm is proposed by utilizing the morphology metric for the cluster feature of the scatterers of the scenes/targets of interests. By introducing structural prior in terms of morphology norm, the intended structure features can be highlighted via convex regularization. More specifically, to accommodate to complicated scenarios or targets, the structure element in the morphology regularizer is designed to be spatially variant under structure tensor representation. Different from conventional convex optimizations, the proposed SG-SAR algorithm is solved under alternating direction method of multipliers (ADMM), which is flexible to incorporate with the super-resolution imagery. In such cases, both sparse and structural features can be simultaneously enhanced, even with limited measurements and in low signal-to-noise ratio (SNR). Superior convergence and robustness can be guaranteed. Moreover, a grouping mask scheme is used to accommodate to the complex-valued SAR data. Finally, both simulated and measured SAR data are applied for the validation. Comparisons with the conventions are performed in terms of phase transition analysis, so as to verify the superiority of the proposed algorithm both qualitatively and quantitatively.

Tensor-Based Online Network Anomaly Detection and Diagnosis

This paper presents an online anomaly detection system capable of handling operational network traffic of large networks (such as an ISP). We also aim for an effective and practical diagnosis of anomalies diagnosis to produce actionable intelligence that enables automated response. To achieve these objectives, we use the following approaches. (1) We model the status of the network by a stream of tensors where each tensor cell contains a time series. (2) We detect anomalous tensors at discrete time steps using an unsupervised tensor representation learning model. (3) We produce actionable intelligence through the diagnosis of anomaly detection results and by identifying the abnormal time series that are the most likely causes of each anomaly in the tensor. (4) We further analyze the traffic corresponding to each anomalous time series by an innovative method that extracts and isolates the attack traffic. (5) We provide solutions for challenges in streaming data anomaly detection such as large volume, high velocity, seasonality, and concept drift. We apply our approach to the complete test set of UGR data to show its practicality and effectiveness. Not only can we detect and isolate most of the labelled attack traffic, but we also identify many organic attack activities in the UGR data. Our results on the complete UGR dataset show high detection and isolation rate for the labelled attacks in the dataset. We also report on additional organic attacks we detected that were originally labelled as background in the dataset. Our analysis shows that the isolated background traffic represents interesting and potentially malicious behavior and can provide invaluable insight for cyber-threat researchers.

3D chemical structures allow robust deep learning models for retention time prediction

We present a robust deep learning method CPORT to predict retention time from 3D molecular structures. It generates 4D tensor representations of 3D conformers, that are processed by a neural network with 3D convolutional and fully-connected layers.

Calculation of electric quadrupole linestrengths for diatomic molecules: Application to the H2, CO, HF, and O2 molecules.

We present a unified variational treatment of the electric quadrupole (E2) matrix elements, Einstein coefficients, and linestrengths for general open-shell diatomic molecules in the general purpose diatomic code Duo. Transformation relations between the Cartesian representation (typically used in electronic structure calculations) to the tensorial representation (required for spectroscopic applications) of the electric quadrupole moment components are derived. The implementation has been validated against accurate theoretical calculations and experimental measurements of quadrupole intensities of 1H2 available in the literature. We also present accurate electronic structure calculations of the electric quadrupole moment functions for the X1Σ+ electronic states of CO and HF, as well as for the a1Δg-b1Σg + quadrupole transition moment of O2 with the MRCI level of theory. Accurate infrared E2 line lists for 12C16O and 1H19F are provided. A demonstration of spectroscopic applications is presented by simulating E2 spectra for 12C16O, H19F, and 16O2 (Noxon a1Δg-b1Σg + band).