Input Tensor Research Articles

A hybrid deep learning model for air quality time series prediction

Air quality (mainly PM2.5) forecasting plays an important role in the early detection and control of air pollution. In recent times, numerous deep learning-based models have been proposed to forecast air quality more accurately. The success of these deep learning models heavily depends on the two key factors viz. proper representation of the input data and preservation of temporal order of the input data during the feature’s extraction phase. Here we propose a hybrid deep neural network (HDNN) framework to forecast the PM2.5 by integrating two popular deep learning architectures, viz. Convolutional neural network (CNN) and bidirectional long short-term memory (BDLSTM) network. Here we build a 3D input tensor so that CNN can extract the trends and spatial features more accurately within the input window. Here we also introduce a linking layer between CNN and BDLSTM to maintain the temporal ordering of feature vectors. In the end, our proposed HDNN framework is compared with the state-of-the-art models, and we show that HDNN outruns other models in terms of prediction accuracy.

Distribution adaptation deep transfer learning method for cross-structure health monitoring using guided waves

Deep learning algorithm can effectively obtain damage information using labeled samples, and has become a promising feature extraction tool for ultrasonic guided wave detection. But it is difficult to apply the monitoring expertise of structure A to structure B in most cases due to the differences in the dispersion and receiving modes of different waveguides. For multi-structure monitoring at the system level, how to transfer a trained structural health monitoring model to another different structure remains a major challenge. In this article, a cross-structure ultrasonic guided wave structural health monitoring method based on distribution adaptation deep transfer learning is proposed to solve the feature generalization problem in different monitoring structures. First, the joint distribution adaptation method is employed to adapt both the marginal distribution and conditional distribution of the guided wave signals from different structures. Second, convolutional long short-term memory network is constructed to learn the mapping relationship from adapted training samples in source domain. Batch normalization layer is implemented to balance the input tensors of each sample to the same distribution. Finally, the multi-sensor damage indexes are utilized to visually present the damage by probability imaging. The experimental results show that proposed method can utilize the single-sensor monitoring data in one structure to implement the multi-sensor damage monitoring in another structure and achieve the damage imaging visualization. The imaging performance is significantly superior to the existing principal component analysis, transfer component analysis, and other state-of-art comparison methods.

Deep-Learning Based Decentralized Frame-to-Frame Trajectory Prediction Over Binary Range-Angle Maps for Automotive Radars

Reliable trajectory prediction methods are critical in providing predictive safety intelligence for vital decision making in intelligent transportation systems to further enhance the safety of drivers and passengers. To tackle complicated maneuvering and interactions between objects, learning-based algorithms were used to replace classic model-based trajectory prediction algorithms. However, most algorithms implicitly presume that they are executed at centralized processing units after gathering data from edge sensors and delivering the results back to the on-board units. This causes an increase in computation time and latency; thus, reducing the reaction time of the drivers. To reduce the computation time and latency and consider the robustness of local sensors, we propose a decentralized radar-dedicated framework with a deep-learning (DL) model, called predictive RadarNet, to predict future trajectories over binary range angle (RA) maps with a probabilistic representation according to the original radar RA maps for presenting the uncertainty of the estimated trajectories. In addition, to reduce the model size for low-complexity, we designed a prepossessing technique that can largely reduce the size of the input tensors without losing information. Moreover, we found that the functions of the DL-model consist of two operations: future inference of original radar RA maps and transformation to binary RA maps. Thus, we designed two models with different kernels that are suitable for dealing with the two operations. Simulations show that the proposed decentralized framework using predictive RadarNet can provide reliable prediction results with a low computation time.

SWIFT: Scalable Wasserstein Factorization for Sparse Nonnegative Tensors

Existing tensor factorization methods assume that the input tensor follows some specific distribution (i.e. Poisson, Bernoulli, and Gaussian), and solve the factorization by minimizing some empirical loss functions defined based on the corresponding distribution. However, it suffers from several drawbacks: 1) In reality, the underlying distributions are complicated and unknown, making it infeasible to be approximated by a simple distribution. 2) The correlation across dimensions of the input tensor is not well utilized, leading to sub-optimal performance. Although heuristics were proposed to incorporate such correlation as side information under Gaussian distribution, they can not easily be generalized to other distributions. Thus, a more principled way of utilizing the correlation in tensor factorization models is still an open challenge. Without assuming any explicit distribution, we formulate the tensor factorization as an optimal transport problem with Wasserstein distance, which can handle non-negative inputs. We introduce SWIFT, which minimizes the Wasserstein distance that measures the distance between the input tensor and that of the reconstruction. In particular, we define the N-th order tensor Wasserstein loss for the widely used tensor CP factorization and derive the optimization algorithm that minimizes it. By leveraging sparsity structure and different equivalent formulations for optimizing computational efficiency, SWIFT is as scalable as other well-known CP algorithms. Using the factor matrices as features, SWIFT achieves up to 9.65% and 11.31% relative improvement over baselines for downstream prediction tasks. Under the noisy conditions, SWIFT achieves up to 15% and 17% relative improvements over the best competitors for the prediction tasks.

Density Guarantee on Finding Multiple Subgraphs and Subtensors

Dense subregion (subgraph & subtensor) detection is a well-studied area, with a wide range of applications, and numerous efficient approaches and algorithms have been proposed. Approximation approaches are commonly used for detecting dense subregions due to the complexity of the exact methods. Existing algorithms are generally efficient for dense subtensor and subgraph detection, and can perform well in many applications. However, most of the existing works utilize the state-or-the-art greedy 2-approximation algorithm to capably provide solutions with a loose theoretical density guarantee. The main drawback of most of these algorithms is that they can estimate only one subtensor, or subgraph, at a time, with a low guarantee on its density. While some methods can, on the other hand, estimate multiple subtensors, they can give a guarantee on the density with respect to the input tensor for the first estimated subsensor only. We address these drawbacks by providing both theoretical and practical solution for estimating multiple dense subtensors in tensor data and giving a higher lower bound of the density. In particular, we guarantee and prove a higher bound of the lower-bound density of the estimated subgraph and subtensors. We also propose a novel approach to show that there are multiple dense subtensors with a guarantee on its density that is greater than the lower bound used in the state-of-the-art algorithms. We evaluate our approach with extensive experiments on several real-world datasets, which demonstrates its efficiency and feasibility.

Efficient Alternating Least Squares Algorithms for Low Multilinear Rank Approximation of Tensors

The low multilinear rank approximation, also known as the truncated Tucker decomposition, has been extensively utilized in many applications that involve higher-order tensors. Popular methods for low multilinear rank approximation usually rely directly on matrix SVD, therefore often suffer from the notorious intermediate data explosion issue and are not easy to parallelize, especially when the input tensor is large. In this paper, we propose a new class of truncated HOSVD algorithms based on alternating least squares (ALS) for efficiently computing the low multilinear rank approximation of tensors. The proposed ALS-based approaches are able to eliminate the redundant computations of the singular vectors of intermediate matrices and are therefore free of data explosion. Also, the new methods are more flexible with adjustable convergence tolerance and are intrinsically parallelizable on high-performance computers. Theoretical analysis reveals that the ALS iteration in the proposed algorithms is q-linear convergent with a relatively wide convergence region. Numerical experiments with large-scale tensors from both synthetic and real-world applications demonstrate that ALS-based methods can substantially reduce the total cost of the original ones and are highly scalable for parallel computing.

Short-Term Multi-Step Ahead Wind Power Predictions Based On A Novel Deep Convolutional Recurrent Network Method

In this paper, a novel deep convolutional recurrent network method, the K-shape and K-means guided Convolutional Neural Network integrating Gated Recurrent Units (KK-CNN-GRU), for short-term multi-step ahead predictions of wind turbine power generations is proposed. The KK-CNN-GRU is composed of three modules, an input tensor construction module, a two-layer clustering module, and a prediction module. In the input tensor construction module, data of spatial, temporal, and physical meaning features are fused and organized as a tensor. Meanwhile, the two-layer clustering module is used to recognize patterns of a wind power time series and offer a cluster membership to the associated input tensor. Finally, predictions are generated by feeding the input tensor into the prediction module, which is composed of a feature weighting unit, a convolutional neural network, and a set of gated recurrent unit cells with fully connected layers switched according to the cluster membership of the input tensor. In experimental studies, two sets of six-step ahead prediction experiments are conducted with considered prediction horizons from 7.1 seconds to 42.6 seconds and from 10 minutes to 1 hour, respectively. The high prediction accuracy of the proposed KK-CNN-GRU is validated by comparing with state-of-the-art benchmarking methods.

Dynamic Path Flow Estimation Using Automatic Vehicle Identification and Probe Vehicle Trajectory Data: A 3D Convolutional Neural Network Model

Dynamic path flows, referring to the number of vehicles that choose each path in a network over time, are generally estimated with the partial observations as the input. The automatic vehicle identification (AVI) system and probe vehicle trajectories are now popular and can provide rich and complementary trip information, but the data fusion was rarely explored. Therefore, in this paper, the dynamic path flow estimation is based on these two data sources and transformed into a feature learning problem. To fuse the two data sources belonging to different detection ways at the data level, the virtual AVI points, analogous to the real AVI points (turning movements at nodes with AVI detectors), are defined and selected to statically observe the dynamic movement of the probe vehicles. The corresponding selection principles and a programming model considering the distribution of real AVI points are first established. The selected virtual AVI points are used to construct the input tensor, and the turning movement-based observations from both the data sources can be extracted and fused. Then, a three-dimensional (3D) convolutional neural network (CNN) model is designed to exploit the hidden patterns from the tensor and establish the high-dimensional correlations with path flows. As the path flow labels commonly with noises, the bootstrapping method is adopted for model training and the corresponding relabeling principle is defined to purify the noisy labels. The entire model is extensively tested based on a realistic road network, and the results show that the designed CNN model with the presented data fusion method can perform well in training time and estimation accuracy. The robustness of a model to noisy labels is also improved through the bootstrapping method. The dynamic path flows estimated by the trained model can be applied to travel information provision, proactive route guidance, and signal control with high real-time requirements.

Att-Net: Enhanced emotion recognition system using lightweight self-attention module

Speech emotion recognition (SER) is an active research field of digital signal processing and plays a crucial role in numerous applications of Human–computer interaction (HCI). Nowadays, the baseline state of the art systems has quite a low accuracy and high computations, which needs upgrading to make it reasonable for real-time industrial uses such as detection of content from speech data. The main intent for low recognition rate and high computational cost is a scarceness of datasets, model configuration, and patterns recognition that is the supreme stimulating work for building a robust SER system. In this study, we address these problems and propose a simple and lightweight deep learning-based self-attention module (SAM) for SER system. The transitional features map is given to SAM, which produces efficiently the channel and spatial axes attention map with insignificant overheads. We use a multi-layer perceptron (MLP) in channel attention to extracting global cues and a special dilated convolutional neural network (CNN) in spatial attention to extract spatial info from input tensor. Moreover, we merge, spatial and channel attention maps to produce a combine attention weights as a self-attention module. We placed SAM in the middle of convolutional and connected layers and trained it in an end-to-end mode. The ablation study and comprehensive experimentations are accompanied over IEMOCAP, RAVDESS, and EMO-DB speech emotion datasets. The proposed SER system shows consistent improvements in overall experiments for all datasets and shows 78.01%, 80.00%, and 93.00% average recall, respectively.

Monaural Speech Separation Using Speaker Embedding From Preliminary Separation

In speech separation, the identities of the speakers may be an important cue to discriminate speeches in the mixture and separate them better. A few recent researches used the speaker embedding as an additional information, but they often require prior information about the target speaker or used noisy speaker embedding extracted from the mixture signal. In this article, we propose monaural speech separation that utilizes the speaker embedding in the later separator blocks, which is extracted from the intermediate separated results obtained by the early stages of the separator network. The later blocks in the separator networks consisting of repeated blocks such as the fully-convolutional time-domain audio separation network (Conv-TasNet) or the successive downsampling and resampling of multi-resolution features (SuDoRM-RF) are modified to take the speaker information as a form of affine transformation or addition to the original input tensor. The experimental results showed that the proposed methods significantly improved the performances of existing separation systems with a moderate number of additional parameters.

Attention-based Convolutional Neural Network for MRI Gibbs-ringing Artifact Suppression

Gibbs-ringing artifact is a common artifact in MRI image processing. As MRI raw data is taken in a frequency domain, 2D in- verse discrete Fourier transform is applied to visualize data. Inability to take inverse Fourier transform of full spectrum (full k-space) leads to the insufficient sampling of the high frequency data and results in a well-known Gibbs phenomenon. It is worth to notice that truncation of high frequency information generates a significant blur, thus some techniques from other image restoration problems (for example, image deblur task) can be successfully used. We propose attention-based convolutional neural network for Gibbs-ringing reduction which is the extension of recently proposed GAS-CNN (Gibbs-ringing Artifact Suppression Convolutional Neural Network). Proposed method includes simplified non-linear mapping, amended by LRNN (Layer Recurrent Neural Network) refinement block with feature attention module, controlling the correlation between input and output tensors of the refinement unit. The research shows that the proposed post-processing refinement construction considerably simplifies the non-linear mapping.

Learning Inter-Modal Correspondence and Phenotypes From Multi-Modal Electronic Health Records

Non-negative tensor factorization has been shown a practical solution to automatically discover phenotypes from the electronic health records (EHR) with minimal human supervision. Such methods generally require an input tensor describing the inter-modal interactions to be pre-established; however, the correspondence between different modalities (e.g., correspondence between medications and diagnoses) can often be missing in practice. Although heuristic methods can be applied to estimate them, they inevitably introduce errors, and leads to sub-optimal phenotype quality. This is particularly important for patients with complex health conditions (e.g., in critical care) as multiple diagnoses and medications are simultaneously present in the records. To alleviate this problem and discover phenotypes from EHR with unobserved inter-modal correspondence, we propose the collective hidden interaction tensor factorization (cHITF) to infer the correspondence between multiple modalities jointly with the phenotype discovery. We assume that the observed matrix for each modality is marginalization of the unobserved inter-modal correspondence, which are reconstructed by maximizing the likelihood of the observed matrices. Extensive experiments conducted on the real-world MIMIC-III dataset demonstrate that cHITF effectively infers clinically meaningful inter-modal correspondence, discovers phenotypes that are more clinically relevant and diverse, and achieves better predictive performance compared with a number of state-of-the-art computational phenotyping models.

A Convolutional Neural Network Using Dinucleotide One-hot Encoder for identifying DNA N6-Methyladenine Sites in the Rice Genome

N6-methyladenine (m6A) is one of the crucial epigenetic modifications and is related to the control of various DNA processes. Carrying out a genome-wide m6A analysis via wet experiments is fundamental but takes a long time. As complementary methods, computing tools, especially those based on machine learning, are urgently needed. A new protocol, iRicem6A-CNN, for identifying m6A sites in the rice genome was developed. This protocol was designed to use dinucleotide one-hot encoding to generate input tensors for predictions by convolutional neutral networks, and achieved five-fold cross-validation and independent testing accuracy values of 93.82% and 96.19%, respectively, performing better than those of other available predictors. The experiment results demonstrates that only the ability of iRicem6A-CNN based on 2-mer one-hot encoding is to display high performance but also to perform more stably and robustly than models using 1-mer one-hot encoding. A webserver is accessible at http://iRicem6A-CNN.aibiochem.net

Multi-channel spectrograms for speech processing applications using deep learning methods

Time–frequency representations of the speech signals provide dynamic information about how the frequency component changes with time. In order to process this information, deep learning models with convolution layers can be used to obtain feature maps. In many speech processing applications, the time–frequency representations are obtained by applying the short-time Fourier transform and using single-channel input tensors to feed the models. However, this may limit the potential of convolutional networks to learn different representations of the audio signal. In this paper, we propose a methodology to combine three different time–frequency representations of the signals by computing continuous wavelet transform, Mel-spectrograms, and Gammatone spectrograms and combining then into 3D-channel spectrograms to analyze speech in two different applications: (1) automatic detection of speech deficits in cochlear implant users and (2) phoneme class recognition to extract phone-attribute features. For this, two different deep learning-based models are considered: convolutional neural networks and recurrent neural networks with convolution layers.

Hierarchical multiscale permutation entropy-based feature extraction and fuzzy support tensor machine with pinball loss for bearing fault identification

Recently, tensor learning has been applied to intelligent bearing health monitoring successfully. In the past tensor learning based fault identification approaches, the clipped time–frequency images (TFIs) are usually taken as the input tensor data. However, the manual clipping method is usually used to reduce the dimension of TFIs, which may easily lose some critical fault information. Hence, we present hierarchical multiscale permutation entropy (HMPE) as a new feature representation method to avert this drawback from the perspective of statistical measure. In practice, the extracted fault feature (HMPE) may often be corrupted by noise. Hence, fuzzy support tensor machine with pinball loss (Pin-FSTM) as a new tensor-based classifier is developed, which can reduce noise sensitivity and minimize classification errors. Finally, a novel bearing health monitoring strategy based on HMPE and Pin-FSTM is presented, which has been demonstrated to have an excellent identification performance through two experiments. Compared with the state-of-the-art permutation entropy-based fault identification algorithms, the presented scheme is more superior in accuracy and stability. Besides, this research provides a new direction for tensor learning further applying in bearing health monitoring.

Deep Learning for Price Movement Prediction Using Convolutional Neural Network and Long Short-Term Memory

The prediction of stock price movement direction is significant in financial studies. In recent years, a number of deep learning models have gradually been applied for stock predictions. This paper presents a deep learning framework to predict price movement direction based on historical information in financial time series. The framework combines a convolutional neural network (CNN) for feature extraction and a long short-term memory (LSTM) network for prediction. We specifically use a three-dimensional CNN for data input in the framework, including the information on time series, technical indicators, and the correlation between stock indices. And in the three-dimensional input tensor, the technical indicators are converted into deterministic trend signals and the stock indices are ranked by Pearson product-moment correlation coefficient (PPMCC). When training, a fully connected network is used to drive the CNN to learn a feature vector, which acts as the input of concatenated LSTM. After both the CNN and the LSTM are trained well, they are finally used for prediction in the testing set. The experimental results demonstrate that the framework outperforms state-of-the-art models in predicting stock price movement direction.

Model order reduction assisted by deep neural networks (ROM-net)

In this paper, we propose a general framework for projection-based model order reduction assisted by deep neural networks. The proposed methodology, called ROM-net, consists in using deep learning techniques to adapt the reduced-order model to a stochastic input tensor whose nonparametrized variabilities strongly influence the quantities of interest for a given physics problem. In particular, we introduce the concept of dictionary-based ROM-nets, where deep neural networks recommend a suitable local reduced-order model from a dictionary. The dictionary of local reduced-order models is constructed from a clustering of simplified simulations enabling the identification of the subspaces in which the solutions evolve for different input tensors. The training examples are represented by points on a Grassmann manifold, on which distances are computed for clustering. This methodology is applied to an anisothermal elastoplastic problem in structural mechanics, where the damage field depends on a random temperature field. When using deep neural networks, the selection of the best reduced-order model for a given thermal loading is 60 times faster than when following the clustering procedure used in the training phase.

A tensor compression algorithm using Tucker decomposition and dictionary dimensionality reduction

Tensor compression algorithms play an important role in the processing of multidimensional signals. In previous work, tensor data structures are usually destroyed by vectorization operations, resulting in information loss and new noise. To this end, this article proposes a tensor compression algorithm using Tucker decomposition and dictionary dimensionality reduction, which mainly includes three parts: tensor dictionary representation, dictionary preprocessing, and dictionary update. Specifically, the tensor is respectively performed by the sparse representation and Tucker decomposition, from which one can obtain the dictionary, sparse coefficient, and core tensor. Furthermore, the sparse representation can be obtained through the relationship between sparse coefficient and core tensor. In addition, the dimensionality of the input tensor is reduced by using the concentrated dictionary learning. Finally, some experiments show that, compared with other algorithms, the proposed algorithm has obvious advantages in preserving the original data information and denoising ability.

Attention-Guided Multi-Scale Segmentation Neural Network for Interactive Extraction of Region Objects from High-Resolution Satellite Imagery

Automatic extraction of region objects from high-resolution satellite imagery presents a great challenge, because there may be very large variations of the objects in terms of their size, texture, shape, and contextual complexity in the image. To handle these issues, we present a novel, deep-learning-based approach to interactively extract non-artificial region objects, such as water bodies, woodland, farmland, etc., from high-resolution satellite imagery. First, our algorithm transforms user-provided positive and negative clicks or scribbles into guidance maps, which consist of a relevance map modified from Euclidean distance maps, two geodesic distance maps (for positive and negative, respectively), and a sampling map. Then, feature maps are extracted by applying a VGG convolutional neural network pre-trained on the ImageNet dataset to the image X, and they are then upsampled to the resolution of X. Image X, guidance maps, and feature maps are integrated as the input tensor. We feed the proposed attention-guided, multi-scale segmentation neural network (AGMSSeg-Net) with the input tensor above to obtain the mask that assigns a binary label to each pixel. After a post-processing operation based on a fully connected Conditional Random Field (CRF), we extract the selected object boundary from the segmentation result. Experiments were conducted on two typical datasets with diverse region object types from complex scenes. The results demonstrate the effectiveness of the proposed method, and our approach outperforms existing methods for interactive image segmentation.

Multiple Tensor-on-Tensor Regression: An Approach for Modeling Processes With Heterogeneous Sources of Data

In recent years, measurement or collection of heterogeneous sets of data such as those containing scalars, waveform signals, images, and even structured point clouds, has become more common. Statistical models based on such heterogeneous sets of data that represent the behavior of an underlying system can be used in the monitoring, control, and optimization of the system. Unfortunately, available methods mainly focus on the scalars and profiles and do not provide a general framework for integrating different sources of data to construct a model. This article addresses the problem of estimating a process output, measured by a scalar, curve, image, or structured point cloud by a set of heterogeneous process variables such as scalar process setting, profile sensor readings, and images. We introduce a general multiple tensor-on-tensor regression approach in which each set of input data (predictor) and output measurements are represented by tensors. We formulate a linear regression model between the input and output tensors and estimate the parameters by minimizing a least square loss function. To avoid overfitting and reduce the number of parameters to be estimated, we decompose the model parameters using several basis matrices that span the input and output spaces, and provide efficient optimization algorithms for learning the basis and coefficients. Through several simulation and case studies, we evaluate the performance of the proposed method. The results reveal the advantage of the proposed method over some benchmarks in the literature in terms of the mean square prediction error. Supplementary materials for this article are available online.