Dilated Causal Convolutions Research Articles

Short-term prediction for wind power based on temporal convolutional network

The fluctuation and intermittence of wind power bring great challenges to the operation and control of the distribution network. Accurate short-term prediction for wind power is helpful to avoid the risk caused by the uncertainties of wind powers. To improve the accuracy of short-term prediction for wind power, the temporal convolutional network (TCN) is proposed in this paper. The proposed method solves the problem of long-term dependencies and performance degradation of deep convolutional model in sequence prediction by dilated causal convolutions and residual connections. The simulation results show that the training process of TCN is very stable and it has strong generalization ability. Furthermore, TCN shows higher forecasting accuracy than existing predictors such as the support vector machine, multi-layer perceptron, long short-term memory network, and gated recurrent unit network.

Multimodal interaction-aware motion prediction for autonomous street crossing

For mobile robots navigating on sidewalks, the ability to safely cross street intersections is essential. Most existing approaches rely on the recognition of the traffic light signal to make an informed crossing decision. Although these approaches have been crucial enablers for urban navigation, the capabilities of robots employing such approaches are still limited to navigating only on streets that contain signalized intersections. In this article, we address this challenge and propose a multimodal convolutional neural network framework to predict the safety of a street intersection for crossing. Our architecture consists of two subnetworks: an interaction-aware trajectory estimation stream (interaction-aware temporal convolutional neural network (IA-TCNN)), that predicts the future states of all observed traffic participants in the scene; and a traffic light recognition stream AtteNet. Our IA-TCNN utilizes dilated causal convolutions to model the behavior of all the observable dynamic agents in the scene without explicitly assigning priorities to the interactions among them, whereas AtteNet utilizes squeeze-excitation blocks to learn a content-aware mechanism for selecting the relevant features from the data, thereby improving the noise robustness. Learned representations from the traffic light recognition stream are fused with the estimated trajectories from the motion prediction stream to learn the crossing decision. Incorporating the uncertainty information from both modules enables our architecture to learn a likelihood function that is robust to noise and mispredictions from either subnetworks. Simultaneously, by learning to estimate motion trajectories of the surrounding traffic participants and incorporating knowledge of the traffic light signal, our network learns a robust crossing procedure that is invariant to the type of street intersection. Furthermore, we extend our previously introduced Freiburg Street Crossing dataset with sequences captured at multiple intersections of varying types, demonstrating complex interactions among the traffic participants as well as various lighting and weather conditions. We perform comprehensive experimental evaluations on public datasets as well as our Freiburg Street Crossing dataset, which demonstrate that our network achieves state-of-the-art performance for each of the subtasks, as well as for the crossing safety prediction. Moreover, we deploy the proposed architectural framework on a robotic platform and conduct real-world experiments that demonstrate the suitability of the approach for real-time deployment and robustness to various environments.

Temporal Convolutional Networks Applied to Energy-Related Time Series Forecasting

Modern energy systems collect high volumes of data that can provide valuable information about energy consumption. Electric companies can now use historical data to make informed decisions on energy production by forecasting the expected demand. Many deep learning models have been proposed to deal with these types of time series forecasting problems. Deep neural networks, such as recurrent or convolutional, can automatically capture complex patterns in time series data and provide accurate predictions. In particular, Temporal Convolutional Networks (TCN) are a specialised architecture that has advantages over recurrent networks for forecasting tasks. TCNs are able to extract long-term patterns using dilated causal convolutions and residual blocks, and can also be more efficient in terms of computation time. In this work, we propose a TCN-based deep learning model to improve the predictive performance in energy demand forecasting. Two energy-related time series with data from Spain have been studied: the national electric demand and the power demand at charging stations for electric vehicles. An extensive experimental study has been conducted, involving more than 1900 models with different architectures and parametrisations. The TCN proposal outperforms the forecasting accuracy of Long Short-Term Memory (LSTM) recurrent networks, which are considered the state-of-the-art in the field.

Surface EMG-Based Hand Gesture Recognition via Hybrid and Dilated Deep Neural Network Architectures for Neurorobotic Prostheses

Motivated by the potentials of deep learning models in significantly improving myoelectric control of neuroprosthetic robotic limbs, this paper proposes two novel deep learning architectures, namely the [Formula: see text] ([Formula: see text]) and the [Formula: see text] ([Formula: see text]), for performing Hand Gesture Recognition (HGR) via multi-channel surface Electromyography (sEMG) signals. The work is aimed at enhancing the accuracy of myoelectric systems, which can be used for realizing an accurate and resilient man–machine interface for myocontrol of neurorobotic systems. The HRM is developed based on an innovative, unconventional, and particular hybridization of two parallel paths (one convolutional and one recurrent) coupled via a fully-connected multilayer network acting as the fusion center providing robustness across different scenarios. The hybrid design is specifically proposed to treat temporal and spatial features in two parallel processing pipelines and to augment the discriminative power of the model to reduce the required computational complexity and construct a compact HGR model. We designed a second architecture, the [Formula: see text], as a compact architecture. It is worth mentioning that efficiency of a designed deep model, especially its memory usage and number of parameters, is as important as its achievable accuracy in practice. The [Formula: see text] has significantly less memory requirement in training when compared to the HRM due to implementation of novel dilated causal convolutions that gradually increase the receptive field of the network and utilize shared filter parameters. The NinaPro DB2 dataset is utilized for evaluation purposes. The proposed [Formula: see text] significantly outperforms its counterparts achieving an exceptionally-high HGR performance of [Formula: see text]%. The TCNM with the accuracy of [Formula: see text]% also outperforms existing solutions while maintaining low computational requirements.

A Spatiotemporal Dilated Convolutional Generative Network for Point-Of-Interest Recommendation

With the growing popularity of location-based social media applications, point-of-interest (POI) recommendation has become important in recent years. Several techniques, especially the collaborative filtering (CF), Markov chain (MC), and recurrent neural network (RNN) based methods, have been recently proposed for the POI recommendation service. However, CF-based methods and MC-based methods are ineffective to represent complicated interaction relations in the historical check-in sequences. Although recurrent neural networks (RNNs) and its variants have been successfully employed in POI recommendation, they depend on a hidden state of the entire past that cannot fully utilize parallel computation within a check-in sequence. To address these above limitations, we propose a spatiotemporal dilated convolutional generative network (ST-DCGN) for POI recommendation in this study. Firstly, inspired by the Google DeepMind’ WaveNet model, we introduce a simple but very effective dilated convolutional generative network as a solution to POI recommendation, which can efficiently model the user’s complicated short- and long-range check-in sequence by using a stack of dilated causal convolution layers and residual block structure. Then, we propose to acquire user’s spatial preference by modeling continuous geographical distances, and to capture user’s temporal preference by considering two types of time periodic patterns (i.e., hours in a day and days in a week). Moreover, we conducted an extensive performance evaluation using two large-scale real-world datasets, namely Foursquare and Instagram. Experimental results show that the proposed ST-DCGN model is well-suited for POI recommendation problems and can effectively learn dependencies in and between the check-in sequences. The proposed model attains state-of-the-art accuracy with less training time in the POI recommendation task.

A Gated Dilated Causal Convolution Based Encoder-Decoder for Network Traffic Forecasting

The accurate estimation of future network traffic is a key enabler for early warning of network degradation and automated orchestration of network resources. The long short-term memory neural network (LSTM) is a popular architecture for network traffic forecasting, and has been successfully used in many applications. However, it has been observed that LSTMs suffer from limited memory capacity problems when the sequence is long. In this paper, we propose a gated dilated causal convolution based encoder-decoder (GDCC-ED) model for network traffic forecasting. The GDCC-ED learns a vector representation in the encoder from historical network traffic series, in which gated dilated causal convolutions are adopted to expand the long-range memory capacity. Moreover, different types of features in various perspectives, including temporal-independent and temporal-related features, are incorporated. In the decoder, the GDCC-ED exploits an RNN with LSTM units to map the vector representation back to a variable-length target sequence. Besides, a sequence data augmentation technique is designed to solve the problem of data scarcity. Experimental results demonstrate that our model achieves superior performance than state-of-the-art algorithms by 11.6%.

Prediction of mutation effects using a deep temporal convolutional network.

Accurate prediction of the effects of genetic variation is a major goal in biological research. Towards this goal, numerous machine learning models have been developed to learn information from evolutionary sequence data. The most effective method so far is a deep generative model based on the variational autoencoder (VAE) that models the distributions using a latent variable. In this study, we propose a deep autoregressive generative model named mutationTCN, which employs dilated causal convolutions and attention mechanism for the modeling of inter-residue correlations in a biological sequence. We show that this model is competitive with the VAE model when tested against a set of 42 high-throughput mutation scan experiments, with the mean improvement in Spearman rank correlation ∼0.023. In particular, our model can more efficiently capture information from multiple sequence alignments with lower effective number of sequences, such as in viral sequence families, compared with the latent variable model. Also, we extend this architecture to a semi-supervised learning framework, which shows high prediction accuracy. We show that our model enables a direct optimization of the data likelihood and allows for a simple and stable training process. Source code is available at https://github.com/ha01994/mutationTCN. Supplementary data are available at Bioinformatics online.

A novel time series forecasting model with deep learning

Time series forecasting is emerging as one of the most important branches of big data analysis. However, traditional time series forecasting models can not effectively extract good enough sequence data features and often result in poor forecasting accuracy. In this paper, a novel time series forecasting model, named SeriesNet, which can fully learn features of time series data in different interval lengths. The SeriesNet consists of two networks. The LSTM network aims to learn holistic features and to reduce dimensionality of multi-conditional data, and the dilated causal convolution network aims to learn different time interval. This model can learn multi-range and multi-level features from time series data, and has higher predictive accuracy compared those models using fixed time intervals. Moreover, this model adopts residual learning and batch normalization to improve generalization. Experimental results show our model has higher forecasting accuracy and has greater stableness on several typical time series datasets.