WaveNet Model Research Articles

Long-time prediction of multi-indicator water quality based on the improved WaveNet under time series decomposition and phase space reconstruction

Abstract Long–time prediction of water quality indicator such as chlorophyll–a (Chl–a) is crucial for water process engineering and environmental management. In order to capture the characteristics of long–time series and reduce the limitations of traditional long–time prediction strategies, this paper proposes a novel hybrid model by combining data decomposition, phase space reconstruction, feature fusion and improved WaveNet. Firstly, the original data is decomposed into several subsequences through time series decomposition. Then, the subsequences with chaotic characteristics are integrated with multiple features for phase space reconstruction. Next, the decomposed and reconstructed subsequences are fed back into the improved WaveNet model separately. Finally, the prediction results are obtained by summing the predicted values of the subsequences. In this paper, the reliability of the method is assessed using the dissolved oxygen, water temperature, pH and Chl–a data of a monitoring station in the Beihai coastal sea area, ablation experiments are conducted to demonstrate the effectiveness of each module in the proposed model, and comparisons with multiple benchmark and hybrid models show that the proposed model exhibits better performance in long–time prediction of coastal water quality in the next fourteen days.

Predicting stock price crash risk in China: A modified graph WaveNet model

The stock price of a firm is dynamically influenced by its own factors as well as those of its peers. In this study, we introduce a Graph Attention Network (GAT) integrated with WaveNet architecture—termed the GAT-WaveNet model—to capture both time-series and spatial dependencies for forecasting the stock price crash risk of Chinese listed firms from 2012 to 2023. Utilizing node-rolling techniques to prevent overfitting, our results show that the GAT-WaveNet model significantly outperforms traditional machine learning models in prediction accuracy. Moreover, investment portfolios leveraging the GAT-WaveNet model substantially exceed the cumulative returns of those based on other models.

A Study of Artificial Intelligence-Assisted Listening Training in College English Teaching

Abstract The rapid development of intelligent technology and its wide application in various industrial fields have caused a big change in the teaching mode of university English listening training. Based on the C/S architecture, this paper establishes an intelligent English teaching listening training system by combining artificial intelligence technology, analyzes the auxiliary role of artificial intelligence technology for listening training, and designs an intelligent English listening training teaching mode based on this system. In order to obtain a diversity of English listening training materials, this paper first utilizes speech signal pre-emphasis, frame-splitting, windowing, and other methods to carry out signal preprocessing and then realizes end-point detection of speech signals through the improved double-threshold algorithm. Based on the characteristics of English speech signals, the Seq2Seq model is optimized by the attention mechanism, and the WaveNet model is introduced to establish a speech synthesis model for listening training. Pre-emphasis can improve the clarity of the speech signal by 50%, and the improved double threshold algorithm improves the recognition effect of speech segment clear tone, and the mean value of speech synthesis MOS score is 4.39. After adopting the intelligent English listening training teaching model, the English listening scores of the students in the observation group increased by 8.21 points compared with those of the control group, and the students who were very uninterested in English listening learning decreased by 20 percentage points. The introduction of artificial intelligence technology can provide students with more diverse English listening training materials, which can help them improve their listening level and learnability.

Automatic microseismic signal classification for mining safety monitoring using the WaveNet classifier

AbstractMicroseismic monitoring is a promising method for the safety monitoring of underground mines. However, it is crucial to isolate microseismic signals related to the collapse of a mine from others for successful monitoring because the monitoring system records various signals. Recently, deep learning‐based classification techniques have achieved high performance in such data classification problems. In this context, we develop an automatic signal classification technique using the modified WaveNet classifier. The main characteristic of the WaveNet structure is its ability to extract features at various frequencies from very long time‐series data, and such an advantage makes the WaveNet suitable for seismic data processing. The data imbalance problem coming from the safe condition of the monitoring target is solved by augmenting the training data with those acquired from another mine and employing class weighting. After training, an optimal classifier is chosen considering the loss function, accuracy and Fβ score. The optimal classifier shows very high accuracy and excellent performance for the test data prediction. Compared to the random forest model and another one‐dimensional convolutional neural network–based network, the suggested classifier has higher reliability in predicting microseismic signals. Even though the proposed WaveNet model has a much more complex structure than the random forest model, the actual application examples demonstrate that the proposed model achieves high efficiency without any preprocessing. The automatic signal classifier developed in this study can be directly applied to various safety monitoring problems, not only mines, to improve the efficiency and reliability of monitoring systems.

WaveNet-based approximation of a cochlear filtering and hair cell transduction model

Computational auditory models are important tools for gaining new insights into hearing mechanisms, and they can provide a foundation for bio-inspired speech and audio processing algorithms. However, accurate models often entail an immense computational effort, rendering their application unfeasible if quick execution is required. This paper presents a WaveNet-based approximation of the normal-hearing cochlear filtering and inner hair cell (IHC) transduction stages of a widely used auditory model [Zilany and Bruce (2006). J. Acoust. Soc. Am. 120(3), 1446–1466]. The WaveNet model was trained and optimized using a large dataset of clean speech, noisy speech, and music for a wide range of sound pressure levels (SPLs) and characteristic frequencies between 125 Hz and 8 kHz. The model was evaluated with unseen (noisy) speech, music signals, sine tones, and click signals at SPLs between 30 and 100 dB. It provides accurate predictions of the IHC receptor potentials for a given input stimulus and allows an efficient execution with processing times up to 250 times lower compared to an already optimized reference implementation of the original auditory model. The WaveNet model is fully differentiable, thus, allowing its application in the context of deep-learning-based speech and audio enhancement algorithms.

Nonlinear System Identification Using Audio-Inspired WaveNet Deep Neural Networks

Nonlinear system identification is a challenging task that requires accurate estimation of the structural model from observations of nonlinear behavior. The WaveNet, which was originally a neural network architecture for audio processing, has been modified and first introduced to the analysis of mechanical signals to capture long-term dependencies in mechanical systems and generate high-quality signals. A novel nonlinear system identification method has been proposed using a modified WaveNet-based approach that constructs a relationship between the vibration response and the nonlinear elements in the inverse model without the need for a definite structural model. This approach uses dilated convolution for feature extraction and a multilayer perceptron for feature transition, with the addition of average pooling along the time dimension for adaptive processing of varying length data, which are more computationally efficient and widely applicable. The 13-layer modified WaveNet models have been designed and applied to the problem. Comparisons with other baseline models were made to demonstrate the method’s superiority in terms of accuracy, effectiveness, and robustness. Additionally, the method has been applied to predict composite models of friction and elastic curves, demonstrating its ability to handle diverse and complex problems.

Dynamic Spatial Correlation in Graph WaveNet for Road Traffic Prediction

Road traffic prediction is a crucial area currently investigated under the umbrella of the intelligent transportation system. Timely and accurate traffic prediction is a challenging problem because of the diverse nature of roads, abrupt changes in speed, and the existence of dependencies between road segments. A critical component of this research is addressing the dynamic spatial and non-linear temporal dependencies in the road network. The traffic conditions in a traffic road network change continuously, and for precise predictions, time-varying spatial correlation needs to be integrated into the model. This study intends to incorporate dynamic spatial dependencies in the Graph WaveNet model by applying attention mechanisms that can compute attention scores for the self-adaptive adjacency matrix in the time domain. We compared the computation cost of our proposed model of a graph attention network with multi-head attention with Graph WaveNet for up to 60 min. Our model gave the best result for 60-min prediction with an average percentage decrease of 3.4% and 4.76% in root-mean-square error on PEMS-BAY and METR-LA datasets, respectively. However, the model training time is increased because of the added computation of attention scores.

Shale Crack Identification Based on Acoustic Emission Experiment and Wavenet Data Recovery

This work provides a general analysis method for the crack detection in shales. The acoustic emission (AE) testing with data recovery is proposed for determining the crack modes and positions in the test and analysis process of the shale fracturing experiment. A fracturing and in situ AE monitoring system is constructed to collect the experimental data in at least six channels for the crack detection, and the source positions output from AE testing represent the positions of cracks. Due to some uncontrollable reasons, such as the poor coupling between sensors and sample and the sudden failure of the sensors, some parts of experimental data are missing during the experiments. Therefore, a data recovery neural network algorithm based on wavenet model is introduced to reconstruct the missing parts of experimental data in the waveforms. Since the accuracy of data recovery is not satisfying based on the collected experimental data, the interpolation of experimental data is performed to refine the data which can obviously improve the accuracy of data recovery. After all the required experimental data have been recovered, the crack mode for each crack can be determined based on the moment tensor analysis. This analysis method can be extensively applied to the shale crack detection.

Wave-Net: A lightweight deep network for retinal vessel segmentation from fundus images

Accurate segmentation of retinal vessels from fundus images is fundamental for the diagnosis of numerous diseases of eye, and an automated vessel segmentation method can effectively help clinicians to make accurate diagnosis for the patients and provide the appropriate treatment schemes. It is important to note that both thick and thin vessels play the key role for disease judgements. Because of complex factors, the precise segmentation of thin vessels is still a great challenge, such as the presence of various lesions, image noise, complex backgrounds and poor contrast in the fundus images. Recently, because of the advantage of context feature representation learning capabilities, deep learning has shown a remarkable segmentation performance on retinal vessels. However, it still has some shortcomings on high-precision retinal vessel extraction due to some factors, such as semantic information loss caused by pooling operations, limited receptive field, etc. To address these problems, this paper proposes a new lightweight segmentation network for precise retinal vessel segmentation, which is called as Wave-Net model on account of the whole shape. To alleviate the influence of semantic information loss problem to thin vessels, to acquire more contexts about micro structures and details, a detail enhancement and denoising block (DED) is proposed to improve the segmentation precision on thin vessels, which replaces the simple skip connections of original U-Net. On the other hand, it could well alleviate the influence of the semantic gap problem. Further, faced with limited receptive field, for multi-scale vessel detection, a multi-scale feature fusion block (MFF) is proposed to fuse cross-scale contexts to achieve higher segmentation accuracy and realize effective processing of local feature maps. Experiments indicate that proposed Wave-Net achieves an excellent performance on retinal vessel segmentation while maintaining a lightweight network design compared to other advanced segmentation methods, and it also has shown a better segmentation ability to thin vessels.

Improved WaveNet for pressurized water reactor accident prediction

Many studies have proposed deep learning models to diagnose faults and predict accidents in nuclear power reactors. However, the training data in these studies are deterministic, and neglect the parameter uncertainty caused by sensor failure in accident scenario. Developing a robust deep learning model requires consideration for uncertainties and possible malfunctions of the sensors. Consequently, this work presents a novel application of wave network (WaveNet) architecture for pressurized water reactor accident prediction using an augmented dataset that reflects the sensor status during accidents. The WaveNet is developed with stacked dilated convolution networks with non-causal layers. The dilated convolution is separated by LeakyRelu and Batch Normalization layers. A flatten layer is added before the output layer to reduce the three-dimensional convolution output to 2D output. Two dense layers are placed at the top to output the model prediction. In this study, four reactor accidents (LOCA, steam generator tube rupture, loss of AC power, and steam line break inside containment) are simulated with twelve augmented reactor coolant parameters used to train and test the model’s predictive performance. The model performance is evaluated using the inference time accuracy, precision, recall, and the F1 score. Different model architecture is also evaluated to determine the best model hyperparameters for accurate prediction of reactor accidents. Finally, the predictive capability of the WaveNet model is compared with the convolution long short-term memory (CNN-LSTM) model. The result shows that the best WaveNet architecture predicts the simulated accidents with an accuracy of 99.3%.

Biosignal Classification Based on Multi-Feature Multi-Dimensional WaveNet-LSTM Models

Electrocardiogram (ECG) effectively records the difference between body potentials generated during the physiological function of the heart. Both ECG and heartbeat sounds are viewed as powerful tools to diagnose abnormal arrhythmias. In the past, the accuracy of such diagnoses has been significantly improved due to the development of machine-learning algorithms. However, current models still do not provide acceptable performance due to similarities of signal waveforms as well as ambient noises and interferences. In this paper, we propose a novel deep-learning model that incorporates a WaveNet model based on dilated convolutions as the backbone followed by multiple bi-directional long-short-term memory (Bi-LSTM) layers to further enhance the discriminant capabilities of temporal relations. A typical clinical dataset, i.e., the MIT-BIH arrhythmia database, which considers intra-patient and inter-patient paradigms based on the American Association of Medical Instrumentation (AAMI) EC57 standard, is used to demonstrate the performance of the proposed approach. Numerical results show that our model has achieved the state-of-the-art classification accuracies.

Improving closed-loop TMS timing using the Wavenet model

Improving closed-loop TMS timing using the Wavenet model

A Wavenet-Based Virtual Sensor for PM10 Monitoring

In this work, a virtual sensor for PM10 concentration monitoring is presented. The sensor is based on wavenet models and uses daily mean NO2 concentration and meteorological variables (wind speed and rainfall) as input. The methodology has been applied to the reconstruction of PM10 levels measured from 14 monitoring stations in Lombardy region (Italy). This region, usually affected by high levels of PM10, is a challenging benchmarking area for the implemented sensors. Neverthless, the performances are good with relatively low bias and high correlation.

Temporal Feature Aggregation for Text Classification Based on Ensembled Deep-Learning Models

In this paper, we focus on the text classification task, which is a most import task in the area of Natural Language Processing (NLP). We propose an innovative convolutional neural network (CNN) model to perform temporal feature aggregation (TFA) effectively, which has a highly representative capacity to extract sequential features from vectorized numerical embeddings. First, we feed embedded vectors into a bi-directional LSTM (Bi-LSTM) model to capture the contextual information of each word. Afterwards, we propose to use the state-of-the-art deep-learning models as key components of the architecture, i.e., the Xception model and the WaveNet model, to extract temporal features from deep convolutional layers concurrently. To facilitate an effective feature fusion, we concatenate the outputs of two component models before forwarding to a drop-out layer to alleviate over-fitting and subsequently a fully-connected dense layer to perform the final classification of input texts. Experiments demonstrate that the proposed method achieves performance comparable to the state-of-the-art models while at a significantly lower computational complexity. Our approach obtains the cross-validation score of 95.83% for the Quora Insincere Question Classification (QIQC) dataset, and the cross-validation score of 83.10% for the Spooky Author Identification (SAI) dataset, respectively, which are among the best published results. The proposed method can be readily generalized to signal processing tasks, e.g., environmental sound classification (ESC) and machine fault analysis (MFA).

Deep Learning-Based Energy Disaggregation and On/Off Detection of Household Appliances

Energy disaggregation, a.k.a. Non-Intrusive Load Monitoring, aims to separate the energy consumption of individual appliances from the readings of a mains power meter measuring the total energy consumption of, e.g., a whole house. Energy consumption of individual appliances can be useful in many applications, e.g., providing appliance-level feedback to the end users to help them understand their energy consumption and ultimately save energy. Recently, with the availability of large-scale energy consumption datasets, various neural network models such as convolutional neural networks and recurrent neural networks have been investigated to solve the energy disaggregation problem. Neural network models can learn complex patterns from large amounts of data and have been shown to outperform the traditional machine learning methods such as variants of hidden Markov models. However, current neural network methods for energy disaggregation are either computational expensive or are not capable of handling long-term dependencies. In this article, we investigate the application of the recently developed WaveNet models for the task of energy disaggregation. Based on a real-world energy dataset collected from 20 households over 2 years, we show that WaveNet models outperforms the state-of-the-art deep learning methods proposed in the literature for energy disaggregation in terms of both error measures and computational cost. On the basis of energy disaggregation, we then investigate the performance of two deep-learning based frameworks for the task of on/off detection which aims at estimating whether an appliance is in operation or not. The first framework obtains the on/off states of an appliance by binarising the predictions of a regression model trained for energy disaggregation, while the second framework obtains the on/off states of an appliance by directly training a binary classifier with binarised energy readings of the appliance serving as the target values. Based on the same dataset, we show that for the task of on/off detection the second framework, i.e., directly training a binary classifier, achieves better performance in terms of F1 score.

Detection of Spike-type Stall of Axial Compressors Based on Dilated Causal Convolutional Neural Networks

An aerodynamic instability inception and short-length-scale periodic anomaly prior to stall onset known as spike-type stall inception in axial compressors is observed in aero-engine. In this paper, a deep dilated causal convolutional neural network(CNN) named as WaveNet is applied to spike-type stall inception detection and prediction in time-series data of axial compressors. WaveNet can implement fast anomaly detection and spike-type stall prediction in long-time term series data. Furthermore, a single WaveNet can be trained to capture and learn the time-domain statistical characteristics of different spike-type stall inception training data with equal fidelity. The trained WaveNet model can rapidly detect the occurrence of anomaly point and predict the probability of rotating stall and surge in axial compressors as an early warning signal. By comparing with the time domain analysis, the calculation results are represented with experimental data to show the effectiveness and feasibility of spike-type stall detection approach based on WaveNet.

Text to Speech Synthesis in Celebrity’s Voice

This paper is proposed for text to speech synthesis. It uses neural network architecture for generation of speech and its synthesis directly from text in celebrity’s voice. The device is fitted with a recurring sequence-to-sequence prediction that graphs the embedding characters into mel scale spectrograms, followed by an updated WaveNet model that functions as a vocoder to create time-domain waveforms from those spectrograms. Here, project evaluation of the impact of mel spectrograms as the conditioning input to WaveNet rather than linguistic features, length, and F0. This paper further would be showing that utilizing this compact acoustic intermediate representation allows a significant reduction in the size of the WaveNet architecture. Using this technique, we are going to modulate the output of the vocoder according to the frequency and pitch of a specific celebrity. Using a unit selection method of concatenation synthesis, a database of prerecorded voice is collected. This paper includes creating a database of an Indian celebrity, clustering, indexing, and synthesizing it for creating a voice output with respect to the text as input. Also worked on normalization of text which includes abbreviations, acronyms, and linguistic analysis. This paper gives output for phonemic features, like vowel length, vowel height, frontness, consonant voicing, consonant poi, and position in the syllable and word.

A WaveNet based Ion Channel prediction Method

There are many diseases associated with the dysfunction of ion channels. Ion channels are a class of proteins embedded in cell membranes that serve principle physiological functions such as memorization, learning and pain signaling. In order to help scientists learn more about ion channels and how they cause diseases, a team in Liverpool University published a public dataset about Ion channels. This paper presents a Wavenet based model to predict the number of Ion channels open at each time point. Wavenet is a type of deep neural network designed for one-dimensional signals. We used the Kalman filter to denoise the original signals and compared WaveNet with classical machine learning methods like SVM and Naïve Bayes to show the superior performance of the WaveNet model.

Classification of heart sound signals using a novel deep WaveNet model

Background and objectivesThe high mortality rate and increasing prevalence of heart valve diseases globally warrant the need for rapid and accurate diagnosis of such diseases. Phonocardiogram (PCG) signals are used in this study due to the low cost of obtaining the signals. This study classifies five types of heart sounds, namely normal, aortic stenosis, mitral valve prolapse, mitral stenosis, and mitral regurgitation. MethodsWe have proposed a novel in-house developed deep WaveNet model for automated classification of five types of heart sounds. The model is developed using a total of 1000 PCG recordings belonging to five classes with 200 recordings in each class. ResultsWe have achieved a training accuracy of 97% for the classification of heart sounds into five classes. The highest classification accuracy of 98.20% was achieved for the normal class. The developed model was validated with a 10-fold cross-validation, thus affirming its robustness. ConclusionThe study results clearly indicate that the developed model is able to classify five types of heart sounds accurately. The developed system can be used by cardiologists to aid in the detection of heart valve diseases in patients.

Accuracy improvement of various short-term load forecasting models by a novel and unified statistical data-filtering method

ABSTRACTTime-series and machine-learning methods are being strongly exploited to improve the accuracy of short-term load forecasting (STLF) results. In developing countries, power consumption behaviors could be suddenly changed by different customers, e.g. industrial customers, residential customers, so the load-demand dataset is often unstable. Therefore, reliability assessment of the load-demand dataset is obviously necessary for STLF models. Hence, this paper proposes a novel and unified statistical data-filtering method with the best confidence interval to eliminate unexpected noises/outliers of the input dataset before performing various short-term load forecasting models. This proposed novel data-filtering method, so-called the data pre-processing method, is also compared to other existing data-filtering methods (e.g. Kalman filter, Density-Based Spatial Clustering of Applications with Noise, Wavelet transform, and Singular Spectrum Analysis). By using an SCADA system-based database of a typical 22kV distribution network in Vietnam, NYISO database, and PJM-RTO database, case studies of short-term load forecasting have been conducted with a conventional ARIMA model, an ANN forecasting model, an LSTM-RNN model, an LSTM-CNN combined model, a deep auto-encoder (DAE) network, a Wavenet-based model, a Wavenet and LSTM hybrid model, and a Wavelet Neural Network (WNN) model, which are to validate the novel and unified statistical data-filtering method proposed. The achieved numerical results demonstrate which the accuracy of the aforementioned STLF models can be significantly improved due to the proposed statistical data-filtering method with the best confidence interval of the input load dataset. The proposed statistical data-filtering method can considerably outperform the existing data-filtering methods.