Fusion of deep recurrent neural network models and fuzzy decision support system for tweet sentiment analysis and classification

The National Early Warning Score, Modified Early Warning Score, and quick Sepsis-related Organ Failure Assessment can predict clinical deterioration. These scores exhibit only moderate performance and are often evaluated using aggregated measures over time. A simulated prospective validation strategy that assesses multiple predictions per patient-day would provide the best pragmatic evaluation. We developed a deep recurrent neural network deterioration model and conducted a simulated prospective evaluation. Retrospective cohort study. Four hospitals in Pennsylvania. Inpatient adults discharged between July 1, 2017, and June 30, 2019. None. We trained a deep recurrent neural network and logistic regression model using data from electronic health records to predict hourly the 24-hour composite outcome of transfer to ICU or death. We analyzed 146,446 hospitalizations with 16.75 million patient-hours. The hourly event rate was 1.6% (12,842 transfers or deaths, corresponding to 260,295 patient-hours within the predictive horizon). On a hold-out dataset, the deep recurrent neural network achieved an area under the precision-recall curve of 0.042 (95% CI, 0.04-0.043), comparable with logistic regression model (0.043; 95% CI 0.041 to 0.045), and outperformed National Early Warning Score (0.034; 95% CI, 0.032-0.035), Modified Early Warning Score (0.028; 95% CI, 0.027- 0.03), and quick Sepsis-related Organ Failure Assessment (0.021; 95% CI, 0.021-0.022). For a fixed sensitivity of 50%, the deep recurrent neural network achieved a positive predictive value of 3.4% (95% CI, 3.4-3.5) and outperformed logistic regression model (3.1%; 95% CI 3.1-3.2), National Early Warning Score (2.0%; 95% CI, 2.0-2.0), Modified Early Warning Score (1.5%; 95% CI, 1.5-1.5), and quick Sepsis-related Organ Failure Assessment (1.5%; 95% CI, 1.5-1.5). Commonly used early warning scores for clinical decompensation, along with a logistic regression model and a deep recurrent neural network model, show very poor performance characteristics when assessed using a simulated prospective validation. None of these models may be suitable for real-time deployment.

In this paper, we present design, implementation and evaluation of a novel predictive control framework to enable reliable distributed stream data processing, which features a Deep Recurrent Neural Network (DRNN) model for performance prediction, and dynamic grouping for flexible control. Specifically, we present a novel DRNN model, which makes accurate performance prediction with careful consideration for interference of co-located worker processes, according to multilevel runtime statistics. Moreover, we design a new grouping method, dynamic grouping, which can distribute/re-distribute data tuples to downstream tasks according to any given split ratio on the fly. So it can be used to re-direct data tuples to bypass misbehaving workers. We implemented the proposed framework based on a widely used Distributed Stream Data Processing System (DSDPS), Storm. For validation and performance evaluation, we developed two representative stream data processing applications: Windowed URL Count and Continuous Queries. Extensive experimental results show: 1) The proposed DRNN model outperforms widely used baseline solutions, ARIMA and SVR, in terms of prediction accuracy; 2) dynamic grouping works as expected; and 3) the proposed framework enhances reliability by offering minor performance degradation with misbehaving workers.

Dissolved oxygen (DO) concentration in water is one of the key parameters for assessing river water quality. Artificial intelligence (AI) methods have previously proved to be accurate tools for DO concentration prediction. This study presents the implementation of a deep learning approach applied to a recurrent neural network (RNN) algorithm. The proposed deep recurrent neural network (DRNN) model is compared with support vector machine (SVM) and artificial neural network (ANN) models, formerly shown to be robust AI algorithms. The Fanno Creek in Oregon (USA) is selected as a case study and daily values of water temperature, specific conductance, streamflow discharge, pH, and DO concentration are used as input variables to predict DO concentration for three different lead times ("t + 1," "t + 3," and "t + 7"). Based on Pearson's correlation coefficient, several input variable combinations are formed and used for prediction. The model prediction performance is evaluated using various indices such as correlation coefficient, Nash-Sutcliffe efficiency, root mean square error, and mean absolute error. The results identify the DRNN model ([Formula: see text]) as the most accurate among the three models considered, highlighting the potential of deep learning approaches for water quality parameter prediction.

Identifying built environment barriers to walkability is the first step toward monitoring and improving our walking environment. Although conventional approaches (i.e., surveys by experts or pedestrians, walking interviews, etc.) to identify built environment barriers have contributed to improving the walking environment, these approaches may require time and effort. To address the limitations of conventional approaches, wearable sensing technologies and data analysis techniques have recently been adopted in the investigation of the built environment. Among various wearable sensors, an inertial measurement unit (IMU) can continuously capture gait-related data, which can be used to identify built environment barriers to walkability. To propose a more efficient method, the author adopts a cascaded bidirectional and unidirectional long short-term memory (LSTM)-based deep recurrent neural network (DRNN) model for classifying human gait activities (normal and abnormal walking) according to walking environmental conditions (i.e., normal and abnormal conditions). This study uses 101,607 gait data collected from the author’s previous study for training and testing a DRNN model. In addition, 31,142 gait data (20 participants) have been newly collected to validate whether the DRNN model is feasible for newly added gait data. The gait activity classification results show that the proposed method can classify normal gaits and abnormal gaits with an accuracy of about 95%. The results also indicate that the proposed method can be used to monitor environmental barriers and improve the walking environment.

Aim: Continuous blood pressure (BP) monitoring can provide invaluable information for cardiovascular disease (CVD) diagnosis. The purpose of this study is to develop a deep recurrent neural network (RNN) model with an optimal feature set of photoplethysmogram (PPG) and electrocardiogram (ECG) signals for continuous BP estimation. Methods: This paper presents a novel deep recurrent neural network (RNN), which consists of 2-layered bidirectional Long Short-term Memory (Bi-LSTM) and 6-layered LSTM networks. It is used to estimate BP based on the optimal feature set of PPG and ECG signals. In this work, the optimal feature set is determined using five different feature selection methods. Results: The proposed method is evaluated based on 660 subjects from the University of California Irvine (UCI) machine learning repository. The RNN model with optimal feature set achieved root mean square error (RMSE) of 3.223 and 1.781 mmHg for systolic BP (SBP) and diastolic BP (DBP), respectively. It also showed mean absolute error (MAE) of 2.514 and 1.383 mmHg for SBP and DBP, respectively. Regarding the British Hypertension Society (BHS) standard, the results attained grade A for the estimation of SBP and DBP. Conclusion: The experimental results suggest that the proposed deep RNN model with an optimal feature set can improve the performance of BP prediction. Thus, it is possible to further apply our proposed method to develop a wearable device for real-time BP monitoring.

Impact load identification of nonlinear structures using deep Recurrent Neural Network

This paper presents a new model structure and structure selection procedure for the identification of control-oriented affine quasi-LPV (qLPV) models in state-space (SS) representation using deep Recurrent Neural Networks (RNNs). The proposed model structure is intended to be an alternative to the existing black-box approaches [1], [2] for the case where the scheduling variables depend on internal states of the system, which are assumed to be unknown. Existing identification approaches are not able to incorporate deep neural network (DNN) structures but employ (normalized) radial basis functions (RBFs) to model the dependence of the time-varying parameters on the scheduling variables. This may increase the dimension of the time-varying parameter vector unnecessarily, making parameter estimation more difficult and limiting the model's use for LPV controller synthesis. The proposed identification approach aims to reduce the required dimension of the time-varying parameter vector by using a (Deep) Neural Network (NN) to model the parameter variation. In order to curb the complexity of the resulting nonlinear optimization problem and make the developed model approach useful in real-life applications, a structure selection procedure based on an initialization method developed in [3] is proposed. The performance of the presented approach is demonstrated on a nonlinear system identification problem.

Sentiment Analysis of Malayalam tweets is done using Deep Neural Networks (DNN). 1-D Convolutional Neural Networks (1-D CNN), Recurrent Neural Networks (RNN), Long Short Term Memory (LSTM), Bi-directional LSTM (BiLSTM) and Gated Recurrent Unit (GRU) models are used to classify Malayalam tweets as positive and negative. This work compared the results of Sentiment Analysis (SA) of Malayalam tweets with different DNN models on our manually annotated corpus containing 5468 Malayalam tweets. The SA of Malayalam tweets with GRU shows better accuracy compared with other DNN models.

Nowadays, drones are not just deployed for defense and military establishments, but they are widely used in many applications such as natural disaster monitoring, soil and crop analysis, road and traffic surveillance, and consumer product delivery. Some information, such as drone identification and flight modes, can be transmitted to other drones. This information can be shared between drones by using radio frequency (RF) signals and through 5G networks. Recently, few studies have been proposed to use deep neural networks (DNNs) on RF signals for identifying drones and detecting their flight modes, such as off, on and connected, flying, hovering, and video recording. However, transmitting RF signals between drones and 5G nodes needs to be secure and decentralized; in addition, the performance of identification and detection needs to be more accurate. In this article, we introduce a framework that combines a blockchain with a deep recurrent neural network (DRNN) and edge computing for 5G-enabled drone identification and flight mode detection. In the proposed framework, raw RF signals of different drones under several flight modes are remotely sensed and collected on a cloud server to train a DRNN model and then distribute the trained model on edge devices for detecting drones and their flight modes. Blockchain is used in the proposed framework for data integrity and securing data transmission. The DRNN model is evaluated on a public dataset, called DroneRF. Experimental evaluation results show that the DRNN model of the proposed framework can detect drones and their flight modes from real RF signals with high accuracy as compared to recent related work.

In the present era, deep learning algorithms are the key elements of several state-of-the-art solutions. But developing these algorithms for production requires a huge volume of data, computational resources, and human expertise. Thus, illegal reproduction, distribution, and modification of these models can cause economic damage to developers and can lead to copyright infringement. We propose a novel watermarking algorithm for deep recurrent neural networks based on adversarial examples that can verify the ownership of the model in a black-box way. In this paper, a novel algorithm to watermark a popular pre-trained speech-to-text deep recurrent neural network model Deep Speech without affecting the accuracy of the model is demonstrated. Watermarking is done by generating a set of adversarial examples by adding noise to the input such that the DeepSpeech model predicts the given input as the target string. In the case of copyright infringement, these adversarial examples can be used to verify ownership of the model. If the alleged stolen model predicts the same target string for the adversarial examples, the ownership of the model is verified. This novel watermarking algorithm can minimize the economic damage to the owners of the deep learning models due to stealing and plagiarizing.

Daily runoff forecasting by deep recursive neural network

Stock market has received widespread attention from investors. It has always been a hot spot for investors and investment companies to grasp the change regularity of the stock market and predict its trend. Currently, there are many methods for stock price prediction. The prediction methods can be roughly divided into two categories: statistical methods and artificial intelligence methods. Statistical methods include logistic regression model, ARCH model, etc. Artificial intelligence methods include multi-layer perceptron, convolutional neural network, naive Bayes network, back propagation network, single-layer LSTM, support vector machine, recurrent neural network, etc. But these studies predict only one single value. In order to predict multiple values in one model, it need to design a model which can handle multiple inputs and produces multiple associated output values at the same time. For this purpose, it is proposed an associated deep recurrent neural network model with multiple inputs and multiple outputs based on long short-term memory network. The associated network model can predict the opening price, the lowest price and the highest price of a stock simultaneously. The associated network model was compared with LSTM network model and deep recurrent neural network model. The experiments show that the accuracy of the associated model is superior to the other two models in predicting multiple values at the same time, and its prediction accuracy is over 95%.

The stability of engineering structures will be greatly affected when they are subjected to stationary random loads. Such loads must be accurately identified to ensure the structural safety. Due to the complexity of stationary random loads and difficulty of solving inverse problems, traditional methods require improvement in identification accuracy. In this work, a stationary random load identification method based on ensemble empirical mode decomposition (EEMD) and deep recurrent neural network (RNN) is proposed. The method designs a deep RNN model consisting of one gated recurrent unit (GRU) layer, one bidirectional long short-term memory (BLSTM) layer, two long short-term memory (LSTM) layers, and two fully connected (FC) layers in sequence. The load and response signals are decomposed into a set of intrinsic mode functions (IMF) from high to low frequencies using EEMD, with the decomposed signals serving as the output and input of the network, respectively. The effectiveness of the proposed method is verified by the simulation data of a three-degree-of-freedom (3DOF) linear system, and the results show that the proposed method is with higher accuracy than using only deep RNN without signal decomposition. The method is verified by the experimental data of a clamped beam as well, with results showing a high accuracy of stationary random load identification.

Speaker identification is a classification task which aims to identify a subject from a given time-series sequential data. Since the speech signal is a continuous one-dimensional time series, most of the current research methods are based on convolutional neural network (CNN) or recurrent neural network (RNN). Indeed, these methods perform well in many tasks, but there is no attempt to combine these two network models to study the speaker identification task. Due to the spectrogram that a speech signal contains, the spatial features of voiceprint (which corresponds to the voice spectrum) and CNN are effective for spatial feature extraction (which corresponds to modeling spectral correlations in acoustic features). At the same time, the speech signal is in a time series, and deep RNN can better represent long utterances than shallow networks. Considering the advantage of gated recurrent unit (GRU) (compared with traditional RNN) in the segmentation of sequence data, we decide to use stacked GRU layers in our model for frame-level feature extraction. In this paper, we propose a deep neural network (DNN) model based on a two-dimensional convolutional neural network (2-D CNN) and gated recurrent unit (GRU) for speaker identification. In the network model design, the convolutional layer is used for voiceprint feature extraction and reduces dimensionality in both the time and frequency domains, allowing for faster GRU layer computation. In addition, the stacked GRU recurrent network layers can learn a speaker’s acoustic features. During this research, we tried to use various neural network structures, including 2-D CNN, deep RNN, and deep LSTM. The above network models were evaluated on the Aishell-1 speech dataset. The experimental results showed that our proposed DNN model, which we call deep GRU, achieved a high recognition accuracy of 98.96%. At the same time, the results also demonstrate the effectiveness of the proposed deep GRU network model versus other models for speaker identification. Through further optimization, this method could be applied to other research similar to the study of speaker identification.

A large number of videos are generated and uploaded to video websites (like youku, youtube) every day and video websites play more and more important roles in human life. While bringing convenience, the big video data raise the difficulty of video summarization to allow users to browse a video easily. However, although there are many existing video summarization approaches, the key frames selected fail to integrate the large video contexts and the qualities of the summarized results are difficult to evaluate because of the lack of ground-truth. Inspired by the previous methods that extract key frames, we propose a deep recurrent neural network model, which learns to extract category-driven key frames. First, we sequentially extract a fixed number of key frames using time-dependent location networks. Second, we utilize recurrent neural network to integrate information of the key frames to classify the category of the video. Therefore, the quality of the extracted key frames could be evaluated by the categorization accuracy. Experiments on a 500-video dataset show that the proposed scheme extracts reasonable key frames and outperforms other methods by quantitative evaluation.