Deep Learning for Coastal Erosion Assessment: Case Study of Vietnam’s Coastal Regions

Machine translation relates to highly autonomous software which is capable of translating source sentences into different languages. Previously some work was done in this sector where the result was comparatively low. Most of the researchers worked on common languages and none of them gave satisfactory Bilingual Evaluation Understudy (BLEU) score. Depending on these factors, we build a system of Bangla-German translator. This system can be used in various areas (i.e. reliable interpreters, business conduction, e-commerce merchandising, etc.). The system is built based on Gated Recurrent Unit (GRU) which is a gating mechanism of Recurrent Neural Network (RNN). Here, total five types of different RNN algorithms were used like Simple RNN, RNN with Embedding, Encoder-Decoder RNN, Bidirectional RNN, Hybrid RNN. All of them gave good accuracy. But the best result we got from the Hybrid model which was the combination of Embedded and Bidirectional algorithm. The accuracy was 85.69%. For further evaluation, BLEU score was used. The result of BLEU score of unigram to four gram was respectively increasing from 54.40% to 85.88%. Also the comparison between machine translated sentences and Google translated sentences showed that the system works very efficiently.

The electric scooter with nonlinear friction force of the transmission belt made the hybrid recurrent neural network (HRNN) control system with degenerated tracking responses. In order to overcome this problem, a hybrid recurrent wavelet neural network (HRWNN) control system is proposed to control for a permanent magnet synchronous motor (PMSM) driven electric scooter. The HRWNN control system consists of a supervisor control, a RWNN and a compensated control with adaptive law. The on-line parameter training methodology of the RWNN can be derived using adaptation laws and the Lyapunov stability theorem. The RWNN has the on-line learning ability to respond to the system’s nonlinear and time-varying behaviors. To show the effectiveness of the proposed controller, comparative studies with HRNN control system is demonstrated by experimental results.

Bidirectional deep recurrent neural networks for process fault classification

Bidirectional recurrent neural network (BRNN) is a non-causal generalization of recurrent neural networks (RNNs). Due to the problem of vanishing gradients, BRNN cannot learn long-term dependencies efficiently with gradient descent. To tackle the long-term dependency problem, we propose segmented-memory recurrent neural network (SM-RNN) and develop a bidirectional segmented-memory recurrent neural network(BSMRNN). We test the performance of BSMRNN on the problem of information latching. Our experimental results show that BSMRNN outperforms BRNN on long-term dependency problems.

A Recurrent Neural Network (RNN) is a specialized form of neural network that is adept at handling sequential data by retaining information from prior inputs. In contrast to conventional feedforward neural networks, RNNs incorporate loops in their architecture, allowing them to leverage data from previous time steps to affect the current output. This characteristic renders RNNs especially effective for applications that involve sequences, including time-series forecasting, natural language processing, and speech recognition. A fundamental component of RNNs is their hidden state, which acts as a dynamic memory that is refreshed with each incoming input. This allows RNNs to capture dependencies across time steps, which is crucial for understanding context in sequences. In language modeling, the interpretation of a word often relies on the words that come before it, a task that Recurrent Neural Networks (RNNs) handle well. However, RNNs struggle with issues like vanishing gradients, which hinder their ability to capture long-range dependencies. To overcome this, models such as Long Short-Term Memory (LSTM) networks and Gated Recurrent Units (GRUs) were introduced. These models incorporate gates that regulate the flow of information, allowing them to better learn long-term dependencies. RNNs remain a powerful tool for working with sequential data, facilitating the modeling of temporal relationships, but their effectiveness depends on careful design and optimization. Research significance: Recurrent Neural Networks (RNNs) hold significant research value because of their capacity to simulate temporal and sequential data, which is essential in many fields. They are frequently employed in natural language processing for tasks such as sentiment analysis, language translation, and text generation. In time-series analysis, RNNs enable accurate forecasting in finance, healthcare, and climate modeling. They also are essential in speech recognition and video processing, handling dependencies across time steps. Research focuses on improving RNNs, addressing challenges like vanishing gradients, and enhancing efficiency through architectures like LSTMs and GRUs, solidifying their relevance in advancing AI and machine learning applications. Methodology: A technique for analyzing the relationships between several variables, particularly in situations when data is limited or unclear, is called gray relational analysis, or GRA. In order to comprehend the relationships between variables, it evaluates how similar or different they are. GRA aids decision-makers in identifying critical factors, prioritizing actions, and improving processes in complex fields like engineering, finance, and management. By converting both qualitative and quantitative data into gray numbers, GRA addresses uncertainty and provides valuable insights for problem-solving, decision-making, and performance improvement, leading to more informed and effective strategies. Alternative taken as Simple RNN, LSTM, GRU, Bidirectional RNN, Deep RNN, Vanilla RNN, Echo State Network, Attention-based RNN, Transformer RNN, GRU with Attention. Evaluation preference taken as Prediction Accuracy, Model Robstness, Learning Efficiency, Training Time, Complexity. Attention-based RNN has the lowest score, Deep RNN has the highest rank, according to the results.

We presented a learning model that generated natural language description of images. The model utilized the connections between natural language and visual data by produced text line based contents from a given image. Our Hybrid Recurrent Neural Network model is based on the intricacies of Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), and Bi-directional Recurrent Neural Network (BRNN) models. We conducted experiments on three benchmark datasets, e.g., Flickr8K, Flickr30K, and MS COCO. Our hybrid model utilized LSTM model to encode text line or sentences independent of the object location and BRNN for word representation, this reduced the computational complexities without compromising the accuracy of the descriptor. The model produced better accuracy in retrieving natural language based description on the dataset.

Introduction: The nonlinear behaviour of activation functions is vital in Artificial Neural Networks (ANNs) for exploring the complex relationship between the input and output features. However, these are probably going to encounter vanishing gradient problems due to small gradients that lead to training instability, expensive exponent operations, and slow convergence. Objectives: The primary objective of this study is to develop Taylor expansion of the second order to realize the hyperbolic tangent and sigmoid functions. In particular, long short term memory network make extensive use of these functions as well as gating mechanism to control the flow of information and gradients. Both the custom functions can reduce the vanishing gradient issues in recurrent neural networks. Methods: Taylor expansion hyperbolic tangent and sigmoid activation functions based parallel heterogeneous Long Short Term Memory Network integrated with Bayesian hyperparameter Optimization is being proposed for coronavirus multi step time series prediction. a Min-Max Normalization is applied, which produces scaled data in the range (0, 1). The normalized dataset is partitioned into training and testing datasets, with 80% and 20%, respectively. Furthermore, both train and test datasets are prepared as input and target series using a window size of 5-7.The further proposed model is tuned with key hyperparameters such as the number of neurons, learning rate, dropout, and type of optimizer. The remaining model parameters are epochs, batch size, and loss, which are 200, 32, and mean square error, respectively. Results: The proposed model efficacy is evaluated on coronavirus daily cumulative cases, cumulative deaths, daily new cases, and total recovery cases in India. The Analysis reveals that the current model achieves remarkable performance in terms of Mean Absolute Percentage Error (MAPE), Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and Coefficient of determination (R2 Score) when compared to existing models. Conclusions: The study reveals that the proposed framework with the Taylor approximation activation function produces more consistency in prediction than the default activation functions, including Tanh and sigmoid. In spite of that, gradients of Taylor Tanh and sigmoid activation function traits indicate a decline in the possibility of vanishing issue.

Neural Network-based approaches have recently produced good performances in Natural language tasks, such as Supertagging. In the supertagging task, a Supertag (Lexical category) is assigned to each word in an input sequence. Combinatory Categorial Grammar Supertagging is a more challenging problem than various sequence-tagging problems, such as part-of-speech (POS) tagging and named entity recognition due to the large number of the lexical categories. Specifically, simple Recurrent Neural Network (RNN) has shown to significantly outperform the previous state-of-the-art feed-forward neural networks. On the other hand, it is well known that Recurrent Networks fail to learn long dependencies. In this paper, we introduce a new neural network architecture based on backward and Bidirectional Long Short-Term Memory (BLSTM) Networks that has the ability to memorize information for long dependencies and benefit from both past and future information. State-of-the-art methods focus on previous information, whereas BLSTM has access to information in both previous and future directions. Our main findings are that bidirectional networks outperform unidirectional ones, and Long Short-Term Memory (LSTM) networks are more precise and successful than both unidirectional and bidirectional standard RNNs. Experiment results reveal the effectiveness of our proposed method on both in-domain and out-of-domain datasets. Experiments show improvements about (1.2 per cent) over standard RNN.

The formation of protein secondary structure especially the regions of β-sheets involves long-range interactions between amino acids. We propose a novel recurrent neural network architecture called segmented-memory recurrent neural network (SMRNN) and present experimental results showing that SMRNN outperforms conventional recurrent neural networks on long-term dependency problems. In order to capture long-term dependencies in protein sequences for secondary structure prediction, we develop a predictor based on bidirectional segmented-memory recurrent neural network (BSMRNN), which is a noncausal generalization of SMRNN. In comparison with the existing predictor based on bidirectional recurrent neural network (BRNN), the BSMRNN predictor can improve prediction performance especially the recognition accuracy of β-sheets.

Bidirectional recurrent neural network (BRNN) is a noncausal system that captures both upstream and downstream information for protein secondary structure prediction. Due to the problem of vanishing gradients, the BRNN can not learn remote information efficiently. To limit this problem, we propose segmented memory recurrent neural network (SMRNN) and obtain a bidirectional segmented-memory recurrent neural network (BSMRNN) by replacing the standard RNNs in BRNN with SMRNNs. Our experiment with BSMRNN for protein secondary structure prediction on the RS126 set indicates improvement in the prediction accuracy.

Bidirectional recurrent neural network (BRNN) is a non-causal generalization of recurrent neural networks (RNNs). Due to the problem of vanishing gradients, BRNN cannot learn long-term dependencies efficiently with gradient descent. To tackle the long-term dependency problem, we propose segmented-memory recurrent neural network (SM-RNN) and develop a bidirectional segmented-memory recurrent neural network(BSMRNN). We test the performance of BSMRNN on the problem of information latching. Our experimental results show that BSMRNN outperforms BRNN on long-term dependency problems.

To predict the remaining useful life of supercapacitor, a data-based model is established by using a stacked bidirectional long short-term memory recurrent neural network. On the basis of the traditional long short-term memory recurrent neural network, a reverse recurrent layer with t time and subsequent time values in the input sequence is added. A stacked network can ensure enough capacity space. Simulation results show that the network has superior performance when the number of hidden layers is 2, the predicted RMSE and MAE are 0.0275 and 0.0241, respectively. Meanwhile, simulation compares ordinary and bidirectional recurrent neural networks and the bidirectional recurrent neural networks with different recurrent units. For subsequent ameliorate, this project will add swarm intelligence algorithm to optimize the initial weight of neural network and reduce the initial prediction error.

Because the wheel of V-belt continuously variable transmission (CVT) system driven by permanent magnet synchronous motor (PMSM) has much unknown nonlinear and time-varying characteristics, the better control performance design for the linear control design is a time consuming job. In order to overcome difficulties for design of the linear controllers, a hybrid recurrent Chebyshev neural network (NN) control system is proposed to control for a PMSM servo-driven V-belt CVT system under the occurrence of the lumped nonlinear load disturbances. The hybrid recurrent Chebyshev NN control system consists of an inspector control, a recurrent Chebyshev NN control with adaptive law and a recouped control. Moreover, the online parameters tuning methodology of adaptive law in the recurrent Chebyshev NN can be derived according to the Lyapunov stability theorem and the gradient descent method. Furthermore, the optimal learning rate of the parameters based on discrete-type Lyapunov function is derived to achieve fast convergence. The recurrent Chebyshev NN with fast convergence has the online learning ability to respond to the system's nonlinear and time- varying behaviors. Finally, to show the effectiveness of the proposed control scheme, comparative studies are demonstrated by experimental results.

In this paper, a transfer bi-directional recurrent neural networks (RNN) is proposed for named entity recognition (NER) in Chinese electronic medical records (EMRs) that aims to extract medical knowledge such as phrases recording diseases and treatments automatically. We propose a two-step procedure where the first step is to train a shallow bi-directional RNN in the general domain, and the second step is to transfer knowledge from the general domain to train a deeper bi-directional RNN for recognizing medical concepts from Chinese EMRs. Specifically, this is achieved by initializing the shallow parts of the deeper network in the second step with parameter weights from the bi-directional RNN trained in the first step. Then the deeper networks are re-trained on the Chinese EMRs. Experimental results show that NER performances are improved by the transferred knowledge significantly.

A new hybrid recurrent neural network (HRNN) for machining process modeling is presented based on the diagonal recurrent neural network (DRNN). In order to overcome the weakness of back propagation (BP) algorithm, a generalized entropy square error (GESE) criterion is defined and a dynamic recurrent back propagation algorithm is developed to guarantee the global convergence. The HRNN based on the GESE is then used for nonlinear system identification and neural network modeling of the machining process. The numeral experiments results show that the HRNN has better approximate effectiveness, tracking and dynamic performance than traditional BP neural network.