Order Recurrent Neural Network Research Articles

BioSwarmNet: A Revolutionary Approach to Brain Tumour Detection Using Fractional Order Differential Particle Swarm Optimisation and Recurrent Neural Networks

BioSwarmNet: A Revolutionary Approach to Brain Tumour Detection Using Fractional Order Differential Particle Swarm Optimisation and Recurrent Neural Networks

Movie Recommendation System using RNN Approach

The problem of information overload has been made worse by the quick advancement of information technology and the quick expansion of the Internet. In response to this issue, recommender systems have developed, assisting users in finding relevant material. The study of personalized recommendation services has seen a change in focus as a result of the complexity of the social setting. We present a novel recommendation approach based on social network recurrent neural network in order to address the sparsity problem of recommender systems while enhancing their accuracy and diversity in complicated scenarios. Using this approach, we group users and take a number of intricate criteria into account.

Credit Risk Modelling Using RNN-LSTM Hybrid Model for Digital Financial Institutions

In response to the rapidly evolving financial market and the escalating concern surrounding credit risk in digital financial institutions, this project addresses the urgency for accurate credit risk prediction models. Traditional methods such as Neural network models, kernel-based virtual machines, Z-score, and Logit (logistic regression model) have all been used, but their results have proven less than satisfactory. The project focuses on developing a credit scoring model specifically tailored for digital financial institutions, by leveraging a hybrid model that combines long short-term memory (LSTM) networks with recurrent neural networks (RNN). This innovative approach capitalizes on the strengths of the Long-Short Term Memory (LSTM) for long-term predictions and Recurrent Neural Network (RNN) for its recurrent neural network capabilities. A key component of the approach is feature selection, which entails extracting a subset of pertinent features from the credit risk data using RNN in order to help classify loan applications. The researcher chose to use data from Kaggle to study and compare the efficacy of different models. The findings reveal that the RNN-LSTM hybrid model outperforms other RNNs, LSTMs, and traditional models. Specifically, the hybrid model demonstrated distinct advantages, showcasing higher accuracy and a superior Area Under the Curve (AUC) compared to individual RNN and LSTM models. While RNN and LSTM models exhibited slightly lower accuracy individually, their combination in the hybrid model proved to be the optimal choice. In summary, the RNN-LSTM hybrid model developed stands out as the most effective solution for predicting credit risk in digital financial institutions, surpassing the performance of standalone RNN and LSTM models as well as traditional methodologies. This research contributes valuable insights for banks, regulators, and investors seeking robust credit risk assessment tools in the dynamic landscape of digital finance.

Blockchain-Based Decentralized Frequency Control of Microgrids Using Federated Learning Fractional-Order Recurrent Neural Network

This paper presents a blockchain-based decentralized frequency control of an islanded microgrid (MG) using a novel federated learning fractional order recurrent neural network (FL-FORNN). A self-adaptive proportional-integral-derivative (PID) controller is proposed using FL-FORNN to handle the uncertainties of power generation by the prosumers while participating in the trading of the islanded MG. In this work, the frequency oscillations during peer-to-peer (P2P) energy trading are regulated through the appropriate design of an adaptive controller and the implementation of a smart contract participation matrix (SCPM) for balancing the generation and consumer demand. A Proof-of-Authority (PoA) private blockchain-based SCPM has been developed to secure the contract among peers. The SCPM provides the power demand information from a consumer to a prosumer who participates in the P2P energy market. It also provides power reference to the prosumer distributed generation as a supplementary control, in addition to the secondary frequency control of the system. To study this, an islanded MG comprising four source nodes (prosumers) and three demand nodes (consumers) are tested with the implementation of the PoA-based blockchain framework. With its SCPM calculation running in the blockchain framework, the decentralized control of prosumers is implemented by interfacing the OPAL-RT with Raspberry Pi devices. The robustness of the proposed FL-FORNN controller has been verified with the FORNN tuned PID controller. Furthermore, a comparative analysis is made with the centralized scheme and traditional decentralized frequency control technique.

Simulation of liquid production and water cut dynamics using fluid flow model and neural networks

In the oil industry, there is a noticeable tendency to use proxy modeling of various levels of complexity to perform operational predictive calculations, in particular machine learning methods that are actively developing in the context of digitalization and intellectualization of production processes. In this paper, using the example of a synthetic oil reservoir model development element, we present an approach to the joint use of a physically meaningful fluid flow model and machine learning methods for solving adaptation and prediction problems. A feature of the considered synthetic model is the presence of a pronounced zonal inhomogeneity of the permeability field. Within the framework of the proposed approach, a single-phase filtration model, simplified in comparison with the original formulation was used, the history matching of which was carried out by restoring the field of reservoir filtration parameters using a network of radial basis functions. Based on the reconstructed field, the connection coefficients between the wells were calculated, which qualitatively and quantitatively correspond to the true well connections. The next step was to train a recurrent neural network in order to predict the water cut of the produced fluid. The use of a recurrent neural network made it possible to reproduce the characteristic non-monotonic behavior of the water cut of the produced fluid, caused by non-stationary modes of operation of injection and production wells. A combination of the presented models makes it possible to predict the volume of the produced fluid and its phase composition. To assess the predictive properties of the models, the actual data set was divided into training and test intervals.

Recommending Profiles To Social Event Participation In Online Social Network Service Applications Using Deep Learning Techniques

Nowadays, organizing online Social events is growing especially in current pandemic situation due to corona. Thus social network services helps to organize the event oriented online social gathering. The events discussed on social networks can be associated with topics, locations, and time periods. Various literatures have employed clustering mechanism to group the profiles of the user of social networking applications. More research has been carried out in event based online network services as existing models leads to scalability and sparsity problems. Event-based online social networks are used to maintain interest-based groups with high relevancy rate, recommendation quality and predictive accuracy. In order to achieve the above goal, in this paper, we propose a novel framework named as Deep Influence Predict (DIP) which explores the features of Recurrent Neural Network in order identify the target or potential users through different patterns and behaviours of the profile on the social networking service applications. It learns multiple levels of representations and abstractions of the latent data through individual participation record. Further, it extracts the extrinsic and intrinsic properties of the profiles on their social connections and social effects. Specifically, it identifies the distinguishing social groups with different topics and categories as multifaceted interest in iterative process. Finally decision of recommendation for the event is integrated on outcomes of user behaviour model through personnel impact, social relation and equilibrium. Evaluation of the proposed model through various case studies has been implemented using hadoop architecture and validated across various measures such as accuracy on precision, Recall and f measure along scalability and Execution time.

A Parallel Cross Convolutional Recurrent Neural Network for Automatic Imbalanced ECG Arrhythmia Detection with Continuous Wavelet Transform.

Automatic detection of arrhythmia using electrocardiogram (ECG) and deep learning (DL) is very important to reduce the global death rate from cardiovascular diseases (CVD). Previous studies on automatic arrhythmia detection relied largely on various ECG features and have achieved considerable classification accuracy using DL-based models. However, most previous research has ignored multi-class imbalanced problems in ECG arrhythmia detection. Therefore, it remains a challenge to improve the classification performance of the DL-based models. This paper proposes a novel parallel cross convolutional recurrent neural network in order to improve the arrhythmia detection performance of imbalanced ECG signals. The proposed model incorporates a recurrent neural network and a two-dimensional (2D) convolutional neural network (CNN) and can effectively learn temporal characteristics and rich spatial information of raw ECG signals. Continuous wavelet transform (CWT) is used to transform the ECG signals into a 2D scalogram composed of time–frequency components, and subsequently, the 2D-CNN can learn spatial information from the 2D scalogram. The proposed model is not only efficient in learning features with imbalanced samples but can also significantly improve model convergence with higher accuracy. The overall performance of our proposed model is evaluated based on the MIT-BIH arrhythmia dataset. Detailed analysis of evaluation metrics reveals that the proposed model is very effective in arrhythmia detection and significantly better than the existing hierarchical network models.

Approximate inferring with confidence predicting based on uncertain knowledge graph embedding

Uncertainty is a natural character of knowledge, while it is still tough to be encoded into the knowledge graph embedding space that can be employed for machine learning tasks. However, the approximate inference could be performed in the embedding space, if confidence, real-value representation of the uncertainty of knowledge facts, can be learned by neural networks. To tackle this, a simple yet effective confidence predicting method is proposed, and several approximate inferring are efficiently performed based on these predictions. The model is a two-step model: knowledge elements embedding step, in which knowledge facts regarded as short sentences are fed into the natural language model to get entity and relation embedding vectors; and confidence learning step, in which the confidence distribution of knowledge facts in the knowledge graph are learned utilizing the recurrent neural network in order to carry out approximate inference. The experience demonstrates that the model achieves better results than state-of-the-art on the link prediction task over uncertain knowledge graph embedding. Uncertainty inferring grounded on predicted confidence is more accurate, feasible, and meaningful for serval knowledge inferring tasks: transitivity, composition inferring, and probabilistic soft logic inferring. Likewise, the proposed approach achieves the best tradeoff between efficiency and accuracy of uncertain knowledge graph embedding and inferring, and can be used to handle large size knowledge graphs at lower time consumption because of the simplicity.

Interval-Valued Reduced RNN for Fault Detection and Diagnosis for Wind Energy Conversion Systems

Recurrent neural network (RNN) is one of the most used deep learning techniques in fault detection and diagnosis (FDD) of industrial systems. However, its implementation suffers from some limitations presented in the hard training step and the high time complexity. Besides, most used RNN-based FDD techniques do not deal with system uncertainties. Therefore, this paper proposes enhanced RNN techniques that detect and classify faults in wind energy conversion (WEC) systems. First, we develop a reduced RNN in order to simplify the model in terms of training and complexity time as well. Reduced RNN is based on Hierarchical K-means clustering to treat the correlations between samples and extract a reduced number of observations from the training data matrix. Second, two reduced RNN-based interval-valued-data techniques are proposed to distinguish between the different WEC system operating modes. The proposed techniques for interval-valued data are able to improve both fault diagnosis robustness and susceptibility while maintaining a satisfactory and stable performance over long periods of process operation. The presented results confirm the high feasibility and effectiveness of the proposed FDD techniques (an accuracy greater than 98% for all the proposed methods).

Deep Learning Based Phishing Websites Detection

Phishing is a crime that involves the theft of confidential user information. Those targeted by phishing websites include individuals, small businesses, cloud storage providers, and government organisations and websites. The majority of phishing prevention techniques involve hardware-based solutions, although software-based options are preferred due of cost and operational considerations. There is no answer to the problem of zero-day phishing assaults from the present phishing detection approaches since there is no solution to the problem. The Phishing Attack Detector based on Web Crawler, a three-phase attack detection system, was designed to handle these issues and accurately detect phishing incidences using a recurrent neural network in order to resolve these issues. It covers the input aspects of web traffic, web content, and Uniform Resource Locator (URL) based on the classification of phishing and non-phishing pages, as well as the output features of phishing and non-phishing pages.

Computer Network Intrusion Anomaly Detection with Recurrent Neural Network

Network intrusion anomaly detection technique has been widely employed in computer network environments as a highly effective security prevention method. As network technology and network applications have advanced at a rapid pace, so too has network data traffic, resulting in an increase in virus and attack kinds. In the face of large-scale traffic and characteristic information, traditional intrusion detection will have problems such as low detection accuracy, high false negatives, and reliance on dimensionality reduction algorithms. Therefore, it is particularly important to establish a fast and efficient network intrusion anomaly detection method to deal with the current complex network environment. This work designs a computer network intrusion detection model with a recurrent neural network in order to explore a new intrusion detection method. The main purpose of this article include the following: (1) design a network security emergency response system architecture with the recurrent neural network model. This system consists of a management center module, a knowledge database module, a data acquisition module, a risk detection tool module, a risk analysis and processing module, a data protection module, and a remote connection auxiliary module. The modules cooperate with each other to complete system functions. (2) Aiming at the risk analysis and processing module, a network intrusion detection model combining bidirectional long short-term memory (BiLSTM) and deep neural network (DNN) is designed. In view of the lack of consideration of the before-and-after relevance of intrusion data features and the multifeature problem in existing models, the use of BiLSTM to extract the relevance between features and the use of DNN to extract deeper features are proposed. Aiming at the problem that the model lacks consideration of the importance of features, it is proposed to embed an attention mechanism into the network to increase consideration for the importance of features. (3) Massive experiments have verified the reliability and effectiveness of the method proposed in this work.

Regional soil moisture prediction system based on Long Short-Term Memory network

In the context of climate change, drought has been recognised as one of the most severe threats for agricultural production since absence of water is one of the most limiting factors for the growth of plants. In this study, a regional system for soil moisture prediction based on ERA5 climate reanalysis dataset, an open-source meteorological dataset issued by Copernicus Climate Change Service, was developed. It consisted of the relevant meteorological parameters for Serbia, during the period 2011–2020. Daily values of maximum and minimum air temperature, precipitation and vapour pressure deficit were used as features, and they were fed to recurrent neural network in order to predict volumetric soil moisture for three days ahead. A Long Short-Term Memory (LSTM) network was designed and trained at a regional scale using the data from the 2011–2016 period, at 28 locations that cover four major soil types. Validation was done on 2017–2018 data and LSTM was compared with a statistical forecasting technique and a classical machine learning approach. Evaluation was done with error measures commonly used in literature. The resulting network yielded the lowest errors and proved to have good generalisation properties. The system was tested from September 2019 to April 2020 using short-range weather forecasts of Yr service. It will present the cornerstone of the irrigation scheduling service in AgroSense.rs, the Serbian national platform for digital agriculture.

Keyword Search Using Attention-Based End-to-End ASR and Frame-Synchronous Phoneme Alignments

Attention-based end-to-end (E2E) automatic speech recognition (ASR) architectures are now the state-of-the-art in terms of recognition performance. However, despite their effectiveness, they have not been widely applied in keyword search (KWS) tasks yet. In this paper, we propose the Att-E2E-KWS architecture, an attention-based E2E ASR framework for KWS that can afford accurate and reliable keyword retrieval results. First, we design a basic framework to carry out KWS based on attention-based E2E ASR. We adopt the connectionist temporal classification and attention (CTC/Att) joint E2E ASR architecture and exploit the spike posterior property of CTC to provide the keywords time stamps. Second, we introduce the frame-synchronous phonemes modeling and use the dynamic programming (DP) algorithm to provide alignments between E2E grapheme outputs and phoneme outputs. We call this alignment procedure dynamic time alignment (DTA), which can provide the proposed Att-E2E-KWS system with more accurate time stamps and reliable confidence scores. Third, we use the Transformer, a self-attention-based encoder-decoder neural network, in place of conventional recurrent neural networks in order to yield more parallelizable models and increased training speed. We conduct comprehensive experiments on English and Mandarin Chinese. To the best of our knowledge, this is the first practical Att-E2E-KWS framework, and experimental results on Switchboard and HKUST corpora show that our proposed Att-E2E-KWS systems significantly outperform the CTC E2E ASR based KWS baselines.

LOW COMPLEXITY RECURRENT NEURAL NETWORKS FOR EDGE COMPUTING

This paper is dedicated to the development of recurrent neural networks in order to supplement the AI-based devices like microcontrollers and other mist computing systems. Due to the insignificant computational power of the edge deviсes the aim of the study is to design and analyze low complexity sequence models for a basic sensory time series forecasting on an example of univariate indoor temperature data. The description of data preparation and transformation followed by the models configuration via different architectures like simple LSTM and GRU is provided. To calculate an optimal set of hyper-parameters for the multiple neural network architectures a genetic algorithm has been implemented. The results of numerical experiments conducted for each model configuration consisting of both unidirectional and bidirectional cell connections are discussed. In addition to these studies the scheme of deploying the developed low-complexity models on STM32 microcontroller joined with the high-performance hub is proposed. Key words : edge computing, recurrent neural networks, time series, genetic algorithm.

Fractional Complex Dynamical Systems for Trajectory Tracking using Fractional Neural Network via Fractional Order PID Control Law

In this paper the problem of trajectory tracking is studied.Based on the Lyapunov theory, a Fractional Order PIDcontrol lawthat achieves the global asymptotic stability of the trackingerror between a fractional order recurrent neural network anda fractional order complex dynamical network is obtained. Toillustrate the analytic results we present a tracking simulationof a dynamical network with each node being just one frac-tional order Lorenz´ s dynamical system and three identicalfractional order Chen’s dynamical systems.

Fractional Complex Dynamical Systems for Trajectory Tracking using Fractional Neural Network via Fractional Order PID Control Law

In this paper the problem of trajectory tracking is studied.Based on the Lyapunov theory, a Fractional Order PIDcontrol lawthat achieves the global asymptotic stability of the trackingerror between a fractional order recurrent neural network anda fractional order complex dynamical network is obtained. Toillustrate the analytic results we present a tracking simulationof a dynamical network with each node being just one frac-tional order Lorenz´ s dynamical system and three identicalfractional order Chen’s dynamical systems.

Efficient Strategies of Static Features Incorporation into the Recurrent Neural Network

Recurrent neural networks (RNNs) have evolved to become one of the most powerful tools for making predictions on sequenced data, such as time series, textual data, signals, music etc. In many real-life cases, however, sequenced data are additionally characterized by static features which, due to their non-sequential nature, cannot be transferred directly into RNNs. In this paper, we discuss a method which incorporates static features into RNNs in order to influence and generalize the learning process. Furthermore, we will demonstrate that our approach significantly enhances the performance of RNNs, enabling the networks to learn the sequenced data exhibiting varying characteristics and then distinguish between them through the use of static supplementary information. Finally, we will evaluate our model against real energy consumption measurements of energy time series and verify that high-accuracy demand forecasts for different types of customers can be achieved only by way of incorporation of static features.

Интернет-дискуссия как форма электронного участия: российская специфика

Авторский подход, изложенный в статье, предлагает теоретически и технологически обоснованные способы дискурс-анализа повседневных дискуссий граждан в интернете для понимания степени их делиберативности и увязки результатов дискуссий с существующим политическим процессом. В качестве теоретической основы используется теория коммуникативной этики Юргена Хабермаса в рамках его модели делиберативной демократии, т.е. демократии, основанной на нормативной и морально-этической необходимости обсуждения различных точек зрения в публичном пространстве современного общества для их последующего сближения. Для тестирования предлагаемого авторами методологического подхода использовались интернет-дискуссии по вопросам уничтожения антисанкционных продуктов и повышения пенсионного возраста в России. Был проведен дискурс-анализ более пяти тысяч постов, сделанных участниками дискуссии на интернет-форумах в разных российских городах. Кодировался ряд важных параметров интернет-дискуссий с точки зрения их делиберативности. Согласно результатам исследования, анализ текущих интернет-дискуссий как делиберативных практик дает возможность выявить важные нюансы, касающиеся интерпретации значимых событий, которые трудно получить с помощью традиционных социологических методик исследования общественных настроений или с применением таких автоматизированных методов исследования, как анализ тональности высказываний вне прямой связи с их морально-этическим обоснованием. Эмпирические данные, полученные в результате дискурс-анализа, послужили основой для проведения эксперимента по использованию алгоритмов глубинного машинного обучения реккурентной нейросети, с тем чтобы обучить ее предсказывать позиции участников дискуссий на основании их высказываний. Эксперимент выявил возможности, условия и ограничения применения технологий искусственного интеллекта для более глубокого понимания результатов общественных дискуссий.
 Благодарность. Работа выполнена при поддержке РНФ, проект №18-18-00360 «Электронное участие как фактор динамики политического процесса и процесса принятия государственных решений». Авторы благодарят студентов Даниила Волковского (СПбГУ) и Петра Бегена (Университет ИТМО) за участие в эмпирических исследованиях, которые представлены в данной статье.

Combining Convolutional Neural Network With Recursive Neural Network for Blood Cell Image Classification

The diagnosis of blood-related diseases involves the identification and characterization of a patient’s blood sample. As such, automated methods for detecting and classifying the types of blood cells have important medical applications in this field. Although deep convolutional neural network (CNN) and the traditional machine learning methods have shown good results in the classification of blood cell images, they are unable to fully exploit the long-term dependence relationship between certain key features of images and image labels. To resolve this problem, we have introduced the recurrent neural networks (RNNs). Specifically, we combined the CNN and RNN in order to propose the CNN–RNN framework that can deepen the understanding of image content and learn the structured features of images and to begin end-to-end training of big data in medical image analysis. In particular, we apply the transfer learning method to transfer the weight parameters that were pre-trained on the ImageNet dataset to the CNN section and adopted a custom loss function to allow our network to train and converge faster and with more accurate weight parameters. Experimental results show that compared with the other CNN models such as ResNet and Inception V3, our proposed network model is more accurate and efficient in classifying blood cell images.

Neural Machine Translation

From the outset, automatic translation was dominated by systems based on linguistic information, but then later other approaches opened up the way, such as translation memories and statistical machine translation which draw on parallel language corpora. Recently the neuronal machine translation (NMT) models have become the cutting edge in automatic translation and many translation agencies and well-known web pages are successfully using these technologies. One NMT model is a kind of statistical model comprising a group of simple deeply interconnected process units. The parameters of these models are estimated from parallel corpora using efficient automatic learning algorithms and powerful graphic processors. Applying these neural models to automatic translation requires words to be represented in the form of vectors and use recurrent neural networks in order to process phrases.