Deep Echo Research Articles

Newman-Watts-Strogatz topology in deep echo state networks for speech emotion recognition

Cortical neural connectivity exhibits a small-world (SW) network topology. However, its role in neural information processing remains unclear. In this study, we investigate the learning performance of a Deep Echo State Network (DeepESN) that incorporates the Newman-Watts-Strogatz (NWS) graph topology as a reservoir. The short path length of the SW topology allows input information to reach output nodes faster, and the clustered arrangement of this topology can expand the range of the echo state property. Additionally, in this paper, we seek to identify the best set of these key factors, such as the spectral radius of the reservoir’s weight matrix and the hyperparameters neighbors' and probabilities. We evaluate the NWS-DeepESNs on four Speech Emotion Recognition (SER) tasks, which are RAVDESS, Emo-db, SAVEE, and TESS databases. By examining the NWS-DeepESN’s performance on SER tasks, we aim to determine the role of the NWS graph topology in enhancing neural information processing. The obtained results exhibited significant improvements in recognition accuracies (weighted and unweighted) with (76.45%, 75.79%), (87.89%, 87.14%), (99%, 98.62%), and (63%, 62.24%), respectively, for RAVDESS, Emo-db, SAVEE, and TESS databases. The proposed approach improves emotion recognition capabilities and yields more accurate results compared to existing state-of-the-art approaches.

Quantitative ultrasonography reveals skeletal muscle abnormalities in carriers of DMD pathogenic variants.

Carriers of DMD pathogenic variants may become symptomatic and develop muscle-related manifestations. Despite that, few studies have attempted to characterize changes in the muscles of these carriers using imaging tools, particularly muscle ultrasound (MUS). The aim of this study was to compare lower limb MUS findings in carriers of DMD pathogenic variants (cDMD) vs healthy controls. Twenty-eight women (15 cDMD and 13 controls) underwent clinical evaluation and MUS. We collected information about muscle-related symptoms and assessed muscle strength. MUS was performed by a single physician (blind to the genetic status of subjects). The following muscles were assessed: rectus femoris, sartorius, tibialis anterior, and medial gastrocnemius. For each site, we computed data on muscle thickness, cross-sectional area, sound attenuation index, and elastography. Between-group comparisons were assessed using nonparametric tests and p-values <.05 were deemed significant. None of the subjects had objective muscle weakness, but exercise intolerance/fatigue was reported by four cDMDs and only one control. Regarding MUS, sound attenuation indices were significantly higher among carriers for all muscles tested. Longitudinal and axial deep echo intensities for the rectus femoris and tibialis anterior were also higher in the cDMD group compared with controls. No significant between-group differences were noted for elastography values, muscle area, or mean echo intensities. cDMD have skeletal muscle abnormalities that can be detected using quantitative MUS. Further studies are needed to determine whether such abnormalities are related to muscle symptoms in these patients.

Research on Credit Risk Prediction Method of Blockchain Applied to Supply Chain Finance

INTRODUCTION: From the perspective of blockchain, it establishes a credit risk evaluation index system for supply chain finance applicable to blockchain, constructs an accurate credit risk prediction model, and provides a reliable guarantee for the research of credit risk in supply chain finance.OBJECTIVES: To address the inefficiency of the current credit risk prediction and evaluation model for supply chain finance.METHODS: This paper proposes a combined blockchain supply chain financial credit risk prediction and evaluation method based on kernel principal component analysis and intelligent optimisation algorithm to improve Deep Echo State Network. Firstly, the evaluation system is constructed by describing the supply chain financial credit risk prediction and evaluation problem based on blockchain technology, analysing the evaluation indexes, and constructing the evaluation system; then, the parameters of DeepESN network are optimized by combining the kernel principal component analysis method with the JSO algorithm to construct the credit risk prediction and evaluation model of supply chain finance; finally, the effectiveness, robustness, and real-time performance of the proposed method are verified by simulation experiment analysis.RESULTS: The results show that the proposed method improves the prediction efficiency of the prediction model.CONCLUSION: The problems of insufficient scientific construction of index system and poor efficiency of risk prediction model of B2B E-commerce transaction size prediction method are effectively solved.

Deep Echo State Q-Network Aided Trust Sharing Provisioning for Internet of Vehicle

Deep Echo State Q-Network Aided Trust Sharing Provisioning for Internet of Vehicle

Performance Evaluation and Improvement of Deep Echo State Network Models in English Writing Assistance and Grammar Error Correctionn

INTRODUCTION: The research on the performance evaluation model of English writing tutoring and grammar error correction is very necessary, which is not only conducive to the rational allocation of teachers' writing tutoring resources, but also more conducive to the timely and effective correction of students' grammatical errors.OBJCTIVES: Aiming at the problems of non-specific quantification, low precision, and low real-time performance evaluation methods for English writing grammar error correction in current methods.METHODS: This paper proposes a grammar error correction performance evaluation method based on deep echo state network with gold rush optimisation algorithm. Firstly, by analysing the process of English writing assistance and grammatical error correction, we extract the evaluation features of grammatical error correction type and construct the performance evaluation system; then, we improve the deep confidence network through the gold rush optimization algorithm and construct the grammatical error correction performance evaluation model; finally, we analyse it through simulation experiments.RESULTS: The results show that the proposed method improves the evaluation accuracy, robustness. The absolute value of the relative error of the evaluation value of the syntactic error correction performance of the method is controlled within the range of 0.02.CONCLUSION: The problems of non-specific quantification, low precision and low real-time performance of the application of English writing grammar error correction performance assessment methods are solved.

Prediction for nonlinear time series by improved deep echo state network based on reservoir states reconstruction

With the aim to enhance prediction accuracy for nonlinear time series, this paper put forward an improved deep Echo State Network based on reservoir states reconstruction driven by a Self-Normalizing Activation (SNA) function as the replacement for the traditional Hyperbolic tangent activation function to reduce the model’s sensitivity to hyper-parameters. The Strategy was implemented in a two-state reconstruction process by first inputting the time series data to the model separately. Once, the time data passes through the reservoirs and is activated by the SNA activation function, the new state for the reservoirs is created. The state is input to the next layer, and the concatenate states module saves. Pairs of states are selected from the activated multi-layer reservoirs and input into the state reconstruction module. Multiple input states are transformed through the state reconstruction module and finally saved to the concatenate state module. Two evaluation metrics were used to benchmark against three other ESNs with SNA activation functions to achieve better prediction accuracy.

Strategies for Prevention and Management of Burnout Syndrome at the Medical Staff

Introduction. Burnout syndrome is a complex phenomenon of great magnitude, which occurs in extremely demanding professions emotionally and nervously, with deep echoes in the psycho-affective structure of individuals, which makes its mark on professional life, but also that social. Acknowledging the problem and requesting specialized help remain the first step toward healing.&#x0D; Material and methods. This article reviews the literature on the discovery of burnout management strategies and its adverse effects. Articles found through this indexed search (PubMed and Google Scholar: 2017-2021) were reviewed and analyzed manually by the authors to identify relevant studies.&#x0D; Discussions. This study specifically focuses on the strategies that health care workers use to address resource depletion associated with fatigue, reduce work burden, and alter work characteristics to increase motivation.&#x0D; Conclusions. In order to prevent, manage or treat burnout, it is necessary to recognize the problems that contribute to burnout. Focusing on individual approaches, along with organizational ones, seems to be a promising way to deal with burnout and stimulate a healthier workforce.&#x0D;

Design and Application of Adaptive Sparse Deep Echo State Network

Design and Application of Adaptive Sparse Deep Echo State Network

Sequence prediction with different dimensions based on two novel deep echo state network models

The echo state network (ESN) is a typical reservoir computation model, which was first proposed by Jaeger et al. It was widely used in various fields and achieved excellent results for a long time, especially in time series prediction. In recent years, there are few improvements to the ESN structure, and the more famous is the deep echo state network (DESN) model. However, a DESN will cause the loss of input data. How to effectively optimize the structure of ESN and how to scientifically add input data to deep echo are urgent problems to be solved. In this paper, we propose multi-reservoir ESN models based on how the input data participate in the system. Then, we use complex nonlinear chaotic systems with different dimensions to test our model. Finally, we compare it with the traditional model and the recently proposed model, and then find that our models have better predictive performance.

A Deep Echo State Network-Based Novel Signal Processing Approach for Underwater Wireless Optical Communication System with PAM and OFDM Signals

Underwater wireless optical communication (UWOC) plays key role in the underwater wireless sensor networks (UWSNs), which have been widely employed for both scientific and commercial applications. UWOC offers high transmission data rates, high security, and low latency communication between nodes in UWSNs. However, significant absorption and scattering loss in underwater channels, due to ocean water conditions, can introduce highly non-linear distortion in the received signals, which can severely deteriorate communication quality. Consequently, addressing the challenge of processing UWOC signals with low optical signal-to-noise ratios (OSNRs) is critical for UWOC systems. Increasing the transmitting optical power and investigating more advanced signal processing technologies to recover transmitted symbols are two primary approaches to improve system tolerance in noisy UWOC signal channels. In this paper, we propose and demonstrate the application of deep echo state networks (DeepESNs) for channel equalization in high-speed UWOC systems to enhance system performance with both PAM and QPSK-OFDM modulations. Our experimental results demonstrate the effectiveness of DeepESNs in UWOC systems, achieving error-free underwater transmission over 40.5 m with data rates up to 167 Mbps. Moreover, we compare the performance of DeepESNs to conventional echo state networks and provide suggestions on the configuration of a DeepESN for UWOC signals.

English

Cloud Computing (CC) is undoubtedly an indispensable technology across the world. It indicates a revolution in collaborative services and data storage. Yet, security problems have increased with the move to CC, which includes intrusion detection systems (IDS). Anomaly detection (AD) is a vital method that ensures the security of CC environments. It identifies unusual behaviour that specifies a security threat. AD is important in a secure CC environment for identifying breaches or attacks and monitoring system performance. The design of potential AD in a secure CC environment needs a combination of machine learning (ML) algorithms, continuous monitoring, and data analysis. Therefore, this article introduces a new Falcon Optimization Algorithm with Deep Echo State Network for Anomaly Detection (FOADESN-AD) technique for a secure CC environment. The presented FOADESN-AD technique exploits the DL model with a metaheuristic optimizer for anomaly or intrusion detection in the cloud platform. To accomplish this, the FOADESN-AD technique initially performs a Z-score normalization process. For anomaly detection, the FOADESN-AD technique uses the DESN classifier, which accurately detects the presence of anomalies in the cloud environment. Moreover, the FOA is utilized to finetune the hyperparameter values of the DESN model, achieving superior classification results. The performance analysis of the FOADESN-AD method is implemented on the CSE-CICIDS-2018 dataset. The experimental values stated the betterment of the FOADESN-AD method over other existing approaches.

Decentralized Federated Learning for Industrial IoT With Deep Echo State Networks

Federated learning (FL) has recently been adopted to train shared models across industrial Internet of Things (IoT) devices without revealing their private raw data. Conventional FL usually relies on a central server for coordination. However, in reality, the central server is not fully trusted, which means that it may be collecting data, raising concerns about data leakage and misuse. Here, we propose a decentralized FL algorithm based on deep neural networks to address the problem of untrusted central servers. The original problem is decomposed into several subproblems with consensus constraints, which can be solved by local computation and communication. The proposed algorithm combines the decentralized average consensus and alternating direction method of multipliers. Several decentralized algorithms are employed for comparison, and the issue of heterogeneous data is discussed. Experimental evaluation shows the effectiveness of our proposed algorithm.

Nonstationary Time Series Prediction Based on Deep Echo State Network Tuned by Bayesian Optimization

The predictions from time series data can help us sense development trends and make scientific decisions in advance. The commonly used forecasting methods with backpropagation consume a lot of computational resources. The deep echo state network (DeepESN) is an advanced prediction method with a deep neural network structure and training algorithm without backpropagation. In this paper, a Bayesian optimization algorithm (BOA) is proposed to optimize DeepESN to address the problem of increasing parameter scale. Firstly, the DeepESN was studied and constructed as the basic prediction model for the time series data. Secondly, the BOA was reconstructed, based on the DeepESN, for optimal parameter searching. The algorithm is proposed within the framework of the DeepESN. Thirdly, an experiment was conducted to verify the DeepESN with a BOA within three datasets: simulation data generated from computer programs, a real humidity dataset collected from Beijing, and a power load dataset obtained from America. Compared with the models of BP (backpropagation), LSTM (long short-term memory), GRU (gated recurrent unit), and ESN (echo state network), DeepESN obtained optimal results, which were 0.0719, 18.6707, and 764.5281 using RMSE evaluation. While getting better accuracy, the BOA optimization time was only 323.4 s, 563.2 s, and 9854 s for the three datasets. It is more efficient than grid search and grey wolf optimizer.

Improving Deep Echo State Network with Neuronal Similarity-Based Iterative Pruning Merging Algorithm

Recently, a layer-stacked ESN model named deep echo state network (DeepESN) has been established. As an interactional model of a recurrent neural network and deep neural network, investigations of DeepESN are of significant importance in both areas. Optimizing the structure of neural networks remains a common task in artificial neural networks, and the question of how many neurons should be used in each layer of DeepESN must be stressed. In this paper, our aim is to solve the problem of choosing the optimized size of DeepESN. Inspired by the sensitive iterative pruning algorithm, a neuronal similarity-based iterative pruning merging algorithm (NS-IPMA) is proposed to iteratively prune or merge the most similar neurons in DeepESN. Two chaotic time series prediction tasks are applied to demonstrate the effectiveness of NS-IPMA. The results show that the DeepESN pruned by NS-IPMA outperforms the unpruned DeepESN with the same network size, and NS-IPMA is a feasible and superior approach to improving the generalization performance of DeepESN. The newly proposed method has broad application prospects in real-time systems.

Deep echo state networks in data marketplaces

Data Marketplaces are the digital platform for data buyers and data sellers to trade information as valuable products or items. The expectation taken for granted from the users of a data marketplace is the truth of the exchanged information. However, the trade of factual data also means the marketable product is no longer unique but a series of replicas. If every user within the data marketplace owns the same information, this data eventually becomes valueless. There are specific instances where the traded products are sought to be always unique, for instance predictions or digital art. This article applies Deep Echo State Networks (ESNs) in data marketplaces that map tradeable data into a larger dimensional space via the dynamics of reservoirs with fixed and non-linear properties. These reservoirs generate unique tradeable data products that cannot be replicated, therefore ensuring its exclusivity and commercial value. The validation results show that ESNs can also be applied to generate random tradeable products in different dimensional spaces. Specifically, the reservoir with its associated neural perturbation emulates a digital creator that generates unique and exclusive content based on 1D functions and 2D images.

Generation of a synthetic aperture radar deception jamming signal based on a deep echo inversion network

AbstractExisting methods for generating synthetic aperture radar (SAR) deception jamming signals have slow speed, low imaging quality, and insufficient intelligence in complex electromagnetic environments. This paper proposes a deep learning‐based SAR deception jamming signal generation method based on deep echo inversion Unet (DEIUnet). This method has high speed and provides high‐image quality of the interference signal. A Swin Next (SN) block is proposed to combine local and non‐local information in the image and echo data. The Unet structure consists of SN blocks, and a residual connection is used as the jump connection to fuse the multi‐scale feature information from the echo and image data. PixelShuffle is utilised for up‐sampling to generate high‐quality echo data. The experimental results on MSTAR and Sentinel‐1 data sets verify the effectiveness and superiority of DEIUnet for echo inversion. The imaging results of the SAR deception jamming signal generated by DEIUnet on an MSTAR scene confirm the effectiveness of the proposed method.

Dynamic ensemble deep echo state network for significant wave height forecasting

Forecasts of the wave heights can assist in the data-driven control of wave energy systems. However, the dynamic properties and extreme fluctuations of the historical observations pose challenges to the construction of forecasting models. This paper proposes a novel dynamic ensemble deep Echo state networks (ESN) to learn the dynamic characteristics of the significant wave height. The dynamic ensemble ESN creates a profound representation of the input and trains an independent readout module for each reservoir. To begin, numerous reservoir layers are built in a hierarchical order, adopting a reservoir pruning approach to filter out the poorer representations. Finally, a dynamic ensemble block is used to integrate the forecasts of all readout layers. The suggested model has been tested on twelve available datasets and statistically outperforms state-of-the-art approaches.

Grey Wolf Optimization–Based Deep Echo State Network for Time Series Prediction

The Echo State Network (ESN) is a unique type of recurrent neural network. It is built atop a reservoir, which is a sparse, random, and enormous hidden infrastructure. ESN has been successful in dealing with a variety of non-linear issues, including prediction and classification. ESN is utilized in a variety of architectures, including the recently proposed Multi-Layer (ML) architecture. Furthermore, Deep Echo State Network (DeepESN) models, which are multi-layer ESN models, have recently been proved to be successful at predicting high-dimensional complicated non-linear processes. The proper configuration of DeepESN architectures and training parameters is a time-consuming and difficult undertaking. To achieve the lowest learning error, a variety of parameters (hidden neurons, input scaling, the number of layers, and spectral radius) are carefully adjusted. However, the optimum training results may not be guaranteed by this haphazardly created work. The grey wolf optimization (GWO) algorithm is introduced in this study to address these concerns. The DeepESN based on GWO (GWODESN) is utilized in trials to forecast time series, and therefore the results are compared with the regular ESN, LSTM, and ELM models. The findings indicate that the planned model performs the best in terms of prediction.

An Engine Fault Detection Method Based on the Deep Echo State Network and Improved Multi-Verse Optimizer

This paper aims to develop an efficient pattern recognition method for engine fault end-to-end detection based on the echo state network (ESN) and multi-verse optimizer (MVO). Bispectrum is employed to transform the one-dimensional time-dependent vibration signal into a two-dimensional matrix with more impact features. A sparse input weight-generating algorithm is designed for the ESN. Furthermore, a deep ESN model is built by fusing fixed convolution kernels and an autoencoder (AE). A novel traveling distance rate (TDR) and collapse mechanism are studied to optimize the local search of the MVO and speed it up. The improved MVO is employed to optimize the hyper-parameters of the deep ESN for the two-dimensional matrix recognition. The experiment result shows that the proposed method can obtain a recognition rate of 93.10% in complex engine faults. Compared with traditional deep belief networks (DBNs), convolutional neural networks (CNNs), the long short-term memory (LSTM) network, and the gated recurrent unit (GRU), this novel method displays superior performance and could benefit the fault end-to-end detection of rotating machinery.

A Hybrid Sailfish Whale Optimization and Deep Long Short-Term Memory (SWO-DLSTM) Model for Energy Efficient Autonomy in India by 2048

In order to formulate the long-term and short-term development plans to meet the energy needs, there is a great demand for accurate energy forecasting. Energy autonomy helps to decompose a large-scale grid control into a small sized decisions to attain robustness and scalability through energy independence level of a country. Most of the existing energy demand forecasting models predict the amount of energy at a regional or national scale and failed to forecast the demand for power generation for small-scale decentralized energy systems, like micro grids, buildings, and energy communities. A novel model called Sailfish Whale Optimization-based Deep Long Short- Term memory (SWO-based Deep LSTM) to forecast electricity demand in the distribution systems is proposed. The proposed SWO is designed by integrating the Sailfish Optimizer (SO) with the Whale Optimization Algorithm (WOA). The Hilbert-Schmidt Independence Criterion (HSIC) is applied on the dataset, which is collected from the Central electricity authority, Government of India, for selecting the optimal features using the technical indicators. The proposed algorithm is implemented in MATLAB software package and the study was done using real-time data. The optimal features are trained using Deep LSTM model. The results of the proposed model in terms of install capacity prediction, village electrified prediction, length of R &amp; D lines prediction, hydro, coal, diesel, nuclear prediction, etc. are compared with the existing models. The proposed model achieves percentage improvements of 10%, 9.5%,6%, 4% and 3% in terms of Mean Squared Error (MSE) and 26%, 21%, 16%, 12% and 6% in terms of Root Mean Square Error (RMSE) for Bootstrap-based Extreme Learning Machine approach (BELM), Direct Quantile Regression (DQR), Temporally Local Gaussian Process (TLGP), Deep Echo State Network (Deep ESN) and Deep LSTM respectively. The hybrid approach using the optimization algorithm with the deep learning model leads to faster convergence rate during the training process and enables the small-scale decentralized systems to address the challenges of distributed energy resources. The time series datasets of different utilities are trained using the hybrid model and the temporal dependencies in the sequence of data are predicted with point of interval as 5 years-head. Energy autonomy of the country till the year 2048 is assessed and compared.