Faster Training Speed Research Articles

Application of Deep Wavelet Kernel Extreme Learning Machine in Fault Diagnosis of Tamping Vehicle

Since it is easy to overfit due to the long training time of the fault diagnosis model for machinery. Introducing the idea of autoencoder (AE) into the wavelet extreme learning machine (WELM) and then stacking to form WELM-AE can convert the underlying fault features to more abstract and advanced ones. And then the adaptive boosting kernel extreme learning machine (Adaboost-KELM) is used as the top-level classifier for fault recognition. The experimental results verify the feasibility of the proposed algorithm in the fault diagnosis of tamping machine with the characteristics of the fast training speed of the extreme learning machine, and a higher accuracy rate than back propagation (BP), support vector machine (SVM), stacked autoencoder (SAE), and convolutional neural networks (CNN).

SanMove: next location recommendation via self-attention network

PurposeThis paper aims to address the following issues: (1) most existing methods are based on recurrent network, which is time-consuming to train long sequences due to not allowing for full parallelism; (2) personalized preference generally are not considered reasonably; (3) existing methods rarely systematically studied how to efficiently utilize various auxiliary information (e.g. user ID and time stamp) in trajectory data and the spatiotemporal relations among nonconsecutive locations.Design/methodology/approachThe authors propose a novel self-attention network–based model named SanMove to predict the next location via capturing the long- and short-term mobility patterns of users. Specifically, SanMove uses a self-attention module to capture each user's long-term preference, which can represent her personalized location preference. Meanwhile, the authors use a spatial-temporal guided noninvasive self-attention (STNOVA) module to exploit auxiliary information in the trajectory data to learn the user's short-term preference.FindingsThe authors evaluate SanMove on two real-world datasets. The experimental results demonstrate that SanMove is not only faster than the state-of-the-art recurrent neural network (RNN) based predict model but also outperforms the baselines for next location prediction.Originality/valueThe authors propose a self-attention-based sequential model named SanMove to predict the user's trajectory, which comprised long-term and short-term preference learning modules. SanMove allows full parallel processing of trajectories to improve processing efficiency. They propose an STNOVA module to capture the sequential transitions of current trajectories. Moreover, the self-attention module is used to process historical trajectory sequences in order to capture the personalized location preference of each user. The authors conduct extensive experiments on two check-in datasets. The experimental results demonstrate that the model has a fast training speed and excellent performance compared with the existing RNN-based methods for next location prediction.

L1-Norm Robust Regularized Extreme Learning Machine with Asymmetric C-Loss for Regression

Extreme learning machines (ELMs) have recently attracted significant attention due to their fast training speeds and good prediction effect. However, ELMs ignore the inherent distribution of the original samples, and they are prone to overfitting, which fails at achieving good generalization performance. In this paper, based on expectile penalty and correntropy, an asymmetric C-loss function (called AC-loss) is proposed, which is non-convex, bounded, and relatively insensitive to noise. Further, a novel extreme learning machine called L1 norm robust regularized extreme learning machine with asymmetric C-loss (L1-ACELM) is presented to handle the overfitting problem. The proposed algorithm benefits from L1 norm and replaces the square loss function with the AC-loss function. The L1-ACELM can generate a more compact network with fewer hidden nodes and reduce the impact of noise. To evaluate the effectiveness of the proposed algorithm on noisy datasets, different levels of noise are added in numerical experiments. The results for different types of artificial and benchmark datasets demonstrate that L1-ACELM achieves better generalization performance compared to other state-of-the-art algorithms, especially when noise exists in the datasets.

An improved hybrid quantum-classical convolutional neural network for multi-class brain tumor MRI classification

The efficiency of quantum computing has recently been extended to machine learning, which has made a significant impact on quantum machine learning. The hybrid structure of quantum and classical ones has developed into the most successful application mode currently due to noisy intermediate scale quantum limitations. In this paper, an improved hybrid quantum-classic convolutional neural network (HQC-CNN) with fast training speed, lightweight, and high performance is proposed. Its convolution layer realizes feature mapping through parameterized quantum circuit, while other layers keep classic operation and finally complete the task of four classifications of brain tumors. The experiment in this paper is based on kaggle brain tumor magnetic resonance imaging public dataset. The final experimental results show that HQC-CNN can effectively classify meningioma, glioma, pituitary, and no tumor with a classification accuracy of 97.8%. When compared to numerous well-known landmark models, HQC-CNN has obvious advantages.

Research on a percussion-based bolt looseness identification method based on phase feature and convolutional neural network

The percussion-based method has become a hot spot for bolt looseness monitoring due to its advantages of non-contact sensing, portability, and low cost. However, the features of bolt looseness in percussion methods lack phase information. In this paper, a percussion method based on the all-pole group delay function in the phase domain is proposed for the first time, and the bolt looseness is determined by a convolutional neural network. Under the four signal-to-noise ratio levels (0, 2, 4 and 6 dB), the accuracy of the proposed method is 90.25%, 92.75%, 93.5% and 94%. The experiment proves the percussion audio signal of the structural point away from the bolt can reflect the looseness of the bolt. The phase feature can represent the information of bolt looseness and has fast training speed and high recognition accuracy, which is suitable for detecting bolt looseness torque.

Energy efficient resource allocation method for 5G access network based on reinforcement learning algorithm

Edge computing and IIoT (Industrial Internet of Things) are two representative application scenarios in 5G (5th Generation) mobile communication technology network. Therefore, this paper studies the resource allocation methods for these two typical scenarios, and proposes an energy-efficient resource allocation method by using DRL algorithm, which enhances the network performance and reduces the network operation cost. Firstly, according to the characteristics of edge computing, taking the connection relationship between base stations, users and the transmission power allocated by base stations to users as decision variables, minimizing the overall energy efficiency as the goal, and taking the needs of mobile users as constraints, a resource allocation model for user quality of service (QoS) guarantee is designed. Secondly, simulation experiments are used to verify that the proposed method has faster training speed and convergence speed, ensures the quality-of-service requirements of mobile users, and realizes intelligent and energy-saving resource allocation. The approximate amounts of steps to convergence under four cases are 500, 350, 250, and 220, respectively, and the values of awards are 21.76, 21.09, 20.38, and 20.25, respectively. Thirdly, aiming at the IIoT environment scenario, this paper proposes a resource allocation method for 5G wide connection low delay services based on asynchronous dominant action evaluation algorithm (A3C). Firstly, according to the characteristics of IIoT environment, taking the connection relationship between base stations and users and the transmission power allocated by base stations to users as decision variables and maximizing energy efficiency while satisfying needs of each user as optimization goal, a resource allocation model for 5G wide connection low delay services is designed. On this basis, an energy-efficient resource allocation method according to A3C is proposed. The hierarchical aggregation clustering algorithm (HAC) is adopted to determine the connection relationship between the base station and users, and the A3C algorithm is adopted to allocate transmission power to users, so as to maximize the overall energy efficiency and ensure the needs of each user.

Linearized microwave downconversion link based on fast and intelligent impairment equalization for noncooperative systems

We experimentally demonstrated the use of intelligent impairment equalization (IIE) for microwave downconversion link linearization in noncooperative systems. Such an equalizer is realized based on an artificial neural network (ANN). Once the training process is completed, the inverse link transfer function can be determined. With the inverse transformation for the detected signal after transmission, the third-order intermodulation distortion components are suppressed significantly without requiring any prior information from an input RF signal. Furthermore, fast training speed is achieved, since the configuration of ANN-based equalizer is simple. Experimental results show that the spurious-free dynamic range of the proposed link is improved to 106.5 dB · Hz2/3, which is 11.3 dB higher than that of a link without IIE. Meanwhile, the training epochs reduce to only five, which has the potential to meet the practical engineering requirement.

Speech Separation Algorithm Using Gated Recurrent Network Based on Microphone Array

Speech separation is an active research topic that plays an important role in numerous applications, such as speaker recognition, hearing prosthesis, and autonomous robots. Many algorithms have been put forward to improve separation performance. However, speech separation in reverberant noisy environment is still a challenging task. To address this, a novel speech separation algorithm using gate recurrent unit (GRU) network based on microphone array has been proposed in this paper. The main aim of the proposed algorithm is to improve the separation performance and reduce the computational cost. The proposed algorithm extracts the sub-band steered response power-phase transform (SRP-PHAT) weighted by gammatone filter as the speech separation feature due to its discriminative and robust spatial position information. Since the GRU network has the advantage of processing time series data with faster training speed and fewer training parameters, the GRU model is adopted to process the separation features of several sequential frames in the same sub-band to estimate the ideal Ratio Masking (IRM). The proposed algorithm decomposes the mixture signals into time-frequency (TF) units using gammatone filter bank in the frequency domain, and the target speech is reconstructed in the frequency domain by masking the mixture signal according to the estimated IRM. The operations of decomposing the mixture signal and reconstructing the target signal are completed in the frequency domain which can reduce the total computational cost. Experimental results demonstrate that the proposed algorithm realizes omnidirectional speech separation in noisy and reverberant environments, provides good performance in terms of speech quality and intelligibility, and has the generalization capacity to reverberate.

Enhanced Hybrid Hierarchical Federated Edge Learning Over Heterogeneous Networks

In this work, a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Hybrid Hierarchical Federated Edge Learning</i> (HHFEL) architecture that consists of a device layer, an edge layer, and a cloud layer over heterogeneous networks, is investigated for large-scale model training. In such systems, learning efficiency is severely degraded by limited communication resources and device heterogeneity in terms of local data distribution and computation capability, especially for synchronous FL mechanisms where the training of each round should wait for the slowest device. To tackle this issue, asynchronous FL is proposed, which allows the devices with powerful computation and communication capabilities exchanging information with the server more frequently. However, this asynchronous FL framework faces a new challenge of low accuracy caused by the imbalanced local model updating. To overcome the shortage of both synchronous and asynchronous FLs, we propose an enhanced online semi-asynchronous FL mechanism between the edge-device layers, where each device trains its local model with the newly generated data and each edge server aggregates a number of local models based on their arrival order in each round. Particularly, devices with faster training speeds would fully utilize the idle time by training their local models repetitively. Meanwhile, synchronous FL with an edge elastic update strategy is adopted to the cloud-edge layers for personalized information exchange. Considering the continuous data generation feature, we formulate the objective problem as an online <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Markov Decision Process</i> (MDP) to realize efficient communication-and-computing HHFEL via joint device selection and resource allocation. Due to the non-convex and combinatorial problem structure, we develop a hybrid <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Deep Q-Network</i> (DQN) and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Deep Deterministic Policy Gradient</i> (DDPG) approach with low computational complexity to adapt the device selection and resource allocation strategies. Numerical results show the effectiveness of the proposed mechanism compared with existing benchmarks.

Self-Supervised Modality-Aware Multiple Granularity Pre-Training for RGB-Infrared Person Re-Identification

RGB-Infrared person re-identification (RGB-IR ReID) aims to associate people across disjoint RGB and IR camera views. Currently, state-of-the-art performance of RGB-IR ReID is not as impressive as that of conventional ReID. Much of that is due to the notorious modality bias training issue brought by the single-modality ImageNet pre-training, which might yield RGB-biased representations that severely hinder the cross-modality image retrieval. This paper makes first attempt to tackle the task from a pre-training perspective. We propose a self-supervised pre-training solution, named Modality-Aware Multiple Granularity Learning (MMGL), which directly trains models from scratch only on multi-modal ReID datasets, but achieving competitive results against ImageNet pre-training, without using any external data or sophisticated tuning tricks. First, we develop a simple-but-effective ‘permutation recovery’ pretext task that globally maps shuffled RGB-IR images into a shared latent permutation space, providing modality-invariant global representations for downstream ReID tasks. Second, we present a part-aware cycle-contrastive (PCC) learning strategy that utilizes cross-modality cycle-consistency to maximize agreement between semantically similar RGB-IR image patches. This enables contrastive learning for the unpaired multi-modal scenarios, further improving the discriminability of local features without laborious instance augmentation. Based on these designs, MMGL effectively alleviates the modality bias training problem. Extensive experiments demonstrate that it learns better representations (+8.03% Rank-1 accuracy) with faster training speed (converge only in few hours) and higher data efficiency (<5% data size) than ImageNet pre-training. The results also suggest it generalizes well to various existing models, losses and has promising transferability across datasets. The code will be released at https://github.com/hansonchen1996/MMGL.

Twin Broad Learning System for Fault Diagnosis of Rotating Machinery

As rotating machines are more and more widely used in modem equipment, their fault diagnosis is important to guarantee the instrument’s reliability and safety. Although intelligent fault diagnosis based on deep learning (DL) has achieved great performance in many fault diagnosis tasks, these models are highly dependent on the large-scale training dataset and can be time-consuming during training and testing. These limitations largely affect the efficiency of diagnosis in real-world applications. An effective alternative way is to develop the single-layer feedforward networks (SLFNs) based Broad Learning System (BLS) for fault diagnosis tasks, which enjoy fast training speed as well as strong generalization ability. However, we found that the least square classifier used in classical BLS can have a problem distinguishing the possibly overlapping fault patterns. In this paper, we propose a novel twin broad learning system (TBLS) for fault diagnosis of rotating machinery. Rather than using the classical least square method, the proposed TBLS learns to find two nonparallel hyper-planes to deal with the classification problem, which shows stronger generalization ability in fault diagnosis problems. Experimental results on two fault diagnosis benchmark datasets for typical rotating machinery further illustrate the effectiveness of the TBLS methods, which offer a high-efficiency solution for rotating machinery fault diagnosis.

A Brand-New Simple, Fast, and Effective Residual-Based Method for Radial Basis Function Neural Networks Training

The radial basis function (RBF) neural network is a type of universal approximator, and has been widely used in various fields. Improving the training speed and compactness of RBF networks are critical for promoting their applications. In the present study, we propose a simple, fast, and effective RBF networks training method, which is based on the residual extreme points and their neighborhoods (thus called the REN method for short in this paper). The REN method calculates RBF centers and widths through a two-level iterative process, and realizes two main functionalities, namely 1) adding multiple centers within one pass through the whole data set, and 2) calculating RBF widths specifically for each center. The use of this algorithm does not need any parameter adjustments, and the models for approximation or classification can be obtained by only one run. The performance of the proposed REN algorithm is compared with the classic and powerful orthogonal least squares (OLS) algorithm. By reaching the same accuracies, the REN algorithm trains RBF networks 50 and 320 times faster, in the chirp (0˜50 Hz, 2 s, 1 kHz, 2001 samples) and two-dimensional peaks (2401 samples) signal approximation tasks respectively, than the OLS algorithm does, and the number of centers obtained by the REN algorithm is reduced by half. When incorporating the same number of centers, the REN algorithm achieves accuracies up to 3 orders of magnitude higher than the best results obtained by the OLS algorithm. In the classification task of a real discrete breast cancer data, both methods result in accuracies comparable to many existent methods, but the REN algorithm has the advantages of fast training speeds and no requirements for parameter adjustments. The REN algorithm proposed in this study may potentially be used for tasks with large scale of data or applications that require high model performances.

Gas Turbine Model Identification Based on Online Sequential Regularization Extreme Learning Machine with a Forgetting Factor

Due to the advantages of high convergence accuracy, fast training speed, and good generalization performance, the extreme learning machine is widely used in model identification. However, a gas turbine is a complex nonlinear system, and its sampling data are often time-sensitive and have measurement noise. This article proposes an online sequential regularization extreme learning machine algorithm based on the forgetting factor (FOS_RELM) to improve gas turbine identification performance. The proposed FOS_RELM not only retains the advantages of the extreme learning machine algorithm but also enhances the learning effect by rapidly discarding obsolete data during the learning process and improves the anti-interference performance by using the regularization principle. A detailed performance comparison of the FOS_RELM with the extreme learning machine algorithm and regularized extreme learning machine algorithm is carried out in the model identification of a gas turbine. The results show that the FOS_RELM has higher accuracy and better robustness than the extreme learning machine algorithm and regularized extreme learning machine algorithm. All in all, the proposed algorithm provides a candidate technique for modeling actual gas turbine units.

SSD-SGD: Communication Sparsification for Distributed Deep Learning Training

Intensive communication and synchronization cost for gradients and parameters is the well-known bottleneck of distributed deep learning training. Based on the observations that Synchronous SGD (SSGD) obtains good convergence accuracy while asynchronous SGD (ASGD) delivers a faster raw training speed, we propose Several Steps Delay SGD (SSD-SGD) to combine their merits, aiming at tackling the communication bottleneck via communication sparsification. SSD-SGD explores both global synchronous updates in the parameter servers and asynchronous local updates in the workers in each periodic iteration. The periodic and flexible synchronization makes SSD-SGD achieve good convergence accuracy and fast training speed. To the best of our knowledge, we strike the new balance between synchronization quality and communication sparsification, and improve the tradeoff between accuracy and training speed. Specifically, the core components of SSD-SGD include proper warm-up stage, steps delay stage, and the novel algorithm of global gradient for local update (GLU). GLU is critical for local update operations by using global gradient information to effectively compensate for the delayed local weights. Furthermore, we implement SSD-SGD on MXNet framework and comprehensively evaluate its performance with CIFAR-10 and ImageNet datasets. Experimental results show that SSD-SGD can accelerate distributed training speed under different experimental configurations, by up to 110% (or 2.1× of the original speed), while achieving good convergence accuracy.

Edge Device Identification Based on Federated Learning and Network Traffic Feature Engineering

With the ubiquitous deployment and applications of Internet of Things (IoT), security issues pose a critical challenge to IoT devices. External attackers often utilize vulnerable IoT devices to invade the target’s internal network and then further cause a security threat to the whole network. To prevent such attacks, it is necessary to develop a security mechanism to control the access of suspicious IoT devices and manage the internal devices. In recent years, deep learning (DL) algorithm has been widely used in the field of edge device identification (EDI), and has made great achievements. However, these previous methods are essentially centralized learning-based EDI (CentEDI) that trains all data together, which can not guarantee data security and not conducive to deployment on edge devices. To address this problem, we introduce a federated learning-based EDI (FedeEDI) method via network traffic to automatically identify edge devices connected to the whole network. Experimental results show that the training efficiency of our proposed FedeEDI method is much higher than that of the CentEDI method, although its classification accuracy is slightly reduced. In contrast to the CentEDI method, the proposed FedeEDI method has two main advantages: faster training speed and safer training process.

Domain Knowledge-Assisted Deep Reinforcement Learning Power Allocation for MIMO Radar Detection

The power allocation of multiple-input multiple-output (MIMO) radars is a key point in target tracking and detection. The optimality of a multitarget and multiconstraint optimization problem strictly depends on a priori model, which is difficult to obtain in time-varying, complex, and noncooperative environments. Recently, deep reinforcement learning (DRL) has been applied for target tracking tasks, which provides a trial-and-error interactive learning mechanism to improve the policy. Unlike tracking tasks with complete target state transition models, it remains an open issue for DRL-based MIMO radar detection that requires efficiently adapting the control policy to the environment of randomly appearing targets and extensive power transmission actions, which leads to sparse final task rewards and hence slow policy learning for the agent. Through introducing both the analytic model (radar equation) and empirical rules (expert preferences) for domain knowledge, this article proposes a domain-knowledge-assisted DRL (DKADRL) framework in which a domain-knowledge-based timely reward generator is utilized to generate timely rewards that assist the agent’s policy learning. To adjust the role of the timely rewards and the final task rewards, a reward fusion module is designed, which gradually increases the role of the final task rewards as the training process progresses, thus allows agent’s policy to converge to the final optimization goal. The algorithm is validated under two target motion scenarios, showing the higher target detection probability and the faster training speed, compared to equal power allocation and proximal policy optimization (PPO)-based power allocation.

A Scaling Transition Method from SGDM to SGD with 2ExpLR Strategy

In deep learning, the vanilla stochastic gradient descent (SGD) and SGD with heavy-ball momentum (SGDM) methods have a wide range of applications due to their simplicity and great generalization. This paper uses an exponential scaling method to realize a smooth and stable transition from SGDM to SGD, which combines the advantages of the fast training speed of SGDM and the accurate convergence of SGD (named TSGD). We also provide some theoretical results on the convergence of this algorithm. At the same time, we take advantage of the learning rate warmup strategy’s stability and the learning rate decay strategy’s high accuracy. A warmup–decay learning rate strategy with double exponential functions is proposed (named 2ExpLR). The experimental results on different datasets for the proposed algorithms indicate that the accuracy is improved significantly and that the training is faster and more stable.

Data information processing of traffic digital twins in smart cities using edge intelligent federation learning

The present work analyzes the application of deep learning in the context of digital twins (DTs) to promote the development of smart cities. According to the theoretical basis of DTs and the smart city construction, the five-dimensional DTs model is discussed to propose the conceptual framework of the DTs city. Then, edge computing technology is introduced to build an intelligent traffic perception system based on edge computing combined with DTs. Moreover, to improve the traffic scene recognition accuracy, the Single Shot MultiBox Detector (SSD) algorithm is optimized by the residual network, form the SSD-ResNet50 algorithm, and the DarkNet-53 is also improved. Finally, experiments are conducted to verify the effects of the improved algorithms and the data enhancement method. The experimental results indicate that the SSD-ResNet50 and the improved DarkNet-53 algorithm show fast training speed, high recognition accuracy, and favorable training effect. Compared with the original algorithms, the recognition time of the SSD-ResNet50 algorithm and the improved DarkNet-53 algorithm is reduced by 6.37ms and 4.25ms, respectively. The data enhancement method used in the present work is not only suitable for the algorithms reported here, but also has a good influence on other deep learning algorithms. Moreover, SSD-ResNet50 and improved DarkNet-53 algorithms have significant applicable advantages in the research of traffic sign target recognition. The rigorous research with appropriate methods and comprehensive results can offer effective reference for subsequent research on DTs cities.

A Comparative Study of Shallow Machine Learning Models and Deep Learning Models for Landslide Susceptibility Assessment Based on Imbalanced Data

A landslide is a type of geological disaster that poses a threat to human lives and property. Landslide susceptibility assessment (LSA) is a crucial tool for landslide prevention. This paper’s primary objective is to compare the performances of conventional shallow machine learning methods and deep learning methods in LSA based on imbalanced data to evaluate the applicability of the two types of LSA models when class-weighted strategies are applied. In this article, logistic regression (LR), random forest (RF), deep fully connected neural network (DFCNN), and long short-term memory (LSTM) neural networks were employed for modeling in the Zigui-Badong area of the Three Gorges Reservoir area, China. Eighteen landslide influence factors were introduced to compare the performance of four models under a class balanced strategy versus a class imbalanced strategy. The Spearman rank correlation coefficient (SRCC) was applied for factor correlation analysis. The results reveal that the elevation and distance to rivers play a dominant role in LSA tasks. It was observed that DFCNN (AUC = 0.87, F1-score = 0.60) and LSTM (AUC = 0.89, F1-score = 0.61) significantly outperformed LR (AUC = 0.89, F1-score = 0.50) and RF (AUC = 0.88, F1-score = 0.50) under the class imbalanced strategy. The RF model achieved comparable outcomes (AUC = 0.90, F1-score = 0.61) to deep learning models under the class balanced strategy and ran at a faster training speed (up to 63 times faster than deep learning models). The LR model performance was inferior to that of the other three models under the balanced strategy. Meanwhile, the deep learning models and the shallow machine learning models showed significant differences in susceptibility spatial patterns. This paper’s findings will aid researchers in selecting appropriate LSA models. It is also valuable for land management policy making and disaster prevention and mitigation.

Automatic topology optimization of echo state network based on particle swarm optimization

The task of time series forecasting is to predict the future trend of data based on the collected historical data, providing theoretical and data support for human judgment and decision making. Randomization-based echo state networks (ESNs) are widely used in the research and application field of time series analysis for their simple structure and fast training speed. The core of the ESN is its dynamic reservoir, which the original reservoir are randomly generated and controlled only by parameter sparsity, often performing poorly on complex tasks and affecting the performance of networks. Manual design of the topology of reservoir is difficult, time-consuming and inconvenient to operate, which is not conducive to the development of ESNs. The construction of a suitable reservoir topology for practical application problems to enrich reservoir dynamics is a hot research point for researchers. In this paper, an automatic optimization method is introduced into the topology optimization of ESN (TP-ESN), and the particle swarm optimization algorithm is used to optimize the topological construction of the ESN. The connection structure between the reservoir neurons is first encoded and then iteratively optimized. The optimized structure is decoded and then the reservoir is initialized for ESN training. Prediction results on Mackey–Glass benchmark time series and two electroencephalogram (EEG) datasets demonstrate that TP-ESN method can have better adaptability, stronger prediction ability and stability than several other manually designed ESN reservoir topologies in the case of relatively complex tasks.