Input-output Mapping Research Articles

Repeated sequential learning increases memory capacity via effective decorrelation in a recurrent neural network

Memories in neural system are shaped through the interplay of neural and learning dynamics under external inputs. By introducing a simple local learning rule to a neural network, we found that the memory capacity is drastically increased by sequentially repeating the learning steps of input-output mappings. The origin of this enhancement is attributed to the generation of a Psuedo-inverse correlation in the connectivity. This is associated with the emergence of spontaneous activity that intermittently exhibits neural patterns corresponding to embedded memories. Stablization of memories is achieved by a distinct bifurcation from the spontaneous activity under the application of each input.

A note on applications of a trajectory-based small-gain theorem to decentralized stabilization of switching networks with generalized dead zones

We propose a method for uniform decentralized stabilitzation for a class of large-scale networks of nonlinear switched systems in lower-triangular form with unknown arbitrary switchings. In contrast to the class of networks with the same structure of interconnections that was considered in [1], the dynamics of each subsystem of our network satisfies more general conditions and its input–output maps are not necessarily right invertible, for instance, they can have dead zones. Another difference from [1] is that the current result is proved by a different approach. Instead of using a small-gain theorem in terms of ISS Lyapunov functions as in [1], we use the trajectory-based small-gain theorem for switched systems proved in [2].

Analysis and Forecast of Urban Economic Vitality in Northeast China

Taking into account the passage of time, the original economic vitality index will vary with changes in social development, we use the BP neural network nearly a decade as the original GDP data for the next 30 years the GDP forecast. BP neural network in 1985, proposed by Rumelhart, the algorithm solves the system of learning problems multilayer neural network connection weights hidden layer [1].It consists of an input layer, a hidden layer, and an output layer. The principle is to continuously adjust the network weights and thresholds by transmitting errors backward and then correcting the errors to achieve the desired input-output mapping.

Operation Skill Acquisition and Fuzzy-Rule Extraction for Drone Control Based on Visual Information Using Deep Learning

In recent years, much research on the unmanned control of a moving vehicle has been conducted, and various robots and motor vehicles moving automatically are being used. However, the more complicated the environment is, the more difficult it is for the autonomous vehicles to move automatically. Even in such a challenging environment, however, an expert with the necessary operation skill can sometimes perform the appropriate control of the moving vehicle. In this research, a method for learning a human’s operation skill using a convolutional neural network (CNN) and setting visual information for input is proposed for learning more complicated environmental information. A CNN is a kind of deep-learning network, and it exhibits high performance in the field of image recognition. In this experiment, the operation knowledge was also visualized using a fuzzy neural network with obtained input-output maps to create fuzzy rules. To verify the effectiveness of this method, an experiment involving operation skill acquisition by some subjects using a drone control simulator was conducted.

Grey wolf optimization-tuned convolutional neural network for transmission line protection with immunity against symmetrical and asymmetrical power swing

The similar current–voltage profile during power swing and fault quite often leads to maloperation of distance relays. As compared to symmetrical power swings, discriminating a swing scenario from a fault becomes more challenging during asymmetrical swings arising due to single-pole tripping. Unlike symmetrical power swings, the presence of zero-sequence and negative-sequence current during asymmetrical swing scenarios hinders the application of classical power swing blocking schemes. In this regard, a convolutional neural network (CNN)-based protection scheme has been proposed in this paper, which, in addition to detecting, classifying, and locating faults, is also able to discriminate between power swing (both symmetrical and asymmetrical) and faults. The discrimination avoids possible maloperation during the non-faulty stressed conditions, thereby overcoming the limitation of the existing protection scheme. With the convolutional neural network, the raw signals are directly fed to the classifier, thus avoiding the computational cost associated with feature extraction in time and frequency domains. With the aim of achieving improved input–output mapping capability of CNN for larger datasets, an evolutionary optimization technique, i.e., grey wolf optimization, has been utilized for determining the optimal values of CNN tuning parameters. The performance of the proposed scheme has been extensively validated for a wide range of fault and power swing conditions in terms of standard indices, i.e., dependability, security, and accuracy. The effectiveness of the proposed scheme has also been evaluated for practical setting by performing real-time simulation on OPAL-RT digital simulator.

ARFIS: Adaptive-Receiver-Based Fuzzy Inference System for Diffusion- Based Molecular Communications

Background: The architecture and sequential learning rule-based underlying ARFIS (adaptive-receiver-based fuzzy inference system) are proposed to estimate and predict the adaptive threshold-based detection scheme for diffusion-based molecular communication (DMC). Method: The proposed system forwards an estimate of the received bits based on the current molecular cumulative concentration, which is derived using sequential training-based principle with weight and bias and an input-output mapping based on both human knowledge in the form of fuzzy IFTHEN rules. The ARFIS architecture is employed to model nonlinear molecular communication to predict the received bits over time series. Result: This procedure is suitable for binary On-OFF-Keying (Book signaling), where the receiver bio-nanomachine (Rx Bio-NM) adapts the 1/0-bit detection threshold based on all previous received molecular cumulative concentrations to alleviate the inter-symbol interference (ISI) problem and reception noise. Conclusion: Theoretical and simulation results show the improvement in diffusion-based molecular throughput and the optimal number of molecules in transmission. Furthermore, the performance evaluation in various noisy channel sources shows promising improvement in the un-coded bit error rate (BER) compared with other threshold-based detection schemes in the literature.

Formal verification of input-output mappings of tree ensembles

Recent advances in machine learning and artificial intelligence are now being considered in safety-critical autonomous systems where software defects may cause severe harm to humans and the environment. Design organizations in these domains are currently unable to provide convincing arguments that their systems are safe to operate when machine learning algorithms are used to implement their software. In this paper, we present an efficient method to extract equivalence classes from decision trees and tree ensembles, and to formally verify that their input-output mappings comply with requirements. The idea is that, given that safety requirements can be traced to desirable properties on system input-output patterns, we can use positive verification outcomes in safety arguments. This paper presents the implementation of the method in the tool VoTE (Verifier of Tree Ensembles), and evaluates its scalability on two case studies presented in current literature. We demonstrate that our method is practical for tree ensembles trained on low-dimensional data with up to 25 decision trees and tree depths of up to 20. Our work also studies the limitations of the method with high-dimensional data and preliminarily investigates the trade-off between large number of trees and time taken for verification.

Generic predictions of output probability based on complexities of inputs and outputs

For a broad class of input-output maps, arguments based on the coding theorem from algorithmic information theory (AIT) predict that simple (low Kolmogorov complexity) outputs are exponentially more likely to occur upon uniform random sampling of inputs than complex outputs are. Here, we derive probability bounds that are based on the complexities of the inputs as well as the outputs, rather than just on the complexities of the outputs. The more that outputs deviate from the coding theorem bound, the lower the complexity of their inputs. Since the number of low complexity inputs is limited, this behaviour leads to an effective lower bound on the probability. Our new bounds are tested for an RNA sequence to structure map, a finite state transducer and a perceptron. The success of these new methods opens avenues for AIT to be more widely used.

Climate-induced thermoregulatory responses in a non-linear thermal environment: investigating the inter-dependencies using a facile artificial neural network-based predictive strategy

Objectives. Given the burgeoning impacts of climatic variability on human health, suitable computational paradigms are used to explore the subsequent ergonomic repercussions. The artificial neural network (ANN), in particular, exhibits near-accurate input–output mapping. However, employment of the ANN to trace the inter-dependencies between the climatic and human thermoregulatory parameters in real-world fuzzy problem landscapes is relatively inadequate. In the present study, the ANN models examined the relationships between climatic, behavioral and intrinsic input factors and the thermoregulatory outputs, namely, sweating and the evaporative heat transfer at the skin surface (E sk). Methods. The data were obtained from nearly 1800 subjects who were exposed to a hot and humid climate outdoors. The ANN models were trained using the Levenberg–Marquardt algorithm combined with Bayesian regularization. Results. The predictability of the ANN models was statistically substantiated. The clothing insulation factor was not included as an input parameter, given its similar values. Intriguingly, the ANN results indicated that fabrics with similar thermal resistances could still affect E sk, plausibly owing to the temporal variation in the evaporative resistance of fabrics among individuals. Conclusion. The reasonably accurate results affirmed the suitability of ANN as a pragmatic technique that could elucidate heat-induced ergonomic challenges.

Design of highly effective multilayer feedforward neural network by using genetic algorithm

AbstractThis paper presents a highly effective and precise neural network method for choosing the activation functions (AFs) and tuning the learning parameters (LPs) of a multilayer feedforward neural network by using a genetic algorithm (GA). The performance of the neural network mainly depends on the learning algorithms and the network structure. The backpropagation learning algorithm is used for tuning the network connection weights, and the LPs are obtained by the GA to provide both fast and reliable learning. Also, the AFs of each neuron in the network are automatically chosen by a GA. The present study consists of 10 different functions to accomplish a better convergence of the desired input–output mapping. Test studies are performed to solve a set of two‐dimensional regression problems for the proposed genetic‐based neural network (GNN) and conventional neural network having sigmoid AFs and constant learning parameters. The proposed GNN has also been tested by applying it to three real problems in the fields of environment, medicine, and economics. Obtained results prove that the proposed GNN is more effective and reliable when compared with the classical neural network structure.

A novel purification machine and fuzzy inference method based hybrid model for wind speed forecasting

As one kind of readily available renewable energy sources, wind is widely used in power generation where wind speed plays an important role. Generally speaking, we need to forecast the wind speed for improving the controllability of wind power generation. However, there exists considerable randomness and instabilities in wind speed data so that it is difficult to obtain accurate forecasting results. In this paper, we propose a novel fuzzy inference method based hybrid model for accurate wind speed forecasting. In this hybrid model, we adopt two strategies to enhance the estimation performance. On one hand, we propose the purification machine which utilize the Irregular Information Reduction Module (IIRM) and the Irrelevant Variable Reduction Module (IVRM) to reduce the randomness and instabilities of the data and to eliminate the variables with zero or negative effect in the wind speed time series. On the other hand, we adopt the developed Single-Input-Rule-Modules based Fuzzy Inference System (SIRM-FIS), the functionally weighted SIRM-FIS (FWSIRM-FIS) to realize the prediction of wind speed. This FWSIRM-FIS utilizes the multi-variable functional weights to dynamically measure the importance of the input variables so that the input-output mapping can be strengthened and more accurate forecasting results can be achieved. Furthermore, detailed experiments and comparisons are given. Experimental results demonstrate that the proposed FWSIRM-FIS and purification machine contributes greatly to deal with the randomness and instability in the wind speed data and yield more accurate forecasting results than those existing excellent forecasting models.

An Effective Wind Speed Estimation Based Extended Optimal Torque Control for Maximum Wind Energy Capture

Most variable-speed wind turbines employ advanced control scheme to improve their performance. In this paper, an extended optimal torque controller is designed based on effective wind speed estimation to control the variable-speed wind turbine. To do this, multilayer perceptron based nonlinear input-output mapping is firstly used for approximating the nonlinear aerodynamics of the wind turbine. In other words, based on this nonlinear mapping, effective wind speed is estimated from the measured rotor speed, the measured pitch angle, and the observed aerodynamic torque by the disturbance observer. After that, the optimal rotor speed command for capturing maximum wind energy is derived from the estimated effective wind speed. And then the optimal torque command is calculated by combing the standard optimal torque formula and a proportional control loop that is added to effectively reduce the moment of inertia. At last, some simulation results are validated to display the availability of the improved effective wind speed estimation algorithm and control strategy. Moreover, the corresponding simulation results indicate that compared with the existing methods, the proposed method increases the accuracy of the effective wind speed estimation by 2-7% and the energy production efficiency by 0.35%.

Design of bonding process parameters for experimentation and ANN-GA model development to maximise diffusion bond strength

Challenges in joining dissimilar aluminium alloys like AA1100 and AA7075 by conventional methods find an alternate methodology in diffusion bonding. The major process parameters of diffusion bonding, the temperature, pressure and holding time were appropriated to maximise the joint strength. Experimental parameters were designed at strategical points to cover the domain of its influence with design expert software and the empirical results were analysed using response surface methodology (RSM). Input-output mapping of results was also done by stochastic modelling tool, the artificial neural network (ANN) and later the process parameter was optimised with genetic algorithm (GA). It is found that the prediction accuracy of ANN model was twice accurate than that of RSM. The optimised temperature, pressure and holding time for sound bonding is 380°C, 10 MPa and 46 min respectively, which were confirmed by experimental results.

A novel Multiplicative Polynomial Kernel for Volterra series identification

Volterra series is especially useful for nonlinear system identification, also thanks to its capability to approximate a broad range of input-output maps. However, its identification from a finite set of data is hard, due to the curse of dimensionality. Recent approaches have shown how regularization strategies can be useful for this task. In this paper, we propose a new regularization network for Volterra models identification. It relies on a new kernel given by the product of basic building blocks. Each block contains some unknown parameters that can be estimated from data using marginal likelihood optimization or cross-validation. In comparison with other algorithms proposed in the literature, numerical experiments show that our approach allows to better select the monomials that really influence the system output, much increasing the prediction capability of the model. The method immediately extends also to polynomial NARMAX models.

CONTROLLABILITY AND GRAMIANS OF 2D CONTINUOUS TIME LINEAR SYSTEMS

The controllability of a class of 2D linear time varying continuous time control systems is studied. The state space representation is provided and the formulas of the states and the input-output map of these systems are derived. The fundamental concepts of controllability and reachability are analysed and suitable controllability and reachability Gramians are constructed to characterize the controllable and the reachable time varying systems. In the case of time invariant 2D systems, some algorithms are developed to calculate different controllability Gramians as solutions of adequate Lyapunov type equations. Corresponding Matlab programs are implemented to solve these Lyapunov equations.

Hamiltonian and Q-Inspired Neural Network-Based Machine Learning

The goal of this study is to present a universal large-scale machine learning model based on spectral processing. By machine learning, we mean input-output mapping approximation generated by training sets. We treat tasks such as pattern recognition and classification as special problems in mapping approximation. The structures of the approximators are implemented using Hamiltonian neural network-based biorthogonal and orthogonal transformations. From a mathematical point of view, these structures can be seen as an implementation of non-expansive mappings. An interesting property of approximators is the reconstruction and recognition of incomplete or distorted patterns. The reconstruction property gives rise to a proposition of a superposition processor and reversible computations. Finally, the models of machine learning described here are adequate for processing data with real and complex values by defining Q-inspired neural networks.

A systematic approach for segmenting voiced/unvoiced signals using fuzzy-logic system and general fusion of neural network models for phonemes-based speech recognition

In this paper, a speech-to-text translation model has been developed for Malaysian speakers based on 41 classes of Phonemes. A simple data acquisition algorithm has been used to develop a MATLAB graphical user interface (GUI) for recording the isolated word speech signals from 35 non-native Malaysian speakers. The collected database consists of 86 words with 41 classes of phoneme based on Affricatives, Diphthongs, Fricatives, Liquid, Nasals, Semivowels and Glides, Stop and Vowels. The speech samples are preprocessed to eliminate the undesirable artifacts and the fuzzy voice classifier has been employed to classify the samples into voiced sequence and unvoiced sequence. The voiced sequences are divided into frame segments and for each frame, the Linear Predictive co-efficients features are obtained from the voiced sequence. Then the feature sets are formed by deriving the LPC features from all the extracted voiced sequences, and used for classification. The isolated words chosen based on the phonemes are associated with the extracted features to establish classification system input-output mapping. The data are then normalized and randomized to rearrange the values into definite range. The Multilayer Neural Network (MLNN) model has been developed with four combinations of input and hidden activation functions. The neural network models are trained with 60%, 70% and 80% of the total data samples. The neural network architecture was aimed at creating a robust model with 60%, 70%, and 80% of the feature set with 25 trials. The trained network model is validated by simulating the network with the remaining 40%, 30%, and 20% of the set. The reliability of trained network models were compared by measuring true-positive, false-negative, and network classification accuracy. The LPC features show better discrimination and the MLNN neural network models trained using the LPC spectral band features gives better recognition.

T–S Fuzzy Control of Travelling-Wave Ultrasonic Motor

The operation of ultrasonic motor is characterized by obvious nonlinearity. Appropriate motor control strategy is an important prerequisite for promoting the wider application of ultrasonic motors. PID controller is widely used in motor motion control products. Due to the nonlinearity of the ultrasonic motor, the PID control parameter values that can achieve good performance are very different at different speeds. In this paper, aiming at the application requirements of ultrasonic motor speed control, a fuzzy controller with simple structure is proposed to realize nonlinear variable-parameter PID control by using nonlinear input–output mapping ability of fuzzy logic. The proposed fuzzy controller has only one input variable with three linguistic values. In order to match the nonlinear operating characteristics of the ultrasonic motor and improve the control precision, the fuzzy rule is designed as the Takagi–Sugeno form. Takagi–Sugeno defuzzification method is adopted. The calculation formula of control quantity given by the fuzzy controller is similar to that of the PI controller, but the control parameters are nonlinearly changed. Compared with the traditional fuzzy controller, the structure of the proposed controller is simple and the amount of online computations is greatly reduced. Experimental results under various operating conditions show that the proposed controller has good control effect and it has robustness to disturbances such as drive voltage amplitude and load torque variation.

A hybrid learning method for the data-driven design of linguistic dynamic systems

In lots of data based prediction or modeling applications, uncertainties and&#x002F or noises in the observed data cannot be avoided. In such cases, it is more preferable and reasonable to provide linguistic &#x0028 fuzzy &#x0029 predicted results described by fuzzy memberships or fuzzy sets instead of the crisp estimates depicted by numbers. Linguistic dynamic system &#x0028 LDS&#x0029 provides a powerful tool for yielding linguistic &#x0028 fuzzy &#x0029 results. However, it is still difficult to construct LDS models from observed data. To solve this issue, this paper first presents a simplified LDS whose input-output mapping can be determined by closed-form formulas. Then, a hybrid learning method is proposed to construct the data-driven LDS model. The proposed hybrid learning method firstly generates fuzzy rules by the subtractive clustering method, then carries out further optimization of centers of the consequent triangular fuzzy sets in the fuzzy rules, and finally adopts multi-objective optimization algorithm to determine the left and right end-points of the consequent triangular fuzzy sets. The proposed approach is successfully applied to three real-world prediction applications which are: prediction of energy consumption of a building, forecasting of the traffic flow, and prediction of the wind speed. Simulation results show that the uncertainties in the data can be effectively captured by the linguistic &#x0028 fuzzy&#x0029 estimates. It can also be extended to some other prediction or modeling problems, in which observed data have high levels of uncertainties.

Comparison of support vector regression and extreme gradient boosting for decomposition-based data-driven 10-day streamflow forecasting

We recently addressed data-driven monthly streamflow forecasting using a decomposition-based model that relies on Fourier transform (FT) to decompose the monthly streamflow time series into 4 components, with each component comprising of contiguous frequencies and being forecasted independently by support vector regression (SVR). The model, called FT-SVR, is capable of achieving near-perfect monthly inflow forecasting for China’s Three Gorges Dam (TGD). In this paper, we adapt FT-SVR to 10-day streamflow forecasting and the TGD is still taken as the case for study. The 10-day inflow time series is decomposed into 7 components, and each component contains an appropriate small number of contiguous frequencies. We also investigate forecasting each decomposed component by extreme gradient boosting (XGBoost) and the investigated model is accordingly named as FT-XGBoost. XGBoost uses an ensemble of trees as the input–output mapping. The maximum number of iterations for solving convex quadratic programming in SVR is considerably reduced and the parameter calibration process to develop each independent SVR or XGBoost model is enhanced. We compare FT-SVR and FT-XGBoost on three different decomposition strategies: the centered 10-day inflow time series, 4 decomposed components like our previous work, and 7 components. Experimental results on the TGD case study demonstrate that FT-SVR with the decomposition strategy of 7 components is able to derive again near-perfect 10-day streamflow forecasting results, significantly better than FT-SVR with the other decomposition strategies and FT-XGBoost with all the decomposition strategies, while with parameter calibration being performed as efficiently as FT-XGBoost on each decomposition strategy.