Deterministic Learning Algorithm Research Articles

Recurrence-Aware Long-Term Cognitive Network for Explainable Pattern Classification.

Machine-learning solutions for pattern classification problems are nowadays widely deployed in society and industry. However, the lack of transparency and accountability of most accurate models often hinders their safe use. Thus, there is a clear need for developing explainable artificial intelligence mechanisms. There exist model-agnostic methods that summarize feature contributions, but their interpretability is limited to predictions made by black-box models. An open challenge is to develop models that have intrinsic interpretability and produce their own explanations, even for classes of models that are traditionally considered black boxes like (recurrent) neural networks. In this article, we propose a long-term cognitive network (LTCN) for interpretable pattern classification of structured data. Our method brings its own mechanism for providing explanations by quantifying the relevance of each feature in the decision process. For supporting the interpretability without affecting the performance, the model incorporates more flexibility through a quasi-nonlinear reasoning rule that allows controlling nonlinearity. Besides, we propose a recurrence-aware decision model that evades the issues posed by the unique fixed point while introducing a deterministic learning algorithm to compute the tunable parameters. The simulations show that our interpretable model obtains competitive results when compared to state-of-the-art white and black-box models.

Early Detection of Coronary Microvascular Dysfunction Using Machine Learning Algorithm Based on Vectorcardiography and Cardiodynamicsgram Features

PurposeAs a main etiology of myocardial ischemia, coronary microvascular dysfunction (CMD) can occur in patients with or without obstructive coronary artery disease. Currently, there is a lack of a non-invasive approach for early detection of CMD. AimWe aim to develop a multilayer perceptron (MLP) algorithm to achieve non-invasive early detection of CMD based on vectorcardiography (VCG) and cardiodynamicsgram (CDG) features. MethodsElectrocardiograms of 82 CMD patients and 107 healthy controls were collected and synthesized into VCGs. The VCGs' ST-T segments were extracted and fed into a deterministic learning algorithm to develop CDGs. Temporal heterogeneity index, spatial heterogeneity index, sample entropy, approximate entropy, and complexity index were extracted from VCGs' ST-T segments and CDGs, entitled as STT- and CDG-based features, respectively. The most effective feature subsets were determined from CDG-based, STT-based, and the combined features (i.e., all features) via the sequential backward selection algorithm as inputs for CDG-, STT-, and CDG-STT-based MLP models optimized with an improved sparrow search algorithm, respectively. Finally, the classification capacity of the corresponding models was evaluated via five-fold cross-validations and tested on a testing dataset to verify the optimal one. ResultsThe CDG-STT-based MLP model had significantly higher evaluated metrics than CDG- and STT-based ones on the validation dataset, with the accuracy, sensitivity, specificity, F1 score, and AUC of 0.904, 0.925, 0.870, 0.870, and 0.897 on the testing dataset respectively. ConclusionsThe MLP model based on VCG and CDG features showed high efficiency in identifying CMD. The CDG-STT-based MLP model may afford a potential computer-aided tool for non-invasive detection of CMD.

Early and Accurate Detection of Radiation-induced Heart Damage by Cardiodynamicsgram.

Cardiodynamicsgram (CDG) has emerged recently as a noninvasive spatiotemporal electrocardiographic method for subtle cardiac dynamics information analysis within electrocardiogram (ECG). This study explored the feasibility of CDG for detecting radiation-induced heart damage (RIHD) in a rat model. A single radiation dose of 40Gy was delivered to the cardiac apex of female Wistar rats. First, CDG was generated through dynamic modeling of ECG signals using the deterministic learning algorithm. Furthermore, CDG indexes were calculated using the wavelet transform and entropy. In this model, CDG entropy indexes decreased significantly after radiotherapy. The shape of CDG changed significantly after radiotherapy (irregular shape) compared with controls (regular shape). Macrophage and fibrosis in myocardium of rats increased significantly after radiotherapy. CDG changes after radiotherapy were significantly correlated with histopathological changes and occurred significantly earlier than histopathological changes. This study provides an experimental basis for the clinical application of CDG for the early detection of RIHD.

Early detection of myocardial ischemia in 12‐lead ECG using deterministic learning and ensemble learning

Background and objective: Early detection of myocardial ischemia is a necessary but difficult problem in cardiovascular diseases. Approaches that exclusively rely on classical ST and T wave changes on the standard 12-lead electrocardiogram (ECG) lack sufficient accuracy in detecting myocardial ischemia. This study aims to construct generalizable models for the detection of myocardial ischemia in patients with subtle ECG waveform changes (namely non-diagnostic ECG) using ensemble learning to integrate ECG dynamic features acquired via deterministic learning. Methods: First, cardiodynamicsgram (CDG), a noninvasive spatiotemporal electrocardiographic method, is generated through dynamic modeling of ECG signals using the deterministic learning algorithm. Then, the spectral fitting exponent, Lyapunov exponent, and Lempel-Ziv complexity are extracted from CDG. Subsequently, the bagging-based heterogeneous ensemble algorithm is applied on CDG features to generate diverse base classifiers and aggregate them with weighted voting to obtain an ensemble model for myocardial ischemia detection. Finally, we train and test the proposed heterogeneous ensemble model on a real-world clinical dataset. This dataset consists of 499 non-diagnostic 12-lead ECG records from 499 patients collected from three independent medical centers, including 383 patients with myocardial ischemia and 116 patients without ischemia. Results: With 10-times 5-fold cross-validation technology, our proposed method achieves an average accuracy of 89.10%, sensitivity of 91.72%, and specificity of 82.69% using the heterogeneous ensemble algorithm on the real-world clinical dataset. On three independent medical centers, our ensemble model also achieves accuracy performance over 82% for patients with non-diagnostic ECG. Furthermore, our ensemble model trained with real-world clinical data yields promising results of 91.11% accuracy, 90.49% sensitivity, and 92.88% specificity on the external test set of the public PTB dataset. Conclusion: The experimental results demonstrate that the proposed model combining ensemble learning and deterministic learning presents excellent diagnostic accuracy and generalization in clinical practice, and could be implemented as a complement to the standard ECG in the clinical diagnosis of myocardial ischemia.

Long short-term cognitive networks

In this paper, we present a recurrent neural system named long short-term cognitive networks (LSTCNs) as a generalization of the short-term cognitive network (STCN) model. Such a generalization is motivated by the difficulty of forecasting very long time series efficiently. The LSTCN model can be defined as a collection of STCN blocks, each processing a specific time patch of the (multivariate) time series being modeled. In this neural ensemble, each block passes information to the subsequent one in the form of weight matrices representing the prior knowledge. As a second contribution, we propose a deterministic learning algorithm to compute the learnable weights while preserving the prior knowledge resulting from previous learning processes. As a third contribution, we introduce a feature influence score as a proxy to explain the forecasting process in multivariate time series. The simulations using three case studies show that our neural system reports small forecasting errors while being significantly faster than state-of-the-art recurrent models.

Topology identification and dynamical pattern recognition for Hindmarsh–Rose neuron model via deterministic learning

Studies have shown that Parkinson’s, epilepsy and other brain deficits are closely related to the ability of neurons to synchronize with their neighbors. Therefore, the neurobiological mechanism and synchronization behavior of neurons has attracted much attention in recent years. In this contribution, it is numerically investigated the complex nonlinear behaviour of the Hindmarsh–Rose neuron system through the time responses, system bifurcation diagram and Lyapunov exponent under different system parameters. The system presents different and complex dynamic behaviors with the variation of parameter. Then, the identification of the nonlinear dynamics and topologies of the Hindmarsh–Rose neural networks under unknown dynamical environment is discussed. By using the deterministic learning algorithm, the unknown dynamics and topologies of the Hindmarsh–Rose system are locally accurately identified. Additionally, the identified system dynamics can be stored and represented in the form of constant neural networks due to the convergence of system parameters. Finally, based on the time-invariant representation of system dynamics, a fast dynamical pattern recognition method via system synchronization is constructed. The achievements of this work will provide more incentives and possibilities for biological experiments and medical treatment as well as other related clinical researches, such as the quantifying and explaining of neurobiological mechanism, early diagnosis, classification and control (treatment) of neurologic diseases, such as Parkinson’s and epilepsy. Simulations are included to verify the effectiveness of the proposed method.

Topology Identification of the Hindmarsh-Rose Model via Deterministic Learning

In this paper, the complex dynamic behavior of the Hindmarsh-Rose (HR) model which characterizes the neuron cell is analyzed numerically. And the unknown topology of the system in dynamic environment is locally accurately identified based on the deterministic learning (DL) algorithm. Firstly, the influence of different parameters on the dynamic behavior of the HR model are investigated. Then, the nonlinear dynamics of the HR model under unknown dynamic environment is locally accurately identified. In addition, the identified system dynamics can be stored in the form of constant neural network. The achievement of this work can provide more incentives and possibilities for the application of HR model in clinic and other related researches. Simulation studies are included to demonstrate the effectiveness.

A Deterministic Learning Algorithm Estimating the Q-Matrix for Cognitive Diagnosis Models

The goal of an exam in cognitive diagnostic assessment is to uncover whether an examinee has mastered certain attributes. Different cognitive diagnosis models (CDMs) have been developed for this purpose. The core of these CDMs is the Q-matrix, which is an item-to-attribute mapping, traditionally designed by domain experts. An expert designed Q-matrix is not without issues. For example, domain experts might neglect some attributes or have different opinions about the inclusion of some entries in the Q-matrix. It is therefore of practical importance to develop an automated method to estimate the Q-matrix. This research proposes a deterministic learning algorithm for estimating the Q-matrix. To obtain a sensible binary Q-matrix, a dichotomizing method is also devised. Results from the simulation study shows that the proposed method for estimating the Q-matrix is useful. The empirical study analyzes the ECPE data. The estimated Q-matrix is compared with the expert-designed one. All analyses in this research are carried out in R.

Revisiting Online Quantum State Learning

In this paper, we study the online quantum state learning problem which is recently proposed by Aaronson et al. (2018). In this problem, the learning algorithm sequentially predicts quantum states based on observed measurements and losses and the goal is to minimize the regret. In the previous work, the existing algorithms may output mixed quantum states. However, in many scenarios, the prediction of a pure quantum state is required. In this paper, we first propose a Follow-the-Perturbed-Leader (FTPL) algorithm that can guarantee to predict pure quantum states. Theoretical analysis shows that our algorithm can achieve an O(√T) expected regret under some reasonable settings. In the case that the pure state prediction is not mandatory, we propose another deterministic learning algorithm which is simpler and more efficient. The algorithm is based on the online gradient descent (OGD) method and can also achieve an O(√T) regret bound. The main technical contribution of this result is an algorithm of projecting an arbitrary Hermitian matrix onto the set of density matrices with respect to the Frobenius norm. We think this subroutine is of independent interest and can be widely used in many other problems in the quantum computing area. In addition to the theoretical analysis, we evaluate the algorithms with a series of simulation experiments. The experimental results show that our FTPL method and OGD method outperform the existing RFTL approach proposed by Aaronson et al. (2018) in almost all settings. In the implementation of the RFTL approach, we give a closed-form solution to the algorithm. This provides an efficient, accurate, and completely executable solution to the RFTL method.

Local Sigmoid Method: Non-Iterative Deterministic Learning Algorithm for Automatic Model Construction of Neural Network

A non-iterative learning algorithm for artificial neural networks is an alternative to optimize the neural network parameters with extremely fast convergence time. Extreme learning machine (ELM) is one of the fastest learning algorithms based on a non-iterative method for a single hidden layer feedforward neural network (SLFN) model. ELM uses a randomization technique that requires a large number of hidden nodes to achieve the high accuracy. This leads to a large and complex model, which is slow at the inference time. Previously, we reported analytical incremental learning (AIL) algorithm, which is a compact model and a non-iterative deterministic learning algorithm, to be used as an alternative. However, AIL cannot grow its set of hidden nodes, due to the node saturation problem. Here, we describe a local sigmoid method (LSM) that is also a sufficiently compact model and a non-iterative deterministic learning algorithm to overcome both the ELM randomization and AIL node saturation problems. The LSM algorithm is based on “divide and conquer” method that divides the dataset into several subsets which are easier to optimize separately. Each subset can be associated with a local segment represented as a hidden node that preserves local information of the subset. This technique helps us to understand the function of each hidden node of the network built. Moreover, we can use such a technique to explain the function of hidden nodes learned by backpropagation, the iterative algorithm. Based on our experimental results, LSM is more accurate than other non-iterative learning algorithms and one of the most compact models.

Human gait recognition based on deterministic learning and knowledge fusion through multiple walking views

Deformation of gait silhouettes caused by different view angles heavily affects the performance of gait recognition. In this paper, a new method based on deterministic learning and knowledge fusion is proposed to eliminate the effect of view angle for efficient view-invariant gait recognition. First, the binarized walking silhouettes are characterized with three kinds of time-varying width parameters. The nonlinear dynamics underlying different individuals’ width parameters is effectively approximated by radial basis function (RBF) neural networks through deterministic learning algorithm. The extracted gait dynamics captures the spatio-temporal characteristics of human walking, represents the dynamics of gait motion, and is shown to be insensitive to the variance across various view angles. The learned knowledge of gait dynamics is stored in constant RBF networks and used as the gait pattern. Second, in order to handle the problem of view change no matter the variation is small or large, the learned knowledge of gait dynamics from different views is fused by constructing a deep convolutional and recurrent neural network (CRNN) model for later human identification task. This knowledge fusion strategy can take advantage of the encoded local characteristics extracted from the CNN and the long-term dependencies captured by the RNN. Experimental results show that promising recognition accuracy can be achieved.

Deterministic Learning-Based Methodology for Detecting Abnormal Dynamics of Cardiac Repolarization During Ischemia.

This study concentrates on subtle electrocardiogram (ECG) spatiotemporal characteristics in the repolarization phase, and describes a deterministic learning-based methodology for the detection of abnormal cardiac dynamics induced by ischemia. ST-T complex of the surface 12-lead ECG signals are identified and extracted. Cardiac dynamics underlying ST-T complex signals is captured using deterministic learning algorithm. This kind of dynamics information represents the beat-to-beat temporal change of electrophysiological modifications in ventricular repolarization, which is shown to be sensitive to the variance during myocardial ischemia. Cardiodynamicsgram (CDG) is proposed as the three-dimensional graphic representation of cardiac dynamics information. Encouraging evaluation results are achieved on electrocardiograms from public PTB database and hospital patients. Significant correlations are found between the CDG morphology and ischemia. Anormal dynamics of cardiac repolarization during ischemia can be detected using a deterministic learning-based methodology. The extracted cardiac dynamics information within routine ECG is expected to provide early detection for latent ischemia before obvious pathological changes are present in ECG. The proposed techniques can be considered as a complementary tool to the generally accepted ECG method for detection of abnormal dynamics in cardiac repolarization, which are important for identifying patients at risk of myocardial ischemia.

An integrated critic-actor neural network for reinforcement learning with application of DERs control in grid frequency regulation

As the electronically-interfaced distributed energy resources (DERs) grow rapidly in power grid, power demand satisfaction and frequency regulation are two main challenges in control area. However, it is difficult to model the analysis of a large-scale grid as well as design a stable and optimal control scheme. With the support of DERs, this paper proposes an actor-critic neural network that integrates a distributed reinforcement learning control scheme to compensate frequency regulation of power grid. The short-term performance and stability is improved by a deterministic learning algorithm that is used to obtain the approximation of desired control output. Meanwhile, a long-term strategic utility function is estimated by the integrated actor-critic neural network. The mapping from system state and control output to the strategic utility function value is identified by neural network, as well as utilized in sub-optimal control learning for further improvement of long-term system performance. Theoretical analysis guarantees the stability. Frequency deviation, tie-line power flow, and long-term cost are coincident with uniform ultimate boundness (UUB). In addition, the upper bound of long-term system cost is also reckoned. The effectiveness and advantages of proposed scheme are illustrated in two case studies. The simulation results indicate that the proposed scheme has better performance under certain condition, compared with some actor-critic network control schemes in frequency regulation of power grid.

Abrupt stall detection for axial compressors with non-uniform inflow via deterministic learning

This paper focuses on the early detection problem of abrupt stall inception in axial compressors subject to non-uniform inflow. The considered stall inception is small and abrupt. Such a stall inception is not only easy to cause system instability, but also difficult to be rapidly detected. In this paper, a novel abrupt stall detection approach is proposed for axial compressors with non-uniform inflow by using the deterministic learning algorithm. Firstly, a high-order distortion dynamic mode is derived to generate the normal and stall inception situations. Subsequently, dynamic features of these different situations are extracted and stored in constant radial basis function networks, which are regarded as the training patterns. Based on dynamic pattern recognition, the average L1 norms of the state errors are used as the measure of the dynamical differences between the test pattern and the training patterns, and then a stall detection method is developed to rapidly decide whether the test patterns are the abrupt stall inception. Simulation studies are given to show that the proposed stall detection approach is sensitive to small dynamic changes of axial compressors, thereby rapidly detecting the abrupt stall inception.

A Stall Warning Scheme for Aircraft Engines With Inlet Distortion via Deterministic Learning

Inlet flow nonuniformity (inlet distortion) can influence the transient behavior of stall disturbances and trigger the compression system instability in aircraft engines. Stall warning provides an approach to avoiding the occurrence of the instability. In this brief, based on the high-order distortion model developed recently, a stall warning scheme employing deterministic learning (DL) algorithm is proposed for aircraft engines with inlet distortion. The high-order distortion model can capture the transient behavior of stall inception for a test rig and be suitable for the analysis of stall warning. First, the system dynamics underlying normal and stall inception are locally accurately approximated through DL. The obtained knowledge of system dynamics is stored in constant radial basis function (RBF) networks. Secondly, a bank of estimators is constructed using the stored constant RBF networks to represent the learning normal and stall inception patterns. By comparing each estimator with a test system, the average $L_{1}$ norms of the residuals are taken as the measure of the dynamical differences between the test system and the learning patterns. The occurrence of stall inception as an early warning signal can be rapidly detected according to the smallest residual principle. Finally, numerical simulation results are given to show the effectiveness of stall warning approach.

Individual identification using a gait dynamics graph

In this paper, we propose a new gait representation—gait dynamics graph (GDG) for individual identification. For each gait sequence, lower limbs joint angles are extracted as gait parameters, and gait system dynamics underlying time-varying gait parameter trajectories is captured by using deterministic learning algorithm. Gait dynamics graph (GDG) is then generated by plotting the extracted dynamics information into three-dimensional graphic. Unlike other gait representations, which are not embedded with dynamics information, GDG demonstrates nonlinear gait dynamics in a new, visually intuitive manner using three-dimensional graphic representation. Both direct matching method and nonlinear dynamics analysis method can be used for GDG recognition independently. The performance of the proposed representation is evaluated and compared with the other representations experimentally on five large benchmark gait databases. This kind of gait representation is embedded with more distinctive information and preserves temporal dynamics information of human walking, which does not rely on shape or silhouettes information. Experimental results show that the GDG representation can further improve recognition rates and avoid the great drop of recognition rate when the training and test sets are under different walking conditions.

Fusion of spatial-temporal and kinematic features for gait recognition with deterministic learning

For obtaining optimal performance, as many informative cues as possible should be involved in the gait recognition algorithm. This paper describes a gait recognition algorithm by combining spatial-temporal and kinematic gait features. For each walking sequence, the binary silhouettes are characterized with four time-varying spatial-temporal parameters, including three lower limbs silhouette widths and one holistic silhouette area. Using deterministic learning algorithm, spatial-temporal gait features can be represented as the gait dynamics underlying the trajectories of lower limbs silhouette widths and holistic silhouette area, which can implicitly reflect the temporal changes of silhouette shape. In addition, a model-based method is proposed to extract joint-angle trajectories of lower limbs. Kinematic gait features can be represented as the gait dynamics underlying the trajectories of joint angles, which can represent the temporal changes of body structure and dynamics. Both spatial-temporal and kinematic cues can be used separately for gait recognition using smallest error principle. They are fused on the decision level using different combination rules to improve the gait recognition performance. The fusion of two different kinds of features provides a comprehensive characterization of gait dynamics, which is not sensitive to the walking conditions variation. The proposed method can still achieve superior performance when the testing walking conditions are different from the corresponding training conditions. Experimental results show that encouraging recognition accuracy can be achieved on five public gait databases: CASIA-B, CASIA-C, TUM GAID, OU-ISIR, USF HumanID.

Prediction of Period-Doubling Bifurcation Based on Dynamic Recognition and Its Application to Power Systems

In this paper, a bifurcation prediction approach is proposed based on dynamic recognition and further applied to predict the period-doubling bifurcation (PDB) of power systems. Firstly, modeling of the internal dynamics of nonlinear systems is obtained through deterministic learning (DL), and the modeling results are applied for constructing the dynamic training pattern database. Specifically, training patterns are chosen according to the hierarchical structured knowledge representation based on the qualitative property of dynamical systems, which is capable of arranging the dynamical models into a specific order in the pattern database. Then, a dynamic recognition-based bifurcation prediction approach is suggested. As a result, perturbations implying PDB on the testing patterns can be predicted through the minimum dynamic error between the training patterns and testing patterns by recalling the knowledge restored in the pattern database. Finally, the second-order single-machine to infinite bus power system model is introduced to check the effectiveness of this prediction approach, which implies PDB under small periodic parameter perturbations. The key point that determines the prediction effect mainly lies in two methods: (1) accurate approximation of the unknown system dynamics through DL guarantees the feasibility of the prediction process; (2) the qualitative property of PDB and the generalization ability of DL algorithm ensure the validity of the selected training patterns. Simulations are included to illustrate the effectiveness of the proposed approach.

Human gait recognition based on deterministic learning through multiple views fusion

Gait characteristics extracted from one single camera are limited and not comprehensive enough to develop a robust recognition system. This paper proposes a robust gait recognition method using multiple views fusion and deterministic learning. First, a multiple-views fusion strategy is introduced, in which gaits collected under different views are synthesized as a kind of synthesized silhouette images. Second, the synthesized silhouettes are characterized with four kinds of time-varying gait features, including three width features of the silhouette and one silhouette area feature. Third, gait variability underlying different individuals’ time-varying gait features is effectively modeled by using deterministic learning algorithm. This kind of variability reflects the change of synthesized silhouettes while preserving temporal dynamics information of human walking. Gait patterns are represented as the gait variability underlying time-varying gait features and a rapid recognition scheme is presented in published gait databases. Experimental results show that encouraging recognition accuracy can be achieved.

Fault detection for nonlinear discrete-time systems via deterministic learning

Recently, an approach for the rapid detection of small oscillation faults based on deterministic learning theory was proposed for continuous-time systems. In this paper, a fault detection scheme is proposed for a class of nonlinear discrete-time systems via deterministic learning. By using a discrete-time extension of deterministic learning algorithm, the general fault functions (i.e., the internal dynamics) underlying normal and fault modes of nonlinear discrete-time systems are locally-accurately approximated by discrete-time dynamical radial basis function (RBF) networks. Then, a bank of estimators with the obtained knowledge of system dynamics embedded is constructed, and a set of residuals are obtained and used to measure the differences between the dynamics of the monitored system and the dynamics of the trained systems. A fault detection decision scheme is presented according to the smallest residual principle, i.e., the occurrence of a fault can be detected in a discrete-time setting by comparing the magnitude of residuals. The fault detectability analysis is carried out and the upper bound of detection time is derived. A simulation example is given to illustrate the effectiveness of the proposed scheme.