Hybrid Learning Scheme Research Articles

Optimal Adaptive Tracking Control of Partially Uncertain Nonlinear Discrete-Time Systems Using Lifelong Hybrid Learning.

This article addresses a multilayer neural network (MNN)-based optimal adaptive tracking of partially uncertain nonlinear discrete-time (DT) systems in affine form. By employing an actor-critic neural network (NN) to approximate the value function and optimal control policy, the critic NN is updated via a novel hybrid learning scheme, where its weights are adjusted once at a sampling instant and also in a finite iterative manner within the instants to enhance the convergence rate. Moreover, to deal with the persistency of excitation (PE) condition, a replay buffer is incorporated into the critic update law through concurrent learning. To address the vanishing gradient issue, the actor and critic MNN weights are tuned using control input and temporal difference errors (TDEs), respectively. In addition, a weight consolidation scheme is incorporated into the critic MNN update law to attain lifelong learning and overcome catastrophic forgetting, thus lowering the cumulative cost. The tracking error, and the actor and critic weight estimation errors are shown to be bounded using the Lyapunov analysis. Simulation results using the proposed approach on a two-link robot manipulator show a significant reduction in tracking error by 44% and cumulative cost by 31% in a multitask environment.

Differential Neural Networks Prediction Using Slow and Fast Hybrid Learning: Application to Prognosis of Infectionsand Deaths of COVID-19 Dynamics.

This essay discusses a potential method for predicting the behavior of various physical processes and uses the COVID-19 outbreak to demonstrate its applicability. This study assumes that the current data set reflects the output of a dynamic system that is governed by a nonlinear ordinary differential equation. This dynamic system may be described by a Differential Neural Network (DNN) with time-varying weights matrix parameters. A new hybrid learning scheme based on the decomposition of the signal to be predicted. The decomposition considers the slow and fast components of the signal which is more natural to signals such as the ones corresponding to the number of infected and deceased patients who suffered of COVID 2019 sickness. The paper results demonstrate the recommended method offers competitive performance (70 days of COVID prediction) in comparison to similar studies.

A hybrid learning scheme towards authenticating hand-geometry using multi-modal features

<span lang="EN-US">Usage of hand geometry towards biometric-based authentication mechanism has been commercially practiced since last decade. However, there is a rising security problem being surfaced owing to the fluctuating features of hand-geometry during authentication mechanism. Review of existing research techniques exhibits the usage of singular features of hand-geometric along with sophisticated learning schemes where accuracy is accomplished at the higher cost of computational effort. Hence, the proposed study introduces a simplified analytical method which considers multi-modal features extracted from hand geometry which could further improve upon robust recognition system. For this purpose, the system considers implementing hybrid learning scheme using convolution neural network and Siamese algorithm where the former is used for feature extraction and latter is used for recognition of person on the basis of authenticated hand geometry. The main results show that proposed scheme offers 12.2% of improvement in accuracy compared to existing models exhibiting that with simpler amendment by inclusion of multi-modalities, accuracy can be significantly improve without computational burden.</span>

Integrating Domain Knowledge in Data-Driven Earth Observation With Process Convolutions

The modelling of Earth observation data is a challenging problem, typically approached by either purely mechanistic or purely data-driven methods. Mechanistic models encode the domain knowledge and physical rules governing the system. Such models, however, need the correct specification of all interactions between variables in the problem and the appropriate parameterization is a challenge in itself. On the other hand, machine learning approaches are flexible data-driven tools, able to approximate arbitrarily complex functions, but lack interpretability and struggle when data is scarce or in extrapolation regimes. In this paper, we argue that hybrid learning schemes that combine both approaches can address all these issues efficiently. We introduce Gaussian process (GP) convolution models for hybrid modelling in Earth observation (EO) problems. We specifically propose the use of a class of GP convolution models called latent force models (LFMs) for EO time series modelling, analysis and understanding. LFMs are hybrid models that incorporate physical knowledge encoded in differential equations into a multioutput GP model. LFMs can transfer information across time-series, cope with missing observations, infer explicit latent functions forcing the system, and learn parameterizations which are very helpful for system analysis and interpretability. We consider time series of soil moisture from active (ASCAT) and passive (SMOS, AMSR2) microwave satellites. We show how assuming a first order differential equation as governing equation, the model automatically estimates the e-folding time or decay rate related to soil moisture persistence and discovers latent forces related to precipitation. The proposed hybrid methodology reconciles the two main approaches in remote sensing parameter estimation by blending statistical learning and mechanistic modeling.

Training of quantum circuits on a hybrid quantum computer.

Generative modeling is a flavor of machine learning with applications ranging from computer vision to chemical design. It is expected to be one of the techniques most suited to take advantage of the additional resources provided by near-term quantum computers. Here, we implement a data-driven quantum circuit training algorithm on the canonical Bars-and-Stripes dataset using a quantum-classical hybrid machine. The training proceeds by running parameterized circuits on a trapped ion quantum computer and feeding the results to a classical optimizer. We apply two separate strategies, Particle Swarm and Bayesian optimization to this task. We show that the convergence of the quantum circuit to the target distribution depends critically on both the quantum hardware and classical optimization strategy. Our study represents the first successful training of a high-dimensional universal quantum circuit and highlights the promise and challenges associated with hybrid learning schemes.

Active learning strategy and hybrid training for infarct segmentation on diffusion MRI with a U-shaped network.

Automatic and reliable stroke lesion segmentation from diffusion magnetic resonance imaging (MRI) is critical for patient care. Methods using neural networks have been developed, but the rate of false positives limits their use in clinical practice. A training strategy applied to three-dimensional deconvolutional neural networks for stroke lesion segmentation on diffusion MRI was proposed. Infarcts were segmented by experts on diffusion MRI for 929 patients. We divided each database as follows: 60% for a training set, 20% for validation, and 20% for testing. Our hypothesis was a two-phase hybrid learning scheme, in which the network was first trained with whole MRI (regular phase) and then, in a second phase (hybrid phase), alternately with whole MRI and patches. Patches were actively selected from the discrepancy between expert and model segmentation at the beginning of each batch. On the test population, the performances after the regular and hybrid phases were compared. A statistically significant Dice improvement with hybrid training compared with regular training was demonstrated ( ). The mean Dice reached . False positives were reduced by almost 30% with hybrid training ( ). Our hybrid training strategy empowered deep neural networks for more accurate infarct segmentations on diffusion MRI.

Complex neural fuzzy system and its application on multi-class prediction — A novel approach using complex fuzzy sets, IIM and multi-swarm learning

In this paper, we present a novel complex neural fuzzy approach to multi-class prediction. The complex neural fuzzy system (CNFS) is proposed using complex fuzzy sets (CFSs), fuzzy causalities and multi-swarm machine learning. In general, CFSs are regarded as advanced fuzzy sets with membership degrees defined in the unit disk of the complex plane, in contrast to regular fuzzy sets with membership degrees in the real-valued unit interval [0,1]. The proposed model is composed of the premises designed by CFSs, the consequences designed by Takagi–Sugeno linear functions, and a fuzzy causality layer connecting the premises toward the consequences, and it is able to perform prediction of multiple targets. The usage of fuzzy causality in the proposed model makes difference to traditional fuzzy models using If-Then rules, and gives the freedom and flexibility for model construction. To optimize the proposed CNFS, we present a hybrid learning scheme using the particle swarm optimization with multiple swarms (denoted as PSOmsw) and the Kalman filtering algorithm (denoted as KFA). In the hybrid learning method, the KFA updates the consequence parameters while the PSOmsw evolves the rest parameters of the model. The proposed approach has been tested with experiments using several real-world stock market datasets. Compared with other methods, the proposed approach has shown excellent performance.

Approximate Optimal Distributed Control of Nonlinear Interconnected Systems Using Event-Triggered Nonzero-Sum Games.

In this paper, approximate optimal distributed control schemes for a class of nonlinear interconnected systems with strong interconnections are presented using continuous and event-sampled feedback information. The optimal control design is formulated as an N -player nonzero-sum game where the control policies of the subsystems act as players. An approximate Nash equilibrium solution to the game, which is the solution to the coupled Hamilton-Jacobi equation, is obtained using the approximate dynamic programming-based approach. A critic neural network (NN) at each subsystem is utilized to approximate the Nash solution and novel event-sampling conditions, that are decentralized, are designed to asynchronously orchestrate the sampling and transmission of state vector at each subsystem. To ensure the local ultimate boundedness of the closed-loop system state and NN parameter estimation errors, a hybrid-learning scheme is introduced and the stability is guaranteed using Lyapunov-based stability analysis. Finally, implementation of the proposed event-based distributed control scheme for linear interconnected systems is discussed. For completeness, Zeno-free behavior of the event-sampled system is shown analytically and a numerical example is included to support the analytical results.

Evolutional RBFNs image model describing-based segmentation system designs

Knowledge discovered-based radial basis function neural networks (RBFNs) model can describe an appropriate behaviors of identified image patterns through the multiple and hybrid learning schemes. The image data extraction learning algorithm (IDELA) with dynamic recognitions to automatically match the appropriate feature with a suitable number of radial basis function (RBFs). This first step approaches their associated centers positions to extract initial prototypes. The approximated image model as a describer is automatically generated by the RBFPSO learning scheme, which is contained hybrid bacterial foraging particle swarm optimization (BFPSO) algorithm and recursive least-squares (RLS) iterations to deeply approach the image feature. Due to the limitations and possible local learning trap, K-means, differential evolution (DE) and particle swarm optimization (PSO) learning algorithms cannot obtain the most smaller Root-Mean-Square Error (RMSE) to achieve an appropriate image segmentation in all experiment cases. The constructed RBFNs image model is generated by the support of multiple image self-extraction feature machine (MISEFM), which combined IDELA and RBFPSO algorithms to develop the universal RBFNs image describers. Simulations compared with other K-means, PSO and DE learning methods, show the average great performance in several real image segmentation applications. The peak signal-to-noise ratio (PSNR) index is selected to evaluate the quality of the reconstructed images. Simulations show that the evolutional hybrid and multi-level RBFNs image model-based system is determined to simultaneously achieve both high performance indexes on accuracy (RMSE) and a high image quality description (PSNR) for matching the desired characters and behaviors of image patterns within a fewer RBFs functions.

Event-Triggered Distributed Approximate Optimal State and Output Control of Affine Nonlinear Interconnected Systems.

This paper presents an approximate optimal distributed control scheme for a known interconnected system composed of input affine nonlinear subsystems using event-triggered state and output feedback via a novel hybrid learning scheme. First, the cost function for the overall system is redefined as the sum of cost functions of individual subsystems. A distributed optimal control policy for the interconnected system is developed using the optimal value function of each subsystem. To generate the optimal control policy, forward-in-time, neural networks are employed to reconstruct the unknown optimal value function at each subsystem online. In order to retain the advantages of event-triggered feedback for an adaptive optimal controller, a novel hybrid learning scheme is proposed to reduce the convergence time for the learning algorithm. The development is based on the observation that, in the event-triggered feedback, the sampling instants are dynamic and results in variable interevent time. To relax the requirement of entire state measurements, an extended nonlinear observer is designed at each subsystem to recover the system internal states from the measurable feedback. Using a Lyapunov-based analysis, it is demonstrated that the system states and the observer errors remain locally uniformly ultimately bounded and the control policy converges to a neighborhood of the optimal policy. Simulation results are presented to demonstrate the performance of the developed controller.

Thermal error modelling of motorised spindle in large-sized gear grinding machine

Thermal errors are one of the most significant factors that influence the machining precision of machine tools. For large-sized gear grinding machine tools, thermal errors of beds, columns and rotary tables are decreased by their huge heat capacity. However, different from machine tools of normal sizes, thermal errors increase with greater power in motorised spindles. Thermal error compensation is generally considered as a relatively effective, convenient and cost-efficient approach in thermal error control and reduction. This article proposes two thermal error prediction models for motorised spindles based on an adaptive neuro-fuzzy inference system and support vector machine, respectively. In the adaptive neuro-fuzzy inference system–based model, the temperature values are divided into different groups using subtractive clustering. A hybrid learning scheme is adopted to adjust membership functions so as to learn from the input data. In the particle swarm optimisation support vector machine–based model, particle swarm optimisation is used to optimise the hyperparameters of the established model. Thermal balance experiments are conducted on a large-sized computer numerical control gear grinding machine tool to establish the prediction models. Comparative results show that the adaptive neuro-fuzzy inference system model has higher prediction accuracy (with residual errors within ±2.5 μm in the radial direction and ±3 μm in the axial direction) than the support vector machine model.

Navigational strategy for underwater mobile robot based on adaptive neuro-fuzzy inference system model embedded with shuffled frog leaping algorithm–based hybrid learning approach

In this research article, a novel navigational approach has been introduced for underwater robot based on learning and self-adaptation ability of adaptive neuro-fuzzy inference system. For avoiding obstacles during three-dimensional navigation, two adaptive neuro-fuzzy inference system models have been coupled to find out required change in heading angles of underwater robot in horizontal and vertical planes, respectively. A new hybrid learning scheme has been proposed for adaptive neuro-fuzzy inference system. Here, memetic approach based shuffled frog leaping algorithm has been used to tune the premise parameters and consequent parameters has been estimated through recursive least square estimation. Minimization of error in output of adaptive neuro-fuzzy inference system model has been treated as major objective of evolutionary-based training algorithm. Preliminary robotic behaviors of underwater robot have been successfully executed by implementing such well-trained adaptive neuro-fuzzy inference system architecture within three-dimensional unspecified workspace. Navigational performance of adaptive neuro-fuzzy inference system trained with the proposed hybrid learning algorithm has been compared with other three-dimensional navigational approaches in simulation mode for authentication purpose. Experimental verification has also been carried out to validate the feasibility and efficiency of the proposed navigational strategy.

Analysis of Blended Learning Scheme Based on Cloud Computing Assisted Instructions

With the development of science and technology, cloud computing technology has been widely applied in a variety of engineering fields, while its application and promotion in the education is always neglected. In this paper, the cloud computing assisted hybrid learning scheme is put forward based on the analysis of the cloud computing technology and the hybrid method. Firstly, the cloud computing platform architecture and key technology are explained. Secondly, the four stages of the blended learning scheme model are established in accordance with its guiding ideology and construction principle. Furthermore, the four stages are given a detailed description and analysis in which seven blended teaching links are concluded in combination with Baihui cloud platform application characteristics. Finally, the scores from students of different grades and different years are compared. The comparison result verifies the blended learning scheme is effective in the classroom, and is helpful to stimulate the students’ learning interest and enthusiasm, which is significant for the further in-depth study.

An ensemble neuro-fuzzy radial basis network with self-adaptive swarm based supervisor and negative correlation for modeling automotive engine coldstart hydrocarbon emissions: A soft solution to a crucial automotive problem

In this paper, the authors propose a novel intelligent framework to identify the exhaust gas temperature (Texh) and the engine-out hydrocarbon emission (HCraw) during the coldstart operation of an automotive engine. These are two key variables affecting the cumulative tailpipe emissions (HCcum) over the coldstart phase, which is the number one emission-related problem for today's spark-ignited (SI) engine vehicles. The coldstart operation is regarded as a highly nonlinear, transient and uncertain phenomenon. The proposed identifier integrates different soft computational strategies, i.e. neuro-fuzzy computing, fuzzy controller, swarm intelligent computing, and ensemble network design, beneficial for capturing both uncertainty and nonlinearity of the problem at hand. Furthermore, concepts of negative correlation topology design and hierarchical pair competition based parallel training are extracted from literature to form a diverse and robust ensemble identifier. Training of each neuro-fuzzy sub-component in ensemble network is carried out using a hybrid learning scheme. One feature of the antecedent part of neuro-fuzzy system, i.e. number of linguistic terms for each variable, as well as characteristics of rules in rule base are adjusted using hierarchical fair competition-based parallel adaptive particle swarm optimization (HFC-APSO) and the rest of features, i.e. the shape of (membership functions) MFs and the consequent variables of each rule, are tuned using back-propagation (BP) and steepest descent techniques. As it was mentioned, the authors try to design an ensemble identifier with acceptable rate of generalization, robustness and accuracy. These features help them to tame the intuitive uncertainties associated with the rate of Texh and HCraw emission over the coldstart period. To do so, the potential characteristics of sub-components (solution domain of network design) are divided into a set of partitions and then HFC-APSO is utilized to explore/exploit each of those partitions. The exploration/exploitation rate of PSO (the core of HFC-APSO) is dynamically controlled by a fuzzy logic based controller. Hence, it is expected that HFC-APSO yields a set of accurate sub-identifiers with different operating characteristics. To further foster the diversity of the ensemble, negative correlation criterion is considered which obstructs the integration of identical sub-identifiers. The identification results demonstrate that the method is highly capable of providing an authentic model for estimation of Texh and HCraw emission during the coldstart period.

A Novel Learning Scheme for Chebyshev Functional Link Neural Networks

A hybrid learning scheme (ePSO‐BP) to train Chebyshev Functional Link Neural Network (CFLNN) for classification is presented. The proposed method is referred as hybrid CFLNN (HCFLNN). The HCFLNN is a type of feed‐forward neural networks which have the ability to transform the nonlinear input space into higher dimensional‐space where linear separability is possible. Moreover, the proposed HCFLNN combines the best attribute of particle swarm optimization (PSO), back propagation learning (BP learning), and functional link neural networks (FLNNs). The proposed method eliminates the need of hidden layer by expanding the input patterns using Chebyshev orthogonal polynomials. We have shown its effectiveness of classifying the unknown pattern using the publicly available datasets obtained from UCI repository. The computational results are then compared with functional link neural network (FLNN) with a generic basis functions, PSO‐based FLNN, and EFLN. From the comparative study, we observed that the performance of the HCFLNN outperforms FLNN, PSO‐based FLNN, and EFLN in terms of classification accuracy.

Applications of machine learning approach on multi-queue message scheduling

Due to limited resource contentions and deadline constraints, messages on the controller area network (CAN) are competing for service from the common resources. This problem can be resolved by assigning priorities to different message classes to satisfy time-critical applications. Actually, because of the fluctuation of network traffic or an inefficient use of resources, these static or dynamic priority policies may not guarantee flexibility for different kinds of messages in real-time scheduling. Consequently, the message transmission which cannot comply with the timing requirements or deadlines may deteriorate system performance significantly. In this paper, we have proposed a controller-plant model, where the plant is analogous to a message queue pool (MQP) and the message scheduling controller (MSC) is responsible to dispatch resources for queued messages according to the feedback information from the MQP. The message scheduling controller, which is realized by the radial basis function (RBF) network, is designed with machine learning algorithm to compensate the variations in plant dynamics. The MSC with the novel hybrid learning schemes can ensure a low and stable message waiting time variance (or a uniform distribution of waiting time) and lower transmission failures. A significant emphasis of the MSC is the variable structure of the RBF model to accommodate to complex scheduling situations. Simulation experiments have shown that several variants of the MSC significantly improve overall system performance over the static scheduling strategies and the dynamic earliest-deadline first (EDF) algorithms under a wide range of workload characteristics and execution environments.

A comprehensive survey on functional link neural networks and an adaptive PSO–BP learning for CFLNN

Functional link neural network (FLNN) is a class of higher order neural networks (HONs) and have gained extensive popularity in recent years. FLNN have been successfully used in many applications such as system identification, channel equalization, short-term electric-load forecasting, and some of the tasks of data mining. The goals of this paper are to: (1) provide readers who are novice to this area with a basis of understanding FLNN and a comprehensive survey, while offering specialists an updated picture of the depth and breadth of the theory and applications; (2) present a new hybrid learning scheme for Chebyshev functional link neural network (CFLNN); and (3) suggest possible remedies and guidelines for practical applications in data mining. We then validate the proposed learning scheme for CFLNN in classification by an extensive simulation study. Comprehensive performance comparisons with a number of existing methods are presented.

HYBRID AND INCREMENTAL FUZZY LEARNING FOR HUMAN SKIN DETECTION

In this paper, a framework for detection of human skin in digital images is proposed. This framework is composed of a training phase and a detection phase. A skin class model is learned during the training phase by processing several training images in a hybrid and incremental fuzzy learning scheme. This scheme combines unsupervised- and supervised-learning: unsupervised, by fuzzy clustering, to obtain clusters of color groups from training images; and supervised to select groups that represent skin color. At the end of the training phase, aggregation operators are used to provide combinations of selected groups into a skin model. In the detection phase, the learned skin model is used to detect human skin in an efficient way. Experimental results show robust and accurate human skin detection performed by the proposed framework.

Data classification using hybrid GrayART network

A hybrid GrayART network consisting of an unsupervised clustering stage and a supervised adjusting stage is proposed in this paper. In this hybrid learning scheme, the unsupervised clustering stage estimates cluster centers of a given dataset, and the supervised adjusting stage follows to tune the cluster centers for improving the training result. With a proper vigilance threshold value, the overall learning process needs only two epochs, one for each stage. The hybrid network is applied to solve the Iris dataset for illustration. Simulation results demonstrate the classification ability and effectiveness of the proposed network.

Repetitive Learning Control for Time-varying Robotic Systems: A Hybrid Learning Scheme

Repetitive learning control is presented for finite-time-trajectory tracking of uncertain time-varying robotic systems. A hybrid learning scheme is given to cope with the constant and time-varying unknowns in system dynamics, where the time functions are learned in an iterative learning way, without the aid of Taylor expression, while the conventional differential learning method is suggested for estimating the constant ones. It is distinct that the presented repetitive learning control avoids the requirement for initial repositioning at the beginning of each cycle, and the time-varying unknowns are not necessary to be periodic. It is shown that with the adoption of hybrid learning, the boundedness of state variables of the closed-loop system is guaranteed and the tracking error is ensured to converge to zero as iteration increases. The effectiveness of the proposed scheme is demonstrated through numerical simulation.