Parameter Learning Algorithm Research Articles

Data-Driven Fault Prognosis Based on Incomplete Time Slice Dynamic Bayesian Network

Based on a dynamic Bayesian network with an incomplete time slice and a mixture of the Gaussian outputs, a data-driven fault prognosis method for model-unknown processes is proposed in this article. First, according to the requirement of fault prognosis, an incomplete time slice Bayesian network with unknown future observed node is constructed. Moreover, the future states are described by the current measurements and his historic data in the form of conditional probability. Second, according to the completed part of historical data, a parameter-learning algorithm is used to obtain network parameters and the weight coefficients of distribution components. After that, using such weight coefficients as input-output data, the subspace identification method is employed to build a forecasting model which can predict weight coefficients at next sampling time. To achieve fault prognosis, an inference algorithm is developed to predict hidden faults based on the distribution of the measurements directly. Furthermore, the remaining useful life of process is estimated via iterative one-step ahead prognosis. As an example, the proposed method is applied to a continuous stirred tank reactor system. The results demonstrate that the proposed method can efficiently predict and identify the fault, and estimate the remaining useful life of process, even though the measurements are partly missing.

Adaptive Neural Fuzzy Inference System Disturbance Observer-Based Control for Reaching Movement of Musculoskeletal Arm Model

Traditional disturbance observer techniques have limitations for multi-input multi-output-coupled systems, time-varying systems with large parameters, and complex systems which are difficult to obtain accurate model, such as the human musculoskeletal arm system. The neural network has advantages of generalized approximation ability, learning ability, and self-adaptive ability. Therefore, this paper develops an adaptive neural fuzzy inference system disturbance observer-based control to achieve the point-to-point control and the path tracking control of the end point of the musculoskeletal arm model. The adaptivity is improved by adjusting the learning algorithm of neural network parameters and the network structure in real time. The uncertainty of the inverse model of the system is solved by constructing a pseudo-system. The adaptive neural fuzzy inference system is used to identify the inverse model of the pseudo-system and to design a compound controller based on feedback control method. The stability is analyzed by Lyapunov function in detail. Furthermore, internal disturbances are suppressed by the learning algorithm of the adaptive neural fuzzy inference system network while external disturbances may be estimated by the multi-input multi-output disturbance observer at the same time. Simulations are performed to verify the point-to-point control and the path tracking control. Results demonstrate that both of the adaptivity and the accuracy are enhanced so that the system can achieve accurate tracking control of any arbitrary trajectory.

Evaluation for Sortie Generation Capacity of the Carrier Aircraft Based on the Variable Structure RBF Neural Network with the Fast Learning Rate

The neural network has the advantages of self‐learning, self‐adaptation, and fault tolerance. It can establish a qualitative and quantitative evaluation model which is closer to human thought patterns. However, the structure and the convergence rate of the radial basis function (RBF) neural network need to be improved. This paper proposes a new variable structure radial basis function (VS‐RBF) with a fast learning rate, in order to solve the problem of structural optimization design and parameter learning algorithm for the radial basis function neural network. The number of neurons in the hidden layer is adjusted by calculating the output information of neurons in the hidden layer and the multi‐information between neurons in the hidden layer and output layer. This method effectively solves the problem that the RBF neural network structure is too large or too small. The convergence rate of the RBF neural network is improved by using the robust regression algorithm and the fast learning rate algorithm. At the same time, the convergence analysis of the VS‐RBF neural network is given to ensure the stability of the RBF neural network. Compared with other self‐organizing RBF neural networks (self‐organizing RBF (SORBF) and rough RBF neural networks (RS‐RBF)), VS‐RBF has a more compact structure, faster dynamic response speed, and better generalization ability. The simulations of approximating a typical nonlinear function, identifying UCI datasets, and evaluating sortie generation capacity of an carrier aircraft show the effectiveness of VS‐RBF.

Learning to rank in PRISM

Learning parameters associated with propositions is one of the main tasks of probabilistic logic programming (PLP), and learning algorithms for PLP have been primarily developed based on maximum likelihood estimation or the optimization of discriminative criteria. This paper explores yet another innovative approach to parameter learning, learning to rank or rank learning, that has been studied mainly in the field of preference learning. We combine learning to rank with techniques developed in PLP to make the latter applicable to a variety of ranking problems such as information retrieval. We implement our approach in PRISM, a PLP system based on the distribution semantics. It supports many parameter learning algorithms such as the expectation maximization algorithm, the variational Bayes algorithm and an algorithm for Viterbi training efficiently by mapping them onto a single data structure called explanation graph. To ensure the same efficiency for parameter learning by learning to rank as in the current PRISM, we introduce a gradient-based learning method that takes advantage of dynamic programming on the explanation graph. This paper also presents three experimental results. The first one is with synthetic data to check the learning behaviors of the proposed approach. The second one uses a knowledge base (knowledge graph) and apply rank learning to a DistMult model for the task of deciding whether relations over entities exist or not. The last one tackles the problem of parsing by a probabilistic context free grammar whose parameters are learned from a tree corpus by rank learning. These experiments successfully demonstrated the potential and effectiveness of learning to rank in PLP. We plan to release a new version of PRISM augmented with the ability of learning to rank in the near future.

Sequential Monte Carlo Smoothing with Parameter Estimation

We propose two new sequential Monte Carlo (SMC) smoothing methods for general state-space models with unknown parameters. The first is a modification of the particle learning and smoothing (PLS) algorithm of Carvalho, Johannes, Lopes, and Polson (2010), with an adjustment in the backward resampling weights. The second, called Refiltering, is a two-stage method that combines sequential parameter learning and particle smoothing algorithms. We illustrate the methods on three benchmark models using simulated data, and apply them to a stochastic volatility model for daily S&P 500 index returns during the financial crisis. We show that both new methods outperform existing SMC approaches, and that Refiltering is competitive with smoothing approaches based on Markov chain Monte Carlo (MCMC) and Particle MCMC.

Naïve Bayes switching linear dynamical system: A model for dynamic system modelling, classification, and information fusion

The Naïve Bayes Switching Linear Dynamical System (NB-SLDS) is proposed as a novel variant of the switching linear dynamical system (SLDS). The variant models multi-variable systems that undergo regime changes in their dynamics. The model may be applied to identify regime changes or classify systems according to their dynamics. The NB-SLDS provides the means to fuse multiple sequential data sources into a single model. A key feature of the model is that it is able to handle missing and unsynchronised data. Filtering and smoothing algorithms for inference and an expectation maximisation algorithm for parameter learning in the NB-SLDS are presented. The model is demonstrated and compared to the SLDS and hidden Markov model (HMM) in a human action recognition problem.

Dynamic Bayesian Networks for Student Modeling

Intelligent tutoring systems adapt the curriculum to the needs of the individual student. Therefore, an accurate representation and prediction of student knowledge is essential. Bayesian Knowledge Tracing (BKT) is a popular approach for student modeling. The structure of BKT models, however, makes it impossible to represent the hierarchy and relationships between the different skills of a learning domain. Dynamic Bayesian networks (DBN) on the other hand are able to represent multiple skills jointly within one model. In this work, we suggest the use of DBNs for student modeling. We introduce a constrained optimization algorithm for parameter learning of such models. We extensively evaluate and interpret the prediction accuracy of our approach on five large-scale data sets of different learning domains such as mathematics, spelling learning, and physics. We furthermore provide comparisons to previous student modeling approaches and analyze the influence of the different student modeling techniques on instructional policies. We demonstrate that our approach outperforms previous techniques in prediction accuracy on unseen data across all learning domains and yields meaningful instructional policies.

An oblique elliptical basis function network approach for supervised learning applications

We propose a neural network architecture based on the oblique elliptical basis function for supervised learning problems. In classification, a category can be a disconnected or non-convex region involving several overlapping or disjoint sub-regions of the feature space. Other existing supervised learning methods may have the restriction that only allows decision regions to be convex. Our proposed method overcomes this restriction by employing a rotational self-constructing clustering algorithm to decompose the feature space into a collection of sub-regions which can then be combined to make up individual categories. An unseen instance is classified to a certain category if its similarity to the category exceeds a threshold. The whole framework fits in a five-layer network consisting of input, component-similarity, cluster-similarity, aggregation, and output layers. A similar idea also applies to solving regression problems. A parameter learning algorithm based on least squares estimation is used to derive the weights of the underlying network. Our approach can offer some advantages in practicality. Through the incorporation of rotation, data can be clustered more appropriately than by standard elliptical basis functions. Also, our approach is applicable to single-label classification, multi-label classification, as well as regression problems. A number of experiments are conducted to show the effectiveness of the proposed approach.

Nonlinear adaptive tracking control for a small-scale unmanned helicopter using a learning algorithm with the least parameters

This paper puts forward a new nonlinear adaptive controller for a small-scale unmanned helicopter with unknown mass. The controller is developed under the framework of backstepping technique, with the unknown mass estimated by a novel identifier and the internal and external uncertainties approximated by radial basis function neural networks (RBFNNs). The overall closed-loop system, which consists of three parts: longitudinal–lateral subsystem, heave subsystem, and heading subsystem, is proved to be semi-globally uniformly ultimately bounded by the strict Lyapunov stability theory. Furthermore, the proposed method is more practical in actual applications with an improved online learning algorithm of the least parameters used in the RBFNNs. Finally, the effectiveness and the robustness of the proposed strategy are exhibited through two simulations compared with the classic PID method.

Tracking multiple moving objects in images using Markov Chain Monte Carlo

A new Bayesian state and parameter learning algorithm for multiple target tracking (MTT) models with image observations is proposed. Specifically, a Markov chain Monte Carlo algorithm is designed to sample from the posterior distribution of the unknown number of targets, their birth and death times, states and model parameters, which constitutes the complete solution to the tracking problem. The conventional approach is to pre-process the images to extract point observations and then perform tracking. We model the image generation process directly to avoid potential loss of information when extracting point observations. Numerical examples show that our algorithm has improved tracking performance over commonly used techniques, for both synthetic examples and real florescent microscopy data, especially in the case of dim targets with overlapping illuminated regions.

Data Missing Bayesian Network Parameters Learning Optimization Algorithm Based on EM

The paper proposes an optimization algorithm for the parameter learning of missing data Bayesian networks. Expectation Maximization (EM) algorithm is the common parameter learning algorithms. The maximum likelihood estimation (MLE) and maximum a posterity estimation (MAP) of EM are local estimate rather than global estimate and are not easy to achieve the global optimal. So this paper puts forward a point estimate relative error minimum optimization algorithm which based on EM algorithm (EM-MLE-MAP). Applying the algorithm to the fault diagnosis of the rotor vibration Bayesian network, simulations and experiments show that the improved algorithm has good precision in diagnosing vibration fault when the loss ratio less than 3 percent.

Functional-type single-input-rule-modules connected neural fuzzy system for wind speed prediction

Wind is one kind of clean and free renewable energy sources. Wind speed plays a pivotal role in the wind power output. However, due to the random and unstable nature of the wind, accurate prediction of wind speed is a particularly challenging task. This paper presents a novel neural fuzzy method for the hourly wind speed prediction. Firstly, a neural structure is proposed for the functional-type single-input-rule-modules U+0028 FSIRMs U+0029 connected fuzzy inference system U+0028 FIS U+0029 to combine the merits of both the FSIRMs connected FIS and the neural network. Then, in order to achieve both the smallest training errors and the smallest parameters, a least square method based parameter learning algorithm is presented for the proposed FSIRMs connected neural fuzzy system U+0028 FSIRMNFS U+0029. Further, the proposed FSIRMNFS and its parameter learning algorithm are applied to the hourly wind speed prediction. Experiments and comparisons are also made to show the effectiveness and advantages of the proposed approach. Experimental results verified that our study has presented an effective approach for the hourly wind speed prediction. The proposed approach can also be used for the prediction of wind direction, wind power and some other prediction applications in the research field of renewable energy.

Vehicles detection in complex urban traffic scenes using Gaussian mixture model with confidence measurement

Aiming to efficiently resolve the problem that the subtraction background model is easily contaminated by slow-moving or temporarily stopped vehicles, the Gaussian mixture model with confidence measurement (GMMCM) is proposed for vehicle detection in complex urban traffic scenes. According to the current traffic state, each pixel of background model is set a CM. Whether to update the background model and the corresponding adaptive learning rate depends on if the current pixel point is in confidence period. Using the real-world urban traffic videos, the first experiments are conducted by GMMCM, compared with three commonly used models including GMM, self-adaptive GMM (SAGMM) and local parameter learning algorithm for the GMM (LPLGMM). The first experimental results show that GMMCM excels GMM, SAGMM and LPLGMM in keeping the background model being unpolluted from slow-moving or temporarily stopped vehicles. The second experiments are conducted by GMMCM, compared with visual background extractor, sigma-delta with CM, SAGMM, LPLGMM and GMM. The average recalls of six methods are 0.899, 0.753, 0.679, 0.420, 0.447 and 0.205, and the average F-measures of six methods are 0.636, 0.612, 0.592, 0.373, 0.330 and 0.179, respectively. All experimental results show the effectiveness of the proposed GMMCM in vehicles detection of complex urban traffic scenes.

A Method for Predicting Wikipedia Editors' Editing Interest

Recruiting or recommending appropriate potential Wikipedia editors to edit a specific Wikipedia entry (or article) can play an important role in improving the quality and credibility of Wikipedia. According to empirical observations based on a small-scale dataset collected from Wikipedia, this paper proposes an Interest Prediction Factor Graph (IPFG) model, which is characterized by editor's social properties, hyperlinks between Wikipedia entries, the categories of an entry and other important features, to predict an editor's editing interest in types of Wikipedia entries. Furthermore, the paper suggests a parameter learning algorithm based on the gradient descent algorithm and the Loopy Sum-Product algorithm for factor graphs. An experiment on a Wikipedia dataset (with different frequencies of data collection) shows that the average prediction accuracy (F1 score) of the IPFG model for data collected quarterly could be up to 0.875, which is approximately 0.49 higher than that of a collaborative filtering approach. In addition, the paper analyzes how incomplete social properties and editing bursts affect the prediction accuracy of the IPFG model. The authors' results can provide insight into effective Wikipedia article tossing and can improve the quality of special entries that belong to specific categories by means of collective collaboration.

Particle filtering and marginalization for parameter identification in structural systems

In structural health monitoring, one wishes to use available measurements from a structure to assess structural condition, localize damage if present, and quantify remaining life. Nonlinear system identification methods are considered that use a parametric, nonlinear, physics-based model of the system, cast in the state-space framework. Various nonlinear filters and parameter learning algorithms can then be used to recover the parameters and quantify uncertainty. This paper focuses on the particle filter (PF), which shows the advantage of not assuming Gaussianity of the posterior densities. However, the PF is known to behave poorly in high dimensional spaces, especially when static parameters are added to the state vector. To improve the efficiency of the PF, the concept of Rao–Blackwellisation is applied, that is, we use conditional linearities present in the equations to marginalize out some of the states/parameters and infer their conditional posterior pdf using the Kalman filtering equations. This method has been studied extensively in the particle filtering literature, and we start our discussion by improving upon and applying two well-known algorithms on a benchmark structural system. Then, noticing that in structural systems, high nonlinearities are often localized while the remaining equations are bilinear in the states and parameters, a novel algorithm is proposed, which combines this marginalization approach with a second-order extended Kalman filter. This new approach enables us to marginalize out all the states/parameters, which do not contribute to any high nonlinearity in the equations and, thus, improve identification of the unknown parameters. Copyright © 2016 John Wiley & Sons, Ltd.

An Automatic Process Monitoring Method Using Recurrence Plot in Progressive Stamping Processes

In progressive stamping processes, condition monitoring based on tonnage signals is of great practical significance. One typical fault in progressive stamping processes is a missing part in one of the die stations due to malfunction of part transfer in the press. One challenging question is how to detect the fault due to the missing part in certain die stations as such a fault often results in die or press damage, but only provides a small change in the tonnage signals. To address this issue, this article proposes a novel automatic process monitoring method using the recurrence plot (RP) method. Along with the developed method, we also provide a detailed interpretation of the representative patterns in the recurrence plot. Then, the corresponding relationship between the RPs and the tonnage signals under different process conditions is fully investigated. To differentiate the tonnage signals under normal and faulty conditions, we adopt the recurrence quantification analysis (RQA) to characterize the critical patterns in the RPs. A parameter learning algorithm is developed to set up the appropriate parameter of the RP method for progressive stamping processes. A real case study is provided to validate our approach, and the results are compared with the existing literature to demonstrate the outperformance of this proposed monitoring method.

Online parameter learning for data-driven crowd simulation and content generation

We present an online parameter learning algorithm for data-driven crowd simulation and crowd content generation. Our formulation is based on incrementally learning pedestrian motion models and behaviors from crowd videos. We combine the learned crowd-simulation model with an online tracker to compute accurate, smooth pedestrian trajectories. We refine the motion model using an optimization technique to estimate the agents׳ simulation parameters. We also use an adaptive-particle filtering scheme for improved computational efficiency. We highlight the benefits of our approach for improved data-driven crowd simulation, including crowd replication, augmented crowds and merging the behavior of pedestrians from multiple videos. We highlight our algorithm׳s performance in various test scenarios containing tens of human-like agents and evaluate it using standard metrics.

Lifted generative learning of Markov logic networks

Markov logic networks (MLNs) are a well-known statistical relational learning formalism that combines Markov networks with first-order logic. MLNs attach weights to formulas in first-order logic. Learning MLNs from data is a challenging task as it requires searching through the huge space of possible theories. Additionally, evaluating a theory's likelihood requires learning the weight of all formulas in the theory. This in turn requires performing probabilistic inference, which, in general, is intractable in MLNs. Lifted inference speeds up probabilistic inference by exploiting symmetries in a model. We explore how to use lifted inference when learning MLNs. Specifically, we investigate generative learning where the goal is to maximize the likelihood of the model given the data. First, we provide a generic algorithm for learning maximum likelihood weights that works with any exact lifted inference approach. In contrast, most existing approaches optimize approximate measures such as the pseudo-likelihood. Second, we provide a concrete parameter learning algorithm based on first-order knowledge compilation. Third, we propose a structure learning algorithm that learns liftable MLNs, which is the first MLN structure learning algorithm that exactly optimizes the likelihood of the model. Finally, we perform an empirical evaluation on three real-world datasets. Our parameter learning algorithm results in more accurate models than several competing approximate approaches. It learns more accurate models in terms of test-set log-likelihood as well as prediction tasks. Furthermore, our tractable learner outperforms intractable models on prediction tasks suggesting that liftable models are a powerful hypothesis space, which may be sufficient for many standard learning problems.

Parameter optimization in differential geometry based solvation models.

Differential geometry (DG) based solvation models are a new class of variational implicit solvent approaches that are able to avoid unphysical solvent-solute boundary definitions and associated geometric singularities, and dynamically couple polar and non-polar interactions in a self-consistent framework. Our earlier study indicates that DG based non-polar solvation model outperforms other methods in non-polar solvation energy predictions. However, the DG based full solvation model has not shown its superiority in solvation analysis, due to its difficulty in parametrization, which must ensure the stability of the solution of strongly coupled nonlinear Laplace-Beltrami and Poisson-Boltzmann equations. In this work, we introduce new parameter learning algorithms based on perturbation and convex optimization theories to stabilize the numerical solution and thus achieve an optimal parametrization of the DG based solvation models. An interesting feature of the present DG based solvation model is that it provides accurate solvation free energy predictions for both polar and non-polar molecules in a unified formulation. Extensive numerical experiment demonstrates that the present DG based solvation model delivers some of the most accurate predictions of the solvation free energies for a large number of molecules.

A Novel Interactively Recurrent Self-Evolving Fuzzy CMAC and Its Classification Applications

In this paper, an Interactively Recurrent Self-evolving Fuzzy Cerebellar Model Articulation Controller (IRSFCMAC) model is developed for solving classification problems. The proposed IRSFCMAC classifier consists of internal feedback and external loops, which are generated by the hypercube cell firing strength to itself and other hypercube cells. The learning process of the IRSFCMAC gets started with an empty hypercube base, and then all of hypercube cells are generated and learned online via structure and parameter learning, respectively. The structure learning algorithm is based on the degree measure to determine the number of hypercube cells. The parameter learning algorithm, based on the gradient descent method, adjusts the shapes of the membership functions and the corresponding fuzzy weights of the IRSFCMAC. Finally, the proposed IRSFCMAC model is tested by four benchmark classification problems. Experimental results show that the proposed IRSFCMAC model has superior performance than traditional FCMAC and other models.