Input-output Space Research Articles

A Hierarchical Procedure for the Synthesis of ANFIS Networks

Adaptive neurofuzzy inference systems (ANFIS) represent an efficient technique for the solution of function approximation problems. When numerical samples are available in this regard, the synthesis of ANFIS networks can be carried out exploiting clustering algorithms. Starting from a hyperplane clustering synthesis in the joint input-output space, a computationally efficient optimization of ANFIS networks is proposed in this paper. It is based on a hierarchical constructive procedure, by which the number of rules is progressively increased and the optimal one is automatically determined on the basis of learning theory in order to maximize the generalization capability of the resulting ANFIS network. Extensive computer simulations prove the validity of the proposed algorithm and show a favorable comparison with other well-established techniques.

Three new fuzzy neural networks learning algorithms based on clustering, training error and genetic algorithm

Three new learning algorithms for Takagi-Sugeno-Kang fuzzy system based on training error and genetic algorithm are proposed. The first two algorithms are consisted of two phases. In the first phase, the initial structure of neuro-fuzzy network is created by estimating the optimum points of training data in input-output space using KNN (for the first algorithm) and Mean-Shift methods (for the second algorithm) and keeps adding new neurons based on an error-based algorithm. Then in the second phase, redundant neurons are recognized and removed using a genetic algorithm. The third algorithm then builds the network in one phase using a modified version of error algorithm used in the first two methods. The KNN method is shown to be invariant to parameter K in KNN algorithm and in two simulated examples outperforms other neuro-fuzzy approaches in both performance and network compactness.

Fuzzy Model Identification With Enhanced Validity Criterion for Mechanical System Design

Model identification for machine system design, design optimization, and manufacturing planning is an important method that has high prediction accuracy and could become an essential stage in practical applications. In this paper, an effective fuzzy model identification algorithm for mechanical system design is developed. First, a fuzzy c-regression model clustering algorithm, in which hyperplane-shaped cluster representatives are utilized to provide a mathematical tool to partition the input–output space reasonably, is introduced. Then, an enhanced cluster validity criterion, in which the structural information hidden in the clusters can be reflected in the index, is proposed to choose the optimal number of clusters. In the proposed architecture, an improved Takagi–Sugeno fuzzy model is proposed to describe the system. Two illustrative examples under various conditions are provided, and their performances are indicated in comparison with other published works. In comparison to these fuzzy works, the proposed fuzzy model identification requires fewer fuzzy rules and a shorter tuning time.

A new method for hybrid-fuzzy identification

AbstractIn this paper a new identification method for non-linear hybrid systems that have mixed continuous and discrete states by using fuzzy clustering and principal component analysis is described. The method first determines the hybrid characteristic of the system inspired by an inverse form of the merge method for clusters, which makes it possible to identify the unknown switching points of a process based on just input-output data. Using the switching points, a hard partition of the input-output space is obtained. Then, we propose to use Takagi-Sugeno (TS) fuzzy models with Gaussian MFs as sub-models for each partition. Thus, the overall model is hybrid-fuzzy and will include explicitly the hybrid behavior of the system (the detected switching points) by means of binary MFs, and in each partition all the other non-linearities by means of TS sub-models. An illustrative experiment on a hybrid-tank system is conducted to present the benefits of the proposed approach.

Human Motion Tracking by Temporal-Spatial Local Gaussian Process Experts

Human pose estimation via motion tracking systems can be considered as a regression problem within a discriminative framework. It is always a challenging task to model the mapping from observation space to state space because of the high-dimensional characteristic in the multimodal conditional distribution. In order to build the mapping, existing techniques usually involve a large set of training samples in the learning process which are limited in their capability to deal with multimodality. We propose, in this work, a novel online sparse Gaussian Process (GP) regression model to recover 3-D human motion in monocular videos. Particularly, we investigate the fact that for a given test input, its output is mainly determined by the training samples potentially residing in its local neighborhood and defined in the unified input-output space. This leads to a local mixture GP experts system composed of different local GP experts, each of which dominates a mapping behavior with the specific covariance function adapting to a local region. To handle the multimodality, we combine both temporal and spatial information therefore to obtain two categories of local experts. The temporal and spatial experts are integrated into a seamless hybrid system, which is automatically self-initialized and robust for visual tracking of nonlinear human motion. Learning and inference are extremely efficient as all the local experts are defined online within very small neighborhoods. Extensive experiments on two real-world databases, HumanEva and PEAR, demonstrate the effectiveness of our proposed model, which significantly improve the performance of existing models.

How Synaptic Release Probability Shapes Neuronal Transmission: Information-Theoretic Analysis in a Cerebellar Granule Cell

A nerve cell receives multiple inputs from upstream neurons by way of its synapses. Neuron processing functions are thus influenced by changes in the biophysical properties of the synapse, such as long-term potentiation (LTP) or depression (LTD). This observation has opened new perspectives on the biophysical basis of learning and memory, but its quantitative impact on the information transmission of a neuron remains partially elucidated. One major obstacle is the high dimensionality of the neuronal input-output space, which makes it unfeasible to perform a thorough computational analysis of a neuron with multiple synaptic inputs. In this work, information theory was employed to characterize the information transmission of a cerebellar granule cell over a region of its excitatory input space following synaptic changes. Granule cells have a small dendritic tree (on average, they receive only four mossy fiber afferents), which greatly bounds the input combinatorial space, reducing the complexity of information-theoretic calculations. Numerical simulations and LTP experiments quantified how changes in neurotransmitter release probability (p) modulated information transmission of a cerebellar granule cell. Numerical simulations showed that p shaped the neurotransmission landscape in unexpected ways. As p increased, the optimality of the information transmission of most stimuli did not increase strictly monotonically; instead it reached a plateau at intermediate p levels. Furthermore, our results showed that the spatiotemporal characteristics of the inputs determine the effect of p on neurotransmission, thus permitting the selection of distinctive preferred stimuli for different p values. These selective mechanisms may have important consequences on the encoding of cerebellar mossy fiber inputs and the plasticity and computation at the next circuit stage, including the parallel fiber-Purkinje cell synapses.

A Computationally Efficient, Consistent Bootstrap for Inference with Non-parametric DEA Estimators

We develop a tractable, consistent bootstrap algorithm for inference about Farrell---Debreu efficiency scores estimated by non-parametric data envelopment analysis (DEA) methods. The algorithm allows for very general situations where the distribution of the inefficiencies in the input-output space may be heterogeneous. Computational efficiency and tractability are achieved by avoiding the complex double-smoothing procedure in the algorithm proposed by Kneip et al. (Econometric Theory 24:1663---1697, 2008). In particular, we avoid technical difficulties in the earlier algorithm associated with smoothed estimates of a density with unknown, nonlinear, multivariate bounded support requiring complicated reflection methods. The new procedure described here is relatively simple and easy to implement: for particular values of a pair of smoothing parameters, the computational complexity is the same as the (inconsistent) naive bootstrap. The resulting computational speed allows the bootstrap to be iterated in order to optimize the smoothing parameters. From a practical viewpoint, only standard packages for computing DEA efficiency estimates, i.e., solving linear problems, are required for implementation. The performance of the method in finite samples is illustrated through some simulated examples.

A BIAS-CORRECTED NONPARAMETRIC ENVELOPMENT ESTIMATOR OF FRONTIERS

In efficiency analysis, the production frontier is defined as the set of the most efficient alternatives among all possible combinations in the input-output space. The nonparametric envelopment estimators rely on the assumption that all the observations fall on the same side of the frontier. The free disposal hull (FDH) estimator of the attainable set is the smallest free disposal set covering all the observations. By construction, the FDH estimator is an inward-biased estimator of the frontier.The univariate extreme values representation of the FDH allows us to derive a bias-corrected estimator for the frontier. The presentation is based on a probabilistic formulation where the input-output pairs are realizations of independent random variables drawn from a joint distribution whose support is the production set. The bias-corrected estimator shares the asymptotic properties of the FDH estimator. But in finite samples, Monte Carlo experiments indicate that our bias-corrected estimator reduces significantly not only the bias of the FDH estimator but also its mean squared error, with no computational cost. The method is also illustrated with a real data example. A comparison with the parametric stochastic frontier indicates that the parametric approach can easily fail under wrong specification of the model.

Design of cluster-wise optimal fuzzy logic controllers to model input-output relationships of some manufacturing processes

In the present study, fuzzy logic (FL)-based approaches have been developed to determine the input-output relationships of some manufacturing processes, which may be non-linear in nature. Moreover, the degree of non-linearity may not be the same over the entire range of the variables. The input-output space has been clustered based on the similarity of the data points and cluster-wise linear regressions have been carried out. Takagi and Sugeno's approach of fuzzy logic controller (FLC) has been implemented cluster-wise using the pre-determined regression equations. A genetic algorithm (GA) has been utilised to optimise both cluster-properties as well as knowledge base of the FLCs developed based on two types of clustering algorithm, namely entropy-based approach and fuzzy C-means algorithm. The performances of above two FLCs have been compared in the present work.

Characterisation of magnetic materials: determination of parameters of Preisach models of hysteresis

Preisach model is a well known general model of hysteresis behaviour in magnetic materials. A method for identification of the parameters of the Classical Scalar Preisach Model (CSPM) of hysteresis in magnetic materials is presented in this article. This new method has a major advantage over other existing methods, as it requires only the measurement of energy dissipation per cycle, while in other methods, the material has to be driven along particular trajectories in the input-output space. The measurements of the dissipated energy can be performed accurately, even in the presence of significant noise, by the use of a special lock-in measurement technique, also presented in this article. The characterisation method has been tested numerically, on a surrogate system, and its performance has been analysed and quantified, for different levels of measurement noise.

Online optimal fuzzy identification using improved nearest-neighbor clustering method

To investigate online self-adaptive identification,a method based on improved nearest-neighborhood clustering with optimal fuzzy logic system was proposed.The advanced clustering way considered the input-output space,adjusted the centre point by calculating the means of the whole clustering space and judging the rule's deletion by potential function,and combined with optimal fuzzy logic system which proposed by WANG Li-xin.High identification speed and precision were achieved.The simulation result illustrates the effectiveness of the proposed method.

Exploiting maximum energy from variable speed wind power generation systems by using an adaptive Takagi–Sugeno–Kang fuzzy model

Nowadays, incentives and financing options for developing renewable energy facilities and the new development in variable speed wind technology make wind energy a competitive source if compared with conventional generation ones. In order to improve the effectiveness of variable speed wind systems, adaptive control systems able to cope with time variances of the system under control are necessary. On these basis, a data driven designing methodology for TSK fuzzy models design is presented in this paper. The methodology, on the basis of given input-output numerical data, generates the ‘best’ TSK fuzzy model able to estimate with high accuracy the maximum extractable power from a variable speed wind turbine. The design methodology is based on fuzzy clustering methods for partitioning the input-output space combined with genetic algorithms (GA), and recursive least-squares (LS) optimization methods for model parameter adaptation.

Designing an Adaptive Fuzzy Controller for Maximum Wind Energy Extraction

The wind power production spreading, also aided by the transition from constant to variable speed operation, involves the development of efficient control systems to improve the effectiveness of power production systems. This paper presents a data-driven design methodology able to generate a Takagi-Sugeno-Kang (TSK) fuzzy model for maximum energy extraction from variable speed wind turbines. In order to obtain the TSK model, fuzzy clustering methods for partitioning the input-output space, combined with genetic algorithms, and recursive least-squares optimization methods for model parameter adaptation are used. The implemented TSK fuzzy model, as confirmed by some simulation results on a doubly fed induction generator connected to a power system, exhibits high speed of computation, low memory occupancy, fault tolerance, and learning capability.

A fuzzy controller for maximum energy extraction from variable speed wind power generation systems

The wind power production spreading, also aided by the transition from constant to variable speed operation, involves the development of efficient control systems to improve the effectiveness of wind systems. This paper presents a data-driven design methodology able to generate a Takagi–Sugeno–Kang (TSK) fuzzy model for maximum energy extraction from variable speed wind turbines. In order to obtain the TSK model, fuzzy clustering methods for partitioning the input–output space, combined with genetic algorithms (GA), and recursive least-squares (LS) optimization methods for model parameter adaptation are used. The implemented TSK fuzzy model, as confirmed by some simulation results on a doubly fed induction generator connected to a power system, exhibits high speed of computation, low memory occupancy, fault tolerance and learning capability.

A self-organizing fuzzy neural network and its applications to function approximation and forecast modeling

To solve the problem of conventional input–output space partitioning, a new learning algorithm for creating self-organizing fuzzy neural networks (SOFNN) is proposed, which automates structure and parameter identification simultaneously based on input-target samples. First, a self-organizing clustering approach is used to establish the structure of the network and obtain the initial values of its parameters, then a supervised learning method to optimize these parameters. Two specific implementations of the algorithm, including function approximation and forecast modeling of the wastewater treatment system, are developed, comprehensive comparisons are made with other approaches in both of the examples. Simulation studies demonstrate the presented algorithm is superior in terms of compact structure and learning efficiency.

Comparison of different strategies of utilizing fuzzy clustering in structure identification

Fuzzy systems approximate highly nonlinear systems by means of fuzzy “if–then” rules. In the literature, various algorithms are proposed for mining. These algorithms commonly utilize fuzzy clustering in structure identification. Basically, there are three different approaches in which one can utilize fuzzy clustering; the first one is based on input space clustering, the second one considers clustering realized in the output space, while the third one is concerned with clustering realized in the combined input–output space. In this study, we analyze these three approaches. We discuss each of the algorithms in great detail and offer a thorough comparative analysis. Finally, we compare the performances of these algorithms in a medical diagnosis classification problem, namely Aachen Aphasia Test. The experiment and the results provide a valuable insight about the merits and the shortcomings of these three clustering approaches.

Multi-Output Regularized Feature Projection

Dimensionality reduction by feature projection is widely used in pattern recognition, information retrieval, and statistics. When there are some outputs available (e.g., regression values or classification results), it is often beneficial to consider supervised projection, which is based not only on the inputs, but also on the target values. While this applies to a single-output setting, we are more interested in applications with multiple outputs, where several tasks need to be learned simultaneously. In this paper, we introduce a novel projection approach called multi-output regularized feature projection (MORP), which preserves the information of input features and, meanwhile, captures the correlations between inputs/outputs and (if applicable) between multiple outputs. This is done by introducing a latent variable model on the joint input-output space and minimizing the reconstruction errors for both inputs and outputs. It turns out that the mappings can be found by solving a generalized eigenvalue problem and are ready to extend to nonlinear mappings. Prediction accuracy can be greatly improved by using the new features since the structure of outputs is explored. We validate our approach in two applications. In the first setting, we predict users' preferences for a set of paintings. The second is concerned with image and text categorization where each image (or document) may belong to multiple categories. The proposed algorithm produces very encouraging results in both settings

Fuzzy identification of a greenhouse

Nonlinear dynamic systems’ modelling is difficult. The solutions proposed are generally based on the linearization of the process behaviour around the operating points. Other researches were carried out on this technique of linearization not only around the operating points, but also in all the input–output space allowing the obtaining of several local linear models. The major difficulty with this technique is the model transition. Fuzzy logic makes it possible to solve this problem thanks to its properties of universal approximator. Indeed, many techniques of modelling and identification based on fuzzy logic are often used for this type of systems. Among these techniques, we find those based on the fuzzy clustering technique. The proposed method uses in a first stage the fuzzy clustering technique to determine both the premises and the consequent parameters of the fuzzy Takagi–Sugeno rules. In a second stage these consequent parameters are adapted by using the recursive weighted least squares algorithm with a forgetting factor. We will try in this paper to apply this method to model the air temperature and humidity inside the greenhouse.

Algorithm of dynamic plant identification with regard to a priori information

Consideration was given to an identification algorithm taking into account the a priori information about the parameters. The original experimental data from the input-output space are transformed in the space of parameter estimates where the set of estimates satisfying the a priori constraints is specified. The parameter estimates are constructed on this set using the probabilistic criterion. An example illustrating effectiveness of the proposed procedure was presented.

Adaptive noise cancellation using enhanced dynamic fuzzy neural networks

In this paper, a novel adaptive noise cancellation algorithm using enhanced dynamic fuzzy neural networks (EDFNNs) is described. In the proposed algorithm, termed EDFNN learning algorithm, the number of radial basis function (RBF) neurons (fuzzy rules) and input-output space clustering is adaptively determined. Furthermore, the structure of the system and the parameters of the corresponding RBF units are trained online automatically and relatively rapid adaptation is attained. By virtue of the self-organizing mapping (SOM) and the recursive least square error (RLSE) estimator techniques, the proposed algorithm is suitable for real-time applications. Results of simulation studies using different noise sources and noise passage dynamics show that superior performance can be achieved.