Orthogonal Least-squares Method Research Articles

Bi-static high resolution imaging for ship targets with sparse apertures and its application in InISAR

ABSTRACTIn this letter, we focus on the high-resolution imaging of ship targets under sparse aperture condition and its application in three-dimensional (3D) imaging. A bi-static high-resolution algorithm dealing with the sparse echo condition is proposed, which consists of the following processing steps: (1) A side-lobe apodization–multiple orthogonal least-square method is proposed to recover the complete signal and reduce the influence of side-lobe at the same time, which enhances the precision of phase extraction in interferometric inverse synthetic aperture radar (InISAR) imaging; (2) The bi-static configuration is used to accentuate the relative motion between the target and radar, such that the requirement for long coherence processing time can be avoided and (3) The 3D InISAR technique is employed to obtain the 3D images of the ship targets. Experiments with simulated and real data are utilized to demonstrate the effectiveness of the proposed method.

A novel identification method for Takagi–Sugeno fuzzy model

Based on the Xie–Beni index and an improved particle swarm optimization algorithm, a novel identification method for the Takagi–Sugeno fuzzy model is proposed in this paper. Firstly, Xie–Beni indices with a fuzzy c-means clustering algorithm are adopted to find the rule number of the Takagi–Sugeno fuzzy model. By utilizing the particle swarm optimization algorithm, the initial membership function and the consequent parameters of the fuzzy model are obtained. In addition, through an improved fuzzy c-regression model and orthogonal least-square method, the premise structure and consequent parameters can be obtained to establish the Takagi–Sugeno fuzzy model. Some well-known models are used to demonstrate that the proposed method outperforms some existing methods.

A Novel Fuzzy Identification Method Based on Ant Colony Optimization Algorithm

In this paper, an identification problem for nonlinear models is explored and a novel fuzzy identification method based on the ant colony optimization algorithm is proposed. First, a modified cluster validity criterion with a fuzzy $c$ -regression model is adopted to find appropriate rule numbers of the Takagi-Sugeno fuzzy model. Then, the ant colony optimization algorithm is adopted and the sifted initial membership function and the consequent parameters of the fuzzy model are obtained. Through an improved fuzzy $c$ -regression model and the orthogonal least-squares method, the premise structure and the consequent parameters can be obtained to establish the Takagi-Sugeno fuzzy model. Some examples are illustrated to show that the proposed method provides better approximation results and robustness than those obtained using some of the existing methods.

A new discrete-continuous algorithm for radial basis function networks construction.

The construction of a radial basis function (RBF) network involves the determination of the model size, hidden nodes, and output weights. Least squares-based subset selection methods can determine a RBF model size and its parameters simultaneously. Although these methods are robust, they may not achieve optimal results. Alternatively, gradient methods are widely used to optimize all the parameters. The drawback is that most algorithms may converge slowly as they treat hidden nodes and output weights separately and ignore their correlations. In this paper, a new discrete-continuous algorithm is proposed for the construction of a RBF model. First, the orthogonal least squares (OLS)-based forward stepwise selection constructs an initial model by selecting model terms one by one from a candidate term pool. Then a new Levenberg-Marquardt (LM)-based parameter optimization is proposed to further optimize the hidden nodes and output weights in the continuous space. To speed up the convergence, the proposed parameter optimization method considers the correlation between the hidden nodes and output weights, which is achieved by translating the output weights to dependent parameters using the OLS method. The correlation is also used by the previously proposed continuous forward algorithm (CFA). However, unlike the CFA, the new method optimizes all the parameters simultaneously. In addition, an equivalent recursive sum of squared error is derived to reduce the computation demanding for the first derivatives used in the LM method. Computational complexity is given to confirm the new method is much more computationally efficient than the CFA. Different numerical examples are presented to illustrate the effectiveness of the proposed method. Further, Friedman statistical tests on 13 classification problems are performed, and the results demonstrate that RBF networks built by the new method are very competitive in comparison with some popular classifiers.

Interpretable support vector regression

This paper deals with transforming Support vector regression (SVR) models into fuzzy systems (FIS). It is highlighted that trained support vector based models can be used for the construction of fuzzy rule-based regression models. However, the transformed support vector model does not automatically result in an interpretable fuzzy model. Training of a support vector model results a complex rule base, where the number of rules are approximately 40-60% of the number of the training data, therefore reduction of the support vector model initialized fuzzy model is an essential task. For this purpose, a three-step reduction algorithm is used based on the combination of previously published model reduction techniques, namely the reduced set method to decrease number of kernel functions, then after the reduced support vector model is transformed into fuzzy rule base similarity measure based merging and orthogonal least-squares methods are utilized. The proposed approach is applied for nonlinear system identification, the identification of a Hammerstein system is used to demonstrate accuracy of the technique with fulfilling the criteria of interpretability.

Hybrid learning for interval type-2 fuzzy logic systems based on orthogonal least-squares and back-propagation methods

This paper presents a novel learning methodology based on a hybrid algorithm for interval type-2 fuzzy logic systems. Since only the back-propagation method has been proposed in the literature for the tuning of both the antecedent and the consequent parameters of type-2 fuzzy logic systems, a hybrid learning algorithm has been developed. The hybrid method uses a recursive orthogonal least-squares method for tuning the consequent parameters and the back-propagation method for tuning the antecedent parameters. Systems were tested for three types of inputs: (a) interval singleton, (b) interval type-1 non-singleton, and (c) interval type-2 non-singleton. Experiments were carried out on the application of hybrid interval type-2 fuzzy logic systems for prediction of the scale breaker entry temperature in a real hot strip mill for three different types of coil. The results proved the feasibility of the systems developed here for scale breaker entry temperature prediction. Comparison with type-1 fuzzy logic systems shows that hybrid learning interval type-2 fuzzy logic systems provide improved performance under the conditions tested.

Adaptive fuzzy wavelet network control design for nonlinear systems

This paper presents a new adaptive fuzzy wavelet network controller (A-FWNC) for control of nonlinear affine systems, inspired by the theory of multiresolution analysis (MRA) of wavelet transforms and fuzzy concepts. The proposed adaptive gain controller, which results from the direct adaptive approach, has the ability to tune the adaptation parameter in the THEN-part of each fuzzy rule during real-time operation. Each fuzzy rule corresponds to a sub-wavelet neural network (sub-WNN) and one adaptation parameter. Each sub-WNN consists of wavelets with a specified dilation value. The degree of contribution of each sub-WNN can be controlled flexibly. Orthogonal least square (OLS) method is used to determine the number of fuzzy rules and to purify the wavelets for each sub-WNN. Since the efficient procedure of selecting wavelets used in the OLS method is not very sensitive to the input dimension, the dimension of the approximated function does not cause the bottleneck for constructing FWN. FWN is constructed based on the training data set of the nominal system and the constructed fuzzy rules can be adjusted by learning the translation parameters of the selected wavelets and also determining the shape of membership functions. Then, the constructed adaptive FWN controller is employed, such that the feedback linearization control input can be best approximated and the closed-loop stability is guaranteed. The performance of the proposed A-FWNC is illustrated by applying a second-order nonlinear inverted pendulum system and compared with previously published methods. Simulation results indicate the remarkable capabilities of the proposed control algorithm. It is worth noting that the proposed controller significantly improves the transient response characteristics and the number of fuzzy rules and on-line adjustable parameters are reduced.

Interpetable Support Vector Machines in Regression and Classification – Application in Process Engineering

Tools from the armoury of soft computing have been in focus of researches recently, since soft computing techniques are used for fault detection (classification techniques), forecasting of time-series data, inference, hypothesis testing, and modelling of causal relationships (regression techniques) in process engineering. These techniques solve two cardinal problems: learning from experimental data by neural networks and support vector based techniques and embedding existing structured human knowledge into fuzzy models. Support vector based models are one of the most commonly used soft computing techniques. Support vector based models are strong in feature selection and to achieve robust models and fuzzy logic helps to improve the interpretability of models. This paper deals with combining these existing soft computing techniques to get interpretable but accurate models for industrial purposes. The paper describes that trained support vector based models can be used for the construction of fuzzy rule-based classifier or regression models. However, the transformed support vector model does not automatically result in an interpretable fuzzy model because the support vector model results in a complex rulebase, where the number of rules is approximately 40-60% of the number of the training data. Hence, reduction of the support model-initialized fuzzy model is an essential task. For this purpose, a three-step reduction algorithm is used on the combination of previously published model reduction techniques. In the first step, the identification of the SV model is followed by the application of the Reduced Set method to decrease the number of kernel functions. The reduced SV model is then transformed into a fuzzy rule-based model. The interpretability of a fuzzy model highly depends on the distribution of the membership functions. Hence, the second reduction step is achieved by merging similar fuzzy sets based on a similarity measure. Finally, in the third step, an orthogonal least-squares method is used to reduce the number of rules and re-estimate the consequent parameters of the fuzzy rule-based model. The proposed approach is applied for classification problems and applied for Hammerstein system identification to illustrate the effectiveness of the technique.

A recurrent fuzzy filter for the analysis of lung sounds

This paper presents a recurrent fuzzy-neural filter for real-time separation of discontinuous adventitious sounds from vesicular sounds. The filter uses two dynamic neuron-based fuzzy neural networks to perform the task of separation. The networks are generated by the dynamic orthogonal least-squares method and are applied to all kinds of lung sounds. Extensive experimental results are given and a performance comparison with a series of other models is conducted, underlining the effectiveness of the proposed filter and its superior performance over its competing rivals.

Identification of immune models for fault detection

In this paper, a novel approach to model-based fault detection for non-linear systems is presented. An immune model of the system is used for the generation of residual. The orthogonal least-squares method is implemented to select the significant receptor vectors of the immune model. After the model identification, the filtered residual scheme and the fault alarm concentration are applied for the fault detection. To verify and demonstrate the performance of the proposed methodology, a simulation example on a two-link robot was studied. The results show the effectiveness and robustness in both system identification and fault detection.

Identification of immune models for fault detection

In this paper, a novel approach to model-based fault detection for non-linear systems is presented. An immune model of the system is used for the generation of residual. The orthogonal least-squares method is implemented to select the significant receptor vectors of the immune model. After the model identification, the filtered residual scheme and the fault alarm concentration are applied for the fault detection. To verify and demonstrate the performance of the proposed methodology, a simulation example on a two-link robot was studied. The results show the effectiveness and robustness in both system identification and fault detection.

An orthogonal least-squares method for recurrent fuzzy-neural modeling

This paper presents an orthogonal least-squares (OLS)-based modeling method, named dynamic OLS (D-OLS), for generating recurrent fuzzy models. A dynamic-neuron-based fuzzy neural network is proposed, comprising generalized Takagi–Sugeno–Kang (TSK) fuzzy rules, whose consequent parts consist of dynamic neurons with local output feedback. From an arbitrarily large set of candidate dynamic neurons, the D-OLS method selects automatically the most important ones. Thus, in the resulting model, the consequent part of each fuzzy rule contains dynamic neurons with different time delays. The proposed dynamic model, equipped with the learning algorithm, is applied to two temporal problems, where the effectiveness of the suggested method as well as the advantages of the resulting dynamic model are demonstrated.

Rule base reduction: some comments on the use of orthogonal transforms

Comments on recent publications about the use of orthogonal transforms to order and select rules in a fuzzy rule base. The techniques are well-known from linear algebra, and we comment on their usefulness in fuzzy modeling. The application of rank-revealing methods based on singular value decomposition (SVD) to rule reduction gives rather conservative results. They are essentially subset selection methods, and we show that such methods do not produce an "importance ordering", contrary to what has been stated in the literature. The orthogonal least-squares (OLS) method, which evaluates the contribution of the rules to the output, is more attractive for systems modeling. However, it has been shown to sometimes assign high importance to rules that are correlated in the premise. This hampers the generalization capabilities of the resulting model. We discuss the performance of rank-revealing reduction methods and advocate the use of a less complex method based on the pivoted QR decomposition. Further, we show how detection of redundant rules can be introduced in OLS by a simple extension of the algorithm. The methods are applied to a problem known from the literature and compared to results reported by other researchers.

A synthesis technique of time-domain interconnect models by MIMO type of selective orthogonal least-square method

This paper describes an efficient synthesis technique of time-domain models for interconnects characterized by sampled data. In this method, poles are derived from the sampled data in the frequency domain. Next, the dominant poles and the residues are obtained efficiently by multiinput-multioutput type of selective orthogonal least-square method. Furthermore, time-domain models are derived from the frequency-domain models. Finally, the accuracy and efficiency of the proposed method are substantiated by transient simulation of example circuits using this model.

A constrained orthogonal least-squares method for generating TSK fuzzy models: Application to short-term load forecasting

In this paper, an orthogonal least-squares (OLS) based modeling method is developed, named the constrained OLS (C-OLS), for generating simple and efficient TSK fuzzy models. The method is a two-stage model building technique, where both premise and consequent identification are simultaneously performed. The fuzzy system is considered as a linear regression model by decomposing the TSK model into a collection of generic rules. The C-OLS algorithm is employed at stage-1 to identify the structure of the model. Given a model building data set, the algorithm selects a subset of most significant regressors which should be included in the model. Based on the similarity measure, a classification tool is developed, which organizes the selected terms into groups with similar premise parts, forming TSK rules. Additionally, input variable selection for the consequent part is performed. The resulting model is reduced in complexity by discarding the unnecessary terms, and is optimized at stage-2 using a richer training data set. This method is used to generate fuzzy models for a real-world problem, the load forecasting of the Greek power system. Extensive simulation results are given and discussed, demonstrating the effectiveness of the suggested method.

A Genetic Algorithm Approach to Identification of Nonlinear Polynomial Models

In the last two decades, linear-in-parameter nonlinear polynomial models for NARX (Nonlinear Auto-Regressive with eXogenous input) systems have received considerable attention. The keypoint of polynomial model identification is how to select a set of significant terms employed to approximate the NARX system under study, from a large number of candidates. To this end, the orthogonal least-squares method which is a local search procedure, and the genetic algorithm approach which has a high potential for global optimization have been proposed in the literature. However, it is considered that the methods reported so far in the literature still lack potential to identify the polynomial models with relatively high-order. This limits the applicability of the polynomial models to the real complex nonlinear systems. Motivated by this fact, in this paper, a new genetic algorithm approach to polynomial model identification is proposed. Our contribution in this paper is to introduce a novel hierarchical encoding technique which is considered to be suitable to the structure of the polynomial models. Simulation and application results are also included to verify the efficiency of the proposed identification algorithm.

Hybrid training of radial basis function networks in a partitioning context of classification

The design of radial basis function networks is rather complex because of the great number of parameters that must be adjusted: positioning and number of kernels, choice of the distance type and centre widths, weight values. This article details these points in the framework of classification tasks with a partitioning approach. An adaptation of the orthogonal least-squares method is presented in order to select the centres of each sub-classifier in connection with a particular stopping criterion based on the addition of a random centre. Moreover, different choices of distance and centre widths are compared and illustrated by a 4-class problem in the non-destructive evaluation domain.

Least-squares adjustment of mathematical model of heat and mass transfer processes during solidification of binary alloys

Classical boundary and initial-boundary value problems in heat and mass transfer are generally formulated in a mathematically unique way. Boundary and initial conditions together with physical properties of the thermodynamic system are treated as exactly known. The influence of different kinds of mathematical model simplifications on the accuracy of solution and reliability of the model are not usually analyzed. The problems become more complicated when inverse ill-posed initial-boundary problems are considered. The widely used procedure of model validation is based on direct comparison of analytical or numerical solution, unique in a mathematical sense, with measurement results. The main feature of the method presented in this article is that all experimental results are included into the mathematical model. Thus, because of the inevitable errors of measurements, the system of model equations becomes internally contradicted as the number of unknown variables is less than the number of equations. In consequence, basic laws of energy and mass conservation are not satisfied. To adjust the experimental data to the mathematical model, an orthogonal least-squares method is proposed. Special attention has been paid to the coupling of experimental data with the nucleation and grain growth models formulated by Rappaz and co-workers. Theoretical considerations are illustrated with experimental data for an Al-Si alloy.

Predictive control based on local linear fuzzy models

This paper deals with predictive control based on fuzzy models. A novel algorithm (LOLIMOT) is proposed for the construction of Takagi-Sugeno fuzzy models. The rule consequents are optimized by a local orthogonal least-squares method that selects the significant regressors. The rule premises are optimized by a tree construction algorithm which partitions the input space in hyper-rectangles. A generalized predictive controller (GPC) and a dynamic matrix controller (DMC) are designed. Both controllers require the extraction of a linear model from the Takagi-Sugeno fuzzy model. For the GPC a new technique called local dynamic linearization is proposed that exploits the special structure of the local linear models. The DMC is based on the evaluation of a step response. The effectiveness of both the identification algorithm and the predictive controllers is shown by application to temperature control of an industrial-scale cross-flow heat exchanger.

Direct control with radial basis function networks: Stability analysis and applications

Abstract In the present paper, neural control and identification of general nonlinear plants are accomplished using radial basis function (RBF) networks. A neural controller is adjusted oil-line by using the orthogonal least squares (OLS) method. A stability analysis has been performed using the conicity criterion, and based upon this a new training data set is elicited so that a stable neural controller is obtained. Applications both to a nonlinear fluid level system and to an inverted pendulum are detailed, demonstrating the effectiveness of the proposed method. Training time with the OLS method are reduced compared to standard backpropagation technique.