Extended Least Squares Research Articles

Extended least squares (ELS) and generalized least squares (GLS) for clutter suppression in hyperspectral images: A theoretical description

Hyperspectral imaging (HSI) has been demonstrated to be useful for estimating spatial distributions of different chemical constituents on surfaces and in imaged scenes, as well as for detecting anomalies and known targets that may be present. In target detection, the objective is to find pixels that contain significant signal attributable to a known “target” spectrum. Unfortunately, measured images are often dominated by “non-target” interference signals with unknown spectra that vary image-to-image. To be effective, target detection algorithms must account for the non-target signal, commonly referred to as “clutter.” The utility of HSI for target detection in spatially complex scenarios can be attributed to two aspects associated with the sensing system: 1) The target can dominate the measured signal in a few individual pixels, although it may have only a minor contribution to the overall signal, and 2) HSI measures many pixels relevant for characterizing clutter that can be used in clutter suppression algorithms to account for interferences.Clutter suppression enables the detection of minor signal sources in HSI, resulting in a sensing system uniquely suited to highly variable environmental scenarios. Popular clutter suppression methods such as extended least squares (ELS) and generalized least squares (GLS) can be described as extensions of the classical least squares (CLS) model. The main advantage of these methods is that they are fast and easy to train on an image-to-image basis providing highly sensitive, image-specific target detection systems. However, the optimization of such models affects the sensitivity and selectivity of the detection. In an effort to assist practitioners interested in clutter suppression, this paper presents a detailed description of the theoretical aspects of ELS and GLS methods used for target detection on hyperspectral images. The concepts are demonstrated with practical examples of the detection of an acrylonitrile butadiene styrene (ABS) particle in sand based on a single near infrared hyperspectral image.

Support Recovery and Parameter Identification of Multivariate ARMA Systems with Exogenous Inputs

Focusing on identification, support recovery, and parameter estimation of multivariate ARMA systems with exogenous inputs (ARMAX) is considered. It is assumed that sparsity appears not only in the unknown parameter matrix of the system inputs and outputs but also in the coefficients of system noises, and with exogenous inputs the system admits feedback control and thus the observation of the ARMAX system may be nonstationary. A two-stage sparse identification algorithm is introduced. First, the extended least squares (ELS) algorithm is applied to obtain an initial estimate for the unknown parameter matrix of the ARMAX system. Second, a convex optimization criterion is introduced based on norm with regularization, where adaptive weights generated from ELS algorithms are adopted in the optimization variables in the term. Third, it is proved that by optimizing the criterion, the sets of the zero and the nonzero entries of the parameter matrix, i.e., the support set of the ARMAX system, can be correctly identified with a finite number of observations, and estimates of the nonzero parameters converge to the true values with probability one. Fourth, the strictly positive realness (SPR) condition required by the ELS algorithm is relaxed by an overparametrization technique. The performance of the algorithm is testified through simulation examples.

A Method of Reducing Invalid Steering for AUVs Based on a Wave Peak Frequency Tracker

The motion control of autonomous underwater vehicles (AUVs) is affected by waves near the ocean surface or in shallow-water areas. Therefore, to counteract the influence of waves, we need to remove them by designing a filter. The wave peak frequency is important in wave filter design. This paper focuses on the identification of the wave peak frequency using the least-squares parameter estimation algorithm. The input–output expression of the wave disturbance model is derived by eliminating the intermediate variable. Based on the obtained identification model, an auxiliary model-based recursive extended least-squares identification algorithm is developed to estimate the model parameters. The effectiveness of the proposed method is verified with simulated tests of the heading control system of an AUV. The simulation results demonstrate that the proposed method is effective for the identification of the wave peak frequency, and an observer with a wave peak frequency tracker can significantly reduce invalid steering.

Treatment of Bone Marrow Cancer Based on Model Predictive Control

Bone marrow is a spongy tissue that contains stem cells that are found inside some bones, including the hip and femur. Bone marrow cancer is a type of cancer that is caused by stem cells that make up the blood cells in the bone marrow. Sometimes these cells grow too fast or abnormally, which is called bone marrow cancer. Bone tissue cells are mainly composed of osteoblasts and osteoclasts. Osteoblast cells constantly build new bone throughout the life of each bone, and other cells called osteoclasts constantly absorb pieces of bone, so the bone is constantly being renewed. In this paper, mathematical models of tumors, the effect of the body on the drug, and the drug on the body are introduced, and then the appropriate dose of the drug to reduce tumor density is calculated using the model predictive control (MPC) algorithm. To obtain an adaptive MPC strategy, the extended least squares (ELS) method developed to learn the parameters of the tumor growth model is used. Finally, the simulation in MATLAB, assuming the model is correct, shows that the tumor is gone, and the bone mass improves over a period of time. The results demonstrate that the proposed method is effective for the treatment of bone marrow cancer.

Improved least‐squares identification for multiple‐output non‐linear stochastic systems

This study considers the identification problems of multiple-output non-linear equation-error moving average systems. There exist the product items of the parameters between the non-linear and linear parts. To solve this difficulty, the key term separation technique is adopted. By using the model decomposition technique and the hierarchical identification principle, a maximum likelihood-based recursive extended least-squares estimation algorithm with reduced computational complexity is presented to estimate the parameters of the non-linear part and the linear part interactively. The simulation results demonstrate the effectiveness of the proposed method.

Recursive Least-squares Estimation for Multivariable Systems Based on the Maximum Likelihood Principle

This paper studies the identification problem of multivariable controlled autoregressive moving average systems. For the case with a parameter matrix and an unmeasurable vector in the system identification model, we transform the model into several submodels based on the number of the outputs. A maximum likelihood-based recursive least-squares algorithm is derived to identify the parameters of each submodel. A multivariable recursive extended least-squares algorithm is provided as a comparison. The effectiveness of the proposed identification algorithm is verified by simulation examples.

Derivation of NARX models by expanding activation functions in neural networks

A method was developed to derive an NARX model from a neural network so that the usability of open‐source libraries for network learning was combined with the NARX advantage of revealing the system structure. After the neural network model was trained on input and output data, the sigmoid activation functions were expanded into Taylor series. Candidate parameters in the NARX model were calculated from the connection weights in the neural network and coefficients in the series. The NARX model structure was determined by the extended least‐squares (ELS) method and the Bayesian information criterion (BIC). Correct NARX models were successfully detected in computer simulations and an experiment. The developed method can be used for any activation functions, including the sigmoid function, if they are expanded into Taylor series. © 2019 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

Maximum likelihood-based recursive least-squares estimation for multivariable systems using the data filtering technique

ABSTRACTFor multivariable equation-error systems with an autoregressive moving average noise, this paper applies the decomposition technique to transform a multivariable model into several identification sub-models based on the number of the system outputs, and derives a data filtering and maximum likelihood-based recursive least-squares algorithm to reduce the computation complexity and improve the parameter estimation accuracy. A multivariable recursive generalised extended least-squares method and a filtering-based recursive extended least-squares method are presented to show the effectiveness of the proposed algorithm. The simulation results indicate that the proposed method is effective and can produce more accurate parameter estimates than the compared methods.

Identification of Flip Folder Model on Folding Machine

Folding Machine is an indispensable tool in the small-scale laundry industry that aims to save time and energy. The main component of this equipment is a flip folder that will be controlled in the form of deflection of the angle position in order to operate the folded clothes. In the design of the controller, what needs to be done is the identification of the plant to determine the characteristics of the system. Modelling section in this paper is using relationships between input variables; input components are often difficult to measure the parameters of the plant. Therefore, this research will identify the system based on the offline input and output data measurement by using Extended Least Square (ELS) and Auto Regressive Exogenous (ARX) model approach. Based on the results of the identification, the second order damped plant equation can be approximated by first order reduction model with the characteristic transient response that having a time constant (τ) of 0.1706 seconds, the rise time of 0.500 seconds, the settling time of 0.511 seconds and the delay time of 0.1178 seconds.

Recursive identification for Hammerstein systems with diminishing excitation signals

In this paper, we consider the identification of Hammerstein systems where the nonlinearity is described by a combination of basis functions with unknown coefficients. The extended least squares (ELS) algorithm is applied to estimate the unknown parameters in the system. Contrary to the classical excitation signals for identification of Hammerstein systems, i.e., the periodic inputs or stationary random signals, here we choose a sequence of diminishing excitation signals as the system inputs. We prove that the strong consistency of the ELS algorithm still holds true and the convergence rate is obtained as well. A numerical example is given to verify the performance of the identification method.

Maximum Likelihood-Based Recursive Least-Squares Algorithm for Multivariable Systems with Colored Noises Using the Decomposition Technique

This paper considers the parameter estimation problems for a class of multivariable equation-error systems with colored noises. By using the decomposition technique, a multivariable system is transformed into several subsystems to reduce the computational burden, and a maximum likelihood-based recursive least-squares identification algorithm is developed for estimating the parameters of each subsystem. As a comparison, a multivariable recursive extended least-squares algorithm is presented. The analysis indicates that the proposed algorithm has lower computational complexity than the multivariable recursive extended least-squares algorithm, and the numerical simulation results demonstrate that the proposed method is effective.

Chaos synchronization and Nelder-Mead search for parameter estimation in nonlinear pharmacological systems: Estimating tumor antigenicity in a model of immunotherapy

In mathematical pharmacology, models are constructed to confer a robust method for optimizing treatment. The predictive capability of pharmacological models depends heavily on the ability to track the system and to accurately determine parameters with reference to the sensitivity in projected outcomes. To closely track chaotic systems, one may choose to apply chaos synchronization. An advantageous byproduct of this methodology is the ability to quantify model parameters. In this paper, we illustrate the use of chaos synchronization combined with Nelder-Mead search to estimate parameters of the well-known Kirschner-Panetta model of IL-2 immunotherapy from noisy data. Chaos synchronization with Nelder-Mead search is shown to provide more accurate and reliable estimates than Nelder-Mead search based on an extended least squares (ELS) objective function. Our results underline the strength of this approach to parameter estimation and provide a broader framework of parameter identification for nonlinear models in pharmacology.

Nonlinear Feed Forward Control of a Perturbed Satellite using Extended Least Squares Adaptation and a Luenberger Observer

Three adaptive approaches for a non-linear feed-forward controller are combined with two physics-inspired sinusoidal trajectory planners in a spacecraft attitude control model for large slew maneuvers. The basis of the model is a space based satellite sensor which has suffered an unwanted collision where the inertial matrix of the craft is no longer similar to the originally measured inertial matrix. This causes a large inherent error in the feedforward control needed for system maneuvers due to the mismatch of expected dynamics. Trajectory generation, feedforward control, feedback control, filters, observers, and system stability are discussed in relation to the non-linear dynamics under simulation. The adaptive feed-forward controllers discussed include a proportional-derivative (PD) adaptive controller, a Recursive Least Square (RLS) Method, and an Extended Least Squares (ELS) Method. Mean control effort stayed relatively constant between configurations. The controller configuration with ELS feedforward, PID feed-back, and an extended sinusoidal trajectory outperformed the baseline adaptive controller. Mean error was decreased by 23.4%, error standard deviation by 34.0%, and maximum error by 33.0% from a similar case using RLS adaptation. This improvement is entirely based on a need to correct for un-modeled or mis-modeled dynamics. This scenario occurs in actual operation during spacecraft launch, collisions with debris, or can be caused by fuel slosh or loose components.

Acceleration tracking control combining adaptive control and off-line compensators for six-degree-of-freedom electro-hydraulic shaking tables

An electro-hydraulic shaking table (EHST) is an essential experimental facility to simulate in real-time actual vibration situations. An adaptive controller combined with off-line compensators is proposed to improve the acceleration frequency bandwidth and tracking accuracy of a six-degree-of-freedom (6-DOF) EHST. A servo controller has been employed to implement acceleration closed-loop and coordinate control of the 6-DOF EHST. A recursive extended least-squares algorithm is employed to identify acceleration closed-loop transfer functions and a zero-phase-error tracking controller is used to design off-line inverse model compensators using the identified transfer functions. However, the off-line compensators cannot compensate in real-time varying dynamics of the 6-DOF EHST; so an online adaptive controller with a least-mean-squares (LMS) algorithm based on a delay compensator is employed. The proposed controller combines advantages of the off-line compensators and the online adaptive controller, which guarantees both a fast rate of convergence of the LMS algorithm and high-fidelity acceleration tracking accuracy of the 6-DOF EHST. Some experimental studies have been conducted on a 6-DOF EHST and experimental results show that acceleration tracking control performances, including the rate of convergence of the LMS algorithm and acceleration tracking accuracy, have been improved compared to a conventional three-variable controller and adaptive controllers.

Experimental evaluation of the parameter-based closed-loop transfer function identification for electro-hydraulic servo systems

Closed-loop systems of an electro-hydraulic servo system including position, acceleration, and force closed-loop systems and their closed-loop transfer functions based on parameter model are adaptive identified using a recursive extended least-squares algorithm. The position and force closed-loop tracking controllers are designed by a proportional–integral–derivative controller and are tuned by the position and force step signals. The acceleration closed-loop tracking controller is designed by a three-variable controller and the three states include position, velocity, and acceleration. Experimental results of the estimated position, acceleration, and force closed-loop transfer functions are performed on an actual electro-hydraulic servo system using xPC rapid prototyping technology, which clearly demonstrate the benefit of the adaptive identification method.

Acceleration waveform replication on six-degree-of-freedom redundant electro-hydraulic shaking tables using an inverse model controller with a modelling error

A redundant electro-hydraulic shaking table (REST) of six degrees-of-freedom (6 DOFs) with eight hydraulic actuators is an essential experimental tool in many industrial applications for real-time simulation of actual vibrations, such as structural vibration, earthquake simulation and fatigue testing. In order to obtain a high-fidelity acceleration waveform on the REST, a feed-forward inverse model (FFIM) controller with a modelling error compensator is proposed in this study. A recursive extended least-squares algorithm is employed to identify an acceleration closed-loop transfer function of the REST. A zero phase error compensation technology is employed to guarantee stability of the designed FFIM because the identified acceleration closed-loop transfer function is a typical non-minimum phase system and its direct inverse transfer function is unstable. The modelling error compensator is designed to compensate for the modelling error between the identified transfer function and the actual experimental REST, which deteriorates the acceleration waveform replication accuracy of the REST. A 6 DOF REST experimental system was used to verify the proposed controller. Experimental results demonstrated that the proposed controller gave satisfactory acceleration tracking performances on the REST.

Improved ETSI advanced front-end for automatic speech recognition using quadrature mirror filter synthesis based on robust complex speech analysis

An automatic speech recognition (ASR) is commonly used in these days. Current ASR systems perform well in ideal environment; however, it does not perform well in realistic noisy environment. As a robust ASR, ETSI has standardized Advanced Front-End (AFE) that adopts two-stage of iterative Wiener filter (IWF) to realize a speech enhancement as the front-end of ASR. In the ETSI AFE, 16 kHz speech is divided uniformly by QMF (Quadrature Mirror Filter) into lower-band and higher-band signal and only lower-band signal is emphasized by the IWF and MFCC (Mel Frequency Cepstral Coefficient) is extracted from both the second-stage of emphasized lower-band signal and non-emphasized higher-band signal. FFT is used to estimate speech spectrum that designs the Wiener filter. On the other hand, we have already proposed robust complex speech analysis for an analytic signal. It can estimate more robust and more accurate speech spectrum due to the introduced robust criterion and nature of analytic signal. This paper proposes an improved AFE using wide-band robust ELS (Extended Least Square) complex analysis and real-valued analysis instead of FFT. Moreover, the QMF synthesis is also introduced for second-stage of wide-band analysis. The experimental results using the CENSREC-2 speech database demonstrate that the performance is improved.

Adaptive modeling of waterflooding process in oil reservoirs

Waterflooding is a common solution for enhancing oil recovery in oil reservoirs in which water is injected to the reservoir. For this purpose, injection rates should be determined to optimally control the desired oil production from each well to accomplish reservoir management strategies. This goal can be achieved by precisely modeling the waterflooding phenomena. In this paper, an applicable algorithm for modeling the waterflooding process in oil reservoirs has been developed. Using proper System Identification techniques will help to consider any change in the inherent reservoir specifications in addition to operating point variations due to injection alternation rate and also severe nonlinear nature of the reservoir and consequently to obtain adaptive model for the reservoir which is able to reflect all the mentioned phenomena. The reservoir has been considered as a Multi-Input/Multi-Output (MIMO) plant with several manipulated variables and outputs to mimic the variables dynamics of the reservoir during its operational life. A Recursive Least Square (RLS) approach has been utilized with proper forgetting factor and sliding window length selection to extract the reservoir dynamic model in the content of a MIMO model structure. The developed approach has been evaluated for SPE10 model as a well-known reservoir case study in this paper.The observed results demonstrate that the proposed method has acceptable capability, compared to the conventional off-line Least Square (LS) and Extended Least Square (ELS) approaches which may lead to Auto Regressive model with eXogenous input (ARX) and Auto Regressive-Moving-Average model with eXogenous inputs (ARMAX) structures, for modeling the dynamical behaviors of the reservoir to be employed as a powerful tool for control and optimization applications.

Accelerating extended least-squares migration with weighted conjugate gradient iteration

Least-squares migration (LSM) iteratively achieves a mean-square best fit to seismic reflection data, provided that a kinematically accurate velocity model is available. The subsurface offset extension adds extra degrees of freedom to the model, thereby allowing LSM to fit the data even in the event of significant velocity error. This type of extension also implies additional computational expense per iteration from crosscorrelating source and receiver wavefields over the subsurface offset, and therefore places a premium on rapid convergence. We have accelerated the convergence of extended least-squares migration by combining the conjugate gradient algorithm with weighted norms in range (data) and domain (model) spaces that render the extended Born modeling operator approximately unitary. We have developed numerical examples that demonstrate that the proposed algorithm dramatically reduces the number of iterations required to achieve a given level of fit or gradient reduction compared with conjugate gradient iteration with Euclidean (unweighted) norms.

A RESPIRATORY MECHANICAL PARAMETERS ESTIMATION TECHNOLOGY BASED ON EXTENDED LEAST SQUARES

Respiratory mechanical parameters of ventilated patients are usually referred in the respiratory diagnosis and treatment. However, the effectiveness of the modern estimation methods is limited. To estimate the overall breathing resistance, overall respiratory compliance, and residual volume simultaneously, a new mathematical model of mechanical ventilation system was proposed. Furthermore, to improve the estimation accuracy, the noise model of mechanical ventilation system was taken into consideration. Based on the mathematical model, a respiratory mechanical parameters estimation method based on extended least squares (ELS) algorithm was derived. Finally, to test the respiratory mechanical parameters estimation method, it was studied experimentally and numerically, and it was approved that the proposed method is effective to estimate the three respiratory mechanical parameters simultaneously and precisely. The estimated values of the parameters can be adopted in the clinical practice. The study provides a new method to estimate respiratory mechanical parameters.