Classical Least-squares Method Research Articles

Application of Bayesian analysis to the doubly labelled water method for total energy expenditure in humans.

RationaleThe doubly labelled water (DLW) method is the reference method for the estimation of free‐living total energy expenditure (TEE). In this method, where both 2H and 18O are employed, different approaches have been adopted to deal with the non‐conformity observed regarding the distribution space for the labels being non‐coincident with total body water. However, the method adopted can have a significant effect on the estimated TEE.MethodsWe proposed a Bayesian reasoning approach to modify an assumed prior distribution for the space ratio using experimental data to derive the TEE. A Bayesian hierarchical approach was also investigated. The dataset was obtained from 59 adults (37 women) who underwent a DLW experiment during which the 2H and 18O enrichments were measured using isotope ratio mass spectrometry (IRMS).ResultsTEE was estimated at 9925 (9106‐11236) [median and interquartile range], 9646 (9167–10540), and 9,638 (9220–10340) kJ·day−1 for women and at 13961 (12851–15347), 13353 (12651–15088) and 13211 (12653–14238) kJ·day−1 for men, using normalized non‐Bayesian, independent Bayesian and hierarchical Bayesian approaches, respectively. A comparison of hierarchical Bayesian with normalized non‐Bayesian methods indicated a marked difference in behaviour between genders. The median difference was −287 kJ·day−1 for women, and −750 kJ·day−1 for men. In men there is an appreciable compression of the TEE distribution obtained from the hierarchical model compared with the normalized non‐Bayesian methods (range of TEE 11234–15431 kJ·day−1 vs 10786–18221 kJ·day−1). An analogous, yet smaller, compression is seen in women (7081–12287 kJ·day−1 vs 6989–13775 kJ·day−1).ConclusionsThe Bayesian analysis is an appealing method to estimate TEE during DLW experiments. The principal advantages over those obtained using the classical least‐squares method is the generation of potentially more useful estimates of TEE, and improved handling of outliers and missing data scenarios, particularly if a hierarchical model is used.

Earth orientation parameters from VLBI determined with a Kalman filter

This paper introduces the reader to our Kalman filter developed for geodetic VLBI (very long baseline interferometry) data analysis. The focus lies on the EOP (Earth Orientation Parameter) determination based on the Continuous VLBI Campaign 2014 (CONT14) data, but also earlier CONT campaigns are analyzed. For validation and comparison purposes we use EOP determined with the classical LSM (least squares method) estimated from the same VLBI data set as the Kalman solution with a daily resolution. To gain higher resolved EOP from LSM we run solutions which yield hourly estimates for polar motion and dUT1 = Universal Time (UT1) – Coordinated Universal Time (UTC). As an external validation data set we use a GPS (Global Positioning System) solution providing hourly polar motion results.Further, we describe our approach for determining the noise driving the Kalman filter. It has to be chosen carefully, since it can lead to a significant degradation of the results. We illustrate this issue in context with the de-correlation of polar motion and nutation.Finally, we find that the agreement with respect to GPS can be improved by up to 50% using our filter compared to the LSM approach, reaching a similar precision than the GPS solution. Especially the power of erroneous high-frequency signals can be reduced dramatically, opening up new possibilities for high-frequency EOP studies and investigations of the models involved in VLBI data analysis.We prove that the Kalman filter is more than on par with the classical least squares method and that it is a valuable alternative, especially on the advent of the VLBI2010 Global Observing System and within the GGOS frame work.

Distributed TLS over multitask networks with adaptive intertask cooperation

Distributed estimation is an important technique for in-network signal processing. In this work, we consider a general case where the unknown parameter vectors (tasks) for different nodes can be different, which is different from the common single-task problem and is known as the multitask problem. Besides, it is assumed that there are some similarities among these tasks. Thus, the performance may be improved by performing the intertask cooperation. To improve robustness against different degrees of difference among the tasks, an adaptive intertask cooperation strategy is proposed. On the other hand, in most of the existing distributed algorithms, it is usually assumed that noise/errors are confined to the output signal. However, in many real environments, the input and output signals may be both corrupted by noise, which is described by the errors-in-variables (EIV) model. In such a case, it has been demonstrated that the total least-squares (TLS) method based on minimizing the squared total error outperforms the classical least-squares method. In this paper, we consider the EIV model with several parameter vectors to be estimated simultaneously, and we derive a distributed TLS algorithm with adaptive intertask cooperation for the in-network cooperative estimation problem. For this algorithm, theoretical performance analysis is provided. Besides, to verify its effectiveness, numerical simulations are given.

Iterative reweighted least M-estimate AVO inversion

The l1 norm and its variants, such as the hybrid l1/l2 norm and the Huber norm, that are used to solve amplitude variation with offset (AVO) inversion optimisation problems, are mostly known to give a more robust solution than the classical least-squares (l2 norm) method by reducing the influence of outliers significantly, although never ignoring it. To deal with data having many outliers, biweight norm using iteratively reweighted least-squares (IRLS) as robust inversion method can improve robustness by ignoring outliers in computing the misfit measure. However, biweight norm uses a higher-order descending weighting on the measured data, which results in poor performance when dealing with the well measured data. Hampel’s three-part redescending M-estimate function as robust measure could be considered as a three-part combination of the l2 norm and l1 norm with excluding outliers, which could perform better. This paper describes an iterative reweighted least M-estimate (IRLM) algorithm as a robust AVO inversion. The synthetic and field seismic data tests show that the IRLM algorithm gives far more robust model estimates than the conventional Huber norm and biweight norm.

Mean centering of ratio spectra and concentration augmented classical least squares in a comparative approach for quantitation of spectrally overlapped bands of antihypertensives in formulations

Two different methods manipulating spectrophotometric data have been developed, validated and compared. One is capable of removing the signal of any interfering components at the selected wavelength of the component of interest (univariate). The other includes more variables and extracts maximum information to determine the component of interest in the presence of other components (multivariate). The applied methods are smart, simple, accurate, sensitive, precise and capable of determination of spectrally overlapped antihypertensives; hydrochlorothiazide (HCT), irbesartan (IRB) and candesartan (CAN). Mean centering of ratio spectra (MCR) and concentration residual augmented classical least-squares method (CRACLS) were developed and their efficiency was compared. CRACLS is a simple method that is capable of extracting the pure spectral profiles of each component in a mixture. Correlation was calculated between the estimated and pure spectra and was found to be 0.9998, 0.9987 and 0.9992 for HCT, IRB and CAN, respectively. The methods were successfully determined the three components in bulk powder, laboratory-prepared mixtures, and combined dosage forms. The results obtained were compared statistically with each other and to those of the official methods.

Prediction model of electric energy consumption of airport lighting equipment

A method of using multivariate regression model of predicting electric energy consumption of airport lighting for managing the efficient use of energy resources of airlines is proposed in the paper. Prediction using the correlation and regression methods involves finding mathematical equations, describing the statistical relationship of one indicator to another (pair correlation), with a group of other indicators (multiple correlation) or between the values of one indicator (autocorrelation). For obtaining a general linear multivariate prediction model, a classical method of least squares was used by the authors. Using the developed model will allow to develop a complex of energy saving measures, taking into account plans of reconstructing the existing infrastructure and manage electric energy consumption of the lighting equipment, taking into consideration peculiarities of its operation. The obtained model was tested on the statistical data of Kiev International airport (Zhuliany).

In Situ Attenuated Total Reflectance Fourier Transform Infrared (ATR FT-IR) Spectroscopy Monitoring of 1,2-Butylene Oxide Polymerization Reaction by Using Iterative Concentration-Guided Classical Least Squares

There has been rapid growth in the application of in situ optical spectroscopy techniques for reaction and process monitoring recently in both academia and industry. Vibrational spectroscopies such as mid-infrared, near-infrared spectroscopy, and Raman spectroscopy have proven to be versatile and informative. Accurate determination of concentrations, based on highly overlapped spectra, remains a challenge. As an example, 1,2-butylene oxide (BO) polymerization, an important industrial reaction, initiated by propylene glycol (PG) and catalyzed by KOH, is studied in this work in a semi-batch fashion by using in situ attenuated total reflectance Fourier transform infrared spectroscopy (ATR FT-IR) monitoring. The weak BO absorbance, the constantly changing interference from the product oligomers throughout the course of the reaction, and the change in BO spectral features with system polarity posed challenges for quantitative spectral analysis based on conventional methods. An iterative concentration-guided classical least-squares (ICG-CLS) method was developed to overcome these challenges. Taking advantage of the concentration-domain information, ICG-CLS enabled the estimation of the pure oligomer product spectra at different stages of the semi-batch process, which in turn was used to construct valid CLS models. The ICG-CLS algorithm provides an in situ calibration method that can be broadly applied to reactions of known order. Caveats in its applications are also discussed.

Sosyal Bilimlerde Alternatif Regresyon Yöntemi Kullanımı ve En Küçük Kareler ile M Tahmin Yöntemlerinin Belirleyicilik Katsayısı Açısından Karşılaştırılması

AbstractThe purpose of this study is two-fold; the first aim being to show the effect of outliers on the widely used least squares estimator in social sciences. The second aim is to compare the classical method of least squares with the robust M-estimator using the determination of coefficient (R2). For this purpose, analyzes were performed on three data sets. The first set of data is hypothetical, consisting of 15 students' general mathematic and linear algebra final scores. The second set of data was collected from 231 adolescents at- tending different high schools in Turkey. The data were collected using the Scale of Aggressiveness, Academic Self-efficacy Scale, Scale of Peer Pressure, and Trait Anxiety Inventory. The third set of data was collected from 1,385 high school students. This data were collected using the Maslach Burnout Inventory-Students Survey, Coping Styles of Stress Scale, Test Anxiety Inventory, Adolescence Self-Efficacy Scale, and Parental Attitude Scale. It was seen that, comparisons with small, medium and large volume samples, especially for the data sets including outiler/outliers, R2 in M estimate is better alternatives than those having least squares. The findings are discussed in light of the recommendations presented in the literature.Key WordsCoefficient of Determination, Least Squares, M-Estimator, Outlier, Regression Analysis, Robust Statistics.(ProQuest: ... denotes text missing in the original.)In scientific research projects, finding a relationship between two or more variables and then expressing it in a mathematical equation is an important dimension needed in order to make future predictions. This mathematical relationship does not only refer to functional relationship, but also shows that one of the variables of a predetermined value provides estimation of the other.The method that permits one to depict the relationship between variables in an equation is called regression analysis, a method which has applications in almost every field (Ariel, 1991).Regression analysis has an important role in scientific research projects because it allows a researcher to predict the future, which is one of the most important missions of science. In fact, analysis may be the most widely used statistical technique (Buyukozturk, 2005; Buyukozturk, Cokluk, K Montgomery, Peck, & Vining, 2001).It is more convenient to deal with multiple models if they are expressed in matrix notation. This allows for a very compact display of the model, data, and results. In matrix notation, the model given by Eq. (2) is:... (3)where... (4)In general, y is an n x 1 vector of the observations, X is an n x p matrix of the levels of the regressor variables, b is an p x 1 vector of the coefficients, and £ is an n x 1 vector of random errors (Montgomery et al., 2001). The major assumptions made thus far in the present study of analysis are as follows:i) The relationship between the response y and the regressors is linear, at least approximately.ii) The errors term £ has a zero mean. …

Robust inversion using biweight norm and its application to seismic inversion

In spite of some minor drawbacks such as nonuniqueness and higher computational cost, finding the least-absolute (l1 norm) error solution to solve an optimisation problem is mostly known to give a better answer than the classical least-squares (l2 norm) method. This is because the robust property of the median value is affected little by outlier values and the solution of the least l1 norm error corresponds to the solution of minimum median error. Several variants of the l1 norm such as the Huber norm and the Hybrid norm have the same robust properties as the l1 norm. The optimisation methods based on l1 norm obtain their robustness by reducing the influence of outliers significantly, although never ignoring it. Therefore, if the proportion of outliers increases, most of the methods based on l1 norm may begin to be affected by the outliers. In such a case, other types of robust measures such as Tukey’s Biweight (Bisquare weight) norm, which excludes outliers in computing the misfit measure, could perform better. This paper describes the application of the Biweight norm using the IRLS (iteratively reweighted least-squares) method as a robust inversion and shows its possible improvement in robustness when dealing with data having many outliers.

Improving multiple removal using least-squares dip filters and independent component analysis

The suppression of multiple events is a crucial task in seismic data processing, and the adaptive subtraction of the predicted multiples is recognized as one of the main challenges for the success of the surface-related multiple elimination technique. The traditional least-squares matching approach can affect the primary events because the estimated multiples tend to adapt to the primaries under the minimum energy condition. We investigate two filtering techniques for improving the multiple removal results. In the first proposed method, we combine the advantages of the least-squares and pattern dip-based subtraction methods. Doing so, we exploit the separation of primaries and multiples in the dip domain, and then we apply the least-squares adaptive subtraction in each dip band before recomposing the data to obtain the final subtraction result. As a result of the dip decomposition, the primary-multiple interferences are reduced, allowing for a more reliable least-squares filtering. In the second method, we propose to replace the multiple subtraction step by a separation step using independent component analysis (ICA) methods. We employ the ICA method after least-squares adaptive filtering. Because of the non-Gaussian distributions of the involved signals, primaries and multiples can be separated by computing the optimal rotation between these two signals. We apply the ICA method in local 2D time-space windows to better compensate the space and time variant character of the data. Two-dimensional synthetic and field data examples demonstrate that the multiple subtraction results of both methods are indeed improved with respect to the classical least-squares method.

SPECTRUM PURIFICATION AUGMENTED CLASSICAL LEAST-SQUARES (CLS): A ROBUST MULTIVARIATE CALIBRATION METHOD FOR COMPOUND QUANTITATIVE DETERMINATION

An improved classical least-squares (CLS) method that can correct the detrimental effects of traditional spectroscopic data analysis CLS models has been developed. An additional noise (introduced by unmodeled concentrations) model is constructed, and the original mixed spectrum is purified using least squares to eliminate the additional noise signals. The extracted spectral signals are then used to augment the concentration predictive power. An application study is presented, which involves data recorded from an experiment of multicomponent concentration determination using Raman spectroscopy, and two designed strategies are exploited to test the predictive power of the proposed method. Results indicate that this method is an applicable spectral analysis tool for compound concentration determination against traditional CLS model ineffectiveness rendered by unmodeled concentrations, and the predictive power of the proposed method is comparable to existing methods.

Application of the on-line recursive least-squares method to perform structural damage assessment

The identification of structural damage is an important objective of health monitoring for civil infrastructure. An on-line recursive least-squares (RLS) identification technique is applied in this study to identify the time-varying dynamic parameters of structures subjected to earthquake loadings. Based on the framework of adaptive filters, the observations are obtained sequentially in real time. It is desirable to perform the identification tasks recursively to save computation time and to be able to observe the variations of parameters on-line. Computation of the classical least-squares (LS) method can be arranged recursively so that the estimated parameters at previous step can be used to predict the responses at current time. The one-step ahead predicted error between estimated response and measured response is calculated by the on-line RLS method and the dynamic properties of system can be identified as well. The purpose of this study is to apply the RLS method to verify the identification procedures and to perform the damage assessment on a three-floor shaking table benchmark model tested at NCREE in Taiwan. Furthermore, both classical LS and RLS methods are implemented to investigate the recorded strong-motion data of Tai-Tung Fire Bureau Building located at Tai-Tung City in Taiwan, which had been demolished due to severe damage after a magnitude 6.2 earthquake in 2006. By observing the variations of the identified time-varying modal properties of both benchmark model and real building, global damage behavior due to weak element or failure of components can be revealed. Copyright © 2009 John Wiley & Sons, Ltd.

Partitioned least-squares operator for large-scale geophysical inversion

Least-squares (LS) problems are encountered in many geophysical estimation and data analysis problems where a large number of observations (data) are combined to determine a model (some aspect of the earth structure). Examples of least squares in seismic exploration include several data processing algorithms, theoretically accurate LS migration, inversion for reservoir parameters, and background velocity estimation. A frequently encountered problem is that the volume of data in 3D is so large that the matrices required for the LS solution cannot be stored within the memory of a single computer. A new technique is described for parallel computation of the LS operator that is based on a partitioned-matrix algorithm. The classical LS method for solution of block-Toeplitz systems of normal equation (NE) to the general case of block-Hermitian and non-Toeplitz systems of NE. is generalized. Specifically, a solution of a block-Hermitian system of NE is shown that may be obtained recursively by linearly combining the solutions of lesser order that are related to the forward and backward subsystems of equations. This results in an efficient parallel algorithm in which each partitioned system can be evaluated independently. The application of the algorithm to the problem of 3D plane wave transformation is demonstrated.

Quantitative Analysis of the Cholesterol-Lowering Drugs Ezetimibe and Simvastatin in Pure Powder, Binary Mixtures, and a Combined Dosage Form by Spectrophotometry, Chemometry, and High-Performance Column Liquid Chromatography

Simple, accurate, sensitive, and precise UV spectrophotometric, chemometric, and HPLC methods were developed for simultaneous determination of a two-component drug mixture of ezetimibe (EZ) and simvastatin (SM) in laboratory-prepared mixtures and a combined tablet dosage form. Four spectrophotometric methods were developed, namely, ratio spectra derivative, ratio subtraction, isosbestic point, and mean centering of ratio spectra. The developed chemometric-assisted spectrophotometric method was the concentration residual augmented classical least-squares method; its prediction ability was assessed and compared to the conventional partial least-squares method. The developed HPLC method used an RP ZORBAX C18 column (5 microm particle size, 250 x 4.6 mm id) with isocratic elution. The mobile phase was acetonitrile-pH 3.5 phosphate buffer (40 + 60, v/v) at a flow rate of 1.0 mL/min, with UV detection at 230 nm. The accuracy, precision, and linearity ranges of the developed methods were determined. The developed methods were successfully applied for determination of EZ and SM in bulk powder, laboratory-prepared mixtures, and a combined dosage form. The results obtained were compared statistically with each other and to those of a reported HPLC method; there was no significant difference between the proposed methods and the reported method regarding both accuracy and precision.

Two-Variable Linear Regression: Modeling with Orthogonal Least-Squares Analysis

In many fields of chemistry, the ordinary least-squares method is preferentially used to fit data. Nevertheless, univariate linear regression by least-squares analysis, as devised by Gauss and Legendre, has some drawbacks that are usually overlooked in experimental science courses and even in many chemical research papers. Orthogonal least-squares fitting is a good method to avoid the unsymmetrical treatment of the data and also to avoid effects of regression toward the mean. These result because the classical least-squares method only minimizes the squared distance parallel to the y axis between the experimental points and the fitting line, whereas the distance parallel to the x axis is not considered because it is understood or assumed to be error free. The orthogonal least-squares regression is an alternative for obtaining symmetric treatments because it minimizes the sum of quadratic orthogonal distances from the points to the fitted line. The method is also related to the first-principal component computation, as it is shown here.

Estimation of Spatial Influence Models Using Mixed-integer Programming

Estimation of ecosystem models is an important task and many studies have been carried out on the problem. However, estimating some models may be difficult. Here, we want to estimate two nonlinear spatial influence models by using the classical least-squares method, and that requires the solution of difficult nonlinear optimization problems. The aim of this paper is to show that both models can be efficiently estimated using mathematical programming. The estimation problems are first formulated as nonlinear optimization problems which are then transformed into convex quadratic mixed-integer programs. The transformation is based on the discretization of some variables and on the linearization of the product of a Boolean variable with a real variable. The approach which allows to find the best estimation with a certain precision and in the least-squares framework, is interesting for several reasons: the definition of the mathematical programs is relatively simple, they are easy to implement using a mathematical programming language together with a quadratic mixed-integer programming software, and computational experiments carried out on large sets of simulated data show the excellent performance of the approach. Moreover, the ideas underlying the method can be used for other difficult least-squares estimations. These results suggest that mixed-integer programming may be an efficient tool for practitioners and researchers in environmental modeling.

Application and Comparison of Robust Linear Regression Methods for Trend Estimation

Abstract In this study, robust parametric regression methods are applied to temperature and precipitation time series in Switzerland and the trend results are compared with trends from classical least squares (LS) regression and nonparametric approaches. It is found that in individual time series statistically outlying observations are present that influence the LS trend estimate severely. In some cases, these outlying observations lead to an over-/underestimation of the trends or even to a trend masking. In comparison with the classical LS method and standard nonparametric techniques, the use of robust methods yields more reliable trend estimations and outlier detection.

Nonparametric inference of discretely sampled stable Lévy processes

We study nonparametric inference of stochastic models driven by stable Lévy processes. We introduce a nonparametric estimator of the stable index that achieves the parametric n rate of convergence. For the volatility function, due to the heavy-tailedness, the classical least-squares method is not applicable. We then propose a nonparametric least-absolute-deviation or median-quantile estimator and study its asymptotic behavior, including asymptotic normality and maximal deviations, by establishing a representation of Bahadur–Kiefer type. The result is applied to several major foreign exchange rates.

Regularized Estimation of Retinal Vascular Oxygen Tension From Phosphorescence Images

The level of retinal oxygenation is potentially an important cue to the onset or presence of some common retinal diseases. An improved method for assessing oxygen tension in retinal blood vessels from phosphorescence lifetime imaging data is reported in this paper. The optimum estimate for phosphorescence lifetime and oxygen tension is obtained by regularizing the least-squares (LS) method. The estimation method is implemented with an iterative algorithm to minimize a regularized LS cost function. The effectiveness of the proposed method is demonstrated by applying it to simulated data as well as image data acquired from rat retinas. The method is shown to yield estimates that are robust to noise and whose variance is lower than that obtained with the classical LS method.

Model Estimation of Cerebral Hemodynamics Between Blood Flow and Volume Changes: A Data-Based Modeling Approach

It is well known that there is a dynamic relationship between cerebral blood flow (CBF) and cerebral blood volume (CBV). With increasing applications of functional MRI, where the blood oxygen-level-dependent signals are recorded, the understanding and accurate modeling of the hemodynamic relationship between CBF and CBV becomes increasingly important. This study presents an empirical and data-based modeling framework for model identification from CBF and CBV experimental data. It is shown that the relationship between the changes in CBF and CBV can be described using a parsimonious autoregressive with exogenous input model structure. It is observed that neither the ordinary least-squares (LS) method nor the classical total least-squares (TLS) method can produce accurate estimates from the original noisy CBF and CBV data. A regularized total least-squares (RTLS) method is thus introduced and extended to solve such an error-in-the-variables problem. Quantitative results show that the RTLS method works very well on the noisy CBF and CBV data. Finally, a combination of RTLS with a filtering method can lead to a parsimonious but very effective model that can characterize the relationship between the changes in CBF and CBV.