Least-squares Theory Research Articles

On the Use of the Generalised Extreme-Value Distribution in Estimating Extreme Percentiles

The application of extreme-value theory for estimation of extreme percentiles of arbitrary continuous distributions is discussed and the use of a generalised extremevalue distribution is proposed. This distribution is given in a form containing three parameters: a location parameter, a scale parameter and a shape parameter. A method of estimating the parameters of the distribution is developed. It uses order statistics and general least-squares theory and is shown to be suitable for censored as well as uncensored samples. Tables of expectations of order statistics for certain sample sizes and certain values of the shape parameter are provided. An application, illustrating the numerical procedures is given, using data from the field of Applied Physiology.

A General Computer Programme for the Analysis of Factorial Experiments

A general programme for the analysis of factorial experiments is described. The programme was written in Extended Mercury Autocode for the I.C.T. Orion computer at Rothamsted. It can deal with experiments with up to seven factors, including those with complete or partial confounding and multiple splits, arranged in randomised blocks, Latin or quasi-Latin squares, or criss-cross designs. Partially confounded contrasts have to be specified by the user, but for the commoner types of design the correct arrangement of the analysis of variance is provided automatically. Properly adjusted tables of treatment means are produced and covariance and missing values are provided for. An outline of the least-square theory of confounded designs is appended. Improvements incorporated in a revised version of the programme are described in an addendum.

Linear Statistical Inference and Its Applications

R. Rao would be found in almost any statistician's list of five outstanding workers in the world of Mathematical Statistics today. His book represents a comprehensive account of the main body of results that comprise modern statistical theory. -W. G. Cochran [C. R. Rao is] one of the pioneers who laid the foundations of statistics which grew from ad hoc origins into a firmly grounded mathematical science. -B. Efrom Translated into six major languages of the world, C. R. Rao's Linear Statistical Inference and Its Applications is one of the foremost works in statistical inference in the literature. Incorporating the important developments in the subject that have taken place in the last three decades, this paperback reprint of his classic work on statistical inference remains highly applicable to statistical analysis. Presenting the theory and techniques of statistical inference in a logically integrated and practical form, it covers: * The algebra of vectors and matrices * Probability theory, tools, and techniques * Continuous probability models * The theory of least squares and the analysis of variance * Criteria and methods of estimation * Large sample theory and methods * The theory of statistical inference * Multivariate normal distribution Written for the student and professional with a basic knowledge of statistics, this practical paperback edition gives this industry standard new life as a key resource for practicing statisticians and statisticians-in-training.

The theory of least squares when the parameters are stochastic and its application to the analysis of growth curves

Journal Article The theory of least squares when the parameters are stochastic and its application to the analysis of growth curves Get access C. RADHAKRISHNA RAO C. RADHAKRISHNA RAO Indian Statistical InstituteCalcutta Search for other works by this author on: Oxford Academic Google Scholar Biometrika, Volume 52, Issue 3-4, December 1965, Pages 447–458, https://doi.org/10.1093/biomet/52.3-4.447 Published: 01 December 1965

The Theory of Least Squares When the Parameters are Stochastic and Its Application to the Analysis of Growth Curves

The Theory of Least Squares When the Parameters are Stochastic and Its Application to the Analysis of Growth Curves

Robust Estimation of a Location Parameter

Robust Estimation of a Location Parameter

A Note on a Generalized Inverse of a Matrix with Applications to Problems in Mathematical Statistics

SUMMARY Some years ago the author defined a pseudo inverse of a singular matrix and used it in representing a solution of normal equations and for obtaining variances and covariances of estimates in the theory of least squares (Rao, 1955). This provided a unified approach to least squares theory, including the case when the normal equations become singular. This note attempts to collect a few mathematical results, some of which are known in literature, associated with the inversion of singular and rectangular matrices, and to indicate briefly their use in problems of mathematical statistics.

A variant least-squares method of solution of a system of observation equations

Introduction. The theory of least squares is specifically designed for application to systems in which the probability distribution of the errors is known to be gaussian and free of systematic errors, Consequently one seeks to remove any observation which has a systematic error or a large random error. In this note we shall show a mathematical device for using a suspect observation equation as a quasi-normal equation, so that if desired, this equation which is placed last in the normal equations, may be dropped from the solution, thereby precluding the necessity for the formation of a new set of normal equations and a new forward solution for the effect of the removal of the suspected observation.

A theory for determining upper-atmosphere winds from radio observations on meteor trails

Least-squares theory is applied to observations on drifting meteor trails to obtain the determination of a general number of parameters, characterizing the wind structure. The results enable vertical air motion and time- and height-variations in the wind structure to be examined, whereas previously a constant horizontal wind was assumed. It is shown how, on the basis of the least-squares solution, the errors in the parameters, and hence in the calculated wind structure, can be estimated. The method is applied to a typical set of data.

Statistical design and analysis of multiply-instrumented control systems

It is the purpose of this paper to show how Wiener's linear least-square filter theory may be applied to some common types of multiply-instrumented control systems. By multiply-instrumented control systems are meant those in which more than one sensing instrument is used.

The rôle of cracovians in astronomy

The importance and usefulness of cracovians—matrices obeying special rules of multiplication—is made evident in the treatment of astronomical problems. To show this, simple applications are dealt with in the fields of spherical astronomy, transformation of co-ordinates, and the theory of least squares. The discussion emphasizes the theoretical interest of cracovians, and brings out the simplicity, ease, and reliability of cracovian operations.

De theoretische zijde van de methode der kleinste kwadraten

SummaryThe theoretical aspect of least squares.This article contains a slightly modified presentation of the Markoff theory of least squares as developed along different lines by Aitken and by David and Neyman. The modifications aim at a more complete treatment and a geometrical illustration of the connection between best linear estimates and generalized least squares. The unbiasedness of ordinary least squares estimates in the case of heteroscedastic and correlated errors is stressed and the loss of efficiency is shown to be generally small. Topics like orthogonalization, partial correlation and what is called “over‐correlation” are treated in passing.Matrices are constantly used, being the adequate tools in this matter. In the appendix a special relevant matrix theorem is derived, viz. a generalization of the well known Cauchy inequality.

On the Theory of Prediction of Nonstationary Stochastic Processes

We consider the following problem of prediction: During a finite time interval T the real valued function S(t)+N(t) is observed, in which S(t) is a signal and N(t) is a linearly superimposed noise disturbance. The problem is to predict the value of a given linear functional of S(t), the predictor formula having certain preassigned ``optimum properties'' among a certain class of predictors. In the case in which the mean value of S(t) is known, the random components of S(t) and N(t) are strictly stationary, and the time interval T is infinite, a complete solution to this problem has been given by N. Wiener. (In the case of discrete time series, the solution was given by A. Kolmogoroff.) This theory has been extended by L. Zadeh and J. Ragazzini [J. Appl. Phys. 21, 645 (1950] to the case in which T is a finite time interval and the mean value of S(t) is unknown but is restricted to be a polynomial in time. We extend the above theories to the case in which the random components of both S(t) and N(t) are nonstationary in time and merely possess finite continuous covariance and cross-covariance functions. The analytical tools of probability theory which we use are those developed independently by M. Loève and K. Karhunen in their studies of a class of stochastic processes usually termed processes of second order. Apparently these techniques are practically unknown outside of certain mathematical circles. In the opinion of the author these techniques are extremely powerful in the analysis of transient random phenomena in linear systems. Finally we give an exposition of a method of prediction by the theory of conditional probabilities. This method is applicable when the form of the joint probability distribution of signal and noise is known and hence is applicable to many practical problems, since this distribution is often Gaussian. In this particular case the predictor formula given by the theory of conditional probabilities is identical with the usual linear predictor formula given by the theory of least squares. In case the signal is a Markoff process, the method of conditional probabilities yields a much simpler prediction formula than the usual method of least squares.

Solution of simultaneous linear equations by the method of relaxation

Introduction. In surveying problems involving the use of the theory of least squares a penultimate step in the numerical computation is the solving of normal equations in several unknowns.

Nonindependent Observational Equations in the Theory of Least Squares

Nonindependent Observational Equations in the Theory of Least Squares

Total and uncharged nuclei at Washington, D.C.

Observations of the concentration of both uncharged and total condensation‐nuclei were made at different times and places near Washington, using an Aitken nuclei‐counter in conjunction with a cylindrical, electrical condenser through which air was drawn at a desired rate by a spring‐driven turbine. The ratio (S) of the concentration of uncharged to that of all nuclei serves as a measure of the relative magnitude of the coefficients of combination and those of attachment between small ions and charged and uncharged nuclei, respectively, provided the assumptions usually made in this connection are valid. Diverse values of S have been found heretofore when these were determined from the same data but by different methods. It is shown in the theory of least squares that the method of determining the best estimate of a quantity of this character depends upon the manner in which weights should be assigned to the data. It is shown that the weight assigned to the observed number of particles falling on a square of the counter should vary inversely as that number. The proper method of determining the best estimate of S, when weights are assigned in this manner, and also two other simpler methods were applied to these data. The surprisingly small differences in the three results indicate these data are homogeneous.

Total and uncharged nuclei at Washington, D.C.

Observations of the concentration of both uncharged and total condensation‐nuclei were made at different times and places near Washington, using an Aitken nuclei‐counter in conjunction with a cylindrical, electrical condenser through which air was drawn at a desired rate by a spring‐driven turbine. The ratio (S) of the concentration of uncharged to that of all nuclei, serves as a measure of the relative magnitude of the coefficients of combination and those of attachment between small ions and charged and uncharged nuclei, respectively, provided the assumptions usually made in this connection are valid. Diverse values of S have been found heretofore when these were determined from the same data but by different methods. It is shown in the theory of least squares that the method of determining the best estimate of a quantity of this character depends upon the manner in which weights should be assigned to the data. It is shown here that the weight assigned to the observed number of particles falling on a square of the counter, should vary inversely as that number. The proper method of determining the best estimate of S, when weights are assigned in this manner, and also two other simpler methods were applied to these data. The surprisingly small differences in the three results, indicates these data are homogeneous.From 620 individual samples of both uncharged and total nuclei, S was found to be 0.75 with a standard error of 0.02. This value of S is distinctly greater than that reported for other places but it is in close agreement with values previously determined at Washington with other equipment. No significant evidence of a dependence of S, upon time, upon place, or upon concentration of nuclei, such as some observers report, is provided by these data. An examination of the standard deviation of S for a single sample indicates that the variations which do appear are no greater than those which are to be expected from random errors. Systematic errors in all measurements of this sort may arise from the adsorption of nuclei on the walls of the apparatus. The differences in the values of S found at the different places may in part be attributed to this source. Four hundred individual samples taken, both before and after the air passed through the condenser with no field applied, indicate that 0.85 of the nuclei which enter this condenser get through. The procedure followed here was such that errors in the calculated value of S from this source are eliminated provided charged and uncharged nuclei are adsorbed with equal facility.

An Application of Orthogonalization Process to the Theory of Least Squares

An Application of Orthogonalization Process to the Theory of Least Squares

Carl Friedrich Gauss

At an early age, Gauss showed unusual ability in mathematics. In fact, some say that he was only three when he corrected his father's calculations of the pay due men working under him. The father was a brick layer and Gauss was brought up in circumstances that barely escaped poverty. At the elementary school in Brunswick, he attracted the attention of an assistant teacher Bartels who was later to become professor of mathematics at Kasan in Russia and then at Dorpat in Germany. At Brunswick, Bartels' duties included cutting quill pens for the younger boys and helping them with their writing. Bartels read mathematics with Gauss and introduced him to the binomial theorem and to infinite series when Gauss was only twelve. Gauss attended the gymnasium in Brunswick and in 1792 with the financial support of the duke of Brunswick who had become interested in him, he went to Caroline College in Brunswick and later to GÖttingen. At Caroline College Gauss worked in mathematics and in languages. When he entered the university in 1795, he had made progress in the theory of least squares, but he was still undecided whether to work in philology or in mathematics. His career to this point has been called his “prehistoric period.” During this time, his study in the theory of numbers was largely experimental—the assembling of many cases, the forming of a rule, the proving of the theorem.

On the theory of errors and least squares

1. In my “Scientific Inference,” chapter V, I found that the usual presentation of the theory of errors of observation needed some modification, even where the probability of error is distributed according to the normal law. One change made was in the distribution of the prior probability of the precision constant h . Whereas this is usually taken as uniform (or ignored), I considered it better to assume that the prior probability that the constant lies in a range dh is proportional to dh/h . This is equivalent to assuming that if h 1 / h 2 = h 3 / h 4 , h is as likely to lie between h 1 and h 2 as between h 3 and h 4 ; this was thought to be the best way of expressing the condition that there is no previous know­ ledge of the magnitude of the errors. The relation must break down for very small h , comparable with the reciprocal of the whole length of the scale used, and for large h comparable with the reciprocal of the step of the scale; but for the range of practically admissible values it appeared to be the most plausible distribution. The argument for this law can now be expressed in an alternative form. The normal law of error is supposed to hold, but the true value x and the pre­cision constant h are unknown. Two measures are made: what is the pro ability that the third observation will lie between them ? The answer is easily seen to be one-third.