Linear Algebra Concepts Research Articles

Analytical solutions to the optimization of a quadratic cost function subject to linear and quadratic equality constraints

In the area of broad-band antenna array signal processing, the global minimum of a quadratic equality constrained quadratic cost minimization problem is often required. The problem posed is usually characterized by a large optimization space (around 50–90 tuples), a large number of linear equality constraints, and a few quadratic equality constraints each having very low rank quadratic constraint matrices. Two main difficulties arise in this class of problem. Firstly, the feasibility region is nonconvex and multiple local minima abound. This makes conventional numerical search techniques unattractive as they are unable to locate the global optimum consistently (unless a finite search area is specified). Secondly, the large optimization space makes the use of decision-method algorithms for the theory of the reals unattractive. This is because these algorithms involve the solution of the roots of univariate polynomials of order to the square of the optimization space. In this paper we present a new algorithm which exploits the structure of the constraints to reduce the optimization space to a more manageable size. The new algorithm relies on linear-algebra concepts, basic optimization theory, and a multivariate polynomial root-solving tool often used by decision-method algorithms.

A NEURAL NETWORK PRIMER

Neural networks are composed of basic units somewhat analogous to neurons. These units are linked to each other by connections whose strength is modifiable as a result of a learning process or algorithm. Each of these units integrates independently (in paral lel) the information provided by its synapses in order to evaluate its state of activation. The unit response is then a linear or nonlinear function of its activation. Linear algebra concepts are used, in general, to analyze linear units, with eigenvectors and eigenvalues being the core concepts involved. This analysis makes clear the strong similarity between linear neural networks and the general linear model developed by statisticians. The linear models presented here are the perceptron and the linear associator. The behavior of nonlinear networks can be described within the framework of optimization and approximation techniques with dynamical systems (e.g., like those used to model spin glasses). One of the main notions used with nonlinear unit networks is the notion of attractor. When the task of the network is to associate a response with some specific input patterns, the most popular nonlinear technique consists of using hidden layers of neurons trained with back-propagation of error. The nonlinear models presented are the Hopfield network, the Boltzmann machine, the back-propagation network and the radial basis function network.

Advanced Calculus With Applications in Statistics

Preface. Preface to the First Edition. 1. An Introduction to Set Theory. 1.1. The Concept of a Set. 1.2. Set Operations. 1.3. Relations and Functions. 1.4. Finite, Countable, and Uncountable Sets. 1.5. Bounded Sets. 1.6. Some Basic Topological Concepts. 1.7. Examples in Probability and Statistics. 2. Basic Concepts in Linear Algebra. 2.1. Vector Spaces and Subspaces. 2.2. Linear Transformations. 2.3. Matrices and Determinants. 2.4. Applications of Matrices in Statistics. 3. Limits and Continuity of Functions. 3.1. Limits of a Function. 3.2. Some Properties Associated with Limits of Functions. 3.3. The o, O Notation. 3.4. Continuous Functions. 3.5. Inverse Functions. 3.6. Convex Functions. 3.7. Continuous and Convex Functions in Statistics. 4. Differentiation. 4.1. The Derivative of a Function. 4.2. The Mean Value Theorem. 4.3. Taylor's Theorem. 4.4. Maxima and Minima of a Function. 4.5. Applications in Statistics. 5. Infinite Sequences and Series. 5.1. Infinite Sequences. 5.2. Infinite Series. 5.3. Sequences and Series of Functions. 5.4. Power Series. 5.5. Sequences and Series of Matrices. 5.6. Applications in Statistics. 6. Integration. 6.1. Some Basic Definitions. 6.2. The Existence of the Riemann Integral. 6.3. Some Classes of Functions That Are Riemann Integrable. 6.4. Properties of the Riemann Integral. 6.5. Improper Riemann Integrals. 6.6. Convergence of a Sequence of Riemann Integrals. 6.7. Some Fundamental Inequalities. 6.8. RiemannStieltjes Integral. 6.9. Applications in Statistics. 7. Multidimensional Calculus. 7.1. Some Basic Definitions. 7.2. Limits of a Multivariable Function. 7.3. Continuity of a Multivariable Function. 7.4. Derivatives of a Multivariable Function. 7.5. Taylor's Theorem for a Multivariable Function. 7.6. Inverse and Implicit Function Theorems. 7.7. Optima of a Multivariable Function. 7.8. The Method of Lagrange Multipliers. 7.9. The Riemann Integral of a Multivariable Function. 7.10. Differentiation under the Integral Sign. 7.11. Applications in Statistics. 8. Optimization in Statistics. 8.1. The Gradient Methods. 8.2. The Direct Search Methods. 8.3. Optimization Techniques in Response Surface Methodology. 8.4. Response Surface Designs. 8.5. Alphabetic Optimality of Designs. 8.6. Designs for Nonlinear Models. 8.7. Multiresponse Optimization. 8.8. Maximum Likelihood Estimation and the EM Algorithm. 8.9. Minimum Norm Quadratic Unbiased Estimation of Variance Components. 8.10. Scheffe's Confidence Intervals. 9. Approximation of Functions. 9.1. Weierstrass Approximation. 9.2. Approximation by Polynomial Interpolation. 9.3. Approximation by Spline Functions. 9.4. Applications in Statistics. 10. Orthogonal Polynomials. 10.1. Introduction. 10.2. Legendre Polynomials. 10.3. Jacobi Polynomials. 10.4. Chebyshev Polynomials. 10.5. Hermite Polynomials. 10.6. Laguerre Polynomials. 10.7. Least-Squares Approximation with Orthogonal Polynomials. 10.8. Orthogonal Polynomials Defined on a Finite Set. 10.9. Applications in Statistics. 11. Fourier Series. 11.1. Introduction. 11.2. Convergence of Fourier Series. 11.3. Differentiation and Integration of Fourier Series. 11.4. The Fourier Integral. 11.5. Approximation of Functions by Trigonometric Polynomials. 11.6. The Fourier Transform. 11.7. Applications in Statistics. 12. Approximation of Integrals. 12.1. The Trapezoidal Method. 12.2. Simpson's Method. 12.3. NewtonCotes Methods. 12.4. Gaussian Quadrature. 12.5. Approximation over an Infinite Interval. 12.6. The Method of Laplace. 12.7. Multiple Integrals. 12.8. The Monte Carlo Method. 12.9. Applications in Statistics. Appendix. Solutions to Selected Exercises. General Bibliography. Index.

Matrices with Rank Deficiency Two in Eigenvalue Problems and Dynamical Systems

Let $K(\alpha ,\beta )$ ($\alpha $, $\beta $ real) be a family of real square matrices. Several computational problems are equivalent to the calculation of a pair $(\alpha ^0 ,\beta ^0 )$ of parameter values for which $K(\alpha ^0 ,\beta ^0 )$ has rank deficiency 2. Among these problems are the computation of a conjugate pair of complex eigenvalues of a given real matrix, the computation of a Hopf bifurcation point on a branch of stationary solutions to a parametrized differential equation, and the computation of a Takens–Bogdanov point in the two-dimensional solution manifold of a set of nonlinear equations. Developing ideas of Griewank and Reddien, the authors define scalar functions $g_1 (\alpha ,\beta ),g_2 (\alpha ,\beta )$ that vanish in $(\alpha ^0 ,\beta ^0 )$. A nondegeneracy condition NDC, which expresses the fact that $(\alpha ^0 ,\beta ^0 )$ is in a natural sense an isolated point in $(\alpha ,\beta )$-space, is introduced. It is proved that under certain conditions on the family $K(\alpha ,\beta )$ the Jacobian of $g_1 $, $g_2 $ with respect to $\alpha $, $\beta $ is nonsingular if and only if NDC holds. A Newton method to compute $(\alpha ^0 ,\beta ^0 )$ is then described. The problems mentioned above and some related ones are analysed in detail. The derived algorithms for the dynamical systems problems have the following features: (1) they are simple and natural, being based on linear algebra concepts only; (2) they treat the two parameters in a symmetric way; (3) they do not lead to formally large systems; and (4) NDC is expressed in terms independent of the particular problem. Numerical results are given which illustrate the quadratic convergence of the Newton algorithm during the computation of a Hopf bifurcation point arising in a model of a tubular reactor.

Deconvolutions based on singular value decomposition and the pseudoinverse: a guide for beginners.

Singular value decomposition (SVD) is deeply rooted in the theory of linear algebra, and because of this is not readily understood by a large group of researchers who could profit from its application. In this paper, we discuss the subject on a level that should be understandable to scientists who are not well versed in linear algebra. However, because it is necessary that certain key concepts in linear algebra be appreciated in order to comprehend what is accomplished by SVD, we present the section, 'Bare basics of linear algebra'. This is followed by a discussion of the theory of SVD. Next we present step-by-step examples to illustrate how SVD is applied to deconvolute a titration involving a mixture of three pH indicators. One noiseless case is presented as well as two cases where either a fixed or varying noise level is present. Finally, we discuss additional deconvolutions of mixed spectra based on the use of the pseudoinverse.

Econometrics: Theory and Applications

This volume deals with problems of estimating and testing socio-economic relations arising in single and simultaneous equations. It incorporates applications from diverse areas including industrial organization, labour, education, public finance, trade, money, and financial economics. Ghosh discusses some of the most recent techniques and models in the discipline and provides an extensive survey of real-world applied econometric studies. The text offers a discussion of an economic problem in the introduction to most chapters. It includes a brief presentation of linear algebra and statistical concepts and results and also presents some classic and modern applied studies in consumption, production and investment functions, applied macro-econometric models and models of rational expectations. The book contains easy-to-follow discussions on the most recent econometric topics such as modelling or model selection, sample selectivity bias, Wald, Likelihood Ratio, and Lagrangean Multiplier tests. The book includes illustrations of equations, models, flow charts, tables of statistical results and appendices as well as a bibliography, end-of-chapter problems and exercises drawn from different socio-economic areas.

Dual-scale topology optoelectronic processor

The dual-scale topology optoelectronic processor (D-STOP) is a parallel optoelectronic architecture for matrix algebraic processing. The architecture can be used for matrix-vector multiplication and two types of vector outer product. The computations are performed electronically, which allows multiplication and summation concepts in linear algebra to be generalized to various nonlinear or symbolic operations. This generalization permits the application of D-STOP to many computational problems. The architecture uses a minimum number of optical transmitters, which thereby reduces fabrication requirements while maintaining area-efficient electronics. The necessary optical interconnections are space invariant, minimizing space-bandwidth requirements.

Introduction to the Theory and Practice of Econometrics

This book interweaves inferential approaches and theory and practice in econometrics. Basic statistical and linear algebra concepts are introduced as they are needed to give life to the statistical model under study. Most econometric applications start with a tentative theory or hypothesis a sample of data and the goal of learning something about the phenomena under study from the limited set of observations. Therefore a sample of data that may be used to investigate a particular economic hypothesis is presented to motivate the analysis of each of the statistical models presented. This linkage between the economic process that is thought to have generated the data and a particular statistical model is a unifying theme throughout the book. It progresses from the special case of investigating the possibilities for determining the location and scale parameters for a population from a sample of observations to investigating a complex simultaneous system of structural equations under general stochastic assumptions. To ensure that the reader understands the basic concepts and conclusions as they relate to linear statistical models simple special case models are evaluated and then the analysis is repeated for the general case. The 1st half of book gives the student a solid introduction to the formulation and use of linear statistical models. The 2nd half introduces the student to the econometric problems that arise when it is taken into account that economic data are stochastic dynamic and simultaneous and that the optimal statistical procedure sometimes changes as we change the statistical model the amount and type of information used and the measure of performance.

On the performance analysis of the MVDR beamformer in the presence of correlated interference

Expressions are developed that describe the output of an adaptive array, based on the minimum variance distortionless response (MVDR) beamformer when multiple correlated interfering signals are present. Such signals may arise in certain multipath scenarios or in the case of smart jamming. Previous analyses considered only the case of single correlated interferer. Three scenarios of interest are examined: very high signal-to-noise ratio (SNR), moderately high SNR, and very low SNR. The analysis is based on simpler linear algebra concepts and generalizes a number of previously reported results proved for special cases. In addition, a modification of MVDR beamforming based on total least squares is introduced as a means of mitigating the sensitivity of the MVDR beamformer to uncertainties in element positions, pointing angle, etc. >

Symmetry and Positive Definiteness in Oriented Matroids

We show the properties of a class of oriented matroids that properly generalizes the class of oriented matroids that can be represented by matrices of the form ( I , − A ), where A is a real n × n symmetric matrix and I is the n × n identity matrix. Several results from linear algebra about positive definiteness, symmetry, and eigenvalues are shown to have natural generalizations in the context of oriented matroids. The relationships among the oriented matroid generalizations of the linear algebraic concepts are seen to be analogous to the relationships among the original linear algebraic concepts.

Functional behavior of nondeterministic and concurrent programs

The functions behavior of a deterministic program segment is a function f : D → D , where D is some set of states for the computation. This notion of functional behavior can be extended to nondeterministic and concurrent programs using techniques from linear algebra. In particular, the functional behavior of a nondeterministic program segment is a linear transformation f : A → A , where A is a free semiring module. Other notions from linear algebra carry over into this setting. For example, weakest preconditions and predicate transformers correspond to well-studied concepts in linear algebra. Using multilinear algebra, programs with tuples of inputs and outputs can be handled. For nondeterministic concurrent programs, the functional behavior is a linear transformation f : A → A , where A is a free semiring algebra. In this case, f may also be an algebra morphism, which indicates that the program involves no interprocess communication. Finally, a model of syntax for programs is studied whose semantics is given using linear algebra. It is shown that in this model, free interpretations (essentially Herbrand universes) do not generally exist.

An Elementary Approach to Jordan Theory

The usual proofs of the existence and uniqueness of the Jordan canonical form employ a certain amount of machinery: module theory, quotient -spaces, or at least a fairly sophisticated manipulation of polynomial matrices. We offer an alternative and, to the best of our knowledge, new development that uses only the elementary concepts of linear algebra. The distinctive feature of this approach is that it works with what we call matrices, rather than with nilpotent operators. As usual, we define the k x k X-Jordan block (or X-block) to be the k x k matrix

Algebraic structure theory of stochastic machines

In the present paper, an algebraic structure theory of stochastic machines is developed which contains both Bacon's theory [1] and Hartmanis' theory for deterministic machines [5] as special cases. Although the present theory is patterned after both theories, it exhibits many interesting features which are here-to-fore unknown. Among other things, it is shown that the basic concepts involved, e.g., partitions with substitution property, partition pairs, state-behavior realizations, are all equivalent to well-known concepts in linear algebra. Section II introduces the model of stochastic machines to be used in subsequent developments. In Section III, various types of realizations and assignments are introduced. The concepts of partitions with substitution property, partition pairs, etc. are introduced in Section III. The last section gives a formulation of a network of concurrently operating QSMs.