Orthogonal Least Squares Regression Research Articles

Supervised Dimensionality Reduction Methods via Recursive Regression.

In this article, the recursive problems of both orthogonal linear discriminant analysis (OLDA) and orthogonal least squares regression (OLSR) are investigated. Different from other works, the associated recursive problems are addressed via a novel recursive regression method, which achieves the dimensionality reduction in the orthogonal complement space heuristically. As for the OLDA, an efficient method is developed to obtain the associated optimal subspace, which is closely related to the orthonormal basis of the optimal solution to the ridge regression. As for the OLSR, the scalable subspace is introduced to build up an original OLSR with optimal scaling (OS). Through further relaxing the proposed problem into a convex parameterized orthogonal quadratic problem, an effective approach is derived, such that not only the optimal subspace can be achieved but also the OS could be obtained automatically. Accordingly, two supervised dimensionality reduction methods are proposed via obtaining the heuristic solutions to the recursive problems of the OLDA and the OLSR.

Feature selection under regularized orthogonal least square regression with optimal scaling

Due to lack of scale change in orthogonal least square regression (OLSR), the scaling term is introduced to OLSR to build up a novel orthogonal least square regression with optimal scaling (OLSR-OS) problem in this paper. In addition, the proposed OLSR-OS problem is proven to be numerically better than the OLSR problem. In order to select relevant features under the proposed OLSR-OS problem, ℓ2, 1-norm regularization is further introduced, such that row-sparse projection is achieved. Accordingly, a novel parameterized expansion balanced feature selection (PEB-FS) method is derived based on an extension balanced counterpart. Moreover, not only the convergence of the proposed PEB-FS method is provided but the optimal scaling can be automatically achieved as well. Consequently, the effectiveness and the superiority of the proposed PEB-FS method are verified both theoretically and experimentally.

A generalized power iteration method for solving quadratic problem on the Stiefel manifold

In this paper, we first propose a novel generalized power iteration method (GPI) to solve the quadratic problem on the Stiefel manifold (QPSM) as min_{W^TW=I}Tr(W^TAW-2W^TB) along with the theoretical analysis. Accordingly, its special case known as the orthogonal least square regression (OLSR) is under further investigation. Based on the aforementioned studies, we then cast major focus on solving the unbalanced orthogonal procrustes problem (UOPP). As a result, not only a general convergent algorithm is derived theoretically but the efficiency of the proposed approach is verified empirically as well.

Tree Based Orthogonal Least Squares Regression with Repeated Weighted Boosting Search

Orthogonal Least Squares Regression (OLSR) selects each regressor by repeated weighted boosting search (RWBS). This kind of OLSR is known to be capable of producing a much sparser model than many other kernel methods. With the aid of tree structure search, this paper is to construct an even sparser regression model in the framework of OLSR with RWBS. When RWBS being used to solve the optimization at each regression stage, OLSR is extended by keeping the k ( k >1)excellent regressors, which minimize the modeling MSE, rather than only choose the best one at each iteration. In this way, the next regressor will be searched in k subspaces instead of in only one subspace as the conventional method. Furthermore we propose a subtree search to decrease experimental time complexity, by specifying the total number of children in every tree depth. The new schemes are shown to outperform the traditional method in the applications, such as component detection, sparse representation for ECG signal and 2-d time series modeling. Besides, experimental results also indicate that subtree based algorithm is with much lower time complexity than tree based one.

Two-Variable Linear Regression: Modeling with Orthogonal Least-Squares Analysis

In many fields of chemistry, the ordinary least-squares method is preferentially used to fit data. Nevertheless, univariate linear regression by least-squares analysis, as devised by Gauss and Legendre, has some drawbacks that are usually overlooked in experimental science courses and even in many chemical research papers. Orthogonal least-squares fitting is a good method to avoid the unsymmetrical treatment of the data and also to avoid effects of regression toward the mean. These result because the classical least-squares method only minimizes the squared distance parallel to the y axis between the experimental points and the fitting line, whereas the distance parallel to the x axis is not considered because it is understood or assumed to be error free. The orthogonal least-squares regression is an alternative for obtaining symmetric treatments because it minimizes the sum of quadratic orthogonal distances from the points to the fitted line. The method is also related to the first-principal component computation, as it is shown here.

Relationship between the metabolism of antipyrine, hexobarbital and theophylline in patients with liver disease as assessed by a ‘cocktail’ approach

Antipyrine (AP), hexobarbital (HB) and theophylline (TH) were administered simultaneously ('cocktail' design) to 24 patients with various types of liver disease. Clearance (Cl) of AP, HB and TH and formation clearance of the AP-metabolites 3-hydroxymethylantipyrine (HMA), norantipyrine (NORA) and 4-hydroxyantipyrine (OHA) were determined and correlation coefficients and orthogonal least-squares regression lines calculated between the clearance and formation clearance parameters. The results were compared with those obtained in a study in which the same 'cocktail' was administered to 26 healthy control subjects. In the patients ClAP, ClHB and ClTH were 23.0 +/- 14.3 ml min-1, 206 +/- 128 ml min-1 and 39.9 +/- 26.1 ml min-1 respectively. All values were considerably lower than those found in the control subjects. With regard to AP metabolism preferential impairment of NORA formation was observed. Relatively high correlation coefficients were found between ClAP, ClHB and ClTH, which suggests, like the results of orthogonal regression analysis, a strong correlation between total metabolism of these probe drugs. Therefore it is likely that impairment in oxidation in patients with liver disease not only leads to reduction in clearance but also to reduced substrate selectivity of cytochrome P-450 isozymes.