Conventional Least-squares Regression Research Articles

Imbalanced least squares regression with adaptive weight learning

Least squares regression (LSR) has demonstrated promising performance in various classification tasks owing to its effectiveness and efficiency. However, there are some deficiencies that seriously hinder its application in imbalanced data scenarios. The first is that LSR strongly relies on a balanced class distribution. A severely imbalanced class distribution may seriously damage the effectiveness of the algorithm. Second, the utilized binary label matrix in the conventional LSR model may be too strict to learn a discriminative transformation matrix for imbalanced learning. To address the above issues, in this paper, an adaptive weight learning mechanism and label relaxation constraint are proposed and incorporated into the framework of LSR to tackle the imbalanced classification problem. The weight of each sample can be adaptively obtained according to the original distribution information of the imbalanced data, in which the importance of minority class samples can be better reflected with larger weights. A new label relaxation matrix consisting of the original label matrix and auxiliary matrix is constructed to widen the margins between different classes. Further, we provide an iterative algorithm with fast convergence to solve the resulting optimization problem. Extensive experimental results on diverse binary-class and multi-class imbalanced datasets show that the proposed method outperforms many other state-of-the-art imbalanced learning approaches.

A new Bayesian approach toward improved regression of low-count U[sbnd]Pb geochronology data generated by LA-ICPMS

UPb dating of hydrogenic and fossil material can be a significant challenge because samples often contain low U levels and variable amounts of non-radiogenic Pb. Line scans using LA-ICPMS provide the widest range of ratios available for defining the radiogenic to common Pb mixing line, hence the best age estimate, but extracting this information requires processing many low-count data. Conventional least-squares regression of ratios based on such data can give inconsistent results. A new approach is introduced where data are regressed as count rates in a 3D space rather than ratios on a 2D plot. This is a maximum likelihood approach in that mixing planes covering a range of ages and common 207Pb/206Pb ratios (PbC) are compared to the data set and used to determine a relative probability density surface whose maximum should fix the best-fit age and common 207Pb/206Pb ratio. 95% confidence errors are determined from T and PbC values that cut off 2.5% of the area to the left and right of 2D plots generated by integration of the 3D plot over PbC and age, respectively. Results from a rugose coral fossil show that this approach gives age results similar to regressions of concordia data and more accurate than Tera-Wasserburg regressions on data from natural samples. Tests on randomly generated synthetic data do not show significant discrepancy between Isoplot ages but they do between values of PbC. The 3D approach is the only one that gives accurate results for both age and PbC.

QLBS: Q-Learner in the Black-Scholes(-Merton) Worlds

This article presents a discrete-time option pricing model that is rooted in reinforcement learning (RL), and more specifically in the famous Q-Learning method of RL. We construct a risk-adjusted Markov Decision Process for a discrete-time version of the classical Black-Scholes-Merton (BSM) model, where the option price is an optimal Q-function, while the optimal hedge is a second argument of this optimal Q-function, so that both the price and hedge are parts of the same formula. Pricing is done by learning to dynamically optimize risk-adjusted returns for an option replicating portfolio, as in Markowitz portfolio theory. Using Q-Learning and related methods, once created in a parametric setting, the model can go model-free and learn to price and hedge an option directly from data, without an explicit model of the world. This suggests that RL may provide efficient data-driven and model-free methods for the optimal pricing and hedging of options. Once we depart from the academic continuous-time limit, and vice versa, option pricing methods developed in Mathematical Finance may be viewed as special cases of model-based reinforcement learning. Further, due to the simplicity and tractability of our model, which only needs basic linear algebra (plus Monte Carlo simulation, if we work with synthetic data), and its close relationship to the original BSM model, we suggest that our model could be used in the benchmarking of different RL algorithms for financial trading applications. TOPICS:Derivatives, options Key Findings • Reinforcement learning (RL) is the most natural way for pricing and hedging of options that relies directly on data and not on a specific model of asset pricing. • The discrete-time RL approach to option pricing generalizes classical continuous-time methods, enables tracking mis-hedging risk, which disappears in the formal continuous-time limit, and provides a consistent framework for using options for both hedging and speculation. • A simple quadratic reward function, which presents a minimal extension of the classical Black-Scholes framework when combined with the Q-learning method of RL, gives rise to a particularly simple computational scheme where option pricing and hedging are semi-analytical, as they amount to multiple uses of a conventional least-squares regression.

A constrained least squares regression model

Least squares regression (LSR) is a widely used regression technique for multicategory classification. Conventional LSR model assumes that during the learning phase, the labeled samples can be exactly transformed into a discrete label matrix, which is too strict to learn a regression matrix for fitting the labels. To overcome this drawback, lots of LSR’s variants utilize the soft target label, which contains the continuous values, to replace this discrete label to improve the learning performance. Since the regression matrix can be learnt from these soft target labels, it is reasonable to assume that the samples in the same class have similar soft target labels. Nevertheless, most of existing LSR-based models don’t adequately consider this similarity assumption. In this paper, we propose a constrained least squares regression (CLSR) model for multicategory classification. The main motivation of CLSR is to force the samples in the same class to obtain the similar soft target labels. To effectively optimize CLSR, we propose a novel alternating algorithm, which can converge to the globally optimal solution. Extensive experiments results on face and digit data sets confirm the effectiveness of our proposed model.

Diffusion parameter mapping with the combined intravoxel incoherent motion and kurtosis model using artificial neural networks at 3T.

Artificial neural networks (ANNs) were used for voxel-wise parameter estimation with the combined intravoxel incoherent motion (IVIM) and kurtosis model facilitating robust diffusion parameter mapping in the human brain. The proposed ANN approach was compared with conventional least-squares regression (LSR) and state-of-the-art multi-step fitting (LSR-MS) in Monte-Carlo simulations and in vivo in terms of estimation accuracy and precision, number of outliers and sensitivity in the distinction between grey (GM) and white (WM) matter. Both the proposed ANN approach and LSR-MS yielded visually increased parameter map quality. Estimations of all parameters (perfusion fraction f, diffusion coefficient D, pseudo-diffusion coefficient D*, kurtosis K) were in good agreement with the literature using ANN, whereas LSR-MS resulted in D* overestimation and LSR yielded increased values for f and D*, as well as decreased values for K. Using ANN, outliers were reduced for the parameters f (ANN, 1%; LSR-MS, 19%; LSR, 8%), D* (ANN, 21%; LSR-MS, 25%; LSR, 23%) and K (ANN, 0%; LSR-MS, 0%; LSR, 15%). Moreover, ANN enabled significant distinction between GM and WM based on all parameters, whereas LSR facilitated this distinction only based on D and LSR-MS on f, D and K. Overall, the proposed ANN approach was found to be superior to conventional LSR, posing a powerful alternative to the state-of-the-art method LSR-MS with several advantages in the estimation of IVIM-kurtosis parameters, which might facilitate increased applicability of enhanced diffusion models at clinical scan times.

Oxygen extraction fraction mapping at 3 Tesla using an artificial neural network: A feasibility study.

The oxygen extraction fraction (OEF) is an important biomarker for tissue-viability. MRI enables noninvasive estimation of the OEF based on the blood-oxygenation-level-dependent (BOLD) effect. Quantitative OEF-mapping is commonly applied using least-squares regression (LSR) to an analytical tissue model. However, the LSR method has not yet become clinically established due to the necessity for long acquisition times. Artificial neural networks (ANNs) recently have received increasing interest for robust curve-fitting and might pose an alternative to the conventional LSR method for reduced acquisition times. This study presents in vivo OEF mapping results using the conventional LSR and the proposed ANN method. In vivo data of five healthy volunteers and one patient with a primary brain tumor were acquired at 3T using a gradient-echo sampled spin-echo (GESSE) sequence. The ANN was trained with simulated BOLD data. In healthy subjects, the mean OEF was 36 ± 2% (LSR) and 40 ± 1% (ANN). The OEF variance within subjects was reduced from 8% to 6% using the ANN method. In the patient, both methods revealed a distinct OEF hotspot in the tumor area, whereas ANN showed less apparent artifacts in surrounding tissue. In clinical scan times, the ANN analysis enables OEF mapping with reduced variance, which could facilitate its integration into clinical protocols. Magn Reson Med 79:890-899, 2018. © 2017 International Society for Magnetic Resonance in Medicine.

Basal Metabolic Rate in Carnivores Is Associated with Diet after Controlling for Phylogeny

Studies of basal metabolic rate (BMR), the minimum metabolic rate of postabsorptive, inactive endotherms while in their rest phase and thermal neutral zone, have contributed significantly to our understanding of animal energetics. Besides body mass, the main determinant of BMR, researchers have invoked diet and phylogenetic history as important factors that influence BMR, although their relative importance has been controversial. For 58 species within the Carnivora, we tested the hypothesis that BMR is correlated with home range size, a proxy for level of activity, and diet, using conventional least squares regression (CLSR) and regression based on phylogenetic independent contrasts (PIC). Results showed that BMR of Carnivora was positively correlated with home range size after controlling for body mass, regardless of the statistical method employed. We also found that diet and mass-adjusted home range size were correlated. When we simultaneously tested the effect of diet and mass-adjusted home range on mass-adjusted BMR, home range size was insignificant because of its colinearity with diet. Then we eliminated home range size from our model, and diet proved to be significant with both CLSR and PIC. We concluded that species that eat meat have larger home ranges and higher BMR than species that eat vegetable matter. To advance our understanding of the potential mechanisms that might explain our results, we propose the "muscle performance hypothesis," which suggests that selection for different muscle fiber types can account for the differences in BMR observed between meat eaters and vegetarian species within the Carnivora.

An Updated Comparison of Drug Dosing Methods

The resurgence of the use of and interest in vancomycin, in conjunction with the high degree of interpatient variability in its pharmacokinetic profile, has prompted the development of many and varied dosing methods. Several dosing nomograms have been proposed and evaluated, methods which are useful for initial dosing but do not allow for individualisation of dosage. Given these constraints, several investigators have attempted to apply conventional least-squares regression techniques and, more recently, Bayesian methodologies using either 1- or 2-compartment pharmacokinetic models. Comparative information evaluating algorithmic methods demonstrates that those of Moellering and Lake offer the least biased and most precise predictions of vancomycin dosage. Patient individualisation using conventional least-squares methodology offers some improvement over nomogram-based methods, both in predictive performance and in dosage adjustment once serum concentration data are available. Overall, the latest data indicate that regimens which incorporate Bayesian principles tend to give better results than nomogram-based or conventional least-squares dosing methods for this drug. Despite the advances in methods for dosing vancomycin, several questions remain to be answered. A lack of convincing evidence of a correlation between serum concentrations and therapeutic outcome has prompted debate over the need for serum concentration monitoring and, if it is needed, over which patient population would most benefit. Secondly, little comparative information is currently available as to the dosing of vancomycin in paediatric and neonatal patient populations. Several nomograms for initial dosing have been proposed, but only 2 have been subject to subsequent testing. Finally, information regarding cost-effectiveness and the quality of patient outcome is lacking from the current literature.

Linear Modelling with Clustered Observations: An Illustrative Example of Earnings in the Engineering Industry

Conventional least-squares regression can lead to misleading results if the data have a hierarchical structure. An appropriate linear model is presented for such data. The model has conventional regression, variance-component, and random coefficient models as special cases and may be calibrated by use of recently available software. The effectiveness of the model is demonstrated by an analysis of earnings in the engineering industry. Particular attention is given to the problems of interpreting the parameter estimates and residuals.