Multiple Linear Regression Problem Research Articles

Solving multiple linear regression problem using artificial neural network

<span>Multiple linear regressions are an important tool used to find the relationship between a set of variables used in various scientific experiments. In this article we are going to introduce a simple method of solving a multiple rectilinear regressions (MLR) problem that uses an artificial neural network to find the accurate and expected output from MLR problem. Different artificial neural network (ANN) types with different architecture will be tested, the error between the target outputs and the calculated ANN outputs will be investigated. A recommendation of using a certain type of ANN based on the experimental results will be raised.</span>

Using Parameter Elimination to Solve Discrete Linear Chebyshev Approximation Problems

We consider discrete linear Chebyshev approximation problems in which the unknown parameters of linear function are fitted by minimizing the least maximum absolute deviation of errors. Such problems find application in the solution of overdetermined systems of linear equations that appear in many practical contexts. The least maximum absolute deviation estimator is used in regression analysis in statistics when the distribution of errors has bounded support. To derive a direct solution of the problem, we propose an algebraic approach based on a parameter elimination technique. As a key component of the approach, an elimination lemma is proved to handle the problem by reducing it to a problem with one parameter eliminated, together with a box constraint imposed on this parameter. We demonstrate the application of the lemma to the direct solution of linear regression problems with one and two parameters. We develop a procedure to solve multidimensional approximation (multiple linear regression) problems in a finite number of steps. The procedure follows a method that comprises two phases: backward elimination and forward substitution of parameters. We describe the main components of the procedure and estimate its computational complexity. We implement symbolic computations in MATLAB to obtain exact solutions for two numerical examples.

Quantum circuit design methodology for multiple linear regression

Multiple linear regression assumes an imperative role in supervised machine learning. In 2009, Harrow et al. [Phys. Rev. Lett. 103, 150502 (2009)] showed that their HHL algorithm can be used to sample the solution of a linear system $\mathbf{Ax=b}$ exponentially faster than any existing classical algorithm, with some manageable caveats. The entire field of quantum machine learning gained considerable traction after the discovery of this celebrated algorithm. However, effective practical applications and experimental implementations of HHL are still sparse in the literature. Here, we demonstrate a potential practical utility of HHL, in the context of regression analysis, using the remarkable fact that there exists a natural reduction of any multiple linear regression problem to an equivalent linear systems problem. We put forward a $7$-qubit quantum circuit design, motivated from an earlier work by Cao et al. [Mol. Phys. 110, 1675 (2012)], to solve a $3$-variable regression problem, using only elementary quantum gates. We also implement the Group Leaders Optimization Algorithm (GLOA) [Mol. Phys. 109 (5), 761 (2011)] and elaborate on the advantages of using such stochastic algorithms in creating low-cost circuit approximations for the Hamiltonian simulation. We believe that this application of GLOA and similar stochastic algorithms in circuit approximation will boost time- and cost-efficient circuit designing for various quantum machine learning protocols. Further, we discuss our Qiskit simulation and explore certain generalizations to the circuit design.

Detecting malicious nodes via gradient descent and support vector machine in Internet of Things

IoT devices have become much popular in our daily lives, while attackers often invade network nodes to launch various attacks. In this work, we focus on the detection of insider attacks in IoT networks. Most existing algorithms calculate the reputation of all nodes based on the routing path. However, they rely heavily on the assumption that different nodes in the same routing path have equal reputation, which may be not invalid in practice and cause inaccurate detection results. To solve this issue, we formulate it as a multivariate multiple linear regression problem and use the K-means classification algorithm to detect malicious nodes. Further, we optimize the routing path and design an enhanced detection scheme. Our results indicate that our proposed methods could achieve a detection accuracy rate of 90% or above in a common case, and the enhanced scheme could reach an even lower false detection rate, i.e., below 5%.

Satellite-augmented diffuse solar radiation separation models

Separation models predict diffuse horizontal irradiance from other meteorological parameters such as the global horizontal irradiance or zenith angle. From a mathematical point of view, the separation modeling problem is a regression, where the regressors are observed or computed variables and the regressand is the unobserved diffuse fraction. The most successful minute-resolution separation model prior to 2016 was proposed by Engerer, which is constructed using a trend component (cloud enhancement) and a main effect (logistic function). Subsequently, the Starke model published in 2018 further improved the Engerer model. It is herein shown that the logistic-function type of model, and many other separation models, can be transformed into a (condition-based) multiple linear regression problem. Under this transformation framework, two new models are proposed, which strictly dominate the performance of the Engerer model and the Starke model, at all 7 test sites across the continental United States, making them probably the most accurate separation models to date. The new models are also tested at 5 European sites with unseen data (i.e., not involved during model parameter fitting); their performance again dominates all benchmarking models. The new separation models leverage half-hourly satellite-derived diffuse fraction. Since satellite data are available globally, the satellite-augmented separation models have universal applicability. However, despite their good performance, empirical separation models suffer from a series of issues. Hence, models driven by atmospheric physics are the “true gems” that one should pursue.

Fog-Based Evaluation Approach for Trustworthy Communication in Sensor-Cloud System

A sensor-cloud system is a combination of wireless sensor networks and cloud computing that is equipped with ubiquitous physical sensing ability, high-speed computation, huge storage, and so on. However, sensor-cloud systems suffer from various types of malicious attacks that can cause sensor communications to become unreliable. Establishing a trust evaluation method to ensure members’ reliability in sensor cloud is an effective way to resist malicious attacks. However, most current trust evaluation methods are constrained to specific attacks or applications, and they lack compatibility, verifiability, and scalability. To solve these problems, we formulated the trust evaluation issue as a multiple linear regression problem. Considering energy restrictions, we adopt fog nodes to assist in the trust computation. Moreover, the least squares algorithm is used to find the fitting function between the communication feature and the trust value. The experimental results show that our approach can find the best trust evaluation model and improve the compatibility, verifiability, and accuracy of trust evaluation.

Predicting Hope Levels of High School Students: The Role of Academic Self-Efficacy and Problem Solving

The current study presented how well academic self-efficacy, problem solving, gender, and age predict dispositional and state hope levels of high school students in Turkey. The participants were consisted of 392 high school students. Of the participants, 230 (58.7%) were girls and 162 (41.3%) were boys. Questionnaire measures of dispositional and state hope, academic self-efficacy, and problem solving were administered to the participants. Two multiple linear regression analyses were performed to examine to what extent the independent variables predict dispositional and state hope levels of participants. The results indicated that the predictors significantly accounted for 22% of the variance of dispositional hope, 25% of the variance of state hope. In both multiple linear regression models, academic self-efficacy and problem solving were significant predictors of students’ dispositional and state hope scores. Higher hope scores were significantly associated with higher academic self-efficacy beliefs and better problem solving skills. The findings were discussed in the light of the literature with some implications for school counselors and educators.

Intelligent optimization models based on hard-ridge penalty and RBF for forecasting global solar radiation

Due to the scarcity of equipment and the high costs of maintenance, far fewer observations of solar radiation are made than observations of temperature, precipitation and other weather factors. Therefore, it is increasingly important to study several relevant meteorological factors to accurately forecast solar radiation. For this research, monthly average global solar radiation and 12 meteorological parameters from 1998 to 2010 at four sites in the United States were collected. Pearson correlation coefficients and Apriori association rules were successfully used to analyze correlations between the data, which provided a basis for these relative parameters as input variables. Two effective and innovative methods were developed to forecast monthly average global solar radiation by converting a RBF neural network into a multiple linear regression problem, adding a hard-ridge penalty to reduce the number of nodes in the hidden layer, and applying intelligent optimization algorithms, such as the cuckoo search algorithm (CS) and differential evolution (DE), to determine the optimal center and scale parameters. The experimental results show that the proposed models produce much more accurate forecasts than other models.

A Dirty Model for Multiple Sparse Regression

The task of sparse linear regression consists of finding an unknown sparse vector from linear measurements. Solving this task even under “high-dimensional” settings, where the number of samples is fewer than the number of variables, is now known to be possible via methods such as the LASSO. We consider the multiple sparse linear regression problem, where the task consists of recovering several related sparse vectors at once. A simple approach to this task would involve solving independent sparse linear regression problems, but a natural question is whether one can reduce the overall number of samples required by leveraging partial sharing of the support sets, or nonzero patterns, of the signal vectors. A line of recent research has studied the use of l1/lq norm block-regularizations with q > 1 for such problems. However, depending on the level of sharing, these could actually perform worse in sample complexity when compared to solving each problem independently. We present a new “adaptive” method for multiple sparse linear regression that can leverage support and parameter overlap when it exists, but not pay a penalty when it does not. We show how to achieve this using a very simple idea: decompose the parameters into two components and regularize these differently. We show, theoretically and empirically, that our method strictly and noticeably outperforms both l1 or l1/lq methods, over the entire range of possible overlaps (except at boundary cases, where we match the best method), even under high-dimensional scaling.

Estimating standard errors in feature network models

Feature network models are graphical structures that represent proximity data in a discrete space while using the same formalism that is the basis of least squares methods employed in multidimensional scaling. Existing methods to derive a network model from empirical data only give the best-fitting network and yield no standard errors for the parameter estimates. The additivity properties of networks make it possible to consider the model as a univariate (multiple) linear regression problem with positivity restrictions on the parameters. In the present study, both theoretical and empirical standard errors are obtained for the constrained regression parameters of a network model with known features. The performance of both types of standard error is evaluated using Monte Carlo techniques.

An [formula omitted]-square coefficient based on final prediction error

In this paper, we propose an R 2 coefficient to measure the percentage of variance which can be explained by the fitted model (not by the true model) in a multiple linear regression problem. This new coefficient, denoted R FPE 2 , is a reparametrization of the FPE criterion in use for model selection, such that maximizing R FPE 2 over a set of candidate models is the same as minimizing FPE. Thus, R FPE 2 can be used simultaneously for assessing goodness of fit and for model selection. At each step of a model selection procedure, the user can then quantify what is gained/lost when adding/removing a variable in/from the model, which should facilitate practice and comprehension of model selection.

Seasonal climate prediction for South America with FSU Multi-model Synthetic Superensemble algorithm

Objective combination schemes of predictions from different models have been applied to seasonal climate forecasts. These schemes are successful in producing a deterministic forecast superior to individual member models and better than the multi-model ensemble mean forecast. Recently, a variant of the conventional superensemble formulation was created to improve skills for seasonal climate forecasts, the Florida State University (FSU) Synthetic Superensemble. The idea of the synthetic algorithm is to generate a new data set from the predicted multimodel datasets for multiple linear regression. The synthetic data is created from the original dataset by finding a consistent spatial pattern between the observed analysis and the forecast data set. This procedure is a multiple linear regression problem in EOF space. The main contribution this paper is to discuss the feasibility of seasonal prediction based on the synthetic superensemble approach and to demonstrate that the use of this method in coupled models dataset can reduce the errors of seasonal climate forecasts over South America. In this study, a suite of FSU coupled atmospheric oceanic models was used. In evaluation the results from the FSU synthetic superensemble demonstrate greater skill for most of the variables tested here. The forecast produced by the proposed method out performs other conventional forecasts. These results suggest that the methodology and database employed are able to improve seasonal climate prediction over South America when compared to the use of single climate models or from the conventional ensemble averaging. The results show that anomalous conditions simulated over South America are reasonably realistic. The negative (positive) precipitation anomalies for the summer monsoon season of 1997/98 (2001/02) were predicted by Synthetic Superensemble formulation quite well. In summary, the forecast produced by the Synthetic Superensemble approach outperforms the other conventional forecasts.

A multi-model superensemble algorithm for seasonal climate prediction using DEMETER forecasts

In this paper, a multi-model ensemble approach with statistical correction for seasonal precipitation forecasts using a coupled DEMETER model data set is presented. Despite the continuous improvement of coupled models, they have serious systematic errors in terms of the mean, the annual cycle and the interannual variability; consequently, the predictive skill of extended forecasts remains quite low. One of the approaches to the improvement of seasonal prediction is the empirical weighted multi-model ensemble, or superensemble, combination. In the superensemble approach, the different model forecasts are statistically combined during the training phase using multiple linear regression, with the skill of each ensemble member implicitly factored into the superensemble forecast. The skill of a superensemble relies strongly on the past performance of the individual member models used in its construction. The algorithm proposed here involves empirical orthogonal function (EOF) filtering of the actual data set prior to the construction of a multimodel ensemble or superensemble as an alternative solution for seasonal prediction. This algorithm generates a new data set from the input multi-model data set by finding a consistent spatial pattern between the observed analysis and the individual model forecast. This procedure is a multiple linear regression problem in the EOF space. The newly generated EOF-filtered data set is then used as an input data set for the construction of a multi-model ensemble and superensemble. The skill of forecast anomalies is assessed using statistics of categorical forecast, spatial anomaly correlation and root mean square (RMS) errors. The various verifications show that the unbiased multi-model ensemble of DEMETER forecasts improves the prediction of spatial patterns (i.e. the anomaly correlation), but it shows poor skill in categorical forecast. Due to the removal of seasonal mean biases of the different models, the forecast errors of the bias-corrected multi-model ensemble and superensemble are already quite small. Based on the anomaly correlation and RMS measures, the forecasts produced by the proposed method slightly outperform the other conventional forecasts.

A multi-model superensemble algorithm for seasonal climate prediction using DEMETER forecasts

In this paper, a multi-model ensemble approach with statistical correction for seasonal precipitation forecasts using a coupled DEMETER model data set is presented. Despite the continuous improvement of coupled models, they have serious systematic errors in terms of the mean, the annual cycle and the interannual variability; consequently, the predictive skill of extended forecasts remains quite low. One of the approaches to the improvement of seasonal prediction is the empirical weighted multi-model ensemble, or superensemble, combination. In the superensemble approach, the different model forecasts are statistically combined during the training phase using multiple linear regression, with the skill of each ensemble member implicitly factored into the superensemble forecast. The skill of a superensemble relies strongly on the past performance of the individual member models used in its construction. The algorithm proposed here involves empirical orthogonal function (EOF) filtering of the actual data set prior to the construction of a multimodel ensemble or superensemble as an alternative solution for seasonal prediction. This algorithm generates a new data set from the input multi-model data set by finding a consistent spatial pattern between the observed analysis and the individual model forecast. This procedure is a multiple linear regression problem in the EOF space. The newly generated EOF-filtered data set is then used as an input data set for the construction of a multi-model ensemble and superensemble. The skill of forecast anomalies is assessed using statistics of categorical forecast, spatial anomaly correlation and root mean square (RMS) errors. The various verifications show that the unbiased multi-model ensemble of DEMETER forecasts improves the prediction of spatial patterns (i.e. the anomaly correlation), but it shows poor skill in categorical forecast. Due to the removal of seasonal mean biases of the different models, the forecast errors of the bias-corrected multi-model ensemble and superensemble are already quite small. Based on the anomaly correlation and RMS measures, the forecasts produced by the proposed method slightly outperform the other conventional forecasts.

THE GENERAL BOX–COX TRANSFORMATIONS IN MULTIPLE LINEAR REGRESSION ANALYSIS

ABSTRACT A general Box–Cox transformation method in multiple linear regressions is investigated. An algorithm is proposed to identify optimal general Box–Cox transformations based on kernel density estimation techniques. It is shown that for a multiple linear regression problem, the optimal general Box–Cox transformation can be derived through solving a matrix eigenvector problem, while the regression coefficients are estimated by least squares approach. Examples are given to illustrate the proposed method.

Marker pair selection for mapping quantitative trait loci.

Mapping of quantitative trait loci (QTL) for backcross and F(2) populations may be set up as a multiple linear regression problem, where marker types are the regressor variables. It has been shown previously that flanking markers absorb all information on isolated QTL. Therefore, selection of pairs of markers flanking QTL is useful as a direct approach to QTL detection. Alternatively, selected pairs of flanking markers can be used as cofactors in composite interval mapping (CIM). Overfitting is a serious problem, especially if the number of regressor variables is large. We suggest a procedure denoted as marker pair selection (MPS) that uses model selection criteria for multiple linear regression. Markers enter the model in pairs, which reduces the number of models to be considered, thus alleviating the problem of overfitting and increasing the chances of detecting QTL. MPS entails an exhaustive search per chromosome to maximize the chance of finding the best-fitting models. A simulation study is conducted to study the merits of different model selection criteria for MPS. On the basis of our results, we recommend the Schwarz Bayesian criterion (SBC) for use in practice.

Automated Well Test Analysis Using Robust (LAV) Nonlinear Parameter Estimation

ABSTRACT Most of the nonlinear regression algorithms used in automated well test interpretation perform a minimization of an objective function formed by the sum of the squares of the differences between the data and the model values. This constitutes a typical nonlinear least-squares problem. One of the drawbacks of the least squares approach is that it is significantly affected by outliers, which are data points that can be considered bad observations. This study proposes a robust algorithm that uses the LAV (Least Absolute Value) as the criterion for the minimization. One of the greatest advantages of the LAV approach is that it provides a smooth transition between full acceptance and total rejection of a given observation, providing a systematic way of rejecting outliers by automatically assigning them less weight in the objective function. The technique described in this work is based on the expansion of the model function in a Taylor series up to the first order terms. This poses a new problem which is solved iteratively after being transformed into a multiple linear regression parameter estimation. The procedure becomes therefore a sequence of multiple linear regression problems, which are solved using the least absolute value criterion. Because the equations of condition are solved, instead of the so-called normal equations (as in several implementations of nonlinear parameter estimation algorithms discussed in the literature), the method avoids unnecessary exacerbation of the ill-conditioning of the design matrix. Examples where no large errors in the data are present show that this method produces results similar to least-squares minimization. When outliers are introduced into the observations, however, the present technique yields much better estimates. Also, the least absolute value approach works better than the least-squares criterion when estimating parameters whose sensitivity is small, as in the case of obtaining the storativity ratio and the interporosity flow coefficient in a naturally fractured reservoir with double-porosity behavior.

Multivariate multiple linear regression based on the minimum sum of absolute errors criterion

We propose the minimum sum of absolute errors (MSAE) criterion for estimating the unknown parameters of a multivariate multiple linear regression model. It is less sensitive to outliers than the popular least squares procedure. A multivariate multiple linear regression problem may be viewed as a multiple criteria decision problem. Using the MSAE criterion the estimation problem can be formulated and solved as a multiple objective linear programming problem. We illustrate the idea with a bicriteria example.

A solution to multiple linear regression problems with ordered attributes

A class of multiple linear regression techniques is discussed, in which the order of magnitude is constrained among regression coefficients. Each predictor variable is a qualitative variate having some categories which are on an ordinal scale. The criterion variable is quantitative. The problem to be solved is reduced to a quadratic programming problem in which the objective function is the residual sum of the squares in regression, and the constraints are linear ones imposed on the regression coefficients. Under some conditions for the observed data, this problem can be solved numerically. The proposed technique works effectively for some types of regression analysis.

RECRES: A FORTRAN-77 program for detecting model misspecification in multiple linear regression

RECRES is an interactive FORTRAN-77 program for detecting model misspecification in multiple linear regression problems. The program produces plots of ordinary least-squares (OLS) residuals, normal probability plots of recursive residuals, cumulative sums of recursive residuals, the Quandt log-likelihood statistic, and recursive estimates of the model parameters. It uses orthogonal triangularization to compute diagnostic statistics and allows flexibility in the production of diagnostic plots. Examples show that the program yields reliable results and can detect model misspecifications that are not apparent readily to users of conventional multiple linear regression programs. A source listing of the program and an example of an interactive session are provided.