Parameters Of Linear Regression Model Research Articles

A new approach using the genetic algorithm for parameter estimation in multiple linear regression with long-tailed symmetric distributed error terms: An application to the Covid-19 data

Maximum likelihood (ML) estimators of the model parameters in multiple linear regression are obtained using genetic algorithm (GA) when the distribution of the error terms is long-tailed symmetric. We compare the efficiencies of the ML estimators obtained using GA with the corresponding ML estimators obtained using other iterative techniques via an extensive Monte Carlo simulation study. Robust confidence intervals based on modified ML estimators are used as the search space in GA. Our simulation study shows that GA outperforms traditional algorithms in most cases. Therefore, we suggest using GA to obtain the ML estimates of the multiple linear regression model parameters when the distribution of the error terms is LTS. Finally, real data of the Covid-19 pandemic, a global health crisis in early 2020, is presented for illustrative purposes.

Comparison of Some Methods for Estimating Parameters of General Linear Model in Presence of Heteroscedastic Problem and High Leverage Points

Linear regression is one of the most important statistical tools through which it is possible to know the relationship between the response variable and one variable (or more) of the independent variable(s), which is often used in various fields of science. Heteroscedastic is one of the linear regression problems, the effect of which leads to inaccurate conclusions. The problem of heteroscedastic may be accompanied by the presence of extreme outliers in the independent variables (High leverage points) (HLPs), the presence of (HLPs) in the data set result unrealistic estimates and misleading inferences. In this paper, we review some of the robust weighted estimation methods that accommodate both Robust and classical methods in the detection of extreme outliers (High leverage points) (HLPs) and the determination of weights. The methods include both Diagnostic Robust Generalized Potential Based on Minimum Volume Ellipsoid (DRGP (MVE)), Diagnostic Robust Generalized Potential Based on Minimum Covariance Determinant (DRGP (MCD)), and Diagnostic Robust Generalized Potential Based on Index Set Equality (DRGP (ISE)). The comparison was made according to the standard error criterion of the estimated parameters SE ( ) and SE ( ) of general linear regression model, for sample sizes (n=60, n=100, n=160), with different degree (severity) of heterogeneity, and contamination percentage (HLPs) are (τ =10%, τ=30%). it was found through comparison that weighted least squares estimation based on the weights of the DRGP (ISE) method are considered the best in estimating the parameters of the multiple linear regression model because they have the lowest standard error values of the estimators ( ) and ( ) as compared to other methods.
 Paper type: A case study

Likelihood-Based Confidence Intervals for the Parameters of a Simple Linear Regression Model with Cauchy Errors

The estimators of the slope and the intercept of simple linear regression model with normal errors are normally distributed and their exact confidence intervals are constructed using the t-distribution. However, when the normality assumption is not fulfilled, it is not possible to obtain exact confidence intervals. The Wald method of interval estimation is commonly used to provide approximate confidence intervals in such cases, and since it is derived from the central limit theorem it requires large samples in order to provide reliable approximate confidence intervals. This paper considers an alternative method of constructing approximate confidence intervals for the parameters of a simple linear regression model with Cauchy errors which is based the normal approximation to the Cauchy likelihood. The normal approximation to the Cauchy likelihood is obtained by a Tailor series expansion of the Cauchy log-likelihood function about the maximum likelihood estimate of the parameters and ignoring terms of order greater two. The maximized relative log-likelihood function for each parameter is then derived from the normal Cauchy relative log-likelihood function. The approximate confidence intervals for the parameters are constructed from their respective maximized relative log-likelihood functions. These confidence intervals have closed form confidence limits, are short and have coverage probabilities close to the nominal value 0.95.

The GSp criterion on subset selection method for regression in the presence of outliers and multicollinearity

Linear regression models a relation between a response variable with regressor variables and produce regression model. Ordinary least squares (OLS) method is often utilized to estimate parameters of linear regression model. OLS method will produce the best linear unbiased estimates when all the assumptions are met. The “best” means the estimate is unbiased and has the minimum variance. However, in real data, those assumptions are often violated, such as the multicollinearity and outliers. Multicollinearity will produce a large variance of the regression parameters, while outliers will cause a biased estimate. Jackknife ridge M-estimator is recommended to be implemented in the model when these problems are present. Jackknife ridge M-estimator is a parameter estimation method of the regression with robust property, which makes robust to outliers, and ridge method is utilized to overcome multicollinearity. Whereas the jackknife method is utilized to reduce the bias from the result of the ridge method. In multiple linear regression, regression model could possibly contain many candidates for regressor variables. Subset selection method is utilized to select some of those regressors that best predicts the response variable. The selection criterion that can be utilized on the subset selection method when outlier and multicollinearity are present is the GSp criterion. The reason for this is because the GSp criterion is based on jackknife ridge M-estimator, which is able to solve the problem of outlier and multicollinearity. Furthermore, GSp criterion is implemented to obtain the best model on predicting IQ based on several predictors. It was found that IQ could be predicted by only doing 3 personality test which is test 1, test 3 and test 5 instead of all 5 different personality test.

Using Bayes weights to remedy the heterogeneity problem of random error variance in linear models

In this research, it was suggested to use the InformativeBayes method in calculating the Bayes weights and use them to treat the of heterogeneity problem when estimating the linear regression model parameters using the weighted least squares method (BWLS). And compare it with the classical method through an experimental side to simulate the generated data from a normal distribution and for several different cases as well as an applied side of real data. The results of the research provided the preference of the proposed method on the classical method by relying on some statistical criteria through a program designed for this purpose in the language of MATLAB.

A Multi Linear Regression Model to Derive Dust PM10 in the Sahel Using AERONET Aerosol Optical Depth and CALIOP Aerosol Layer Products

Due to a limited number of monitoring stations in Western Africa, the impact of mineral dust on PM10 surface concentrations is still poorly known. We propose a new method to retrieve PM10 dust surface concentrations from sun photometer aerosol optical depth (AOD) and CALIPSO/CALIOP Level 2 aerosol layer products. The method is based on a multi linear regression model that is trained using co-located PM10, AERONET and CALIOP observations at 3 different locations in the Sahel. In addition to the sun photometer AOD, the regression model uses the CALIOP-derived base and top altitude of the lowermost dust layer, its AOD, the columnar total and columnar dust AOD. Due to the low revisit period of the CALIPSO satellite, the monthly mean annual cycles of the parameters are used as predictor variables rather than instantaneous observations. The regression model improves the correlation coefficient between monthly mean PM10 and AOD from 0.15 (AERONET AOD only) to 0.75 (AERONET AOD and CALIOP parameters). The respective high and low PM10 concentration during the winter dry season and summer season are well produced. Days with surface PM10 above 100 μg/m3 are better identified when using the CALIOP parameters in the multi linear regression model. The number of true positives (actual and predicted concentrations above the threshold) is increased and leads to an improvement in the classification sensitivity (recall) by a factor 1.8. Our methodology can be extrapolated to the whole Sahel area provided that satellite derived AOD maps are used in order to create a new dataset on population exposure to dust events in this area.

Least Quartic Regression Criterion to Evaluate Systematic Risk in the Presence of Co-Skewness and Co-Kurtosis

This article proposes a new method for the estimation of the parameters of a simple linear regression model which is based on the minimization of a quartic loss function. The aim is to extend the traditional methodology, based on the normality assumption, to also take into account higher moments and to provide a measure for situations where the phenomenon is characterized by strong non-Gaussian distribution like outliers, multimodality, skewness and kurtosis. Although the proposed method is very general, along with the description of the methodology, we examine its application to finance. In fact, in this field, the contribution of the co-moments in explaining the return-generating process is of paramount importance when evaluating the systematic risk of an asset within the framework of the Capital Asset Pricing Model. We also illustrate a Monte Carlo test of significance on the estimated slope parameter and an application of the method based on the top 300 market capitalization components of the STOXX® Europe 600. A comparison between the slope coefficients evaluated using the ordinary Least Squares (LS) approach and the new Least Quartic (LQ) technique shows that the perception of market risk exposure is best captured by the proposed estimator during market turmoil, and it seems to anticipate the market risk increase typical of these periods. Moreover, by analyzing the out-of-sample risk-adjusted returns we show that the proposed method outperforms the ordinary LS estimator in terms of the most common performance indices. Finally, a bootstrap analysis suggests that significantly different Sharpe ratios between LS and LQ yields and Value at Risk estimates can be considered more accurate in the LQ framework. This study adds insights into market analysis and helps in identifying more precisely potentially risky assets whose extreme behavior is strongly dependent on market behavior.

Analyses of replicated spectrophotometric data by using soft computing methods

Soft computing based chemometric studies are needed to model and to optimize complex processes, such as the extraction stage. In this study, it was aimed to obtain optimal values of ethanol (EtOH) concentration, extraction time, extraction temperature, solvent/solid ratio for the ultrasound-assisted extraction (UAE) of grape seed polyphenols to maximize the total phenolic content (TPC) and total antioxidant activity (TAA) in multi-objective perspective by using soft computing methods. The experimental data set was composed with replicated response measures to see the behavior of the responses. The replicated response measured (RRM) data was recently modeled by using fuzzy linear regression in which the replicated responses were considered as triangular linear fuzzy numbers (TLFNs). Also, polynomial type fuzzy linear regression model parameters were dealt as TLFNs whereas the experiment conditions were crisp. The predicted fuzzy linear models were obtained by using least square (LS) approach. The predicted fuzzy models were optimized through the Non-dominated Sorting Genetic Algorithm-II (NSGA-II) and Pareto set was obtained. The obtained Pareto solution set was experimentally verified. A compromise solution was chosen among many non-dominated solutions of experimental conditions by using a distance function based on root mean square of fuzzy errors. According to the results, it is possible to say that the proposed soft computing based modeling and multi-objective optimization (MOO) approaches can be used as flexible analyses tools in chemometry.

A parametric recurrent neural network scheme for solving a class of fuzzy regression models with some real-world applications

In this paper, a hybrid scheme based on recurrent neural networks for approximate fuzzy coefficients (parameters) of fuzzy linear and polynomial regression models with fuzzy output and crisp inputs is presented. Here, a neural network is first constructed based on some concepts of convex optimization and stability theory. The suggested neural network model guarantees to find the approximate parameters of the fuzzy regression problem. The existence and convergence of the trajectories of the neural network are studied. The Lyapunov stability for the neural network is also shown. Some illustrative examples provide a further demonstration of the effectiveness of the method.

Self-tuning robust adjustment within multivariate regression time series models with vector-autoregressive random errors

The iteratively reweighted least-squares approach to self-tuning robust adjustment of parameters in linear regression models with autoregressive (AR) and t-distributed random errors, previously established in Kargoll et al. (in J Geod 92(3):271–297, 2018. https://doi.org/10.1007/s00190-017-1062-6), is extended to multivariate approaches. Multivariate models are used to describe the behavior of multiple observables measured contemporaneously. The proposed approaches allow for the modeling of both auto- and cross-correlations through a vector-autoregressive (VAR) process, where the components of the white-noise input vector are modeled at every time instance either as stochastically independent t-distributed (herein called “stochastic model A”) or as multivariate t-distributed random variables (herein called “stochastic model B”). Both stochastic models are complementary in the sense that the former allows for group-specific degrees of freedom (df) of the t-distributions (thus, sensor-component-specific tail or outlier characteristics) but not for correlations within each white-noise vector, whereas the latter allows for such correlations but not for different dfs. Within the observation equations, nonlinear (differentiable) regression models are generally allowed for. Two different generalized expectation maximization (GEM) algorithms are derived to estimate the regression model parameters jointly with the VAR coefficients, the variance components (in case of stochastic model A) or the cofactor matrix (for stochastic model B), and the df(s). To enable the validation of the fitted VAR model and the selection of the best model order, the multivariate portmanteau test and Akaike’s information criterion are applied. The performance of the algorithms and of the white noise test is evaluated by means of Monte Carlo simulations. Furthermore, the suitability of one of the proposed models and the corresponding GEM algorithm is investigated within a case study involving the multivariate modeling and adjustment of time-series data at four GPS stations in the EUREF Permanent Network (EPN).

Least Squares Estimations for the General Linear Model Parameters with Epsilon Skew Normal Error Term

Examination of skewness makes academics more aware of the importance of accurate statistical analysis. Undoubtedly, most phenomena contain a certain percentage of skewness which resulted to the appearance of what is -called "asymmetry" and, consequently, the importance of the skew normal family . The epsilon skew normal distribution ESN (μ, σ, ε) is one of the probability distributions which provide a more flexible model because the skewness parameter provides the possibility to fluctuate from normal to skewed distribution. Theoretically, the estimation of linear regression model parameters, with an average error value that is not zero, is considered a major challenge due to having difficulties, as no explicit formula to calculate these estimates can be obtained. Practically, values for these estimates can be obtained only by referring to numerical methods. This research paper is dedicated to estimate parameters of the Epsilon Skew Normal General Linear Model (ESNGLM) using an adaptive least squares method, as along with the employment of the ordinary least squares method for estimating parameters of the General Linear Model (GLM). In addition, the coefficient of determination was used as a criterion to compare the models’ preference. These methods were applied to real data represented by dollar exchange rates. The Matlab software was applied in this work and the results showed that the ESNGLM represents a satisfactory model.

A New Method to Estimate Parameters in the Simple Regression Linear Equation

Linear regression is widely used in various fields. Research on linear regression uses the OLS and ML method in estimating its parameters. OLS and ML method require many assumptions to complete. It is frequently found there is an unconditional assumption that both methods are not successfully used. This paper proposes a new method which does not require any assumption with a condition. The new method is called SAM (Simple Averaging Method) to estimate parameters in the simple linear regression model. The method may be used without fulfilling assumptions in the regression model. Three new theorems are formulated to simplify the estimation of parameters in the simple linear regression model with SAM. By using the same data, the simple linear regression model parameter estimation is conducted using SAM. The result shows that the obtained regression parameter is not quite far different. However, to measure the accuracy of both methods, a comparison of errors made by each method is conducted using Root Mean Square Error (RMSE) and Mean Averaged Error (MAE). By comparing the values of RMSE and MAE for both methods, SAM method may be used to estimate parameters in the regression equation. The advantage of SAM is free from all assumptions required by regression, such as error normality assumption while the data should be from the normal distribution.

Hidden Markov Linear Regression Model and its Parameter Estimation

This article first defines a hidden Markov linear regression model for the purpose of further studying the mutual transformation between different states in the linear regression model, and the regression relationship between the dependent variable and the independent variable in each state. And then, K-means clustering analysis methods are used to identify the hidden states of observed data, and the maximum likelihood estimation of the hidden state transition probability matrix elements is obtained by using the maximum likelihood estimation method, and parameter estimation of unknown parameters in linear regression model is also presented by using the least squares method. Finally, the observation vector set is generated according to the defined model, and the empirical simulation demonstrates that the parameter estimation method shown in this work is reliable.

Determination of the main factors of the gross regional product level by econometric modeling methods for the totality of regions of the Russian Federation

The purpose of the research is to identify factors that determine the level of gross regional product, using the construction of econometric models. In terms of the economic theory the volume of gross added value of the region depends on existence of resources, efficiency of use of resources, level of effective demand for the consumption of gross added value, level of export, structure of the regional economy. In the article, by constructing econometric models with a different set of factors, the most essential and statistically significant factors determining the level of gross regional product were identified.Materials and methods. Official data of Federal State Statistics Service for 2016 formed empirical base of the research. The aggregate of the constituent entities of the Russian Federation was considered as a statistical aggregate as an object of study, and both the complete aggregate of subjects and the aggregate of entities with a gross regional product level of less than and more than a trillion rubles were studied. Research method is econometric modeling of gross regional product based on spatial data on constituent entities of the Russian Federation. The parameters of the multiple linear regression models in absolute values and in logarithms were estimated using the least squares method. To test the null hypotheses, a five percent significance level was adopted. Interval estimation of model parameters was carried out taking into account standard errors consistent under heteroskedasticity. The quality of models was defined by coefficient of determination, mean relative error of approximation, Ramsey’s test checking a null hypothesis about lack of the passed variables in model. Implementation of the research was carried out in the RStudio software product.Results. Eight qualitative models of multiple linear regression are constructed. Two models are constructed on the general set of constituent entities of the Russian Federation: 1) model in absolute values with three quantitative signs and one attributive sign characterizing group of entities on the level of a gross regional product; 2) three-factor model in logarithms. Two models are constructed on set of entities with a gross regional product less than one trillion rubles: 1) model in absolute values with seven quantitative signs, five of which were statistically significant; 2) four-factor model in logarithms. Four two-factor models are constructed on set of entities with a gross regional product more than one trillion rubles.Conclusion. In all constructed models the most significant factors of the level of gross regional product are: a) factor of fixed capital (average annual residual cost of fixed assets); b) factor of effective demand (expressed by one of the following indicators: social expenses of the budget, total income of the population, wage fund); c) factor of level of financial resources (expressed by the volume of average annual balances on ruble accounts of legal persons and balance of incomes and expenses of the enterprises and organizations). At creation of econometric models statistically insignificant were factors of resource efficiency (capital productivity, labor productivity, profitability), factors characterizing the volume of foreign economic activity and a share of the extractive industries. The economy of the regions in 2016 can be described as an economy of an extensive type, focusing on the use of domestic capital and financial resources.

Ordered partially ordered judgment subset sampling with applications to parametric inference

ABSTRACTIn this paper, we use the idea of order statistics from independent and non-identically distributed random variables to propose ordered partially ordered judgment subset sampling (OPOJSS) and then develop optimal linear parametric inferences. The best linear unbiased and invariant estimators of the location and scale parameters of a location-scale family are developed based on OPOJSS. It is shown that, despite the presence or absence of ranking errors, the proposed estimators with OPOJSS are uniformly better than the existing estimators with simple random sampling (SRS), ranked set sampling (RSS), ordered RSS (ORSS) and partially ordered judgment subset sampling (POJSS). Moreover, we also derive the best linear unbiased estimators (BLUEs) of the unknown parameters of the simple linear regression model with replicated observations using POJSS and OPOJSS. It is found that the BLUEs with OPOJSS are more precise than the BLUEs based on SRS, RSS, ORSS and POJSS.

Multivariable Regression Equivalent Model of Interconnected Active Distribution Networks Based on Boundary Measurement

With the increasing complexity of the active distribution network (ADN) due to distributed generation (DG) integration, together with the electricity market evolution, the traditional ADN is divided into multiple areas to operate independently. Due to technical problems or business privacy, the internal network regional control center cannot grasp the changes of the external regional network in time. In order to accurately reflect the distribution network operation state, a multivariable regression equivalent model is proposed in this paper. Firstly, the external network is made equivalent to a multi-port Norton model. The multivariable linear regression model is then derived based on the equivalent distribution network, and the regression model variables are constructed using boundary node information collected by the measurement equipment. Finally, the maximum likelihood estimation (MLE) is used to estimate the parameters of the multivariable linear regression model. Furthermore, case studies demonstrate the effectiveness and robustness of the proposed method, and detailed information of external ADN is unnecessary, except for the boundary node information. The proposed method can also be applied for three-phase unbalanced ADN efficiently.

A New Class of Change Point Test Statistics of Rényi Type

A new class of change point test statistics is proposed that utilizes a weighting and trimming scheme for the cumulative sum (CUSUM) process inspired by Rényi. A thorough asymptotic analysis and simulations both demonstrate that this new class of statistics possess superior power compared to traditional change point statistics based on the CUSUM process when the change point is near the beginning or end of the sample. Generalizations of these “Rényi” statistics are also developed to test for changes in the parameters in linear and nonlinear regression models, and in generalized method of moments estimation. In these contexts, we applied the proposed statistics, as well as several others, to test for changes in the coefficients of Fama–French factor models. We observed that the Rényi statistic was the most effective in terms of retrospectively detecting change points that occur near the endpoints of the sample.

Calibration of low-cost NO2 sensors in an urban air quality network

Low-cost air quality sensors measuring air quality at fine spatio-temporal resolutions, typically suffer from sensor drift and interference. Field calibration is typically performed at one location, while little is known about the spatial transferability of correction factors. We evaluated three calibration methods using a year of hourly nitrogen dioxide (NO2) observations from low-cost sensors, collocated at two sites with a conventional monitor as reference: (1) an iterative Bayesian approach for daily estimation of the parameters in a multiple linear regression model, (2) a daily updated correction factor and (3) a correction factor updated only when concentrations are uniformly low. We compared the performance of the calibration methods in terms of temporal stability, spatial transferability, and sensor specificity. We documented drift within the 1-year period. The correction factor updated under uniformly low concentrations performed poorly. The iterative Bayesian approach and daily correction factor reduced the root mean squared error (RMSE) by 21–46% at the calibration locations, but did not reduce RMSE at the other location. By examining the posterior distributions of the regression coefficients, we found that the poor spatial transferability is consistent with different responses of individual sensors to environmental factors. We conclude that the spatial and temporal variability in the calibration parameters requires them to be updated regularly, including sensor-specific recalibrations.

Asymptotic Distribution of the Bias Corrected Least Squares Estimators in Measurement Error Linear Regression Models Under Long Memory

This article derives the consistency and asymptotic distribution of the bias corrected least squares estimators (LSEs) of the regression parameters in linear regression models when covariates have measurement error (ME) and errors and covariates form mutually independent long memory moving average processes. In the structural ME linear regression model, the nature of the asymptotic distribution of suitably standardized bias corrected LSEs depends on the range of the values of where d X ,d u , and d ε are the LM parameters of the covariate, ME and regression error processes respectively. This limiting distribution is Gaussian when and non‐Gaussian in the case . In the former case some consistent estimators of the asymptotic variances of these estimators and a log(n)‐consistent estimator of an underlying LM parameter are also provided. They are useful in the construction of the large sample confidence intervals for regression parameters. The article also discusses the asymptotic distribution of these estimators in some functional ME linear regression models, where the unobservable covariate is non‐random. In these models, the limiting distribution of the bias corrected LSEs is always a Gaussian distribution determined by the range of the values of d ε − d u .

Design and implementation of the fuzzy expert system in Monte Carlo methods for fuzzy linear regression

In this study, fuzzy expert system (FES) in Monte Carlo (MC) method, which is used for estimating fuzzy linear regression model (FLRM) parameters, is applied to determine the parameter intervals, for the first time in the literature. MC method in estimating FLRM parameters is a new field of study that is very useful and time saving. However a major problem might occur in determining the parameter intervals from which the regression model parameters are supposed to come. If the intervals are calculated too large, FLRM error will be very large. Accordingly, the actual model parameters will not be obtained if the intervals are calculated too narrow. This drawback has not been addressed in the literature before and only optimization methods have been applied to achieve the best interval values. In this article, the FES is used for the first time in order to solve the problem in parameter estimation process for the FLRM in the field of statistics. For this purpose, the difference between the fuzzy observation value and fuzzy estimation value’s support set (W) is taken into account. The most appropriate intervals calculated for the parameters are those that make W as small as possible. Thus, FES is designed to determine the best intervals for the model parameters. The system knowledge base is composed of 7 fuzzy rules. As a result, it is deduced that the FLRM parameter estimates obtained from the MC method using FES are very close to the real values. The real impact of this paper will be in showing the applicability of FESs in order to solve problems that we encounter in the field of statistics by the help of linguistic expressions. Moreover, these outcomes will be useful for enriching the studies that have already focused on FLRMs and will encourage researchers to use FES to solve problems in statistics. To sum up, this study demonstrates that FESs which is used in technological devices and makes our lives easier can also be used in solving problems that we confront in the field of statistics efficiently with using linguistic expressions like human inference system.