Classical Linear Regression Model Research Articles

CryptoNet: Using Auto-Regressive Multi-Layer Artificial Neural Networks to Predict Financial Time Series

When analyzing a financial asset, it is essential to study the trend of its time series. It is also necessary to examine its evolution and activity over time to statistically analyze its possible future behavior. Both retail and institutional investors base their trading strategies on these analyses. One of the most used techniques to study financial time series is to analyze its dynamic structure using auto-regressive models, simple moving average models (SMA), and mixed auto-regressive moving average models (ARMA). These techniques, unfortunately, do not always provide appreciable results both at a statistical level and as the Risk-Reward Ratio (RRR); above all, each system has its pros and cons. In this paper, we present CryptoNet; this system is based on the time series extraction exploiting the vast potential of artificial intelligence (AI) and machine learning (ML). Specifically, we focused on time series trends extraction by developing an artificial neural network, trained and tested on two famous crypto-currencies: Bitcoinand Ether. CryptoNet learning algorithm improved the classic linear regression model up to 31% of MAE (mean absolute error). Results from this work should encourage machine learning techniques in sectors classically reluctant to adopt non-standard approaches.

Semiparametric partial linear modeling of risk factors for ear infections: the Early Childhood Longitudinal Study

The Early Childhood Longitudinal Study–Kindergarten Class of 2010–2011 (ECLS-K:2011) ascertained timing of ear infections within age specified intervals and parent's/caregiver's report of medically diagnosed hearing loss. In this nationally representative, school-based sample of children followed from kindergarten entry through fifth grade, academic performance in reading, mathematics, and science was assessed longitudinally. Prior investigations of this ECLS-K:2011 cohort showed that age has a non-linear, monotonically increasing functional relationship with academic performance. Because of this knowledge, a semiparametric partial linear model is proposed, in which the effect of age is modeled by an unknown monotonically increasing function along with other regression parameters. The parameters are estimated by a semiparametric maximum likelihood estimator. A test of a constant effect of age is also proposed. Simulation studies are conducted to evaluate the performance of the proposed method, as compared with the commonly used linear model; the former outperforms the latter based on several criteria. We then analyzed ECLS-K:2011 data to compare results of the partial linear parametric model estimation with that of classical linear regression models.

Comparison of variable selection procedures and investigation of the role of shrinkage in linear regression-protocol of a simulation study in low-dimensional data.

In low-dimensional data and within the framework of a classical linear regression model, we intend to compare variable selection methods and investigate the role of shrinkage of regression estimates in a simulation study. Our primary aim is to build descriptive models that capture the data structure parsimoniously, while our secondary aim is to derive a prediction model. Simulation studies are an important tool in statistical methodology research if they are well designed, executed, and reported. However, bias in favor of an “own” preferred method is prevalent in most simulation studies in which a new method is proposed and compared with existing methods. To overcome such bias, neutral comparison studies, which disregard the superiority or inferiority of a particular method, have been proposed. In this paper, we designed a simulation study with key principles of neutral comparison studies in mind, though certain unintentional biases cannot be ruled out. To improve the design and reporting of a simulation study, we followed the recently proposed ADEMP structure, which entails defining the aims (A), data-generating mechanisms (D), estimand/target of analysis (E), methods (M), and performance measures (P). To ensure the reproducibility of results, we published the protocol before conducting the study. In addition, we presented earlier versions of the design to several experts whose feedback influenced certain aspects of the design. We will compare popular penalized regression methods (lasso, adaptive lasso, relaxed lasso, and nonnegative garrote) that combine variable selection and shrinkage with classical variable selection methods (best subset selection and backward elimination) with and without post-estimation shrinkage of parameter estimates.

Variable selection in linear-circular regression models

ABSTRACT Applications of circular regression models are ubiquitous in many disciplines, particularly in meteorology, biology and geology. In circular regression models, variable selection problem continues to be a remarkable open question. In this paper, we address variable selection in linear-circular regression models where uni-variate linear dependent and a mixed set of circular and linear independent variables constitute the data set. We consider Bayesian lasso which is a popular choice for variable selection in classical linear regression models. We show that Bayesian lasso in linear-circular regression models is not able to produce robust inference as the coefficient estimates are sensitive to the choice of hyper-prior setting for the tuning parameter. To eradicate the problem, we propose a robustified Bayesian lasso that is based on an empirical Bayes (EB) type methodology to construct a hyper-prior for the tuning parameter while using Gibbs Sampling. This hyper-prior construction is computationally more feasible than the hyper-priors that are based on correlation measures. We show in a comprehensive simulation study that Bayesian lasso with EB-GS hyper-prior leads to a more robust inference. Overall, the method offers an efficient Bayesian lasso for variable selection in linear-circular regression while reducing model complexity.

Forecasting annual electric power consumption using a random parameters model with heterogeneity in means and variances

This paper develops heterogeneity-based econometric models that forecast the yearly consumption of electricity. Unlike previous research, this paper departs from classical econometric regression models and explicitly accounts for unobserved heterogeneity and corresponding interactions. More specifically, three modeling approaches are applied: a random parameter linear regression model (RPLRM), a correlated random parameter linear model (C-RPLRM), and a random parameter linear model with heterogeneity in means and variances (RP-HMV). In addition, a grey model is estimated based on historical consumption data. The estimation results clearly demonstrate that the random parameter methodology is statistically superior to the classical multiple linear regression model. Moreover, based on the test results reported in this paper, as well as the forecasting accuracy measures, the RP-HMV model is shown to be statistically outstanding, with a very low forecasting error (MAPE of 0.04%) compared to the other models, including the grey model. Our estimation results show that two variables, namely the average electricity price and contribution of renewable energy to the national and distribution grids, produced statistically significant random parameters with heterogeneous variances and means. Moreover, the results show that factors including the number of households on the distribution grid, average electric power price, and availability of all-season air conditioners with hot and cold inverter characteristics significantly influenced the electricity consumption. These empirical results reveal that random-parameters models with heterogeneity in their means and variances can provide a detailed analysis of the predictor variables shaping the annual electricity consumption, and they highlight the importance of including unobserved heterogeneity and related interactions in econometric models.

The Impact of Industrial Intelligence on Energy Intensity: Evidence from China

With the sustainable development of cyber-physical science and information technologies, artificial intelligence technology is becoming more and more mature and has been used widely in various walks of life. As one part of this development, industrial intelligence has been applied diffusely to improve the productivity and energy efficiency of factories and governments. Meanwhile, the social ecological environment change has also caused widespread social concern in recent years, and energy efficiency, which is related to climate change, has forced almost every country to reduce their carbon emissions for bettering environmental quality. However, there is little research that has studied this problem from the perspective of industrial robots, even though they are an indispensable part in modern industrial systems. In order to promote the development of artificial intelligence and its application in industrial fields effectively and raise the energy consumption efficiency of production, this paper investigates the impact of industrial intelligence on energy intensity in China, as it is the largest manufacturing and energy consumption country in the world, and we also hope that the experimental results in this study can guide relevant departments and governments to formulate reasonable policies to enhance the utilization efficiency of energy and improve the environmental quality synchronously. For the sake of the rigor of this research and the accuracy of the experimental results, this study explores the corresponding effect mechanisms of industrial intelligence on China’s energy intensity from 2008 to 2019 by using the classical linear regression model OLS (Ordinary Least Squares) and WLS (Weighted Least Squares) separately, which were applied in the previous studies. The results of this study reveal three major findings. The first is that it further proves that the application of artificial intelligence can indeed reduce energy intensity, and the wide applications of artificial intelligence can reduce energy intensity significantly by reducing energy consumption. Besides, the ownership structure of state-owned enterprises will have a positive impact on energy efficiency. The environmental performance of state-owned enterprises is better than that of foreign-funded and private enterprises. Finally, the models further verify the significant impact of the enterprise scale effect on energy intensity. It will bring about the improvement of economic efficiency, and the larger the enterprise, the more obvious the economies of scale effect and the lower the energy intensity.

A catchment-scale model of river water quality by Machine Learning

Water quality is a concern in most river basins worldwide due to the widespread release of pollutants which impacts the freshwater ecosystems. Exploring the relationships between drivers and water quality parameters at the regional scale is key in the identification of appropriate actions for the reduction of water pollution. Regional models are the appropriate tool to achieve this, though their development poses relevant challenges because of the complexity and non-linearity of such relationships. Among the available approaches, Machine Learning (ML) is promising because of its capability to detect complex nonlinear relationships and flexibility in the parameterization, which is learned from data. In this work, we developed regional models of water temperature, dissolved oxygen, arsenic, sulfate and chloride concentrations, as well as electrical conductivity, by using two ML algorithms, Random Forest and Deep feed-forward Neural Network, and compared their performances against the standard Linear Regression model. Our results indicate that the two ML algorithms are much more accurate models for such variables than the classical Linear Regression model, with Deep feed-forward Neural Network being the most effective in identifying the reciprocal importance of the drivers and capturing nonlinear relationships between drivers and water quality variables. Our analysis also revealed that the Julian day and year at which the sample was taken surrogate the air temperature in modeling water temperature and dissolved oxygen, with only a slight performance reduction. Arsenic, sulfate, and chloride show more complex behaviors in which geogenic and anthropogenic sources are intertwined. Dilution exerts a role chiefly for arsenic concentration, which suggests a non-uniform, in space, geogenic origin for this variable.

Network Gradient Descent Algorithm for Decentralized Federated Learning

We study a fully decentralized federated learning algorithm, which is a novel gradient descent algorithm executed on a communication-based network. For convenience, we refer to it as a network gradient descent (NGD) method. In the NGD method, only statistics (e.g., parameter estimates) need to be communicated, minimizing the risk of privacy. Meanwhile, different clients communicate with each other directly according to a carefully designed network structure without a central master. This greatly enhances the reliability of the entire algorithm. Those nice properties inspire us to carefully study the NGD method both theoretically and numerically. Theoretically, we start with a classical linear regression model. We find that both the learning rate and the network structure play significant roles in determining the NGD estimator’s statistical efficiency. The resulting NGD estimator can be statistically as efficient as the global estimator, if the learning rate is sufficiently small and the network structure is weakly balanced, even if the data are distributed heterogeneously. Those interesting findings are then extended to general models and loss functions. Extensive numerical studies are presented to corroborate our theoretical findings. Classical deep learning models are also presented for illustration purpose.

Regional thermal analysis approach: A management tool for predicting water temperature metrics relevant for thermal fish habitat

The field of water resource management, including fisheries, is facing new challenges associated with climate change. This study sheds light on the modeling of water temperature indices (metrics) that describe critical thermal maxima of the Atlantic salmon (salmo salar). These thermal metrics include MaxWaterTmax (interannual mean of maximum summer temperature), MaxNumDay (interannual mean of the number of consecutive days with maximum water temperature > 25 °C and minimum water temperature > 20 °C). The latter is an important indicator to evaluate thermal variability. Three other parameters of a Gaussian function fitted to the interannual daily mean temperatures characterizing the thermal regime of 146 stations located in Eastern Canada were estimated. These three parameters are Gaussian_a (maximum of interannual daily mean temperature), Gaussian_b (mean duration of the warm period), and Gaussian_c (date of occurrence of the interannual maximum temperature). The classical Multiple linear regression model (MLR) and the non-linear Generalized additive model (GAM) were tested and compared to estimate the five thermal metrics. The regression-based approaches involve the identification of thermally homogeneous regions based on three approaches: hierarchical clustering analysis (HCA), regions of influence (ROI) as well as canonical correlation analysis (CCA). Then, the regional MLR and GAM models were applied within the delineated homogenous regions. Also, the regional models were compared to models encompassing all stations (i.e., one region). For each regional estimation model and each thermal metric, a set of optimal explanatory variables were selected using a forward stepwise procedure. The database consisted of 22 environmental predictors related to physiography, topography, climate, land cover and surface deposits. To assess performance of the models, the following statistical metrics were used: coefficient of determination R2, root mean square error (RMSE), bias, relative root mean square error (RRMSE) and percent bias (PBias). The results demonstrate that the non-linear GAM model was consistently better than the simpler MLR model for estimating the five thermal metrics. Results also show that the best practice consists in delineating homogeneous regions before applying the regional GAM model. According to all performance criteria, delineation of regions with the HCA approach is considered to be more flexible and to lead to better performances than neighborhood-based approaches (CCA and ROI).

Predicting Outcome of Traumatic Brain Injury: Is Machine Learning the Best Way?

One of the main challenges in traumatic brain injury (TBI) patients is to achieve an early and definite prognosis. Despite the recent development of algorithms based on artificial intelligence for the identification of these prognostic factors relevant for clinical practice, the literature lacks a rigorous comparison among classical regression and machine learning (ML) models. This study aims at providing this comparison on a sample of TBI patients evaluated at baseline (T0), after 3 months from the event (T1), and at discharge (T2). A Classical Linear Regression Model (LM) was compared with independent performances of Support Vector Machine (SVM), k-Nearest Neighbors (k-NN), Naïve Bayes (NB) and Decision Tree (DT) algorithms, together with an ensemble ML approach. The accuracy was similar among LM and ML algorithms on the analyzed sample when two classes of outcome (Positive vs. Negative) approach was used, whereas the NB algorithm showed the worst performance. This study highlights the utility of comparing traditional regression modeling to ML, particularly when using a small number of reliable predictor variables after TBI. The dataset of clinical data used to train ML algorithms will be publicly available to other researchers for future comparisons.

Empirical Research on the Relationship between Industry Working Capital Shortfall and Company Cash Holding in the Same Industry

Affected by the fluctuation of the market and economic environment during the epidemic period, the capital pressure of companies has increased sharply, which increases the possibility of risk transmission in the same industry and poses new challenges to the operation and financing of companies. From the perspective of preventive motivation of corporate cash holding, we creatively use the industry working capital shortfall as the explanatory variable to construct an Extended Cash Holding Model and a Cash Holding Value Regression Model. Taking the panel data of A-share listed companies in Shanghai and Shenzhen as samples, this paper uses the Classical Linear Regression Model and Fixed Effects Regression Model to study the relationship between industry working capital shortfall and cash holdings in the same industry, as well as the relationship between industry working capital shortfall and cash holding value. The empirical results show that industry working capital shortfall has an important impact on the cash holding level within the same industry, and the cash holding level is significantly and positively correlated with industry working capital shortfall. Moreover, this study also reveals that the industry working capital shortfall has a dual impact on cash holdings. Specifically, the higher the risk of industry working capital shortfall, the lower the value of company cash holdings. The conclusions of this paper not only extend the research on cash holding but also provide support and reference for companies to optimize the cash holding value.

An Exploratory Analysis of Related Macroeconomic Indicators as Determinants to Economic Growth

The relative movement of certain individual economic indicators to the movement of per capita income is a well-researched field. In addition to that space, this study aimed at regressing economic development in terms of per capita incomes to selected miscellaneous indicators. This by itself may not be a new approach to understanding the interrelationships that exist between other relevant economic parameters, but localized and exploratory research in this regard does prove to have some key insights regarding the interactions of economic indicators in the Philippines. Using the Classical Linear Regression Model (CLRM), The results show that increases in Net Domestic Credit, Foreign Direct Investment, and Ratio of Female to Male Labor Participation Rate increased per capita income significantly, while increases in Real Interest Rate and Carbon Dioxide Emissions decreased per capita income significantly. This meant that increasing credit and investment and incorporating more women into the labor force is a significant impetus for growth and development, while environmental degradation and the high cost of borrowing harms it.

Data-Driven Thermal Deviation Prediction in Turning Machine-Tool - A Comparative Analysis of Machine Learning Algorithms

Thermal error significantly impacts the machining precision of machine-tools. Thermal deformations in the machine-tool structure caused by the various machine heat sources is at the origin of this phenomenon. In order to ensure the expected quality of the parts, manufacturer have to run the machine-tools for hours before start producing in order to reach the machine thermal stability. This heating phase has a high negative impact on the machine productivity on one hand and on its ecological footprint on the other. This paper presents a data-driven approach to model and predict the thermal error in order to correct the tool reference position accordingly. The automatic adjustment of tool position allows to produce parts with the expected quality and precision regardless of the thermal state of the machines, which substantially increase their productivity. For this purpose, temperature sensors as well as high precision tool position measurement instruments are deployed on a Tornos SwissNano4 machine-tool. A set of experiments are conducted to collect data related to these two measurements. Four major Machine Learning algorithms are trained using a subset of the collected data and tested with the remaining data subset. Quantitative and comparative analysis shows that three of the four algorithms have a prediction with a mean Absolute Error (MAE) below 1µm and a Correlation Coefficient higher than 90%. Even classical linear regression models are able to predict the thermal error with high accuracy. Advanced Machine Learning techniques show high potential to provide a better prediction accuracy.

Financial Management Practices and Performance of Commercial and Services Companies Listed at Nairobi Securities Exchange, Kenya

The listed firms’ contribution to the economies of countries such as Kenya is tremendous. Despite the contribution by the commercial and services companies listed at Nairobi Securities Exchange towards the economy of the country, majority have for long time been experiencing poor and declining performance and some have been suspended and even end up being delisted from the activities at Nairobi Securities Exchange in the last few years. The survival of businesses, companies and firms is a matter of global concern as much of their failure is mainly attributed to poor financial management approaches in the firms. Determining the effect of financial management practices and examining the effect of working capital management, cash budgeting, fixed asset management and capital structure on financial performance of companies listed at Nairobi Securities Exchange under commercial and services segment were respectively the general and specific objectives of this study. Trade–off theory, Contingency theory, Modigliani Miller capital structure theory and Fisher separation theorem were adopted in this study. This study adopted the positivism research philosophy concept that linked with objectivism concept. The study employed explanatory research design and the target population were eight commercial and service companies listed at NSE and were operational between the year 2009 and 2020. The study relied on secondary panel data which was obtained from the various sources such as financial statements and annual statements which were available at the firms’ websites and Nairobi Securities Exchange data. Descriptive as well as inferential analysis were utilized in analyzing the data. Descriptive statistics was able to produce trends, frequencies, means and standard deviations while inferential analysis entailed correlation and regression analysis. The study conducted the diagnostic tests namely normality, linearity, multicollinearity, unit root test, Heteroscedasticity, autocorrelation and Hausman tests. The diagnostic tests performed on the data gave an indication that there were no violations of the classical linear regression model assumptions and that data was not biased, had no inconsistencies or biased parameters. The key findings from the study were that working capital management, cash budgeting and fixed asset management have a positive and significant influence on the financial performance of commercial and services companies while capital structure negatively and insignificantly influenced the financial performance. The study concluded that working capital management, cash budgeting and fixed asset management positively and significantly affected financial performance while capital structure negatively and insignificantly affected financial performance of the companies listed at Nairobi Securities Exchange under commercial and services segment. The study recommends that firms have to ensure they have the required liquidity by ensuring that their assets and liabilities are well managed; entities have to ensure they have enough cash to meet their daily operations, finance their growth, ensure unexpected payments are met as well as sustained and firms have to ensure their non-current assets are well tracked and safeguarded in order to increase their fixed asset turnover ratios. Furthermore, the study recommends the firms to establish their optimal capital structure levels so as to ensure it positively influences the financial performance which can be achieved by ensuring the firms reach the optimal debt to equity ratios.

Artificial Intelligence and Machine Learning: What You Always Wanted to Know but Were Afraid to Ask

Artificial Intelligence and Machine Learning: What You Always Wanted to Know but Were Afraid to Ask

The Impact of Tax Revenue on Economic Growth in Somalia

The study used a simple endogenous growth model to analyze the impact of tax revenue on economic growth in Somalia. The study used a classical linear regression model based on the OLS estimation method to establish the nature and strength of relationship between taxation and economic growth. Pre-estimation tests were carried out to determine heteroskasdicity, correlation, multicollinearity and normality of the variables. The results revealed overall significance of the explanatory variables in explaining GDP. In view of these findings, the coefficient of determination showed that 49.3 percent of the variation in GDP is explained by income and corporate taxes, international trade taxes and domestic indirect taxes. The findings further revealed that the coefficient of domestic indirect taxes was negative and influencing the economic growth in Somalia. The implications of this is that policy makers should focus more on income and corporate taxes and international trade taxes so as to generate more revenue by improving the tax collection system, eliminating fraud, evasion and corruption Keywords: Tax revenue, GDP, evasion and corruption DOI: 10.7176/JESD/13-2-07 Publication date: January 31 st 2022

Accumulated long-term care benefits by risk assessment scales for incident functional disability: A six-year follow-up study of long-term care receipt data

Objectives This study aims to evaluate the differences in the cumulative benefit costs of public long-term care [LTC] insurance services, using a risk assessment scale score, which predicts incident functional disability among older people.Methods A baseline survey was conducted in 2010 involving individuals aged 65 and above from 12 municipalities in Japan who were not eligible for public LTC insurance benefits (response rate: 64.7%). Using public LTC claim records, we followed LTC service costs among 46,616 individuals over a period of about six years (up to 76 months). We used risk assessment scales to assess incident functional disability (0-48). We adopted a classical linear regression model, Tobit regression model, and linear regression with multiple imputation for missing values.Results Overall, 7,348 (15.8%) of the participants had used LTC services during the follow-up period. The risk assessment score for incident functional disability was positively associated with the cumulative costs of LTC services per person, length of usage period of LTC services, and proportion of people certified for long-term care/support need and for over long-term care level 2. After adjusting for confounding variables, the six-year cumulative costs of LTC services were around JPY 31.6 thousand higher per point of risk score (95% confidence interval [CI]: 28.3 to 35.0). The costs were around JPY 8.9 thousand (95%CI: 6.5 to 11.3)higher in the low score group (risk score ≤ 16), and JPY 75.3 thousand (95%CI: 67.4 to 83.1) higher in the high score group (risk score ≥ 17). When we adopted other estimated models, the major results and trends were not largely different.Conclusions In this study, the risk assessment scale score could estimate subsequent LTC benefit costs. Community interventions to improve and maintain variable aspects of risk assessment scores may help contribute to a reduction in public LTC benefits within municipalities.

Macroeconomic determinants of savings in a developing economy: a new empirical evidence from Nigeria

Abstract Subject and purpose of work: The issue of savings and what motivates it has continued to generate argument. This study, therefore, investigated the determinants of savings in Nigeria over the period 1980 to 2017. Materials and methods: The study employed the Classical Linear Regression Model in its analysis. Results: The results showed that the determinants of savings include per capita income, gross fixed capital formation, financial deepening and exchange rate. Interest rate and inflation rate showed negative impact on savings. Conclusions: The study recommended that the variables that showed positive impact on savings rate should be properly directed with the relevant policy tools to ensure higher saving rates. Again, the government should direct spending towards economic activities that encourage the creation of more jobs and investments. This will enable individuals, firms and governments to have more money to save. Finally, the Monetary Authorities should pursue financial deepening policies and implement strategies that will enhance the increase of savings in Nigeria.

Модели машинного обучения в задаче прогнозирования природно-ресурсного потенциала Пермского края

In the article we consider a complex indicator of region natural resource potential modeling and forecasting quality improvement using different machine learning models. Problem under consideration importance is determined by the fact that the models traditionally used for these purposes demonstrate either low quality, or high configuration and parameters evaluation difficulty. The aim of the study is determination of machine learning models that provide the optimal values of various modeling quality metrics. Materials and methods. For this study purposes we considered the multiple linear regression, decision tree, random forest, gradient boosting and multilayer perceptron mo¬dels. We used the determination coefficient R2, the root mean square error of modeling RMSE, the average absolute error of modeling MAE, and the relative error of prediction for 1 and 2 time intervals as quality metrics. This study is based on data of the complex indicator of the Perm Region natural resource potential and the system of its determining factors in the time interval from 2001 to 2018. We evaluate models and calculate quality metrics using Pandas and Scikit-learn Python libraries in Jupiter Notebook environment. Results. According to our research the classical multiple linear regression model demonstrates the worst results for all quality metrics under consideration. The decision tree model demonstrates determination coefficient maximum value and minimum root mean square error and mean absolute error. Minimum relative forecasting error for 2017 is provided by the gradient boosting model, for 2018 – by the multilayer perceptron model. Conclusion. Our study allows us to affirm that nonlinear machine learning models for the task of region natural resource potential modeling and forecasting demonstrate better approximating and predictive properties compared to multiple linear regression and thus can be used to improve the quality of natural resource management.

The Impact of Foreign Direct Investment on the Economic Growth of South Caucasus Countries

This paper focuses on the effects of foreign direct investment inflows on the economic growth in a panel of three South Caucasus countries using data from 1996-2019 periods. In this study, we applied the following control variables; trade openness, investment, real exchange rate, and population growth. Classical linear regression model was employed in this paper. Ordinary least squares methods are used for estimation. Empirical results revealed that there is no significant effect of FDI inflows on economic growth. The results show that inward FDI stock-to-GDP ratio and real GDP growth rate are positively correlated.