Classical Regression Approach Research Articles

Generalized Support Vector Regression and Symmetry Functional Regression Approaches to Model the High-Dimensional Data

The analysis of the high-dimensional dataset when the number of explanatory variables is greater than the observations using classical regression approaches is not applicable and the results may be misleading. In this research, we proposed to analyze such data by introducing modern and up-to-date techniques such as support vector regression, symmetry functional regression, ridge, and lasso regression methods. In this study, we developed the support vector regression approach called generalized support vector regression to provide more efficient shrinkage estimation and variable selection in high-dimensional datasets. The generalized support vector regression can improve the performance of the support vector regression by employing an accurate algorithm for obtaining the optimum value of the penalty parameter using a cross-validation score, which is an asymptotically unbiased feasible estimator of the risk function. In this regard, using the proposed methods to analyze two real high-dimensional datasets (yeast gene data and riboflavin data) and a simulated dataset, the most efficient model is determined based on three criteria (correlation squared, mean squared error, and mean absolute error percentage deviation) according to the type of datasets. On the basis of the above criteria, the efficiency of the proposed estimators is evaluated.

A statistical approach on distribution and seasonal habitat use of waterfowl and shorebirds in Çıldır Lake (Ardahan, Türkiye).

Wetlands are crucial habitats for both migrant and resident bird assemblages. The distribution and habitat preferences of birds in aquatic ecosystems are significantly influenced by environmental and ecological factors that critically impact the relevant habitats. In order to reveal the distribution and habitat preferences of the birds, many statistical models and methodologies are employed in ecology and conservation biology. Herein, we investigated the effects of year, season, habitat, and species variables on the distribution and population dynamics of waterfowls and shorebirds associated with the wetland. In this regard, field surveys were carried out in and around Çıldır Lake (Ardahan, Türkiye) between April 2017 and September 2018 to examine the distribution of waterfowls and shorebirds and variations in population sizes. As an experimental design, a stratified random sampling design was used to assess bird fauna in the four dominant habitat types (open water surface, reeds, grasslands, and agricultural areas) in the study area. Accordingly, a total of 51 waterfowl and shorebird species were identified during the study period. Of the identified families, Anatidae (n = 18), Scolopacidae (n = 8), and Ardeidae (n = 8) were the most common families. Considering bird species, common coot Fulica atra and mallard Anas platyrhynchos were the most abundant species. The dependent variable (bird populations) was compared with the independent variables (year, season, habitat, and species). The population in 2018 decreased by 13% in comparison to the population in 2017 (p < 0.05). Once the reed area was considered as the reference, the population density in the water surface habitat increased by 65% (p < 0.001). In relation to seasonal reference, a 65% increase in population growth in spring was recorded in comparison to the growth in fall (p < 0.001). On the other hand, no statistical differences were noted in population growth in winter and summer ((p > 0.05). With respect to the reference species (Anas crecca), critical differences in species fluctuation were observed among species (p < 0.001). Consequently, the findings of the present study suggest that seasonal factor might be of the substantial factors linked to the habitat composition. However, more descriptive and predictive analytical methods are needed beyond classical regression approaches in habitat use and selection studies at bird ecology.

Double burden of malnutrition among women of reproductive age in Bangladesh: A comparative study of classical and Bayesian logistic regression approach.

Although the prevalence of undernutrition among women of reproductive age has declined in Bangladesh, the increase in the prevalence of overnutrition remains a major challenge. To achieve Sustainable Development Goal 2.2, it is important to identify the drivers of the double burden of malnutrition on women in Bangladesh. The Bangladesh Demographic and Health Survey, 2017-2018 was used to model the relationship between the double burden of malnutrition among women and the risk factors using a logistic regression model under the classical and Bayesian frameworks and performed the comparison between the regression models based on the narrowest confidence interval. Regarding the Bayesian application, the Metropolis-Hastings algorithm with two types of prior information (historical and noninformative prior) was used to simulate parameter estimates from the posterior distributions. The Boruta algorithm was used to determine the significant predictors. Almost half of reproductive aged women experienced a form of malnutrition (12% were underweight, 26.1% were overweight, and 6.8% were obese). In terms of the narrowest interval estimate, it was found that Bayesian logistic regression with informative priors performs better than the noninformative priors and the classical logistic regression model. Women who were older, highly educated, from rich families, unemployed, and from urban residences were more likely to experience the double burden of malnutrition. This study recommended using the historical prior as the informative prior rather than the flat/noninformative prior to estimating the parameter uncertainty if historical data are available. The double burden of malnutrition among women is a major public health challenge in Bangladesh. This study was to determine the impact of effective risk factors on the double burden of malnutrition among women by applying the Bayesian framework. Using both informative and noninformative priors, "historical prior" was proposed as informative prior information. The main strength is that the proposed prior (historical prior) provided improved estimation as compared to the flat prior distribution.

Impact of Using Double Positive Samples in Deming Regression

In the method comparison approach, two measurement errors are observed. The classical regression approach (linear regression) method cannot be used for the analysis because the method may yield biased and inefficient estimates. In view of that, the Deming regression is preferred over the classical regression. The focus of this work is to assess the impact of censored data on the traditional regression, which deletes the censored observations compared to an adapted version of the Deming regression that takes into account the censored data. The study was done based on simulation studies with NLMIXED being used as a tool to analyse the data. Eight different simulation studies were run in this study. Each of the simulation is made up of 100 datasets with 300 observations. Simulation studies suggest that the traditional Deming regression which deletes censored observations gives biased estimates and a low coverage, whereas the adapted Deming regression that takes censoring into account gives estimates that are close to the true value making them unbiased and gives a high coverage. When the analytical error ratio is misspecified, the estimates are as well not reliable and biased.

Coming of Age: Renovation Premiums in Housing Markets

Abstract We rely on novel textual analysis of real estate listings and identify renovated dwellings in a dataset of Norwegian transactions to estimate the renovation premium in an urban housing market. The renovation premium is estimated in a hedonic framework by classical regression approaches and a random forest algorithm. The strength of the latter is that it allows for a more complex interplay between the renovation premium and explanatory variables. We estimate a significant positive renovation premium of 5–7 percent for renovated dwellings and a negative premium of 9–10 percent for unmaintained/neglected dwellings. These averages mask significant variations in these premiums over time, particularly, a counter-cyclical effect. Omitting renovation information also has implications for estimated short-term house price growth. Unmaintained dwellings tend to transact more in the fourth quarter, indicating that parts of the seasonal price variation reported in the literature are due to compositional variation with respect to renovation. This composition effect bias price movement estimates downward, if uncontrolled for, as unmaintained dwellings transact at significantly lower prices.

Differential richness inference for 16S rRNA marker gene surveys

BackgroundIndividual and environmental health outcomes are frequently linked to changes in the diversity of associated microbial communities. Thus, deriving health indicators based on microbiome diversity measures is essential. While microbiome data generated using high-throughput 16S rRNA marker gene surveys are appealing for this purpose, 16S surveys also generate a plethora of spurious microbial taxa.ResultsWhen this artificial inflation in the observed number of taxa is ignored, we find that changes in the abundance of detected taxa confound current methods for inferring differences in richness. Experimental evidence, theory-guided exploratory data analyses, and existing literature support the conclusion that most sub-genus discoveries are spurious artifacts of clustering 16S sequencing reads. We proceed to model a 16S survey’s systematic patterns of sub-genus taxa generation as a function of genus abundance to derive a robust control for false taxa accumulation. These controls unlock classical regression approaches for highly flexible differential richness inference at various levels of the surveyed microbial assemblage: from sample groups to specific taxa collections. The proposed methodology for differential richness inference is available through an R package, Prokounter.ConclusionsFalse species discoveries bias richness estimation and confound differential richness inference. In the case of 16S microbiome surveys, supporting evidence indicate that most sub-genus taxa are spurious. Based on this finding, a flexible method is proposed and is shown to overcome the confounding problem noted with current approaches for differential richness inference.Package availability: https://github.com/mskb01/prokounter

Recurrent neural network-based prediction of critical heat flux in rod bundles with non-uniform axial power shape

The Critical Heat Flux (CHF) prediction is one of the most important phenomena for the safety analyses of nuclear reactors. Current state-of-the-art CHF correlations for PWR fuel assemblies are based on classical regression approaches. In this paper a new method based on Recurrent Neural Networks (RNNs) for the calculation of the CHF in fuel assemblies is proposed. The sequential nature of the RNN allows it to learn the influence of the geometrical features of the fuel assembly and of the history effect by calculating the CHF not only based on the local flow conditions at the CHF location, but also upstream from it. The model was tested on two rod bundle datasets. The results show that the neural network model can reasonably well predict both the value of the CHF and its location without the need of any additional correction factors.

Modelling of South African Hypertension: Comparative Analysis of the Classical and Bayesian Quantile Regression Approaches.

Hypertension has become a major public health challenge and a crucial area of research due to its high prevalence across the world including the sub-Saharan Africa. No previous study in South Africa has investigated the impact of blood pressure risk factors on different specific conditional quantile functions of systolic and diastolic blood pressure using Bayesian quantile regression. Therefore, this study presents a comparative analysis of the classical and Bayesian inference techniques to quantile regression. Both classical and Bayesian inference techniques were demonstrated on a sample of secondary data obtained from South African National Income Dynamics Study (2017–2018). Age, BMI, gender male, cigarette consumption and exercises presented statistically significant associations with both SBP and DBP across all the upper quantiles . The white noise phenomenon was observed on the diagnostic tests of convergence used in the study. Results suggested that the Bayesian approach to quantile regression reveals more precise estimates than the frequentist approach due to narrower width of the 95% credible intervals than the width of the 95% confidence intervals. It is therefore suggested that Bayesian approach to quantile regression modelling to be used to estimate hypertension.

Thermal performance of hybrid fly ash and copper nanofluid in various mixture ratios: Experimental investigation and application of a modern ensemble machine learning approach

The purpose of the current research work is to investigate the various properties like thermal conductivity, stability and viscosity of copper and a mixture of fly ash-Copper (FA:Cu) nanoparticles (NP) suspended in water. The research experiments were conducted for a concentration of 1.0 vol% of FA:Cu hybrid nanofluid (HNF) with various mixture ratios. The measurements of thermal conductivity and viscosity were performed in the 30–60 °C temperature range. The highest thermal conductivity and viscosity values for HNF with a mixture ratio of 20:80 were obtained with a maximum amplification exceeding 83.2% and 65% than the base fluid, respectively. The properties enhancement ratio (PER) reveals that HNF with 1.0 vol% concentration enhances heat transfer for all defined mixture ratios in the reported research work. Finally, the predictability potential of an ensemble-based machine learning technique called Boosted Regression Tree (BRT), based on nanofluids temperature (T) and mixture ratio (R), were compared with the classical regression approach to simulate the thermo-physical properties of HNF. The outcomes of the BRT model in terms of (r(viscosity) = 0.9953 and r(thermal conductivity) = 0.9991) superior to the regression method with (r(viscosity) = 0.9695 and r(thermal conductivity) = 0.9539) over the viscosity and thermal conductivity prediction process.

Regression with Stable Errors Based on Order Statistics

Classical regression approaches are not robust when errors are heavy-tailed or asymmetric. That may be due to the non-existence of the mean or variance of the error distribution. Estimation based on trimmed data, which ignored outlier or leverage points, has an old history and frequently used. This procedure chooses fixed cut-off points. In this work, we use this idea recently applied for initial estimates of regression coefficients with heavy-tailed stable errors. We propose an effective procedure to calculate the cut-off points based on the tail index and skewness parameters of errors. We use the property of the existence of some moments of stable distribution order statistics. Data are trimmed based on ordered residuals of a least square regression. However, the trimmed data’s optimal number is determined based on the number of error order statistics whose variance exists. Then, we use the rest of the ordered data to estimate the regression coefficients. Based on these order statistics’ joint distribution, we analytically compute the bias and variance of the introduced estimator of regression parameters that was impossible for regression with stable errors.

Why Are Interest Rates on Bank Deposits so Low?

Using granular data of German banks for the 2003 to 2018 period, we analyze the determinants of bank rates on retail deposits. We find that a bank’s rate on sight deposits is especially low if the bank operates in rural districts, if it is not exposed to strong competition and if it provides much service. Regarding the rates on term deposits, we find that the bank’s cost situation plays a role: if the bank’s costs are high, its deposit rates are low. By transferring concepts from portfolio theory to the pass-through topic, we show that replicating portfolio approaches are often equivalent to regression approaches and that, under some assumptions, the classical regression approach corresponds to a replicating portfolio approach.

Machine learning techniques for relating liquid limit obtained by Casagrande cup and fall cone test in low-medium plasticity fine grained soils

The liquid limit is a key property of fine soils closely related to the stress-strain behaviour and other relevant characteristics of soils such as the expansive potential. There are two standardized methods for its determination, the Casagrande cup and the fall cone test, among which there are many correlations that offer heterogeneous results. In the present study, a compilation of 113 data from fine soil samples with low-medium plasticity has been carried out. Then, a comparative study of different machine learning algorithms was carried out to relate the liquid limit obtained from both methods, having in consideration other parameters such as the plastic limit and the percentages of passing through the 0.40 and 0.075 mm sieves. The result of this study has shown that extremely randomized trees algorithm provides the best performance. Consequently, the algorithm has been tuned to enhance the precision, obtaining a coefficient of determination (R2) value of 0.99. The results demonstrate the potential of machine learning techniques for relating liquid limit obtained by Casagrande's method and fall cone test in fine-grained soils with low-medium plasticity, mainly for values of the liquid limit higher than 30 for which classical linear regression approaches provide lower performance metrics.

Deep and interpretable regression models for ordinal outcomes

Outcomes with a natural order commonly occur in prediction problems and often the available input data are a mixture of complex data like images and tabular predictors. Deep Learning (DL) models are state-of-the-art for image classification tasks but frequently treat ordinal outcomes as unordered and lack interpretability. In contrast, classical ordinal regression models consider the outcome’s order and yield interpretable predictor effects but are limited to tabular data. We present ordinal neural network transformation models (ontrams), which unite DL with classical ordinal regression approaches. ontrams are a special case of transformation models and trade off flexibility and interpretability by additively decomposing the transformation function into terms for image and tabular data using jointly trained neural networks. The performance of the most flexible ontram is by definition equivalent to a standard multi-class DL model trained with cross-entropy while being faster in training when facing ordinal outcomes. Lastly, we discuss how to interpret model components for both tabular and image data on two publicly available datasets.

Bayesian estimation of impact in experimental interventions with continuous outcome

Impact evaluation in experimental intervention is an estimation of how design and implementation strategies affect the overall outcome and effectiveness of interventions by establishing causation. It provides the approach for determining the “Average treatment effect” and the “Average effect of treatment on the treated” otherwise known as impact. One of several approaches to estimating impact in experimental intervention is the use of the Difference-in-Differences (DID) estimation procedure based on classical regression approach which often overstates reality and rely on assumptions that are often violated in real life. Bayesian estimation is popular in literature but not widely applied in evaluation of experimental intervention. This study developed an approach for estimating impact based on the Bayesian estimation procedure. It derived a distribution for the difference of difference variable when outcome is continuous and normally distributed using the convolution procedure. The likelihood of distribution of the difference of difference variable was combined with the normal prior, at a specific value of the prior hyperparameter obtained from previous study, to estimate the posterior distribution. Using data from computer simulation and secondary data, the posterior mean was estimated. Also, classical regression approach was used to estimate the impact. Results from the Bayesian approach produced lower impact estimate and lower mean square error compared with the classical approach. This study provides a better alternative to estimating impact in experimental interventions.

Estimating Pole Capacity From Radio Network Performance Statistics by Supervised Learning

Network dimensioning is a critical task for cellular operators to avoid degraded user experience and unnecessary upgrades of network resources with changing mobile traffic patterns. For this purpose, smart network planning tools require accurate cell and user capacity estimates. In these tools, throughput is often used as a capacity metric due to its close relationship with user satisfaction. In this work, a comprehensive analysis is carried out to compare different well-known Supervised Learning (SL) algorithms for estimating cell and user throughput in the DownLink in busy hours from radio measurements collected on a cell basis in the Operation Support System (OSS). The considered SL approaches include random forest, shallow multi-layer perceptron, support vector regression and k-nearest neighbors. Such algorithms are compared with classical multiple linear regression and deep learning approaches considered in previous works. All these algorithms are tested in two radio access technologies: High Speed DownLink Packet Access (HSDPA) and Long Term Evolution (LTE). To this end, two datasets with the most relevant performance indicators per technology are collected from live cellular networks. Results show that non-deep SL algorithms are the most appropriate option for applications with storage constraints, such as network planning tools, since they provide a higher accuracy with reduced datasets.

Diversity-driven ANN-based ensemble framework for seasonal low-flow analysis at ungauged sites

Low-flow estimation at ungagged sites is a challenging task. Ensemble-based machine learning regression has recently been utilized in modeling hydrologic phenomena and showed improved performance compared to classical regional regression approaches. Ensemble modeling mainly revolves around developing a proper training framework of the individual learners and combiners. An ensemble framework is proposed in this study to drive the generalization ability of the sub-ensemble models and the ensemble combiners. Information mixtures between the subsamples are introduced and, unlike common ensemble frameworks, are explicitly devoted to the ensemble members as well as ensemble combiners. The homogeneity paradigm is developed via a two-stage resampling approach, which creates sub-samples with controlled information mixture levels for the training of the individual learners. Artificial neural networks are used as sub-ensemble members in combination with a number of ensemble integration techniques. The proposed model is applied to estimate summer and winter low-flow quantiles for catchments in the province of Québec, Canada. The results show significant improvement when compared to the other models presented in the literature. The obtained homogeneity levels from the optimum ensemble models demonstrate the importance of utilizing the diversity concept in ensemble learning applications.

Sensitivity analysis of treatment effect to unmeasured confounding in observational studies with survival and competing risks outcomes.

No unmeasured confounding is often assumed in estimating treatment effects in observational data, whether using classical regression models or approaches such as propensity scores and inverse probability weighting. However, in many such studies collection of confounders cannot possibly be exhaustive in practice, and it is crucial to examine the extent to which the resulting estimate is sensitive to the unmeasured confounders. We consider this problem for survival and competing risks data. Due to the complexity of models for such data, we adapt the simulated potential confounder approach of Carnegie et al(2016), which provides a general tool for sensitivity analysis due to unmeasured confounding. More specifically, we specify one sensitivity parameter to quantify the association between an unmeasured confounder and the exposure or treatment received, and another set of parameters to quantify the association between the confounder and the time-to-event outcomes. By varying the magnitudes of the sensitivity parameters, we estimate the treatment effect of interest using the stochastic expectation-maximization (EM) and the EM algorithms. We demonstrate the performance of our methods on simulated data, and apply them to a comparative effectiveness study in inflammatory bowel disease. An R package "survSens" is available on CRAN that implements the proposed methodology.

Modeling Burglar Incidents Data Using Generalized and Quasi Poisson Regression Models: A Case Study of Nairobi City County, Kenya

Serious violent crimes including Burglary, dangerous drug trafficking and sexual offenses make up the bulk of incidents filed at police stations daily. These crimes related activities poses a serious threat to the peace and serenity of a nation as far as safety is concerned. Burglar incidents data are often discrete and do not conform to the general assumptions of the linear model and its variants. Ordinarily, such data could be modeled using a linear regression approach to derive the relationship between the response variable to the underlying covariates. However, the narrowing of the gap between city and suburban burglar crime rates brings about variability invalidating the application of Ordinary linear regression approaches. The main objective of this study focused on the comparative use of Generalized Poisson and Quasi-Poisson models as an alternative to the classical linear regression approach in modeling Burglar incidents in Nairobi City County, Kenya. The prime advantage of applying Quasi Poisson in count data analysis is that it fixes the basic fallacy of assuming homogeneity in data and allows estimation of dispersion. The study used secondary data covering Eight (8) Nairobi's Administrative Divisions from the National Crime Research Center (NCRC) for the period 2016-2018. The comparison criteria were the Akaike Information (AIC) Criterion and Deviance Information Criterion (DIC) alongside other model diagnostics tests. Application of this results in burglar events revealed that the number of incidents in the study area are Under-dispersed with the risks of experiencing Burglar crime being above 5% in all the locations surveyed. In an attempt to explore Burglar to location relationship, results from study proved that Generalized Poisson Model performed better than the Quasi Poisson model having posted the lowest AIC value.

Flexible modeling of ratio outcomes in clinical and epidemiological research.

In medical studies one frequently encounters ratio outcomes. For modeling these right-skewed positive variables, two approaches are in common use. The first one assumes that the outcome follows a normal distribution after transformation (e.g. a log-normal distribution), and the second one assumes gamma distributed outcome values. Classical regression approaches relate the mean ratio to a set of explanatory variables and treat the other parameters of the underlying distribution as nuisance parameters. Here, more flexible extensions for modeling ratio outcomes are proposed that allow to relate all the distribution parameters to explanatory variables. The models are embedded into the framework of generalized additive models for location, scale and shape (GAMLSS), and can be fitted using a component-wise gradient boosting algorithm. The added value of the new modeling approach is demonstrated by the analysis of the LDL/HDL cholesterol ratio, which is a strong predictor of cardiovascular events, using data from the German Chronic Kidney Disease Study. Particularly, our results confirm various important findings on risk factors for cardiovascular events.

Stochastic search variable selection for split-plot and blocked screening designs

Split-plot definitive screening and blocked definitive screening designs have been developed for detecting active main effects and second-order effects in screening experiments when split-plot and block structures exist. In the literature, multistage regression and forward stepwise regression methods were proposed for data analysis on the two types of designs. However, classical regression approaches present limitations and potential problems. First, the degrees of freedom may not be large enough to estimate all active effects. Second, the restricted maximum-likelihood estimates for variances of whole-plot and block errors can be zero. To solve these problems and enhance detection capability, we propose a stochastic search variable selection (SSVS) method based on Bayesian theory. Different from the existing Bayesian approaches for split-plot and blocked designs, the proposed SSVS method can perform variable selections and choose models that follow the effect heredity principle. Markov chain Monte Carlo and Gibbs sampling are applied and a general WinBUGS code, which can be used for any split-plot and blocked screening design, is provided. Simulation studies are conducted and their results show that the proposed SSVS method effectively controls the false discovery rate and has higher detection capability than the two regression methods.