Conventional Regression Models Research Articles

Fuzzy linear regression model on mulberry silk cocoon characteristics

Purpose This study aims to propose a fuzzy linear regression (FLR) model to deal with the vagueness or fuzziness of the underlying relationship between silk cocoon and yarn quality. Design/methodology/approach Shell ratio percentage, defective cocoon percentage and cocoon volume are considered as significant independent variables to predict the quality of silk cocoons. Input and output parameters of the FLR model are considered as non-fuzzy, but the underlying relationship between the variables is assumed to be fuzzy. Findings The fuzzy regression model shows its superiority against conventional multiple linear regression model for estimation of silk cocoon characteristics. It is inferred that the fuzziness in underlying relationship between the parameters can be handled efficiently by FLR model. Originality/value A rigorous experimental work has been carried out on 40 lots of mulberry silk cocoons to generate real-world data set to characterize silk cocoons’ quality in a fuzzy environment.

Spatial-Temporal Analysis of Injury Severity with Geographically Weighted Panel Logistic Regression Model

This study is intended to investigate the influencing factors of injury severity by considering the heterogeneity issue of unobserved factors at different arterials and the spatial attributes in geographically weighted regression models. To achieve the objectives, geographically weighted panel logistic regression model was developed, in which the geographically weighted logistic regression model addressed the injury severity from the spatial perspective, while the panel data model accommodated the heterogeneity attributed to unobserved factors from the temporal perspective. The geo-crash data of Las Vegas metropolitan area from 2014 to 2016 was collected, involving 27 arterials with 25,029 injury samples. By comparing the conventional logistic regression model and geographically weighted logistic regression models, the geographically weighted panel logistic regression model showed preference to the other models. Results revealed that four main factors, human-beings (drivers/pedestrians/cyclists), vehicles, roadway, and environment, were potentially significant factors of increasing the injury severity. The findings provide useful insights for practitioners and policy makers to improve safety along arterials.

Spatial Auto-regressive Dependency Interpretable Learning Based on Spatial Topological Constraints

Spatial regression models are widely used in numerous areas, including detecting and predicting traffic volume, air pollution, and housing prices. Unlike conventional regression models, which commonly assume independent and identical distributions among observations, existing spatial regression requires the prior knowledge of spatial dependency among the observations in different spatial locations. Such a spatial dependency is typically predefined by domain experts or heuristics. However, without sufficient consideration on the context of the specific prediction task, it is prohibitively difficult for one to pre-define the numerical values of the spatial dependency without bias. More importantly, in many situations, the existing techniques are insufficient to sense the complete connectivity and topological patterns among spatial locations (e.g., in underground water networks and human brain networks). Until now, these issues have been extremely difficult to address and little attention has been paid to the automatic optimization of spatial dependency in relation to a prediction task, due to three challenges: (1) necessity and complexity of modeling the spatial topological constraints; (2) incomplete prior spatial knowledge; and (3) difficulty in optimizing under spatial topological constraints that are usually discrete or nonconvex. To address these challenges, this article proposes a novel convex framework that automatically jointly learns the prediction mapping and spatial dependency based on spatial topological constraints. There are two different scenarios to be addressed. First, when the prior knowledge on existence of conditional independence among spatial locations is known (e.g., via spatial contiguity), we propose the first model named Spatial-Autoregressive Dependency Learning I (SADL-I) to further quantify such spatial dependency. However, when the knowledge on the conditional independence is unknown or incomplete, our second model named Spatial-Autoregressive Dependency Learning II (SADL-II) is proposed to automatically learn the conditional independence pattern as well as quantify the numerical values of the spatial dependency based on spatial topological constraints. Topological constraints are usually discrete and nonconvex, which makes them extremely difficult to be optimized together with continuous optimization problems of spatial regression. To address this, we propose a convex and continuous equivalence of the original discrete topological constraints with a theoretical guarantee. The proposed models are then transferred to convex problems that can be iteratively optimized by our new efficient algorithms until convergence to a global optimal solution. Extensive experimentation using several real-world datasets demonstrates the outstanding performance of the proposed models. The code of our SADL framework is available at: http://mason.gmu.edu/∼lzhao9/materials/codes/SADL.

Eigenvector Spatial Filtering-Based Logistic Regression for Landslide Susceptibility Assessment

Logistic regression methods have been widely used for landslide research. However, previous studies have seldom paid attention to the frequent occurrence of spatial autocorrelated residuals in regression models, which indicate a model misspecification problem and unreliable results. This study accounts for spatial autocorrelation by implementing eigenvector spatial filtering (ESF) into logistic regression for landslide susceptibility assessment. Based on a landslide inventory map and 11 landslide predisposing factors, we developed the eigenvector spatial filtering-based logistic regression (ESFLR) model, as well as a conventional logistic regression (LR) model and an autologistic regression (ALR) model for comparison. The three models were evaluated and compared in terms of their prediction capability and model fit. The ESFLR model performed better than the other two models. The overall predictive accuracy of the ESFLR model was 90.53%, followed by the ALR model (76.21%) and the LR model (74.76%), and the areas under the ROC curves for the ESFLR, ALR and LR models were 0.957, 0.828 and 0.818, respectively. The ESFLR model adequately addressed the spatial autocorrelation of residuals by reducing the Moran’s I value of the residuals to 0.0270. In conclusion, the ESFLR model is an effective and flexible method for landslide analysis.

Sediment transport modeling in rigid boundary open channels using generalize structure of group method of data handling

Sediment transport in open channels has complicated nature and finding the analytical models applicable for channel design in practice is a quite difficult task. To this end, behind theoretical consideration of the open channel sediment transport through incorporating of four fundamental characteristics of fluid, flow, sediment and channel, recently, machine learning techniques are used for modeling of sediment transport in open channels. However, most of the studies in the literature used limited number of data for model development neglecting some effective parameters involved which may affect their performances. Moreover, most of this studies had not provided a comprehensive explicit equation for future use. Accordingly, this study applied four machine learning techniques of Gene Expression Programming (GEP), Extreme Learning Machine (ELM), Generalized Structure Group Method of Data Handling (GS-GMDH) and Fuzzy c-means based Adaptive Neuro-Fuzzy Inference System (FCM-ANFIS) to model sediment transport in open channels. Four existing data sets in the literature with wide ranges of pipe size, sediment size, sediment volumetric concentration, channel bed slope and flow depth are used for the model development. The recommended models are compared with their corresponding conventional regression models taken from the literature in terms of different statistical performance indices. Results indicate superiority of the machine leaning techniques to the conventional multiple non-linear regression models. Although, developed GEP, ELM, GS-GMDH and FCM-ANFIS models have almost same performances, GS-GMDH gives slightly better performance which can be linked to the generalized structure of this approach. A MATLAB code is provided to calculate the sediment transport in open channel for practical engineering.

Predictive Analysis of First Abbreviated New Drug Application Submission for New Chemical Entities Based on Machine Learning Methodology.

Generic drug products are approved by the US Food and Drug Administration (FDA) through Abbreviated New Drug Applications (ANDAs). The ANDA review and approval involves multiple offices across the FDA. Forecasting ANDA submissions can critically inform resource allocation and workload management. In this work, we used machine learning (ML) methodologies to predict the time to first ANDA submissions referencing new chemical entities following their earliest lawful ANDA submission dates. Drug product information, regulatory factors, and pharmacoeconomic factors were used as modeling inputs. The random survival forest ML method, as well as the conventional Cox model, was used for ANDA submission predictions. The ML method outperformed the conventional Cox regression model in predictive performance that was adequately assessed by both internal and external validations. In conclusion, it can potentially serve as an effective forecasting tool for strategic workload and research planning for generic applications.

A novel distribution-free hybrid regression model for manufacturing process efficiency improvement

This work is motivated by a particular problem of a modern paper manufacturing industry, in which the maximum efficiency of the fiber-filler recovery process is desired. A lot of unwanted materials along with valuable fibers and fillers come out as a by-product of the paper manufacturing process and mostly goes out as waste. The job of an efficient Krofta supracell is to separate the unwanted materials from the valuable ones so that fibers and fillers can be collected from the waste materials and reused in the manufacturing process. The efficiency of Krofta depends on several critical process parameters and monitoring them is a difficult proposition. To solve this problem, we propose a novel hybridization of regression trees (RT) and artificial neural networks (ANN), namely hybrid RT-ANN model, to solve the problem of low recovery percentage of the supracell. This model is used to achieve the goal of improving supracell efficiency, viz., gain in percentage recovery. Also, theoretical results for the universal consistency of the proposed model are given with the optimal value of a vital model parameter. Experimental findings show that the proposed hybrid RT-ANN model achieves higher accuracy in predicting Krofta recovery percentage than other conventional regression models for solving the Krofta efficiency problem. This work will help the paper manufacturing company to become environmentally friendly with minimal ecological damage and improvement in their waste recovery process.

Individualized Multilayer Tensor Learning With an Application in Imaging Analysis

This work is motivated by multimodality breast cancer imaging data, which is quite challenging in that the signals of discrete tumor-associated microvesicles are randomly distributed with heterogeneous patterns. This imposes a significant challenge for conventional imaging regression and dimension reduction models assuming a homogeneous feature structure. We develop an innovative multilayer tensor learning method to incorporate heterogeneity to a higher-order tensor decomposition and predict disease status effectively through utilizing subject-wise imaging features and multimodality information. Specifically, we construct a multilayer decomposition which leverages an individualized imaging layer in addition to a modality-specific tensor structure. One major advantage of our approach is that we are able to efficiently capture the heterogeneous spatial features of signals that are not characterized by a population structure as well as integrating multimodality information simultaneously. To achieve scalable computing, we develop a new bi-level block improvement algorithm. In theory, we investigate both the algorithm convergence property, tensor signal recovery error bound and asymptotic consistency for prediction model estimation. We also apply the proposed method for simulated and human breast cancer imaging data. Numerical results demonstrate that the proposed method outperforms other existing competing methods. Supplementary materials for this article are available online.

Diabetes and cancer risk: A Mendelian randomization study.

Earlier cohort studies using conventional regression models have consistently shown an increased cancer risk among individuals with type 2 diabetes. However, reverse causality and residual confounding due to common risk factors could exist, and it remains unclear whether diabetes per se contributes to cancer development. Mendelian randomization analyses might clarify the true association between diabetes and cancer risk. We conducted a case–cohort study with 10,536 subcohort subjects and 3,541 newly diagnosed cancer cases derived from 32,949 eligible participants aged 40–69 years within the Japan Public Health Center‐based Prospective Study. With 29 known type 2 diabetes susceptibility variants, we used an inverse variance‐weighted method to estimate hazard ratios for the associations of diabetes with risks of total and site‐specific cancers. The hazard ratios of cancer per doubling of the probability of diabetes were 1.03 (95% confidence interval [CI], 0.92–1.15) overall, 1.08 (95% CI: 0.73–1.59) for the pancreas, 0.80 (95% CI: 0.57–1.14) for the liver and 0.90 (95% CI: 0.74–1.10) for the colorectum. Additional analyses, using publicly available large‐scale genome‐wide association study data on colorectal cancer in Japan, resulted in a narrower CI (hazard ratio: 1.00; 95% CI: 0.93–1.07). In this prospective Mendelian randomization study with a large number of incident cancer cases, we found no strong evidence to support associations between diabetes and overall and site‐specific cancer risks. Our findings suggest that there is little evidence to support the genetic role of type 2 diabetes in cancer development in the Japanese population.

Random Forests and Cubist Algorithms for Predicting Shear Strengths of Rockfill Materials

The shear strength of rockfill materials (RFM) is an important engineering parameter in the design and audit of geotechnical structures. In this paper, the predictive reliability and feasibility of random forests and Cubist models were analyzed by estimating the shear strength from the relative density, particle size, distribution (gradation), material hardness, gradation and fineness modulus, and confining (normal) stress. For this purpose, case studies of 165 rockfill samples have been applied to generate training and testing datasets to construct and validate the models. Thirteen key material properties for rockfill characterization were selected to develop the proposed models. Validation and comparison of the models have been performed using the root mean square error (RMSE), coefficient of determination (R2), and mean estimation error (MAE) between the measured and estimated values. A sensitivity analysis was also conducted to ascertain the importance of various inputs in the prediction of the output. The results demonstrated that the Cubist model has the highest prediction performance with (RMSE = 0.0959, R2 = 0.9697 and MAE = 0.0671), followed by the random forests model with (RMSE = 0.1133, R2 = 0.9548 and MAE= 0.0665), the artificial neural network (ANN) model with (RMSE = 0.1320, R2 = 0.9386 and MAE = 0.0841), and the conventional multiple linear regression technique with (RMSE = 0.1361, R2 = 0.9345 and MAE = 0.0888). The results indicated that the Cubist and random forests models are able to generate better predictive results of the shear strength of RFM than ANN and conventional regression models. The Cubist model was considered to be more promising for interpreting the complex relationships between the influential properties of RFM and the shear strengths of RFM to some extent, which can be extremely helpful in estimating the shear strength of rockfill materials.

An Empirical Comparison of Multiple Linear Regression and Artificial Neural Network for Concrete Dam Deformation Modelling

Deformation predicting models are essential for evaluating the health status of concrete dams. Nevertheless, the application of the conventional multiple linear regression model has been limited due to the particular structure, random loading, and strong nonlinear deformation of concrete dams. Conversely, the artificial neural network (ANN) model shows good adaptability to complex and highly nonlinear behaviors. This paper aims to evaluate the specific performance of the multiple linear regression (MLR) and artificial neural network (ANN) model in characterizing concrete dam deformation under environmental loads. In this study, four models, namely, the multiple linear regression (MLR), stepwise regression (SR), backpropagation (BP) neural network, and extreme learning machine (ELM) model, are employed to simulate dam deformation from two aspects: single measurement point and multiple measurement points, approximately 11 years of historical dam operation records. Results showed that the prediction accuracy of the multipoint model was higher than that of the single point model except the MLR model. Moreover, the prediction accuracy of the ELM model was always higher than the other three models. All discussions would be conducted in conjunction with a gravity dam study.

Regression convolutional neural network for improved simultaneous EMG control

Objective. Deep learning models can learn representations of data that extract useful information in order to perform prediction without feature engineering. In this paper, an electromyography (EMG) control scheme with a regression convolutional neural network (CNN) is proposed as a substitute of conventional regression models that use purposefully designed features. Approach. The usability of the regression CNN model is validated for the first time, using an online Fitts’ law style test with both individual and simultaneous wrist motions. Results were compared to that of a support vector regression-based scheme with a group of widely used extracted features. Main results. In spite of the proven efficiency of these well-known features, the CNN-based system outperformed the support vector machine (SVM) based scheme in throughput, due to higher regression accuracies especially with high EMG amplitudes. Significance. These results indicate that the CNN model can extract underlying motor control information from EMG signals during single and multiple degree-of-freedom (DoF) tasks. The advantage of regression CNN over classification CNN (studied previously) is that it allows independent and simultaneous control of motions.

Estimation of Tunnel Desilter Sediment Removal Efficiency by ANFIS

The tunnel desilter is a simple and economical fluidic device which is the most suitable over other alternative devices for the region if water is abundantly available. The flow mechanism in the tunnel desilter is so complex that it is difficult to estimate the sediment removal efficiency accurately using a conventional regression. Hence, in the present study AI-based techniques, adaptive neurofuzzy interface system (ANFIS) and artificial neural network (ANN), were employed to estimate the sediment removal efficiency of the tunnel desilter using the data-sets collected by conducting the laboratory test. Findings of the sensitivity analysis showed that the size of the sediment was the most significant parameter followed by the concentration in the estimation of removal efficiency. The results of AI-based modeling were also compared with the available conventional predictive regression models, and it was found that the triangular membership function-based ANFIS model outperformed the other considered models. Further, ANN was also found to be giving comparable results.

A generalized Bayesian nonlinear mixed-effects regression model for zero-inflated longitudinal count data in tuberculosis trials.

In this paper, we investigate Bayesian generalized nonlinear mixed-effects (NLME) regression models for zero-inflated longitudinal count data. The methodology is motivated by and applied to colony forming unit (CFU) counts in extended bactericidal activity tuberculosis(TB) trials. Furthermore, for model comparisons, we present a generalized method for calculating the marginal likelihoods required to determine Bayes factors. A simulation study shows that the proposed zero-inflated negative binomial regression model has good accuracy, precision, and credibility interval coverage. In contrast, conventional normal NLME regression models applied to log-transformed count data, which handle zero counts as left censored values, may yield credibility intervals that undercover the true bactericidal activity of anti-TB drugs. We therefore recommend that zero-inflated NLME regression models should be fitted to CFU count on the original scale, as an alternative to conventional normal NLME regression models on the logarithmic scale.

Developing Machine Learning Models to Predict Roadway Traffic Noise: An Opportunity to Escape Conventional Techniques

Accurate prediction of roadway traffic noise remains challenging. Many researchers continue to improve the performance of their models by either adding more variables or improving their modeling algorithms. In this research, machine learning (ML) modeling techniques were developed to predict roadway traffic noise accurately. The ML techniques applied were: regression decision trees, support vector machine, ensembles, and artificial neural network. The parameters of each of these models were fine-tuned to achieve the best performance results. In addition, a state-of-the-art hybrid feature-selection technique has been employed to select a minimum set of input features (variables) while maintaining the accuracy of the developed models. By optimizing the number of features used in the model, the resources needed to develop and utilize a model to predict roadway noise would be less, hence decreasing the development cost. The proposed approach has been applied to develop a free-field roadway traffic noise model for Sharjah City in the United Arab Emirates. The best developed ML model was compared with a conventional regression model which was developed earlier under the same conditions. The cross-validated results clearly indicate that the best ML model outperformed the regression modeling. The performance of the ML model was also assessed after reducing the number of its input features based on the outcome of the feature-selection algorithm; the model performance was slightly affected. This result emphasizes the importance of considering only features that greatly influence the roadway traffic noise.

Modal Regression-Based Atomic Representation for Robust Face Recognition and Reconstruction.

Representation-based classification (RC) methods, such as sparse RC, have shown great potential in face recognition (FR) in recent years. Most previous RC methods are based on the conventional regression models, such as lasso regression, ridge regression, or group lasso regression. These regression models essentially impose a predefined assumption on the distribution of the noise variable in the query sample, such as the Gaussian or Laplacian distribution. However, the complicated noises in practice may violate the assumptions and impede the performance of these RC methods. In this paper, we propose a modal regression (MR)-based atomic representation and classification (MRARC) framework to alleviate such limitations. MR is a robust regression framework which aims to reveal the relationship between the input and response variables by regressing toward the conditional mode function. Atomic representation is a general atomic norm regularized linear representation framework which includes many popular representation methods, such as sparse representation, collaborative representation, and low-rank representation as special cases. Unlike previous RC methods, the MRARC framework does not require the noise variable to follow any specific predefined distributions. This gives rise to the capability of MRARC in handling various complex noises in reality. Using MRARC as a general platform, we also develop four novel RC methods for unimodal and multimodal FR, respectively. In addition, we devise a general optimization algorithm for the unified MRARC framework based on the alternating direction method of multipliers and half-quadratic theory. The experiments on real-world data validate the efficacy of MRARC for robust FR and reconstruction.

Decision tree (DT), generalized regression neural network (GR) and multivariate adaptive regression splines (MARS) models for sediment transport in sewer pipes.

Sediment deposition in sewers and urban drainage systems has great effect on the hydraulic capacity of the channel. In this respect, the self-cleansing concept has been widely used for sewers and urban drainage systems design. This study investigates the bed load sediment transport in sewer pipes with particular reference to the non-deposition condition in clean bed channels. Four data sets available in the literature covering wide ranges of pipe size, sediment size and sediment volumetric concentration have been utilized through applying decision tree (DT), generalized regression neural network (GR) and multivariate adaptive regression splines (MARS) techniques for modeling. The developed models have been compared with conventional regression models available in the literature. The model performance indicators, showed that DT, GR and MARS models outperform conventional regression models. Result shows that GR and MARS models are comparable in terms of calculating particle Froude number and performing better than DT. It is concluded that conventional regression models generally overestimate particle Froude number for the non-deposition condition of sediment transport, while DT, GR and MARS outputs are close to their measured counterparts.

A Status-transition Model for CO2 Heat Pump Water Heater Based on Modified Lorentz cycle

Energy management is a systematic activity for improving energy performances of a target system, and an energy management system is expected to solve operational planning problems and report or suggest opportunities for performance improvement. An equipment model is required to reflect the characteristics of the actual equipment’s performance and to have a simple structure to apply to operational planning problems. The model should be able to diagnose changes with performance degradation over time. In this study, we proposed a thermodynamically-sound model of a CO2 heat pump water heater, suitable for solving operational planning problems and diagnosing degradation of equipment. The proposed model consists of a heat pump unit (HP) and a hot water storage tank (ST). The HP model is a status-transition model, constructed based on the Lorentz efficiency, which is identified by experimental values and a theoretical maximum coefficient of performance (COP) for a trans-critical heat pump cycle. The ST model is simplified and can describe temperature distribution in the ST because the unit COP of the HP influences the thermal stratification of the ST. The proposed model is preferable in its simplicity and robust performance for a wide temperature range by comparison with a conventional statistical regression model.

Hierarchical Rough-to-Fine Model for Infant Age Prediction Based on Cortical Features.

Prediction of the chronological age based on neuroimaging data is important for brain development analysis and brain disease diagnosis. Although many researches have been conducted for age prediction of older children and adults, little work has been dedicated to infants. To this end, this paper focuses on predicting infant age from birth to 2-year old using brain MR images, as well as identifying some related biomarkers. However, brain development during infancy is too rapid and heterogeneous to be accurately modeled by the conventional regression models. To address this issue, a two-stage prediction method is proposed. Specifically, our method first roughly predicts the age range of an infant and then finely predicts the accurate chronological age based on a learned, age-group-specific regression model. Combining this two-stage prediction method with another complementary one-stage prediction method, a hierarchical rough-to-fine (HRtoF) model is built. HRtoF effectively splits the rapid and heterogeneous changes during a long time period into several short time ranges and further mines the discrimination capability of cortical features, thus reaching high accuracy in infant age prediction. Taking 8 types of cortical morphometric features from structural MRI as predictors, the effectiveness of our proposed HRtoF model is validated using an infant dataset including 50 healthy subjects with 251 longitudinal MRI scans from 14 to 797 days. Comparing with five state-of-the-art regression methods, HRtoF model reduces the mean absolute error of the prediction from >48 days to 32.1 days. The correlation coefficient of the predicted age and the chronological age reaches 0.963. Moreover, based on HRtoF, the relative contributions of the eight types of cortical features for age prediction are also studied.

Light and heavy drinking in jurisdictions with different alcohol policy environments

BackgroundA basic, yet untested tenet underlying alcohol control policies is that they should affect both light and heavy drinking, thereby shifting the entire population in a favourable direction. The aim of this study was to test this assumption in young Swiss men. MethodsCross-sectional self-reported data — from 5755 young Swiss men participating in the Cohort Study on Substance Use Risk Factors (C-SURF), a large cohort study on young men living within 21 jurisdictions across Switzerland — were analysed via nested logistic regression. With this approach, a set of increasingly-heavy drinking patterns was broken down into a set of nested regression models, each one estimating the probability of heavier drinking, conditional on the lighter drinking pattern. Drinking patterns relating to heavy episodic drinking (HED), heavy volume drinking (HVD) on weekends, and workweek drinking, as well as alcohol use disorder (AUD) were examined. The explanatory variable was a previously-used alcohol policy environment index (APEI) reflecting the number of alcohol control policies implemented in each jurisdiction. Conventional and multilevel logistic regression models were tested, adjusted for age, education, linguistic region, urban/rural status, attention-deficit/hyperactivity disorder, depression, sensation seeking, antisocial personality disorder, and unobserved heterogeneity between jurisdictions. ResultsFor HED, weekend HVD, and AUD, negative relationships with the APEI were found, such that with a higher APEI the probability of lighter drinking patterns was increased while the probability of heavier patterns was reduced, including a reduced probability of the heaviest patterns. These relationships were non-linear, however, and tapered off towards the heavy end of the drinking spectrum. No relationship was identified between the APEI and workweek drinking patterns. ConclusionAmong young Swiss men, stricter alcohol policy environments were associated with a global shift towards lighter drinking, consistent with the basic tenet behind the universal prevention approach.