Classical Regression Methods Research Articles

Bioinspired Hybrid Model to Predict the Hydrogen Inlet Fuel Cell Flow Change of an Energy Storage System

The present research work deals with prediction of hydrogen consumption of a fuel cell in an energy storage system. Due to the fact that these kind of systems have a very nonlinear behaviour, the use of traditional techniques based on parametric models and other more sophisticated techniques such as soft computing methods, seems not to be accurate enough to generate good models of the system under study. Due to that, a hybrid intelligent system, based on clustering and regression techniques, has been developed and implemented to predict the necessary variation of the hydrogen flow consumption to satisfy the variation of demanded power to the fuel cell. In this research, a hybrid intelligent model was created and validated over a dataset from a fuel cell energy storage system. Obtained results validate the proposal, achieving better performance than other well-known classical regression methods, allowing us to predict the hydrogen consumption with a Mean Absolute Error (MAE) of 3.73 with the validation dataset.

Image Set-Oriented Dual Linear Discriminant Regression Classification and Its Kernel Extension

Along with the rapid development of computer and image processing technology, it is definitely convenient to obtain various images for subjects, which can be more robust to classification as more feature information is contained. However, how to effectively exploit the rich discriminative information within image sets is the key problem. In this paper, based on the concept of dual linear regression classification method for image set classification, we propose a novel discriminative framework to exploit the superiority of discriminant regression mechanism. We aim to learn a projection matrix to force the represented image points from the same class to be close and those from different class are better separated. The feature extraction strategy in our discriminative framework can appropriately work with the corresponding classification strategy, thus, better classification performance can be achieved. Moreover, we propose a kernel discriminative extension method to address the non-linearity problem by adopting the kernel trick. From the experimental results, our proposed method can obtain competitive recognition rates on face recognition tasks via mapping the original image sets into a more discriminative feature space. Besides, it also shows the effectiveness for object classification task with small image sizes and different number of frames.

Application of Sentinel-1 Data to Estimate Height and Biomass of Rice Crop in Astaneh-ye Ashrafiyeh, Iran

Crops monitoring is a challengeable subject that radar images can help it. The applicability of Sentinel-1 SAR data with dual polarization provided a splendid opportunity to develop a method for estimating rice parameters. Heights of cereal and biomass are two significant characteristics of rice that can be estimated with assessing satellite data and field measurements by classical regression methods [multiple linear regression (MLR), relevance vector regression (RVR), and support vector regression (SVR)]. In this study, Sentinel-1 SAR data from April 2018 to September 2018 in Astaneh-ye Ashrafiyeh region in the north of Iran were used. To evaluate and analyze validation of regression methods, field measurements (gathered from 15 plots) were utilized. The efficiency of nonparametric methods (SVR and RVR) is much better than that of the parametric regression (MLR) for rice parameter estimations. Among nonparametric approaches, RVR method has better results than SVR, because of the highest correlation coefficient (R2) and lowest root mean square error (RMSE). R2 = 0.92, RMSE = 162.1, and MAE = 971.9 and R2 = 0.92, RMSE = 10.9, and MAE = 70.71 are the results of height and biomass, respectively.

The calibration of the hailpads upon the Greek National Hail Suppression Program, using the Classical and Inverse Regression methods

The hailpad, constructed from a plate of Styrofoam, is a simple instrument for recording hailfall. In addition to simply recording the hailfall, calibration of the instrument is required to obtain quantitative measurements of the hail. The calibration is a process leading to a calibration equation, a polynomial establishing a relationship between the diameter of a hailstone and the dent the hailstone is left on the surface of the hailpad. A hailpad network, consisted of 154 instruments, has established inGreece, in the context of the Greek National Hail Suppression Program operating for the protection of the agricultural cultivations from hail damage. For the calibration of the haipads of the network the well known “Energy Matching technique” has adopted and the Inverse Regression method is applied from the beginning for the obtainment of the calibration equation. In the present study along with the Inverse Regression method hitherto applied, the Classical Regression method is examined and presented and inferential statistics are also introduced in both methods in order to establish a more stringent statistical procedure for the calibration of the hailpads. After the theoretical analysis the data from a calibration experiment were analyzed, calibration models obtained using both methods of regression, hail diameters were predicted with the two models when new observations were available and the results compared to each other. The comparison of the two models' predictions showed that the results are almost the same so there is no good reason to replace the hitherto Inverse Regression method. However, it would be good to introduce the Classical Regression method alongside the Inverse. In addition, prediction bands for both methods should be introduced giving to the results the range of the confidence interval of the predictions.

Two new methodological approaches for assessing skeletal maturity in archeological human remains based on the femoral distal epiphysis

This study presents two new methodological approaches for estimating skeletal age from maturational changes in the femoral distal epiphysis. In the first approach, five maturity stages were coded based on morphological changes in the epiphysis that encompass the overall developmental process. Data were presented as age ranges for the different maturity stages in the reference sample. As this approach has a number of shortcomings for age assessment, a probabilistic approach was also used. Cross-validation was then used to compare the accuracy of the age estimation from the maturity stages with that from Pyle and Hoerr’s atlas. This study’s findings showed that Pyle and Hoerr’s atlas is more precise than our qualitative method in the oldest age categories. Nonetheless, results from the test of agreement between methods showed that skeletal age estimates from both methods are interchangeable. In the second approach, the overall shape of the femoral distal epiphyses was first analyzed based on elliptical Fourier descriptors (EFDs). Since the number of EFDs is excessively large, a principal component analysis (PCA) of these EFDs was carried out. PC1 scores were used to model the relationship between age and overall shape in a sample of 110 cases of the femoral distal epiphysis. Inverse and classical regression methods of calibration were used to explore the relationship. Based on our results, we recommend the use of a classical calibration model for those cases in which we suspect that the growth and development of the target individual is advanced or delayed relative to those of the Portuguese sample. Otherwise, the inverse calibration model is preferable. Both, quantitative and qualitative methods presented herein notably improve our abilities to estimate skeletal age using incomplete femora from skeletal samples.

A Hybrid Supervised Approach to Human Population Identification Using Genomics Data.

Single nucleotide polymorphisms (SNPs) are one type of genetic variations and each SNP represents a difference in a single DNA building block, namely a nucleotide. Previous research demonstrated that SNPs can be used to identify the correct source population of an individual. In addition, variations in the DNA sequences have an influence on human diseases. In this regard, SNPs studies are helpful for personalized medicine and treatment. In the literature, unsupervised clustering methods especially principal component analysis (PCA) have been popular for studying population structure. In this study, we investigate supervised approaches, particularly the LASSO multinomial regression classification method, for recognizing individuals' origin genetic population. Then, we introduce PCA-LASSO as an extension of LASSO method that benefits from advantageous characteristics of both PCA and LASSO regression. The experimental results obtained on the 1,000 genome project dataset show PCA-LASSO's significantly high accuracy in prediction of individual's origin population.

Active Polynomial Chaos Expansion for Reliability-Based Design Optimization

The aim of the present paper is to develop a strategy for solving reliability-based design optimization (RBDO) problems by sparse polynomial chaos expansion (PCE). Because classic sparse regression methods cannot provide the surrogate error measure that can be employed to improve the sampling performance for failure probability estimation in RBDO, Bayesian compressed sensing with state-of-the-art performance is employed to build sparse PCE in the paper. In the meanwhile, a new active learning function is proposed to adaptively select new training points. Two goals can be achieved using this criterion; that is, most of the selected training points are selected from the desired regions to approximate limit state surfaces, and these points tend to be far away from the existing points in the current design to avoid the clustering problem. Because the sparse PCEs are built in an augmented space, it is made numerically affordable to solve the RBDO problem with no extra computational cost. The computation capability of the proposed method is demonstrated by several analytical RBDO problems. Meanwhile, the design optimization of a stiffened rib of the wing edge in a certain aircraft also verifies its good engineering applicability.

Two-Dimensional (2D) particle swarms for structure selection of nonlinear systems

The present study proposes a new structure selection approach for non-linear system identification based on Two-Dimensional particle swarms (2D-UPSO). The 2D learning framework essentially extends the learning dimension of the conventional particle swarms and explicitly incorporates information about the cardinality (i.e., number of terms) into the search process. This property of the 2D-UPSO is exploited to determine the correct structure of the non-linear systems. The efficacy of the proposed approach is demonstrated by considering several simulated benchmark nonlinear systems in discrete and continuous domains. In addition, the proposed approach is applied to identify a parsimonious structure from practical non-linear wave-force data. The results of the comparative investigation with four meta-heuristic algorithms and classical orthogonal forward regression methods illustrate that the proposed 2D-UPSO can successfully detect the correct structure of the non-linear systems.

Wide-Ranging Temporal Variation in Transoceanic Movement and Population Mixing of Bluefin Tuna in the North Atlantic Ocean

Uncertainty regarding the movement and population exchange of Atlantic bluefin tuna (Thunnus thynnus) from the two primary spawning areas (Gulf of Mexico, Mediterranean Sea) is increasingly implicated as a major impediment for the conservation of this species. Here, two mixture methods were applied to natural chemical markers (18O and 13C) in otoliths (ear stones) to comprehensively investigate the nature and degree of transoceanic movement and mixing of eastern and western populations in several areas of the North Atlantic Ocean that potentially represent mixing hotspots. Areas investigated occurred on both sides of the 45W management boundary as well as waters off the coast of Africa (Morocco, Canary Islands) where both populations are known to occur. Projections of population composition (i.e., natal or nursery origin) from a multinomial logistic regression (MLR) classification method with different probability thresholds were generally in agreement with maximum likelihood estimates from the commonly used mixed-population program HISEA; however, predicted contributions for the less abundant population were occasionally higher for MLR estimates. Both MLR and HISEA clearly showed that mixing of Atlantic bluefin tuna in the Central North Atlantic Ocean was highly variable from year to year with expatriates of eastern or western origin commonly crossing into the other management area. Pronounced transoceanic movement and mixing of western migrants was also present off the coast of Africa, with the occurrence of western migrants in the Canary Islands and Morocco ranging from zero to the majority of the individuals assayed for the years examined. Results indicate highly variable rates of movement and population exchange for Atlantic bluefin tuna, highlighting the need for temporally resolved estimates of natal origin in mixing hotspots to improve population models used to evaluate the status of this threatened species.

Graphical user interface (GUI) for the least absolute shrinkage and selection operator (LASSO) regression

In Data Science, we usually encounter High-dimensional data. In this situation, the Classical Regression method usually cannot perform well because it is impossible to include all covariates in the model since the number of a parameter to be estimated is larger than the sample size. Least absolute shrinkage and selection operator (Lasso) method is one of the methods which can deal with this problem. Lasso regression perform the selection of covariates so that only the most influential covariates are used in the model. Unfortunately, most of Lasso method should be performed in CLI Software which is difficult to use for the general user. For this reason, we develop a web application by using Shiny to perform the Lasso method based on GUI which is easier to use. It allows users to analyze high-dimensional data without using programming language. The paper contains an implementation of Lasso Regression using web application on olive pomade oil data.

Compound Regression and Constrained Regression: Nonparametric Regression Frameworks for EIV Models

Errors-in-variable (EIV) regression is often used to gauge linear relationship between two variables both suffering from measurement and other errors, such as, the comparison of two measurement platforms (e.g., RNA sequencing vs. microarray). Scientists are often at a loss as to which EIV regression model to use for there are infinite many choices. We provide sound guidelines toward viable solutions to this dilemma by introducing two general nonparametric EIV regression frameworks: the compound regression and the constrained regression. It is shown that these approaches are equivalent to each other and, to the general parametric structural modeling approach. The advantages of these methods lie in their intuitive geometric representations, their distribution free nature, and their ability to offer candidate solutions with various optimal properties when the ratio of the error variances is unknown. Each includes the classic nonparametric regression methods of ordinary least squares, geometric mean regression (GMR), and orthogonal regression as special cases. Under these general frameworks, one can readily uncover some surprising optimal properties of the GMR, and truly comprehend the benefit of data normalization. Supplementary materials for this article are available online.

On Regularisation Methods for Analysis of High Dimensional Data

High dimensional data are rapidly growing in many domains due to the development of technological advances which helps collect data with a large number of variables to better understand a given phenomenon of interest. Particular examples appear in genomics, fMRI data analysis, large-scale healthcare analytics, text/image analysis and astronomy. In the last two decades regularisation approaches have become the methods of choice for analysing such high dimensional data. This paper aims to study the performance of regularisation methods, including the recently proposed method called de-biased lasso, for the analysis of high dimensional data under different sparse and non-sparse situations. Our investigation concerns prediction, parameter estimation and variable selection. We particularly study the effects of correlated variables, covariate location and effect size which have not been well investigated. We find that correlated data when associated with important variables improve those common regularisation methods in all aspects, and that the level of sparsity can be reflected not only from the number of important variables but also from their overall effect size and locations. The latter may be seen under a non-sparse data structure. We demonstrate that the de-biased lasso performs well especially in low dimensional data, however it still suffers from issues, such as multicollinearity and multiple hypothesis testing, similar to the classical regression methods.

The relationship between tax avoidance and firm value with income smoothing

Purpose The purpose of this paper is to investigate the relationship between tax avoidance, firm value and managerial ability in Tehran Stock Exchange and Over the Counter (OTC), according to the related theoretical foundations. Design/methodology/approach To calculate the managerial ability in this study, DEA is used based on the accounting data, company profile and industry and the hypotheses are estimated in a period of 12 years during 2004 to 2015 in TSE and OTC. Within the previous studies, to test the hypotheses, only the classical regression method is usually used and most of the times the effect of macroeconomic variables is not considered. In this study, in a new act for testing the hypotheses, three statistical methods are used, that is, classical regression models, mixed effects multilevel models and Bayesian multilevel models. Also in this study, the test of structural change is used to control the effects of macroeconomic variables, like inflation and other economic and political influence, on the results. Findings The results of these three methods show that the effect of income smoothing and earnings quality on the relationship between tax avoidance and firm value are significant. Originality/value Although several studies are conducted so far on the subject of the study, the current study is the first project which combined Bayesian econometrics. Therefore, the results are quite noble.

Financial and Social Success of University Graduates in the Ural Region

The quality of human capital is a priority factor for the development of modern economy. Apparently, the system of higher education has a significant impact on the human development. There are a number of approaches to assessing this impact and to enhancing its positive outcomes. Studying the professional and social trajectories of graduates allows obtaining an objective picture of a practice-oriented influence of higher education on the human capital formation. The article explores young professionals’ career development six months after graduation. We analyse the characteristics of young professionals’ occupational and educational trajectories through examining the indicators of graduates’ employment monitoring in 2014–2016. The data are supplemented by the results of the graduates’ survey conducted in 2017 using administrative data. The data were processed using the methods of cluster and regression analysis, classification method and expert estimates. We have disproved the hypothesized significant impact of graduates’ performance at the university and their training basis (fee-payment or budgetary funding) on the financial success of graduates. Furthermore, we have identified five tendencies. Firstly, the graduates are increasingly involved in unstable employment. Secondly, working experience has a significant impact on successful employment. Thirdly, graduates continue their training to adapt themselves to the labour market. Fourthly, the financial and social success is highly dependent on the choice of training specialities, which are in demand in the modern economy. And last, there is a high mobility among graduates. Up to 30 % of them are moving to another subject of the Russian Federation. The study results can be applied to improve the quality of bachelor and master degree programmes, taking into account the identified specifics of young professionals’ unstable employment and to determine areas for effective government investment in the development of the Ural region human potential.

Wastewater treatment plant performance analysis using artificial intelligence - an ensemble approach.

In the present study, three different artificial intelligence based non-linear models, i.e. feed forward neural network (FFNN), adaptive neuro fuzzy inference system (ANFIS), support vector machine (SVM) approaches and a classical multi-linear regression (MLR) method were applied for predicting the performance of Nicosia wastewater treatment plant (NWWTP), in terms of effluent biological oxygen demand (BODeff), chemical oxygen demand (CODeff) and total nitrogen (TNeff). The daily data were used to develop single and ensemble models to improve the prediction ability of the methods. The obtained results of single models proved that, ANFIS model provides effective outcomes in comparison with single models. In the ensemble modeling, simple averaging ensemble, weighted averaging ensemble and neural network ensemble techniques were proposed subsequently to improve the performance of the single models. The results showed that in prediction of BODeff, the ensemble models of simple averaging ensemble (SAE), weighted averaging ensemble (WAE) and neural network ensemble (NNE), increased the performance efficiency of artificial intelligence (AI) modeling up to 14%, 20% and 24% at verification phase, respectively, and less than or equal to 5% for both CODeff and TNeff in calibration phase. This shows that NNE model is more robust and reliable ensemble method for predicting the NWWTP performance due to its non-linear averaging kernel.

A Kalman Filter-Based Queue Length Estimation Method with Low-Penetration Mobile Sensor Data at Signalized Intersections

Queue length estimation is of great importance for signal performance measures and signal optimization. With the development of connected vehicle technology and mobile internet technology, using mobile sensor data instead of fixed detector data to estimate queue length has become a significant research topic. This study proposes a queue length estimation method using low-penetration mobile sensor data as the only input. The proposed method is based on the combination of Kalman Filtering and shockwave theory. The critical points are identified from raw spatiotemporal points and allocated to different cycles for subsequent estimation. To apply the Kalman Filter, a state-space model with two state variables and the system noise determined by queue-forming acceleration is established, which can characterize the stochastic property of queue forming. The Kalman Filter with joining points as measurement input recursively estimates real-time queue lengths; on the other hand, queue-discharging waves are estimated with a line fitted to leaving points. By calculating the crossing point of the queue-forming wave and the queue-discharging wave of a cycle, the maximum queue length is also estimated. A case study with DiDi mobile sensor data and ground truth maximum queue lengths at Huanggang-Fuzhong intersection, Shenzhen, China, shows that the mean absolute percentage error is only 11.2%. Moreover, the sensitivity analysis shows that the proposed estimation method achieves much better performance than the classical linear regression method, especially in extremely low penetration rates.

Robust Compositional Analysis of Physical Activity and Sedentary Behaviour Data.

Although there is an increasing awareness of the suitability of using compositional data methodology in public health research, classical methods of statistical analysis have been primarily used so far. The present study aims to illustrate the potential of robust statistics to model movement behaviour using Czech adolescent data. We investigated: (1) the inter-relationship between various physical activity (PA) intensities, extended to model relationships by age; and (2) the associations between adolescents’ PA and sedentary behavior (SB) structure and obesity. These research questions were addressed using three different types of compositional regression analysis—compositional covariates, compositional response, and regression between compositional parts. Robust counterparts of classical regression methods were used to lessen the influence of possible outliers. We outlined the differences in both classical and robust methods of compositional data analysis. There was a pattern in Czech adolescents’ movement/non-movement behavior—extensive SB was related to higher amounts of light-intensity PA, and vigorous PA ratios formed the main source of potential aberrant observations; aging is associated with more SB and vigorous PA at the expense of light-intensity PA and moderate-intensity PA. The robust counterparts indicated that they might provide more stable estimates in the presence of outlying observations. The findings suggested that replacing time spent in SB with vigorous PA may be a powerful tool against adolescents’ obesity.

Some variants of spiral LBP in texture recognition

Texture classification is one of the recently popular study topics in pattern recognition. Local binary pattern (LBP) is a very efficient local texture descriptor and is used for feature extraction in texture recognition. There are five main steps in representation of texture images: neighbourhood topology and sampling, thresholding and quantisation, encoding and regrouping, combining complementary features. In this study, the authors used symmetric two spirals LBP to measure the grey-scale difference between the centre pixel and its neighbours. They also extended the proposed method by using four spirals LBP to generate the LBP code. For classification, linear regression classification method, which is generally used to solve the face recognition problems, is used. The authors tested the performance of their method on UIUC and CUReT texture image databases. It is experimentally demonstrated that the proposed method achieves the highest classification accuracy among the comparative methods on texture databases.

Efficiency of Bayesian Approaches in Quantile Regression with Small Sample Size


 
 
 
 Quantile regression is a statistical technique intended to estimate, and conduct inference about the conditional quantile functions. Just as the classical linear regression methods estimate model for the conditional mean function, quantile regression offers a mechanism for estimating models for the conditional median function, and the full range of other conditional quantile functions. In the Bayesian approach to variable selection prior distributions representing the subjective beliefs about the parameters are assigned to the regression coefficients. The estimation of parameters and the selection of the best subset of variables is accomplished by using adaptive lasso quantile regression. In this paper we describe, compare, and apply the two suggested Bayesian approaches. The two suggested Bayesian suggested approaches are used to select the best subset of variables and estimate the parameters of the quantile regression equation when small sample sizes are used. Simulations show that the proposed approaches are very competitive in terms of variable selection, estimation accuracy and efficient when small sample sizes are used. 
 
 
 
 

Importance of the spatial extent for using soil properties estimated by laboratory VNIR/SWIR spectroscopy: Examples of the clay and calcium carbonate content

Visible, near-infrared and short-wave infrared (VNIR/SWIR, 400–2500 nm) laboratory soil spectrometry is now considered to provide accurate estimations of primary soil properties (clay, calcium carbonate, iron, soil organic carbon, etc.). The performances of primary soil property prediction models are evaluated in regard to figures of merit calculated over calibration and validation databases but not in regard to the spatial extent of predicted soil samples. The objective of this study was to analyze regional model performances for soil property prediction at regional and within-field extents within contrasted representative geopedological situations. This study used a database of 240 soil samples collected over eight vineyard fields located in the Languedoc Region (southern France) (between 20 and 36 soil samples per field) for which VNIR/SWIR laboratory spectra were acquired and two soil physico-chemical properties (clay and calcium carbonate) were measured. Soil property prediction models were built using the classical partial least square regression (PLSR) method, which links the VNIR/SWIR laboratory spectra and the physico-chemical soil property. Our results showed that both clay and calcium carbonate prediction models are accurate at the regional extent, whereas prediction model performances at the within-field extent depend on the model robustness. Therefore, primary soil properties predicted by VNIR/SWIR laboratory spectra must be used with care at different extents.