Proposed Regression Method Research Articles

Low-rank tensor regression for selection of grouped variables

Low-rank tensor regression (LRTR) problems are widely studied in statistics and machine learning, in which the regressors are generally grouped by clustering strongly correlated variables or variables corresponding to different levels of the same predictive factor in many practical applications. By virtue of the idea of group selection in the classical linear regression framework, we propose an LRTR method for adaptive selection of grouped variables in this article, which is formulated as a group SLOPE penalized low-rank, orthogonally decomposable tensor optimization problem. Moreover, we introduce the notion of tensor group false discovery rate (TgFDR) to measure the group selection performance. The proposed regression method provably controls TgFDR and achieves the asymptotically minimax estimate under the assumption that variable groups are orthogonal to each other. Finally, an alternating minimization algorithm is developed for efficient problem resolution. We demonstrate the performance of our proposed method in group selection and low-rank estimation through simulation studies and real dataset analysis.

Experimental investigation and regression analysis on the characteristics of the JT refrigerator operating with natural refrigerant mixtures

This paper presents the results of the analysis of the experimental characteristics of the closed-cycle Joule-Thomson (JT) refrigerator operating on a nitrogen-hydrocarbon refrigerant mixture. Designing such energy-efficient refrigeration machines involves the calculation and experimental determination of the optimal values of the “charge” composition of the mixture components. The design process must take into account the change in the “circulating” composition of the mixture components. In the paper, we present the results of regression analysis of the “circulating” composition of the mixture components of a JT low-performance refrigerator (compressor volumetric flow rate less than 25 m3/h). Such refrigerators can be used for thermostatting objects at the temperatures down to −160 °C. The results are given in the form of relationships for specific refrigeration capacity and a number of efficiency indicators. In the considered component concentration range, the relative root-mean-square regression error was less than 12.6 %. Finally, we determined the optimal concentrations of the mixture components of the JT refrigerator having a compressor with a volumetric flow rate of 4.36 m3/h which corresponds to the maximum value of specific refrigerating capacity Qt/Ve [19,87 W/(m3/h)] at the temperature of the evaporator inlet of −150 °C. The results of approbation the proposed regression method on a prototype low-temperature freezer based on a JT refrigerator operating with MMR and the limits of applicability of the developed regression models are determined. These relationships provide valuable insights for designing stock-produced devices of varying standard sizes, reducing development time and cost while leveraging comparable operating cycles and working body compositions.

A regression approach for seismic first-break picking

The picking efficiency of seismic first breaks (FBs) has been greatly accelerated by deep learning (DL) technology. However, the picking accuracy and efficiency of DL methods still face huge challenges in low signal-to-noise ratio (SNR) situations. To address this issue, we propose a regression approach to pick FBs based on bidirectional long short-term memory (BiLSTM) neural network by learning the implicit Eikonal equation of 3D inhomogeneous media with rugged topography in the target region. We employ a regressive model that represents the relationships among the elevation of shots, offset and the elevation of receivers with their seismic traveltime to predict the unknown FBs, from common-shot gathers with sparsely distributed traces. Different from image segmentation methods which automatically extract image features and classify FBs from seismic data, the proposed method can learn the inner relationship between field geometry and FBs. In addition, the predicted results by the regressive model are continuous values of FBs rather than the discrete ones of the binary distribution. The picking results of synthetic data shows that the proposed method has low dependence on label data, and can obtain reliable and similar predicted results using two types of label data with large differences. The picking results of 9380 shots for 3D seismic data generated by vibroseis indicate that the proposed method can still accurately predict FBs in low SNR data. The subsequent stacked profiles further illustrate the reliability and effectiveness of the proposed method. The results of model data and field seismic data demonstrate that the proposed regression method is a robust first-break picker with high potential for field application.

Inferring the probability distribution over strain tensors in polycrystals from diffraction based measurements

Polycrystals illuminated by high-energy X-rays or neutrons produce diffraction patterns in which the measured diffraction peaks encode the individual single crystal strain states. While state of the art X-ray and neutron diffraction approaches can be used to routinely recover per grain mean strain tensors, less work has been produced on the recovery of higher order statistics of the strain distributions across the individual grains. In the setting of small deformations, we consider the problem of estimating the crystal elastic strain tensor probability distribution from diffraction data. For the special case of multivariate Gaussian strain tensor probability distributions, we show that while the mean of the distribution is well defined from measurements, the covariance of strain has a null-space. We show that there exist exactly 6 orthogonal perturbations to this covariance matrix under which the measured strain signal is invariant. In particular, we provide analytical parametrisations of these perturbations together with the set of possible maximum-likelihood estimates for a multivariate Gaussian fit to data. The parametric description of the null-space provides insights into the strain PDF modes that cannot be accurately estimated from the diffraction data. Understanding these modes prevents erroneous conclusions from being drawn based on the data. Beyond Gaussian strain tensor probability densities, we derive an iterative radial basis regression scheme in which the strain tensor probability density is estimated by a sparse finite basis expansion. This is made possible by showing that the operator mapping the strain tensor probability density onto the measured histograms of directional strain is linear, without approximation. The utility of the proposed algorithm is demonstrated by numerical simulations in the setting of single crystal monochromatic X-ray scattering. The proposed regression methods were found to robustly reject outliers and accurately predict the strain tensor probability distributions in the presence of Gaussian measurement noise.

Rapid Preliminary Sailplane Design using Regression Method

The article describes a new method for the preliminary design of a Sailplane based on technical data from similar Sailplanes, enabling quick design using regression methods. The authors chose FX 66-168 as the main wing airfoil for a new sailplane with a gross weight of 602 kg with a flight parameter Angle-of-Attack (AoA) 00 in minimum velocity of 49.5 m/s to cruise. As an example, we illustrate the proposed regression method. Our method allows for improving the effective preliminary design of the sailplane. The effectiveness evaluation validation of the new model is conducted with the numerical calculation using XFLR Opensource software. The research results improve design techniques and could be used as a manual for an aircraft designer.

Experimental Study on Mechanical Properties of Concrete at Super-Early Age.

Few studies have reported the cohesion and friction angle of concrete at a super early age. However, these two mechanical parameters are necessary to study the influence of engineering vibration on super-early-age concrete. In view of this, the mechanical properties of the super-early age-concrete are investigated in this work by direct shear testing. Firstly, the shear strength of the super-early-age concrete is measured by the direct shear experiment under different normal pressures at different times. Secondly, the cohesion and friction angle of the super early age concrete are calculated according to the Mohr-Coulomb criterion of failure. To overcome the great discreteness and randomness in the measured data, a new robust regression analysis algorithm is presented to replace the traditional regression analysis method to obtain more reliable and reasonable mechanical parameters. According to the experimental and theoretical analysis results, it is found that the friction angles of the super early age concrete are located in the interval of [50°, 70°]. The cohesion of the concrete is about 78.7 kPa at the initial setting state and about 190.9 kPa at the final setting state, respectively. It has been shown that the cohesion of the concrete at a super-early age tends to increase rapidly with time. The method and test results of this work can be used as a reference for relevant engineering practice. Specifically, the proposed regression method can be extended to the data analysis of other mechanical parameters of concrete, as well as other brittle materials such as rock. The test results of early concrete cohesion and friction angle can be used to analyze the adverse effects of vibration on newly cast concrete members in pile driving and blasting engineering.

System norm regularization methods for Koopman operator approximation

Approximating the Koopman operator from data is numerically challenging when many lifting functions are considered. Even low-dimensional systems can yield unstable or ill-conditioned results in a high-dimensional lifted space. In this paper, Extended Dynamic Mode Decomposition (DMD) and DMD with control, two methods for approximating the Koopman operator, are reformulated as convex optimization problems with linear matrix inequality constraints. Asymptotic stability constraints and system norm regularizers are then incorporated as methods to improve the numerical conditioning of the Koopman operator. Specifically, the H ∞ norm is used to penalize the input–output gain of the Koopman system. Weighting functions are then applied to penalize the system gain at specific frequencies. These constraints and regularizers introduce bilinear matrix inequality constraints to the regression problem, which are handled by solving a sequence of convex optimization problems. Experimental results using data from an aircraft fatigue structural test rig and a soft robot arm highlight the advantages of the proposed regression methods.

Comparative study for multi-variable regression methods based on Laguerre polynomial and manifolds optimization

PurposeThe purpose of this paper is to overcome the application limitations of other multi-variable regression based on polynomials due to the huge computation room and time cost.Design/methodology/approachIn this paper, based on the idea of feature selection and cascaded regression, two strategies including Laguerre polynomials and manifolds optimization are proposed to enhance the accuracy of multi-variable regression. Laguerre polynomials were combined with the genetic algorithm to enhance the capacity of polynomials approximation and the manifolds optimization method was introduced to solve the co-related optimization problem.FindingsTwo multi-variable Laguerre polynomials regression methods are designed. Firstly, Laguerre polynomials are combined with feature selection method. Secondly, manifolds component analysis is adopted in cascaded Laguerre polynomials regression method. Two methods are brought to enhance the accuracy of multi-variable regression method.Research limitations/implicationsWith the increasing number of variables in regression problem, the stable accuracy performance might not be kept by using manifold-based optimization method. Moreover, the methods mentioned in this paper are not suitable for the classification problem.Originality/valueExperiments are conducted on three types of datasets to evaluate the performance of the proposed regression methods. The best accuracy was achieved by the combination of cascade, manifold optimization and Chebyshev polynomials, which implies that the manifolds optimization has stronger contribution than the genetic algorithm and Laguerre polynomials.

Locally weighted PCA regression to recover missing markers in human motion data.

“Missing markers problem”, that is, missing markers during a motion capture session, has been raised for many years in Motion Capture field. We propose the locally weighted principal component analysis (PCA) regression method to deal with this challenge. The main merit is to introduce the sparsity of observation datasets through the multivariate tapering approach into traditional least square methods and develop it into a new kind of least square methods with the sparsity constraints. To the best of our knowledge, it is the first least square method with the sparsity constraints. Our experiments show that the proposed regression method can reach high estimation accuracy and has a good numerical stability.

A musculoskeletal model driven by muscle synergy-derived excitations for hand and wrist movements

Objective. Musculoskeletal model (MM) driven by electromyography (EMG) signals has been identified as a promising approach to predicting human motions in the control of prostheses and robots. However, muscle excitations in MMs are generally derived from the EMG signals of the targeted sensor covering the muscle, inconsistent with the fact that signals of a sensor are from multiple muscles considering signal crosstalk in actual situation. To identify more accurate muscle excitations for MM in the presence of crosstalk, we proposed a novel excitation-extracting method inspired by muscle synergy for simultaneously estimating hand and wrist movements. Approach. Muscle excitations were firstly extracted using a two-step muscle synergy-derived method. Specifically, we calculated subject-specific muscle weighting matrix and corresponding profiles according to contributions of different muscles for movements derived from synergistic motion relation. Then, the improved excitations were used to simultaneously estimate hand and wrist movements through musculoskeletal modeling. Moreover, the offline comparison among the proposed method, traditional MM and regression methods, and an online test of the proposed method were conducted. Main results. The offline experiments demonstrated that the proposed approach outperformed the EMG envelope-driven MM and three regression models with higher R and lower NRMSE. Furthermore, the comparison of excitations of two MMs validated the effectiveness of the proposed approach in extracting muscle excitations in the presence of crosstalk. The online test further indicated the superior performance of the proposed method than the MM driven by EMG envelopes. Significance. The proposed excitation-extracting method identified more accurate neural commands for MMs, providing a promising approach in rehabilitation and robot control to model the transformation from surface EMG to joint kinematics.

Similarity-Based Prediction for Channel Mapping and User Positioning

In a wireless network, gathering information at the base station about mobile users based only on uplink channel measurements is an interesting challenge. Indeed, accessing the users locations and predicting their downlink channels would be particularly useful in order to optimize the network efficiency. In this letter, a supervised machine learning approach addressing these tasks in an unified way is proposed. It relies on a labeled database that can be acquired in a simple way by the base station while operating. The proposed regression method can be seen as a computationally efficient two layers neural network initialized with a non-parametric estimator. It is illustrated on realistic channel data, both for the positioning and channel mapping tasks, achieving better results than previously proposed approaches, at a lower cost.

Improve Regression Network on Depth Hand Pose Estimation With Auxiliary Variable

The regression based deep neural networks have achieved state-of-the-arts performance on depth 3D hand pose estimation task. This paper focuses on improving the regression mapping between features and pose joints. Inspired by the distribution modeling ability of Variational Autoencoders, we introduce an auxiliary variable into the regression network. During training, the auxiliary variable is modeled by an inference distribution that learns the underlying structural kinematics of human hand. Different with other regression methods on hand poses, our network estimates the pose joints from input depth features and the learned auxiliary variable as well. We show that by introducing the auxiliary variable, the regression is benefited from 1) regularization modeled by inference distribution; and 2) prior information carried by the auxiliary model. The effectiveness of the proposed regression method is evaluated with extensively self-comparative experiments and in comparison with other regression methods on hand pose datasets. The proposed network is easy to train in an end-to-end manner and can work with various feature extraction methods. We apply the proposed regression method to an existing hand pose estimation system, and improves the estimation accuracy by 18.35% and 16.65% on public hand pose datasets.

Robust twin support vector regression based on Huber loss function

Construction of robust regression learning models to fit data with noise is an important and challenging problem of data regression. One of the ways to tackle this problem is the selection of a proper loss function showing insensitivity to noise present in the data. Since Huber function has the property that inputs with large deviations of misfit are penalized linearly and small errors are squared, we present novel robust regularized twin support vector machines for data regression based on Huber and e-insensitive Huber loss functions in this study. The proposed regression models result in solving a pair of strongly convex minimization problems in simple form in primal whose solutions are obtained by functional and Newton–Armijo iterative algorithms. The finite convergence of Newton–Armijo algorithm is proved. Numerical tests are performed on noisy synthetic and benchmark datasets, and their results are compared with few popular regression learning algorithms. The comparative study clearly shows the robustness of the proposed regression methods and further demonstrates their effectiveness and suitability.

Predictive modeling of biomass gasification with machine learning-based regression methods

Biomass gasification is a promising power generation process due to its ability to utilize waste materials and similar renewable energy sources. Predicting the outcomes of this process is a critical step to efficiently obtain the optimal amount of products. For this purpose, various kinetic and equilibrium models are proposed, but the assumptions made in these models significantly reduced their practical usability and consistency. More recently, machine learning methods have started been employed, but the limited selection of methods and lack of implementation of cross-validation techniques caused insufficiency to obtain unbiased performance evaluations. In this study, we employed four regression techniques, i.e., polynomial regression, support vector regression, decision tree regression and multilayer perceptron to predict CO, CO2, CH4, H2 and HHV outputs of the biomass gasification process. The data set is experimentally collected via downdraft fixed-bed gasifier. PCA technique is applied to the extracted features to prevent multicollinearity and to increase computational efficiency. Performances of the proposed regression methods are evaluated with k-fold cross validation. Multilayer perceptron and decision tree regression performed the best among other methods by achieving R2> 0.9 for the majority of outputs and were able to outperform other modeling approaches.

A tree ensemble-based two-stage model for advanced-stage colorectal cancer survival prediction

Classification techniques have widely been applied to cancer survival prediction for predicting survival or death of patients. However, little attention has been paid to patients who are predicted to die. In this work, we consider survival prediction as a two-stage task, where the first stage is to predict whether the outcome is survival or not, and the second stage is to predict the remaining lifespan for patients whose predicted outcome is death. To this end, we propose a two-stage machine learning model to enhance cancer survival prediction. At the first stage, a tree-based imbalanced ensemble classification method is proposed for classification of the survivability of advanced-stage cancer patients. At the second stage, a selective ensemble regression method is proposed for survival time prediction, where a priori knowledge is adopted for feature selection and the mean proportion of error interval is proposed for selecting base learners. Extensive computational studies performed on colorectal cancer data from SEER database demonstrate that the proposed two-stage model can achieve a more accurate prediction compared to the one-stage regression model. The results show that the proposed classification approach can effectively handle the imbalanced survivability data, and the proposed regression method outperforms several state-of-the-art regression models.

Two regression methods for hesitant multiplicative preference relations with different consistencies

Multiplicative preference relation (MPR) is an efficient and widely used tool in describing the preferences of decision makers. The hesitant multiplicative preference relation (HMPR), as an extension of the MPR, is commonly used in collecting and representing all hesitant preferences and judgements of the decision makers. Considering that there usually appear some random or illogical preference degrees in the process of constructing the HMPRs, so it is necessary to derive a pithy preference relation (such as MPR) from the original HMPR through selecting the most proper and abandoning improper preference degrees, at the same time, the derived MPR is with the advantage of both terseness and the best consistency. Based on the multiplicative consistency measure of HMPRs, this paper proposes two regression methods to transform the HMPR into MPRs (called reduced MPRs). With the error analysis, the first proposed regression method provides some steps to not only extract the reduced MPRs from original HMPR but also calculate the consistencies of both the HMPR and the reduced MPRs. A case study reflects that the reduced MPR is with the highest consistency degree among those possible separated MPRs from HMPR. To apply this method to solve practical problems conveniently, a group decision-making procedure with hesitant multiplicative information is further given based on the first regression method. Time complexity comparison between our proposed method and an existing method indicates the effectiveness of our method. In addition, according to the weak consistency, we develop the second regression method and design an algorithm to obtain the reduced MPRs from the HMPR. Furthermore, we provide a method to check the weak consistency of the HMPR and repair the inconsistent one. Numerical examples verify that the second regression method proposed in this paper is an effective technique for checking the weak consistency and modifying the inconsistency HMPR to the one with weak consistency.

Maximum correntropy criterion partial least squares

Partial least squares (PLS) has been extensively used to solve problems such as infrared quantitative analysis, economic data analysis, object tracking. PLS finds a linear regression model by projecting the predicted variables and the response to a new space. A major drawback of existing PLS methods is that regression coefficient will be affected by outliers. Thus, partial least squares experience significant performance degradation when gross outliers are presented. The problem of robust partial least squares has been relatively unexplored in Chemometrics and other related fields. In this paper, a new maximum correntropy based partial least squares is proposed to build robust model. We then proposed a solution algorithm for proposed model. Moreover, we also conducted convergence analysis to mathematically support the proposed model. Extensive experiments are conducted with four public data sets and experimental results demonstrate that our proposed regression method can lead to better accuracy to existing methods.

Adaptive Weather Correction of Energy Consumption Data

A framework for adaptive weather correction of energy consumption data is presented. The procedure is conducted in two steps: I) a regression model is trained on a building's recent historical energy consumption, weather and calendar data; II) energy consumption is predicted by using long term weather data as input to the trained model. Thus the buildings long term energy consumption is obtained, from which the building's expected (alias normalised or weather corrected) yearly energy consumption is derived. For older Swedish residential buildings, the proposed regression method matches traditional heating degree days method in accuracy. But for low energy and near zero energy buildings the regression method is more accurate, especially for years of extreme weather and for building with more complex installations such as heat pumps or solar thermal panels. The main benefit of the developed weather correction method is that it adapts to the data, therefore most buildings (or any kinds of weather dependent processes) can be weather corrected in an automated way.

Simulating land-use changes by incorporating spatial autocorrelation and self-organization in CLUE-S modeling: a case study in Zengcheng District, Guangzhou, China

The Conversion of Land Use and its Effects at Small regional extent (CLUE-S), which is a widely used model for land-use simulation, utilizes logistic regression to estimate the relationships between land use and its drivers, and thus, predict land-use change probabilities. However, logistic regression disregards possible spatial autocorrelation and self-organization in land-use data. Autologistic regression can depict spatial autocorrelation but cannot address self-organization, while logistic regression by considering only self-organization (NElogistic regression) fails to capture spatial autocorrelation. Therefore, this study developed a regression (NE-autologistic regression) method, which incorporated both spatial autocorrelation and self-organization, to improve CLUE-S. The Zengcheng District of Guangzhou, China was selected as the study area. The land-use data of 2001, 2005, and 2009, as well as 10 typical driving factors, were used to validate the proposed regression method and the improved CLUE-S model. Then, three future land-use scenarios in 2020: the natural growth scenario, ecological protection scenario, and economic development scenario, were simulated using the improved model. Validation results showed that NE-autologistic regression performed better than logistic regression, autologistic regression, and NE-logistic regression in predicting land-use change probabilities. The spatial allocation accuracy and kappa values of NE-autologistic-CLUE-S were higher than those of logistic-CLUE-S, autologistic-CLUE-S, and NE-logistic-CLUE-S for the simulations of two periods, 2001–2009 and 2005–2009, which proved that the improved CLUE-S model achieved the best simulation and was thereby effective to a certain extent. The scenario simulation results indicated that under all three scenarios, traffic land and residential/industrial land would increase, whereas arable land and unused land would decrease during 2009–2020. Apparent differences also existed in the simulated change sizes and locations of each land-use type under different scenarios. The results not only demonstrate the validity of the improved model but also provide a valuable reference for relevant policy-makers.

A Group Decision-Making Model Based on Regression Method with Hesitant Fuzzy Preference Relations

In recent years, the decision-making models with hesitant fuzzy preference relations (HFPRs) have received a lot of attention by some researchers. Meanwhile, the previous studies normally adopt normalization technical means to ensure the same number for all elements, which biases original information of decision-makers. In order to overcome this problem, in this paper, the multiplicative consistency of HFPRs is defined and the highest consistent reduced HFPRs are obtained by means of fuzzy linear programming method from given HFPRs. The proposed regression method eliminates the unreasonable information and retains the reasonable information from a given HFPR. In addition, the proposed method overcomes drawbacks of Zhu and Xu’s regression method and is more simple and effective. On account of the obtained reduced HFPRs by the proposed regression method, a GDM model is established. Finally, a supplier selection problem was researched to present the effectiveness and pragmatism of the proposed approach, which proved that the method could offer beneficial insights into the GDM procedure.