Selection Of Regressors Research Articles

A New Linear Model for the Calculation of Routing Metrics in 802.11s Using ns-3 and RStudio

Wireless mesh networks (WMNs) offer a pragmatic solution with a cost-effective ratio when provisioning ubiquitous broadband internet access and diverse telecommunication systems. The conceptual underpinning of mesh networks finds application not only in IEEE networks, but also in 3GPP networks like LTE and the low-power wide area network (LPWAN) tailored for the burgeoning Internet of Things (IoT) landscape. IEEE 802.11s is well known for its facto standard for WMN, which defines the hybrid wireless mesh protocol (HWMP) as a layer-2 routing protocol and airtime link (ALM) as a metric. In this intricate landscape, artificial intelligence (AI) plays a prominent role in the industry, particularly within the technology and telecommunication realms. This study presents a novel methodology for the computation of routing metrics, specifically the ALM. This methodology implements the network simulator ns-3 and the RStudio as a statistical computing environment for data analysis. The former has enabled for the creation of scripts that elicit a variety of scenarios for WMN where information is gathered and stored in databases. The latter (RStudio) takes this information, and at this point, two linear predictions are supported. The first uses linear models (lm) and the second employs general linear models (glm). To conclude this process, statistical tests are applied to the original model, as well as to the new suggested ones. This work substantially contributes in two ways: first, through the methodological tool for the metric calculation of the HWMP protocol that belongs to the IEEE 802.11s standard, using lm and glm for the selection and validation of the model regressors. At this stage the ANOVA and STEPWIZE tools of RStudio are used. The second contribution is a linear predictor that improves the WMN’s performance as a priori mechanism before the use of the ns-3 simulator. The ANCOVA tool of RStudio is employed in the latter.

House Price Prediction Using Texture and Visual Features

Real estate sector has been growing at a rate never seen before. For this sector, a key role is played by the pricing of the property. Gone are the days when the price of the property was based on whims and facies of the real estate dealers. Machine learning has numerous applications in the domain of real estate, and one of the most popular ones is predicting house prices. The application of machine learning in house price prediction involves training a model on a dataset that includes a variety of visual and texture features related to the property. The model is then used to predict the price of a new property based on its features.
 This paper successfully explores machine learning based house price prediction. The methodology followed was to first use data sets to train the model. Later, using correlation-based hybrid GA-reinforcement strategy, a suitable set of features has been selected. In the end, these features are applied to a XG boost regressor to get results. The accuracies are compared with the cases of without feature selection of different regressors. This algorithm, if successfully deployed will be beneficial to both sellers and buyers, because it sets a data-based benchmarking for pricing the property.

Sparse Bayesian deep learning for dynamic system identification

This paper proposes a sparse Bayesian treatment of deep neural networks (DNNs) for system identification. Although DNNs show impressive approximation ability in various fields, several challenges still exist for system identification problems. First, DNNs are known to be too complex that they can easily overfit the training data. Second, the selection of the input regressors for system identification is nontrivial. Third, uncertainty quantification of the model parameters and predictions are necessary. The proposed Bayesian approach offers a principled way to alleviate the above challenges by marginal likelihood/model evidence approximation and structured group sparsity-inducing priors construction. The identification algorithm is derived as an iterative regularised optimisation procedure that can be solved as efficiently as training typical DNNs. Remarkably, an efficient and recursive Hessian calculation method for each layer of DNNs is developed, turning the intractable training/optimisation process into a tractable one. Furthermore, a practical calculation approach based on the Monte-Carlo integration method is derived to quantify the uncertainty of the parameters and predictions. The effectiveness of the proposed Bayesian approach is demonstrated on several linear and nonlinear system identification benchmarks by achieving good and competitive simulation accuracy. The code to reproduce the experimental results is open-sourced and available online.

Progress prediction of Parkinson's disease based on graph wavelet transform and attention weighted random forest

The progress prediction of Parkinson's disease (PD) is one of the most important issues in early diagnosis of PD. Many researches have been conducted in this field, however, most existing methods focus on the selection of baseline features and regressors to reduce prediction errors. Different from the previous studies, the main goal of this paper is to obtain more effective features by feature transformation of baseline features to improve the prediction performance. Therefore, this paper proposes a prediction model based on graph wavelet transform (GWT) and attention weighted random forest (RF). Firstly, a clustering algorithm is adopted to reduce the prediction error of the model. Next, a multi-scale analysis of the feature vectors by GWT is conducted to yield a frequency feature representation that is more relevant to the target value. Finally, the frequency features are input into the attention weighted RF to predict the severity of PD, allowing the results of decision trees with better predictive performance in the RF to be highlighted while reducing the risk of overfitting. The effectiveness of the method is evaluated on the Parkinson's telemonitoring dataset collected by the University of Oxford. The experimental results show that the mean absolute error and root mean squared error of the proposed method for predicting PD severity (motor- and total-UPDRS) are 1.53, 2.13 and 1.91, 2.70, respectively. Compared with the quoted optimal method, the errors are reduced by 7.27%, 4.05% and 5.45%, 1.10%, respectively. This indicates that the proposed method has better prediction performance.

Lossless Compression of Plenoptic Camera Sensor Images

We propose a codec for the lossless compression of plenoptic camera sensor images. The proposed encoder starts by splitting the input lenslet image into rectangular patches, with each patch corresponding to a microlens image. The encoder and decoder exploit the correlation between the pixels in neighbor patches using a patch-by-patch prediction mechanism where each pixel of a patch has its own dedicated sparse predictor designed to utilize the most relevant pixels from the neighbor patch to the left. An intra-patch prediction mask together with the pixels from the neighbor left patch form the final prediction template. The encoder performs the design of sparse predictors by first finding the relevant regressors in the final template. The patches are classified into M classes according to two possible mechanisms (either based on depth information or based on Bayer mask colors), and the sparse predictor design is performed for each pair (class label; patch pixel index). A relevant context selection mirrors the selection of relevant regressors, thereby providing the arithmetic coding with skewed coding distributions at each context. We show examples of the application of the proposed methods on sensor images from the JPEG Pleno database, thus demonstrating the improved performance of the proposed methods compared to the existing predictive methods for encoding camera sensor images.

Artificial Intelligence for the Prediction of Exhaust Back Pressure Effect on the Performance of Diesel Engines

The actual trade-off among engine emissions and performance requires detailed investigations into exhaust system configurations. Correlations among engine data acquired by sensors are susceptible to artificial intelligence (AI)-driven performance assessment. The influence of exhaust back pressure (EBP) on engine performance, mainly on effective power, was investigated on a turbocharged diesel engine tested on an instrumented dynamometric test-bench. The EBP was externally applied at steady state operation modes defined by speed and load. A complete dataset was collected to supply the statistical analysis and machine learning phases—the training and testing of all the AI solutions developed in order to predict the effective power. By extending the cloud-/edge-computing model with the cloud AI/edge AI paradigm, comprehensive research was conducted on the algorithms and software frameworks most suited to vehicular smart devices. A selection of artificial neural networks (ANNs) and regressors was implemented and evaluated. Two proof-of concept smart devices were built using state-of-the-art technology—one with hardware acceleration for “complete cycle” AI and the other with a compact code and size (“AI in a nut-shell”) with ANN coefficients embedded in the code and occasionally offline “statistical re-calibration”.

Distributed Estimation Over an Adaptive Diffusion Network Based on the Family of Affine Projection Algorithms

This paper utilizes the family of affine projection algorithms (APAs) for distributed estimation in the adaptive diffusion networks. The diffusion APA (DAPA), the diffusion selective partial update (SPU) APA (DSPU-APA), the diffusion selective regressor (SR) APA (DSR-APA), and the diffusion dynamic selection (DS) APA (DDS-APA) are introduced in a unified way. In DSPU-APA, the weight coefficients are partially updated at each node during the adaptation. Therefore, the DSPU-APA has lower computational complexity in comparison to the DAPA. In addition, the convergence speed of the DSPU-APA is close to the DAPA. In DSR-APA, a subset of input regressors is optimally selected at each node during the adaptation. The dynamic selection of input regressors is performed in the DDS-APA. These strategies improve the performance of the conventional DAPA in terms of the steady-state error and computational complexity features. Also, by combining these algorithms, the DSPU-SR-APA and the DSPU-DS-APA are established, which are computationally efficient. The mean-square performance of the proposed algorithms is analyzed in the nonstationary environment and the generic relations for the theoretical learning curve and the steady-state error are derived. The analysis is based on the spatial-temporal energy conservation relation. The validity of the theoretical results and the good performance of the introduced algorithms are demonstrated by several computer simulations in diffusion networks.

Diffusion improved multiband-structured subband adaptive filter algorithms with dynamic selection of regressors and subbands over distributed networks

The present study solves the problem of distributed estimation in the diffusion networks based on the family of improved multiband-structured subband adaptive filters (IMSAFs). The diffusion IMSAF ...

Diffusion improved multiband-structured subband adaptive filter algorithms with dynamic selection of regressors and subbands over distributed networks

The present study solves the problem of distributed estimation in the diffusion networks based on the family of improved multiband-structured subband adaptive filters (IMSAFs). The diffusion IMSAF (DIMSAF), the DIMSAF with dynamic selection of regressors (DIMSAF-DSR), and the DIMSAF with dynamic selection of subbands (DIMSAF-DSS) are established. The DIMSAF-DSS and the DIMSAF-DSR algorithms, while benefiting from high convergence speed in DIMSAF, have lower computational complexity and lower steady-state error. During the weight coefficients adaptation in DIMSAF-DSR, the input signal regressors are dynamically selected at each subband of different nodes. In DIMSAF-DSS, the subbands are dynamically selected at each node. In the following, the introduced algorithms are established based on a general update equation. Accordingly, the mean-square performance analysis of the algorithms is studied in a unified way. The theoretical results and the good performance of proposed algorithms are justified by several computer simulations in adaptive diffusion networks.

Group SLOPE – Adaptive Selection of Groups of Predictors

ABSTRACTSorted L-One Penalized Estimation (SLOPE; Bogdan et al. 2013, 2015) is a relatively new convex optimization procedure, which allows for adaptive selection of regressors under sparse high-dimensional designs. Here, we extend the idea of SLOPE to deal with the situation when one aims at selecting whole groups of explanatory variables instead of single regressors. Such groups can be formed by clustering strongly correlated predictors or groups of dummy variables corresponding to different levels of the same qualitative predictor. We formulate the respective convex optimization problem, group SLOPE (gSLOPE), and propose an efficient algorithm for its solution. We also define a notion of the group false discovery rate (gFDR) and provide a choice of the sequence of tuning parameters for gSLOPE so that gFDR is provably controlled at a prespecified level if the groups of variables are orthogonal to each other. Moreover, we prove that the resulting procedure adapts to unknown sparsity and is asymptotically minimax with respect to the estimation of the proportions of variance of the response variable explained by regressors from different groups. We also provide a method for the choice of the regularizing sequence when variables in different groups are not orthogonal but statistically independent and illustrate its good properties with computer simulations. Finally, we illustrate the advantages of gSLOPE in the context of Genome Wide Association Studies. R package grpSLOPE with an implementation of our method is available on The Comprehensive R Archive Network.

Robust sparse linear regression for tokamak plasma boundary estimation using variational Bayes

Precise control of the shape of plasma in a tokamak requires reliable reconstruction of the plasma boundary. The problem of boundary estimation can be reduced to a simple linear regression with a potentially infinite amount of regressors. This regression problem poses some difficulties for classical methods. The selection of regressors significantly influences the reconstructed boundary. Also, the underlying model may not be valid during certain phases of the plasma discharge. Formal model structure estimation technique based on the automatic relevance principle yields a version of sparse least squares estimator. In this contribution, we extend the previous method by relaxing the assumption of Gaussian noise and using Student’s t-distribution instead. Such a model is less sensitive to potential outliers in the measurement. We show on simulations and real data that the proposed modification improves estimation of the plasma boundary in some stages of a plasma discharge. Performance of the resulting algorithm is evaluated with respect to a more detailed and computationally costly model which is considered to be the “ground truth” The results are also compared to those of Lasso and Tikhonov regularization techniques.

Identification of time-varying neural dynamics from spike train data using multiwavelet basis functions

BackgroundTracking the changes of neural dynamics based on neuronal spiking activities is a critical step to understand the neurobiological basis of learning from behaving animals. These dynamical neurobiological processes associated with learning are also time-varying, which makes the modeling problem challenging. New methodWe developed a novel multiwavelet-based time-varying generalized Laguerre–Volterra (TVGLV) modeling framework to study the time-varying neural dynamical systems using natural spike train data. By projecting the time-varying parameters in the TVGLV model onto a finite sequence of multiwavelet basis functions, the time-varying identification problem is converted into a time invariant linear-in-the-parameters one. An effective forward orthogonal regression (FOR) algorithm aided by mutual information (MI) criterion is then applied for the selection of significant model regressors or terms and the refinement of model structure. A generalized linear model fit approach is finally employed for parameter estimation from spike train data. ResultsThe proposed multiwavelet-based TVGLV approach is used to identify both synthetic input-output spike trains and spontaneous retinal spike train recordings. The proposed method gives excellent the performance of tracking either sharply or slowly changing parameters with high sensitivity and accuracy regardless of the a priori knowledge of spike trains, which these results indicate that the proposed method is shown to deal well with spike train data. Comparison with existing methodsThe proposed multiwavelet-based TVGLV approach was compared with several state-of-art parametric estimation methods like the steepest descent point process filter (SDPPF) or Chebyshev polynomial expansion method. The conventional SDPPF algorithm, or SDPPF with B-splines wavelet expansion method was shown to have the poor performance of tracking the time-varying system changes with the synthetic spike train data due to the slow convergence of the adaptive filter methods. Although the Chebyshev polynomial basis function method gave the good parametric estimation results, it requires prior parameter estimation. It was shown that the proposed multiwavelet-based TVGLV method can track the time-varying parameter changes rapidly and accurately. ConclusionsThe multiwavelet-based TVGLV modeling framework developed in this paper can not only provide a computational modeling scheme for investigating such nonstationary properties, track more general forms of changes in time-varying neural dynamics, and but also may potentially be applied to investigate the spatial-temporal information underlying biomedical spiking signals.

Shrinkage estimation of dynamic panel data models with interactive fixed effects

We consider the problem of determining the number of factors and selecting the proper regressors in linear dynamic panel data models with interactive fixed effects. Based on the preliminary estimates of the slope parameters and factors a la Bai (2009) and Moon and Weidner (2015), we propose a method for simultaneous selection of regressors and factors and estimation through the method of adaptive group Lasso (least absolute shrinkage and selection operator). We show that with probability approaching one, our method can correctly select all relevant regressors and factors and shrink the coefficients of irrelevant regressors and redundant factors to zero. Further, we demonstrate that our shrinkage estimators of the nonzero slope parameters exhibit some oracle property. We conduct Monte Carlo simulations to demonstrate the superb finite-sample performance of the proposed method. We apply our method to study the determinants of economic growth and find that in addition to three common unobserved factors selected by our method, government consumption share has negative effects, whereas investment share and lagged economic growth have positive effects on economic growth.

Shrinkage Estimation of Dynamic Panel Data Models with Interactive Fixed Effects

We consider the problem of determining the number of factors and selecting the proper regressors in linear dynamic panel data models with interactive fixed effects. Based on the preliminary estimates of the slope parameters and factors a la Bai and Ng (2009) and Moon and Weidner (2014a), we propose a method for simultaneous selection of regressors and factors and estimation through the method of adaptive group Lasso (least absolute shrinkage and selection operator). We show that with probability approaching one, our method can correctly select all relevant regressors and factors and shrink the coefficients of irrelevant regressors and redundant factors to zero. Further, we demonstrate that our shrinkage estimators of the nonzero slope parameters exhibit some oracle property. We conduct Monte Carlo simulations to demonstrate the superb finite-sample performance of the proposed method. We apply our method to study the determinants of economic growth and find that in addition to three common unobserved factors selected by our method, government consumption share has negative effects, whereas investment share and lagged economic growth have positive effects on economic growth.

Multivariable Linear Regression for SOFC Stack Temperature Estimation under Degradation Effects

Multivariable linear regression (MLR) models are developed for the estimation of the maximum temperature and the temperature difference over a cell inside a solid oxide fuel cell (SOFC) stack. Empirical test data from both long-term tests (3500 hours) and a full three-factor designed experiment on a 10 kW SOFC system are utilized in the work. It is shown that accurate estimation can be carried out effectively based on systematic short-term experiments and by using simple and reliable measurements even under the effects of stack performance degradation. After proper data is obtained, selection of suitable MLR model regressors is crucial to obtaining good estimates. The cathode outlet temperature was found useful for the estimation of the stack maximum temperature and the stack voltage for the estimation of the temperature difference over a cell. Also, analysis of the measurement data shows that the experiment design can be considerably reduced without significant reduction in obtained information. The importance of using both long-term testing data as well as short-term designed experiments from an invariant system as the basis for modeling is underlined.

Low computational complexity family of affine projection algorithms over adaptive distributed incremental networks

This paper presents the problem of distributed estimation in an incremental network based on the family of affine projection (AP) adaptive algorithms. The distributed selective partial update normalized least mean squares (dSPU-NLMS), the distributed SPU-AP algorithm (dSPU-APA), the distributed selective regressor APA (dSR-APA), the distributed dynamic selection of APA (dDS-APA), dSPU-SR-APA and dSPU-DS-APA are introduced in a unified way. These algorithms have low computational complexity feature and close convergence speed to ordinary distributed adaptive algorithms. In dSPU-NLMS and dSPU-APA, the weight coefficients are partially updated at each node during the adaptation. In dSR-APA, the optimum number of input regressors is selected during the weight coefficients update. The dynamic selection of input regressors is used in dDS-APA. dSPU-SR-APA and dSPU-DS-APA combine SPU with SR and DS approaches. In these algorithms, the weight coefficients are partially updated and the input regressors are optimally/dynamically selected at every iteration for each node. In addition, a unified approach for mean-square performance analysis of each individual node is presented. This approach can be used to establish a performance analysis of classical distributed adaptive algorithms as well. The theoretical expressions for stability bounds, transient, and steady-state performance analysis of various distributed APAs are introduced. The validity of the theoretical results and the good performance of dAPAs are demonstrated by several computer simulations.

Prediction of water intake and excretion flows in Holstein dairy cows under thermoneutral conditions

The increase in the worldwide demand for dairy products, associated with global warming, will emphasize the issue of water use efficiency in dairy systems. The evaluation of environmental issues related to the management of animal dejections will also require precise biotechnical models that can predict effluent management in farms. In this study, equations were developed and evaluated for predicting the main water flows at the dairy cow level, based on parameters related to cow productive performance and diet under thermoneutral conditions. Two datasets were gathered. The first one comprised 342 individual measurements of water balance in dairy cows obtained during 18 trials at the experimental farm of Méjussaume (INRA, France). Predictive equations of water intake, urine and fecal water excretion were developed by multiple regression using a stepwise selection of regressors from a list of seven candidate parameters, which were milk yield, dry matter intake (DMI), body weight, diet dry matter content (DM), proportion of concentrate (CONC) and content of crude protein (CP) ingested with forage and concentrate (CPf and CPc, g/kg DM). The second dataset was used for external validation of the developed equations and comprised 196 water flow measurements on experimental lots obtained from 43 published papers related to water balance or digestibility measurements in dairy cows. Although DMI was the first predictor of the total water intake (TWI), with a partial r2 of 0.51, DM was the first predictive parameter of free water intake (FWI), with a partial r2 of 0.57, likely due to the large variability of DM in the first dataset (from 11.5 to 91.4 g/100 g). This confirmed the compensation between water drunk and ingested with diet when DM changes. The variability of urine volume was explained mainly by the CPf associated with DMI (r.s.d. 5.4 kg/day for an average flow of 24.0 kg/day) and that of fecal water was explained by the proportion of CONC in the diet and DMI. External validation showed that predictive equations excluding DMI as predictive parameters could be used for FWI, urine and fecal water predictions if cows were fed a well-known total mixed ration. It also appeared that TWI and FWI were underestimated when ambient temperature increased above 25°C and possible means of including climatic parameters in future predictive equations were proposed.

A family of variable step-size affine projection adaptive filter algorithms using statistics of channel impulse response

Abstract This paper extends the recently introduced variable step-size (VSS) approach to the family of adaptive filter algorithms. This method uses prior knowledge of the channel impulse response statistic. Accordingly, optimal step-size vector is obtained by minimizing the mean-square deviation (MSD). The presented algorithms are the VSS affine projection algorithm (VSS-APA), the VSS selective partial update NLMS (VSS-SPU-NLMS), the VSS-SPU-APA, and the VSS selective regressor APA (VSS-SR-APA). In VSS-SPU adaptive algorithms the filter coefficients are partially updated which reduce the computational complexity. In VSS-SR-APA, the optimal selection of input regressors is performed during the adaptation. The presented algorithms have good convergence speed, low steady state mean square error (MSE), and low computational complexity features. We demonstrate the good performance of the proposed algorithms through several simulations in system identification scenario.

PENALIZED-R2CRITERIA FOR MODEL SELECTION

It is beneficial to observe that popular model selection criteria for the linear model are equivalent to penalized versions of R2. Let PR2 refer to any one of these model selection criteria. Then PR2 serves the dual purpose of selecting the model and summarizing the resulting fit subject to the penalty function. Furthermore, it is straightforward to extend the logic of PR2 to instrumental variables estimation and the non-parametric selection of regressors. For two-stage least squares estimation, a simulation study investigates the finite-sample performance of PR2 to select the correct model in cases of either strong or weak instruments.

Optimization of Temporal Processes: A Model Predictive Control Approach

A dynamic predictive-control model of a nonlinear and temporal process is considered. Evolutionary computation and data mining algorithms are integrated for solving the model. Data-mining algorithms learn dynamic equations from process data. Evolutionary algorithms are then applied to solve the optimization problem guided by the knowledge extracted by data-mining algorithms. Several properties of the optimization model are shown in detail, in particular, a selection of regressors, time delays, prediction and control horizons, and weights. The concepts proposed in this paper are illustrated with an industrial case study in combustion process.