Nonparametric Regression Method Research Articles

A multigrid preconditioner for tensor product spline smoothing

Penalized spline smoothing is a well-established, nonparametric regression method that is efficient for one and two covariates. Its extension to more than two covariates is straightforward but suffers from exponentially increasing memory demands and computational complexity, which brings the method to its numerical limit. Penalized spline smoothing with multiple covariates requires solving a large-scale, regularized least-squares problem where the occurring matrices do not fit into storage of common computer systems. To overcome this restriction, we introduce a matrix-free implementation of the conjugate gradient method. We further present a matrix-free implementation of a simple diagonal as well as more advanced geometric multigrid preconditioner to significantly speed up convergence of the conjugate gradient method. All algorithms require a negligible amount of memory and therefore allow for penalized spline smoothing with multiple covariates. Moreover, for arbitrary but fixed covariate dimension, we show grid independent convergence of the multigrid preconditioner which is fundamental to achieve algorithmic scalability.

Experimental Approach for the Evaluation of the Performance of a Satellite Module in the CanSat Form Factor for In Situ Monitoring and Remote Sensing Applications

This article includes the phases of conceptualization and validation of a picosatellite prototype named Simple-2 for remote sensing activities using COTS (Commercial-Off-The-Shelf) components and the modular design methodology. To evaluate its performance and ensure the precision and accuracy of the measurements made by the satellite prototype, a methodology was designed and implemented for the characterization and qualification of CanSats (soda can satellites) through statistical tests and techniques of DoE (Design of Experiments) based on CubeSat aerospace standards and regulations, in the absence of official test procedures for these kinds of satellite form factor. For the above, two experimental units were used, and all the performance variables of the different satellite subsystems were discriminated. For the above, two experimental units were used, and all the performance variables of the different satellite subsystems were discriminated against. These were grouped according to the dependence of the treatments formulated in thermal and dynamic variables. For the tests of the first variables, a one-factor design was established using dependent samples on each of the treatments. Then, hypothesis tests were performed for equality of medians, using nonparametric analysis of the Kruskal-Wallis variance. Additionally, multivariate analysis of variance was carried out for nonparametric samples (nonparametric multivariate tests), and the application of post hoc multiple-range tests to identify the treatments that presented significant differences within a margin of acceptability. To know the dynamic response and ensure the structural integrity of the satellite module, shock, oscillation, and sinusoidal tests were applied through a shaker. Having applied the experimental methodology to the different units, the results of a real experiment are illustrated in which a high-altitude balloon was used through the application of nonparametric regression methods. This experiment’s interest measured thermodynamic variables and the concentration of pollutants in the stratosphere to corroborate the operating ranges planned in the above experiments using on-flight conditions and estimate the TLR (technology readiness level) of future prototypes.

PENERAPAN REGRESI NONPARAMETRIK KERNEL DAN SPLINE DALAM MEMODELKAN RETURN ON ASSET (ROA) BANK SYARIAH DI INDONESIA

Sharia Bank Return On Assets (ROA) modeling in Indonesia in 2018 aims to analyze the relationship pattern of Retturn On Assets (ROA) with interest rates. The analysis that is often used for modeling is regression analysis. Regression analysis is divided into two, namely parametric and nonparametric. The most commonly used nonparametric regression methods are kernel and spline regression. In this study, the nonparametric regression used was kernel regression with the Nadaraya-Watson (NWE) estimator and Local Polynomial (LPE) estimator, while the spline regression was smoothing spline and B-splines. The fitting curve results show that the best model is the B-splines regression model with a degree of 3 and the number of knots 5. This is because the B-splines regression model has a smooth curve and more closely follows the distribution of data compared to other regression curves. The B-splines regression model has a determination coefficient of R ^ 2 of 74.92%,%, meaning that the amount of variation in the ROA variable described by the B-splines regression model is 74.92%, while the remaining 25.8% is explained by other variables not included in the model.

Modification of Multivariate Adaptive Regression Spline (MARS)

Multivariate Adaptive Regression Spline (MARS) is a nonparametric regression method that can accommodate additive effects and interaction effects between predictor variables. Generally, MARS has been used for modeling pairs of data with continuous or categorical responses. One type of categorical data that needs special attention in modeling is count data. The count data is often encountered, especially in the health sector. The existence of count data motivates the development of the theory and application of the MARS method, which is the Multivariate Adaptive Poisson Regression Spline (MAPRS). The MAPRS is a combination of MARS and Poisson regression. It can accommodate and analyze the data according to its type and distribution. The application of MAPRS to model the count of Tuberculosis (TB) shows that it outperforms the Poisson regression.

Mixture Model Nonparametric Regression and Its Application

Regression analysis is a statistical analysis used to determine the pattern of relationships between predictor variables and response variables. There are two models of estimation approaches in regression analysis, namely parametric regression, and nonparametric regression. The parametric regression approach is used in the shape of the regression curve is known. In cases with unknown relationship patterns, the development is done using nonparametric regression. Nonparametric regression is a model estimation method which is based on an approach that is not bound by certain assumptions of the regression curve shape. Nonparametric regression varies greatly with variable curves that are different between one predictor variable with another predictor variable. In nonparametric regression, there are several types of the recommended kernel, spline, and Fourier series. In many cases, however, these conservative nonparametric regression methods cannot handle more complex problems. Mixture methods by combining several methods such as a mixture of spline and Fourier series, kernel and Fourier series, and so on, give a better result. This study aims to obtain estimates of a mixture of a truncated spline, kernel, and Fourier series by using the Ordinary Least Square (OLS) method and obtain methods for selecting knots, bandwidth, and optimal oscillation parameters with the smallest GCV. The results of this study are the formulation of a mixed estimation model of the truncated spline, kernel, and Fourier series and the smallest GCV formula to obtain the optimum location and number of points of knots, bandwidth, and oscillations.

Parametric Versus Semi and Nonparametric Regression Models

There are three common types of regression models: parametric, semiparametric and nonparametric regression. The model should be used to fit the real data depends on how much information is available about the form of the relationship between the response variable and explanatory variables, and the random error distribution that is assumed. Researchers need to be familiar with each modeling approach requirements. In this paper, differences between these models, common estimation methods, robust estimation, and applications are introduced. For parametric models, there are many known methods of estimation, such as least squares and maximum likelihood methods which are extensively studied but they require strong assumptions. On the other hand, nonparametric regression models are free of assumptions regarding the form of the response-explanatory variables relationships but estimation methods, such as kernel and spline smoothing are computationally expensive and smoothing parameters need to be obtained. For kernel smoothing there two common estimators: local constant and local linear smoothing methods. In terms of bias, especially at the boundaries of the data range, local linear is better than local constant estimator.  Robust estimation methods for linear models are well studied, however the robust estimation methods in nonparametric regression methods are limited. A robust estimation method for the semiparametric and nonparametric regression models is introduced.

New Risk Bounds for 2D Total Variation Denoising

2D Total Variation Denoising (TVD) is a widely used technique for image denoising. It is also an important nonparametric regression method for estimating functions with heterogenous smoothness. Recent results have shown the TVD estimator to be nearly minimax rate optimal for the class of functions with bounded variation. In this paper, we complement these worst case guarantees by investigating the adaptivity of the TVD estimator to functions which are piecewise constant on axis aligned rectangles. We rigorously show that, when the truth is piecewise constant, the ideally tuned TVD estimator performs better than in the worst case. We also study the issue of choosing the tuning parameter. In particular, we propose a fully data driven version of the TVD estimator which enjoys similar worst case risk guarantees as the ideally tuned TVD estimator.

Qualifying drug dosing regimens in pediatrics using Gaussian processes.

Drug development commonly studies an adult population first and then the pediatric population. The knowledge from the adult population is taken advantage of for the design of the pediatric trials. Adjusted drug doses for these are often derived from adult pharmacokinetic (PK) models which are extrapolated to patients with smaller body size. This extrapolation is based on scaling physiologic model parameters with a body size measure accounting for organ size differences. The inherent assumption is that children are merely small adults. However, children can be subject to additional effects such as organ maturation. These effects are not present in the adult population and are possibly overlooked at the design stage of the pediatric trials. It is thus crucial to qualify the extrapolation assumptions once the pediatric trial data are available. In this work, we propose a model based on a non-parametric regression method called Gaussian process (GP) to detect deviations from the made extrapolation assumptions. We introduce the theoretical background of this model and compare its performance to a parametric expansion of the adult model. The comparison includes simulations and a clinical study data example. The results show that the GP approach can reliably detect maturation trends from sparse pediatric data.

A nonparametric analysis of household-level food insecurity and its determinant factors: exploratory study in Ethiopia and Nigeria

Given the fundamental importance of food to human well-being, understanding food insecurity is crucial for sustainable development. However, due to the complex nature of food insecurity, traditional linear methods of empirical analysis may mask critical relationships between food insecurity and demographic, agricultural, and environmental factors. Here we show, using two years of household-level survey data from Ethiopia and Nigeria, that nonparametric regression (“random forest”, in this study) enables enhanced insight into the factors associated with self-reported food security and household dietary diversity score. We observe nonlinearities and thresholds in the relationships between the measures of food security, livestock ownership, and climatic conditions. The threshold-based relationships suggest that policies aimed at increasing agricultural productivity (e.g., livestock holdings) may only be beneficial up to an extent. While it is intuitive that some level of diminishing returns will exist, our nonparametric analysis could be used as a first step to discern the levels to which policies may be beneficial. Additionally, our results indicate that the random forest (and perhaps nonparametric regression and classification methods more generally) may be especially well-positioned to uncover nuances in these relationships in years with suboptimal climatic conditions (such as during the 2015 drought in Ethiopia). Ultimately, we argue that nonparametric approaches, when informed by existing theory, provide an insightful complement to inform the analysis of agricultural and development policy.

Outlier detection under a covariate-adjusted exponential regression model with censored data

Exponential regression models with censored data are most widely used in practice. In the modeling process, there exist situations where the covariates are not directly observed but are observed after being contaminated by unknown functions of an observable confounder in a multiplicative manner. The problem of outlier detection is a fundamental and important problem in applied statistics. In this paper, we use a nonparametric regression method to adjust the covariates and recast the outlier detection issue into a high-dimensional regularization regression issue in the covariate-adjusted exponential regression model with censored data. We propose a smoothly clipped absolute deviation (SCAD) penalized likelihood method to detect the possible outliers, which features that the proposed method can simultaneously deal with outlier detection and estimations for the regression coefficients. The coordinate descent algorithm is employed to facilitate computation. Simulation studies are conducted to evaluate the finite-sample performance of our proposed method. An application to a German breast cancer study demonstrates the utility of the proposed method in practice.

MODELLING NONLINEAR RELATION BY USING RUNNING INTERVAL SMOOTHER, CONSTRAINED B-SPLINE SMOOTHING AND DIFFERENT QUANTILE ESTIMATORS

This paper compares the small-sample properties of two non-parametric regression methods, running interval smoother and constrained b-spline smoothing. The running interval smoother method deals with estimation of a conditional quantile (or a measure of location) using different estimators and here our focus is on Harrell-Davis and newly proposed NO quantile estimators. The constrained b-spline smoothing method uses the quantile regression estimator while obtaining conditional quantile estimates. Constrained b-spline smoothing and running interval smoother methods are compared with a simulation study by using theoretical distributions. Furthermore, the methods are examined graphically to understand how they can model the relationship between variables. Constrained b-spline smoothing and running interval smoother with NO estimator outperformed running interval smoother with Harrell-Davis estimator in terms of mean squared error.

Regresi Nonparametrik Kernel Gaussian pada Pemodelan Angka Kelahiran Kasar di Provinsi Nusa Tenggara Barat

This study aims to model Crude Birth Rates (CBR) in West Nusa Tenggara Province. The nonparametric regression method was used in this research by considering data distribution patterns that do not show a linear relationship between variables. In this case, the kernel nonparametric regression using the Gaussian function and the Nadaraya-Watson estimator. The results showed optimal bandwidths of 0.55542837, 1.29042927, 0.94706041, and 0.92278896 with a value of minimum Generalized Cross-Validation (GCV) of 0.000000000432613511, which was minimized by the simulated annealing algorithm. The resulting model's accuracy can be seen from the coefficient of determination (R2) of 99.23% and the Mean Absolute Percentage Error (MAPE) of 0.007049%.

Stem taper modeling equation for dahurian larch based on nonparametric regression methods

Stem taper modeling equation for dahurian larch based on nonparametric regression methods

PEMODELAN INDEKS HARGA SAHAM GABUNGAN (IHSG) DAN JAKARTA ISLAMIC INDEX (JII) MENGGUNAKAN REGRESI BIRESPON SPLINE TRUNCATED BERBASIS GUI R

The capital market is one of the economic drivers and representations for assessing the condition of companies in a country. Indonesia Stock Exchange (IDX) as one of the institutions in the capital market has 24 types of indexes that can be used as main indicators that reflect the performance of capital market, two of them are the Composite Stock Price Index (CSPI) and the Jakarta Islamic Index (JII). CSPI and JII movements are influenced by several factors, both from domestic and from foreign, such as inflation and the Dow Jones Industrial Average (DJIA). Modeling of CSPI and JII in this study was carried out using biresponses spline truncated nonparametric regression methods using Graphical User Interface (GUI) R with the intention of facilitating the analysis process. This method is used because there is a correlation between CSPI and JII and there is no specific relationship pattern between the response variable (CSPI and JII) and the predictor variable (inflation and DJIA). The best biresponses spline truncated model is determined by the order, number and location of the knots seen based on minimum GCV criteria. By using monthly data from January 2016 to December 2019, the best biresponses spline truncated model is obtained when the model for CSPI is in order 2 and the model for JII is in order 3 with 2 knots for each predictor variable. This model has a coefficient of determination of 85,54437% and MAPE of 2,65595% so that it has a very good ability in forecasting.

Abstract 370: Electrical Features Of Pulseless Electrical Activity Associated With Cardiac Arrest Outcomes

Introduction: The cardiac arrest rhythm of pulseless electrical activity (PEA) poses various diagnostic and therapeutic challenges. PEA may represent a spectrum of arrest conditions with variable responses to resuscitation care. Aim: We analyzed PEA rhythms to identify diagnostic patterns associated with survival in cardiac arrest. Methods: In this retrospective cohort study, we utilized the Portland Resuscitation Outcomes Consortium database of out-of-hospital cardiac arrests compiled by the Tualatin Valley Fire and Rescue from 2006-2016. Recordings from defibrillation pads included compression waveforms, electrocardiogram, and transthoracic impedance signals. For each patient, we analyzed the first two pauses in chest compressions, characterized by flat compression and impedance signals. Features extracted from raw ECG signals included contraction frequency and variability. Signal Fourier transformation and 0-100 Hz band pass filtering yielded signals’ distribution across a frequency spectrum from which signal power was extracted. Extraction of the three most prominent frequencies was performed from the Gaussian filtered frequency spectrum. Non-parametric tests (Mann-Whitney, Fisher) and logistic regression methods were used for analysis. Results: Fifty-nine ECG recordings were analyzed corresponding to 7 (11.9%) survivors and 52 (88.1%) non-survivors. Median age was 72 (IQR 20), and 28.8% (17/59) were female. No significant differences were noted in sex or median age between survivors and non-survivors. Analysis of the first ECG pause showed a higher first peak median frequency among survivors (2.15 vs 0.06 Hz, p=0.049). We did not find a significant association between the second peak median frequency of the first ECG segment (6.46 vs 1.49 Hz, p=0.882) or the signal power of the second ECG segment (108.04 vs 100.77 Hz, p=0.647) with survival. Regression analysis did not provide reliable outcome prediction models for survival in this preliminary cohort. Conclusion: Computerized analysis of PEA ECG waveforms offers alternate approaches to bedside signal interpretation that may correlate with survival. Our preliminary work offers a potential approach to PEA analysis that will require application to a larger PEA arrest cohort.

Local Linear Forests

Random forests are a powerful method for nonparametric regression, but are limited in their ability to fit smooth signals. Taking the perspective of random forests as an adaptive kernel method, we pair the forest kernel with a local linear regression adjustment to better capture smoothness. The resulting procedure, local linear forests, enables us to improve on asymptotic rates of convergence for random forests with smooth signals, and provides substantial gains in accuracy on both real and simulated data. We prove a central limit theorem valid under regularity conditions on the forest and smoothness constraints, and propose a computationally efficient construction for confidence intervals. Moving to a causal inference application, we discuss the merits of local regression adjustments for heterogeneous treatment effect estimation, and give an example on a dataset exploring the effect word choice has on attitudes to the social safety net. Last, we include simulation results on real and generated data. A software implementation is available in the R package grf. Supplementary materials for this article are available online.

Model robust profile monitoring for the generalized linear mixed model for Phase I analysis

AbstractThe generalized linear mixed model (GLMM) becomes very popular in profile monitoring, especially when the production processes follow nonnormal distribution. In most of the real‐life applications in industry, medicine, biology…and so on researchers assume that the response variable follows a Bernoulli or Binomial distribution. The majority of previous studies in profile monitoring focused on parametric modeling using the logistic regression model, with both fixed or random effects, under the assumption of correct model specification. This research considers those cases where the parametric logistic regression model for the family of profiles is unknown or at least uncertain. Consequently, we propose two mixed model methods to monitor profiles from the exponential family: a nonparametric (NP) regression method based on the penalized spline regression technique and a semiparametric method (model robust profile monitoring for the generalized linear mixed model) which combines the advantages of both the parametric and NP methods. Several Hotelling T2 charts that have been studied for a binary response variable with replicates for Phase I profile monitoring. The performance of the proposed method is evaluated by using mean squares of errors and probability of signals criteria. The results showed satisfactory performance of the proposed control charts.

Pemodelan Regresi Nonparametrik Spline Linear Persentase Penduduk Miskin di Kalimantan

Poverty is a social problem faced in almost every country. Based on BPS data published in 2018, East Kalimantan Province has a population of poor people of 222.39 thousand people or around 6.06 percent. In March 2018, the number of poor people was 218.90 thousand people or about 6.03 percent, which means the number of poor people had increased by an absolute 3.49 thousand people, this caused the percentage of poor people to rise 0.03 percent. In this study identified factors that influence the percentage of poor population using a linear spline nonparametric regression model. The data used in this study are data from the Central Statistics Agency (BPS) in 5 provinces in Kalimantan. In the nonparametric linear spline regression method using the optimal knot point based on the smallest GCV value. The results obtained an R2 value of 74.48% which shows that the model formed is feasible to be used to model the data pattern and there are 5 variables that have a significant effect on the Percentage of Poor Population, namely Population Growth Rate, School Length Average, School Old School Expectancy Rate, Level Open Unemployment, Labor Force Participation Rate.

Asymptotics for L_{1}-wavelet method for nonparametric regression

Wavelets are particularly useful because of their natural adaptive ability to characterize data with intrinsically local properties. When the data contain outliers or come from a population with a heavy-tailed distribution, L_{1}-estimation should obtain a better fit. In this paper, we propose a L_{1}-wavelet method for nonparametric regression, and derive the asymptotic properties of the L_{1}-wavelet estimator, including the Bahadur representation, the rate of convergence and asymptotic normality. The rate of convergence of it is comparable with the optimal convergence rate of the nonparametric estimation in nonparametric models, and it does not require the continuously differentiable conditions of a nonparametric function.

Submerged macrophyte assessment in rivers: An automatic mapping method using Pléiades imagery

Submerged macrophyte monitoring is a major concern for hydrosystem management, particularly for understanding and preventing the potential impacts of global change on ecological functions and services. Macrophyte distribution assessments in rivers are still primarily realized using field monitoring or manual photo-interpretation of aerial images. Considering the lack of applications in fluvial environments, developing operational, low-cost and less time-consuming tools able to automatically map and monitor submerged macrophyte distribution is therefore crucial to support effective management programs. In this study, the suitability of very fine-scale resolution (50 cm) multispectral Pléiades satellite imagery to estimate submerged macrophyte cover, at the scale of a 1 km river section, was investigated. The performance of nonparametric regression methods (based on two reliable and well-known machine learning algorithms for remote sensing applications, Random Forest and Support Vector Regression) were compared for several spectral datasets, testing the relevance of 4 spectral bands (red, green, blue and near-infrared) and two vegetation indices (the Normalized Difference Vegetation Index, NDVI, and the Green-Red Vegetation Index, GRVI), and for several field sampling configurations. Both machine learning algorithms applied to a Pléiades image were able to reasonably well predict macrophyte cover in river ecosystems with promising performance metrics (R² above 0.7 and RMSE around 20%). The Random Forest algorithm combined to the 4 spectral bands from Pléiades image was the most efficient, particularly for extreme cover values (0% and 100%). Our study also demonstrated that a larger number of fine-scale field sampling entities clearly involved better cover predictions than a smaller number of larger sampling entities.