Kernel Smoothing Research Articles

On the Performance of Semi- and Nonparametric Item Response Functions in Computer Adaptive Tests.

Large-scale assessments often use a computer adaptive test (CAT) for selection of items and for scoring respondents. Such tests often assume a parametric form for the relationship between item responses and the underlying construct. Although semi- and nonparametric response functions could be used, there is scant research on their performance in a CAT. In this work, we compare parametric response functions versus those estimated using kernel smoothing and a logistic function of a monotonic polynomial. Monotonic polynomial items can be used with traditional CAT item selection algorithms that use analytical derivatives. We compared these approaches in CAT simulations with a variety of item selection algorithms. Our simulations also varied the features of the calibration and item pool: sample size, the presence of missing data, and the percentage of nonstandard items. In general, the results support the use of semi- and nonparametric item response functions in a CAT.

Relation between the habitat niche breadth and the geographic range size: A case study on palearctic voles (Mammalia: Rodentia: Arvicolinae)

Habitat niche breadth for Palearctic Arvicolinae species was estimated at both local (α- niche) and global (the entire geographic range, γ-niche) scales using occurrence records of species and environmental (climate, topography, and vegetation) data. Niche breadth was estimated in the space of the first two principal components of environmental variables using kernel smoothing of the densities of species occurrence points. The breadth of α-niches was estimated for a set of random points inside the geographic range in a series of buffers of increasing size around these points. Within each buffer, we calculated the overlap between the distribution of environment values for the kernel smoothed densities of species occurrence points and the distribution of environment values in the background environment. The α-niche breadth was calculated as the slope of the linear regression of the niche breadth for buffers of different size by the ln area of these buffers with a zero intercept. The γ-niche breadth was calculated as the overlap between the distributions of environmental values for the kernel smoothed densities of species occurrence points over the whole geographic range and the distribution of environmental values in the background environment and also approximated by linear regression of the species’ average α-niche to the geographic range area of this species. The results demonstrated that the geographic range size was significantly related with the α- and γ-niche breadth. The γ-niche breadth was significantly positively correlated with the α-niche breadth. Finally, the differences between the γ-niche breadth values that were directly estimated and extrapolated from the α-niche breadth (Δ) values were positively correlated with the geographic range size. Thus, we conclude that the species occupy larger geographic ranges because they have broader niches. Our estimations of the γ-niche breadth increase with the geographic range size not due to a parallel increase of the environmental diversity (spatial autocorrelation in the environment).

Improved composite quantile regression and variable selection with nonignorable dropouts

With nonignorable dropouts and outliers, we propose robust statistical inference and variable selection methods for linear quantile regression models based on composite quantile regression and empirical likelihood (EL) that accommodate both the within-subject correlations and nonignorable dropouts. The purpose of our study is threefold. First, we apply the generalized method of moments to estimate the parameters in the nonignorable dropout propensity based on an instrument. Subsequently, the inverse probability weighting and kernel smoothing approaches are applied to obtain the smoothed and bias-corrected generalized estimating equations. Second, we borrow the idea of quadratic inference function to construct the improved EL procedure for nonignorable dropouts. The asymptotic properties of the proposed estimators and their confidence regions are derived. Third, the penalized EL method and algorithm for variable selection are investigated. The finite-sample performance of the proposed estimators is studied through simulation, and an application to HIV-CD4 data set is also presented.

Nonparametric estimation in an illness-death model with component-wise censoring.

In disease settings where study participants are at risk for death and a serious nonfatal event, composite endpoints defined as the time until the earliest of death or the nonfatal event are often used as the primary endpoint in clinical trials. In practice, if the nonfatal event can only be detected at clinic visits and the death time is known exactly, the resulting composite endpoint exhibits "component-wise censoring." The standard method used to estimate event-free survival in this setting fails to account for component-wise censoring. We apply a kernel smoothing method previously proposed for a marker process in a novel way to produce a nonparametric estimator for event-free survival that accounts for component-wise censoring. The key insight that allows us to apply this kernel method is thinking of nonfatal event status as an intermittently observed binary time-dependent variable rather than thinking of time to the nonfatal event as interval-censored. We also propose estimators for the probability in state and restricted mean time in state for reversible or irreversible illness-death models, under component-wise censoring, and derive their large-sample properties. We perform a simulation study to compare our method to existing multistate survival methods and apply the methods on data from a large randomized trial studying a multifactor intervention for reducing morbidity and mortality among men at above average risk of coronary heartdisease.

Kernel density estimation for partial linear multivariate responses models

We propose a kernel density based estimation by constructing a nonparametric kernel version of the maximum profile likelihood estimator for partial linear multivariate responses regression models. The method proposed in this article makes use of multivariate kernel smoothing nonparametric techniques to estimate the unknown multivariate density function. For the hypothesis testing of parametric components, restricted estimators under the null hypothesis and test statistics are proposed. The asymptotic properties for the estimators and test statistics are established. We illustrate our proposals through simulations and an analysis of the energy efficiency data. Our analysis provides strong evidence that the proposed kernel density based estimator is superior than the profile least squares estimator, particularly for multimodal or heavy-tailed distributions of the model errors.

A Framework of Learning Through Empirical Gain Maximization.

We develop in this letter a framework of empirical gain maximization (EGM) to address the robust regression problem where heavy-tailed noise or outliers may be present in the response variable. The idea of EGM is to approximate the density function of the noise distribution instead of approximating the truth function directly as usual. Unlike the classical maximum likelihood estimation that encourages equal importance of all observations and could be problematic in the presence of abnormal observations, EGM schemes can be interpreted from a minimum distance estimation viewpoint and allow the ignorance of those observations. Furthermore, we show that several well-known robust nonconvex regression paradigms, such as Tukey regression and truncated least square regression, can be reformulated into this new framework. We then develop a learning theory for EGM by means of which a unified analysis can be conducted for these well-established but not fully understood regression approaches. This new framework leads to a novel interpretation of existing bounded nonconvex loss functions. Within this new framework, the two seemingly irrelevant terminologies, the well-known Tukey's biweight loss for robust regression and the triweight kernel for nonparametric smoothing, are closely related. More precisely, we show that Tukey's biweight loss can be derived from the triweight kernel. Other frequently employed bounded nonconvex loss functions in machine learning, such as the truncated square loss, the Geman-McClure loss, and the exponential squared loss, can also be reformulated from certain smoothing kernels in statistics. In addition, the new framework enables us to devise new bounded nonconvex loss functions for robust learning.

Investigating Clustering and Violence Interruption in Gang-Related Violent Crime Data Using Spatial–Temporal Point Processes With Covariates

Reported gang-related violent crimes in Los Angeles, California, from 1/1/14 to 12/31/17 are modeled using spatial–temporal marked Hawkes point processes with covariates. We propose an algorithm to estimate the spatial-temporally varying background rate nonparametrically as a function of demographic covariates. Kernel smoothing and generalized additive models are used in an attempt to model the background rate as closely as possible in an effort to differentiate inhomogeneity in the background rate from causal clustering or triggering of events. The models are fit to data from 2014 to 2016 and evaluated using data from 2017, based on log-likelihood and superthinned residuals. The impact of nonrandomized violence interruption performed by The City of Los Angeles Mayor’s Office of Gang Reduction Youth Development (GRYD) Incident Response (IR) Program is assessed by comparing the triggering associated with GRYD IR Program events to the triggering associated with sub-sampled non-GRYD events selected to have a similar spatial–temporal distribution. The results suggest that GRYD IR Program violence interruption yields a reduction of approximately 18.3% in the retaliation rate in locations more than 130 m from the original reported crimes, and a reduction of 14.2% in retaliations within 130 m.

Particle-Swarm-Optimization-Enhanced Radial-Basis-Function-Kernel-Based Adaptive Filtering Applied to Maritime Data

The real-life signals captured by different measurement systems (such as modern maritime transport characterized by challenging and varying operating conditions) are often subject to various types of noise and other external factors in the data collection and transmission processes. Therefore, the filtering algorithms are required to reduce the noise level in measured signals, thus enabling more efficient extraction of useful information. This paper proposes a locally-adaptive filtering algorithm based on the radial basis function (RBF) kernel smoother with variable width. The kernel width is calculated using the asymmetrical combined-window relative intersection of confidence intervals (RICI) algorithm, whose parameters are adjusted by applying the particle swarm optimization (PSO) based procedure. The proposed RBF-RICI algorithm’s filtering performances are analyzed on several simulated, synthetic noisy signals, showing its efficiency in noise suppression and filtering error reduction. Moreover, compared to the competing filtering algorithms, the proposed algorithm provides better or competitive filtering performance in most considered test cases. Finally, the proposed algorithm is applied to the noisy measured maritime data, proving to be a possible solution for a successful practical application in data filtering in maritime transport and other sectors.

Fuzzy Regression Analysis Based on Fuzzy Neural Networks Using Trapezoidal Data

Fuzzy regression is a generalized regression model to represent the relationship between dependent and independent variables in a fuzzy environment. The fuzzy linear regression analysis seeks for regression models fitting well all the data based on a specific criterion. In this paper, an adaptive neuro-fuzzy inference system (ANFIS) is employed for the analysis and prediction of a nonparametric fuzzy regression function with non-fuzzy inputs and symmetric trapezoidal fuzzy outputs. To this end, two new hybrid algorithms are proposed in which the fuzzy least squares and linear programming have been used to optimize the secondary weights. The algorithms are applied to a multi-layered validation method to confirm the models’ reliability. In addition, three methods of nonparametric fuzzy regression with crisp inputs and asymmetric trapezoidal fuzzy outputs, are compared. Three nonparametric techniques in statistics, namely local linear smoothing (L-L-S), K-nearest neighbor smoothing (K-NN) and kernel smoothing (K-S) with trapezoidal fuzzy data have been analyzed to obtain the best smoothing parameters. The performance of the models is illustrated through numerical examples and simulations. More specifically, the accuracy of the algorithms is confirmed by exhaustive simulations.

Towards Including Dynamic Vegetation Parameters in the EUMETSAT H SAF ASCAT Soil Moisture Products

The TU Wien Soil Moisture Retrieval (TUW SMR) approach is used to produce several operational soil moisture products from the Advanced Scatterometer (ASCAT) on the Metop series of satellites as part of the EUMETSAT Satellite Application Facility on Support to Operational Hydrology and Water Management (H SAF). The incidence angle dependence of backscatter is described by a second-order Taylor polynomial, the coefficients of which are used to normalize ASCAT observations to the reference incidence angle of 40∘ and for correcting vegetation effects. Recently, a kernel smoother was developed to estimate the coefficients dynamically, in order to account for interannual variability. In this study, we used the kernel smoother for estimating these coefficients, where we distinguished for the first time between their two uses, meaning that we used a short and fixed window width for the backscatter normalisation while we tested different window widths for optimizing the vegetation correction. In particular, we investigated the impact of using the dynamic vegetation parameters on soil moisture retrieval. We compared soil moisture retrievals based on the dynamic vegetation parameters to those estimated using the current operational approach by examining their agreement, in terms of the Pearson correlation coefficient, unbiased RMSE and bias with respect to in situ soil moisture. Data from the United States Climate Research Network were used to study the influence of climate class and land cover type on performance. The sensitivity to the kernel smoother half-width was also investigated. Results show that estimating the vegetation parameters with the kernel smoother can yield an improvement when there is interannual variability in vegetation due to a trend or a change in the amplitude or timing of the seasonal cycle. However, using the kernel smoother introduces high-frequency variability in the dynamic vegetation parameters, particularly for shorter kernel half-widths.

Comparison of parametric and semiparametric survival regression models with kernel estimation

The modelling of censored survival data is based on different estimations of the conditional hazard function. When survival time follows a known distribution, parametric models are useful. This strong assumption is replaced by a weaker in the case of semiparametric models. For instance, the frequently used model suggested by Cox is based on the proportionality of hazards. These models use non-parametric methods to estimate some baseline hazard and parametric methods to estimate the influence of a covariate. An alternative approach is to use smoothing that is more flexible. In this paper, two types of kernel smoothing and some bandwidth selection techniques are introduced. Application to real data shows different interpretations for each approach. The extensive simulation study is aimed at comparing different approaches and assessing their benefits. Kernel estimation is demonstrated to be very helpful for verifying assumptions of parametric or semiparametric models and is able to capture changes in the hazard function in both time and covariate directions.

A joint introduction to Gaussian Processes and Relevance Vector Machines with connections to Kalman filtering and other kernel smoothers

The expressive power of Bayesian kernel-based methods has led them to become an important tool across many different facets of artificial intelligence, and useful to a plethora of modern application domains, providing both power and interpretability via uncertainty analysis. This article introduces and discusses two methods which straddle the areas of probabilistic Bayesian schemes and kernel methods for regression: Gaussian Processes and Relevance Vector Machines. Our focus is on developing a common framework with which to view these methods, via intermediate methods a probabilistic version of the well-known kernel ridge regression, and drawing connections among them, via dual formulations, and discussion of their application in the context of major tasks: regression, smoothing, interpolation, and filtering. Overall, we provide understanding of the mathematical concepts behind these models, and we summarize and discuss in depth different interpretations and highlight the relationship to other methods, such as linear kernel smoothers, Kalman filtering and Fourier approximations. Throughout, we provide numerous figures to promote understanding, and we make numerous recommendations to practitioners. Benefits and drawbacks of the different techniques are highlighted. To our knowledge, this is the most in-depth study of its kind to date focused on these two methods, and will be relevant to theoretical understanding and practitioners throughout the domains of data-science, signal processing, machine learning, and artificial intelligence in general.

Assessing movement patterns using Bayesian state space models on Lake Winnipeg walleye

Acoustic telemetry systems technology is useful for studying fish movement patterns and habitat use; however, the data generated from omnidirectional acoustic receivers are prone to large observation errors because the tagged animal can be anywhere in the detection range of the receiver. In this study, we used the Bayesian state space modeling (SSM) approach and different smoothing methods including kernel smoothing and cross-validated local polynomial regression to reconstruct fish movement paths of walleye (Sander vitreus) using data obtained from a telemetry receiver grid in Lake Winnipeg, Manitoba, Canada. Using the SSM approach, we obtained more realistic movement paths, compared with the smoothing methods. In addition, we have highlighted the advantages of the SSM approach to estimate undetected movement paths, over simple smoothing techniques, by comparing ecological metrics such as path length and tortuosity between different reconstruction approaches. Reconstructed paths could be useful in making effective fishery management decisions on Lake Winnipeg in the future by providing information on how walleye move and distribute in Lake Winnipeg over space and time.

Tornado wind hazard mapping and equivalent tornado design wind profile for Canada

Tornadoes cause property damage and fatalities. A systematic assessment of tornado wind velocity hazard is presented in the present study for Canada based on reported historical tornado catalogue. For the assessment, the Poisson model, zero-inflated Poisson model, and negative binomial model, as well as the adaptive Gaussian kernel smoothing technique, are used to model tornado occurrence. The modeling takes into account the tornado reporting bias due to population density and uses the cloud-to-ground lightning flash density and the thunderstorm days as the explanatory variables. Statistical analysis results indicate that the negative binomial model is preferable.The tornado wind velocity hazard maps for Canada, in terms of wind velocity at 10 m height above the ground surface and return period T, VT(10), are developed based on an available probabilistic tornado wind field model. Also, a simple equivalent tornado design wind profile is developed based on the return period value of tornado wind speed at different heights. The developed tornado design wind profile only depends on VT(10), which greatly facilitates its potential use for structural design subjected to tornado wind load and its possible implementation in structural design code and standards.

Cox regression analysis for distorted covariates with an unknown distortion function.

We study inference for censored survival data where some covariates are distorted by some unknown functions of an observable confounding variable in a multiplicative form. An example of this kind of data in medical studies is normalizing some important observed exposure variables by patients' body mass index , weight, or age. Such a phenomenon also appears frequently in environmental studies where an ambient measure is used for normalization and in genomic studies where the library size needs to be normalized for the next generation sequencing of data. We propose a new covariate-adjusted Cox proportional hazards regression model and utilize the kernel smoothing method to estimate the distorting function, then employ an estimated maximum likelihood method to derive the estimator for the regression parameters. We establish the large sample properties of the proposed estimator. Extensive simulation studies demonstrate that the proposed estimator performs well in correcting the bias arising from distortion. A real dataset from the National Wilms' Tumor Study is used to illustrate the proposedapproach.

Varying coefficient frailty models with applications in single molecular experiments.

Motivated by an analysis of single molecular experiments in the study of T-cell signaling, a new model called varying coefficient frailty model with local linear estimation is proposed. Frailty models have been extensively studied, but extensions to nonconstant coefficients are limited to spline-based methods that tend to produce estimation bias near the boundary. To address this problem, we introduce a local polynomial kernel smoothing technique with a modified expectation-maximization algorithm to estimate the unknown parameters. Theoretical properties of the estimators, including their unbiased property near the boundary, are derived along with discussions on the asymptotic bias-variance trade-off. The finite sample performance is examined by simulation studies, and comparisons with existing spline-based approaches are conducted to show the potential advantages of the proposed approach. The proposed method is implemented for the analysis of T-cell signaling. The fitted varying coefficient model provides a rigorous quantification of an early and rapid impact on T-cell signaling from the accumulation of bond lifetime, which can shed new light on the fundamental understanding of how T cells initiate immuneresponses.

Parametric Versus Semi and Nonparametric Regression Models

There are three common types of regression models: parametric, semiparametric and nonparametric regression. The model should be used to fit the real data depends on how much information is available about the form of the relationship between the response variable and explanatory variables, and the random error distribution that is assumed. Researchers need to be familiar with each modeling approach requirements. In this paper, differences between these models, common estimation methods, robust estimation, and applications are introduced. For parametric models, there are many known methods of estimation, such as least squares and maximum likelihood methods which are extensively studied but they require strong assumptions. On the other hand, nonparametric regression models are free of assumptions regarding the form of the response-explanatory variables relationships but estimation methods, such as kernel and spline smoothing are computationally expensive and smoothing parameters need to be obtained. For kernel smoothing there two common estimators: local constant and local linear smoothing methods. In terms of bias, especially at the boundaries of the data range, local linear is better than local constant estimator.  Robust estimation methods for linear models are well studied, however the robust estimation methods in nonparametric regression methods are limited. A robust estimation method for the semiparametric and nonparametric regression models is introduced.

Small area estimation and childhood obesity surveillance using electronic health records.

There is an urgent need for childhood surveillance systems to design, implement, and evaluate interventions at the local level. We estimated obesity prevalence for individuals aged 5-17 years using a southcentral Wisconsin EHR data repository, Public Health Information Exchange (PHINEX, 2007-2012). The prevalence estimates were calculated by aggregating the estimated probability of each individual being obese, which was obtained via a generalized linear mixed model. We incorporated the random effects at the area level into our model. A weighted procedure was employed to account for missingness in EHR data. A non-parametric kernel smoothing method was used to obtain the prevalence estimates for locations with no or little data (<20 individuals) from the EHR. These estimates were compared to results from newly available obesity atlas (2015-2016) developed from various EHRs with greater statewide representation. The mean of the zip code level obesity prevalence estimates for males and females aged 5-17 years is 16.2% (SD 2.72%); 17.9% (SD 2.14%) for males and 14.4% (SD 2.00%) for females. The results were comparable to the Wisconsin Health Atlas (WHA) estimates, a much larger dataset of local community EHRs in Wisconsin. On average, prevalence estimates were 2.12% lower in this process than the WHA estimates, with lower estimation occurring more frequently for zip codes without data in PHINEX. Using this approach, we can obtain estimates for local areas that lack EHRs data. Generally, lower prevalence estimates were produced for those locations not represented in the PHINEX database when compared to WHA estimates. This underscores the need to ensure that the reference EHRs database can be made sufficiently similar to the geographic areas where synthetic estimates are being created.

The spectrogram, method of reassignment, and frequency-domain beamforming.

A smeared spectrogram is a result of the smoothing kernel in the short-time Fourier-transform (STFT). Besides the smeared energy, time and frequency phase information is also smeared, i.e., spectral components may contain imprecise phase information. The STFT is also used as the basis for more advanced signal processing techniques such as frequency-domain beamforming and cross correlation (CC). Both methods seek the delay time between signals by exploring phase-shifts in the frequency domain. Due to the inexact phase information in some of the time-frequency elements, their phase shifts are incorrect. This study re-introduces the reassigned spectrogram (RS) as a measure to fix the STFT artifacts. Moreover, it is shown that by using the RS, phase shifts can be optimized and improve beamforming and CC results. Synthetic and recorded data are used to show the advantage of using the RS in time-frequency analysis, CC, and beamforming. Results show that, subject to certain constraints, the RS provides exact time-frequency representation of deterministic signals and significantly improve CC and beamforming results. Array analysis of infrasonic signals shows that better results are obtained by either the RS- or STFT-based analysis depending on the signals' spectral components and noise levels.

Nonparametric estimation of aggregated Sobol’ indices: Application to a depth averaged snow avalanche model

Avalanche models are increasingly employed for elaborating land-use maps and designing defense structures, but they rely on poorly known parameters. Careful uncertainty assessment is thus required but difficulty arises from the nature of the outputs of these models, which are commonly both functional and scalar. Hence, so far in the avalanche field, few sensitivity analyses have been performed. In this work, we propose to determine the most influential inputs of an avalanche model by estimating aggregated first-order Sobol’ indices. We propose a nonparametric estimation procedure based on the Nadaraya–Watson kernel smoother, which allows to estimate the aggregated Sobol’ indices from a given random sample of small to moderate size. Due to the limited size of the sample, the kernel estimation is biased. Therefore, we propose a bootstrap based bias correction before selecting the bandwidth by cross-validation. To estimate the aggregated Sobol’ indices, we reduce the dimension of the output using principal components analysis. After different test cases showing the efficiency of our approach, it is applied to a real avalanche case. Results show that the friction parameters and the snow depth in the release zone are the most influential parameters determining the avalanche characteristics.