Local Density Estimation Research Articles

Bias and bandwidth for local likelihood density estimation

A local likelihood density estimator is shown to have asymptotic bias depending on the dimension of the local parameterization. Comparing with kernel estimation it is demonstrated using a variety of bandwidths that we may obtain as good and potentially even better estimates using local likelihood. Boundary effects are also examined.

Segmentation of Low-Cost Remote Sensing Images Combining Vegetation Indices and Mean Shift

The development of low-cost remote sensing systems is important in small agriculture business, particularly in developing countries, to allow feasible use of images to gather information. However, images obtained through such systems with uncalibrated cameras have often illumination variations, shadows, and other elements that can hinder the analysis by image processing techniques. This letter investigates the combination of vegetation indices (color index of vegetation extraction, visual vegetation index, and excess green) and the mean-shift algorithm, based on the local density estimation in the color space on images acquired by a low-cost system. The objective is to detect green coverage, gaps, and degraded areas. The results showed that combining local density estimation and vegetation indices improves the segmentation accuracy when compared with the competing methods. It deals well with images in different conditions and with regions of imbalanced sizes, confirming the practical application of the low-cost system.

Nonlinear Incentive Schemes and Corruption in Public Procurement: Evidence from the Czech Republic

This article uses data on Czech public procurement contracts from 2005-2010 in order to uncover patterns suggestive of corrupt behavior of procuring officials. Using polynomial regressions and local linear density estimators, the article provides evidence that procurement officials manipulate anticipated values of procurements so that contracts can be awarded through less transparent procedures with restricted entry. Manipulations manifest through emergence of sharp discontinuities in the anticipated value distribution. Procurements excessively bunch below statutory thresholds, which determine officials’ scope of discretion, entry-restrictiveness and transparency of the contract-awarding process. The first appearance of discontinuities coincides almost exactly with thresholds being introduced into the procurement legislation. Manipulations occur only in procedures restricted by thresholds and are prevalent only among a narrow group of procuring bodies. The last finding is consistent with manipulations being driven by corruption of procurement officials. Manipulations concern 8.6% of all below-limit procurements.

Shallow Bathymetric Mapping via Multistop Single Photoelectron Sensitivity Laser Ranging

We discuss the optimization of components in a single-wavelength airborne laser bathymeter that is intended for a low-power unmanned aerial vehicle platform. The theoretical minimum energy requirement to detect the submerged sea floor in shallow (<; 5 m) water using a low signal-to-noise ratio (LSNR) detection methodology is calculated. Results are presented from tests of a prototype light detection and ranging (LiDAR) instrument that was developed by the University of Florida, Gainesville. A green wavelength (532 nm), 100-beamlet, low-energy (35-nJ/beamlet), short-pulse (480 ps) laser ranging system was operated from a low-altitude (500-m) aircraft, with a multichannel sensor that is capable of single photoelectron sensitivity and multiple stops. Data that were collected during tests display vertical structure in shallow-water areas based on fixed threshold crossings at a single-photon sensitivity level. A major concern for the binary detection strategy is the reliable identification and removal of noise events. Potential causes of ranging errors related to photomultiplier tube afterpulsing, impedance mismatching, and gain block overdrive are described. Data collection/processing solutions based on local density estimation are explored. Previous studies on LSNR performance metrics showed that short (15-cm) dead time could be expected in the case of multiple scattering objects, indicating the possibility of seamless topographic/bathymetric mapping with minimal discontinuity at the waterline. LiDAR depth estimates from airborne profiles are compared to on-site measurements, and near-shore submerged feature identification is presented.

Real‐time deployment of artificial neural network forecasting models: Understanding the range of applicability

When an operational artificial neural network (ANN) model is deployed, new input patterns are collected in order to make real‐time forecasts. However, ANNs (like other empirical and statistical methods) are unable to reliably extrapolate beyond the calibration range. Consequently, when deployed in real‐time operation there is a need to determine if new input patterns are representative of the data used in calibrating the model. To address this problem, a novel detection system for identifying uncharacteristic data patterns is presented. This approach combines a self‐organizing map (SOM), to partition the data set, with nonparametric kernel density estimators to calculate local density estimates (LDE). The SOM‐LDE method determines the degree to which a new input pattern can be considered to be contained within the domain of the calibration set. If a new pattern is found to be uncharacteristic, a warning can be issued with the forecast, and the ANN model retrained to include the new pattern. This approach of selectively retraining the model is compared to no retraining and the more computationally onerous case of retraining the model after each new sample. These three approaches are applied to forecast flow in the Kentucky River, USA, using multilayer perceptron (MLP) models. The results demonstrate that there is a significant advantage in retraining an ANN that has been deployed as a real‐time, operational model, and that the SOM‐LDE classifier is an effective approach for identifying the model's range of applicability and assessing the usefulness of the forecast.

Smoothing survival densities in practice

Many nonparametric smoothing procedures consider independent identically distributed stochastic variables. There are also many important nonparametric smoothing applications where the data is more complicated. Survival data or filtered data, defined as following Aalen’s multiplicative hazard model and aggregated versions of this model, are considered. Aalen’s model based on counting process theory allows multiple left truncations and multiple right censoring to be present in the data. This type of filtering is omnipresent in biostatistical and demographical applications, where people can join a study, leave the study and perhaps join the study again. The estimation methodology is based on a recent class of local linear density estimators. A new stable bandwidth-selector is developed for these estimators. A data application to aggregated national mortality data is provided, where immigrations to and from the country correspond to respectively left truncation and right censoring. The aggregated mortality data study illustrates that the new practical density estimators provide an important extra element in the visual toolbox for understanding survival data.

Filtering with state space localized Kalman gain

It is well known that particle filters and Markov chain Monte Carlo methods are too expensive for high dimensional models; more robust techniques such as low rank versions of the Kalman filter are usually applied in large scale models. Common feature of all the ensemble Kalman type filters is that the model state and parameters are updated using the covariance matrix between the model states and the measurements, often estimated by a finite number of samples. In many nonlinear problems a global linearization of the model may be inadequate to describe the relationship between the observations and the model state and parameters. Hence the Kalman type updates may cause severe bias in the posterior estimates. In this paper we investigate the effect of using a local linearization. Motivated by local Gaussian density estimation we describe ways to estimate the covariance locally in the state space resulting in local ensemble filters. This is not to be confused with covariance localization which often refers to localization in the physical space. The filters are tested on the three dimensional Lorenz model and a three layer petroleum reservoir type model.The results clearly show that using a Kalman update based on local covariances allows resolution of small scale structures in the posterior distribution that cannot be resolved using a global covariance matrix. In particular we show that the new local filters are able to approximate posterior distributions with multiple modes contrary to the ensemble Kalman filter.

Estimating Brownian motion dispersal rate, longevity and population density from spatially explicit mark–recapture data on tropical butterflies

1. We develop a Bayesian method for analysing mark-recapture data in continuous habitat using a model in which individuals movement paths are Brownian motions, life spans are exponentially distributed and capture events occur at given instants in time if individuals are within a certain attractive distance of the traps. 2. The joint posterior distribution of the dispersal rate, longevity, trap attraction distances and a number of latent variables representing the unobserved movement paths and time of death of all individuals is computed using Gibbs sampling. 3. An estimate of absolute local population density is obtained simply by dividing the Poisson counts of individuals captured at given points in time by the estimated total attraction area of all traps. Our approach for estimating population density in continuous habitat avoids the need to define an arbitrary effective trapping area that characterized previous mark-recapture methods in continuous habitat. 4. We applied our method to estimate spatial demography parameters in nine species of neotropical butterflies. Path analysis of interspecific variation in demographic parameters and mean wing length revealed a simple network of strong causation. Larger wing length increases dispersal rate, which in turn increases trap attraction distance. However, higher dispersal rate also decreases longevity, thus explaining the surprising observation of a negative correlation between wing length and longevity.

Estimating Snow Water Equivalent Using Snow Depth Data and Climate Classes

Abstract In many practical applications snow depth is known, but snow water equivalent (SWE) is needed as well. Measuring SWE takes ∼20 times as long as measuring depth, which in part is why depth measurements outnumber SWE measurements worldwide. Here a method of estimating snow bulk density is presented and then used to convert snow depth to SWE. The method is grounded in the fact that depth varies over a range that is many times greater than that of bulk density. Consequently, estimates derived from measured depths and modeled densities generally fall close to measured values of SWE. Knowledge of snow climate classes is used to improve the accuracy of the estimation procedure. A statistical model based on a Bayesian analysis of a set of 25 688 depth–density–SWE data collected in the United States, Canada, and Switzerland takes snow depth, day of the year, and the climate class of snow at a selected location from which it produces a local bulk density estimate. When converted to SWE and tested against two continental-scale datasets, 90% of the computed SWE values fell within ±8 cm of the measured values, with most estimates falling much closer.

Micro-spatial genetic structure in song sparrows (Melospiza melodia)

The spatial genetic structure of populations is strongly influenced by current and historical patterns of gene flow and drift, which in the simplest case, is limited by geographic distance. We examined the microspatial genetic structure within 33 populations of song sparrows (Melospiza melodia) which included eight subspecies located across coastal areas in southern British Columbia (BC) and California. We also examined the effect of water barriers and local density estimates on genetic structuring. Across both regions, positive genetic structure was detectable at distances of less than 10 km. Genetic divergence was highest in Californian subspecies, perhaps due to reduced gene flow across sub-specific contact zones. In BC, populations distributed across islands displayed greater genetic structuring over similar spatial scales than those across mainland sites, supporting the prediction that water barriers reduce gene flow in this species. Our results confirm both the expectation for fine-scale genetic structure in these generally sedentary subspecies, and the role of landscape features in generating geographic variation in genetic structure.

Multivariate density estimation using dimension reducing information and tail flattening transformations for truncated or censored data

This paper introduces a multivariate density estimator for truncated and censored data with special emphasis on extreme values based on survival analysis. A local constant density estimator is considered. We extend this estimator by means of tail flattening transformation, dimension reducing prior knowledge and a combination of both. The asymptotic theory is derived for the proposed estimators. It shows that the extensions might improve the performance of the density estimator when the transformation and the prior knowledge is not too far away from the true distribution. A simulation study shows that the density estimator based on tail flattening transformation and prior knowledge substantially outperforms the one without prior knowledge, and therefore confirms the asymptotic results. The proposed estimators are illustrated and compared in a data study of fire insurance claims.

A method to estimate roe deer Capreolus capreolus density at various spatial scales in a fragmented landscape

AbstractEstimating the density of wildlife populations is still a difficult task, especially when you work with spatially open populations and you must relax the assumption of closure, which is the basis of most methods currently used. Further difficulties arise when obtaining density estimates at small spatial scales. Using eight years (1996‐2003) to monitor data from a roe deer Capreolus capreolus population that lives in a sub‐Mediterranean environment in central Italy, we were able to estimate local density (at a spatial scale of one home range) by using a large sample of radio‐marked animals. Local density estimates could be obtained only in zones in which radio‐marked deer were available in sufficient numbers. To estimate local density in the whole study area, we developed a calibration model, which allowed us to infer density where radio‐marked deer were absent or scarce. To do this, we computed the mark‐resight density estimates (using radio‐marked animals) and related these estimates to linear and non‐linear functions of animal count and surface area of fields, to obtain a set of density estimators. Then, we selected a linear combination of such estimators, whose quality was assessed by cross‐validation. Our results show that the method we propose can be effective in investigating small‐scale spatial structure of density in a roe deer population. We see several potential applications of this method for both research and management purposes.

Multivariate Density Estimation Using Dimension Reducing Information and Tail Flattening Transformations for Truncated or Censored Data

This paper introduces a multivariate density estimator for truncated and censored data with special emphasis on extreme values based on survival analysis. A local constant density estimator is considered. We extend this estimator by means of tail flattening transformation, dimension reducing prior knowledge and a combination of both. The asymptotic theory is derived for the proposed estimators. It shows that the extensions might improve the performance of the density estimator when the transformation and the prior knowledge is not too far away from the true distribution. A simulation study shows that the density estimator based on tail flattening transformation and prior knowledge substantially outperforms the one without prior knowledge, and therefore confirms the asymptotic results. The proposed estimators are illustrated and comparedin a data study of fire insurance claims.

Local Likelihood Density Estimation and Value‐at‐Risk

This paper presents a new nonparametric method for computing the conditional Value‐at‐Risk, based on a local approximation of the conditional density function in a neighborhood of a predetermined extreme value for univariate and multivariate series of portfolio returns. For illustration, the method is applied to intraday VaR estimation on portfolios of two stocks traded on the Toronto Stock Exchange. The performance of the new VaR computation method is compared to the historical simulation, variance‐covariance, and J. P. Morgan methods.

Measurement of Steady‐State Soil Water Flux across a Soil Horizon Interface

Soil horizon interfaces have been shown to be focal points for localized, three‐dimensional redistribution of water and solutes in field soils. Therefore, understanding of the physics of water flow and transport in layered soils requires experimental observations of the magnitude and variability of local soil water flux under a variety of well‐defined boundary conditions. We developed a time domain reflectometry (TDR) method to measure the spatial pattern of steady‐state, local soil water flux density above and below a soil horizon interface under quasi‐steady surface water application and implemented it in laboratory and field experiments. Time series of TDR‐measured bulk soil electrical conductivity and TDR‐measured soil volumetric water content at each location are used to quantify the solute travel time under steady‐state flow conditions, which is then used to quantify steady‐state, local soil water flux density. Results from laboratory and field experiments showed that the proposed methodology yielded local soil water flux density estimates that were, on average, 104% of the applied surface water flux density (i.e., mass recovery = 104%), which is consistent with transient, local soil water flux density estimates from a previous study. For the field experiments, steady‐state, local soil water flux density estimates above and below an A/B horizon interface (measured across the length of a 6.75‐m transect) were negatively correlated to each other. The strength of the negative correlation, however, decreased with increasing surface water application rate, suggesting that the hydrologic response of the soil horizon interface is flux‐dependent. This flux‐dependent correlation between A and B horizon steady‐state soil water flux density is probably attributable to the spatial covariance between A and B horizon soil hydraulic properties.

Nonparametric density estimation for positive time series

The Gaussian kernel density estimator is known to have substantial problems for bounded random variables with high density at the boundaries. For independent and identically distributed data, several solutions have been put forward to solve this boundary problem. In this paper, we propose the gamma kernel estimator as a density estimator for positive time series data from a stationary α -mixing process. We derive the mean (integrated) squared error and asymptotic normality. In a Monte Carlo simulation, we generate data from an autoregressive conditional duration model and a stochastic volatility model. We study the local and global behavior of the estimator and we find that the gamma kernel estimator outperforms the local linear density estimator and the Gaussian kernel estimator based on log-transformed data. We also illustrate the good performance of the h -block cross-validation method as a bandwidth selection procedure. An application to data from financial transaction durations and realized volatility is provided.

Local winter deer density: Effects of forest structure and snow in a managed forest landscape

Abstract Background/Questions/Methods White-tailed deer (Odocoileus virginianus) are a ‘keystone herbivore’ with the potential to cause tree regeneration failure and greatly affect vegetation dynamics, stand structure and ecological function of forests across eastern North America. In northern mixed conifer-hardwood forests, local winter-time deer populations are dependent on habitat characterized by patterns of forest cover that provide shelter from snow and cold temperatures (lowland conifer stands) in close proximity to winter food (deciduous hardwood stands). Stand structure may also influence winter spatial deer distribution. Consequently, modification of forest cover patterns and stand structure by timber harvesting will affect local spatial deer distributions, with potential ecological and economic consequences. Here, we ask if forest cover pattern and stand structure, and their interactions with snow depth, can explain winter deer density in the managed forests of the central Upper Peninsula of Michigan, USA. For each local winter deer density estimate (from fecal pellet counts) we calculate stand-level characteristics for surrounding ‘landscapes of influence’ of radius 200 m. For these data, and modeled snow depth estimates, we use multivariate techniques to produce predictive models and to identify the most important factors driving local deer densities across our 400,000 ha study area. Results/Conclusions Distance to the nearest conifer stand consistently explains the most variance in univariate regression models. Deer densities are highest near lowland conifer stands in areas where mean hardwood tree diameter-at-breast-height is low. Multiple regression models including these factors explain 22% of variance in deer density and have up to a 65% chance of correctly ranking a site’s deer density (relative to other sites within our study area). We use model ensembles to produce maps of estimated deer density (and associated uncertainty) for a subset of our study area and show how managers might use these maps to aid co-management of deer and forest regeneration. Our results highlight the importance of local and regional factors (forest cover-type pattern, stand structure, climate) on winter white-tailed deer density in managed hardwood-conifer forests. Use of these results, and the simulation model being developed, will help identify management practices that can decrease deer impacts and ensure the ecological and economic sustainability of forests in which deer browse is proving problematic for tree regeneration.

Probabilistic learning for fully automatic face recognition across pose

Recent pose invariant methods try to model the subject specific appearance change across pose. For this, however, almost all of the existing methods require a perfect alignment between a gallery and a probe image. In this paper we present a pose invariant face recognition method that does not require the facial landmarks to be detected as such and is able to work with only single training image of the subject. We propose novel extensions by introducing to use a more robust feature description as opposed to pixel-based appearances. Using such features we put forward to synthesize the non-frontal views to frontal. Furthermore, using local kernel density estimation, instead of commonly used normal density assumption, is suggested to derive the prior models. Our method does not require any strict alignment between gallery and probe images which makes it particularly attractive as compared to the existing state of the art methods. Improved recognition across a wide range of poses has been achieved using these extensions.

A Novel Criterion for Writer Enrolment Based on a Time-Normalized Signature Sample Entropy Measure

This paper proposes a novel criterion for an improved writer enrolment based on an entropy measure for online genuine signatures. As online signature is a temporal signal, we measure the time-normalized entropy of each genuine signature, namely, its average entropy per second. Entropy is computed locally, on portions of a genuine signature, based on local density estimation by a Client-Hidden Markov Model. The average time-normalized entropy computed on a set of genuine signatures allows then categorizing writers in an unsupervised way, using a K-Means algorithm. Linearly separable and visually coherent classes of writers are obtained on MCYT-100 database and on a subset of BioSecure DS2 containing 104 persons (DS2-104). These categories can be analyzed in terms of variability and complexity measures that we have defined in this work. Moreover, as each category can be associated with a signature prototype inherited from the K-Means procedure, we can generalize the writer categorization process on the large subset DS2-382 from the same DS2 database, containing 382 persons. Performance assessment shows that one category of signatures is significantly more reliable in the recognition phase, and given the fact that our categorization can be used online, we propose a novel criterion for enhanced writer enrolment.

ASYMPTOTIC THEORY FOR LOCAL TIME DENSITY ESTIMATION AND NONPARAMETRIC COINTEGRATING REGRESSION

Asymptotic theory is developed for local time density estimation for a general class of functionals of integrated and fractionally integrated time series. The main result provides a convenient basis for developing a limit theory for nonparametric cointegrating regression and nonstationary autoregression. The treatment directly involves local time estimation and the density function of the processes under consideration, providing an alternative approach to the Markov chain and Fourier integral methods that have been used in other recent work on these problems.