Nonparametric Estimation Method Research Articles

Quantitative Assessment of Ecological Flow in the Yellow River Under Changing Environments

ABSTRACTStudying the streamflow characteristics of the Yellow River mainstem under changing environmental conditions is crucial for the management and sustainable development of water resources within its basin. This research employs a long short‐term memory (LSTM) model to restore the flow characteristics of the Yellow River's mainstream under natural conditions. Additionally, the range of variation approach (RVA) and nonparametric kernel density estimation (KDE) method are integrated to quantitatively assess the impact of environmental changes on streamflow. The findings indicate that: (1) Hydrological variability in the Yellow River was observed in 1985, with a degree of variability ranging from 26% to 58%, classified as moderate. (2) The annual ecological flow value of the Yellow River is 560–1001 m3/s, and the average annual ecological flow assurance is 43%. (3) Based on LSTM simulation results (NSE > 0.7, R2 > 0.8), it is concluded that the ecological flow assurance under natural conditions in the Yellow River exceeds the measured values, primarily due to human activities, which contribute over 52% to this discrepancy. These results suggest that the river ecosystem of the Yellow River's mainstem is relatively unstable and requires further management.

Identifying the number of latent factors of stochastic volatility models

AbstractWe provide a procedure to identify the number of latent factors of stochastic volatility models. The methodology relies on the non-parametric Fourier estimation method introduced by Malliavin and Mancino (Finance Stoch 4:49–61, 2002) and applies to high-frequency data. Based on the Fourier analysis, we first estimate the latent volatility process and then the volatilities and covariances of the processes that are gradually identified, such as volatility of volatility and leverage. The analysis of the eigenvalue spectrum of the Gram matrix can reveal information about the actual number of factors driving the process at hand. We corroborate our analysis by numerical simulations on single and multi factor models. Finally, we apply our methodology to intraday prices from the S &P 500 index futures.

Distribution reconstruction and reliability assessment of complex LSFs via an adaptive Non-parametric Density Estimation Method

Complex limit state functions (LSFs), often characterized by strong nonlinearity, non-smoothness, or discontinuity, pose challenges for structural reliability analysis in engineering practices. Conventional methods for uncertainty propagation and reliability assessment may struggle to handle these issues effectively. This paper introduces a novel approach to adaptively reconstruct the unknown distributions of complex LSFs. The Non-parametric Density Estimation Method based on Harmonic Transform (NDEM-HT) is employed as the tool for this purpose. An adaptive strategy is then proposed to determine the number of harmonic moments required in NDEM-HT for achieving high accuracy. Specifically, the Adaptive Kernel Density Estimation (AKDE) method is also adopted to provide an initial estimation of the rough distribution. Subsequently, the optimal number of harmonic moments is determined by minimizing the relative entropy between the distributions obtained by AKDE and NDEM-HT. The efficacy of the proposed method is demonstrated through five numerical examples, considering various types of complex LSFs. Comparative results are also provided employing MCS along with both conventional and state-of-the-art methods.

Efficient Nonparametric Estimation of 3D Point Cloud Signals through Distributed Learning

Advancements in technology have elevated the prominence of 3D point cloud data, making its analysis increasingly vital across various applications. This need drives the demand for advanced statistical analytic approaches to handle challenges such as size, sparsity, and irregularity in 3D point clouds while ensuring accurate and efficient information extraction. This paper introduces a novel nonparametric distributed (NPD) learning framework that utilizes trivariate spline smoothing over a triangulation of domain. The proposed NPD algorithm features a straightforward, scalable, and communication-efficient implementation scheme that can achieve near-linear speedup. In addition, we provide rigorous theoretical support for the NPD estimation framework and demonstrate that the NPD spline estimators attain the same convergence rate as the global spline estimators obtained using the entire dataset and achieve the optimal nonparametric convergence rate established by Stone (1982) under some regularity conditions. To evaluate the efficacy of the proposed NPD method, we conduct simulation studies comparing it with several global nonparametric estimation methods used to smooth the 3D data. The results demonstrate the superior performance of the NPD method in accurately and efficiently processing and learning from 3D point clouds, highlighting its potential to advance large and complex data analysis. Supplementary material, which contains related technical details, proofs of the theoretical results, and additional results in simulation studies, is available online.

Girth sizes of the body in Ukrainian men and women with various forms of urticaria

The relevance of studying body girth in patients with different forms of urticaria is due to the importance of anthropometric indicators in the diagnosis and prognosis of the disease. Urticaria, as one of the most common dermatological conditions, can present in various clinical forms, each of which may differently impact the body, particularly by altering body circumferences. Examining these indicators in Ukrainian men and women allows for the identification of gender-specific and individual characteristics, which could be crucial for improving treatment and prevention methods for urticaria. The purpose of the work is to establish the peculiarities and sex differences in body girth measurements in young Ukrainian men and women with acute and chronic mild and severe urticaria. Body girth measurements were determined in 40 Ukrainian men and 40 young women (25-44 years according to the age periodization of WHO, 2015) patients with acute (AU) and chronic (CU) urticaria of a mild (MU) and severe (SU) course. Urticaria was diagnosed according to the EAACI/GA²LEN/EuroGuiDerm/APAAACI international guidelines. Anthropometric measurements were performed on the right side of the body. The girth sizes of the upper and lower limbs, neck and trunk were determined with a centimeter tape. As a control group, the girth measurements of the body of practically healthy Ukrainian men (n=82) and women (n=101) of young age were used, which were taken from the data bank of the research center of the National Pirogov Memorial Medical University, Vinnytsya. Statistical analysis of girth body dimensions was carried out in the license package "Statistica 6.0" using non-parametric estimation methods. When comparing the body girths of men or women with urticaria with practically healthy men or women, significantly larger values or a tendency towards larger values were established: in sick men, most of the girths of the upper limb (mainly in patients with AU/SU), hips, waist, chest during inhalation and at rest (in patients with AU/MU, AU/SU, CU/MU and CU/SU), neck (in patients with AU/MU, AU/SU and CU/MU); in sick women – shoulder girths in stressed and relaxed states, hips, thighs, neck, waist, chest during inhalation, exhalation and in a calm state (in patients with AU/MU, AU/SU, CU/MU and CU/SU ), forearm in the lower part (in patients with AU/MU and CU/MU), crus in the upper and lower part (in patients with AU/MU and CU/MU). When comparing the girths of the body between men and women with urticaria, only a few significant differences or trends were established: between the men, only larger values of the girths of the neck (in patients with AU/MU compared to AU/SU) and feet (in patients with AU/SU vs. CU/SU); between sick women – greater values of shoulder girths in tense and relaxed states, forearm in the lower part, hand, crus in the upper part, foot, thorax on exhalation and in a calm state (in patients with AU/MU compared to AU/SU), crus (in patients with CU/MU compared to CU/SU). Pronounced manifestations of sexual dimorphism of body girths between Ukrainian men and women with urticaria were also established: in sick men, larger values of hand and neck girths were found (in patients with AU/MU, AU/SU, CU/MU and CU/SU), shoulder girths in tense and relaxed states, forearms in the upper and lower part, feet, waist, chest during inhalation, exhalation and in a calm state (in patients with AU/SU), shoulder in a tense state (in patients with CU/SU), forearm in the upper part (in patients with AU/MU), forearm in the lower part (in patients with CU/MU); in female patients – larger values of hip girth (in patients with AU/MU) and hips (in patients with CU/MU).

Irregular surface temperature monitoring in liver procurement via time-vertex signal processing

The liver viability monitoring during its procurement is critical to guarantee the safe liver transportation. Traditionally, the viability is assessed by taking invasive biopsy on the liver surface. Recently, the noninvasive thermal images of liver surface have been used as an alternative assessment way. Researchers have proposed monitoring and classification approaches based on the thermal images. Existing works have demonstrated the importance of the temporal variation or the spatial variation of the thermal images to the viability monitoring. However, there is no prior work to leverage the graph structure of the spatio-temporal processes for the viability monitoring. In this paper, we propose a time-vertex signal processing framework for the irregular thermal data of the pure liver region monitoring. In particular, we extract features of irregular thermal data based on the joint Fourier transform (JFT), which is an integration of graph Fourier transform (GFT) and discrete Fourier transform (DFT). The extracted JFT features can accurately reconstruct the thermal images with a limited number of features. Then, we use a non-parametric online change-point estimation method, based on online scan B statistics, to monitor the JFT features without a parametric distribution. Our proposed framework is applied to both simulation and the liver procurement data, and achieves good monitoring performance.

Spatial non-equilibrium and distribution dynamic evolution of the development level of national physical fitness in China's provinces.

Physical health is fundamental to a country's socio-economic advancement. An in-depth exploration of the spatial distribution and dynamic evolution of national physical fitness across China is crucial for enhancing the country's overall physical health. This study aims to provide valuable insights into the geographical and temporal patterns of physical fitness, informing strategies for national physical fitness improvement. Employing data from China's national physical fitness monitoring of 31 provinces, cities, and districts for the years 2005, 2010, and 2015, this study utilizes the Gini coefficient, its decomposition, and nonparametric density estimation methods. These techniques are applied to analyze the spatial disparities and temporal trends in national physical fitness levels among different demographics, including the overall population, males, females, and urban and rural residents. The study reveals that the regional disparity in China's national physical fitness initially narrowed and then expanded across the general population, females, and in both urban and rural areas, while consistently increasing among males. In terms of geographical distribution, the East, Middle, and West regions show significant heterogeneity, with the East-West gap being the most pronounced (Gini coefficients of 0.0249, 0.0230, 0.0263). The contribution rate of regional gaps was highest (54.40% -64.69%), followed by regional disparities (24.78% -27.15%), and the contribution of hypervariable density difference was the smallest (10.53% -19.75%). Despite a slight improvement in overall national physical fitness, the absolute regional disparities have further widened. Provinces with lower levels of national physical fitness demonstrate a 'club convergence' trend, indicating regional clustering of similar fitness levels. Additionally, a 'catch-up effect' is evident in rural areas, particularly in provinces with historically lower levels of national physical fitness. These findings suggest the need for region-specific public health strategies to address the growing disparities in national physical fitness across China.

Effectiveness and efficiency of urban water access in the Democratic Republic of Congo: A panel directional approach

In the 2030 Sustainable Development Goals (SDG) of the United Nations, countries are committed to achieving universal and equitable access to safe and affordable drinking water for all. The Democratic Republic of Congo is a paradox as it owns the second-largest basin in the world while more than half of the population has no access to basic drinking water. This fact is our starting point to conduct a performance evaluation exercise of the 11 provinces from 2008 to 2019. Our approach has five distinguished features: we take account of population trends; we use a tailored and complete database of urban centres; we define and decompose flexible indicators, and we use a non-parametric estimation method. Our results show that there is inefficiency and ineffectiveness in urban water access. Overall, larger efficiency–effectiveness differences are over time observed mostly due to a lack of technological change and a resource constraint. We also highlight the role of public policies.

Learning real-world heterogeneous noise models with a benchmark dataset

Noise modeling is an important research field in computer vision; the design of an accurate model for imaging sensor noise depends on not only a comprehensive benchmark dataset of the real world, but also a precise design of the noise modeling algorithm. However, due to the inaccurate estimation method of noise-free images and limited shooting scenes, the current realistic datasets could not describe the diverse noise properties sufficiently. Moreover, popular parametric noise models are not sophisticated enough to characterize the real-world noise exactly. In this work, we first construct a more comprehensive dataset of the real world by capturing more indoor and outdoor scenes under different lighting conditions using diverse smartphones, then we propose a non-parametric noise estimation method capable of modeling the spatial heterogeneity of real-world noise patterns. Specifically, in order to characterize the spatial heterogeneity of real-world noise, we assume a non-i.i.d Gaussian distribution and propose a deep convolutional neural network (DCNN)-based approach for learning pixel-wise noise variance maps. To learn the pixel-wise variance map, we have constructed a variance estimation network mapping from the conditional signals (clean image, ISO, and camera model) to surrogate labels obtained from the nonlocal search of similar patches from the clean-noisy image pair. Finally, we conducted denoising and classification experiments using different kinds of simulated noisy images, compared to the Poisson-Gaussian and Noise Flow noise models, the proposed method achieves denoising performance improvements (PSNR) of 1.13 dB and 2.51 dB respectively on the proposed real-world test dataset, denoising and classification results on the real noisy data captured by mobile phones have verified that our approach is more accurate than current noise modeling methods.

Nonparametric receiver operating characteristic curve analysis with an imperfect gold standard.

This article addresses the challenge of estimating receiver operating characteristic (ROC) curves and the areas under these curves (AUC) in the context of an imperfect gold standard, a common issue in diagnostic accuracy studies. We delve into the nonparametric identification and estimation of ROC curves and AUCs when the reference standard for disease status is prone to error. Our approach hinges on the known or estimable accuracy of this imperfect reference standard and the conditional independent assumption, under which we demonstrate the identifiability of ROC curves and propose a nonparametric estimation method. In cases where the accuracy of the imperfect reference standard remains unknown, we establish that while ROC curves are unidentifiable, the sign of the difference between two AUCs is identifiable. This insight leads us to develop a hypothesis-testing method for assessing the relative superiority of AUCs. Compared to the existing methods, the proposed methods are nonparametric so that they do not rely on the parametric model assumptions. In addition, they are applicable to both the ROC/AUC analysis of continuous biomarkers and the AUC analysis of ordinal biomarkers. Our theoretical results and simulation studies validate the proposed methods, which we further illustrate through application in two real-world diagnostic studies.

FFT-Based Probability Density Imaging of Euler Solutions.

When using traditional Euler deconvolution optimization strategies, it is difficult to distinguish between anomalies and their corresponding Euler tails (those solutions are often distributed outside the anomaly source, forming "tail"-shaped spurious solutions, i.e., misplaced Euler solutions, which must be removed or marked) with only the structural index. The nonparametric estimation method based on the normalized B-spline probability density (BSS) is used to separate the Euler solution clusters and mark different anomaly sources according to the similarity and density characteristics of the Euler solutions. For display purposes, the BSS needs to map the samples onto the estimation grid at the points where density will be estimated in order to obtain the probability density distribution. However, if the size of the samples or the estimation grid is too large, this process can lead to high levels of memory consumption and excessive computation times. To address this issue, a fast linear binning approximation algorithm is introduced in the BSS to speed up the computation process and save time. Subsequently, the sample data are quickly projected onto the estimation grid to facilitate the discrete convolution between the grid and the density function using a fast Fourier transform. A method involving multivariate B-spline probability density estimation based on the FFT (BSSFFT), in conjunction with fast linear binning appropriation, is proposed in this paper. The results of two random normal distributions show the correctness of the BSS and BSSFFT algorithms, which is verified via a comparison with the true probability density function (pdf) and Gaussian kernel smoothing estimation algorithms. Then, the Euler solutions of the two synthetic models are analyzed using the BSS and BSSFFT algorithms. The results are consistent with their theoretical values, which verify their correctness regarding Euler solutions. Finally, the BSSFFT is applied to Bishop 5X data, and the numerical results show that the comprehensive analysis of the 3D probability density distributions using the BSSFFT algorithm, derived from the Euler solution subset of x0,y0,z0, can effectively separate and locate adjacent anomaly sources, demonstrating strong adaptability.

Uncertainty impacts on China’s agricultural commodity futures: a quantile perspective

ABSTRACT This study employs rolling quantile regression and non-parametric estimation methods, specifically Quantile-on-Quantile (QQ) and Causality-in-Quantiles (QC), to investigate the impact of uncertainty indices (Geopolitical Uncertainty (GPR), Climate Policy Uncertainty (CPU), and Economic Policy Uncertainty (EPU)) on China’s agricultural commodity futures. The rolling window quantile regression coefficients vary over time, with CPU having the most significant impact on agricultural commodity futures. The empirical results from the non-parametric estimation QQ method reveal when agricultural commodities are in a prosperous market, CPU and EPU exhibit relatively strong correlation, while GPR shows weaker or insignificant correlation. According to the non-parametric estimation QC method, the evidence suggests that the second-order (variance) causality relationship is relatively stronger than the first-order (mean) causality relationship, and the first-order mean causal relationship between CPU and corn and between EPU and wheat is not significant across the entire range of quantiles. The results on Hedging Effectiveness reveal that the efficiency of uncertainty hedging is substantiated. The hedging effectiveness of EPU exhibits improvement post-COVID-19, whereas the effectiveness of GPR and CPU is more pronounced in the pre-COVID-19 period.

Dynamic evolution and spatial spillover effect of agricultural green development on eight economic regions in China

Promoting the green development of agriculture is of great significance to realize agricultural and rural modernization in China. Based on the existing research, this paper innovatively explores the dynamic and spatial effects of agricultural green development in the eight newly zoned regions of China's economy. Based on the panel data of 30 provinces in China from 2013 to 2022, this paper selects 20 indicators to measure the level of agricultural green development from five dimensions such as ecological protection, resource conservation, environment-friendly, green supply and economic growth by entropy weight method and uses non-parametric estimation method to analyze the dynamic evolution trend of agricultural green development in the whole country and its eight economic regions. Then, a spatial econometric model is constructed to further explore the influence mechanism and spatial spillover effect of each influencing factor on agricultural green development. The findings demonstrate that the level of agricultural green development in 30 provinces of China continuously improved during the study period, but the dynamic evolution trend characteristics in the whole country and its eight economic regions are not the same. Specifically, the development differences between the whole country, the northeast region, the eastern coast, the southern coast and the northwest region increased, while that between the northern coast, the Yellow River basin and the middle reaches of the Yangtze River first increased and then decreased, and that in the southwestern region gradually narrowed. There is a significant spatial spillover effect on agricultural green development and its influencing factors. Moreover, there is heterogeneity in the influence characteristics and spatial spillover effects of various influencing factors on agricultural green development among the eight economic regions. Therefore, it is proposed that eight economic regions in China should formulate differentiated development strategies, focus on educational and technological innovation etc., and further promote agricultural green development.

Double hedonic price-characteristics frontier estimation for IoT service providers in the industry 5.0 era: A nonconvex perspective accommodating ratios

The advent of advanced digital technologies, including the Internet of Things (IoT), image processing, artificial intelligence (AI), blockchain, robotics and cognitive computing that have been embedded in Industry 5.0, is considerably improving the sustainability, resilience, and human-centric performance of industrial organizations. Despite the increasing use of Industry 5.0 technologies in smart product platforming in industrial organizations, a critical issue remains how to assess the providers/suppliers of such technologies in highly competitive markets to fulfil personalized products and services. Following Lancaster's characteristics approach to consumer theory, in this study we contribute to assess digital technologies service providers in the Industry 5.0 era by focusing on both theoretical and empirical evidence inquiring about the convexity of conventional nonparametric frontier estimation methods. To do so, a nonparametric double frontier estimation of the hedonic price characteristics relation is developed from both the buyer's and seller's perspectives. Moreover, a separable directional distance function-based optimization model is developed for the efficiency estimation. Furthermore, a comparable estimation of the convex and nonconvex hedonic price function is proposed. We also explicitly test the impact of convexity in evaluating the efficiency of IoT service providers in the Industry 5.0 context. In this study, we also show that the hypothesis of convexity in assessing the efficiency of IoT service providers is rejected using the Li-test comparing entire densities in the case of the seller's perspective without ratio data. Differences are less pronounced for the buyer's perspective and in the case with ratio data.

Stock market prices and Dividends in the US: Bubbles or Long-run equilibria relationships?

This paper presents a novel approach to identifying potential bubbles in the US stock market by employing alternative time series methods based on long memory, including fractional integration and cointegration, as well as duration dependence non-parametric models. To test for duration dependence, the paper employs a unique non-parametric hazard function estimation method, using monotonic P-splines, which enables the estimation of a coherent and sophisticated Bayesian confidence interval for the function. By analyzing historical data of real US stock index from January 1871 to November 2022, the results reveal strong evidence of bubbles in the US stock market. It is worth noting that this is the first time a fractional integration approach and a novel non-parametric hazard function have been utilized to test for rational speculative bubbles in the US stock market, making this paper a significant contribution to the field.

The effects of manager sentiment in financial disclosure: Perspectives of operational efficiency and market reaction

This study constructs a measure of managerial optimism using textual and nonparametric estimation methods, exploring its impact on company operational efficiency and market reactions. Findings indicate that managerial optimism positively influences stock price returns over the subsequent year by enhancing internal operational efficiency and reducing analyst forecast dispersion, with a more pronounced effect in state-owned enterprises. Utilizing information from the MD&A (Manager Discussion and Analysis) section of annual reports, this research demonstrates the role of managerial optimism in company operations and market reactions, offering empirical evidence for corporate behavioral finance research and practical implications for management practice.

Linking Clicks to Bricks: Understanding the Effects of Email Advertising on Multichannel Sales

Businesses have widely used email ads to directly send promotional information to consumers. Whereas email ads serve as a convenient tool that allows firms to target consumers online, there is little evidence of their multichannel impact on consumer spending in both online and brick-and-mortar stores. In this paper, we utilize a unique high-dimensional data set from one of the world’s largest office supplies retailers to link each consumer’s online behaviors to item-level purchase records in physical stores. We employ a doubly robust estimator that incorporates nonparametric machine learning methods for causal estimation of observational data. Our results show that email ads significantly increase the retailer’s sales across different channels. We also investigate the effects of email ads on diverse consumer behaviors along the purchase funnel and find that increased sales result from increased purchase probability and a wider variety of products purchased by consumers. Further, we examine several moderating factors, such as product types and consumer segments, that influence the multichannel effects of email advertising. Our study provides empirical evidence for the economic impact of email ads on consumer behavior across different channels and the underlying mechanisms thereof, offering direct implications for multichannel retailers seeking to improve their digital marketing strategies.

Nonparametric estimation of nonautonomous uncertain differential equations with application to temperature models

In recent years, there has been a great development in parameter estimation methods for uncertain differential equations (UDEs). However, the observations we can obtain in real life are limited, in which case the form of function in a UDE is unknown. When dealing with such UDEs, we may use observational data to make nonparametric estimates. There are many nonautonomous systems in real life, and nonautonomous UDEs can simulate some uncertain nonautonomous dynamical systems well. In this paper, a nonparametric estimation method based on the nonautonomous UDEs of the binary Legendre polynomial is proposed. Then, three numerical examples are given to verify the reliability of nonparametric estimation. As an application, a real data example of global average monthly temperatures is used to illustrate the effectiveness of our method.

Mean Square Error Estimation of Small Area Predictors by Use of Parametric and Nonparametric Bootstrap

In this article, we propose and compare some old and new parametric and nonparametric bootstrap methods for MSE estimation in small area estimation, restricting to the case of the widely used Fay-Herriot model. The parametric method consists of generating parametrically a large number of area bootstrap samples from the model fitted to the original data, re-estimating the model parameters for each bootstrap sample and then estimating the separate components of the MSE. The use of double-bootstrap is also considered. The nonparametric method generates the samples by bootstrapping standardized residuals, estimated from the original sample data. The bootstrap procedures are compared to other methods proposed in the literature in a simulation study, which also examines the robustness of the various methods to non-normality of the model error terms. A design-based MSE estimator for the Fay-Herriot model-dependent predictor is also described and its performance is investigated in a separate simulation study. AMS subject classification: 62F10, 62F40

Maximum likelihood estimation of log-concave densities on tree space

Phylogenetic trees are key data objects in biology, and the method of phylogenetic reconstruction has been highly developed. The space of phylogenetic trees is a nonpositively curved metric space. Recently, statistical methods to analyze samples of trees on this space are being developed utilizing this property. Meanwhile, in Euclidean space, the log-concave maximum likelihood method has emerged as a new nonparametric method for probability density estimation. In this paper, we derive a sufficient condition for the existence and uniqueness of the log-concave maximum likelihood estimator on tree space. We also propose an estimation algorithm for one and two dimensions. Since various factors affect the inferred trees, it is difficult to specify the distribution of a sample of trees. The class of log-concave densities is nonparametric, and yet the estimation can be conducted by the maximum likelihood method without selecting hyperparameters. We compare the estimation performance with a previously developed kernel density estimator numerically. In our examples where the true density is log-concave, we demonstrate that our estimator has a smaller integrated squared error when the sample size is large. We also conduct numerical experiments of clustering using the Expectation-Maximization algorithm and compare the results with k-means++ clustering using Fréchet mean.