Nonparametric Modeling Methods Research Articles

A nonparametric degradation modeling method for remaining useful life prediction with fragment data

Condition-based maintenance (CBM) is an effective way to keep the safety of industrial equipment by predicting the remaining useful life (RUL) and scheduling the maintenance plan before failure. Condition monitoring data is the basis of health state evaluation and RUL prediction. In ideal cases, the monitoring data are collected consecutively starting from the healthy stage until the end of the lifetime. In real industrial cases, however, fragment data often exist due to the interruption of monitoring and/or the loss of sensor readings. The major characteristic of fragment data is that they only record a random period of degradation process. The initial degradation time information is generally lost. Therefore, it is unable to be modeled using the common time-dependent modeling framework. To deal with the above issue, this paper proposes a nonparametric degradation modeling method for RUL prediction with fragment data. In this method, a new state-dependent degradation modeling framework is constructed via two-step axis transform. It formulates the RUL using a function depending on the health state. Based on functional principle component analysis (FPCA), a principal analysis via maximum likelihood estimation (PAMLE) algorithm is developed to recover the missing data of failed units. In addition, a RUL prediction-oriented optimization (POO) algorithm is proposed to predict the RUL of in-service units based on a piece of fragment data. Consequently, the proposed method is capable of dealing with the issues of both data recovery and RUL prediction. The effectiveness of the method is demonstrated using a fatigue crack growth dataset and a lithium-ion (Li-ion) battery degradation dataset.

Adaptive Ensemble of Multi-Kernel Gaussian Process Regressions Based on Heuristic Model Screening for Nonparametric Modeling of Ship Maneuvering Motion

Abstract Gaussian process regression (GPR) is a commonly used approach for establishing the nonparametric models of ship maneuvering motion, and its performance depends on the selection of the kernel function. However, no single kernel function can be universally applied to all nonparametric models of ship maneuvering motion, which may compromise the robustness of GPR. To address this issue, an adaptive ensemble of multi-kernel GPRs based on heuristic model screening (AEGPR-HMS) is proposed in this paper. In the proposed method, four kernel functions are involved in constructing the ensemble model. The HMS method is introduced to determine the weights of individual-based GPR models, which can be adaptively assigned according to the baseline GPR model. To determine the hyper-parameters of these kernel functions, the genetic algorithm is also employed to compute the optimal values. The KVLCC2 tanker provided by the SIMMAN 2008 workshop is used to validate the performance of the proposed method. The results demonstrate that the AEGPR-HMS is an efficient and robust method for nonparametric modeling of ship maneuvering motion.

Nonparametric modeling of a high-speed USV at three speed regions based on Gaussian process regression with a hybrid kernel function

Due to the complexity and variability of the ocean environment, it is difficult for high-speed unmanned surface vehicles (USVs) to construct a unified mathematical model for nonlinear motion under different Froude numbers. In this paper, a nonparametric modeling method for high-speed USVs at three speed regions is presented based on Gaussian process regression with a hybrid kernel function (HKF-GPR). First, the hybrid kernel functions are designed based on the forecasting performance of Gaussian process regression with different single kernel functions (SKF-GPR) using sea trial speed data from the “Jiuhang 750” USV. Second, to better capture the dynamic performance of the USVs, the sea trial speed data are clustered into three speed regions by an improved k-means++ algorithm; these regions are the displacement, semiplanning, and planning regions. Finally, USV modeling and forecasting are performed in the three speed regions using HKF-GPR. The trained model shows good modeling accuracy and forecasting, verifying the effectiveness of the proposed method.

Improved PER-DDPG based nonparametric modeling of ship dynamics with uncertainty

This study contributes to addressing the challenge of quickly obtaining an effective and accurate nonparametric model for describing ship maneuvering motion in three degrees of freedom (3-DOF). To achieve this, an intelligent ship dynamics nonparametric modeling method named improved PER-DDPG is proposed. This method leverages the deep deterministic policy gradient algorithm (DDPG) and prioritized experience replay mechanism (PER) and analyzes the characteristics between the goal of deep reinforcement learning (DRL) and the modeling process of the nonparametric model. The PER mechanism is utilized to enhance the agent’s understanding of the overall mechanism of ship motion by improving the utilization of samples. The meaning of target value is redefined due to transforming DRL aiming at maximizing cumulative rewards into maximizing the set of immediate rewards at each time step. To validate the performance of the proposed modeling method, we conduct simulation studies using a benchmark ship model i.e., a Mariner cargo ship dynamic model, and experimental studies using a real unmanned surface vehicle (USV). In the simulation test, we demonstrate the effectiveness and generalization of the proposed method through zigzag and turning circle tests. Furthermore, we verify the robustness and applicability of the proposed method by using datasets with uncertain environmental disturbances and datasets with different sampling frequencies. Additionally, the experimental tests conducted on the USV indicate the consistency of the proposed approach.

Nonparametric modeling of ship maneuvering motions in calm water and regular waves based on R-LSTM hybrid method

It is essential to establish reliable method to predict the ship maneuvering motions in complex navigation environments for navigation efficiency and safety. In this paper, a novel hybrid nonparametric modeling method (R-LSTM) is developed by coupling Bidirectional Long Short-Term Memory (Bi-LSTM) with residual network and attention mechanism. Firstly, the prediction capability of the proposed hybrid method is evaluated in the predictions of zigzag maneuvers in calm water. The model test data of a KVLCC2 tanker model are utilized for modeling and validation, and the optimal parameters are determined by Tree-structured Parzen Estimator algorithm. The results demonstrate that the proposed method has higher prediction accuracy and better generalization ability than the classical LSTM method. On this basis, the online modeling method for predicting the turning motion in regular waves is developed along with sliding time window and time series split cross-validation. The model test data of a KCS container ship and a ONRT tumblehome ship under different wave conditions are used for online modeling and real-time prediction. The results indicate that the ship trajectories with trajectory drift and fluctuation features during the turning motion under the wave impacts are captured well by the developed method with satisfactory confidence intervals.

Effect of Handedness on Learned Controllers and Sensorimotor Noise During Trajectory-Tracking.

In human-in-the-loop control systems, operators can learn to manually control dynamic machines with either hand using a combination of reactive (feedback) and predictive (feedforward) control. This article studies the effect of handedness on learned controllers and performance during a trajectory-tracking task. In an experiment with 18 participants, subjects perform an assay of unimanual trajectory-tracking and disturbance-rejection tasks through second-order machine dynamics, first with one hand then the other. To assess how hand preference (or dominance) affects learned controllers, we extend, validate, and apply a nonparametric modeling method to estimate the concurrent feedback and feedforward controllers. We find that performance improves because feedback adapts, regardless of the hand used. We do not detect statistically significant differences in performance or learned controllers between hands. Adaptation to reject disturbances arising exogenously (i.e., applied by the experimenter) and endogenously (i.e., generated by sensorimotor noise) explains observed performance improvements.

A new nonparametric degradation modeling method for truncated degradation signals by axis rotation

Nonparametric degradation modeling can automatically formulate the principal degradation pattern of condition monitoring signals. It performs higher flexibility than parametric modeling in describing complicated degradation processes of machinery. Functional principal component analysis (FPCA) is a commonly used nonparametric modeling technique. A basic requirement of FPCA is that different units must share the same scale in X-axis. A FPCA-based degradation modeling method has been specially designed for truncated degradation signals whose amplitudes are truncated at the failure threshold. This method formulates the degradation time as a function of state level by axis rotation. Since the degradation signals of different units are truncated at the same failure threshold level, the axis rotation strategy ensures the same scale in X-axis. However, in cases of noisy signals, one state level may correspond to multiple time points, which means that the time function depending on state level does not exist. In addition, this method assumes that the noise term as well as the functional principal component (FPC) scores follow the Gaussian distribution, which restricts the flexibility in real practice. Aiming at the above two issues, this paper develops a new degradation modeling method based on the FPCA by axis rotation. The proposed method formulates the degradation model on the definition of first pass time (FPT), which ensures the existence of the time function depending on state level. The noise term and the FPC scores are not restricted to the Gaussian distribution. A particle filtering based updating algorithm is developed to deal with the non-Gaussian state estimation issue of the FPC scores. The effectiveness of the proposed method is verified using an experiment of milling cutter wear.

Identification modeling of ship maneuvering motion based on local Gaussian process regression

A fast and accurate nonparametric modeling method based on local Gaussian process regression (LGPR) is proposed for the identification modeling and prediction of ship maneuvering motion. The training dataset collected from the free-running model tests of ship maneuvering is automatically divided into a number of clusters according to the similarity criterion by clustering analysis using k-means algorithm. Utilizing the data in each cluster, the corresponding local nonparametric model is identified. The computational cost of training and prediction based on LGPR is reduced compared to that based on the classic Gaussian process regression (CGPR) using the whole training dataset. Taking the KVLCC2 tanker and an unmanned surface vehicle (USV) as study objects, the nonparametric models are identified based on the experimental data of zigzag maneuvers of the KVLCC2 model and random maneuver of the USV. Using the identified models, the zigzag maneuvers of the KVLCC2 model and the random maneuver of the USV, which are not involved in the training data, are predicted. The results show that LGPR has higher computational efficiency than CGPR with acceptable prediction accuracy.

The Review of Electromagnetic Field Modeling Methods for Permanent-Magnet Linear Motors

Permanent-magnet linear motors (PMLMs) are widely used in various fields of industrial production, and the optimization design of the PMLM is increasingly attracting attention in order to improve the comprehensive performance of the motor. The primary problem of PMLM optimization design is the establishment of a motor model, and this paper summarizes the modeling of the PMLM electromagnetic field. First, PMLM parametric modeling methods (model-driven methods) such as the equivalent circuit method, analytical method, and finite element method, are introduced, and then non-parametric modeling methods (data-driven methods) such as the surrogate model and machine learning are introduced. Non-parametric modeling methods have the characteristics of higher accuracy and faster computation, and are the mainstream approach to motor modeling at present. However, surrogate models and traditional machine learning models such as support vector machine (SVM) and extreme learning machine (ELM) approaches have shortcomings in dealing with the high-dimensional data of motors, and some machine learning methods such as random forest (RF) require a large number of samples to obtain better modeling accuracy. Considering the modeling problem in the case of the high-dimensional electromagnetic field of the motor under the condition of a limited number of samples, this paper introduces the generative adversarial network (GAN) model and the application of the GAN in the electromagnetic field modeling of PMLM, and compares it with the mainstream machine learning models. Finally, the development of motor modeling that combines model-driven and data-driven methods is proposed.

A novel gray wolf optimizer with RNA crossover operation for tackling the non-parametric modeling problem of FCC process

This work proposes a novel gray wolf optimizer (GWO) with RNA crossover-operation and adaptive control parameter scheme (named as RNA-GWO). The RNA crossover-operation can better enhance the population diversity of RNA-GWO, it is designed according to the special pseudoknot structure of RNA molecule. The adaptive control parameter scheme is proposed to replace the linear one to balance the exploration and exploitation capacities during the optimization process of RNA-GWO. The effectiveness of RNA-GWO is verified by a suite of IEEE CEC 2017 benchmark functions. The experimental results prove that RNA-GWO can obtain higher accuracy solutions than the other six meta-heuristic algorithms. The RNA-GWO is also employed to solve the parameter problem of the wavelet neural network (WNN) non-parametric modeling method and applied to model the FCC process. The simulation results demonstrate that the RNA-GWO can provide favorable parameters for WNN, the model outputs of RNA-GWO optimized WNN can better agree with the experimental data of FCC process.

Reliability analysis of multistage transmission system with coupling uncertainties

The coupling uncertainties have already become the greatest difficulty in assessing the reliability of multistage transmission systems. A new reliability assessment method of rotor system with uncertainties is proposed here. The probability density functions (PDF) of the state functions are re-derived by introducing the divergence control parameter of uncertainty from nonparametric modeling method and the Lagrange multipliers involved are calculated by the Sigel function method. Then, the joint PDF of all state functions is calculated so that the reliability prediction of multistage systems can be effectively carried out. Both the coupling uncertainties and the intensity of uncertainties are formulated in the PDF of the state functions. Furthermore, the short-period prediction method is improved to avoid wrong judgments caused by excessive neighbor radius. Numerical simulation suggests a reference for which degree of uncertainty can be accepted. It’s very helpful to predict the evolution of operation status and avoid the occurrence of faults.

Forecasting house prices in Turkey: GLM, VAR and time series approaches

Purpose- A wide range of decision-makers is interested in educated forecasts for house prices. The technical analysis introduced in this study aims to estimate future (forecasted) house prices and provide sufficient evidence in support of the adequacy of the estimated models obtained from parametric and non-parametric modeling methods for Turkey's housing market. Methodology- We employ non-parametric and various time series methods to find appropriate fits to forecast Turkey's house price index (HPI). In our modelling, we consider macroeconomic indicators related to housing markets, such as; gold, interest rate and currencies. In this study, first using the explanatory variables, we construct two Generalized Linear Models (GLM) and a Vector Auto Regressive (VAR) model. Then, we construct two univariate time series models. HPI series inherits seasonality. Even though the HPI contains seasonality, first, we neglect the seasonal effect and come up an Autoregressive Moving Average (ARMA(p,q)) model among many other alternative ARMA models. Second, we consider the seasonality effect on the housing market index and construct a seasonal Autoregressive Integrated Moving Average (ARIMA(p,d,q)(P,D,Q)) and exponential smoothing models. Findings- The analysis identifies forecasts of Turkey’s housing market index from both the seasonal ARIMA(p,d,q) (P,D,Q)_m and Holt Winter models as accurate models compared to classical time series models, namely ARIMA(p,d,q) models, based on the explanation power measure (R^2) values and out-of-sample error measures MSE, RMSE and MAE. Conclusion- The study has three main contributions: i) Our forecast shows Turkey's housing market's return will not increase in the following 12-months. ii) The seasonal ARIMA and exponential smoothing models forecast some negative returns within the given forecasting period, which should be considered a warning for Turkey's housing market for the future. iii) GLM and VAR models illustrate that Turkey's housing market shows a high dependence on gold, inflation, and foreign exchange rates than other well-known economic indicators.

Nonparametric Methods in Population Pharmacokinetics.

Population pharmacokinetic (PK) modeling is a widely used approach to analyze PK data obtained from groups of individuals, in both industry and academic research. The approach can also be used to analyze pharmacodynamic (PD) data and pooled PK/PD data. There are 2 main families of population PK methods: parametric and nonparametric. The objectives of this article are to present an overview of nonparametric methods used in population pharmacokinetic modeling and to explain their specific characteristics to inform scientists and clinicians about their potential value for data analysis, simulation, dosage design, and therapeutic drug monitoring (TDM). Nonparametric methods have several interesting characteristics for population PK analysis, including computation of exact likelihoods, the ability to accommodate parameter probability distributions of any shape (eg, non-Gaussian), and to detect subpopulations and outliers. Nonparametric population methods are also highly relevant for model-based TDM and design of individualized drug dosage regimens. Several algorithms have been developed to estimate model parameter values within an individual and compute that individual's dosage to achieve target drug exposure with maximum precision and accuracy. Nonparametric modeling methods for both population and individual PK analysis are available under user-friendly packages.

Short-Term Intersection Traffic Flow Forecasting

The intersection is a bottleneck in an urban roadway network. As traffic demand increases, there is a growing congestion problem at urban intersections. Short-term traffic flow forecasting is crucial for advanced trip planning and traffic management. However, there are only a handful of existing models for forecasting intersection traffic flow. In addition, previous short-term traffic flow forecasting models usually were for predicting roadway conditions in a very short period, such as one minute or five minutes, which is often too late given that a driver may well be approaching the bottleneck already. Being able to accurately predict traffic congestions in about half-hour advance is very critical for advanced trip planning and traffic management. To fill this gap, this research develops a two-layer stacking model for intersection short-term traffic flow forecasting by integrating the K-nearest neighbor (KNN) and Elman Neural Network modeling methods. It was developed using the 24-h cycle by cycle traffic data collected at a signalized intersection in Jinan, China. The developed model is evaluated by applying it to the same intersection for forecasting the short-term traffic conditions in a different set of days. The prediction performance of this model was compared with four other models developed using some existing non-parametric modeling and machine learning methods, including clustering, backpropagation (BP) neural network, KNN, and Elman Neural Network. The results of this study indicate that the proposed model outperforms other existing models in terms of its prediction accuracy.

Combined Impact of Sample Size and Modeling Approaches for Predicting Stem Volume in Eucalyptus spp. Forest Plantations Using Field and LiDAR Data

Light Detection and Ranging (LiDAR) remote sensing has been established as one of the most promising tools for large-scale forest monitoring and mapping. Continuous advances in computational techniques, such as machine learning algorithms, have been increasingly improving our capability to model forest attributes accurately and at high spatial and temporal resolution. While there have been previous studies exploring the use of LiDAR and machine learning algorithms for forest inventory modeling, as yet, no studies have demonstrated the combined impact of sample size and different modeling techniques for predicting and mapping stem total volume in industrial Eucalyptus spp. tree plantations. This study aimed to compare the combined effects of parametric and nonparametric modeling methods for estimating volume in Eucalyptus spp. tree plantation using airborne LiDAR data while varying the reference data (sample size). The modeling techniques were compared in terms of root mean square error (RMSE), bias, and R2 with 500 simulations. The best performance was verified for the ordinary least-squares (OLS) method, which was able to provide comparable results to the traditional forest inventory approaches using only 40% (n = 63; ~0.04 plots/ha) of the total field plots, followed by the random forest (RF) algorithm with identical sample size values. This study provides solutions for increasing the industry efficiency in monitoring and managing forest plantation stem volume for the paper and pulp supply chain.

The effect of the reaction equilibrium on the kinetics of gas-solid reactions — A non-parametric modeling study

The viability of thermochemical energy storage for a given application is often determined by the reaction kinetics under process conditions. For high exergetic efficiency the process needs to operate in close proximity to the reaction equilibrium. Thus, accurate kinetic models that include the effect of the reaction equilibrium are required.In the present work, different parametrization methods for the equilibrium term in the General Kinetic Equation are evaluated by modeling the kinetics of two reaction systems relevant for thermochemical energy storage (CaC2O4 and CuO) from experimental data. A non-parametric modeling method based on tensor decompositions is used that allows for a purely data driven assessment of different parametrization methods.Our analysis shows that including a suitable equilibrium term is crucial. Omitting the equilibrium term when modeling formation reactions can lead to seemingly negative activation energies. Our tests also show that for formation reactions, the reaction rate decreases much faster towards the equilibrium than theory predicts. We present an empirical modeling approach that can predict the reaction rate of gas-solid reactions, regardless of the shortcomings of theory. In this way, non-parametric modeling offers a powerful tool for applied research and may contribute to the advancement of the thermochemical energy storage technology.

Nonparametric Model Prediction for Intelligent Regulation of Human Cardiorespiratory System to Prescribed Exercise Medicine

Intelligent regulation for human exercise behaviors becomes significantly necessary for exercise medicine after the COVID-19 epidemic. The key issue of exercise regulation and its potential development for intelligent exercise is to describe human exercise physiological behaviors in a more accurate and sufficient manner. Here, a non-parametric modeling method with kernel-based regularization is presented to estimate cardiorespiratory biomarkers (i.e., oxygen uptake ( ${\dot {\text {V}}}$ O2) and carbon dioxide output ( ${\dot {\text {V}}}$ CO2) by merely non-invasively monitoring the indicator of exercise intensity (e.g., walking speed). Using the kernel-based non-parametric modeling, we show that ${\dot {\text {V}}}$ O2 and ${\dot {\text {V}}}$ CO2 behaviors in response to continuous and diversified exercise intensity stimulations can be quantitatively described. Furthermore, the dataset from the stairs experiment with a proper protocol is applied in the kernel parameter selection, and this selection approach is compared with the numerical simulation approach. The comparison results illustrate an improvement of 4.18% for oxygen uptake and 7.63% for carbon dioxide output in a half period, and 11.00% for oxygen uptake and 12.60% for carbon dioxide output in one period when using the kernel parameter selected from the stairs exercise. Moreover, the advantages of using the non-parametric model, the necessity of sufficient stimulation for identification and the importance of the kernel regularization term are also addressed in this paper. This method provides fundamental work for the practice of intelligent exercise.

The use of statistics in social sciences

Purpose The purpose this paper is to review some of the statistical methods used in the field of social sciences. Design/methodology/approach A review of some of the statistical methodologies used in areas like survey methodology, official statistics, sociology, psychology, political science, criminology, public policy, marketing research, demography, education and economics. Findings Several areas are presented such as parametric modeling, nonparametric modeling and multivariate methods. Focus is also given to time series modeling, analysis of categorical data and sampling issues and other useful techniques for the analysis of data in the social sciences. Indicative references are given for all the above methods along with some insights for the application of these techniques. Originality/value This paper reviews some statistical methods that are used in social sciences and the authors draw the attention of researchers on less popular methods. The purpose is not to give technical details and also not to refer to all the existing techniques or to all the possible areas of statistics. The focus is mainly on the applied aspect of the techniques and the authors give insights about techniques that can be used to answer problems in the abovementioned areas of research.

Integrated statistical modeling method: part I—statistical simulations for symmetric distributions

The use of parametric and nonparametric statistical modeling methods differs depending on data sufficiency. For sufficient data, the parametric statistical modeling method is preferred owing to its high convergence to the population distribution. Conversely, for insufficient data, the nonparametric method is preferred owing to its high flexibility and conservative modeling of the given data. However, it is difficult for users to select either a parametric or nonparametric modeling method because the adequacy of using one of these methods depends on how well the given data represent the population model, which is unknown to users. For insufficient data or limited prior information on random variables, the interval approach, which uses interval information of data or random variables, can be used. However, it is still difficult to be used in uncertainty analysis and design, owing to imprecise probabilities. In this study, to overcome this problem, an integrated statistical modeling (ISM) method, which combines the parametric, nonparametric, and interval approaches, is proposed. The ISM method uses the two-sample Kolmogorov–Smirnov (K–S) test to determine whether to use either the parametric or nonparametric method according to data sufficiency. The sequential statistical modeling (SSM) and kernel density estimation with estimated bounded data (KDE-ebd) are used as the parametric and nonparametric methods combined with the interval approach, respectively. To verify the modeling accuracy, conservativeness, and convergence of the proposed method, it is compared with the original SSM and KDE-ebd according to various sample sizes and distribution types in simulation tests. Through an engineering and reliability analysis example, it is shown that the proposed ISM method has the highest accuracy and reliability in the statistical modeling, regardless of data sufficiency. The ISM method is applicable to real engineering data and is conservative in the reliability analysis for insufficient data, unlike the SSM, and converges to an exact probability of failure more rapidly than KDE-ebd as data increase.

Generic Inference on Quantile and Quantile Effect Functions for Discrete Outcomes

Quantile and quantile effect (QE) functions are important tools for descriptive and causal analysis due to their natural and intuitive interpretation. Existing inference methods for these functions do not apply to discrete random variables. This article offers a simple, practical construction of simultaneous confidence bands for quantile and QE functions of possibly discrete random variables. It is based on a natural transformation of simultaneous confidence bands for distribution functions, which are readily available for many problems. The construction is generic and does not depend on the nature of the underlying problem. It works in conjunction with parametric, semiparametric, and nonparametric modeling methods for observed and counterfactual distributions, and does not depend on the sampling scheme. We apply our method to characterize the distributional impact of insurance coverage on health care utilization and obtain the distributional decomposition of the racial test score gap. We find that universal insurance coverage increases the number of doctor visits across the entire distribution, and that the racial test score gap is small at early ages but grows with age due to socio-economic factors especially at the top of the distribution. Supplementary materials (additional results, R package, replication files) for this article are available online.