Orthogonal Series Expansion Research Articles

Probabilistic neural networks for incremental learning over time-varying streaming data with application to air pollution monitoring

This paper proposes a novel algorithm for incremental learning over streaming data in a non-stationary environment. The idea refers to the applicability of Probabilistic Neural Networks (PNNs), commonly used as a fast and robust method for solving classification problems in a stationary scenario. It is well known that PNNs have solid mathematical foundations but they fail in the non-stationary scenario and are not able to deal with concept drift. In this paper, the method based on the orthogonal series expansions of unknown probability densities is proposed. This approach significantly extends the applicability of the PNNs. The nonparametric procedures called Incremental Probabilistic Neural Networks (IPNNs) for tracking drifting probability densities or conditional class densities, in the case of classification, are presented. We prove their convergence as the stream data size is growing to infinity. More precisely, we show convergence in probability and with probability one as the sample size tends to infinity. Compared to the existing literature, almost exclusively based on various heuristics, the proposed method is mathematically justified and has solid theoretical foundations. Simulations performed on synthetic and air pollution data confirm the effectiveness of the algorithm.

A hierarchical Bayesian modeling framework for identification of Non-Gaussian processes

Non-Gaussian processes are frequently encountered in engineering problems, posing a challenge when it comes to identification. The main challenge in the identification arises from the fact that a non-Gaussian process can be treated as a collection of infinite dimensional non-Gaussian variables. The application of the hierarchical Bayesian modeling (HBM) framework is constrained due to the inherent complexity of dimensionality and non-Gaussian characteristics associated with these variables. To tackle the issue of dimensionality, the improved orthogonal series expansion (iOSE) representing a non-Gaussian process by time functions with non-Gaussian coefficients, which are readily obtained from discretizing the process at some specific time points, is introduced within the HBM framework. In particular, the iOSE is embedded to convert the identification of a non-Gaussian process into a finite number of non-Gaussian coefficients. Regarding their non-Gaussian characteristics, polynomial chaos expansion (PCE) is used to quantify the uncertainty of the non-Gaussian coefficients with parameters in PCE treated as hyper parameters to be estimated by the HBM framework. The proposed framework is applicable to the identification of both stationary and nonstationary non-Gaussian processes. The effectiveness of non-Gaussian process quantification by the proposed framework is demonstrated using simulated data of a non-stationary extreme value process. Theapplicability of this approach for non-Gaussian process identification is validated by accurately identifying a stochastic load in a structural dynamic problem. Furthermore, it is successfully applied to the reconstruction of random mode shapes of a building arising from different environmental conditions.

Semi-parametric single-index predictive regression models with cointegrated regressors

This paper considers the estimation of a semi-parametric single-index regression model that allows for nonlinear predictive relationships. This model is useful for predicting financial asset returns, whose observed behaviour resembles a stationary process, if the multiple nonstationary predictors are cointegrated. The presence of cointegrated regressors imposes a single-index structure in the model, and this structure not only balances the nonstationarity properties of the multiple predictors with the stationarity properties of asset returns but also avoids the curse of dimensionality associated with nonparametric regression function estimation. An orthogonal series expansion is used to approximate the unknown link function for the single-index component. We consider the constrained nonlinear least squares estimator of the single-index (or the cointegrating) parameters and the plug-in estimator of the link function, and derive their asymptotic properties. In an empirical application, we find some evidence of in-sample nonlinear predictability of U.S. stock returns using cointegrated predictors. We also find that the single-index model in general produces better out-of-sample forecasts than both the historical average benchmark and the linear predictive regression model.

On the use of L-functionals in regression models

Abstract In this article, we survey and unify a large class or L L -functionals of the conditional distribution of the response variable in regression models. This includes robust measures of location, scale, skewness, and heavytailedness of the response, conditionally on covariates. We generalize the concepts of L L -moments (G. Sillito, Derivation of approximants to the inverse distribution function of a continuous univariate population from the order statistics of a sample, Biometrika 56 (1969), no. 3, 641–650.), L L -skewness, and L L -kurtosis (J. R. M. Hosking, L-moments: analysis and estimation of distributions using linear combinations or order statistics, J. R. Stat. Soc. Ser. B Stat. Methodol. 52 (1990), no. 1, 105–124.) and introduce order numbers for a large class of L L -functionals through orthogonal series expansions of quantile functions. In particular, we motivate why location, scale, skewness, and heavytailedness have order numbers 1, 2, (3,2), and (4,2), respectively, and describe how a family of L L -functionals, with different order numbers, is constructed from Legendre, Hermite, Laguerre, or other types of polynomials. Our framework is applied to models where the relationship between quantiles of the response and the covariates follows a transformed linear model, with a link function that determines the appropriate class of L L -functionals. In this setting, the distribution of the response is treated parametrically or nonparametrically, and the response variable is either censored/truncated or not. We also provide a framework for asymptotic theory of estimates of L L -functionals and illustrate our approach by analyzing the arrival time distribution of migrating birds. In this context, a novel version of the coefficient of determination is introduced, which makes use of the abovementioned orthogonal series expansion.

Neural-network-based wavefront solution algorithm for a wide field survey telescope.

The wide field survey telescope (WFST) is a 2.5m optical survey telescope currently under construction in China. The telescope employs a primary-focus optical design to achieve a wide field of view of 3deg, and its focal plane is equipped with four pairs of curvature sensors to perform wavefront sensing and active optics. Currently, there are several wavefront solution algorithms available for curvature sensors, including the iterative fast Fourier transform method, orthogonal series expansion method, Green's function method, and sensitivity matrix method. However, each of these methods has limitations in practical use. This study proposes a solution method based on a convolutional neural network model with a U-Net structure for the curvature wavefront sensing of the WFST. Numerical simulations show that the model, when properly trained, has a high accuracy and performs a curvature wavefront solution effectively. Upon a comparison with the sensitivity matrix method, this new method demonstrates its superiority. Finally, the study is summarized, and the drawbacks of the proposed method are discussed, which leads to direction for future optimizations.

Application of Nonlinear Differential Equation in Electric Automation Control System

Abstract This article uses fifth-order nonlinear differential equations to describe the dynamic process of electrical automation control systems. This method first derives the equivalent system of the nonlinear fuzzy global system and then uses the orthogonal polynomial series expansion technique and its integral operation matrix. The local manifold at the dominant unstable equilibrium point of a single-machine infinite-bus system after a failure described by a two-dimensional quadratic nonlinear differential equation is calculated, and the stability boundary of the power system is obtained. The research results show that the output frequency fluctuation of the electrical automation control system is small after the algorithm is adopted, and the intelligent control system can accurately diagnose and warn the electrical faults. The system can meet the requirements of online voltage coordinated control.

A hierarchical Bayesian framework embedded with an improved orthogonal series expansion for Gaussian processes and fields identification

A new hierarchical Bayesian framework (HBM) is proposed for identification of Gaussian processes or fields, which are usually used for simulating uncertainty in temporal variability of loads or spatial variability of material properties. An improved orthogonal series expansion (iOSE) is embedded into the proposed framework by simulating the Gaussian process or field through correlated Gaussian variables, and then HBM is applied to quantify their uncertainty. Hyper parameters to be identified are set to be the mean value and standard deviation vectors of these Gaussian variables, as well as the parameters in autocorrelation function (ACF) of the Gaussian process or field which are used to replace correlation coefficients of correlated Gaussian variables for reducing the number of hyper parameters. With the identified hyper parameters, a simulation model of the Gaussian process or field can be obtained based on the iOSE expression. In addition, model class selection is introduced to select the optimal number of orthogonal functions and integral points involved in iOSE as well as select the category of ACF among several alternative models, known to influence the simulated expression and accuracy. Studies conducted on two dynamic examples verify the effectiveness of proposed framework.

Evaluation of Different Methods for Voltage Sag Source Detection

This paper compares and evaluates three different methods for voltage sag source detection. First method (method I) is based on the assumption, that the energy flow at the monitoring point increases during downstream events and decreases during upstream events. Second and third method (methods II and III) are both based on the assumption that currents measured at the monitoring point increase during downstream events and decrease during upstream events. The slope of a current-voltage trajectory is investigated in method II, while a real current component is observed within method III. Both current-based methods (II and III) require fundamental harmonic components of sampled voltages and currents, which are extracted using discrete orthogonal series expansion, such as Fourier or Walsh. Algorithms of this type are especially appropriate for studying steady-state and periodically repeating conditions. Voltage sags are, on the contrary, transient disturbance events. Thus, usage of the discussed algorithms may not be appropriate. Furthermore, criteria within methods II and III are checked for each phase individually. In the cases of asymmetrical voltage sags exact interpretation of the obtained results, therefore, might not be possible. Method I is, on the contrary, based on instantaneous values of line voltages and currents, while three-phase criterion is used. An exact interpretation of the results obtained by this method is, therefore, also possible in cases of asymmetrical voltage sags. All the discussed methods for voltage sag source detection have been tested by applying extensive simulations and field tests. The results for ground faults, asymmetrical voltage sags, upstream events and motor starting have been analyzed in order to evaluate all the discussed methods. The obtained results show that all discussed methods are very successful in cases of heavy motor starting and other symmetrical voltage sags. In cases of asymmetrical voltage sags the methods II and III do not work well, especially for those originating from the upstream side, while the method I is not successful only in particular cases of voltage sags due to upstream ground faults. Based on the performed evaluation it can be concluded, that further development is still needed to increase the degree of confidence in the discussed methods.

Legendre orthogonal polynomial method in calculating reflection and transmission coefficients of fluid-loaded functionally gradient plates

Based on the Legendre orthogonal polynomial series expansion and the partial wave theory, a Legendre orthogonal polynomial method (LOPM) is proposed to calculate the reflection and transmission coefficients of plane waves at the liquid/solid interface of a liquid-loaded functionally gradient material (FGM) plate. The displacement solutions in FGM plate are fitted approximately by Legendre orthogonal polynomial series. The stresses and the governing differential equations of the FGM plate are derived. Based on the boundary conditions of the liquid/solid interface and the governing differential equations, the linearly independent equations are set up to calculate the reflection and transmission coefficients. Meanwhile, the expansion coefficients of the Legendre orthogonal polynomials are also obtained. The power function establishes the gradient model of the mechanical parameters along the thickness direction. The angular spectrums of reflection and transmission coefficients from LOPM are in well agreement with the calculation results from the transfer matrix method. By analyzing the convergence of the reflection coefficient spectrum, the critical value of the truncated order of Legendre orthogonal polynomials is determined. The mapping relationship between the FGM gradient models and the reflection coefficient angular spectrum, also the frequency spectrums and the displacement/stress distributions can be demonstrated simultaneously, which provides the theoretical fundamentals of the ultrasonic non-destructive testing for the mechanical properties of FGM and extends the application of the LOPM.

Reliability analysis of horizontally loaded pile considering spatial variability of soil parameters

In the design and analysis of pile foundations, the horizontal load is an important factor that needs to be considered. The reliability analysis can take the uncertainty of various parameters into consideration and thus reduce the design deviations, especially the uncertainty of soil parameters. This paper proposes a reliability analysis method considering the spatial variability of soil for the horizontally loaded pile. Firstly, a parameter sensitivity analysis was performed by the Sobol sensitivity method, and the critical variables, including the undrained shear strength Cu, the soil strain ε50, which represents the soil strain when principal stress difference reaches half (50%) of the maximum value in triaxial test, and the horizontal load and applied moment, are identified and taken as the random variables. Moreover, the spatial variability of soil parameters (Cu and ε50) are simulated as one-dimensional random field through the orthogonal series expansion method. The failure probabilities of pile foundation under the failure modes of deflection and strength are evaluated by Monte Carlo simulation. Finally, the proposed method is verified through case study in which the influences of the spatial variability of soil parameters, the uncertainty of the horizontal load on the failure probability were investigated. The results show that the uncertainty of horizontal loads and Cu make significant effects on the reliability of pile foundations. The increase of correlation length of Cu and ε50 will lead to the increase of the mean value of pile response, which has different effects on the failure probability under different failure modes.

Statistical Feature Extraction and System Identification Algorithms for Partial Discharge Signal Classification Using Laguerre Polynomial Expansion

In this paper, a novel algorithm for partial discharge (PD) pulse waveform analysis and separation based on orthogonal series expansion using Laguerre polynomials is proposed. A system identification technique is also developed to help increase the accuracy rate of the signal classification. Laguerre polynomial series expansions are applied on the PD pulses to effectively extract important information and well-discriminative features. Laguerre expansion coefficients inherently contain information about the amplitude (intensity) and the relaxation time of a dynamic system and this helps direct characterization of PD pulses. The proposed technique is applied to PD pulses acquired from a laboratory set of multiple PD sources. It is shown that this statistical method can classify partial discharge signals with high accuracies, even when the signals are visually indistinguishable. In this research, two methods are developed for system identification. One is based on a deterministic approach in the form of a recursive formula and the other one is a stochastic method based on a group Lasso methodology. The aim of system identification is to improve the classification accuracy by removing the effect of the measurement system from the observed PD waveforms. The accuracy rate of the proposed algorithm is evaluated using the signals that are captured from simultaneous artificial defects in high pressure SF <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">6</inf> and in low pressure air.

Free vibration analysis of functionally graded porous plates in contact with bounded fluid

This study investigates the free vibrations of functionally graded porous (FGP) plates in contact with fluid. As an illustrative problem, a rectangular FGP plate is considered as the vertical wall of a rigid tank containing liquid. The plate displacement field is approximated based on the first-order shear deformation theory (FSDT). Thus, the expressions of its strain and kinetic energy are obtained. The fluid is assumed to be incompressible, inviscid, and irrotational. Its velocity potential functions are obtained by applying suitable boundary and compatibility conditions. By using the Rayleigh–Ritz method with two-variable orthogonal polynomials series expansion, the natural frequencies of the FGP plate in contact with fluid are calculated. To validate the method, the results obtained for the natural frequencies are compared with those available in the literature for FGP plates without fluid and isotropic plates with fluid. Then, three different types of porosity distribution are considered to evaluate their effects on the natural frequencies of the plate. Besides, the effect are investigated of different parameters, such as the fluid depth, side-to-thickness ratio, length-to-width (aspect) ratio.

A time-variant uncertainty propagation analysis method based on a new technique for simulating non-Gaussian stochastic processes

In this paper, the time-variant uncertainty propagation analysis is defined to solve the output stochastic process of a time-variant function with uncertainty. And a time-variant uncertainty propagation analysis method is constructed with the combination of an extended orthogonal series expansion method (extended OSE) and sparse grid numerical integration (SGNI). The SGNI serving as a classical uncertainty propagation method is utilized here to solve the moments and autocorrelation function at discrete time points of the time-variant performance function. And the extended OSE is proposed to simulate the output stochastic process based on the results from SGNI. By extended OSE, a non-Gaussian stochastic process is represented as the sumof orthogonal time functions with random coefficients, and these coefficients can be directly obtained by discretization of the target process. For these coefficients are correlated and non-Gaussian, the correlated polynomial chaos expansion method (c-PCE) is presented to represent them in terms of correlated standard Gaussian variables, and then the principal component analysis (PCA) is adopted to transform them into independent ones with dimension reduction. Finally we can obtain an explicit expression to represent the non-Gaussian process whatever it is stationary or non-stationary. Three illustrative examples are used to verify the performance of the extended OSE. In addition, two engineering problems are investigated to demonstrate the effectiveness of the time-variant uncertainty propagation method.

A least squares-type density estimator using a polynomial function

Higher-order density approximation and estimation methods using orthogonal series expansion have been extensively discussed in statistical literature and its various fields of application. This study proposes least squares-type estimation for series expansion via minimizing the weighted square difference of series distribution expansion and a benchmarking distribution estimator. As the least squares-type estimator has an explicit expression, similar to the classical moment-matching technique, its asymptotic properties are easily obtained under certain regularity conditions. In addition, we resolve the non-negativity issue of the series expansion using quadratic programming. Numerical examples with various simulated and real datasets demonstrate the superiority of the proposed estimator.

A time-variant extreme-value event evolution method for time-variant reliability analysis

Abstract In this paper, we propose a time-variant extreme-value event evolution method (TEEM). The time-evolution process of extreme-value event is firstly proposed in this paper. And by solving it, we can obtain the time-variant reliability of arbitrary time interval and arbitrary failure threshold. In this method, the random process in limit-state function is firstly expanded by an improved orthogonal series expansion method (iOSE). Second, we introduce the idea of extreme-value event to describe the time-variant reliability problem. And by discretizing the time domain, we can obtain a series of extreme-value events. The moments of extreme-value event in every discrete time interval will be solved by the integration of Broyden–Fletcher–Goldfarb–Shanno (BFGS) method and univariate dimension reduction method (UDRM). Third, a time-dependent polynomial chaos expansion method (t-PCE) is proposed to simulate the extreme-value event's time-evolution process, and it will be simulated as a function in terms of a standard normal variable and time. Finally, Monte Carlo simulation (MCS) is adopted to sample the standard normal variable to obtain the time-variant reliability of arbitrary failure threshold and time interval. Three numerical examples are investigated to demonstrate the effectiveness of the proposed methods.

Dynamic computation for rigid–flexible multibody systems with hybrid uncertainty of randomness and interval

Considering the unavoidable uncertainty of material properties, geometry, and external loads existing in rigid–flexible multibody systems, a new hybrid uncertain computational method is proposed. Two evaluation indexes, namely interval mean and interval error bar, are presented to quantify the system response. The dynamic model of a rigid–flexible multibody system is built by using the absolute node coordinate formula (ANCF). The geometry size and external loads of rigid components are considered as interval variables, while the Young’s modulus and Poisson’s ratio of flexible components are expressed by a random field. The continuous random field is discretized to Gaussian random variables by using the expansion optimal linear estimation (EOLE) method. This paper proposes an orthogonal series expansion method, termed as improved Polynomial-Chaos-Chebyshev-Interval (PCCI) method, which solves the random and interval uncertainty under one integral framework. The improved PCCI method has some sampling points located on the bounds of interval variables, which lead to a higher accuracy in estimating the bounds of multibody systems’ response compared to the PCCI method. A rigid–flexible slider–crank mechanism is used as a numerical example, which demonstrates that the improved PCCI method has a higher accuracy than the PCCI method.

Second-order variational equations for spatial point processes with a view to pair correlation function estimation

Second-order variational type equations for spatial point processes are established. In case of log linear parametric models for pair correlation functions, it is demonstrated that the variational equations can be applied to construct estimating equations with closed form solutions for the parameter estimates. This result is used to fit orthogonal series expansions of log pair correlation functions of general form.

Special Issue on Recent Trends in Stochastic Models, Statistics and Their Applications

This special issue was organized on the occasion of the 13th Workshop on Stochastic Models, Statistics and Their Applications (SMSA 2017) held at Humboldt University of Berlin on February 20-24, 2017. Key areas of the workshop focused on models and methods for problems arising in industry, finance, natural sciences, engineering, and technology. In these areas, solutions are needed, which are able to deal with complex and possibly high-dimensional data, make use of innovative methodologies, and have proven applicability to real-world data. In this special issue, we have 13 papers. The papers consider stochastic models for diverse data sets and streams such as bridge vehicle loads, watermark identification data, text documents, images of digitized handwritten digits, financial asset returns, option prices, and equity data. The methods and algorithms required to handle come from machine learning, stochastic processes, sequential (on-line) change-point detection, regression analysis, image analysis, and robust divergence measures. The work of Barbulescu analyzes the properties of Generalized Regression Neural Network (GRNN) model applied to the prediction of the selected Romanian stock market prices. In the work of Luo et al, a nonlinear PDE model to estimate the American call options and put options under financial crisis scenario is implemented with additional penalty introduced to Black-Scholes formula. The work of Liu et al proposes a compound Poisson bivariate process model of vehicle load with Levy copula function and validates it in a reliability analysis of bridges in China. Migalska proposes in her paper a density-free symmetry verification test for arbitrary symmetries in images. In their work, Darkhovsky et al propose a novel method based on the notion of ϵ-complexity of continuous vector functions for an off-line detection of multiple change-points in multidimensional time series. The work of Rafajlowicz et al proposes a method of spike detection and suppression (SDS method), which extends the jump detector proposed by the Authors in their earlier work and applies it for a laser power control in a 3D additive manufacturing process. Sliwinski et al propose two algorithms for nonlinearity recovery in Hammerstein systems, one based on orthogonal series expansions and the other using aggregation and validate them in numerical simulations. The work of Sousa et al provides an insight into the behavior of GARCH process describing financial markets affected by rumours and other random factors through an analysis of process mean and variance by estimating run length. Kisslinger et al describe a new toolkit based on scaled Bregman divergences for detecting distributional changes in random data, including streams and clouds. The work of Sarnowski et al derives an optimal decision function for a sudden change detection in political business cycle model based on the Markovian process. The work of Walkowiak et al proposes a two-step procedure for open-set classification of text documents and validates it on simulated data and on the collection of Wikipedia documents. The work by Xie et al combines two-tier stochastic frontier model with the traditional Richardson Model and applies it to the Chinese stock market analysis. Finally, the work of Yuan et al derives the Hamilton-Jacobi-Bellman equation for an optimal consumption problem of maximizing the expected discounted value of utility in an economic growth model with random technological shocks. We thank numerous anonymous reviewers for helping in evaluating all submitted manuscripts.

Additive nonparametric models with time variable and both stationary and nonstationary regressors

This paper considers nonparametric additive models that have a deterministic time trend and both stationary and integrated variables as components. The diverse nature of the regressors caters for applications in a variety of settings. In addition, we extend the analysis to allow the stationary regressor to be instead locally stationary, and we allow the models to include a linear form of the integrated variable. Heteroscedasticity is allowed for in all models. We propose an estimation strategy based on orthogonal series expansion that takes account of the different type of stationarity/nonstationarity possessed by each covariate. We establish pointwise asymptotic distribution theory jointly for all estimators of unknown functions and also show the conventional optimal convergence rates jointly in the L2 sense. In spite of the entanglement of different kinds of regressors, we can separate out the distribution theory for each estimator. We provide Monte Carlo simulations that establish the favorable properties of our procedures in moderate sized samples. Finally, we apply our techniques to the study of a pairs trading strategy.

Semiparametric Single-Index Predictive Regression

This paper studies a semi-parametric single-index predictive regression model with multiple nonstationary predictors that exhibit co-movement behaviour. Orthogonal series expansion is employed to approximate the unknown link function in the model and the estimator is derived from an optimization under the constraint of identification condition for index parameter. The main finding includes two types of super-consistency rates of the estimators of the index parameter along two orthogonal directions in a new coordinate system. The central limit theorem is established for a plug-in estimator of the unknown link function. In the empirical studies, we provide evidence in favour of nonlinear predictability of the stock return using long term yield and treasury bill rate.