Prediction Residuals Research Articles

A Novel Ultra-short-term Photovoltaic Power Generation Forecasting Method Based on Seasonal Autoregressive Integrated Moving Average

This study suggests an ultra-short-term photovoltaic (PV) energy generation forecasting model based on Seasonal Autoregressive Integrated Moving Average (SARIMA) and Support Vector Machine (SVM). It can further increase the predictive performance of PV electricity generation electrical output and address the issue of significant fluctuation and instability of PV power generation output power. To achieve combined prediction, SVM is employed in this study to model the nonlinear SARIMA prediction residuals. In comparison to the traditional SVM method and the SARIMA method, the mixed forecasting method has a good forecasting consequence and can serve as a favorable basis for the safe operation and scheduling of the PV grid, according to the results of the simulation of actual data.

An Algorithm for Learning Orthonormal Matrix Codebooks for Adaptive Transform Coding.

This paper proposes a novel data-driven approach to designing orthonormal transform matrix codebooks for adaptive transform coding of any non-stationary vector processes which can be considered locally stationary. Our algorithm, which belongs to the class of block-coordinate descent algorithms, relies on simple probability models such as Gaussian or Laplacian for transform coefficients to directly minimize with respect to the orthonormal transform matrix the mean square error (MSE) of scalar quantization and entropy coding of transform coefficients. A difficulty commonly encountered in such minimization problems is imposing the orthonormality constraint on the matrix solution. We get around this difficulty by mapping the constrained problem in Euclidean space to an unconstrained problem on the Stiefel manifold and leveraging known algorithms for unconstrained optimization on manifolds. While the basic design algorithm directly applies to non-separable transforms, an extension to separable transforms is also proposed. We present experimental results for adaptive transform coding of still images and video inter-frame prediction residuals, comparing the transforms designed using the proposed method and a number of other content-adaptive transforms recently reported in the literature.

Rate-Distortion Optimized Geometry Compression for Spinning LiDAR Point Cloud

Point cloud is a major representation format of 3D objects and scenes. It has been increasingly applied in various applications due to the rapid advances in 3D sensing and rendering technologies. In the field of autonomous driving, point clouds captured by spinning Light Detection And Ranging (LiDAR) devices have become an informative data source for road environment perception and intelligent vehicle control. On the other hand, the massive data volume of point clouds also brings huge challenges to point cloud transmission and storage. Therefore, establishing compression frameworks and algorithms that conform to the characteristics of point cloud data has become an important research topic for both academia and industry. In this paper, a geometry compression method dedicated to spinning LiDAR point cloud was proposed taking advantage of the prior information of the LiDAR acquisition procedure. Rate-distortion optimizations were further integrated into the coding pipeline according to the characteristics of the prediction residuals. Experimental results obtained on different datasets show that the proposed method consistently outperforms the state-of-the-art G-PCC predictive geometry coding method with reduced runtime at both the encoder and decoder sides.

Profitability related industrial-scale batch processes monitoring via deep learning based soft sensor development

Data-driven soft sensor technology has been widely developed to estimate quality-related variables, while following difficulties still limit its application in batch processes, such as different initial conditions, uneven-length of batches, and the extraction of within-batch multiphase features. To address these problems, a quality prediction and monitoring framework is proposed. Variables related to quality-related variables are first selected, and a data stacked strategy is proposed to transform three-dimensional batch data into time-lagged sequences that can be fed into soft sensor models. Aiming to extract the multiphase features, a novel differential recurrent neural networks is established by embedding differential operations into long short-term memory neural networks. Moreover, to ensure profitability, prediction residuals are employed for quality monitoring. Case study on a simulation dataset and an industrial-scale penicillin fermentation process demonstrates the effectiveness of the proposed method and its applicability to batch process monitoring and control in both academic research and industrial operation.

A fault prediction framework for Doubly‐fed induction generator under time‐varying operating conditions driven by digital twin

AbstractAccurate status prediction is critical for detecting potential failures of Doubly‐fed induction generator (DFIG). A unique Digital Twin (DT) fault prediction paradigm is put forth in this research, which uses an edge intelligence paradigm to ensure high‐performance fault prediction. Edge‐based DT provides compute and storage capabilities on edge devices to enable effective data processing. The stator current spectrum and instantaneous power spectrum of the DFIG are simulated and assessed, followed by the development of a verification prototype on the DFIG and the qualitative observation of the internal performance of the DFIG. The encoder is constructed using Squeeze‐and‐Excitation (SE) Convolutional Neural Network (SE‐CNN) with the channel attention mechanism, and the decoder is constructed based on a long short‐term memory (LSTM) network to form the self‐encoding model named SE‐CNN‐LSTM. In order to forecast problems and provide a foundation for predictive maintenance, the status indicator is built based on the prediction residuals of the status variables. Based on the outcomes of edge‐cloud co‐simulation and data collected from the Supervisory Control and Data Acquisition system at the Da Bancheng Wind Farm in Xinjiang, China, it is demonstrated that the framework and technology are workable and capable of predicting generator failures.

Real-time detection of abnormal driving behavior based on long short-term memory network and regression residuals

Abnormal driving behavior is one of the main causes of roadway collisions. In most studies of abnormal driving behavior, the abnormal driving status is detected and analyzed using classification algorithms directly or using unsupervised learning algorithms to classify reconstruction or prediction residuals. However, abnormal driving behavior data are difficult to acquire and label. Also, a class imbalance issue is inherent in the algorithm training process due to the relatively sparse data for abnormal driving behavior. Moreover, current studies that include residual analysis tend to focus on individual points and thus fail to capture the continuity characteristic of abnormal driving behavior.To address these problems, a long short-term memory-residual (LSTM-R) algorithm is proposed to detect abnormal driving behavior in real time. The proposed algorithm (referred to simply as LSTM-R) has two steps. First, an LSTM network is used to fit the current vehicle kinematic data based on historical data to obtain the root mean square residual at each moment. Second, a time window-based residual algorithm is designed and employed to detect abnormal driving behavior according to the magnitude and continuity of the residuals. To verify the effectiveness of LSTM-R, an experimental test was conducted in Nanjing, China. The vehicle kinematic data were collected non-intrusively using a smartphone.In addition, AdaCost, SMOTEBoost, EasyEnsemble, LightGBM-residual, and linear regression-residual algorithms were employed for comparison with the proposed algorithm to assess its effectiveness. The effects of (1) the degree-of-fit of the LSTM network, (2) the LSTM-R parameters, and (3) the abnormal driving behavior percentage on the detection results were analyzed in detail. First, both the underfitting and overfitting of the LSTM network compromise the detection performance. Second, within a certain range of values, the LSTM-R parameters have little effect on the detection results. Third, the detection results are affected only slightly by the abnormal proportion. The results show that LSTM-R, with a maximum F1-score of 0.866, significantly outperforms the other five algorithms. Furthermore, even if only 10% abnormal driving behavior is in the training set, LSTM-R’s F1-score can still be close to 0.86, indicating a significant relaxation of the requirements for labeled data. Also, the required data are easy to collect, which indicates LSTM-R’s extensive application possibilities. This paper thus provides an effective method for the real-time detection of abnormal driving behavior and also supports driving risk assessment and driving behavior improvement with the overall goal to enhance roadway safety.

Exploring Spatial Patterns of Interurban Passenger Flows Using Dual Gravity Models.

Geographical gravity models can be employed to quantitatively describe and predict spatial flows, including migration flows, passenger flows, daily commuting flows, etc. However, how to model spatial flows and reveal the structure of urban traffic networks in the case of missing partial data is still a problem to be solved. This paper is devoted to characterizing the interurban passenger flows in the Beijing-Tianjin-Hebei region of China using dual gravity models and Tencent location big data. The method of parameter estimation is the least squares regression. The main results are as follows. First, both the railway and highway passenger flows can be effectively described by dual gravity models. A small part of missing spatial data can be compensated for by predicted values. Second, the fractal properties of traffic flows can be revealed. The railway passenger flows follow the gravity scaling law better than the highway passenger flows. Third, the prediction residuals indicate the changing trend of interurban connections in the study area in recent years. The center of gravity of the spatial dynamics has shifted from the Beijing-Tianjin-Tangshan triangle to the Beijing-Baoding-Shijiazhuang axis. A conclusion can be reached that the dual gravity model is an effective tool for analyzing spatial structures and dynamics of traffic networks and flows. Moreover, the model provides a new approach to estimating the fractal dimensions of traffic networks and spatial flow patterns.

Deterministic ship roll forecasting model based on multi-objective data fusion and multi-layer error correction

With the increasing frequency of international shipping and marine resources development activities, ship motion prediction plays an increasingly important role in ensuring the safety of offshore operations. However, in the field of ship motion prediction, the deep feature information of raw high-resolution ship motion data, as well as the predictable components in the initial prediction residuals is usually neglected. In this paper, a deterministic ship roll forecasting model based on multi-objective data fusion and multi-layer error correction is proposed. The proposed model consists of three stages, which are data pre-processing stage, multi-objective data fusion forecasting stage, and multi-layer error correction stage. To verify the stability and validity of the proposed model, an experimental study was conducted using three sets of measured ship roll motion data collected in the South China Sea from 2018 to 2020. Taking the 1-step, 5-step, and 10-step predictions of dataset #1 as an example, the RMSE values of the proposed model are 0.0130°, 0.0612°, and 0.0791°, respectively. Through three analytical experiments and four comparison experiments, it is proved that the proposed model is able to obtain accurate deterministic point forecasts, which can better assist the sailor in decision making.

Topological graph representation of stratigraphic properties of spatial-geological characteristics and compression modulus prediction by mechanism-driven learning

The soil's compression modulus (Es) is one of the most critical mechanical parameters for studying land subsidence in urban strata. Meanwhile, the vertical heterogeneity and lateral discontinuity in the distribution of Es is one of the leading causes of TBM (tunnel-boring-machine) entrapment in the field. In this study, we first obtain the minimum redundancy input variables for Es prediction by comparing 20 purely data-driven models. Based on this, we propose a static directed and weighted graph based on spatial correlation and sequence stratigraphy for mechanism-driven learning, the spatial-geological stratigraphic graph (SG-SPG), where edges encode essential spatial-geological information in direction and weights. Then we built a spatial-geological graph attention network (SGGAT) for node-level estimation and edge-level correction of Es prediction, including two attention layers to adequately capture the dependencies between spatial location and lithology. The results show that our method outperforms state-of-the-art models (R2 = 0.99, RMSE = 0.0593 MPa). Furthermore, through network analysis of SG-SPG, we find that increasing global transitivity significantly improves accuracy, while decreasing local connectivity and degree centrality of nodes slightly increases the prediction residuals. Our approach provides interpretability for graph neural network (GNN) and is advantageous for geophysical property prediction problems with Spatio-temporal heterogeneity.

Edge-Based Video Compression Texture Synthesis Using Generative Adversarial Network

It has been recognized that texture patterns with abundant high-frequency components, such as grass and water, produce visual masking effects, and the distortion in textures is hard to be perceived by human eyes than structure regions. However, modern video codecs in a rate-distortion optimized manner usually consume a lot of bits to encode textures, leading to the insufficiency in perceptual coding performance. Nowadays, with the rapid development of deep learning, learning based texture synthesis methods have been proposed to replace the coding process of prediction residuals to reduce the rate cost. In this paper, we present a deep texture synthesizer named edge-based texture synthesis framework (ETSF). At encoder side, the framework detects texture regions by semantic and fidelity classification criteria, and the detected regions are quantized coarsely by the hybrid coding framework. In texture characterization, ETSF extracts low-level edge features representing pixel intensity variation. Feature processing tools are developed to remove the spatiotemporal redundancy of edges. The processed edge information is compressed and transmitted. To effectively recover textures, we design an edge-based texture synthesis generative adversarial network (ETSGAN) at the decoder of ETSF, which can incorporate edge information into convolutional layers and generate realistic textures. Experimental results on a collected texture dataset show that the proposed ETSF can achieve an average of -12.8%, -14.2% and -9.6% MOS BD-rate under lowdelay_B, lowdelay_P and random_access configurations of VVC coding, respectively.

Evaluating and Improving Neonatal Gentamicin Pharmacokinetic Models Using Aggregated Routine Clinical Care Data.

Model-informed precision dosing (MIPD) can aid dose decision-making for drugs such as gentamicin that have high inter-individual variability, a narrow therapeutic window, and a high risk of exposure-related adverse events. However, MIPD in neonates is challenging due to their dynamic development and maturation and by the need to minimize blood sampling due to low blood volume. Here, we investigate the ability of six published neonatal gentamicin population pharmacokinetic models to predict gentamicin concentrations in routine therapeutic drug monitoring from nine sites in the United State (n = 475 patients). We find that four out of six models predicted with acceptable levels of error and bias for clinical use. These models included known important covariates for gentamicin PK, showed little bias in prediction residuals over covariate ranges, and were developed on patient populations with similar covariate distributions as the one assessed here. These four models were refit using the published parameters as informative Bayesian priors or without priors in a continuous learning process. We find that refit models generally reduce error and bias on a held-out validation data set, but that informative prior use is not uniformly advantageous. Our work informs clinicians implementing MIPD of gentamicin in neonates, as well as pharmacometricians developing or improving PK models for use in MIPD.

Detection and quantification of temperature sensor drift using probabilistic neural networks

Temperature effects are a major driver of strain and deformations in weather-exposed civil infrastructure, such as bridges and buildings. For such structures, long-term temperature data holds the potential for data-driven prediction of expected structural behavior, which in turn enables the detection of anomalous structural behavior. For this reason, structural health monitoring (SHM) strategies typically employ temperature sensors. However, the success of SHM is contingent on the quantity and quality of the available temperature data. Hence, accurate automatic methods for the detection and quantification of anomalies in temperature data are needed. In particular, gradual temperature sensor drifts are difficult to detect and can introduce errors into the thermal compensation of strain sensors, which can be erroneously confounded with time-dependent structural behavior. Current data-driven methods use air temperature as predictor because it exhibits good correlation with temperatures in the structure. Sensor drift is typically quantified by analysis of the prediction residuals; however, these methods are not robust to outliers and can be affected by seasonal biases. In this work, a probabilistic neural network is used as the nonlinear data-driven temperature prediction model which enables the introduction of a sensible threshold to mitigate seasonal model bias. Furthermore, a novel drift detection method based on the evolution of parameters of a trinomial probability distribution is introduced, together with a robust drift quantification method. The performance of this method is assessed using real temperature data from a pedestrian bridge, spanning over seven years of the structure’s life.

P-step Kalman-like unbiased FIR algorithm for detecting anomalies of atomic clocks

An extended measurement model anomaly detection algorithm for atomic clocks is proposed based on the p-step Kalman-like iterative Unbiased finite impulse response (UFIR) algorithm. The novel of the proposed algorithm is that it breaks the disadvantage of initializing the noise matrix in the traditional Kalman filter improved algorithms. By accumulating the prediction residuals generated by the process of p-step Kalman-like iterative, the detection statistics are constructed to realize the weaker phase and frequency jump detection of atomic clocks. Simulation results show the effectiveness of the proposed algorithm, and Monte-Carlo (M-C) experiment is used to verify the theoretical calculation of detection probability. We also analyze the measured data of a cesium atomic clock in our timekeeping laboratory, and the results show that the new method is more sensitive to weak frequency jump than the improved algorithm based on the Kalman filter.

Spatial Outlier Accommodation Using a Spatial Variance Shift Outlier Model

Outlier detection has been a long-debated subject among researchers due to its effect on model fitting. Spatial outlier detection has received considerable attention in the recent past. On the other hand, outlier accommodation, particularly in spatial applications, retains vital information about the model. It is pertinent to develop a method that is capable of accommodating detected spatial outliers in a fashion that retains vital information in the spatial models. In this paper, we formulate the variance shift outlier model (SVSOM) in the spatial regression as a robust spatial model using restricted maximum likelihood (REML) and use weights based on the detected outliers in the model. The spatial outliers are accommodated via a revised model for the outlier observations with the help of the SVSOM. Simulation results show that the SVSOM, based on the detected spatial outliers is more efficient than the general spatial model (GSM). The findings of this study also reveal that contamination in the residuals and x variable have little effect on the parameter estimates of the SVSOM, and that outliers in the y variable are always detectable. Asymptotic distribution of the squared spatial prediction residuals are obtained to confirm the outlyingness of an observation. The merit of or proposed SVSOM for the detection and accommodating outliers is also confirmed using artificial and COVID-19 data sets.

Real-Time LiDAR Point Cloud Compression Using Bi-Directional Prediction and Range-Adaptive Floating-Point Coding

Due to the large amount of data involved in the three-dimensional (3D) LiDAR point clouds, point cloud compression (PCC) becomes indispensable to many real-time applications. In autonomous driving of connected vehicles for example, point clouds are constantly acquired along the time and subjected to be compressed. Among the existing PCC methods, very few of them have effectively removed the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">temporal</i> redundancy inherited in the point clouds. To address this issue, a novel lossy LiDAR PCC system is proposed in this paper, which consists of the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">inter</i> -frame coding and the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">intra</i> -frame coding. For the former, a deep-learning approach is proposed to conduct bi-directional frame prediction using an asymmetric residual module and 3D space-time convolutions; the proposed network is called the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">bi-directional prediction network</i> (BPNet). For the latter, a novel <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">range-adaptive floating-point coding</i> (RAFC) algorithm is proposed for encoding the reference frames and the B-frame prediction residuals in the 32-bit floating-point precision. Since the pixel-value distribution of these two types of data are quite different, various encoding modes are designed for providing adaptive selection. Extensive simulation experiments have been conducted using multiple point cloud datasets, and the results clearly show that our proposed PCC system consistently outperforms the state-of-the-art MPEG G-PCC in terms of data fidelity and localization, while delivering real-time performance.

PTR-CNN for in-loop filtering in video coding

A deep learning method called PTR-CNN (Predicted frame with Transform unit partition and prediction Residual aided CNN) is proposed for in-loop filtering in video compression. To reduce the computational complexity of an end-to-end CNN in-loop filter, a non-learning method of reference frame selection is designed to select the highest quality frame based on the frame’s blurriness and smoothiness scores. The transform unit (TU) partition and the prediction residual (PR) of the current frame are used as extra inputs to the neural network as the filtering guidance. The selected similar and high quality reference frame (RF) and the current unfiltered frame (CUF) are input to a CNN based motion compensation module to generate a predicted frame (PF). Finally input the PF, the CUF, the CUF’s TU partition and the CUF’s PR into the main CNN to reconstruct the filtered frame. The model is implemented in Tensorflow and tested in HEVC and AV1. Experimental results show that the complexity of proposed PTR-CNN is less than SOTA CNN-based reference aided in-loop filtering methods and slightly outperforms their RD performance. The scheme introduces a complexity overhead of 7% on the encoder. In particular, for random access, the proposed model achieves 11.78% coding gain over HEVC with DBF/SAO off, while has a gain of 4.76% over HEVC with DBF/SAO on. Ablation study demonstrates that the RF contributes about 10% of the total gain, and the TU and PR contribute over 4% of the total one, proving the effectiveness of each module. Moreover, it is observed that the proposed method can restore detailed structures and textures and hence improve the subjective quality.

High Efficiency Intra Video Coding Based on Data-Driven Transform

In this work, we propose a high efficiency intra video coding based on data-driven transform, which is able to learn the source distributions of intra prediction residuals and improve the subsequent transform coding efficiency. Firstly, we model learning based transform design as an optimization problem of maximizing energy compaction or decorrelation. A data-driven Subspace Approximation with Adjusted Bias (Saab) transform is analyzed and compared with the mainstream Discrete Cosine Transform (DCT) on their energy compaction and decorrelation capabilities. Secondly, we propose a Saab transform based intra video coding framework with offline Saab transform learning. Meanwhile, intra mode dependent Saab transform is developed. Then, Rate-Distortion (RD) gain of Saab transform based intra video coding is theoretically and experimentally analyzed in detail. Finally, three strategies that apply the Saab transform to intra video coding are developed to improve the coding efficiency. Experimental results demonstrate that the proposed <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$8\times 8$ </tex-math></inline-formula> Saab transform based intra coding can achieve Bjønteggard Delta Bit Rate (BDBR) from −1.19% to −10.00% and −3.07% on average as compared with the mainstream <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$8\times 8$ </tex-math></inline-formula> DCT based intra coding. In case of variable size transform unit setting, the proposed algorithm achieves BDBR from −0.17% to −6.09% and −1.80% on average, which outperform the DCT based and the neural network based schemes.

Improving sign-algorithm convergence rate using natural gradient for lossless audio compression

In lossless audio compression, the predictive residuals must remain sparse when entropy coding is applied. The sign algorithm (SA) is a conventional method for minimizing the magnitudes of residuals; however, this approach yields poor convergence performance compared with the least mean square algorithm. To overcome this convergence performance degradation, we propose novel adaptive algorithms based on a natural gradient: the natural-gradient sign algorithm (NGSA) and normalized NGSA. We also propose an efficient natural-gradient update method based on the AR(p) model, which requires mathcal {O}(p) multiply–add operations at every adaptation step. In experiments conducted using toy and real music data, the proposed algorithms achieve superior convergence performance to the SA. Furthermore, we propose a novel lossless audio codec based on the NGSA, called the natural-gradient autoregressive unlossy audio compressor (NARU), which is open-source and implemented in C. In a comparative experiment with existing, well-known codecs, NARU exhibits superior compression performance. These results suggest that the proposed methods are appropriate for practical applications.

Incorporating Spatial Autocorrelation in Machine Learning Models Using Spatial Lag and Eigenvector Spatial Filtering Features

Applications of machine-learning-based approaches in the geosciences have witnessed a substantial increase over the past few years. Here we present an approach that accounts for spatial autocorrelation by introducing spatial features to the models. In particular, we explore two types of spatial features, namely spatial lag and eigenvector spatial filtering (ESF). These features are used within the widely used random forest (RF) method, and their effect is illustrated on two public datasets of varying sizes (Meuse and California housing datasets). The least absolute shrinkage and selection operator (LASSO) is used to determine the best subset of spatial features, and nested cross-validation is used for hyper-parameter tuning and performance evaluation. We utilize Moran’s I and local indicators of spatial association (LISA) to assess how spatial autocorrelation is captured at both global and local scales. Our results show that RF models combined with either spatial lag or ESF features yield lower errors (up to 33% different) and reduce the global spatial autocorrelation of the residuals (up to 95% decrease in Moran’s I) compared to the RF model with no spatial features. The local autocorrelation patterns of the residuals are weakened as well. Compared to benchmark geographically weighted regression (GWR) models, the RF models with spatial features yielded more accurate models with similar levels of global and local autocorrelation in the prediction residuals. This study reveals the effectiveness of spatial features in capturing spatial autocorrelation and provides a generic machine-learning modelling workflow for spatial prediction.

Predicting hydraulic jump characteristics in a gradually expanding stilling basin with roughness elements by Sugeno Fuzzy Logic

Abstract The hydraulic behavior of a hydraulic jump can be challenging to estimate in order to design gradually expanding stilling basins with roughness elements on the bed. In this study, five dependent variables were identified that comprise: (i) the sequent depth ratio; (ii) the relative length of the jump; (iii) the relative roller length of the jump; (iv) the relative energy dissipation; and (v) the water surface profile. This study undertook a set of formulations based on the regression analysis and Sugeno Fuzzy Logic (SFL) to predict these variables based upon experimental data. Results demonstrate that the trained SFLs predicted the behavior of the dependent variables with the Nash–Sutcliffe Coefficient (NSC) greater than 0.96 in the testing phase. In contrast, the NSC values for the regression models are greater than 0.79. The higher accuracy of SFL is attributed to its capability in managing uncertainty and imprecise data owing to water surface profile oscillations of the hydraulic jump. Also, the results indicate that the prediction residuals for SFL are homoscedastic for all hydraulic parameters investigated except for the water surface profile, the prediction residuals of which for the regression equations are heteroscedastic.