Prediction Residuals Research Articles

Using extended Kalman filter for failure detection and prognostic of degradation process in feedback control system

In this article, we study a new approach to predict failures in feedback control system and particularly in actuators. However, we use two-tank control system with a proportional–integral–derivative controller for controlling a process variable. In practice, the actuator is a dynamic operating component in a random environment. Moreover, its capacity decreases over time and becomes valuable information for reliability analysis. The loss of capacity which is related to degradation, either normally or in an accelerated manner, depends on different operational conditions of the feedback control system and environmental factors. For this reason, to improve its working condition, a service life time analysis is necessary. Obviously, one has to predict the trend of future system characteristics, such as the reliability, which is measured by the estimate value of remaining useful life. In this situation, we use the stochastic gamma process model to describe the degradation behavior of the actuator. Generally, the algorithm of extended Kalman filter is a widely used method to overcome the difficulties of estimating the state vector in a nonlinear model of two-tank control system. This algorithm gives an innovation vector or prediction residual which contains fault information, when the system is failed. The prediction residuals can be recursively computed for diagnosis by the generalized likelihood ratio test. However, we use the generalized likelihood ratio test algorithm to estimate the moment at which the prognostic started. Finally, a practical case study is given to show the effectiveness of the proposed approaches for failure detection. Obviously, the simulation results show that the degradation path of the actuator capacity is estimated and the reliability based on remaining useful life predicted is analyzed.

Progressive Transmission of Medical Images via a Bank of Generative Adversarial Networks.

The healthcare sector is currently undergoing a major transformation due to the recent advances in deep learning and artificial intelligence. Despite a significant breakthrough in medical imaging and diagnosis, there are still many open issues and undeveloped applications in the healthcare domain. In particular, transmission of a large volume of medical images proves to be a challenging and time-consuming problem, and yet no prior studies have investigated the use of deep neural networks towards this task. The purpose of this paper is to introduce and develop a deep-learning approach for the efficient transmission of medical images, with a particular interest in the progressive coding of bit-planes. We establish a connection between bit-plane synthesis and image-to-image translation and propose a two-step pipeline for progressive image transmission. First, a bank of generative adversarial networks is trained for predicting bit-planes in a top-down manner, and then prediction residuals are encoded with a tailored adaptive lossless compression algorithm. Experimental results validate the effectiveness of the network bank for generating an accurate low-order bit-plane from high-order bit-planes and demonstrate an advantage of the tailored compression algorithm over conventional arithmetic coding for this special type of prediction residuals in terms of compression ratio.

A hybrid prediction model for wind speed using support vector machine and genetic programming in conjunction with error compensation

Wind energy resources are a clean renewable energy source. Accurate prediction of wind speed has important theoretical significance and practical value for sustainable use of wind energy, energy planning, and economic development. This paper proposes a hybrid model for prediction of wind speed using support vector machine and genetic programming models, with error compensation for the prediction residuals. Wind speed and the residuals generated using the hybrid model are simulated and predicted. The proposed model is applied to monthly average wind speed data observed over the period January 2005–December 2014 from the Tuoli and Hetian stations in Xinjiang, China. For each time series, eight prediction schemes, including independent support vector machine model and genetic programming model and their combination, were tried. At the same time, phase space reconstruction method was used to select model input factors. The results clearly show the performance differences of the eight wind speed prediction schemes in the prediction effect of the two time series. The combination of the support vector machine wind speed prediction model and the genetic programming residual prediction model has the best prediction effect, with a correlation coefficient value above 0.9, mean square value below 0.1, and mean absolute error value below 0.2. This combined model is found to be quite efficient and, therefore, the proposed approach is highly suitable to predict the monthly average wind speed.

Fault detection of wind turbine based on SCADA data analysis using CNN and LSTM with attention mechanism

The complex and changeable working environment of wind turbine often challenges the condition monitoring and fault detection. In this paper, a new method is proposed for fault detection of wind turbine, in which the convolutional neural network (CNN) cascades to the long and short term memory network (LSTM) based on attention mechanism (AM). Supervisory control and data acquisition (SCADA) data are used from wind turbine as input variables and build CNN architecture to extract dynamic changes of data. AM is applied to strengthen the impact of important information. AM can assign different weighs for concentrating the characteristics of LSTM to increase the learning accuracy through mapping weigh and parameter learning. The proposed model can execute early warning for anomaly state and deduce the faulted component by prediction residuals. Finally, through the cases the early failure of the wind turbine is predicted, which verifies the effectiveness of the proposed method.

Application of Singular Spectrum Analysis in Deep-Ocean Tide Reconstruction and Prediction

This study utilized the Singular Spectrum Analysis (SSA) approach to perform time series orthogonalization and demonstrated its use by analyzing vertical tidal acceleration and sea level time series from different deep-ocean locations. This method quantifies astronomical variations by using decomposed vertical tidal acceleration to reconstruct and predict deep-ocean tide. The results show that each decomposed vertical tidal acceleration can be associated with the decomposed sea level having at least 5 astronomical variations. Their associated energies can also be used to diagnose the change of the oceanic tide response to tidal acceleration. Performance evaluation also shows that this method can give comparable reconstruction accuracy and slightly better prediction accuracy compared to the harmonic analysis-based method. It is indicated that the proposed method is accurate enough to be applied in a tsunami detection algorithm. The results also indicate that the proposed method is stable enough to provide unpropagated prediction residuals.

Image Tampering Localization Using Demosaicing Patterns and Singular Value Based Prediction Residue

Almost all image sensors measure only one color per pixel through the color filter array. Missing pixels are estimated using a demosaicing process. For this reason, a demosaiced image leaves a particular trace. When an image is manipulated or tampered, the demosaicing trace can be changed. This change can serve as a basic clue for detecting or localizing image tampering. Demosaicing pattern-based tampering localization algorithms require a re-interpolation process, and the prediction residue between the given image and the re-interpolated image is commonly used to localize tampered regions. However, the prediction residue is not always valid because the demosaicing interpolation kernel cannot be known, which deteriorates the localization performance. This paper presents an effective re-interpolation process using singular value decomposition for an unknown demosaicing method. First, the green channel of the given image is decomposed into four sub-images according to the Bayer pattern. For a small block of each sub-image, the singular value decomposition is performed. The prediction residue is obtained by reconstructing the image block after removing the largest singular value. The feature to localize the forged regions is extracted by the logarithm ratio of the prediction residue variance. The proposed method does not require any statistical model for the extracted feature, because the prediction residue is more accurate than that of conventional methods. We perform intensive experiments for three test datasets and compare the proposed method with state-of-the-art tampering localization methods, the results of which indicate that the proposed scheme outperforms existing approaches.

A CNN-Based Fast Inter Coding Method for VVC

The Versatile Video Coding (VVC) achieves superior coding efficiency as compared with the High Efficiency Video Coding (HEVC), while its excellent coding performance is at the cost of several high computational complexity coding tools, such as Quad-Tree plus Multi-type Tree (QTMT)-based Coding Units (CUs) and multiple inter prediction modes. To reduce the computational complexity of VVC, a CNN-based fast inter coding method is proposed in this paper. First, a multi-information fusion CNN (MF-CNN) model is proposed to early terminate the QTMT-based CU partition process by jointly using the multi-domain information. Then, a content complexity-based early Merge mode decision is proposed to skip the time-consuming inter prediction modes by considering the CU prediction residuals and the confidence of MF-CNN. Experimental results show that the proposed method reduces an average of 30.63% VVC encoding time, and the Bjøontegaard Delta Bit Rate (BDBR) increases about 3%.

Sign prediction and sign regression

Intuitively, model-predicted signs matter a lot in finance, especially for investment strategy constructions. This paper proposes an approach whereby the loss function regularizes the errors in prediction in different ways. In particular, the loss function considers errors in predicted signs and the sizes and signs of the residuals in the model prediction simultaneously. Less weight is given to residuals with correctly predicted signs, while more weight is assigned to residuals with wrongly predicted signs. This is important because agents make decisions according to model predictions, especially the signs of the predictions. At the same time, larger residuals are penalized more and smaller residuals are penalized less. The signs of the residuals are considered in the loss function because they also affect decision-making processes. This paper proposes a new approach, termed “sign regression”, which takes these considerations into account. We show that ordinary least squares estimators generate better Sharpe ratios than sign regression does for most of the assets studied in this paper. However, sign regression can perform better for some assets. (Published at Journal of Investment Strategies, DOI: 10.21314/JOIS.2021.002)

Detection of Glucose Sensor Faults in an Artificial Pancreas via Whiteness Test on Kalman Filter Residuals

Abstract Continuous Glucose Monitoring (CGM) sensors are key components in an artificial pancreas, an emerging tool for type 1 diabetes treatment. Malfunctioning of this component might reduce the efficacy of glucose control achieved by the system and even pose the safety of the patient at risk. Therefore, accurate and prompt detection of these anomalies is an important problem. This paper investigates a model-based method to detect CGM failures. Based on an individualized linear model of the subject, identified on hystorical data, the method predicts future glucose concentration through a one-step ahead Kalman predictor. The correct functioning of the system is then monitored using two different criteria: the first checks the magnitude of prediction residuals. The second checks the whiteness of the residuals through a correlogram test. The effectiveness of the two criteria is investigated and compared by performing tests on an in-silico dataset obtained by means of UVA/Padova Type 1 Diabetes simulator, accepted by the US Food and Drug Administration as a substitute of animal testing prior to artificial pancreas clinical trials on humans.

Efficient Bias Robust Cross Section Factor Models

This paper introduces a theory based robust regression estimator, called the mOpt estimator, that minimizes the maximum bias with respect to a Tukey-Huber mixture model that includes a standard linear regression model with normally distribution errors as a special case, but also allows for a small fraction of unrestricted fat-tailed and skewed non-normal distribution variations from the standard model. The estimator has a very intuitive weighted least squares interpretation based on a data-dependent weight function that is equal to zero for robustly scaled prediction residuals that are larger in magnitude than 3.0, and thereby rejects outliers. We apply the robust regression method to single factor and multiple factor cross-section models for Size, BM, Beta and EP factors, and find that the robust regression results reverse the Fama-French 1992 (FF92) conclusions concerning the significance of the Size, Beta and EP factors. The difference in our results and those of FF92 is that the robust regression rejects approximately 4% to 5% of outliers, most of which, but not all, occur for microcap stocks, with smallcap stocks also having some influential outliers, and even largecaps have a few. We strongly recommend standard use of the mOpt estimator as an important complement to least squares for empirical asset pricing research, as well as for quantitative finance applications in general.

In silico analysis of protein-protein interfaces: Tools and approaches

Two or more proteins interact in vivo to perform complex molecular functions including catalysis, regulation, assembly, immunity and inhibition through the formation of stable interfaces. This interaction is governed by several factors that are selective, sensitive and specific in nature. Several interface features has been documented since 1975. The study of these interface features of proteins and their dynamicity during interaction with different proteins help understanding the mechanisms underlying diverse molecular functions and its biological processes. Computational tools greatly assist in studying such interface features that determine the interaction between two or more proteins, and in this context, this review enumerates the different interface features reported thus far along with the tools that aid in deciphering protein features (physicochemical characteristics, binding site and interface residue prediction and hotspot residues) along with their approaches that are employed in the prediction these features. Also, the review discusses the advantages and limitations of experimental techniques and computational biological tools deployed for deciphering the protein-protein interactions. Altogether, the review will provide insights into the optimal tools and different strategies involved in protein interaction studies that would facilitate the researchers to understand the protein structural features and molecular principles of protein-protein interaction with known functions.

Spatial and temporal variability of tritium in precipitation within South Africa and it's bearing on hydrological studies

Tritium, the radioactive isotope of hydrogen, has been used to understand groundwater recharge processes for decades. The current variation of tritium in the atmosphere is largely attributed to stratospheric production and fall out rates as well as global circulation phenomena controlling the hydrological cycle. Global controls on the variability in atmospheric tritium activity are poorly suited to explain local variation and tritium activities in precipitation are often assumed to be uniform over both local and regional catchments and watersheds. This assumption can result in both over and under estimation of modern recharge within an aquifer when using tritium as the recharge proxy. In order to minimize the inherent prediction residuals associated with tritium based recharge investigations, the variability of tritium in precipitation was modelled from 127 spatial precipitation samples taken over a two year period, combined with a 76 precipitation sample group-set taken over a one year period in a single location. Precipitation events were traced backward in time, from the point of collection, using HYSPLIT modelling to ascertain the origins of moisture content as well as the altitudes of moisture origin reached along the particle track. Tritium activities, collected over a one year period in Paarl, range from 0.45 to 4.16 TU and have a mean of 1.59 TU. Spatial storm events in the Western Cape in 2017 and 2018 had a range from 0 to 2.2 and 0.37 to 3.27 TU, respectively, with mean activities of 1.18 (n = 34) and 1.25 TU (n = 32). Both storm events had similar tritium variability (σ = 0.5 n = 35 and 0.48 n = 32). Regional precipitation events had the largest range of tritium activities (0.55–12.2 TU). Although not all tritium activities can be explained by interrogating the water mass origin, this study suggests that approximately 90% of events can be completely or partially attributed to the origin of the water mass. The variability of tritium, both spatially and temporally, was higher than expected, confirming that when uniform tritium inputs are used, the groundwater system would provide inaccurate modern recharge estimates. Higher spatial resolution of tritium variation in precipitation for a particular region will improve our ability to relate tritium activities in groundwater to local precipitation.

Fault detection based on one-class deep learning for manufacturing applications limited to an imbalanced database

Despite extensive studies for the industrial applications of deep learning, its actual usage in manufacturing sites has been extremely restrained by the difficulty in obtaining sufficient industrial data, especially for production failure cases. In this study, we introduced a fault-detection module based on one-class deep learning for imbalanced industrial time-series data, which consists of three submodules, namely, time-series prediction based on deep learning, residual calculation, and one-class classification using one-class support vector machine and isolation forest. Four different networks were used for the time-series prediction: multilayer perception (MLP), residual network (ResNet), long–short-term memory (LSTM), and ResNet–LSTM, each trained with the one-class data having only the production success cases. We adopted the residuals of the deep-learning prediction as an elaborated feature for the construction of the one-class classification. We also tested the fault-detection module with the actual mass production data of a die-casting process. By adopting the features elaborated by the deep-learning time-series prediction, we showed that the total accuracy of the one-class classification significantly improved from 90.0% to 96.0%. Especially for its capability to detect production failures, the accuracy improved from 84.0% to 96.0%. The area under the receiver operating characteristics (AUROC) also improved from 87.56% to 98.96%. ResNet showed the best performance for detecting production failures, whereas ResNet–LSTM produced better results for ensuring the production success. Our results suggest that the one-class deep learning is a promising approach for extracting important features from time-series data to realize a one-class fault-detection module.

Sign Prediction and Sign Regression

Intuitively, model-predicted signs matter a lot in finance, especially for investment strategy constructions. This paper proposes an approach whereby the loss function regularizes the errors in prediction in different ways. In particular, the loss function considers errors in predicted signs and the sizes and signs of the residuals in the model prediction simultaneously. Less weight is given to residuals with correctly predicted signs, while more weight is assigned to residuals with wrongly predicted signs. This is important because agents make decisions according to model predictions, especially the signs of the predictions. At the same time, larger residuals are penalized more and smaller residuals are penalized less. The signs of the residuals are considered in the loss function because they also affect decision-making processes. This paper proposes a new approach, termed “sign regression”, which takes these considerations into account. We show that ordinary least squares estimators generate better Sharpe ratios than sign regression does for most of the assets studied in this paper. However, sign regression can perform better for some assets. (Published at Journal of Investment Strategies, DOI: 10.21314/JOIS.2021.002)

Empirical fragility assessment using conditional GMPE-based ground shaking fields: application to damage data for 2016 Amatrice Earthquake

Recent earthquakes have exposed the vulnerability of existing buildings; this is demonstrated by damage incurred after moderate-to-high magnitude earthquakes. This stresses the need to exploit available data from different sources to develop reliable seismic risk components. As far as it regards empirical fragility assessment, accurate estimation of ground-shaking at the location of buildings of interest is as crucial as the accurate evaluation of observed damage for these buildings. This implies that explicit consideration of the uncertainties in the prediction of ground shaking leads to more robust empirical fragility curves. In such context, the simulation-based methods can be employed to provide fragility estimates that integrate over the space of plausible ground-shaking fields. These ground-shaking fields are generated according to the joint probability distribution of ground-shaking at the location of the buildings of interest considering the spatial correlation structure in the ground motion prediction residuals and updated based on the registered ground shaking data and observed damage. As an alternative to the embedded coefficients in the ground motion prediction equations accounting for subsoil categories, stratigraphic coefficients can be applied directly to the ground motion fields at the engineering bedrock level. Empirical fragility curves obtained using the observed damage in the aftermath of Amatrice Earthquake for residential masonry buildings show that explicit consideration of the uncertainty in the prediction of ground-shaking significantly affects the results.

Lossless Compression for Hyperspectral Images based on Adaptive Band Selection and Adaptive Predictor Selection

With the wide application of hyperspectral images, it becomes more and more important to compress hyperspectral images. Conventional recursive least squares (CRLS) algorithm has great potentiality in lossless compression for hyperspectral images. The prediction accuracy of CRLS is closely related to the correlations between the reference bands and the current band, and the similarity between pixels in prediction context. According to this characteristic, we present an improved CRLS with adaptive band selection and adaptive predictor selection (CRLS-ABS-APS). Firstly, a spectral vector correlation coefficient-based k-means clustering algorithm is employed to generate clustering map. Afterwards, an adaptive band selection strategy based on inter-spectral correlation coefficient is adopted to select the reference bands for each band. Then, an adaptive predictor selection strategy based on clustering map is adopted to select the optimal CRLS predictor for each pixel. In addition, a double snake scan mode is used to further improve the similarity of prediction context, and a recursive average estimation method is used to accelerate the local average calculation. Finally, the prediction residuals are entropy encoded by arithmetic encoder. Experiments on the Airborne Visible Infrared Imaging Spectrometer (AVIRIS) 2006 data set show that the CRLS-ABS-APS achieves average bit rates of 3.28 bpp, 5.55 bpp and 2.39 bpp on the three subsets, respectively. The results indicate that the CRLS-ABS-APS effectively improves the compression effect with lower computation complexity, and outperforms to the current state-of-the-art methods.

Unsupervised anomaly detection in peripheral venous pressure signals with hidden Markov models

This paper proposes an automatic anomaly detection and removal algorithm for peripheral venous pressure (PVP) signals, which can be used to predict intravascular volume loss in humans. PVP signal collection is a minimally invasive procedure that can be performed by using a standard peripheral intravenous (PIV) catheter and a commercial pressure-monitoring transducer. PVP signals are highly susceptible to motion and noise artifacts such as patient movements or unintended manipulation of PIV lines. Anomalies in PVP signals can corrupt useful information and seriously affect the integrity of PVP signal analysis. We propose to detect and remove such anomalies by exploiting the properties of PVP signals. Specifically, a dynamic linear model (DLM) with a Kalman filter is used to track and predict the time-domain evolution of PVP signals. The prediction residuals of the Kalman filter are then modeled with a hidden Markov model (HMM), with the normal and anomalous status of the signal modeled by using binary states of a hidden Markov chain. The HMM parameters along with the hidden states are iteratively estimated by using an unsupervised learning algorithm with a modified Baum–Welch method. The anomaly detection algorithm is applied to clinical data from a cohort of 24 pediatric patients with hypertrophic pyloric stenosis. Experimental results demonstrate that the proposed unsupervised anomaly detection algorithm can efficiently remove anomalies in PVP signals without the need of a training phase. The algorithm can also be applied to other time series signals, such as Electrocardiography (ECG) and Photoplethysmogram (PPG) signals.

Lifespan prediction of lithium-ion batteries based on various extracted features and gradient boosting regression tree model

Accurate battery lifespan prediction is critical for the quality evaluation and long-term planning of battery management systems. As battery degradation process is typically nonlinear, accurate early prediction of cycle life with significantly less degradation is extremely challenging. Approaches using machine learning techniques, which are mechanism-agnostic alternatives, to predict battery lifespan are therefore becoming more and more attractive. In this paper, a gradient boosting regression tree (GBRT) is proposed to model complex nonlinear battery dynamics and predict battery lifespan through various extracted battery features. Essentially, the GBRT works by constructing additional trees through minimizing the prediction residues from existing base models. It can identify complex and nonlinear end-to-end relationship and meanwhile provide relative importance for each input feature. Various potential features including voltage-related features, capacity-related features, and temperature-related features are constructed in both discharge time dimension and life cycle dimension and explored for effective lifespan prediction. Key hyper-parameters including learning rate, number of trees, and maximum number of splits are investigated for optimal GBRT prediction. Comparative studies confirm that the proposed method is significantly superior to other machine learning algorithms for battery lifespan prediction with limited data samples and various input features, with mean average percentage error around 7%.

Multi-sensor weighted fusion algorithm based on improved TOPSIS

In the process of track fusion of multi-sensor target tracking, multi-sensor weighted fusion is used to overcome the shortcoming of incomplete single information source. In order to change the weight of each sensor participating in fusion in real time according to the merits and demerits of sensor information adaptively, a method of sensor weight calculation based on improved TOPSIS is proposed. Firstly, the local estimation and global prediction residuals of each sensor are composed into residuals matrix, from which the ideal solution is found, and then the relative closeness degree is calculated according to the grey correlation distance between the local estimation residuals and the ideal solution, so as to screen the sensors with high weight to participate in the fusion. The simulation results show that the tracking error of the algorithm after fusion is significantly reduced compared with the local estimation error of each sensor, and the tracking error of multiple sensors for the target can be realized.

A hybrid deep learning-based method for short-term building energy load prediction combined with an interpretation process

Data driven-based building energy load prediction is of great value for building energy management tasks such as fault diagnosis and optimal control. However, there are two challenges for conventional data driven-based prediction methods. The first challenge is that time-lag measurements such as historical cooling loads still cannot be taken full advantage of. To deal with this challenge, a hybrid prediction method is proposed based on long short-term memory networks and artificial neural networks. The second challenge is that data driven-based models are hard to explain by domain knowledge. To deal with this challenge, an interpretation method is proposed based on a dimensionless sensitivity index and a weighted Manhattan distance. Operation data of a public building are utilized to evaluate the proposed methods. Results show that the proposed hybrid prediction method has higher prediction accuracy than conventional prediction methods in one-hour-ahead cooling load prediction. Crucial factors affecting building cooling loads are revealed successfully based on the proposed sensitivity index. Moreover, the weighted Manhattan distance is utilized to quantify the difference between predicted conditions and known conditions of training data. Results show that the prediction accuracy of data driven-based methods is reduced with the increase of the weighted Manhattan distance. It is further discovered that relationships between logarithmic prediction residuals and corresponding logarithmic weighted Manhattan distances are approximatively linear.