Irregular Sampling Rates Research Articles

Computationally Efficient Inference via Time-Aware Modular Control Systems

Control in multi-agent decision-making systems is an important issue with a wide variety of existing approaches. In this work, we offer a new comprehensive framework for distributed control. The main contributions of this paper are summarized as follows. First, we propose PHIMEC (physics-informed meta control)—an architecture for learning optimal control by employing a physics-informed neural network when the state space is too large for reward-based learning. Second, we offer a way to leverage impulse response as a tool for system modeling and control. We propose IMPULSTM, a novel approach for incorporating time awareness into recurrent neural networks designed to accommodate irregular sampling rates in the signal. Third, we propose DIMAS, a modular approach to increasing computational efficiency in distributed control systems via domain-knowledge integration. We analyze the performance of the first two contributions on a set of corresponding benchmarks and then showcase their combined performance as a domain-informed distributed control system. The proposed approaches show satisfactory performance both individually in their respective applications and as a connected system.

Enhancing continuous time series modelling with a latent ODE-LSTM approach

Due to their dynamic properties such as irregular sampling rate and high-frequency sampling, Continuous Time Series (CTS) are found in many applications. Since CTS with irregular sampling rate are difficult to model with standard Recurrent Neural Networks (RNNs), RNNs have been generalised to have continuous-time hidden dynamics defined by a Neural Ordinary Differential Equation (Neural ODE), leading to the ODE-RNN model. Another approach that provides a better modelling is that of the Latent ODE model, which constructs a continuous-time model where a latent state is defined at all times. The Latent ODE model uses a standard RNN as the encoder and a Neural ODE as the decoder. However, since the RNN encoder leads to difficulties with missing data and ill-defined latent variables, a Latent ODE-RNN model has recently been proposed that uses a ODE-RNN model as the encoder instead.Both the Latent ODE and Latent ODE-RNN models are difficult to train due to the vanishing and exploding gradients problem. To overcome this problem, the main contribution of this paper is to propose and illustrate a new model based on a new Latent ODE using an ODE-LSTM (Long Short-Term Memory) network as an encoder - the Latent ODE-LSTM model. To limit the growth of the gradients, the Norm Gradient Clipping strategy was embedded on the Latent ODE-LSTM model.The performance evaluation of the new Latent ODE-LSTM (with and without Norm Gradient Clipping) for modelling CTS with regular and irregular sampling rates is then demonstrated. Numerical experiments show that the new Latent ODE-LSTM performs better than Latent ODE-RNNs and can avoid the vanishing and exploding gradients during training.Code implementations developed in this work are available at github.com/CeciliaCoelho/LatentODELSTM.

An Efficient Dynamic Auto-Regressive CCA for Time Series Imputation With Irregular Sampling

System dynamics are inevitable in industrial processes due to factors such as ambient disturbances and controller tuning. Accurate modeling of these dynamics are of key importance for subsequent process analysis and anomaly detection, and dynamic latent variable methods are widely adopted since they retain good interpretability. However, only dynamic cross-correlations are modeled in existing methods, leaving a large portion of quality information unexploited. In this work, an efficient dynamic auto-regressive canonical correlation analysis (EDACCA) method is proposed with a modified auto-regressive exogenous model to extract dynamics in both auto-correlations and cross-correlations. The flexibility and efficiency of EDACCA are improved with the design of weighting parameters and the economic singular value decomposition. EDACCA is further adapted for multi-step ahead (MS) prediction and missing data imputation. Two industrial processes are employed to evaluate the prediction performance and imputation performance of EDACCA. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —Different sampling rates are usually set for process and quality variables in industrial processes, which leads to less quality samples. Meanwhile, system dynamics are not fully exploited for dynamic predictive modeling in most existing algorithms. The focus of this study is to develop a customized data imputation method for different data volume of process and quality data. An efficient dynamic auto-regressive canonical correlation analysis (EDACCA) is designed to extract temporal relations between process and quality variables, which is also adapted for multi-step-ahead prediction purpose. An EDACCA based data imputation method is also proposed to impute incomplete data caused by irregular sampling rates.

Toward Learning Joint Inference Tasks for IASS-MTS Using Dual Attention Memory With Stochastic Generative Imputation.

Irregularly, asynchronously and sparsely sampled multivariate time series (IASS-MTS) are characterized by sparse and uneven time intervals and nonsynchronous sampling rates, posing significant challenges for machine learning models to learn complex relationships within and beyond IASS-MTS to support various inference tasks. The existing methods typically either focus solely on single-task forecasting or simply concatenate them through a separate preprocessing imputation procedure for the subsequent classification application. However, these methods often ignore valuable annotated labels or fail to discover meaningful patterns from unlabeled data. Moreover, the approach of separate prefilling may introduce errors due to the noise in raw records, and thus degrade the downstream prediction performance. To overcome these challenges, we propose the time-aware dual attention and memory-augmented network (DAMA) with stochastic generative imputation (SGI). Our model constructs a joint task learning architecture that unifies imputation and classification tasks collaboratively. First, we design a new time-aware DAMA that accounts for irregular sampling rates, inherent data nonalignment, and sparse values in IASS-MTS data. The proposed network integrates both attention and memory to effectively analyze complex interactions within and across IASS-MTS for the classification task. Second, we develop the stochastic generative imputation (SGI) network that uses auxiliary information from sequence data for inferring the time series missing observations. By balancing joint tasks, our model facilitates interaction between them, leading to improved performance on both classification and imputation tasks. Third, we evaluate our model on real-world datasets and demonstrate its superior performance in terms of imputation accuracy and classification results, outperforming the baselines.

Leveraging neural differential equations and adaptive delayed feedback to detect unstable periodic orbits based on irregularly sampled time series.

Detecting unstable periodic orbits (UPOs) based solely on time series is an essential data-driven problem, attracting a great deal of attention and arousing numerous efforts, in nonlinear sciences. Previous efforts and their developed algorithms, though falling into a category of model-free methodology, dealt with the time series mostly with a regular sampling rate. Here, we develop a data-driven and model-free framework for detecting UPOs in chaotic systems using the irregularly sampled time series. This framework articulates the neural differential equations (NDEs), a recently developed and powerful machine learning technique, with the adaptive delayed feedback (ADF) technique. Since the NDEs own the exceptional capability of accurate reconstruction of chaotic systems based on the observational time series with irregular sampling rates, UPOs detection in this scenario could be enhanced by an integration of the NDEs and the ADF technique. We demonstrate the effectiveness of the articulated framework on representative examples.

A multi-task Gaussian process self-attention neural network for real-time prediction of the need for mechanical ventilators in COVID-19 patients

ObjectiveThe Coronavirus Disease 2019 (COVID-19) pandemic has overwhelmed the capacity of healthcare resources and posed a challenge for worldwide hospitals. The ability to distinguish potentially deteriorating patients from the rest helps facilitate reasonable allocation of medical resources, such as ventilators, hospital beds, and human resources. The real-time accurate prediction of a patient's risk scores could also help physicians to provide earlier respiratory support for the patient and reduce the risk of mortality. MethodsWe propose a robust real-time prediction model for the in-hospital COVID-19 patients' probability of requiring mechanical ventilation (MV). The end-to-end neural network model incorporates the Multi-task Gaussian Process to handle the irregular sampling rate in observational data together with a self-attention neural network for the prediction task. ResultsWe evaluate our model on a large database with 9,532 nationwide in-hospital patients with COVID-19. The model demonstrates significant robustness and consistency improvements compared to conventional machine learning models. The proposed prediction model also shows performance improvements in terms of area under the receiver operating characteristic curve (AUROC) and area under the precision-recall curve (AUPRC) compared to various deep learning models, especially at early times after a patient's hospital admission. ConclusionThe availability of large and real-time clinical data calls for new methods to make the best use of them for real-time patient risk prediction. It is not ideal for simplifying the data for traditional methods or for making unrealistic assumptions that deviate from observation's true dynamics. We demonstrate a pilot effort to harmonize cross-sectional and longitudinal information for mechanical ventilation needing prediction.

Detecting the periodicity of highly irregularly sampled light curves with Gaussian processes: the case of SDSS J025214.67−002813.7

ABSTRACT Based on a 20-yr-long multiband observation of its light curve, it was conjectured that the quasar SDSS J025214.67−002813.7 has a periodicity of ∼4.4 yr. These observations were acquired at a highly irregular sampling rate and feature long intervals of missing data. In this setting, the inference over the light curve’s spectral content requires, in addition to classic Fourier methods, a proper model of the probability distribution of the missing observations. In this article, we address the detection of the periodicity of a light curve from partial and irregularly sampled observations using Gaussian processes, a Bayesian non-parametric model for time series. This methodology allows us to evaluate the veracity of the claimed periodicity of the above-mentioned quasar and also to estimate its power spectral density. Our main contribution is the confirmation that considering periodic component definitely improves the modelling of the data, although being the source originally selected by a large sample of objects, the possibility that this is a chance result cannot be ruled out.

SCONE: Supernova Classification with a Convolutional Neural Network

Abstract We present a novel method of classifying Type Ia supernovae using convolutional neural networks, a neural network framework typically used for image recognition. Our model is trained on photometric information only, eliminating the need for accurate redshift data. Photometric data is preprocessed via 2D Gaussian process regression into two-dimensional images created from flux values at each location in wavelength-time space. These “flux heatmaps” of each supernova detection, along with “uncertainty heatmaps” of the Gaussian process uncertainty, constitute the data set for our model. This preprocessing step not only smooths over irregular sampling rates between filters but also allows SCONE to be independent of the filter set on which it was trained. Our model has achieved impressive performance without redshift on the in-distribution SNIa classification problem: 99.73 ± 0.26% test accuracy with no over/underfitting on a subset of supernovae from PLAsTiCC’s unblinded test data set. We have also achieved 98.18 ± 0.3% test accuracy performing six-way classification of supernovae by type. The out-of-distribution performance does not fully match the in-distribution results, suggesting that the detailed characteristics of the training sample in comparison to the test sample have a big impact on the performance. We discuss the implication and directions for future work. All of the data processing and model code developed for this paper can be found in the SCONE software package located at github.com/helenqu/scone.

Design And Analysis Of An Efficientbit Based Object Detection

Video understanding can be viewed with useful contextual information in static cameras beyond a few seconds. Subjects may conduct similarly over a number of days and background objects remain static. The frequency of sampling is low, often less than a frame per second, and occasionally irregular because of the power and storage limitation of the motion trigger. If they are to be effective in this setting, models must be robust to irregular sampling rates. Users have developed a new range of EfficientDet Object detectors based on these optimizations and better backbones to improve efficiency over many resources compared to state-of-the-art. CentreNet is the highest speed-precision disruption in MS COCO at 28,1% CA for 142 FPS, 37,4% AP for 52 FPS and 45,1% AP for multi-scale tests with 1.4 FPS. We use the same approach to estimate the 3D border box in the KITTI benchmark and human position in the COCO keyboard dataset. With sophisticated multi-stage methods, our method works competitively and runs in real-time.

Activities of Daily Living Recognition With Binary Environment Sensors Using Deep Learning: A Comparative Study

The power of end-to-end deep learning techniques to automatically learn latent high-level features from raw signals has been demonstrated in numerous application fields, however, few studies systematically investigate how to properly encode the time-series firings of binary environment sensors that typically work in an event-triggering scheme and have irregular sampling rates for in-home human activity recognition. To this end, we here propose two different methods to process the streaming sensor readings and accordingly evaluate their combinations with deep learning models. Specifically, we divide the multichannel sensor events into segments and encode each segment into either a vector or two-dimensional matrix. Particularly, three different feature representations are presented for the vector form. Afterwards, we combine the encoded features with four typical deep learning models and optimize corresponding activity recognizers to study their sensitivity to different feature encodings. Furthermore, we include seven commonly used shallow classification models for comparison purposes. Finally, we conduct extensive experiments on three publicly available smart home datasets. Results indicate that the performance of both deep learning and shallow models is closely associated with the raw signal encodings and demonstrate the superiority of one-dimensional convolutional neural networks over its competitors in terms of generalization across scenarios. Besides, we preliminarily explore the influence of NULL class on an activity recognizer and experimentally show its negative impact on overall accuracy, enlightening relevant studies to consider it in developing a practical activity recognition system.

Testing the disk instability model of cataclysmic variables

Context. The disk instability model (DIM) attributes the outbursts of dwarf novae to a thermal-viscous instability of their accretion disk, an instability to which nova-like stars are not subject. Aims. We aim to test the fundamental prediction of the DIM: the separation of cataclysmic variables (CVs) into nova-likes and dwarf novae depending on orbital period and mass transfer rate from the companion. Methods. We analyzed the light curves from a sample of ≈130 CVs with a parallax distance in the Gaia DR2 catalog to derive their average mass transfer rate. We validated the method for converting optical magnitude to mass accretion rate against theoretical light curves of dwarf novae. Results. Dwarf novae (resp. nova-likes) are consistently placed in the unstable (resp. stable) region of the orbital period – mass transfer rate plane predicted by the DIM. None of the analyzed systems present a challenge to the model. These results are robust against the possible sources of error and bias that we investigated. Light curves from Kepler or, in the future, the LSST or Plato surveys, could alleviate a major source of uncertainty, that is, the irregular sampling rate of the light curves, assuming good constraints can be set on the orbital parameters of the CVs that they happen to target. Conclusions. The disk instability model remains the solid basis on which to construct an understanding of accretion processes in CVs.

Incorporating Telemetry Error into Hidden Markov Models of Animal Movement Using Multiple Imputation

When data streams are observed without error and at regular time intervals, discrete-time hidden Markov models (HMMs) have become immensely popular for the analysis of animal location and auxiliary biotelemetry data. However, measurement error and temporally irregular data are often pervasive in telemetry studies, particularly in marine systems. While relatively small amounts of missing data that are missing-completely-at-random are not typically problematic in HMMs, temporal irregularity can result in few (if any) observations aligning with the regular time steps required by HMMs. Fitting HMMs that explicitly account for uncertainty attributable to location measurement error, temporally irregular observations, or other forms of missing data typically requires computationally demanding techniques, such as Markov chain Monte Carlo (MCMC). Using simulation and a real-world bearded seal (Erignathus barbatus) example, I investigate a practical alternative to incorporating measurement error and temporally irregular observations into HMMs based on multiple imputation of the position process drawn from a single-state continuous-time movement model. This two-stage approach is relatively simple, performed with existing software using efficient maximum likelihood methods, and completely parallelizable. I generally found the approach to perform well across a broad range of simulated measurement error and irregular sampling rates, with latent states and locations reliably recovered in nearly all simulated scenarios. However, high measurement error coupled with low sampling rates often induced bias in both the estimated probability distributions of data streams derived from the imputed position process and the estimated effects of spatial covariates on state transition probabilities. Results from the two-stage analysis of the bearded seal data were similar to a more computationally intensive single-stage MCMC analysis, but the two-stage analysis required much less computation time and no custom model-fitting algorithms. I thus found the two-stage multiple-imputation approach to be promising in terms of its ease of implementation, computation time, and performance. Code for implementing the approach using the R package “momentuHMM” is provided.Supplementary materials accompanying this paper appear online.

Kalman filtering approach to multi-rate information fusion in the presence of irregular sampling rate and variable measurement delay

State estimation for a system with irregular rate and delayed measurements is studied using fusion Kalman filter. Lab data in process plants is usually more accurate compared to other measurements. However, it is often slow rate and subject to variable delay and irregularity in sampling time. Fast rate state estimation can be conducted using fast rate measurement, while the slow rate lab data can be used to improve the accuracy of estimation whenever it is available. For this purpose, two Kalman filters are used to estimate the states based on each type of measurement. The estimates are fused in the next step by considering the correlation between them. An iterative algorithm to obtain the cross-covariance matrix between the estimation errors of the two Kalman filters is presented and employed in the fusion process. The improvement on the accuracy of estimation and comparison with other optimal fusion state estimation techniques are discussed through a simulation example, a pilot-scale experiment and an industrial case study.

Limits of Performance in Systems with Periodic Irregular Sampling and Actuation Rates

This paper examines the limits of performance in systems with periodic irregular sampling rate when the actuation is not necessarily synchronized with the sampling. For such a system, three sampling and actuation schemes are considered: when the sampling and control rate are both regular, when they are both irregular, and when the sampling rate is irregular while the control rate is regular. To ascertain the limits of performance of this type of systems under each sampling and actuation scheme, the system is modeled as a linear periodically time-varying (LPTV) system; optimal LQG control design with a variance constraint is applied to find the smallest achievable mean variance of the performance signal subject to a constraint on the mean control effort variance. In addition, to deal with the computational delay of the controller, an innovative discretization method is proposed which does not introduce any extra states into the state space model. The proposed method is exploited to determine the performance of a hard disk drive (HDD) in track-following mode. A simulation study demonstrates that in the presence of 30% irregularity in sampling time, using the irregular sampling and regular control action scheme for the HDD achieves an RMS 3s value of the position error signal (PES) that is 40% smaller than the corresponding value achieved by using a controller provided by our industry partner, in which both the sampling and control rates are irregular.

Optimal $H_{\infty}$ Control Design and Implementation of Hard Disk Drives With Irregular Sampling Rates

Missing position error signal sampling data in hard disk drives (HDDs) results in their servo systems having irregular sampling rates (ISRs). With the natural periodicity of HDDs, which is related to the disk rotation, HDD servo systems with ISRs can be modeled as linear periodically time-varying systems. Based on our previous results, an explicit minimum entropy <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">H</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</sub> controller for HDD servos with ISRs via discrete Riccati equations is first obtained. The resulting controller is subsequently demonstrated to be periodically time-varying and implementable. Simulation and experimental studies, which have been carried out on a set of hard disk drives, demonstrate that the proposed control synthesis technique is able to handle ISRs and can be used to conveniently design a loop-shaping track-following servo that achieves the robust performance of a desired error rejection function for disturbance attenuation. Experimental results on ten actual 2.5" hard disk drives show around 27% improvement of the 3σ PES was obtained by the proposed control algorithm.

Track-Following Control Design and Implementation for Hard Disk Drives with Missing Samples

AbstractMissing position-error-signal sampling data in hard disk drives (HDDs) results in their servo systems having irregular sampling rates. Due to the natural periodicity of HDDs, which is related to the disk rotation, HDD servo systems with missing samples can be modeled as linear periodically time-varying (LPTV) systems. Based on previous H∞ control synthesis results for general LPTV systems developed by some of the authors, an optimal H∞ track-following control algorithm for HDD servo systems with missing samples is proposed. A simulation and implementation study, which has been carried out on a set of hard disk drives, demonstrates that the proposed control synthesis algorithm is able to handle missing samples and can be used to achieve the robust performance of a desired error rejection function for disturbance attenuation. Implementation results on multiple real hard disk drives validate the predicted robust performance.

Quadrature prefilters for the discrete wavelet transform

Discrepancies between the discrete wavelet transform and the coefficients of the wavelet series are known to be reducible by initialization of input data. Prefilters based on Lagrange interpolants are derived here for biorthogonal compact support wavelet systems, providing exact subspace projection in cases of local polynomial smoothness. The resulting divergence acceleration in a nonpolynomial test case is examined. Irregular sampling rates are also accommodated.