Irregular Sampling Research Articles

Enhancing continuous time series modelling with a latent ODE-LSTM approach

Due to their dynamic properties such as irregular sampling rate and high-frequency sampling, Continuous Time Series (CTS) are found in many applications. Since CTS with irregular sampling rate are difficult to model with standard Recurrent Neural Networks (RNNs), RNNs have been generalised to have continuous-time hidden dynamics defined by a Neural Ordinary Differential Equation (Neural ODE), leading to the ODE-RNN model. Another approach that provides a better modelling is that of the Latent ODE model, which constructs a continuous-time model where a latent state is defined at all times. The Latent ODE model uses a standard RNN as the encoder and a Neural ODE as the decoder. However, since the RNN encoder leads to difficulties with missing data and ill-defined latent variables, a Latent ODE-RNN model has recently been proposed that uses a ODE-RNN model as the encoder instead.Both the Latent ODE and Latent ODE-RNN models are difficult to train due to the vanishing and exploding gradients problem. To overcome this problem, the main contribution of this paper is to propose and illustrate a new model based on a new Latent ODE using an ODE-LSTM (Long Short-Term Memory) network as an encoder - the Latent ODE-LSTM model. To limit the growth of the gradients, the Norm Gradient Clipping strategy was embedded on the Latent ODE-LSTM model.The performance evaluation of the new Latent ODE-LSTM (with and without Norm Gradient Clipping) for modelling CTS with regular and irregular sampling rates is then demonstrated. Numerical experiments show that the new Latent ODE-LSTM performs better than Latent ODE-RNNs and can avoid the vanishing and exploding gradients during training.Code implementations developed in this work are available at github.com/CeciliaCoelho/LatentODELSTM.

Latent Stochastic Differential Equations for Modeling Quasar Variability and Inferring Black Hole Properties

Quasars are bright and unobscured active galactic nuclei (AGN) thought to be powered by the accretion of matter around supermassive black holes at the centers of galaxies. The temporal variability of a quasar’s brightness contains valuable information about its physical properties. The UV/optical variability is thought to be a stochastic process, often represented as a damped random walk described by a stochastic differential equation (SDE). Upcoming wide-field telescopes such as the Rubin Observatory Legacy Survey of Space and Time (LSST) are expected to observe tens of millions of AGN in multiple filters over a ten year period, so there is a need for efficient and automated modeling techniques that can handle the large volume of data. Latent SDEs are machine learning models well suited for modeling quasar variability, as they can explicitly capture the underlying stochastic dynamics. In this work, we adapt latent SDEs to jointly reconstruct multivariate quasar light curves and infer their physical properties such as the black hole mass, inclination angle, and temperature slope. Our model is trained on realistic simulations of LSST ten year quasar light curves, and we demonstrate its ability to reconstruct quasar light curves even in the presence of long seasonal gaps and irregular sampling across different bands, outperforming a multioutput Gaussian process regression baseline. Our method has the potential to provide a deeper understanding of the physical properties of quasars and is applicable to a wide range of other multivariate time series with missing data and irregular sampling.

Analyticity and supershift with irregular sampling

The notion of supershift generalizes that one of superoscillation and expresses the fact that the sampling of a function in an interval allows to compute the values of the function outside the interval. In a previous paper, we discussed the case in which the sampling of the function is regular and we are considering supershift in a bounded set, while here we investigate how irregularity in the sampling may affect the answer to the question of whether there is any relation between supershift and real analyticity on the whole real line. We show that the restriction to R\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\mathbb {R}$$\\end{document} of any entire function displays supershift, whereas the converse is, in general, not true. We conjecture that the converse is true as long as the sampling is regular, we discuss examples in support and we prove that the conjecture is indeed true for periodic functions.

Identification of Continuous-Time Systems With Irregular Samplings by Invariant-Subspace Based Method

Identification of Continuous-Time Systems With Irregular Samplings by Invariant-Subspace Based Method

Efficient anti-aliasing and anti-leakage Fourier transform for high-dimensional seismic data regularization using cube removal and GPU

Seismic data is commonly acquired sparsely and irregularly, which necessitates the regularization of seismic data with anti-aliasing and anti-leakage methods during seismic data processing. We propose a novel method of 4D anti-aliasing and anti-leakage Fourier transform using a cube-removal strategy to address the combination of irregular sampling and aliasing in high-dimensional seismic data. We compute a weighting function by stacking the spectrum along the radial lines, apply this function to suppress the aliasing energy, and then iteratively pick the dominant amplitude cube to construct the Fourier spectrum. The proposed method is very efficient due to a cube removal strategy for accelerating the convergence of Fourier reconstruction and a well-designed parallel architecture using CPU/GPU collaborative computing. To better fill the acquisition holes from 5D seismic data and meanwhile considering the GPU memory limitation, we developed the anti-aliasing and anti-leakage Fourier transform method in 4D with the remaining spatial dimension looped. The entire workflow is composed of three steps: data splitting, 4D regularization, and data merging. Numerical tests on both synthetic and field data examples demonstrate the high efficiency and effectiveness of our approach.

Conditional Generative Adversarial Network for Early Classification of Longitudinal Datasets Using an Imputation Approach

Early classification of longitudinal data remains an active area of research today. The complexity of these datasets and the high rates of missing data caused by irregular sampling present data-level challenges for the Early Longitudinal Data Classification (ELDC) problem. Coupled with the algorithmic challenge of optimising the opposing objectives of early classification (i.e., earliness and accuracy), ELDC becomes a non-trivial task. Inspired by the generative power and utility of the Generative Adversarial Network (GAN), we propose a novel context-conditional, longitudinal early classifier GAN (LEC-GAN). This model utilises informative missingness, static features and earlier observations to improve the ELDC objective. It achieves this by incorporating ELDC as an auxiliary task within an imputation optimization process. Our experiments on several datasets demonstrate that LEC-GAN outperforms all relevant baselines in terms of F1 scores while increasing the earliness of prediction.

CUTS+: High-Dimensional Causal Discovery from Irregular Time-Series

Causal discovery in time-series is a fundamental problem in the machine learning community, enabling causal reasoning and decision-making in complex scenarios. Recently, researchers successfully discover causality by combining neural networks with Granger causality, but their performances degrade largely when encountering high-dimensional data because of the highly redundant network design and huge causal graphs. Moreover, the missing entries in the observations further hamper the causal structural learning. To overcome these limitations, We propose CUTS+, which is built on the Granger-causality-based causal discovery method CUTS and raises the scalability by introducing a technique called Coarse-to-fine-discovery (C2FD) and leveraging a message-passing-based graph neural network (MPGNN). Compared to previous methods on simulated, quasi-real, and real datasets, we show that CUTS+ largely improves the causal discovery performance on high-dimensional data with different types of irregular sampling.

Deep learning for robust forecasting of hot metal silicon content in a blast furnace

AbstractThe hot metal silicon content is a key indicator of the thermal state in the blast furnace and it needs to be kept within a pre-defined range in order to ensure efficient operations. Effective monitoring of silicon content is challenging due to the harsh environment in the furnace and irregularly sampled measurements. Data-driven approaches have been proposed in the literature to predict silicon content using process data and overcome the sparsity of silicon content measurements. However, these approaches rely on the selection of hand-crafted features and ad hoc interpolation methods to deal with irregular sampling of the process variables, adding complexity to model training and optimisation, and requiring significant effort when tuning the model over time to keep it to the required level of accuracy. This paper proposes an improved framework for the prediction of silicon content using a novel deep learning approach based on Phased LSTM. The model has been trained using 3 years of data and validated over a 1-year period using a robust walk-forward validation method, therefore providing confidence in the model performance over time. The Phased LSTM model outperforms competing approaches due to its in-built ability to learn from event-based sequences and scalability for real-world deployments. This is the first time that Phased LSTM has been applied to real-world datasets and results suggest that the ability to learn from event-based data can be beneficial for the process industry where event-driven signals from multiple sensors are common.

Brain Tumor Detection System using Deep Learning

This project's objectives include locating brain tumours and enhancing patient care. Tumours are abnormal cell growths, and malignant tumours are abnormal cell growths. The two types of scans, CT and MRI frequently detect infected brain tissues. Numerous more techniques are employed for the diagnosis of brain tumours, some of which include molecular testing, and positive charges imaging of blood or lymph arteries. In order to identify disease causes like tumors, this article will use various MRI pictures. This study paper's major goals are to 1) recognize irregular sample photos and 2) locate the tumor region. In order to administer the appropriate therapy, the aberrant portions of the photographs will anticipate the levels of tumours. From example photos, deep learning is utilized to identify anomalous areas. The aberrant section will be segmented in this study using VGG-16. The number of pixels that are malignant determines the extent of the contaminated area.

Retrieving Similar Trajectories from Cellular Data of Multiple Carriers at City Scale

Retrieving similar trajectories aims to search for the trajectories that are close to a query trajectory in spatio-temporal domain from a large trajectory dataset. This is critical for a variety of applications, like transportation planning and mobility analysis. Unlike previous studies that perform similar trajectory retrieval on fine-grained GPS data or single cellular carrier, we investigate the feasibility of finding similar trajectories from cellular data of multiple carriers, which provide more comprehensive coverage of population and space. To handle the issues of spatial bias of cellular data from multiple carriers, coarse spatial granularity, and irregular sparse temporal sampling, we develop a holistic system cellSim . Specifically, to avoid the issue of spatial bias, we first propose a novel map matching approach, which transforms the cell tower sequences from multiple carriers to routes on a unified road map. Then, to address the issue of temporal sparse sampling, we generate multiple routes with different confidences to increase the probability of finding truly similar trajectories. Finally, a new trajectory similarity measure is developed for similar trajectory search by calculating the similarities between the irregularly-sampled trajectories. Extensive experiments on a large-scale cellular dataset from two carriers and real-world 1,701 km query trajectories reveal that cellSim provides state-of-the-art performance for similar trajectory retrieval.

Flexible Surface-Enhanced Raman Scattering Tape Based on Ag Nanostructured Substrate for On-Site Analyte Detection.

Surface-enhanced Raman scattering (SERS) has emerged as a powerful surface analytical technique that amplifies Raman scattering signals of molecules adsorbed onto metal nanostructured surfaces. The droplet reaction method has recently been employed to fabricate large-scale microring patterns of silver (Ag) nanostructures on rigid substrates, which enables sensitive detection within the ring area. However, these rigid substrates present limitations for direct on-site detection of analyte residues on irregular sample surfaces. There is a need to develop soft and flexible SERS substrates that can intimately conform to arbitrary surfaces. In this study, we presented a SERS substrate using flexible and adhesive tape as the supporting material. This SERS tape was fabricated by repeatedly transferring presynthesized Ag nanostructures from a rigid substrate to the tape. For a model compound adenine, our SERS tape exhibited a good linear response from 5 × 10-4 M to 5 × 10-5 M with a low limit of detection (LOD) of 5 × 10-7 M and displayed a SERS enhancement factor (EF) of 3.2 × 105. The relative standard deviation (RSD) of SERS intensity achieved was as low as 1.93%, indicating its outstanding uniformity. The as-prepared SERS tape was used for in situ detection of pesticide residue on an apple surface and dye residue on human hair. Leveraging the large surface area of Ag nanostructure patterns from the droplet reaction, the developed SERS tape demonstrates excellent performance in terms of sensitivity and uniformity. The successful detection of analyte residues on arbitrary surfaces of apple and human hair highlights the potential of this flexible SERS tape for real-world applications across various industries for enhanced diagnostic accuracy.

Functional linear model with prior information of subjects’ network

In many modern applications, data samples are interconnected by a network, and network information is a crucial factor in forecasting. However, existing network data analysis methods, which are designed for scalar data, are not effective for infinite-dimensional function data, particularly when functional predictors are observed on an irregular sampling design. In this paper, we propose a functional linear model for network-linked data. To improve the estimation and prediction, the network cohesion is enforced using the Laplace quadratic penalty function. The statistical properties of the proposed model are studied, and an extension to high-dimensional functional data is developed to simultaneously select relevant functional predictors and estimate the coefficient functions. Simulation results and real data application demonstrate the satisfactory performance of the proposed methods.

A metastable RNG using interleaved NAND- and NOR-based TEROs

ABSTRACT This study introduces a novel approach to address the limitations observed in existing random number generator (RNG) designs, focusing on the resolution for insufficiently covered problem statements and study purposes. In response to these challenges, we propose a high-throughput metastable-based RNG tailored for cryptography applications. Our design is featured with a two-layer transient effect ring oscillator (TERO) architecture, meticulously crafted to mitigate the locking phenomenon and enhance RNG randomness. Leveraging interleaved two-layer TEROs, realised through NAND and NOR SR latches, our RNG achieves unpredictable irregular sampling, crucial for cryptographic applications. To further enhance irregular sampling, the first layer of the RNG employs a multiplexer that selects inputs randomly. This multiplexer is governed by selection signals generated by the second layer of the RNG, ensuring superior randomness. Implemented using TSMC 40-nm CMOS process, our RNG passed the FIPS 140–1 randomness test at a clock frequency of 100 MHz, boasting a remarkable throughput of 50 Mbps. This study represents a significant advancement in RNG technology, addressing previous shortcomings and paving the way for high-speed communication systems’ secure cryptographic applications.

Analysis of Key Influencing Factors of Track Dynamic Irregularity Induced by Earthquakes in the High-Speed Railway Track–Bridge System

Stiffness degradation of track–bridge systems under earthquake excitations can lead to track dynamic irregularity, impacting the safety of train operations post-earthquake. In this paper, the nonlinear time-history analysis of CRTS II track–bridge system model established by ANSYS finite element analysis software is carried out under the action of transverse random earthquake. The train–track–bridge system model considering the stiffness degradation caused by earthquake is established by MATLAB, and the earthquake-induced track dynamic irregularity sample library is constructed. The probability distribution mode of the power spectral density sample of the earthquake-induced track dynamic irregularity is explored and the calculation method based on the probability guarantee rate is proposed. The influence of structural damping ratio, train running speed and train type on the power spectral density curve of the earthquake-induced track dynamic irregularity is analyzed. The results show that the power spectrum samples of track dynamic irregularity conform to the hypothesis test of a normal distribution. The power spectral density of earthquake-induced dynamic irregularity primarily consists of medium and low-frequency components. When the structural damping ratio increases from 0.03 to 0.04, 0.04 to 0.05, 0.05 to 0.06, and 0.06 to 0.07, the variation gradients are 0.1701, 0.1240, 0.1034 and 0.0999, respectively, which indicates that the structural damping ratio has a significant effect on the power spectral density of earthquake-induced irregularity, and the impacts of train speed and train type on the power spectral density of near-fault earthquake-induced irregularity are minimal.

A Bayesian framework for modeling COVID-19 case numbers through longitudinal monitoring of SARS-CoV-2 RNA in wastewater.

Wastewater-based surveillance has become an important tool for research groups and public health agencies investigating and monitoring the COVID-19 pandemic and other public health emergencies including other pathogens and drug abuse. While there is an emerging body of evidence exploring the possibility of predicting COVID-19 infections from wastewater signals, there remain significant challenges for statistical modeling. Longitudinal observations of viral copies in municipal wastewater can be influenced by noisy datasets and missing values with irregular and sparse samplings. We propose an integrative Bayesian framework to predict daily positive cases from weekly wastewater observations with missing values via functional data analysis techniques. In a unified procedure, the proposed analysis models severe acute respiratory syndrome coronavirus-2 RNA wastewater signals as a realization of a smooth process with error and combines the smooth process with COVID-19 cases to evaluate the prediction of positive cases. We demonstrate that the proposed framework can achieve these objectives with high predictive accuracies through simulated and observed real data.

A Cloud-Edge Collaborative Soft Sensing Framework for Multi-Performance Indicators of Manufacturing Processes with Irregular Sampling

A Cloud-Edge Collaborative Soft Sensing Framework for Multi-Performance Indicators of Manufacturing Processes with Irregular Sampling

Structure determination using high-order spatial correlations in single-particle X-ray scattering.

Single-particle imaging using X-ray free-electron lasers (XFELs) is a promising technique for observing nanoscale biological samples under near-physiological conditions. However, as the sample's orientation in each diffraction pattern is unknown, advanced algorithms are required to reconstruct the 3D diffraction intensity volume and subsequently the sample's density model. While most approaches perform 3D reconstruction via determining the orientation of each diffraction pattern, a correlation-based approach utilizes the averaged spatial correlations of diffraction intensities over all patterns, making it well suited for processing experimental data with a poor signal-to-noise ratio of individual patterns. Here, a method is proposed to determine the 3D structure of a sample by analyzing the double, triple and quadruple spatial correlations in diffraction patterns. This ab initio method can reconstruct the basic shape of an irregular unsymmetric 3D sample without requiring any prior knowledge of the sample. The impact of background and noise on correlations is investigated and corrected to ensure the success of reconstruction under simulated experimental conditions. Additionally, the feasibility of using the correlation-based approach to process incomplete partial diffraction patterns is demonstrated. The proposed method is a variable addition to existing algorithms for 3D reconstruction and will further promote the development and adoption of XFEL single-particle imaging techniques.

Cross-modal missing time-series imputation using dense spatio-temporal transformer nets.

Due to irregular sampling or device failure, the data collected from sensor network has missing value, that is, missing time-series data occurs. To address this issue, many methods have been proposed to impute random or non-random missing data. However, the imputation accuracy of these methods are not accurate enough to be applied, especially in the case of complete data missing (CDM). Thus, we propose a cross-modal method to impute time-series missing data by dense spatio-temporal transformer nets (DSTTN). This model embeds spatial modal data into time-series data by stacked spatio-temporal transformer blocks and deployment of dense connections. It adopts cross-modal constraints, a graph Laplacian regularization term, to optimize model parameters. When the model is trained, it recovers missing data finally by an end-to-end imputation pipeline. Various baseline models are compared by sufficient experiments. Based on the experimental results, it is verified that DSTTN achieves state-of-the-art imputation performance in the cases of random and non-random missing. Especially, the proposed method provides a new solution to the CDM problem.

De-cluttering Scatterplots with Integral Images.

Scatterplots provide a visual representation of bivariate data (or 2D embeddings of multivariate data) that allows for effective analyses of data dependencies, clusters, trends, and outliers. Unfortunately, classical scatterplots suffer from scalability issues, since growing data sizes eventually lead to overplotting and visual clutter on a screen with a fixed resolution, which hinders the data analysis process. We propose an algorithm that compensates for irregular sample distributions by a smooth transformation of the scatterplot's visual domain. Our algorithm evaluates the scatterplot's density distribution to compute a regularization mapping based on integral images of the rasterized density function. The mapping preserves the samples' neighborhood relations. Few regularization iterations suffice to achieve a nearly uniform sample distribution that efficiently uses the available screen space. We further propose approaches to visually convey the transformation that was applied to the scatterplot and compare them in a user study. We present a novel parallel algorithm for fast GPU-based integral-image computation, which allows for integrating our de-cluttering approach into interactive visual data analysis systems.

An Efficient Dynamic Auto-Regressive CCA for Time Series Imputation With Irregular Sampling

System dynamics are inevitable in industrial processes due to factors such as ambient disturbances and controller tuning. Accurate modeling of these dynamics are of key importance for subsequent process analysis and anomaly detection, and dynamic latent variable methods are widely adopted since they retain good interpretability. However, only dynamic cross-correlations are modeled in existing methods, leaving a large portion of quality information unexploited. In this work, an efficient dynamic auto-regressive canonical correlation analysis (EDACCA) method is proposed with a modified auto-regressive exogenous model to extract dynamics in both auto-correlations and cross-correlations. The flexibility and efficiency of EDACCA are improved with the design of weighting parameters and the economic singular value decomposition. EDACCA is further adapted for multi-step ahead (MS) prediction and missing data imputation. Two industrial processes are employed to evaluate the prediction performance and imputation performance of EDACCA. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —Different sampling rates are usually set for process and quality variables in industrial processes, which leads to less quality samples. Meanwhile, system dynamics are not fully exploited for dynamic predictive modeling in most existing algorithms. The focus of this study is to develop a customized data imputation method for different data volume of process and quality data. An efficient dynamic auto-regressive canonical correlation analysis (EDACCA) is designed to extract temporal relations between process and quality variables, which is also adapted for multi-step-ahead prediction purpose. An EDACCA based data imputation method is also proposed to impute incomplete data caused by irregular sampling rates.