Drift Estimation Research Articles

Traffic Flow Outlier Detection for Smart Mobility Using Gaussian Process Regression Assisted Stochastic Differential Equations

Current methods for detecting outliers in traffic streaming data often struggle to capture real-time dynamic changes in traffic conditions and differentiate between genuine changes and anomalies. This study proposes a novel approach to outlier detection in traffic streaming data that effectively addresses stochasticity and uncertainty in observations. The proposed method utilizes Stochastic Differential Equations (SDEs) and Gaussian Process Regression (GPR). By employing SDEs, we can capture drift and diffusion estimates in traffic streaming data, providing a more comprehensive modeling of the data generation process. Integrating GPR allows precise Bayesian posterior inferences for outlier detection within the SDE framework. To improve practicality, we introduce a flexible threshold-setting mechanism using statistical testing to control the false positive rate. This adaptability helps strike a balance between model fitting and complexity in outlier detection. Compared to traditional SDE-based methods, our SDE-GPR outlier detection method demonstrates enhanced robustness and better adaptability to the complexities of traffic systems. This is evidenced through an empirical study using time series data collected in California, USA. Overall, this study introduces a more advanced and accurate approach to outlier detection in traffic streaming data, paving the way for improved real-time traffic condition monitoring and management.

Power utility maximization with expert opinions at fixed arrival times in a market with hidden Gaussian drift

AbstractIn this paper we study optimal trading strategies in a financial market in which stock returns depend on a hidden Gaussian mean reverting drift process. Investors obtain information on that drift by observing stock returns. Moreover, expert opinions in the form of signals about the current state of the drift arriving at fixed and known dates are included in the analysis. Drift estimates are based on Kalman filter techniques. They are used to transform a power utility maximization problem under partial information into an optimization problem under full information where the state variable is the filter of the drift. The dynamic programming equation for this problem is studied and closed-form solutions for the value function and the optimal trading strategy of an investor are derived. They allow to quantify the monetary value of information delivered by the expert opinions. We illustrate our theoretical findings by results of extensive numerical experiments.

Accelerometer static state detection (SSD)-assisted GNSS/accelerometer bridge monitoring algorithm

Abstract In the field of structural health monitoring (SHM), a loosely coupled (LC) Kalman filtering algorithm that accounts for baseline drift errors is commonly used to integrate GNSS data with accelerometer data. In the LC algorithm, the baseline drift errors are considered unknown parameters that need to be estimated. In scenario of continuous float solutions, the estimation of baseline drift error is often inaccurate, leading to the divergence of monitoring results. Theoretically, as a type of motion sensor, accelerometers are expected to qualitatively determine the priori state of bridges, whether dynamic or static. Utilizing the inherent characteristics of accelerometers and the principle of zero-velocity detection in integrated navigation, we originally propose a bridge static state detection (SSD) method based on low-cost accelerometer, and introduces this prior SSD information as a constraint in GNSS/accelerometer LC algorithm, called SSD-LC bridge monitoring algorithm. Through a simulation platform and real-world bridge monitored tests, the effectiveness of our proposed SSD method has been verified. Furthermore, our proposed SSD-LC bridge monitoring algorithm can effectively mitigate the divergence problem in baseline drift estimation that occurs with continuous GNSS float solutions in traditional algorithms, which can effectively avoid misjudgments and false alarms in bridge monitoring during GNSS anomalies.

Generalized transport characterizations for short oceanic internal waves in a sea of long waves

Wave turbulence provides a conceptual framework for weakly nonlinear interactions in dispersive media. Dating from five decades ago, applications of wave turbulence theory to oceanic internal waves assigned a leading-order role to interactions characterized by a near equivalence between the group velocity of high-frequency internal waves with the phase velocity of near-inertial waves. This scale-separated interaction leads to a Fokker–Planck (generalized diffusion) equation. More recently, starting four decades ago, this scale-separated paradigm has been investigated using ray tracing methods. These ray methods characterize spectral transport of energy by counting the amplitude and net velocity of wave packets in phase space past a high-wavenumber gate prior to ‘breaking’. This explicitly advective characterization is based on an intuitive assignment and lacks theoretical underpinning. When one takes an estimate of the net spectral drift from the wave turbulence derivation and makes the corresponding assessment, one obtains a prediction of spectral transport that is an order of magnitude larger than either observations or reported ray tracing estimates. Motivated by this contradiction, we report two parallel derivations for transport equations describing the refraction of high-frequency internal waves in a sea of random inertial waves. The first uses standard wave turbulence techniques and the second is an ensemble-averaged packet transport equation characterized by the dispersion of wave packets about a mean drift in the spectral domain. The ensemble-averaged transport equation for ray tracing differs in that it contains the intuitively motivated advective term. We conclude that the aforementioned contradiction between theory, numerics and observations needs to be taken at face value and present a pathway for resolving this contradiction.

Hybrid surrogate model combining physics and data for seismic drift estimation of shear‐wall structures

AbstractTo address the issue of costly computational expenditure related to high‐fidelity numerical models, surrogate models have been widely used in various engineering tasks, including design optimization. Despite the successful application of the existing surrogate models, physics‐based models depend largely on simplifications and assumptions, which render parameter calibration challenging; whereas data‐driven models require substantial data to reach their full potential, with their performance often being constrained in tasks when obtaining massive data is difficult. In this study, a hybrid surrogate model is proposed combining physics‐based and data‐driven models to rapidly estimate building seismic responses. The application of this model is exemplified through effective estimation of inter‐story drift ratios (IDRs), being a critical factor in shear‐wall structure design. Initially, a data augmentation technique and a parametric modeling procedure are introduced to significantly enhance the dataset diversity. Subsequently, a task decomposition strategy is proposed to effectively integrate a data‐driven graph neural network (GNN) and a physics‐based flexural‐shear model. Additionally, the output layer and the loss function of the GNN are modified to enhance the estimation accuracy by eliminating fundamental errors. Results of numerical experiments indicate that the proposed hybrid model can complete IDR estimations in an average time of 0.56 s, with a mean absolute percentage error of 12.7%. This performance significantly surpasses that of existing purely data‐driven and physics‐based models. A case study shows that the efficiency of the proposed hybrid model is approximately 100 times greater than that of conventional finite element software. This enables an accurate assessment of the design compliance with code requirements. The results of this study can be applied to the design optimization of seismic‐resistant building structures.

A novel methodology to assess optical profilometer stability to discriminate surface roughness

A new method is proposed to investigate the measurement variation of instruments and their ability to discriminate two TA6V surfaces obtained by grinding with SiC grit papers (80 and 120). Three optical measurement technologies included into a single apparatus, namely Confocal Microscopy (CM), Focus Variation (FV) and Coherence Scanning Interferometry (CSI), were used to measure both ground surfaces according to a specific experimental protocol (MorphoMeca Monitoring) based on measurement iterations and repetitions. From the measurements, four indices were built from the Sa roughness parameter to assess the Quality (ratio of the topographical variability over the measurement noise), the Drift (related to surface fluctuations) and the Stability (estimation of the temporal drift using a second order autoregressive model) of the three measurement technologies (modes), as well as their Relevance (computed from an analysis of variance) to discriminate ground surfaces with close roughness. For these investigated surfaces, it was especially highlighted that the Quality Index is four times higher for the CSI mode than for the FV and CM modes. All the used measurement modes experienced small drifting (Drift Index) during the measurements. The CSI mode was slightly more sensitive to drift than the FV and CM modes. The Stability Index indicated that there was no memory effect between two acquisitions for each measurement mode, i.e., the temporal drift did not depend on the measurement mode. Moreover, the three measurement modes are able to discriminate the two ground surfaces with the same accuracy through the Sa parameter. However, the Sa values may vary depending on the measurement modes, and it was finally shown that the Sa parameter is minimized by 0.03 μm for the FV mode compared with the CSI mode, due to its smoothing effect.

Quantification of Representation Error in the Neutral Winds and Ion Drifts Using Data Assimilation

AbstractIn this work we quantify the representation error of the algorithm Estimating Model Parameters from Ionospheric Reverse Engineering (EMPIRE), which estimates global neutral winds and ion drifts given time‐varying plasma densities. SAMI3 (Sami3 is A Model of the Ionosphere) serves as the background climate model and pseudo‐measurements for the EMPIRE observation system. This configuration allows the data assimilation inputs to be self‐consistent between each other and with the validation data. The estimated neutral winds and ion drifts are compared to the Horizontal Wind Model (HWM14) and SAMI3 “truth.” For both the quiet period on 25 August 2018 and subs7equent storm on 26 August, the EMPIRE estimation of ion drifts is better at low‐to‐mid geomagnetic latitudes with mean error up to 20 m/s. For the high latitudes (poleward of ±60° magnetic), the mean errors exceed 50 m/s with variances up to 200 m/s, and the relative errors are higher than the “truth.” At latitudes of ±87°, the large errors are attributed to a boundary effect. However, the neutral wind mean errors peak at 20 m/s at mid‐latitudes (40°–60° magnetic), with larger uncertainties, then converge to 0 approaching higher latitudes. By conducting this study, we define a method for obtaining the representation error covariance for future use of EMPIRE with SAMI3 as background.

Lineage frequency time series reveal elevated levels of genetic drift in SARS-CoV-2 transmission in England.

Genetic drift in infectious disease transmission results from randomness of transmission and host recovery or death. The strength of genetic drift for SARS-CoV-2 transmission is expected to be high due to high levels of superspreading, and this is expected to substantially impact disease epidemiology and evolution. However, we don't yet have an understanding of how genetic drift changes over time or across locations. Furthermore, noise that results from data collection can potentially confound estimates of genetic drift. To address this challenge, we develop and validate a method to jointly infer genetic drift and measurement noise from time-series lineage frequency data. Our method is highly scalable to increasingly large genomic datasets, which overcomes a limitation in commonly used phylogenetic methods. We apply this method to over 490,000 SARS-CoV-2 genomic sequences from England collected between March 2020 and December 2021 by the COVID-19 Genomics UK (COG-UK) consortium and separately infer the strength of genetic drift for pre-B.1.177, B.1.177, Alpha, and Delta. We find that even after correcting for measurement noise, the strength of genetic drift is consistently, throughout time, higher than that expected from the observed number of COVID-19 positive individuals in England by 1 to 3 orders of magnitude, which cannot be explained by literature values of superspreading. Our estimates of genetic drift suggest low and time-varying establishment probabilities for new mutations, inform the parametrization of SARS-CoV-2 evolutionary models, and motivate future studies of the potential mechanisms for increased stochasticity in this system.

Improved estimation of drift coefficients using optimal local bandwidths

Stochastic differential equations (SDEs) are commonly used to model various systems. Data-driven methods have been widely used to estimate the drift and diffusion terms of a Langevin equation. Among the most commonly used estimation methods is the Nadaraya–Watson estimator, which is a non-parametric data-driven approach. In this study, we propose a method to improve the estimation of the drift coefficient of a stochastic process using optimal local bandwidths that minimize the error of the approximation of the first conditional moments of a univariate system. This approach is compared to a global bandwidth estimation and an estimation based on a fixed number of nearest neighbors. The proposed method has the potential to reduce the error of the drift estimation, thereby improving the accuracy of the model.

The estimation of the long-term drift of the inductance measurement standards

Inductance measurements are important in many fields, especially in electronics, electrical engineering, radio engineering, and other areas. The inductance is often an important parameter in a wide range of applications such as radio transmitters, power circuits, magnetic resonance pulsed sources, etc. The accuracy of the inductance affects the quality of products, especially in devices where inductors are used, such as filters, transformers, inverters, etc. High-precision inductance measurements are used for the product quality control to ensure that manufactured devices meet established specifications and standards.
 Drift is an undesirable property of all measuring instruments and measurement standards during their life cycle. The analysis of the instrumental drift of measurement standards is of great importance in metrology. Reliable drift accounting plays an important role in maintaining measurement accuracy. For electrical measurement standards, the long-term drift is predictable. The drift types and main methods of its estimation for measurement standards between their calibrations were analysed. The drift uncertainty can be evaluated from the history of successive calibrations, and in the absence of such history, the order of magnitude of the calibration uncertainty can be estimated.
 The results of the estimation of the long-term drift of the inductance measurement standards for high-precision calibration of measuring instruments and measurement standards by two methods, polynomial regression curves and Exponentially Weighted Moving Average (EWMA) schemes, are given. The EWMA schemes reduce the lag inherent in traditional moving averages by giving more weight to recent observations. It is shown that the use of the EWMA schemes compared to the regression analysis shows greater sensitivity to the drift changes in the last years of observations. This allows the laboratory to take this factor into account when calibrating measuring instruments and measurement standards.

Quantitative assessment of earthquake-induced building damage at regional scale using LiDAR data

Regional-scale assessment of the damage caused by earthquakes to structures is crucial for post-disaster management. While remote sensing techniques can be of great help for a quick post-event structural assessment of large areas, currently available methods are limited to the detection of severely-damaged buildings. Furthermore, remote sensing-based assessment methods typically provide only qualitative results, as they lack integration with information on the building’s behaviour in response to seismic-induced ground shaking. In this study, we developed a new methodology that uses airborne Light Detection And Ranging (LiDAR) data in combination with structural indicators of building response to provide a quantitative assessment of earthquake-induced damage at a regional scale. LiDAR datasets collected before and after an earthquake are used to measure residual displacements of building roofs. The resulting lateral drift estimations are used to quantify the level of damage for a specific building typology. Application to the LiDAR datasets collected before and after the 2014 earthquake in Napa Valley, California, demonstrates the capability of the proposed method to detect moderate levels of structural damage, proving its potential for faster and more accurate support to post-disaster management.

Robust adaptive Kalman filter for structural performance assessment

SummaryStructural performance assessment is a critical stage in the health monitoring for the maintenance and risk management of engineering structures, in which the interstory drift is a key indicator. In this paper, a robust adaptive Kalman filter is proposed for an interstory drift estimation problem to show the health condition of steel structures in the case that the statistics or internal dynamics describing the signals and measurements are not known precisely. More precisely, we build an adaptive current Jerk model (ACJM) where the model parameters are updated in each time step to presuppose the statistics characterization of the steel dynamic, while the unknown measurement noise covariance is adapted based on a fixed‐lag innovation with respect to measurements. Moreover, a robust adaptive Kalman filter is designed for the modelling uncertainties in each time increment by solving a minimax game: one player tries to select an “actual” model far from the proposed ACJM with an exponential decay tolerance, while its “hostile” player, namely, the optimum filter, is designed by minimizing the estimation error according to the selected “actual” model. Finally, some simulation and experimental results show the effectiveness of the proposed algorithm.

MRMPro: a web-based tool to improve the speed of manual calibration for multiple reaction monitoring data analysis by mass spectrometry.

As a gold-standard quantitative technique based on mass spectrometry, multiple reaction monitoring (MRM) has been widely used in proteomics and metabolomics. In the analysis of MRM data, as no peak picking algorithm can achieve perfect accuracy, manual inspection is necessary to correct the errors. In large cohort analysis scenarios, the time required for manual inspection is often considerable. Apart from the commercial software that comes with mass spectrometers, the open-source and free software Skyline is the most popular software for quantitative omics. However, this software is not optimized for manual inspection of hundreds of samples, the interactive experience also needs to be improved. Here we introduce MRMPro, a web-based MRM data analysis platform for efficient manual inspection. MRMPro supports data analysis of MRM and schedule MRM data acquired by mass spectrometers of mainstream vendors. With the goal of improving the speed of manual inspection, we implemented a collaborative review system based on cloud architecture, allowing multiple users to review through browsers. To reduce bandwidth usage and improve data retrieval speed, we proposed a MRM data compression algorithm, which reduced data volume by more than 60% and 80% respectively compared to vendor and mzML format. To improve the efficiency of manual inspection, we proposed a retention time drift estimation algorithm based on similarity of chromatograms. The estimated retention time drifts were then used for peak alignment and automatic EIC grouping. Compared with Skyline, MRMPro has higher quantification accuracy and better manual inspection support. In this study, we proposed MRMPro to improve the usability of manual calibration for MRM data analysis. MRMPro is free for non-commercial use. Researchers can access MRMPro through http://mrmpro.csibio.com/ . All major mass spectrometry formats (wiff, raw, mzML, etc.) can be analyzed on the platform. The final identification results can be exported to a common.xlsx format for subsequent analysis.

On Lasso and Slope drift estimators for Lévy-driven Ornstein–Uhlenbeck processes

On Lasso and Slope drift estimators for Lévy-driven Ornstein–Uhlenbeck processes

The importance of hazard-consistency when estimating seismic residual drifts in steel moment frames

Residual drifts are routinely used to assess the post-earthquake safety of buildings. Despite their importance, studies on the probabilistic assessment of residual drifts in multi-storey buildings with Steel Moment Resisting Frames (SMRF) are far less common than those dealing with their transient peak drift counterpart. More seriously, although residual drift prediction models have been developed with a particular broad ground-motion type or soil condition in mind, all models proposed to date remain oblivious to the central issue of hazard consistency. Hence, missing the causal connection between the seismic hazard level and the ground-motion suite used and potentially compromising the meaningfulness of their results. This oversight is expected to introduce a bias in the estimation of residual drifts but its magnitude has not yet been properly evaluated. In this paper, we evaluate and quantify the significance of these effects. To this end, we use the Conditional Scenario Spectra framework, which provides a set of realistic earthquake spectra with assigned rates of occurrences based on their spectral shape and intensity, thus preserving the critical relationship of hazard consistency. Nonlinear Response History Analyses (NRHA) of 24 deteriorating SMRFs under 816 ground-motion records are performed to construct a database of residual drift demands. These NRHA results are used to examine the residual drift trends and to construct benchmark residual drift hazard curves. An extensive feature selection process employing several Machine Learning (ML) algorithms precedes the development of hybrid data-driven predictive models. Finally, we compare our hazard-consistent predictions with currently available models and quantify the massive under- and over-estimations associated with previously proposed non-hazard-consistent assumptions.

A contrast in sea ice drift and deformation between winter and spring of 2019 in the Antarctic marginal ice zone

Abstract. Two ensembles of buoys, deployed in the marginal ice zone (MIZ) of the north-eastern Weddell Sea region of the Southern Ocean, are analysed to characterise the dynamics driving sea ice drift and deformation during the winter-growth and the spring-retreat seasons of 2019. The results show that although the two buoy arrays were deployed within the same region of ice-covered ocean, their trajectory patterns were vastly different. This indicates a varied response of sea ice in each season to the local winds and currents. Analyses of the winter data showed that the Antarctic Circumpolar Current modulated the drift near the sea ice edge. This led to a highly energetic and mobile ice cover, characterised by free-drift conditions. The resulting drift and deformation were primarily driven by large-scale atmospheric forcing, with negligible contributions due to the wind-forced inertial response. For this highly advective coupled ice–ocean system, ice drift and deformation linearly depended on atmospheric forcing. We also highlight the limits of commercial floating ice velocity profilers in this regime since they may bias the estimates of sea ice drift and the ice type detection. On the other hand, the spring drift was governed by the inertial response as increased air temperatures caused the ice cover to melt and break up, promoting a counterintuitively less wind-driven ice–ocean system that was more dominated by inertial oscillations. In fact, the deformation spectra indicate a strong decoupling to large-scale atmospheric forcing. Further analyses, extended to include the deformation datasets from different regions around Antarctica, indicate that, for similar spatial scales, the magnitude of deformation varies between seasons, regions, and the proximity to the sea ice edge and the coastline. This implies the need to develop rheology descriptions that are aware of the ice types in the different regions and seasons to better represent sea ice dynamics in the MIZ.

Drift Estimation of the Threshold Ornstein-Uhlenbeck Process From Continuous and Discrete Observations

We refer by threshold Ornstein-Uhlenbeck to a continuous-time threshold autoregressive process. It follows the Ornstein-Uhlenbeck dynamics when above or below a fixed level, yet at this level (threshold) its coefficients can be discontinuous. We discuss (quasi)-maximum likelihood estimation of the drift parameters, both assuming continuous and discrete time observations. In the ergodic case, we derive consistency and speed of convergence of these estimators in long time and high frequency. Based on these results, we develop a test for the presence of a threshold in the dynamics. Finally, we apply these statistical tools to short-term US interest rates modeling.

Critical drift estimates for the frog model on trees

Critical drift estimates for the frog model on trees

Improved critical drift estimates for the frog model on trees

Improved critical drift estimates for the frog model on trees

Estimating the long-term drift of travelling measurement standards for comparisons

In metrology, it is essential to analyse the instrumental drift of measuring instruments and measurement standards. Each reference instrument is periodically calibrated according to a frequency determined by the laboratory. Calibration establishes the metrological state of the instrument on a certain date of calibration. However, it is necessary to know the state of the measuring instrument during or after the calibration.
 Reliable accounting for drift plays an important role in maintaining measurement accuracy. Otherwise, it can lead to significant measurement errors. Accounting for time drift is mandatory when conducting international comparisons of national measurement standards. The drift uncertainty can be evaluated from its history of successive calibrations. In the absence of such a history, the magnitude order of the calibration uncertainty can be estimated.
 The analysis of the long-term drift of travelling measurement standards is limited to examples of key and supplementary comparisons of measurement standards of electrical capacitance. Quite a lot of such comparisons were conducted both by the Consultative Committee for Electricity and Magnetism (CCEM) and by most of the Regional Metrology Organizations (RMOs). There are international standards and guides that describe various statistical methods of analysing the measurement results.
 For capacitance measurement standards, time drift is predictable and nearly linear. For comparisons of measurement standards, a linear model is more than often applied, as a travelling measurement standard with excellent stability characteristics is used. The consistent results have been obtained. The linear model was applied to estimate the drift of travelling measurement standards during the key and supplementary comparisons (COOMET.EM-K4, COOMET.EM-S4, and COOMET.EM-S13) of measurement standards of electrical capacitance. The estimation of the long-term drift of measurement standards of electrical capacitance as travelling measurement standards for comparisons using a polynomial regression are presented.