Time-dependent Data Research Articles

Cluster-based stability evaluation in time series data sets

In modern data analysis, time is often considered just another feature. Yet time has a special role that is regularly overlooked. Procedures are usually only designed for time-independent data and are therefore often unsuitable for the temporal aspect of the data. This is especially the case for clustering algorithms. Although there are a few evolutionary approaches for time-dependent data, the evaluation of these and therefore the selection is difficult for the user. In this paper, we present a general evaluation measure that examines clusterings with respect to their temporal stability and thus provides information about the achieved quality. For this purpose, we examine the temporal stability of time series with respect to their cluster neighbors, the temporal stability of clusters with respect to their composition, and finally conclude on the temporal stability of the entire clustering. We summarise these components in a parameter-free toolkit that we call Cluster Over-Time Stability Evaluation (CLOSE). In addition to that we present a fuzzy variant which we call FCSETS (Fuzzy Clustering Stability Evaluation of Time Series). These toolkits enable a number of advanced applications. One of these is parameter selection for any type of clustering algorithm. We demonstrate parameter selection as an example and evaluate results of classical clustering algorithms against a well-known evolutionary clustering algorithm. We then introduce a method for outlier detection in time series data based on CLOSE. We demonstrate the practicality of our approaches on three real world data sets and one generated data set.

Penetration Routes of Oxygen and Moisture into the Insulation of FR-EPDM Cables for Nuclear Power Plants

The polymeric insulation used in nuclear power plants (NPPs) carries the risk of molecular breakage due to oxidation and hydrolysis in the event of an accident. With this in mind, tubular specimens of flame-retardant ethylene-propylene-diene rubber (FR-EPDM) insulation were obtained by taking conductors out of a cable harvested from an NPP. Similar tubular specimens were made from a newly manufactured cable and those aged artificially using a method called the "superposition of time-dependent data." The inner and outer surfaces of each tubular specimen were subjected to various instrumental analyses to examine their oxidation, moisture uptake, and cross-linking. As a result, it has become clear that oxygen penetrates the cable through gaps between the twisted conductor strands. Meanwhile, water vapor diffuses more often through the sheath than through gaps between the conductor strands. Of the two methods used to simulate design-based accidents in NPPs, the one used to simulate the designed loss-of-coolant accident is more severe to FR-EPDM than the one used to simulate the designed severe accident. In addition, the validity of the method called the "superposition of time-dependent data," which is used to give artificial aging treatments to cable samples, was confirmed. Measurements of spin-spin relaxation time and residual dipolar coupling using time-domain nuclear magnetic resonance were found suitable to use to obtain information on the cross-linking of FR-EPDM insulation.

Explaining the GeV Antiproton Excess, GeV γ -Ray Excess, and W -Boson Mass Anomaly in an Inert Two Higgs Doublet Model

For the newly discovered W-boson mass anomaly, one of the simplest dark matter (DM) models that can account for the anomaly without violating other astrophysical and experimental constraints is the inert two Higgs doublet model, in which the DM mass (m_{S}) is found to be within ∼54-74 GeV. In this model, the annihilation of DM via SS→bb[over ¯] and SS→WW^{*} would produce antiprotons and gamma rays, and may account for the excesses identified previously in both particles. Motivated by this, we reanalyze the AMS-02 antiproton and Fermi-LAT Galactic center γ-ray data. For the antiproton analysis, the novel treatment is the inclusion of the charge-sign-dependent three-dimensional solar modulation model as constrained by the time-dependent proton data. We find that the excess of antiprotons is more distinct than previous results based on the force-field solar modulation model. The interpretation of this excess as the annihilation of SS→WW^{*} (SS→bb[over ¯]) requires a DM mass of ∼40-80 (40-60)GeV and a velocity-averaged cross section of O(10^{-26}) cm^{3} s^{-1}. As for the γ-ray data analysis, besides adopting the widely used spatial template fitting, we employ an orthogonal approach with a data-driven spectral template analysis. The fitting to the GeV γ-ray excess yields DM model parameters overlapped with those to fit the antiproton excess via the WW^{*} channel. The consistency of the DM particle properties required to account for the W-boson mass anomaly, the GeV antiproton excess, and the GeV γ-ray excess suggests a common origin of them.

SGORNN: Combining scalar gates and orthogonal constraints in recurrent networks

Recurrent Neural Network (RNN) models have been applied in different domains, producing high accuracies on time-dependent data. However, RNNs have long suffered from exploding gradients during training, mainly due to their recurrent process. In this context, we propose a variant of the scalar gated FastRNN architecture, called Scalar Gated Orthogonal Recurrent Neural Networks (SGORNN). SGORNN utilizes orthogonal matrices at the recurrent step. Our experiments evaluate SGORNN using two recently proposed orthogonal parametrizations for the recurrent weights of an RNN. We present a constraint on the scalar gates of SGORNN, which is easily enforced at training time to provide a probabilistic generalization gap which grows linearly with the length of sequences processed. Next, we provide bounds on the gradients of SGORNN to show the impossibility of exponentially exploding gradients through time. Our experimental results on the addition problem confirm that our combination of orthogonal and scalar gated RNNs are able to outperform other orthogonal RNNs and LSTM on long sequences. We further evaluate SGORNN on the HAR-2 classification task, where it improves upon the accuracy of several models using far fewer parameters than standard RNNs. Finally, we evaluate SGORNN on the Penn Treebank word-level language modeling task, where it again outperforms its related architectures and shows comparable performance to LSTM using far less parameters. Overall, SGORNN shows higher representation capacity than the other orthogonal RNNs tested, suffers from less overfitting than other models in our experiments, benefits from a decrease in parameter count, and alleviates exploding gradients during backpropagation through time.

Dynamic Community Detection Decouples Multiple Time Scale Behavior of Complex Chemical Systems.

Although community or cluster identification is becoming a standard tool within the simulation community, traditional algorithms are challenging to adapt to time-dependent data. Here, we introduce temporal community identification using the Δ-screening algorithm, which has the flexibility to account for varying community compositions, merging and splitting behaviors within dynamically evolving chemical networks. When applied to a complex chemical system whose varying chemical environments cause multiple time scale behavior, Δ-screening is able to resolve the multiple time scales of temporal communities. This computationally efficient algorithm is easily adapted to a wide range of dynamic chemical systems; flexibility in implementation allows the user to increase or decrease the resolution of temporal features by controlling parameters associated with community composition and fluctuations therein.

Quantification of Deep Neural Network Prediction Uncertainties for VVUQ of Machine Learning Models

Recent performance breakthroughs in artificial intelligence (AI) and machine learning (ML), especially advances in deep learning, the availability of powerful and easy-to-use ML libraries (e.g., scikit-learn, TensorFlow, PyTorch), and increasing computational power, have led to unprecedented interest in AI/ML among nuclear engineers. For physics-based computational models, verification, validation, and uncertainty quantification (VVUQ) processes have been very widely investigated, and many methodologies have been developed. However, VVUQ of ML models has been relatively less studied, especially in nuclear engineering. This work focuses on uncertainty quantification (UQ) of ML models as a preliminary step of ML VVUQ, more specifically Deep Neural Networks (DNNs) because they are the most widely used supervised ML algorithm for both regression and classification tasks. This work aims at quantifying the prediction or approximation uncertainties of DNNs when they are used as surrogate models for expensive physical models. Three techniques for UQ of DNNs are compared, namely, Monte Carlo Dropout (MCD), Deep Ensembles (DE), and Bayesian Neural Networks (BNNs). Two nuclear engineering examples are used to benchmark these methods: (1) time-dependent fission gas release data using the Bison code and (2) void fraction simulation based on the Boiling Water Reactor Full-size Fine-Mesh Bundle Tests (BFBT) benchmark using the TRACE code. It is found that the three methods typically require different DNN architectures and hyperparameters to optimize their performance. The UQ results also depend on the amount of training data available and the nature of the data. Overall, all three methods can provide reasonable estimations of the approximation uncertainties. The uncertainties are generally smaller when the mean predictions are close to the test data while the BNN methods usually produce larger uncertainties than MCD and DE.

Modeling optical second harmonic generation for oxide/semiconductor interface characterization

In this work, we present physics-based numerical modeling of experimental time-dependent optical second harmonic generation data from an oxide/semiconductor (SiO2/Si) interface. A comprehensive numerical solution to the Poisson–Boltzmann equation has been developed here, using the Newton–Raphson method at different time instances. It incorporates the trapping behavior of photo-excited charge carriers at the silicon dioxide/silicon (SiO2/Si) interface, within the silicon dioxide (SiO2) and at the SiO2 surface, in order to model the second harmonic photon count data obtained from our in-house experimental setup. This yields a quantitative analysis of the SiO2/Si interface, oxide, and surface charge densities, and provides a contact-less and non-invasive optical technique for oxide/semiconductor interface characterization.

Variations in age structure and growth in congeners Lacerta viridis and Lacerta media.

Determining the age of any species allows it to be analyzed from the ontogenetic, demographic, and ecological perspectives. In the present study, we tested the hypothesis that the age structure of congener species (Lacerta media and Lacerta viridis) with the same ecological niche may vary in different areas. In this context, we applied skeletochronology method to reveal various demographic parameters, such as age structure, longevity, age at sexual maturity, growth rate, survival rate, adult life expectancy, and the relationship between age and body size in the green lizard, L. viridis, and the medium lizard, L. media. In L. media and L. viridis, the maximum lifespan was 10 and 8 years, respectively. The mean age and body size of females were significantly greater than those of males in L. media. However, in the examined L. viridis population, no appreciable variation in mean age or body size was found to exist between the sexes. It was estimated that the green lizards reach maturity at the age of 2 or 3 years. However, the L. media reached sexual maturity approximately 1 year later than the congener. The body size markedly increased with age in males for both studied populations. However, in females, body size positively increased with age only in L. media. The approach of skeletochronology that we utilized in this study to assess age structure makes it simple to gather a variety of time-dependent ecological data for such ectothermic species.

Survival analysis of localized prostate cancer with deep learning

In recent years, data-driven, deep-learning-based models have shown great promise in medical risk prediction. By utilizing the large-scale Electronic Health Record data found in the U.S. Department of Veterans Affairs, the largest integrated healthcare system in the United States, we have developed an automated, personalized risk prediction model to support the clinical decision-making process for localized prostate cancer patients. This method combines the representative power of deep learning and the analytical interpretability of parametric regression models and can implement both time-dependent and static input data. To collect a comprehensive evaluation of model performances, we calculate time-dependent C-statistics C_{text {td}} over 2-, 5-, and 10-year time horizons using either a composite outcome or prostate cancer mortality as the target event. The composite outcome combines the Prostate-Specific Antigen (PSA) test, metastasis, and prostate cancer mortality. Our longitudinal model Recurrent Deep Survival Machine (RDSM) achieved C_{text {td}} 0.85 (0.83), 0.80 (0.83), and 0.76 (0.81), while the cross-sectional model Deep Survival Machine (DSM) attained C_{text {td}} 0.85 (0.82), 0.80 (0.82), and 0.76 (0.79) for the 2-, 5-, and 10-year composite (mortality) outcomes, respectively. In addition to estimating the survival probability, our method can quantify the uncertainty associated with the prediction. The uncertainty scores show a consistent correlation with the prediction accuracy. We find PSA and prostate cancer stage information are the most important indicators in risk prediction. Our work demonstrates the utility of the data-driven machine learning model in prostate cancer risk prediction, which can play a critical role in the clinical decision system.

Ministerial stability during presidential approval crises: The moderating effect of ministers’ attributes on dismissals in Brazil and Chile

This article analyses the effect of ministers’ exposure to periods of low presidential approval in Brazil and Chile between 1990 and 2014. Approval is explored with quarterly estimates using a dyad-ratios algorithm and merged into a time-dependent cabinet data set to evaluate individual ministerial terminations ( N = 4245). The empirical strategy combines time-varying exposure Cox regressions with observational data and propensity score and matching to estimate the effect of low approval on ministerial survival and perform a moderation analysis with three profiles associated with presidential strategies: (1) nonpartisan ministers to limit agency loss and moral hazard; (2) economists as ministers to optimise cabinet performance and send positive signals to the electorate; and (3) party leaders as ministers to optimise legislative support. The main findings show that risk increases by 135.1% in periods of low approval. In addition, approximately only one in five nonpartisan ministers is removed compared to party members.

Structural identification via the inference of the stochastic volatility model conditioned on the time-dependent bridge deflection

The time-dependent deflection data collected by a structural health monitoring (SHM) system contains information indicating the damage accumulation of prestressed concrete bridges (PSCBs). However, for this information to be used, the results must be fully translated into reliable metrics. Model-based structural identification (SI) targeting the time-dependent deflection of PSCBs is affected by time-dependent material behavior such as creep and shrinkage as well as the unknown path of structural deterioration, which should be considered in a stochastic volatility model (SVM). This paper presents an inference framework for an SVM conditioned on the time-dependent deflection of PSCBs. In this framework, the augmented state space includes the full ranges of the unobserved state variables affecting the time-dependent deflection of PSCBs, namely, structural rigidity, creep, shrinkage, prestress level and dead load level, along with the volatility parameters associated with the deterioration path of structural rigidity, which is modeled via the Wiener process. Exploiting the latent structure of the SVM, a cyclic Markov chain Monte Carlo (MCMC) sampler is proposed to draw samples from the joint posterior distribution of the unobserved state variables and volatility parameters. In an illustrative example, two-year continuous deflection monitoring data of an existing bridge are utilized as the target information for inference. The updated model can indicate the accumulated damage of the case bridge over the monitoring period. The proposed SI framework is able to support accurate probabilistic analysis of the time-dependent deflection of PSCB.

Alignment of highly resolved time-dependent experimental and simulated crash test data

We investigate for car and component crash tests the comparison of highly resolved experimental data with corresponding simulation data. Due to recent advances for optical measurement systems, one can nowadays obtain surface measurement data from a real crash experiment with high resolution in space and time. These advances call for new data processing methods that allow an alignment of this experimental data with numerical simulation results. We propose an approach based on a data representation stemming from a discrete Laplace–Beltrami operator, which allows such an alignment as well as a joint visual comparative analysis of both data sources. The method enables the identification of the best corresponding simulation among several numerical results, which also allows inferring physical quantities that cannot be measured in experiments. We evaluate the procedure on synthetic and real experimental data from two different crashworthiness setups.

An in-silico analysis of experimental designs to study ventricular function: A focus on the right ventricle.

In-vivo studies of pulmonary vascular disease and pulmonary hypertension (PH) have provided key insight into the progression of right ventricular (RV) dysfunction. Additional in-silico experiments using multiscale computational models have provided further details into biventricular mechanics and hemodynamic function in the presence of PH, yet few have assessed whether model parameters are practically identifiable prior to data collection. Moreover, none have used modeling to devise synergistic experimental designs. To address this knowledge gap, we conduct a practical identifiability analysis of a multiscale cardiovascular model across four simulated experimental designs. We determine a set of parameters using a combination of Morris screening and local sensitivity analysis, and test for practical identifiability using profile likelihood-based confidence intervals. We employ Markov chain Monte Carlo (MCMC) techniques to quantify parameter and model forecast uncertainty in the presence of noise corrupted data. Our results show that model calibration to only RV pressure suffers from practical identifiability issues and suffers from large forecast uncertainty in output space. In contrast, parameter and model forecast uncertainty is substantially reduced once additional left ventricular (LV) pressure and volume data is included. A comparison between single point systolic and diastolic LV data and continuous, time-dependent LV pressure-volume data reveals that at least some quantitative data from both ventricles should be included for future experimental studies.

Real-time kinetics and affinity analysis of the interaction between protein A and immunoglobulins G derived from different species on silica colloidal crystal films

Kinetic and affinity analysis of protein interactions reveals information on their related activities in biological processes. Herein, we established a system for evaluating the kinetics and affinity of the interaction between protein A and various IgG species on the surface of silica spheres of silica colloidal crystal (SCC) films by the extraordinary optical interference capabilities of 190 nm silica spheres after self-assembly. The equilibrium association constant (KA) was calculated by the equilibrium Langmuir model and nonlinear least-squares analysis of time-dependent data. The relative protein A/IgG binding affinity is human > rabbit >cow >goat. In addition, the competitive interaction of distinct species of IgG with protein A at the interface of SCC films was studied and performed. These findings may help with the use of protein A and other recognition components in a number of sensor types. Furthermore, this research might offer a novel approach to determining the kinetics and affinity of proteins on the surface of spheres particles, which may contribute to the development of the application of spheres particles in pharmaceutical science, biomedical engineering, and other techniques.

Time-Dependent Receiver Operating Characteristic Analysis and Applications in the Field of Medicine

When there is a time-dependency between the biomarker and the event of interest (death, disease, relapse etc.), classical receiver operating characteristic (ROC) analysis may not be able to estimate the true performance of the biomarker. For such cases, time-dependent ROC, an extended version of the standard ROC, is developed. In this study, the aim is to demonstrate applications of this modified ROC method on medical datasets and find out if it should be preferred over classical ROC for time-dependent situations. Comparison between classical ROC and Kaplan-Meier (KM) estimator, which is one of the two time-dependent ROC estimators, has been made using datasets in this study. Nearest Neighbor Estimator (NNE), the alternative of KM estimator, is also applied on all datasets. Then the findings of these two approaches are compared. It is concluded that time-dependent ROC method is superior to the standard ROC analysis. In general, the closer to the event time, the higher performance is observed. Especially, biomarkers measured at last 12 or 6 months before the event are determined to be better at classification than the earlier measurements. Though in all applications KM and NNE yielded very similar results, the latter is considered to be more appropriate to evaluate the performance of a biomarker when a time dependent data is also censored. Results of this study show that time-dependent ROC analysis performs more accurately when there is a time dependency between the biomarker and the event of interest; therefore, it is recommended.

Numerical simulations for time-dependent corrosion surface of unpainted structural steel plates using spatio-temporal data

Structural failure due to corrosion of steel members is a long-term process. A method to precisely simulate the development of corrosion on steel surfaces is of great significance. In this study, 9 specimens of unpainted steel plates, SM400A were used for the experiments. The experiments were conducted under artificial seawater corrosive environments to accelerate the corrosion process. The corrosion depth data of each unpainted steel plate for the spatial statistical analysis was measured by a laser focus measuring system when the accelerated corrosion experiments of 28, 84, and 168 days. Semivariogram was used in this spatial statistical analysis to research the spatial autocorrelation structures of the corrosion surface. By using this and the technique of ordinary kriging, the method to simulate the spatial features of corrosion surface was proposed. The results of the simulation indicated that the corrosion depth and surface topography of corrosion surfaces were highly consistent with the experiments. In addition, a prediction method for time-dependent corrosion surface was proposed by investigating the relationship between the spatial autocorrelation structures and their time-dependence data.

Development of an observational exposure human biomonitoring study to assess Canadian children's DEET exposure during protective use.

Biomonitoring data of N,N-diethyl-meta-toluamide (DEET) in children is scarce and limited to controlled exposure and surveillance studies. We conducted a 24-hour observational exposure and human biomonitoring study designed to estimate use of and exposure to DEET-based insect repellents by Canadian children in an overnight summer camp setting. Here, we present our study design and methodology. In 2019, children between the ages of 7 and 13 took part in the study (n = 126). Children controlled their use of DEET-based insect repellents, and provided an account of their activities at camp that could impact insect repellent absorption. Children provided a total of 389 urine samples throughout the study day, and reported the time that they applied insect repellent, which allowed us to contextualize urinary DEET and metabolite concentrations with respect to the timing of insect repellent application. DEET (2.3% <Limits of detection (LOD)) and two metabolites, N,N-diethyl-m-(hydroxymethyl)benzamide (DHMB) (0% <LOD) and 3-diethylcarbamoyl benzoic acid (DCBA) (0% <LOD), were measured in urine samples. Three time difference scenarios were established for the data and analysed to account for these complex time-dependent data, which demonstrated the need for DEET biomonitoring to be done in context with the timing of a known DEET exposure or over the course of at least 14 to 24 hours to better capture the excretion curve. To our knowledge, this is the first field-based study of real-world exposure to DEET in children. Our experience and results suggest that this type of real-world observational exposure study with a human biomonitoring component can generate data reflective of actual exposure, but is not without significant logistic, practical, and analytical challenges.

Multi-task longitudinal forecasting with missing values on Alzheimer’s disease

Background and Objective:Machine learning techniques typically used in dementia assessment are not able to learn multiple tasks jointly and deal with time-dependent heterogeneous data containing missing values. In this paper, we reformulate SSHIBA, a recently introduced Bayesian multi-view latent variable model, for jointly learning diagnosis, ventricle volume, and ADAS score in dementia on longitudinal data with missing values.Methods:We propose a novel Bayesian Variational inference framework capable of simultaneously imputing missing values and combining information from several views. This way, we can combine different data views from different time-points in a common latent space and learn the relationships between each time-point, using the semi-supervised formulation to fully exploit the temporal structure of the data and handle missing values. In turn, the model can combine all the available information to simultaneously model and predict multiple output variables.Results:We applied the proposed model to jointly predict diagnosis, ventricle volume, and ADAS score in dementia. The comparison of imputation strategies demonstrated the superior performance of the semi-supervised formulation of the model, improving the best baseline methods. Moreover, the performance in simultaneous prediction of diagnosis, ventricle volume, and ADAS score led to an improved prediction performance over the best baseline method.Conclusions:The results demonstrate that the proposed SSHIBA framework can learn an excellent imputation of the missing values and outperforming the baselines while simultaneously predicting three different tasks.

EFFECT OF PRE-TREATMENTS ASSISTED MICROWAVE DRYING ON KINETICS AND ENERGY ASPECTS OF DRAGON FRUIT

Drying is one of the most effective preservation methods applied to increase the shelf life of agricultural products. In this study, the effect of different pre-treatments on drying dragon fruit with a microwave was investigated. It is also aimed to determine the drying method with the minimum energy consumption. Dragon fruit slices were dried by microwave at 300 W output power in four different pretreatments as ultrasound, gum arabic, sucrose, and without pretreatment. Pretreatment applications were carried out by performing ultrasound for 10 minutes on dragon fruit slices and soaking them in 10% gum arabic and sucrose solutions before drying. Results showed that the shortest and longest drying times were measured in gum arabic and without pretreatment, respectively. Time-dependent estimation data were calculated by six different thin-layer drying models, considering the drying times and moisture rates. Jena&amp;Das model showed the best fitting performance. As a result, it has been determined that pretreatment applications reduce drying time and energy consumption. In addition, gum arabic pre-treatment is advantageous in terms of energy consumption in microwave drying of dragon fruit.

A Comprehensive Analysis of Proportional Intensity-Based Software Reliability Models with Covariates

This paper focuses on the so-called proportional intensity-based software reliability models (PI-SRMs), which are extensions of the common non-homogeneous Poisson process (NHPP)-based SRMs, and describe the probabilistic behavior of software fault-detection process by incorporating the time-dependent software metrics data observed in the development process. The PI-SRM is proposed by Rinsaka et al. in the paper “PISRAT: Proportional Intensity-Based Software Reliability Assessment Tool” in 2006. Specifically, we generalize this seminal model by introducing eleven well-known fault-detection time distributions, and investigate their goodness-of-fit and predictive performances. In numerical illustrations with four data sets collected in real software development projects, we utilize the maximum likelihood estimation to estimate model parameters with three time-dependent covariates (test execution time, failure identification work, and computer time-failure identification), and examine the performances of our PI-SRMs in comparison with the existing NHPP-based SRMs without covariates. It is shown that our PI-STMs could give better goodness-of-fit and predictive performances in many cases.