Arbitrary Data Research Articles

Formal verification for event stream processing: Model checking of BeepBeep stream processing pipelines

Event stream processing (ESP) is the application of a computation to a set of input sequences of arbitrary data objects, called “events”, in order to produce other sequences of data objects. In recent years, a large number of ESP systems have been developed; however, none of them is easily amenable to a formal verification of properties on their execution. In this paper, we show how stream processing pipelines built with an existing ESP library called BeepBeep 3 can be exported as a Kripke structure for the NuXmv model checker. This makes it possible to formally verify properties on these pipelines, and opens the way to the use of such pipelines directly within a model checker as an extension of its specification language.

Analysis Database Systems and Solve Medical Problems

Due to this mechanism, threshold synchronization is achieved with a minimum amount of time. It is usually designed for communication between scalar processors via shared memory. The main function of graphics processors since their inception has been graphics processing. Subsequently, after it became possible to program the processing of model vertices and pixels of rendered three-dimensional scenes using special programs (shaders), the architecture of graphic processors changed significantly. After the advent of the first general-purpose programmable graphics it became possible to process commands not only for graphic data in vector form, but also to perform ordinary calculations for arbitrary data on a variety of special cores, while implementing data parallelism.

A sensorized modular training platform to reduce vascular damage in endovascular surgery.

Endovascular interventions require intense practice to develop sufficient dexterity in catheter handling within the human body. Therefore, we present a modular training platform, featuring 3D-printed vessel phantoms with patient-specific anatomy and integrated piezoresistive impact force sensing of instrument interaction at clinically relevant locations for feedback-based skill training to detect and reduce damage to the delicate vascular wall. The platform was fabricated and then evaluated in a user study by medical ([Formula: see text]) and non-medical ([Formula: see text]) users. The users had to navigate a set of guidewire and catheter through a parkour of 3 modules including an aneurismatic abdominal aorta, while impact force and completion time were recorded. Eventually, a questionnaire was conducted. The platform allowed to perform more than 100 runs in which it proved capable to distinguish between users of different experience levels. Medical experts in the fields of vascular and visceral surgery had a strong performance assessment on the platform. It could be shown, that medical students could improve runtime and impact over 5 runs. The platform was well received and rated as promising for medical education despite the experience of higher friction compared to real human vessels. We investigated an authentic patient-specific training platform with integrated sensor-based feedback functionality for individual skill training in endovascular surgery. The presented method for phantom manufacturing is easily applicable to arbitrary patient-individual imaging data. Further work shall address the implementation of smaller vessel branches, as well as real-time feedback and camera imaging for further improved training experience.

Learning Logical Pauli Noise in Quantum Error Correction.

The characterization of quantum devices is crucial for their practical implementation but can be costly in experimental effort and classical postprocessing. Therefore, it is desirable to measure only the information that is relevant for specific applications and develop protocols that require little additional effort. In this Letter, we focus on the characterization of quantum computers in the context of stabilizer quantum error correction. For arbitrary stabilizer codes, subsystem codes, and data syndrome codes, we prove that the logical error channel induced by Pauli noise can be estimated from syndrome data under minimal conditions. More precisely, for any such code, we show that the estimation is possible as long as the code can correct the noise.

Modified Scattering for the One-Dimensional Schrödinger Equation with a Subcritical Dissipative Nonlinearity

We study the asymptotic behavior in time of solutions to the one dimensional nonlinear Schrödinger equation with a subcritical dissipative nonlinearity $$\lambda |u|^\alpha u$$ , where $$0<\alpha <2$$ , and $$\lambda $$ is a complex constant satisfying $$\text {Im} \lambda >\frac{\alpha |\text {Re} \lambda |}{2\sqrt{ \alpha +1}}$$ . For arbitrary large initial data, we present the uniform time decay estimates when $$4/3\le \alpha <2$$ , and the large time asymptotics of the solution when $$\frac{7+\sqrt{145}}{12}<\alpha <2$$ . The proof is based on the vector fields method and a semiclassical analysis method.

A Framework for Frugal Supervised Learning with Incremental Neural Networks

This study proposes an implementation of an incremental neural network (INN) that was initially designed for affective computing tasks. INNs are a family of machine learning algorithms that combine prototype-based classifiers with neural networks. They achieve state-of-the-art performance with less data than traditional approaches. In this research, we conduct an in-depth review of INN mechanisms and present a research-grade framework that enables the use of INNs on arbitrary data. We evaluated our implementation on two different datasets, including the AVEC2014 Challenge, which involved predicting depressive state from auditive and visual modalities. Our results are encouraging, demonstrating the potential of INNs in situations where approaches have to be explainable or when data are scarce.

Quiver Hecke algebras for Borcherds-Cartan datum

We introduce a family of quiver Hecke algebras which give a categorification of quantum Borcherds algebra associated to an arbitrary Borcherds-Cartan datum.

The pysat ecosystem

The Python Satellite Data Analysis Toolkit (pysat) is an open source package that implements a general data analysis workflow for arbitrary data sets, providing a consistent manner for obtaining, managing, analysing, and processing data, including modelled and observational ground and space-based data sets for the space sciences. Pysat enables systematic and individual treatment of data as well as simplifies rigorous data access and use, allowing larger-scale scientific efforts including machine learning, data assimilation, and constellation instrumentation processing. Since the start of its development pysat has evolved into an ecosystem, separating general file and data handling functionality from both individual data set support and generalized data analysis. This design choice ensures that the core pysat package has only the necessary functionality required to provide data management services for the wider development community. The shift of data and analysis support to ecosystem packages makes it easier for the community to contribute to, as well as use, the full array of features and data sources enabled by pysat. Pysat’s ease of use, and generality, supports adoption outside of professional science to include industry, citizen science, and education.

HDF5eis: A storage and input/output solution for big multidimensional time series data from environmental sensors

Modern high-performance computing (HPC) tasks overwhelm conventional geophysical data formats. We describe a new data schema called HDF5eis (read H-D-F-size) for handling big multidimensional time series data from environmental sensors in HPC applications and implement a freely available Python application programming interface (API) for building and processing HDF5eis files. HDF5eis augments the popular Hierarchical Data Format 5 with a minimal set of additional conventions that facilitate fast and flexible data input and output protocols for regularly sampled (in time) data with any number of dimensions. HDF5eis supports arbitrary ancillary data (e.g., metadata) storage in columnar format or as UTF-8 encoded byte streams alongside time series data. Our HDF5eis API enables simple and efficient access to big data sets distributed across a potentially large number of small heterogeneous files through a single point of access. HDF5eis outperforms conventional seismic data formats by up to two orders of magnitude in terms of random read access times. We contribute HDF5eis as an operational tool and an experimental draft proposal that will help establish the next generation of data standards in the earth sciences.

How Big Should Your Data Really Be? Data-Driven Newsvendor: Learning One Sample at a Time

We study the classical newsvendor problem in which the decision maker must trade off underage and overage costs. In contrast to the typical setting, we assume that the decision maker does not know the underlying distribution driving uncertainty but has only access to historical data. In turn, the key questions are how to map existing data to a decision and what type of performance to expect as a function of the data size. We analyze the classical setting with access to past samples drawn from the distribution (e.g., past demand), focusing not only on asymptotic performance but also on what we call the transient regime of learning, that is, performance for arbitrary data sizes. We evaluate the performance of any algorithm through its worst-case relative expected regret, compared with an oracle with knowledge of the distribution. We provide the first finite sample exact analysis of the classical sample average approximation (SAA) algorithm for this class of problems across all data sizes. This allows one to uncover novel fundamental insights on the value of data: It reveals that tens of samples are sufficient to perform very efficiently but also that more data can lead to worse out-of-sample performance for SAA. We then focus on the general class of mappings from data to decisions without any restriction on the set of policies and derive an optimal algorithm (in the minimax sense) and characterize its associated performance. This leads to significant improvements for limited data sizes and allows to exactly quantify the value of historical information. This paper was accepted by David Simchi-Levi, data science. Supplemental Material: The data files and online appendix are available at https://doi.org/10.1287/mnsc.2023.4725 .

The Hidden Behavior of a D-Latch

For clock and data transitions in close temporal proximity, synchronous memory elements potentially enter metastability, which leads to unintended output behavior. Although respective analyses in literature have already derived suitable explanations, almost all of them modeled the control (clock) signal transition with negligible rise/fall time. In modern circuits this assumption is, however, not reasonable any more. In fact, due to a finite slope, intermediate clock signal values have to be considered during a large share of the storage process, while their concrete impact is not yet sufficiently explored. In this paper we thus use static and dynamic considerations to thoroughly investigate the behavior of a latch for arbitrary analog control, data and output values, i.e., during the storage process. Basic circuit considerations allow us to derive a unified model which identifies the latch as a Schmitt Trigger with vastly varying hysteresis. We verify the correctness of our predictions by comparison to analog SPICE simulations. Finally we are able to generalize our findings and thus provide explanations for yet unexplained behavior reported in literature.

The Impact of Data Quality on Software Testing Effort Prediction

Background: This paper investigates the impact of data quality on the performance of models predicting effort on software testing. Data quality was reflected by training data filtering strategies (data variants) covering combinations of Data Quality Rating, UFP Rating, and a threshold of valid cases. Methods: The experiment used the ISBSG dataset and 16 machine learning models. A process of three-fold cross-validation repeated 20 times was used to train and evaluate each model with each data variant. Model performance was assessed using absolute errors of prediction. A ‘win–tie–loss’ procedure, based on the Wilcoxon signed-rank test, was applied to identify the best models and data variants. Results: Most models, especially the most accurate, performed the best on a complete dataset, even though it contained cases with low data ratings. The detailed results include the rankings of the following: (1) models for particular data variants, (2) data variants for particular models, and (3) the best-performing combinations of models and data variants. Conclusions: Arbitrary and restrictive data selection to only projects with Data Quality Rating and UFP Rating of ‘A’ or ‘B’, commonly used in the literature, does not seem justified. It is recommended not to exclude cases with low data ratings to achieve better accuracy of most predictive models for testing effort prediction.

Wide and deep neural networks achieve consistency for classification

While neural networks are used for classification tasks across domains, a long-standing open problem in machine learning is determining whether neural networks trained using standard procedures are consistent for classification, i.e., whether such models minimize the probability of misclassification for arbitrary data distributions. In this work, we identify and construct an explicit set of neural network classifiers that are consistent. Since effective neural networks in practice are typically both wide and deep, we analyze infinitely wide networks that are also infinitely deep. In particular, using the recent connection between infinitely wide neural networks and neural tangent kernels, we provide explicit activation functions that can be used to construct networks that achieve consistency. Interestingly, these activation functions are simple and easy to implement, yet differ from commonly used activations such as ReLU or sigmoid. More generally, we create a taxonomy of infinitely wide and deep networks and show that these models implement one of three well-known classifiers depending on the activation function used: 1) 1-nearest neighbor (model predictions are given by the label of the nearest training example); 2) majority vote (model predictions are given by the label of the class with the greatest representation in the training set); or 3) singular kernel classifiers (a set of classifiers containing those that achieve consistency). Our results highlight the benefit of using deep networks for classification tasks, in contrast to regression tasks, where excessive depth is harmful.

Minimal bipartite dimers and higher genus Harnack curves

This paper completes the comprehensive study of the dimer model on infinite minimal graphs with Fock's weights [arXiv:1503.00289] initiated in [arXiv:2007.14699]: the latter article dealt with the elliptic case, i.e., models whose associated spectral curve is of genus one, while the present work applies to models of arbitrary genus. This provides a far-reaching extension of the genus zero results of [arXiv:math-ph/0202018, arXiv:math/0311062], from isoradial graphs with critical weights to minimal graphs with weights defining an arbitrary spectral data. For any minimal graph with Fock's weights, we give an explicit local expression for a two-parameter family of inverses of the associated Kasteleyn operator. In the periodic case, this allows us to prove local formulas for all ergodic Gibbs measures, thus providing an alternative description of the measures constructed in [arXiv:math-ph/0311005]. We also compute the corresponding slopes, exhibit an explicit parametrization of the spectral curve, identify the divisor of a vertex, and build on [arXiv:math/0311062, arXiv:1107.5588] to establish a correspondence between Fock's models on periodic minimal graphs and Harnack curves endowed with a standard divisor.

Efficient Two-Stage Analysis for Complex Trait Association with Arbitrary Depth Sequencing Data

Sequencing-based genetic association analysis is typically performed by first generating genotype calls from sequence data and then performing association tests on the called genotypes. Standard approaches require accurate genotype calling (GC), which can be achieved either with high sequencing depth (typically available in a small number of individuals) or via computationally intensive multi-sample linkage disequilibrium (LD)-aware methods. We propose a computationally efficient two-stage combination approach for association analysis, in which single-nucleotide polymorphisms (SNPs) are screened in the first stage via a rapid maximum likelihood (ML)-based method on sequence data directly (without first calling genotypes), and then the selected SNPs are evaluated in the second stage by performing association tests on genotypes from multi-sample LD-aware calling. Extensive simulation- and real data-based studies show that the proposed two-stage approaches can save 80% of the computational costs and still obtain more than 90% of the power of the classical method to genotype all markers at various depths d≥2.

A time-accurate inflow coupling for zonal LES

Generating turbulent inflow data is a challenging task in zonal large eddy simulation (zLES) and often relies on predefined DNS data to generate synthetic turbulence with the correct statistics. The more accurate, but more involved alternative is to use instantaneous data from a precursor simulation. Using instantaneous data as an inflow condition allows to conduct high fidelity simulations of subdomains of, e.g. an aircraft including all non-stationary or rare events. In this paper, we introduce a toolchain that is capable of interchanging highly resolved spatial and temporal data between flow solvers with different discretization schemes. To accomplish this, we use interpolation algorithms suitable for scattered data in order to interpolate spatially. In time, we use one-dimensional interpolation schemes for each degree of freedom. The results show that we can get stable simulations that map all flow features from the source data into a new target domain. Thus, the coupling is capable of mapping arbitrary data distributions and formats into a new domain while also recovering and conserving turbulent structures and scales. The necessary time and space resolution requirements can be defined knowing the resolution requirements of the used numerical scheme in the target domain.

Analysis of the generalized Aw-Rascle model

We consider the multi-dimensional generalization of the Aw-Rascle system for vehicular traffic. For arbitrary large initial data and the periodic boundary conditions, we prove the existence of global-in-time measure-valued solutions. We also show, using the relative energy technique, that the measure-valued solutions coincide with the classical solutions as long as the latter exist.

Optimal False Data Injection Attacks Against Power System Frequency Stability

The automatic generation control (AGC) is one of the core control systems in power grids that regulate frequency within the permissible range. However, its dependence on communication makes it highly vulnerable to cyber-attacks. An arbitrary false data injection attack (FDIA) on AGC frequency and tie-line flow measurements will likely be detectable by bad data detection methods; however, if an attack can be launched optimally, it often becomes stealthy. In this regard, we develop a framework of optimal FDIAs (OFDIAs) to demonstrate the feasibility of such attacks in the power system frequency control loop. We propose a linearized formulation of discretized power systems’ dynamics in an optimization framework to model OFDIAs that compromise the AGC system by corrupting tie-line flow and generators’ frequency measurements. Using the proposed formal modeling, we study the effects of two types of FDIAs, continuous and time-limited, on the frequency behavior in power grids. The results demonstrate that continuous OFDIAs can lead to severe consequences on a power grid’s performance, such as frequency instability. In contrast, the time-limited FDIAs can cause the frequency to fluctuate beyond the acceptable range, which may lead to the triggering of the frequency-based protection relays.

Lessons learned on the use of data surfaces for Ffowcs Williams-Hawkings calculations: Airframe noise applications

The use of integral solutions to the acoustic-analogy-based Ffowcs Williams-Hawkings (FW-H) formulation is the de facto industrial and academic approach for assessing far-field acoustic behavior from airframe noise simulations. The methodology utilizes time-accurate flow variables obtained at arbitrary data surfaces to determine far-field spectra. The solid surface approach, where the time history of pressure is recorded on the surface of the aircraft, is generally preferred. Permeable surfaces, where pressure and velocity are recorded on a user-defined surface enclosing the aircraft, are also used. This paper seeks to share some of the valuable lessons we learned over the past several years on the limitations of the FW-H formulation and best practices for its successful application. We show results for various configurations comprising isolated landing gears, a business jet, a large commercial transport, and a generic low boom supersonic prototype, all at landing conditions. We demonstrate that the solid FW-H approach has significant accuracy limitations for most realistic cases and that the permeable FW-H approach may also fail in specific scenarios, requiring additional costly steps to extract the correct far-field noise spectra. Suggested guidelines for proper FW-H calculations are also given.

Application of dense neural networks for manifold-based modeling of flame-wall interactions

Artifical neural networks (ANNs) are universal approximators capable of learning any correlation between arbitrary input data with corresponding outputs, which can also be exploited to represent a low-dimensional chemistry manifold in the field of combustion. In this work, a procedure is developed to simulate a premixed methane-air flame undergoing side-wall quenching utilizing an ANN chemistry manifold. In the investigated case, the flame characteristics are governed by two canonical problems: the adiabatic flame propagation in the core flow and the non-adiabatic flame- wall interaction governed by enthalpy losses to the wall. Similar to the tabulation of a Quenching Flamelet-Generated Manifold (QFM), the neural network is trained on a 1D head-on quenching flame database to learn the intrinsic chemistry manifold. The control parameters (i.e. the inputs) of the ANN are identified from thermo-chemical state variables by a sparse principal component analysis (PCA) without using prior knowledge about the flame physics. These input quantities are then transported in the coupled CFD solver and used for manifold access during simulation runtime. The chemical source terms are corrected at the manifold boundaries to ensure bounded- ness of the thermo-chemical state at all times. Finally, the ANN model is assessed by comparison to simulation results of the 2D side-wall quenching (SWQ) configuration with detailed chemistry and with a flamelet-based manifold (QFM).