Single Data Structure Research Articles

Non-overlapping descents and ascents in stack-sortable permutations

The Eulerian polynomials An(x) give the distribution of descents over permutations. It is also known that the distribution of descents over stack-sortable permutations (i.e. permutations sortable by a certain algorithm whose internal storage is limited to a single stack data structure) is given by the Narayana numbers 1nnknk+1. On the other hand, as a corollary of a much more general result, the distribution of the statistic “maximum number of non-overlapping descents”, MND, over all permutations is given by ∑n,k≥0Dn,kxktnn!=et1−x(1+(t−1)et).In this paper, we show that the distribution of MND over stack-sortable permutations is given by 1n+1n+12k+1n+kk. We give two proofs of the result via bijections with rooted plane (binary) trees allowing us to control MND. Moreover, we show combinatorially that MND is equidistributed with the statistic MNA, the maximum number of non-overlapping ascents, over stack-sortable permutations. The last fact is obtained by establishing an involution on stack-sortable permutations that gives equidistribution of 8 statistics.

Dezvoltarea algoritmilor genetici din perspective inter/transdisciplinare

The theoretical-practical foundations of Genetic Algorithms, which are built on the principle of "survival of the fittest", enunciated by Charles Darwin, are dealt with in this paper. The paper describes the basic characteristics of the genetic algorithm, highlighting its advantages and disadvantages. Genetic algorithm problems are examined. The Genetic Algorithm is examined from the perspective of examining problems in which finding the optimal solution is not simple or at least inefficient due to the characteristics of the probabilistic search. The steps are shown in which Genetic Algorithms encode a possible solution to a specific problem in a single data structure called a "chromosome" and set the stage for applying genetic operators to these structures in order to maintain critical information.

A Data integration tool for the integrated modeling and analysis for EAST

A large amount of heterogeneous data is generated during the experiments, operations, and simulations of the Experimental Advanced Superconducting Tokamak (EAST). The semantic and formatting inconsistencies of these heterogeneous data barrier the automation of data processes in integrated modeling and analysis. Data integration approaches based on a unified data model are an effective way to address this problem. A data integration tool, SpDB, was developed to unify data access for EAST’s integrated modeling process. SpDB unifies access to heterogeneous data sources under a single entry point in a namespace compatible with Integrated Modeling and Analysis Suite (IMAS). Users can access abstract tree data structures through a unified path without worrying about the format of specific data sources. SpDB enables automating integrated modeling workflows by combining multiple data sources into a single data structure that acts as a unified IO interface to the module.

TechMaps: exploring technology relationships through patent information based proximity.

Our work provides a novel method for rich information discovery about the evolution of technical fields and company developments through patent relationships. A new exploratory method and graphical tool to discover technology proximity based on patent classification information are introduced. By technology we mean a technical field (defined by an International Patent Classification-IPC-code or a combination of them) or an organization (such as a tech company, research center, or institution). A single data structure is used for characterizing both technical fields and organizations, to visualize them as items of the very same body. This new method generates two graphs: the first graph, the TechnologyMap, visualizes technology items in a 2D plot wherein technical fields and companies will appear positioned relative to each other; the. A second graph, the Focused TechnologyMap, visualizes technology items with respect to a selected one, the focus, which is located in the center of a circle whose radii correspond to the complete set of IPC codes. This article represents the process and algorithms used for production of the graphs, and solidifies the assumptions of validity by presenting two of the many successful test cases to which it was applied.

SVJedi-graph: improving the genotyping of close and overlapping structural variants with long reads using a variation graph.

Structural variation (SV) is a class of genetic diversity whose importance is increasingly revealed by genome resequencing, especially with long-read technologies. One crucial problem when analyzing and comparing SVs in several individuals is their accurate genotyping, that is determining whether a described SV is present or absent in one sequenced individual, and if present, in how many copies. There are only a few methods dedicated to SV genotyping with long-read data, and all either suffer of a bias toward the reference allele by not representing equally all alleles, or have difficulties genotyping close or overlapping SVs due to a linear representation of the alleles. We present SVJedi-graph, a novel method for SV genotyping that relies on a variation graph to represent in a single data structure all alleles of a set of SVs. The long reads are mapped on the variation graph and the resulting alignments that cover allele-specific edges in the graph are used to estimate the most likely genotype for each SV. Running SVJedi-graph on simulated sets of close and overlapping deletions showed that this graph model prevents the bias toward the reference alleles and allows maintaining high genotyping accuracy whatever the SV proximity, contrary to other state of the art genotypers. On the human gold standard HG002 dataset, SVJedi-graph obtained the best performances, genotyping 99.5% of the high confidence SV callset with an accuracy of 95% in less than 30 min. SVJedi-graph is distributed under an AGPL license and available on GitHub at https://github.com/SandraLouise/SVJedi-graph and as a BioConda package.

A Dynamic Web-based Application for the Classification of Vascular Anomalies

The International Society for the Study of Vascular Anomalies (ISSVA) released the most recent version of the classification of vascular anomalies in 2018.1 This revision of the 2014 classification expanded the scope of clinical entities and included molecular diagnostics. The 2018 report incorporated recent discoveries in the field and was released as a 20-page PDF file navigable through embedded links. The links lead users through the classification hierarchy, and each page lists anomalies, their corresponding genetic mutations (if known), and other details. Further information is found in five appendices in the last seven pages of the report. The document is comprehensive, but the structure has drawbacks in usability, organization, and sustainability. The format is rigid and difficult to update. The vascular anomaly field is new and ever growing. In just the past few years, there have been significant advances in discovering new clinical entities and uncovering the genetic basis for these conditions. The PDF-based classification cannot be updated routinely without reformatting entirely. Therefore, this format is unsustainable as new discoveries are made. The PDF format classification also limits organization, ease of use, and navigability. It includes a hierarchy of anomalies and categories organized into tables with internal page links that are difficult to navigate. The embedded links guide users to various pages throughout the document, which provides a nonlinear and somewhat confusing user experience. Additionally, the format of each page was unique (to accommodate custom information), and this lack of standardization stifles the user experience because of the inconsistent location of relevant information. Finally, the indices store information that cannot be placed on the appropriate PDF pages (because of space limitations), which makes navigating back and forth tedious. The aim for a new format was to streamline the design and solve the limitations of the current PDF format. We engineered an interactive web app format, which sustainably supports future updates. The classification is organized into a tree, with nested expandable nodes containing the anomalies in a hierarchical structure, similar to that of a file explorer. With this format, a user can expand the tree to view individual anomalies, associated information, and the entire hierarchy in a common location. The visual hierarchy provides a natural organization of the classification for a more intuitive experience. The app has a consistent format throughout, and it is easily modified. Minor additions and changes can be implemented quickly, without altering or compromising the framework of the format. The web app supports including additional anomaly-specific data through the use of popover buttons, which improves organization while providing comprehensive information. Popovers may contain clinical and radiological images (Figure 1), footnotes for unusual or exceptional cases (Figure 2), genomic information, and others if necessary. Additionally, we included search functionality via autocomplete “smart” select boxes (Figure 3). The user can search for a specific term by selecting it from an alphabetical list, or by typing into a free text field that limits selection to specific options dynamically. When searching for a specific anomaly, the web app opens the tree to the selected entry while briefly highlighting the entry itself and its parent nodes. Separate selectable dialog boxes allow the user to search genes and abbreviations without navigating away (Figure 4).Figure 1.: Expanded tree view of vascular anomalies classification hierarchy. Image popover shows clinical image of infantile hemangioma.Figure 2.: Popover with additional information about rapidly involuting congenital hemangioma.Figure 3.: Search function opens the tree to the selectable term (arteriovenous malformation in this case), automatically scrolls down the page, and highlights hierarchy.Figure 4.: Gene Search and Abbreviation Search dialog boxes with option filtering and autofill. The search buttons remain affixed to the bottom for easy look-up even as one scrolls up and down the tree. One can lookup terms without losing one’s place in the tree.The tree is rendered dynamically from a single data structure, which allows for flexible modification and efficiency. When data are changed it is instantly reflected in the classification. This makes minor or major alterations feasible and practical, in contrast to labor-intense reformats of PDF releases. The vascular anomaly field is changing rapidly with the recognition of new clinical entities and the identification of genes responsible for disease. This app format allows for real-time classification updates. The web app was created using React, a JavaScript library. It is a mobile-friendly application currently hosted at https://vascularanomalies.net. A pure client-side program, it can be embedded easily on the ISSVA website and updated to reflect any future changes rapidly.

Data, Anomalies; a Call to Cease and Desist.

Data, Anomalies; a Call to Cease and Desist.

Nested spatial data structures for optimal indexing of LiDAR data

In this paper we present a flexible framework for creating spatial data structures to manage LiDAR point clouds in the context of spatial big data. For this purpose, standard approaches typically include the use of a single data structure to index point clouds. Some of them use a hybrid two-tier solution to optimize specific application purposes such as storage or rendering. In this article we introduce a meta-structure that can have unlimited depth and a custom, user-defined combination of nested structures, such as grids, quadtrees, octrees, or kd-trees. With our approach, the out-of-core indexing of point clouds can be adapted to different types of datasets, taking into account the spatial distribution of the data. Therefore, the most suitable spatial indexing can be achieved for any type of dataset, from small TLS-based scenes to planetary-scale ALS-based scenes. This approach allows us to work with overlapping datasets of different resolutions from different acquisition technologies in the same structure.

Decorated merge trees for persistent topology

This paper introduces decorated merge trees (DMTs) as a novel invariant for persistent spaces. DMTs combine both $$\pi _0$$ and $$H_n$$ information into a single data structure that distinguishes filtrations that merge trees and persistent homology cannot distinguish alone. Three variants on DMTs, which emphasize category theory, representation theory and persistence barcodes, respectively, offer different advantages in terms of theory and computation. Two notions of distance—an interleaving distance and bottleneck distance—for DMTs are defined and a hierarchy of stability results that both refine and generalize existing stability results is proved here. To overcome some of the computational complexity inherent in these distances, we provide a novel use of Gromov-Wasserstein couplings to compute optimal merge tree alignments for a combinatorial version of our interleaving distance which can be tractably estimated. We introduce computational frameworks for generating, visualizing and comparing decorated merge trees derived from synthetic and real data. Example applications include comparison of point clouds, interpretation of persistent homology of sliding window embeddings of time series, visualization of topological features in segmented brain tumor images and topology-driven graph alignment.

Linear unmixing protocol for hyperspectral image fusion analysis applied to a case study of vegetal tissues

Hyperspectral imaging (HSI) is a useful non-invasive technique that offers spatial and chemical information of samples. Often, different HSI techniques are used to obtain complementary information from the sample by combining different image modalities (Image Fusion). However, issues related to the different spatial resolution, sample orientation or area scanned among platforms need to be properly addressed. Unmixing methods are helpful to analyze and interpret the information of HSI related to each of the components contributing to the signal. Among those, Multivariate Curve Resolution-Alternating Least Squares (MCR-ALS) offers very suitable features for image fusion, since it can easily cope with multiset structures formed by blocks of images coming from different samples and platforms and allows the use of optional and diverse constraints to adapt to the specific features of each HSI employed. In this work, a case study based on the investigation of cross-sections from rice leaves by Raman, synchrotron infrared and fluorescence imaging techniques is presented. HSI of these three different techniques are fused for the first time in a single data structure and analyzed by MCR-ALS. This example is challenging in nature and is particularly suitable to describe clearly the necessary steps required to perform unmixing in an image fusion context. Although this protocol is presented and applied to a study of vegetal tissues, it can be generally used in many other samples and combinations of imaging platforms.

Integration of Remote Sensing Data into a Composite Voxel Model for Environmental Performance Analysis of Terraced Vineyards in Tuscany, Italy

Understanding socio-ecological systems and the discovery, recovery and adaptation of land knowledge are key challenges for sustainable land use. The analysis of sustainable agricultural systems and practices, for instance, requires interdisciplinary and transdisciplinary research and coordinated data acquisition, data integration and analysis. However, datasets, which are acquired using remote sensing, geospatial analysis and simulation techniques, are often limited by narrow disciplinary boundaries and therefore fall short in enabling a holistic approach across multiple domains and scales. In this work, we demonstrate a new workflow for interdisciplinary data acquisition and integration, focusing on terraced vineyards in Tuscany, Italy. We used multi-modal data acquisition and performed data integration via a voxelised point cloud that we term a composite voxel model. The latter facilitates a multi-domain and multi-scale data-integrated approach for advancing the discovery and recovery of land knowledge. This approach enables integration, correlation and analysis of data pertaining to different domains and scales in a single data structure.

Intrusion Detection System Based on Integrated System Calls Graph and Neural Networks

Computer security is one of the main challenges of today's technological infrastructures, whereas intrusion detection systems are one of the most widely used technologies to secure computer systems. The intrusion detection systems use a variety of information sources, one of the most important sources are the applications' system calls. The intrusion detection systems use many different detection techniques, e.g. system calls sequences, text classification techniques and system calls graphs. However, existing techniques obtain poor results in the detection of complex attack patterns, so it is necessary to improve the detection results. This paper presents an intrusion detection system model that integrates multiple detection techniques into a single system with the goal of modeling the global behavior of the applications. In addition, the paper proposes a new modified system calls graph to integrate and represent the information of the different techniques in a single data structure. The system uses a deep neural network to combine the results of the different detection techniques used in the global model. The result of the study shows the improvement obtained in the detection results with respect to the use of individual techniques, the proposed model achieves higher detection rates and lower false positives. The proposal has been validated onto three datasets with different levels of complexity.

A multimedia document browser based on multilayer networks

Querying and retrieving relevant information still remains a difficult task, one with a relatively high cognitive cost for users, who usually focus only on the first few pages of results. This issue drives effort to support the exploration of search results through clustering and visualization. This paper contributes to this challenge by providing a visual analytics system that is designed to support search tasks in multimedia document archives. The system provides complex querying, semantic overviews of time, and visual, and textual concepts combined with analysis. All search tasks are supported with linked-highlighting and leapfrog interactions. This is made possible all in a single data structure thanks to multilayer network modelling.

Rasterisation-based progressive photon mapping

Ray tracing on the GPU has been synergistically operating alongside rasterisation in interactive rendering engines for some time now, in order to accurately capture certain illumination effects. In the same spirit, in this paper, we propose an implementation of progressive photon mapping entirely on the rasterisation pipeline, which is agnostic to the specific GPU architecture, in order to synthesise images at interactive rates. While any GPU ray tracing architecture can be used for photon mapping, performing ray traversal in image space minimises acceleration data structure construction time and supports arbitrarily complex and fully dynamic geometry. Furthermore, this strategy maximises data structure reuse by encompassing rasterisation, ray tracing and photon gathering tasks in a single data structure. Both eye and light paths of arbitrary depth are traced on multi-view deep G-buffers, and photon flux is gathered by a properly adapted multi-view photon splatting. In contrast to previous methods exploiting rasterisation to some extent, due to our novel indirect photon splatting approach, any event combination present in photon mapping is captured. We evaluate our method using typical test scenes and scenarios for photon mapping methods and show how our approach outperforms typical GPU-based progressive photon mapping.

A binary-tree element subdivision method for evaluation of singular domain integrals with continuous or discontinuous kernel

A novel element subdivision method based on the binary tree has been proposed for evaluation of singular domain integrals in BEM. In this paper, this element subdivision technique is called the Binary-Tree Subdivision Method (BTSM), which is applicable to arbitrary shape linear and curved volume elements with arbitrary locations of the source point. Compared to the Conventional Subdivision Method (CSM), a significant advantage of the BTSM is that it can handle singular domain integrals with continuous or discontinuous kernel and improve the accuracy of integration even with distorted elements. With the distinct feature that a single binary-tree data structure can efficiently handle volume element subdivision, it is flexible and convenient for the BTSM to be implemented in the formulation of the boundary integral equation which contains volume integrals. In addition, for the volume integrals with discontinuous kernel, an improved general projection algorithm based on Newton iteration has been proposed for curved boundary matching. Experiment results have demonstrated that the volume element is always subdivided by the BTSM in a fully automated manner and high-quality patch generation can be guaranteed in any situation. Several examples are given to verify the validity, robustness and accuracy of the proposed method.

Integration of multispectral visible-infrared imaging and pointwise X-ray fluorescence data for the analysis of a large canvas painting by Carpaccio

In this work, we studied the canvas “The meeting of the pilgrims with the Pope” by Vittore Carpaccio fusing in a single data structure the pointwise XRF analysis with the Vis-IR multispectral data acquired on the painting. We proposed a workflow for analyzing and interpreting the data using Principal Component Analysis. The effectiveness of this approach is evaluated against the analysis of the data analyzed as separated contributions. The proposed procedure is demonstrated on the red pigments, which are predominant in this painting and in Carpaccio's production in general, highlighting information regarding the original artist's technique and regarding restored areas.

YSTAFDB, a unified database of material stocks and flows for sustainability science

We present the Yale Stocks and Flows Database (YSTAFDB), which comprises most of the material stocks and flows (STAF) data generated at the Center for Industrial Ecology at Yale University since the early 2000s. These data describe material cycles, criticality, and recycling in terms of 62 elements and various engineering materials, e.g., steel, on spatial scales and timeframes ranging from cities to global and the 1800s to ca. 2013. YSTAFDB integrates this diverse collection of STAF data, previously scattered across various non-uniformly formatted electronic files, into a single data structure and file format. Here, we discuss this data structure as well as the usage and formatting of data records in YSTAFDB. YSTAFDB contains 100,000+ data records that are all situated in their systems contexts, with additional metadata included as available. YSTAFDB offers a comprehensive basis upon which STAF data can be accumulated, integrated, and exchanged, and thereby improves their accessibility. Therefore, YSTAFDB facilitates deeper understanding of sustainable materials use and management, which are key goals of contemporary sustainability science.

A binary-tree element subdivision method for evaluation of nearly singular domain integrals with continuous or discontinuous kernel

An adaptive and efficient volume element subdivision method using binary tree for evaluation of nearly singular domain integrals with continuous or discontinuous kernel in three-dimensional (3-D) boundary element method (BEM) has been presented. In the Conventional Subdivision Method (CSM) for evaluation of nearly singular integrals, the patches are obtained by simply connecting the source point with each vertex of the element. Thus, the accuracy of the integral obtained with CSM is easily affected by the shape of the element and the location of the source point. In contrast, the proposed Binary-Tree Subdivision Method (BTSM) is more convenient to implement and can guarantee successful patch generation under any circumstances for accurate evaluation of nearly singular domain integrals with continuous or discontinuous kernel. Numerical results for volume elements of arbitrary type with various relative locations of the source point demonstrate robustness, accuracy and efficiency of the proposed method. • The BTSM is proposed for evaluating nearly singular integrals with various kernel. • The BTSM is applicable to different kinds of volume element with arbitrary shape. • The BTSM is more convenient to implement by the single binary-tree data structure. • The BTSM can guarantee the convergence of element subdivision in any situation. • The BTSM possesses better accuracy and efficiency than the conventional method.

Efficient anticorrelated variance reduction for stochastic simulation of biochemical reactions

We investigate the computational challenge of improving the accuracy of the stochastic simulation estimation by inducing negative correlation through the anticorrelated variance reduction technique. A direct application of the technique to the stochastic simulation algorithm (SSA), employing the inverse transformation, is not efficient for simulating large networks because its computational cost is similar to the sum of independent simulation runs. We propose in this study a new algorithm that employs the propensity bounds of reactions, introduced recently in their rejection-based SSA, to correlate and synchronise the trajectories during the simulation. The selection of reaction firings by our approach is exact due to the rejection-based mechanism. In addition, by applying the anticorrelated variance technique to select reaction firings, our approach can induce substantial correlation between realisations, hence reducing the variance of the estimator. The computational advantage of our rejection-based approach in comparison with the traditional inverse transformation is that it only needs to maintain a single data structure storing propensity bounds of reactions, which is updated infrequently, hence achieving better performance.

Efficient finite-difference method for computing sensitivities of biochemical reactions

Sensitivity analysis of biochemical reactions aims at quantifying the dependence of the reaction dynamics on the reaction rates. The computation of the parameter sensitivities, however, poses many computational challenges when taking stochastic noise into account. This paper proposes a new finite-difference method for efficiently computing sensitivities of biochemical reactions. We employ propensity bounds of reactions to couple the simulation of the nominal and perturbed processes. The exactness of the simulation is preserved by applying the rejection-based mechanism. For each simulation step, the nominal and perturbed processes under our coupling strategy are synchronized and often jump together, increasing their positive correlation and hence reducing the variance of the estimator. The distinctive feature of our approach in comparison with existing coupling approaches is that it only needs to maintain a single data structure storing propensity bounds of reactions during the simulation of the nominal and perturbed processes. Our approach allows to compute sensitivities of many reaction rates simultaneously. Moreover, the data structure does not require to be updated frequently, hence improving the computational cost. This feature is especially useful when applied to large reaction networks. We benchmark our method on biological reaction models to prove its applicability and efficiency.