Internal Data Structures Research Articles

Application of the Random Forest classification method for facies zoning according to seismic data

Background. Facial zoning based on seismic data is an important task of dynamic analysis. There are numerous approaches of solving this problem using various algorithms. The most common method is clustering by reflection shape. This approach belongs to unsupervised learning algorithms, due to the mapping of seismic facies is based on the internal data structure and the key feature is the change in the wave packet within the target interval. The disadvantage of this method is requirement of further tying clustering results and geological information. Another way of directed solution of this problem is the use of supervised learning algorithms. This category includes various classification methods that relate to the category of machine learning. In comparison to traditional approaches of seismic facial analysis, this method accounts geological information at the computation stage. Aim. This paper considers the results of a research carried out with the study of the facies structure of the Tyumen formation at a group of fields in the Khanty-Mansiysk Autonomous Region. The Tyumen formation is characterized by the predominance of channel facies associated with the development of complex river systems, which are clearly observed in the dynamic characteristics of the wave field. A complicating factor in the study of these deposits is the rather low coverage of well data, which makes difficult the geological interpretation of the results obtained. Materials and methods. The authors used the Random Forest classification method to deal with the assigned task. The application of the method is considered on the cluster consisting of three seismic surveys obtained at different times. For training, expert marking by area was used based on the distribution of amplitudes along the reflecting horizon. Results. As a result of the research, a probabilistic assessment of the distribution of channel facies was obtained, that is related to the perspective of this type of deposits in the study area. Thus, the authors have developed a methodology that gives an opportunity to obtain an estimate of the probability of the presence of a certain facies using seismic data. Conclusions. The performed study shows the possibility of using the Random Forest classification method to solve the problem of facial zoning.

Privacy-Enhanced Robust Image Hashing with Bloom Filters

Robust image hashes are used to detect known illegal images, even after image processing. This is, for example, interesting for a forensic investigation, or for a company to protect their employees and customers by filtering content. The disadvantage of robust hashes is that they leak structural information of the pictures, which can lead to privacy issues. Our scientific contribution is to extend a robust image hash with privacy protection. We thus introduce and discuss such a privacy-preserving concept. The approach uses a probabilistic data structure -- known as Bloom filter -- to store robust image hashes. Bloom filter store elements by mapping hashes of each element to an internal data structure. We choose a cryptographic hash function to one-way encrypt and store elements. The privacy of the inserted elements is thus protected. We evaluate our implementation, and compare it to its underlying robust image hashing algorithm. Thereby, we show the cost with respect to error rates for introducing a privacy protection into robust hashing. Finally, we discuss our approach's results and usability, and suggest possible future improvements.

Novel Approach to Minimize the Memory Requirements of Frequent Subgraph Mining Techniques

Frequent subgraph mining (FSM) is a subset of the graph mining domain that is extensively used for graph classification and clustering. Over the past decade, many efficient FSM algorithms have been developed with improvements generally focused on reducing the time complexity by changing the algorithm structure or using parallel programming techniques. FSM algorithms also require high memory consumption, which is another problem that should be solved. In this paper, we propose a new approach called Predictive dynamic sized structure packing (PDSSP) to minimize the memory needs of FSM algorithms. Our approach redesigns the internal data structures of FSM algorithms without making algorithmic modifications. PDSSP offers two contributions. The first is the Dynamic Sized Integer Type, a newly designed unsigned integer data type, and the second is a data structure packing technique to change the behavior of the compiler. We examined the effectiveness and efficiency of the PDSSP approach by experimentally embedding it into two state-of-the-art algorithms, gSpan and Gaston. We compared our implementations to the performance of the originals. Nearly all results show that our proposed implementation consumes less memory at each support level, suggesting that PDSSP extensions could save memory, with peak memory usage decreasing up to 38% depending on the dataset.

A New Approach to Minimize Memory Requirements of Frequent Subgraph Mining Algorithms

Frequent subgraph mining (FSM) is a subsection of graph mining domain which is extensively used for graph classification and graph clustering purposes. Over the past decade, many efficient FSM algorithms have been developed. The improvements generally focus on reducing time complexity by changing the algorithm structure or using parallel programming techniques. FSM algorithms have another problem to solve, which is the high memory consumption. In this study, a new approach called Predictive Dynamic Sized Structure Packing (PDSSP) have been proposed to minimize the memory requirement of FSM algorithms. Proposed approach redesigns the internal data structures of FSM algorithms without any algorithmic modifications. PDSSP has two contributions. The first one is the Dynamic Sized Integer Type (ds_Int) which is a newly designed unsigned integer data type. The second contribution is “Data Structure packaging” component that uses a data structure packing technique which changes the behaviour of the compiler. A number of experiments have been conducted to examine the effectiveness and efficiency of the PDSSP approach by embedding it into two state-of-art algorithms called gSpan and Gaston. Proposed implementation have been compared to the official one. Almost all results show that the proposed implementation consumes less memory on each support level. As a result, PDSSP extensions can save memory and the peak memory usage may decrease up to 38% depending on the dataset.

Efficient wide-field radio interferometry response

Radio interferometers do not measure the sky brightness distribution directly, but measure a modified Fourier transform of it. Imaging algorithms therefore need a computational representation of the linear measurement operator and its adjoint, regardless of the specific chosen imaging algorithm. In this paper, we present a C++ implementation of the radio interferometric measurement operator for wide-field measurements that is based on so-called improved w-stacking. It can provide high accuracy (down to ≈10−12), is based on a new gridding kernel that allows smaller kernel support for given accuracy, dynamically chooses kernel, kernel support, and oversampling factor for maximum performance, uses piece-wise polynomial approximation for cheap evaluations of the gridding kernel, treats the visibilities in cache-friendly order, uses explicit vectorisation if available, and comes with a parallelisation scheme that scales well also in the adjoint direction (which is a problem for many previous implementations). The implementation has a small memory footprint in the sense that temporary internal data structures are much smaller than the respective input and output data, allowing in-memory processing of data sets that needed to be read from disk or distributed across several compute nodes before.

On the correctness and efficiency of a novel lock-free hash trie map design

Hash tries are a trie-based data structure with nearly ideal characteristics for the implementation of hash maps. In this paper, we present a novel, simple and scalable hash trie map design that fully supports the concurrent search, insert and remove operations on hash maps. To the best of our knowledge, our proposal is the first that puts together the following characteristics: (i) be lock-free; (ii) use fixed size data structures; and (iii) maintain the access to all internal data structures as persistent memory references. Our design is modular enough to allow different types of configurations aimed for different performances in memory usage and execution time and can be easily implemented in any type of language, library or within other complex data structures. We discuss in detail the key algorithms required to easily reproduce our implementation by others and we present a proof of correctness showing that our proposal is linearizable and lock-free for the search, insert and remove operations. Experimental results show that our proposal is quite competitive when compared against other state-of-the-art proposals implemented in Java.

Target Recognition of SAR Images Based on SVM and KSRC.

A synthetic aperture radar (SAR) target recognition method combining linear and nonlinear feature extraction and classifiers is proposed. The principal component analysis (PCA) and kernel PCA (KPCA) are used to extract feature vectors of the original SAR image, respectively, which are classical and reliable feature extraction algorithms. In addition, KPCA can effectively make up for the weak linear description ability of PCA. Afterwards, support vector machine (SVM) and kernel sparse representation-based classification (KSRC) are used to classify the KPCA and PCA feature vectors, respectively. Similar to the idea of feature extraction, KSRC mainly introduces kernel functions to improve the processing and classification capabilities of nonlinear data. Through the combination of linear and nonlinear features and classifiers, the internal data structure of SAR images and the correspondence between test and training samples can be better investigated. In the experiment, the performance of the proposed method is tested based on the MSTAR dataset. The results show the effectiveness and robustness of the proposed method.

Energy-Efficient Transaction Serialization for IoT Devices

This article presents two designs, the Transaction Serial Format (TSF) and the Transaction Array Model (TAM). Together, they provide full, efficient, transaction serialization facilities for devices with limited onboard energy, such as those in an Internet of Things (IoT) network. TSF provides a compact, non-parsed, format that requires minimal processing for transaction deserialization. TAM provides an internal data structure that needs minimal dynamic storage and directly uses the elements of TSF. The simple lexical units of TSF do not require parsing. The lexical units contain enough information to allocate the internal TAM data structure efficiently. TSF generality is equivalent to XML and JSON. TSF represents any XML document or JSON object without loss of information, including whitespace. The XML equivalence provides a foundation for the performance comparisons. A performance comparison of a C reference implementation of TSF and TAM to the popular Expat XML library, also written in C, shows that TSF reduces deserialization processor time by more than 80%.

Object-Oriented Metrics for Quality Improvement of Object-Oriented Software

Software metric is a computation of characteristics of a programming applications for quality enhancement. Software metrics are used to access the productivity and efficiency of a software product. These are helpful to understand the technical procedure used to create software. Software metric enables software engineers to assess software quality, software process improvement, monitoring and controlling of software. Object-Oriented Software Development is related to real-world objects and their characteristics creation instead of working in software applications. Class Objects have their own internal data structure, which defines their data and methods. Object-Oriented design restrained all the properties and worth of software that is allied to any large or small project. Object-Oriented metric is a measurement term in which a Object-Oriented Software holds features. These are guidelines that give an indication of the progress that a project has made quality.

FieldOpt: A powerful and effective programming framework tailored for field development optimization

Petroleum field development involves critical decisions such as well location, well completion design and optimal control setting that have a significant impact on revenues and costs. These decisions are associated with a large degree of engineering effort that commonly involves time-consuming reservoir simulations to compute the performance of different field development scenarios. Increasingly within the petroleum industry, software tools and optimization methodology are developed and implemented to support and augment the various decision-making processes. The overall aim of these tools is to increase productivity and improve decision quality. Within this context, this work introduces an open-source, extensible, tailor-made programming framework: FieldOpt. FieldOpt’s primary purpose is rapid prototyping and testing of optimization procedures to solve critical field development problems. The framework is implemented in C++ and provides an efficient integration of mathematical optimization procedures with reservoir simulation.FieldOpt’s modular architecture and use of object-oriented programming allow the users to adapt and extend the code with ease. The architecture has proven successful in facilitating new optimization algorithms, new use cases, and new methodology for optimization with minimal effort and change to internal data structures and data flow.Three use cases are presented to demonstrate the optimization capabilities of FieldOpt as well as the flexibility and ease-of-use of the software regarding configuring various optimization procedures to solve a range of field development problems. The three use cases presented optimize on well control, well completion design and well placement parameters, respectively. For each case, both the configuration of the algorithms and problems within FieldOpt as well as the final solution and performance of the different optimization procedures are discussed. In all three cases, FieldOpt was able to find significant improvements or crucial information to decision making.

Problems and features of data structure modeling in distributed Web-oriented production systems

The goal is to develop a system model necessary for structuring the subject area, decomposing the functions performed by a web application; formalize the design methods, determine the general method of synthesizing Internet-oriented information systems. The main part of the development is the creation of a model, which components will be the objects of the domain, while attention should be paid to balance - the flexibility of the model and the development costs, taking into account the possibility of reuse. Too flexible components will have too many properties and will be difficult to customize adjust. On the contrary, models that implement too particular situation will be useless in terms of reuse, and, perhaps, the cost of its implementation will not make sense. For a more complete understanding of the functions and requirements for data processing, it is necessary to identify the main objects of the domain, information flows, logical connections, as well as rules and restrictions when working with data. The result of building a model should be a DBMS-independent data schema (metadata) corresponding to the internal data structure of the Web application, focused on the implementation of the required functions when interfacing with the user.

Concrete Syntax with Black Box Parsers

Context: Meta programming consists for a large part of matching, analyzing, and transforming syntax trees. Many meta programming systems process abstract syntax trees, but this requires intimate knowledge of the structure of the data type describing the abstract syntax. As a result, meta programming is error-prone, and meta programs are not resilient to evolution of the structure of such ASTs, requiring invasive, fault-prone change to these programs. Inquiry: Concrete syntax patterns alleviate this problem by allowing the meta programmer to match and create syntax trees using the actual syntax of the object language. Systems supporting concrete syntax patterns, however, require a concrete grammar of the object language in their own formalism. Creating such grammars is a costly and error-prone process, especially for realistic languages such as Java and C++. Approach: In this paper we present Concretely, a technique to extend meta programming systems with pluggable concrete syntax patterns, based on external, black box parsers. We illustrate Concretely in the context of Rascal, an open-source meta programming system and language workbench, and show how to reuse existing parsers for Java, JavaScript, and C++. Furthermore, we propose Tympanic, a DSL to declaratively map external AST structures to Rascal's internal data structures. Tympanic allows implementors of Concretely to solve the impedance mismatch between object-oriented class hierarchies in Java and Rascal's algebraic data types. Both the algebraic data type and AST marshalling code is automatically generated. Knowledge: The conceptual architecture of Concretely and Tympanic supports the reuse of pre-existing, external parsers, and their AST representation in meta programming systems that feature concrete syntax patterns for matching and constructing syntax trees. As such this opens up concrete syntax pattern matching for a host of realistic languages for which writing a grammar from scratch is time consuming and error-prone, but for which industry-strength parsers exist in the wild. Grounding: We evaluate Concretely in terms of source lines of code (SLOC), relative to the size of the AST data type and marshalling code. We show that for real programming languages such as C++ and Java, adding support for concrete syntax patterns takes an effort only in the order of dozens of SLOC. Similarly, we evaluate Tympanic in terms of SLOC, showing an order of magnitude of reduction in SLOC compared to manual implementation of the AST data types and marshalling code. Importance: Meta programming has applications in reverse engineering, reengineering, source code analysis, static analysis, software renovation, domain-specific language engineering, and many others. Processing of syntax trees is central to all of these tasks. Concrete syntax patterns improve the practice of constructing meta programs. The combination of Concretely and Tympanic has the potential to make concrete syntax patterns available with very little effort, thereby improving and promoting the application of meta programming in the general software engineering context.

A vectorization approach for multifaceted solids in VecGeom

VecGeom [1] is a multi-purpose geometry library targeting the optimisation of the 3D-solids’ algorithms used extensively in particle transport and tracking applications. The implementations of these algorithms are templated on the input data type and are vectorised based on the VecCore [2] abstraction library in case of multiple inputs in a SIMD vector. This provides additional performance for applications supporting a multi-particle flow, such as the GeantV [3] prototype. VecGeom allows also scalar queries for all the supported solids, an option that started being used in Geant4 [4] since the release 10.2, as optional replacement of the geometry functionality provided by the native Geant4 solids. In single particle mode, VecGeom can still issue SIMD instructions by vectorizing the geometry algorithms featuring loops over internal data structures. This approach has proven to bring very large benefits for the tessellated solids represented in terms of triangular facets. To expose more vectorization in the scalar mode we have extended the approach used for the triangular tessellations to other multifaceted shapes, such as the extruded polygon, the poly-hedra and different trapezoids. We hereby present the strategy used to vectorise the different processing phases for tessellated solids, the performance improvements compared to the previous scalar implementations for other solids using this approach, and how this is reflected in Geant4 simulations using VecGeom as geometry engine.

A novel JSON based regular expression language for pattern matching in the internet of things

The Internet of Things work by constantly sensing the physical properties in the vicinity of the user such as ambient light, sounds, motion and temperature. These sensors produce huge volumes of data that has to be efficiently sifted for relevant events required triggering certain actions. In addition, filtering has to be performed to ensure that privacy-sensitive confidential data is not leaked. Efficient and expressive pattern matching is thus a key enabling technology for the full realization of ambient and humanized computing. The bulk of research in this area has focused on the use of specialized hardware and reducing of the memory footprint. Unfortunately, there has been limited work if any on optimizing the core elements of pattern matching- the regular expression language and the compilation process that is responsible for converting patterns into internal data structures. The importance of writing good REs so that on compilation they do not lead to unrealizable data structures is relatively less understood. In the proposed research, we empirically compare different RE processing engines and practically demonstrate that the compilation phase is highly memory intensive and time-consuming as compared to the matching phase -and hence is worth exploring for new techniques and optimizations. As a second important contribution, we propose a novel technique for defining regular expressions by utilizing JavaScript Object Notation. Our evaluation with carefully created patterns shows that the performance of the proposed technique is at par with competing approaches. It is also less ambiguous, extensible, more expressive and much appropriate for defining large and complex patterns.

Algorithmic debugging generalized

Algorithmic debugging is a semi-automatic debugging technique that abstracts the operational details of computations, allowing the programmers to debug their code from an abstract point of view. However, its use in practice is still marginal, and one of the reasons is the lack of precision of this technique when reporting errors (current algorithmic debuggers do not point an expression or line as buggy, but they point a whole procedure/function/method as containing the bug). In this paper, we make a step forward to overcome this problem. We identify two specific causes of that problem in the standard formulation and implementations of algorithmic debugging, and we present a reformulation to solve both problems. We show that the novel ideas included in the reformulation proposed cannot be supported by the standard internal data structures (such as the Execution Tree) used in this technique and, hence, a generalisation of the standard definitions and algorithms is needed. The reformulation has been done in a language-independent manner to make it useful and reusable in different programming languages.

Synchronizing Parallel Geometric Algorithms on Multi-Core Machines

A thread synchronization mechanism dubbed Spatial Locks for parallel geometric algorithms is presented. We show that Spatial Locks ensure thread synchronization on geometric algorithms that perform concurrent operations on geometric surfaces in two-dimensional or three-dimensional space. The proposed technique respects the fact that these operations follow a certain order of processing, i.e. priorities. Parallelizing these kinds of geometric algorithms using Spatial Locks requires only a simple parameter initialization, rather than modifying the algorithms themselves together with their internal data structures. A parallel algorithm for mesh simplification is chosen to show the Spatial Locks usefulness when parallelizing geometric algorithms with ease on multi-core machines. Experimental results illustrate the advantage of using this synchronization mechanism where significant computational improvement can be achieved compared to alternative approaches.

Low-Dimensional Spaces for Relating Sensor Signals with Internal Data Structure in a Propulsion System

Low-Dimensional Spaces for Relating Sensor Signals with Internal Data Structure in a Propulsion System

Smart Objects Assisted Event Detection System Using Wireless Sensor Networks

In recent years, the technological improvement in wireless smart communication (WSC) and mobile computing stimulated a consistent growth both in number and types of implementations for Smart Classes and Objects (SCO). Significantly enlarging their battery longevity and hence their serviceability, it is essential to minimize power consumption and thus ameliorate power efficiency. Before a smart device initiates communication, it must discover the set of nodes that are within its direct communicating range. The gathered relevant information is kept in its internal data structures to apply with routing. The behavior of an ad-hoc and dynamic devices relies on the functioning of its neighboring interconnected devices since it must sense the medium before it starts transmitting packets to nodes in its interfering range, which can give rise to collisions at the other devices. The wireless ad-hoc devices acclimatize dynamically its topology knowledge range, leading to faster convergence of its neighboring nodes to establish smart objects communication.The design goal of SCO is to acquire the real time data acquisition with an impact of supervisory control for large scale Remote Environment. Proposed system considers a logging System for a remote communicationand the power that is being wasted will be saved using sensors to indicate the microcontroller to increase or decrease the power to the required levels. Our proposed system is to help in automation of SCO that in turn makes our lives easier.

FMM/GPU-Accelerated Boundary Element Method for Computational Magnetics and Electrostatics

A fast multipole method (FMM)/graphics processing unit-accelerated boundary element method (BEM) for computational magnetics and electrostatics via the Laplace equation is presented. The BEM is an integral method, but the FMM is typically designed around monopole and dipole sources. To apply the FMM to the integral expressions in the BEM, the internal data structures and logic of the FMM must be changed. However, this can be difficult. For example, computing the multipole expansions due to the boundary elements requires computing single and double surface integrals over them. Moreover, FMM codes for monopole and dipole sources are widely available and highly optimized. This paper describes a method for applying the FMM unchanged to the integral expressions in the BEM. This method, called the correction factor matrix method, works by approximating the integrals using a quadrature. The quadrature points are treated as monopole and dipole sources, which can be plugged directly into current FMM codes. The FMM is effectively treated as a black box. Inaccuracies from the quadrature are corrected during a correction factor step. The method is derived, and example problems are presented showing accuracy and performance.

Efficient spatial co-location pattern mining on multiple GPUs

In this paper, we investigate Co-location Pattern Mining (CPM) from big spatial datasets. CPM consists in searching for types of objects that are frequently located together in a spatial neighborhood. Knowledge about such patterns is very important in fields like biology, environmental sciences, epidemiology etc. However, CPM is computationally challenging, mainly due to the large number of pattern instances hidden in spatial data. In this work, we propose a new solution that can utilize the power of multiple GPUs to increase the performance of CPM. The proposed solution is also capable of coping with the GPU memory limits by dividing the work into multiple packages and compressing internal data structures. Experiments performed on large synthetic and real-world datasets prove that we can achieve an order of magnitude speedups in comparison to the efficient multithreaded CPU implementation. Our solution can greatly improve the performance of data analysis, using widely available and energy efficient graphics cards. As a result, CPM in large datasets is more viable for university researchers as well as smaller companies and organizations.