High-level Data Structures Research Articles

MOBILE APP DEVELOPMENT TO INCREASE STUDENT ENGAGEMENT AND PROBLEM SOLVING SKILLS USING ANDROID TECHNOLOGY

Programming is a fundamental ability for Computer Science Students. Most programming lectures use traditional languages, such as C, C++, or Java, as the rst learning language for beginners, which are effective for real applications and therefore are popular in industry. However, the complex syntax of these languages is challenging for beginners, which becomes an obstacle to their learning. Python has simpler syntax and high-level data structures to enable succinct programs. The multiple paradigms of Python also provide learners opportunities to learn various features of programming languages. Therefore, Python gradually becomes a new option of the rst language for learning. This Project investigates the features of the rst learning programming language by comparing Python with the popular learning language Java. The results has given instructors suggestions for selecting suitable learning languages in their lectures and designing adaptive instructional strategies/materials based on the features of the selected language.

Компилятор с языка программирования EL: усовершенствование и развитие

The implementation of the new multi-paradigm (functionally- imperative) programming language El, developed at the Department of Computer Science of the Novosibirsk State Technical University, in the form of a compiler is associated with the need to find ways to solve a number of complex problems. The current version of the compiler does implement only partially functionality of the language and generates far from optimal executable codes. In this paper, we consider the problem of an efficient compilation of an El-program, taking into account the need to implement new high-level data structures (two-sided lists, vectors with special forms of access, and a number of others) and control structures of the language, which make it possible to uniformly define cyclic and branching computational processes, as well as those laid down in the language a mechanism for explicitly controlling the mutability of variables. The tasks of improving and developing a compiler organized according to the classical multi-platform scheme are briefly considered, in which the front-end (lexical, syntactic, and semantic analyzers) converts the program to be translated into pseudocode of a single format, and used efficient infrastructure for building LLVM compilers as a back-end that turns pseudocode into executable code for different platforms. Execution of all possible operations on elements of high-level data structures (lists, tuples, vectors), as well as on arbitrary-precision numbers, has been moved to the runtime support library and, accordingly, can be deeply optimized. For this structure, the outlined ways of solving the problem of developing and improving the compiler by deep reforming and optimization of the chain of transformations of the translated program implemented by the front-end are formulated. At the initial stage, it is planned to implement a new compiler for two platforms: Linux and Windows.

Fast Search of the Optimal Contraction Sequence in Tensor Networks

Tensor network and tensor computation are widely applied in scientific and engineering domains like quantum physics, electronic design automation, and machine learning. As one of the most fundamental operations for tensor networks, a tensor contraction eliminates the sharing orders among tensors and produces a compact sub-network. Different contraction sequence usually yields distinct storage and compute costs, and searching the optimal sequence is known as a hard problem. Prior work have designed heuristic and fast algorithms to solve this problem, however, several issues still remain unsolved. For example, the data format and data structure are not efficient, the constraints during modeling are impractical, the search of the optimal solution might fail, and the search cost is very high. In this paper, we first introduce a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$log_k$</tex-math></inline-formula> order representation and design an adjacency matrix-based data structure to efficiently accelerate the search of the optimal contraction sequence. Then, we propose an outer product pruning method with acceptable overhead to reduce the search space. Finally, we use a multithread optimization in our implementation to further improve the execution performance. We also present in-depth analysis of factors that influence the search time. This work provides a full-stack solution for optimal contraction sequence search from both high-level data structure and search algorithm to low-level execution parallelism, and it will benefit a broad range of tensor-related applications.

Data Reduction in the String Space for Efficient kNN Classification Through Space Partitioning

Within the Pattern Recognition field, two representations are generally considered for encoding the data: statistical codifications, which describe elements as feature vectors, and structural representations, which encode elements as high-level symbolic data structures such as strings, trees or graphs. While the vast majority of classifiers are capable of addressing statistical spaces, only some particular methods are suitable for structural representations. The kNN classifier constitutes one of the scarce examples of algorithms capable of tackling both statistical and structural spaces. This method is based on the computation of the dissimilarity between all the samples of the set, which is the main reason for its high versatility, but in turn, for its low efficiency as well. Prototype Generation is one of the possibilities for palliating this issue. These mechanisms generate a reduced version of the initial dataset by performing data transformation and aggregation processes on the initial collection. Nevertheless, these generation processes are quite dependent on the data representation considered, being not generally well defined for structural data. In this work we present the adaptation of the generation-based reduction algorithm Reduction through Homogeneous Clusters to the case of string data. This algorithm performs the reduction by partitioning the space into class-homogeneous clusters for then generating a representative prototype as the median value of each group. Thus, the main issue to tackle is the retrieval of the median element of a set of strings. Our comprehensive experimentation comparatively assesses the performance of this algorithm in both the statistical and the string-based spaces. Results prove the relevance of our approach by showing a competitive compromise between classification rate and data reduction.

Using ACL2 in the Design of Efficient, Verifiable Data Structures for High-Assurance Systems

Verification of algorithms and data structures utilized in modern autonomous and semi-autonomous vehicles for land, sea, air, and space presents a significant challenge. Autonomy algorithms, e.g., route planning, pattern matching, and inference, are based on complex data structures such as directed graphs and algebraic data types. Proof techniques for these data structures exist, but are oriented to unbounded, functional realizations, which are not typically efficient in either space or time. Autonomous systems designers, on the other hand, generally limit the space and time allocations for any given function, and require that algorithms deliver results within a finite time, or suffer a watchdog timeout. Furthermore, high-assurance design rules frown on dynamic memory allocation, preferring simple array-based data structure implementations. In order to provide efficient implementations of high-level data structures used in autonomous systems with the high assurance needed for accreditation, we have developed a verifying compilation technique that supports the "natural" functional proof style, but yet applies to more efficient data structure implementations. Our toolchain features code generation to mainstream programming languages, as well as GPU-based and hardware-based realizations. We base the Intermediate Verification Language for our toolchain upon higher-order logic; however, we have used ACL2 to develop our efficient yet verifiable data structure design. ACL2 is particularly well-suited for this work, with its sophisticated libraries for reasoning about aggregate data structures of arbitrary size, efficient execution of formal specifications, as well as its support for "single-threaded objects" -- functional datatypes with imperative "under the hood" implementations. In this paper, we detail our high-assurance data structure design approach, including examples in ACL2 of common algebraic data types implemented using this design approach, proofs of correctness for those data types carried out in ACL2, as well as sample ACL2 implementations of relevant algorithms utilizing these efficient, high-assurance data structures.

A high-performance atmospheric radiation package: With applications to the radiative energy budgets of giant planets

A High-performance Atmospheric Radiation Package (HARP) is developed for studying multiple-scattering planetary atmospheres. HARP is an open-source program written in C + + that utilizes high-level data structure and parallel-computing algorithms. It is generic in three aspects. First, the construction of the model atmospheric profile is generic. The program can either take in an atmospheric profile or construct an adiabatic thermal and compositional profile, taking into account the clouds and latent heat release due to condensation. Second, the calculation of opacity is generic, based on line-by-line molecular transitions and tabulated continuum data, along with a table of correlated-k opacity provided as an option to speed up the calculation of energy fluxes. Third, the selection of the solver for the radiative transfer equation is generic. The solver is not hardwired in the program. Instead, based on the purpose, a variety of radiative transfer solvers can be chosen to couple with the atmosphere model and the opacity model.We use the program to investigate the radiative heating and cooling rates of all four giant planets in the Solar System. Our Jupiter's result is consistent with previous publications. Saturn has nearly perfect balance between the heating rate and cooling rate. Uranus has the least radiative fluxes because of the lack of CH4 and its photochemical products. Both Uranus and Neptune suffer from a severe energy deficit in their stratospheres. Possible ways to resolve this issue are discussed. Finally, we recalculate the radiative time constants of all four giant planet atmospheres and find that the traditional values from (Conrath BJ, Gierasch PJ, Leroy SS. Temperature and Circulation in the Stratosphere of the Outer Planets. Icar. 1990;83:255–81) are significantly overestimated.

ACTS: from ATLAS software towards a common track reconstruction software

Reconstruction of charged particles’ trajectories is a crucial task for most particle physics experiments. The high instantaneous luminosity achieved at the LHC leads to a high number of proton-proton collisions per bunch crossing, which has put the track reconstruction software of the LHC experiments through a thorough test. Preserving track reconstruction performance under increasingly difficult experimental conditions, while keeping the usage of computational resources at a reasonable level, is an inherent problem for many HEP experiments. Exploiting concurrent algorithms and using multivariate techniques for track identification are the primary strategies to achieve that goal.Starting from current ATLAS software, the ACTS project aims to encapsulate track reconstruction software into a generic, framework- and experiment-independent software package. It provides a set of high-level algorithms and data structures for performing track reconstruction tasks as well as fast track simulation. The software is developed with special emphasis on thread-safety to support parallel execution of the code and data structures are optimised for vectorisation to speed up linear algebra operations. The implementation is agnostic to the details of the detection technologies and magnetic field configuration which makes it applicable to many different experiments.

High Level Data Structures for GPGPU Programming in a Statically Typed Language

To increase software performance, it is now common to use hardware accelerators. Currently, GPUs are the most widespread accelerators that can handle general computations. This requires to use GPGPU frameworks such as Cuda or OpenCL. Both are very low-level and make the benefit of GPGPU programming difficult to achieve. In particular, they require to write programs as a combination of two subprograms, and, to manually manage devices and memory transfers. This increases the complexity of the overall software design. The idea we develop in this paper is to guarantee expressiveness and safety for CPU and GPU computations and memory managements with high-level data-structures and static type-checking. In this paper, we present how statically typed languages, compilers and libraries help harness high level GPGPU programming. In particular, we show how we added high-level user-defined data structures to a GPGPU programming framework based on a statically typed programming language: OCaml. Thus, we describe the introduction of records and tagged unions shared between the host program and GPGPU kernels described via a domain specific language as well as a simple pattern matching control structure to manage them. Examples, practical tests and comparisons with state of the art tools, show that our solutions improve code design, productivity, and safety while providing a high level of performance.

LinZinc: A Library for Linearizing Zinc Models

Zinc is a modeling language in which a conceptual model is automatically mapped into different design models using well-known solving techniques: constraint programming, mathematical methods, and local search. As a result, Zinc is solver and technique independent. Since original conceptual models may be nonlinear, mapping to linear solvers requires the models to be linearized first. In this paper, we introduce LinZinc, a new library for Zinc which includes necessary functions to linearize high-level data structures and expressions in Zinc models. The interesting point is that, thanks to the extensibility of Zinc, LinZinc is written in Zinc itself instead of being implemented in the compiler. We have used LinZinc for linearizing a set of well-known nonlinear models. The experimental results show that on the one hand, there is no significant overhead in linearizing models and, on the other hand, the linearized models are sometimes faster than the original models. Using the proposed approach, Zinc users can implement their own linearizing techniques without any need to change the language and without being anxious about the mapping overhead.

Scalable data structure detection and classification for C/C++ binaries

Many existing techniques for reversing data structures in C/C ++ binaries are limited to low-level programming constructs, such as individual variables or structs. Unfortunately, without detailed information about a program's pointer structures, forensics and reverse engineering are exceedingly hard. To fill this gap, we propose MemPick, a tool that detects and classifies high-level data structures used in stripped binaries. By analyzing how links between memory objects evolve throughout the program execution, it distinguishes between many commonly used data structures, such as singly- or doubly-linked lists, many types of trees (e.g., AVL, red-black trees, B-trees), and graphs. We evaluate the technique on 10 real world applications, 4 file system implementations and 16 popular libraries. The results show that MemPick can identify the data structures with high accuracy.

Data Quality Assessment

A state-of-the-art in the domain of data quality assessment and maintaining in modern information systems is presented in the paper. A short historical view on the development of this problem is given. Particular attention is paid to the development of the idea of multi-aspect data quality assessment. The problem of extension of single data quality assessment on higher-level data structures is considered. The methods of multi-aspect data quality measures ordering for comparison is analyzed and its solution based on the concept of semi-ordering in linear vector (Kantorovitsh) space is proposed. Remarks on organizational and technological tools for data quality maintenance in organizations are given. Expected future trends in the development of data quality assessment and maintenance methods are suggested.

Optimizing data structures in high-level programs

High level data structures are a cornerstone of modern programming and at the same time stand in the way of compiler optimizations. In order to reason about user- or library-defined data structures compilers need to be extensible. Common mechanisms to extend compilers fall into two categories. Frontend macros, staging or partial evaluation systems can be used to programmatically remove abstraction and specialize programs before they enter the compiler. Alternatively, some compilers allow extending the internal workings by adding new transformation passes at different points in the compile chain or adding new intermediate representation (IR) types. None of these mechanisms alone is sufficient to handle the challenges posed by high level data structures. This paper shows a novel way to combine them to yield benefits that are greater than the sum of the parts. Instead of using staging merely as a front end, we implement internal compiler passes using staging as well. These internal passes delegate back to program execution to construct the transformed IR. Staging is known to simplify program generation, and in the same way it can simplify program transformation. Defining a transformation as a staged IR interpreter is simpler than implementing a low-level IR to IR transformer. With custom IR nodes, many optimizations that are expressed as rewritings from IR nodes to staged program fragments can be combined into a single pass, mitigating phase ordering problems. Speculative rewriting can preserve optimistic assumptions around loops. We demonstrate several powerful program optimizations using this architecture that are particularly geared towards data structures: a novel loop fusion and deforestation algorithm, array of struct to struct of array conversion, object flattening and code generation for heterogeneous parallel devices. We validate our approach using several non trivial case studies that exhibit order of magnitude speedups in experiments.

High-level data structures

High-level data structures

High-level algorithms and data structures requirements for security-by-contract on Java cards

The Java Card technology has progressed to the point of running web servers and web clients on a smart card. Yet concrete deployments of multi-applications smart cards have remained extremely rare because the business model of the asynchronous download and update of applications by different parties requires the control of interactions among possible applications after the card has been fielded. The current security models and techniques do not support this type of evolution. We propose in this paper to apply the notion of security-by-contract (S×C), that is a specification of security-related behaviour of an application that must be compliant with the security policy of the hosting platform. This compliance can be checked at the application loading time, avoiding in this way the need of costly runtime monitoring. We show how S×C can be used to prevent illegal information exchange among applications on a single smart card platform in presence of dynamic changes on the card.

Realization CNC Controller Enable Machine Condition Monitoring Architecture Based on STEP-NC Data Model

The drive for increased productivity, reduced production time and cost, reduced defective parts and relaxed machine design constraints, pushes for real-time control and optimization for machining operations. This vision can be realized if the current CNC controllers can be adaptable, portable, interoperable and intelligent in responding quickly and efficiently in the product lifecycle domain. In this study, the development of a realization CNC controller enabled machine condition monitoring architecture based on STEP-NC data model to support an automated and intelligent machining environment is introduced. The controller allows canonical machine commands to be executed with a fuzzy feed-rate optimization modul to serve the purpose of allowing in-process optimization to be automatically incorporated during on-going machining operation. This research has shown that it is more prolific to utilize high-level data structure as well as a universal interface for machine execution.

Data centric techniques for mapping performance data to program variables

Traditional methods of performance analysis offer a code centric view, presenting performance data in terms of blocks of contiguous code (statement, basic block, loop, function). Data centric techniques, combined with hardware counter information, allow various program properties including cache misses and cycle count to be mapped directly to variables. We introduce mechanisms for efficiently collecting data centric performance numbers independent of hardware support. We create extended data centric mappings, which we call variable blame, that relates data centric information to high level data structures. Finally, we show performance data gathered from three parallel programs using our technique.

On the performance of distributed lock-based synchronization?

Distributed mutual exclusion locks are the de facto mechanism for concurrency control on distributed data structures. A process accesses the data structure only while holding the lock, and hence the process is guaranteed exclusive access. The popularity of this approach is largely due to the apparently simple programming model of such locks and the availability of efficient implementations. We study the relation between classical types of distributed locking mechanisms and several distributed data structures which use locking for synchronization. Our objectives are: To determine which one of the two classical locking techniques -- token-based locking or permission-based locking -- is more efficient. Our strategy to achieve this objective is to implement several locks and to compare their performance. To propose, implement and test several distributed data structures, namely, two different types of counters, a queue, a stack and a linked list; and for each one of the data structures to determine what is the preferred mutual exclusion lock to be used as the underling locking mechanism. To determine which one of the two proposed counters is better to be used either as a stand-alone data structure or when used as a building block for implementing other high level data structures. Our testing environment is consisting of 20 Intel XEON 2.4 GHz machines running theWindows XP OS with 2GB of RAM and using a JRE version 1.4.2_08. All the machines were located inside the same LAN and were connected using a 20 port Cisco switch.

Memory-access-aware data structure transformations for embedded software with dynamic data accesses

Embedded systems are evolving from traditional, stand-alone devices to devices that participate in Internet activity. The days of simple, manifest embedded software [e.g. a simple finite-impulse response (FIR) algorithm on a digital signal processor (DSP] are over. Complex, nonmanifest code, executed on a variety of embedded platforms in a distributed manner, characterizes next generation embedded software. One dominant niche, which we concentrate on, is embedded, multimedia software. The need is present to map large scale, dynamic, multimedia software onto an embedded system in a systematic and highly optimized manner. The objective of this paper is to introduce high-level, systematically applicable, data structure transformations and to show in detail the practical feasibility of our optimizations on three real-life multimedia case studies. We derive Pareto tradeoff points in terms of accesses versus memory footprint and obtain significant gains in execution time and power consumption with respect to the initial implementation choices. Our approach is a first step to systematically applying high-level data structure transformations in the context of memory-efficient and low-power multimedia systems.

A PDM- and CAD-integrated assembly modelling environment for manufacturing planning

This paper reports the development of an advanced assembly modelling environment for the purposes of manufacturing planning and analysis. computer-aided assembly process planning (CAAPP) is not a mature technology, despite the managerial and technical advantages offered by its implementation. This paper describes the limiting factors in industrial acceptance of CAAPP and discusses methodologies by which these factors can be eliminated. This is done primarily by the integration of CAAPP functionality with a product data management (PDM) tool, providing a data control framework and a high-level data structure to form the basis of planning. The paper also details a system based on the described methodologies. Data control methodologies are presented to aid retrieval, storage and creation. Methods for the definition of liaisons, mating features and fastener information are discussed. The methodologies used for constraint identification, process sequence generation and sequence representation within the system are described. Multiple sequences are represented by means of a tree structure, similar to the AND/OR graph representation. The sequence is navigable and can be used downstream to evaluate possible sequences of generate scheduling data. The underlying object-oriented model developed to facilitate automated assembly process planning is described in detail.

A product data management integrated product configuration and assembly process planning environment

This paper reports part of the results of a 3 year project carried out at Cranfield University in collaboration with companies in the automotive and aerospace industry, aiming to develop new methodologies for computer aided assembly process planning (CAAPP). CAAPP is not a mature technology, despite the managerial and technical advantages offered by its implementation. This paper describes the limiting factors in industrial acceptance of CAAPP and discusses methodologies by which these factors can be eliminated. This is done primarily by the integration of CAAPP functionality with a product data management (PDM) tool, providing a data control framework and a high-level data structure to form the basis of planning. The paper also details a system based on the described approach. The underlying object-oriented model developed to facilitate automated assembly process planning is described in detail. Data control methodologies are presented to aid retrieval, storage and creation. Methods for the definition of liaisons, mating features, are discussed. The methodologies used for constraint identification, process sequence generation and sequence representation within the system are described. Multiple sequences are represented by means of a tree structure, similar to the AND/OR graph representation. The sequence is navigable and can be used for downstream sequence evaluation and simulation or to generate scheduling data.