Data Interchange Format Research Articles

A messaging library for distributed modeling

The ongoing digital transformation of industry is most clearly reflected in the increasing collection and analysis of data from various sources. Among these are sensor equipped machinery, telemetry in logistics or audiovisually monitored production floors. In order to utilize the data, e. g., to predict machinery malfunctions, its technically smooth consolidation is crucial. Therefore, numerous data interchange formats, protocols and middleware emerged over the past years. However, until today there is no gold standard technology stack for industrial data analysis, for multiple reasons, including the applications’ heterogeneity.In this work, we present a software library which aims at decoupling messaging protocols and patterns from their implementation to overcome incompatibilities and, thus, facilitate data consolidation for software engineers. Moreover, we show how to use the library for rapidly modeling distributed cyber-physical systems using an integrated schema generation mechanism. Based on one real-world and one synthetic use case, we evaluate the library's applicability, discuss open issues and outline planned features.

Development of a web application for hotspot token vending machine administration

This article describes the features of a web application to manage the availability and acquisition of wireless internet access credentials of vending machine, it followed the OOHMD agile software development methodology, for the implementation we used the client-server architecture, the JavaScript programming language, the JavaScript runtime environment of Node. js JavaScript runtime environment, Express web development framework for Node.js, Sequelize ORM, MySQL relational database management system, Passport authentication middleware, EJS open source template engine, Bootstrap CSS framework, JSON data interchange format and Visual Studio Code IDE. As a contribution, there is a web application that allows the synchronization of available tokens and the sale of a wireless internet access credential vending machine, which connects through the internet. The methodology used allowed to have an adequate development process to obtain a quality product that resulted in a web application that efficiently provides the available access credentials and manages the access credentials acquired from the vending machine.

DsJSON: A Distributed SQL JSON Processor

The popularity of JSON as a data interchange format resulted in big amounts of datasets available for processing. Users would like to analyze this data using SQL queries but existing distributed systems limit their users to only two specific formats, JSONLine and GeoJSON. The complexity of JSON schema makes it challenging to parse arbitrary files in a modern distributed system while producing records with unified schema that can be processed with SQL. To address these challenges, this paper introduces dsJSON, a state-of-the-art distributed JSON processor that overcomes limitations in existing systems and scales to big and complex data. dsJSON introduces the projection tree, a novel data structure that applies selective parsing of nested attributes to produce records that are ready for SQL processors. The key objective of the projection tree is to parse a big JSON file in parallel to produce records with a unified schema that can be processed with SQL. dsJSON is integrated into SparkSQL which enables users to run arbitrary SQL queries on complex JSON files. It also pushes projection and filter down into the parser for full integration between the parser and the processor. Experiments on up-to two terabytes of real data show that dsJSON performs several times faster than existing systems. It can also efficiently parse extremely large files not supported by existing distributed parsers

Time-frequency catalogue: JSON implementation and python library

TFCat (Time-Frequency Catalogue) is a data interchange format based on JSON (JavaScript Object Notation), which has been initially designed for exchanging low frequency radio events and features. It defines several types of JSON objects and how they are combined to represent data related to temporal-spectral features of a time spectrogram (a.k.a., dynamic spectrum), their properties, and their temporal and spectral extents. This implementation is inheriting from the GeoJSON file format. The TFCat python library is implementing this specification and provides a software interface permitting to create, update and validate TFCat objects efficiently.

Auto JSON: An Automatic Transformation Model for Converting Relational Database to Non-relational Documents

In recent days, the demand for dealing large set of distributed data obsoletes the relational database and its structured query language (SQL) solutions in practice and paves the way for novel solutions in the name of non-relational database as not-only SQL (NoSQL). The NoSQL offers dynamic, flexible, scalable, highly available, greater performance and near real-time access to the distributed nature of voluminous data used for current industrial applications. Apart from these giant features of NoSQL, the SQL is still found to be in operation because of its popularity and standard. This paper projected an algorithm to convert the relational documents of MySQL into any document oriented NoSQL databases automatically without destructing the existing relational database setup and installing the NoSQL from scratch in the core machines. Java Script Object Notation (JSON) is a human readable data interchange format, being used in web development. The characteristics of JSON widened its use cases from web development to database storage. The Mongo DB, one of the most popular document oriented NoSQL adapts JSON format for its storage. The proposed algorithm is built based on its schema definition and the performance is captured through evaluating it against a sample database from hospital management system. The findings are discussed with great interest of addressing the challenges and revealing the scope for improvement.

JSON document clustering based on schema embeddings

The growing popularity of JSON as the data storage and interchange format increases the availability of massive multi-structured data collections. Clustering JSON documents has become a significant issue in organising large data collections. Existing research uses various structural similarity measures to perform clustering. However, differently annotated JSON structures may also encode semantic relatedness, necessitating the use of both syntactic and semantic properties of heterogeneous JSON schemas. Using the SchemaEmbed model, this paper proposes an embedding-based clustering approach for grouping contextually similar JSON documents. The SchemaEmbed model is designed using the pre-trained Word2Vec model and a deep autoencoder that considers both syntactic and semantic information of JSON schemas for clustering the documents. The Word2Vec model learns the attribute embeddings, and a deep autoencoder is designed to generate context-aware schema embeddings. Finally, the context-based similar JSON documents are grouped using a clustering algorithm. The effectiveness of the proposed work is evaluated using both real and synthetic datasets. The results and findings show that the proposed approach improves clustering quality significantly, with a high NMI score of 75%. In addition, we demonstrate that clustering results obtained by contextual similarity are superior to those obtained by traditional semantic similarity models.

Development of an application to anonymize data to be shared in the cloud

A design of an interface is proposed to be developed as a lightweight application that allows the use of different algorithms to anonymize proposed by various authors, the application focuses on adapting data from distributed applications that must interact using the JSON lightweight Data Interchange Format. The user interaction is minimized for the anonymization process, an interface is provided to facilitate the selection process of the anonymization algorithms that users choose.

Development of an application to anonymize data to be shared in the cloud

A design of an interface is proposed to be developed as a lightweight application that allows the use of different algorithms to anonymize proposed by various authors, the application focuses on adapting data from distributed applications that must interact using the JSON lightweight Data Interchange Format. The user interaction is minimized for the anonymization process, an interface is provided to facilitate the selection process of the anonymization algorithms that users choose.

Retracted] Design of Moral Education Management System for Higher Vocational Students Based on Multisource Sensing Data Fusion

Based on the specific circumstances of colleges and universities, this study presents the moral education management system and leverages multisource sensor data and other technologies to develop an information system in line with the moral education management of modern college students. Using multisource sensor data fusion and learning data definition, the fused education data is expressed in a standardized way, resulting in a common and standard data interchange format. This foundation is used to build a shareable and reusable data management system that enables data sharing and interchange across multiple heterogeneous data sources, aids the intelligent education system in obtaining more comprehensive and complete recorded data, and improves data sharing timeliness. As a result, learning behavior analysis results are more objective, immediate, and accurate, allowing the intelligent education system’s response to be more prompt and intelligent. This study, which is based on multisource sensor data fusion, also discusses platform selection, system architecture, database design, and methods and suggestions for overcoming critical challenges and can be used as a reference for other university information management systems with vast amounts of data.

Measurement project interoperability for real-time data gathering systems

The Internet-of-Things devices have allowed the increment of the Spatiotemporal resolution in the Real-time Data Gathering Systems. However, this has increased the complexity due to the heterogeneity of sensors. Thus, it is important to know how data can be understood aware of its context at the edge. Also, a transversal perspective is expected for aligning and reusing as many sensors as possible (based on an experimental design). Because of the hardware limitations, the experimental design should be communicated as simple, integrated, and consistent as possible under single content fostering the projects’ complementarity. This work describes a strategy for defining and communicating a set of measurement projects, following a conceptual hierarchy derived from a measurement ontology. Every project definition is obtained from the user requirements following a well-established strategy. BriefPD is introduced as a data interchange format that is self-contained, partially verifiable (through a Merkle tree insight), consistent, and does not require the use of tags to determine the content meaning. BriefPD generates a project definition in 6.27 ms (1.9 times quicker than JSON-equivalent), consuming 20.39% of JSON’s size. A Github library is provided with the reference implementation jointly with a ‘Code Ocean’ capsule for reproducing the simulation results.

JSON Schemas with Semantic Annotations Supporting Data Translation

As service-oriented architectures are a solution for large distributed systems, interoperability between these systems, which are often heterogeneous, can be a challenge due to the different syntax and semantics of the exchanged messages or even different data interchange formats. This paper addresses the data interchange format and data interoperability issues between XML-based and JSON-based systems. It proposes novel annotation mechanisms to add semantic annotations and complement date values to JSON Schemas, enabling an interoperability approach for JSON-based systems that, until now, was only possible for XML-based systems. A set of algorithms supporting the translation from JSON Schema to XML Schema, JSON to XML, and XML to JSON is also proposed. These algorithms were implemented in an existing prototype tool, which now supports these systems’ interoperability through semantic compatibility verification and the automatic generation of translators.

Regulations and Standards Aware Framework for Recording of mHealth App Vulnerabilities

The authors describe a standards-based security framework for the purposes of recording security and privacy vulnerabilities discovered in mHealth apps. The proposed framework is compliant with the international standard for software architecture descriptions, ISO/IEC/IEEE 42010, relevant state-agency regulations, and US federal healthcare mandates, as well as computing standards for data interchange formats. Future real-life implementations are envisioned to consists of three key components: (1) design and implementation of a repository that links vulnerabilities to concepts from the taxonomy used by legislative and standardization bodies; (2) population of the repository with security vulnerability descriptions that follow a standard format, such as JavaScript Object Notation (JSON); and (3) implementation of a searchable user interface (e.g., Google's Firebase UI), which allows for aggregation statistics, data analytics, as well as public access to the repository. The proposed framework design promotes timely updates of regulations, standardization drafts, and app development platforms.

Encoding JSON by using Base64

Transmitting binary data across a network should generally avoid transmitting raw binary data over the medium for several reasons, one would be that the medium may be a textual one and may not accept or correctly handle raw bitstream, another would be that some protocols may misinterpret the meaning of the bits and causes a problem or even loss of the data. To make the data more readable and would avoid misinterpretation by different systems and environments, this paper introduces encoding two of the most broadly used data interchange formats, XML and JSON, into the Base64 which is an encoding scheme that converts binary data to an ASCII string format by using a radix-64 representation. This process, will, make the data more readable and would avoid misinterpretation by different systems and environments. The results reflect that encoding data in Base64 before the transmission will present many advantages including readability and integrity, it will also enable us to transmit binary data over textual mediums, 7 Bit protocols such as SMTP, and different network hardware without risking misinterpretation.

PsiNorm: A fast, efficient and free open-source software for interpreting, reporting and archiving neuropsychological test results.

In many clinics, calculation and interpretation of neuropsychological test results, along with reporting, data organization and archiving of the data are done manually. In this era where most of the similar processes are automated, manual application may result in excessive time consumption, unnecessary use of qualified work-force, and is also open to error. A software that automates these processes for neuropsychological tests used for dementia assessment may overcome these issues. We aimed to develop a free, open source software not requiring specialized training, which would optimise the calculation, preparation of personal reports and archiving processes of neuropsychological tests, hence would easily be incorporated in to the daily work of psychologists. We've used Python 3.6 as the programming language, and JSON was used as the data interchange format to allow for personal alterations in the content. The tests were selected among those which are in common use for neuropsychological evaluation of adults in Turkey, with available norm values. (Funding: TUBITAK 214S048). PsiNorm was developed, comprising widely used standardized tests for cognitive evaluation of adults in Turkey. The software is lightweight, compatible with most common operating systems, and easy-to-use. We've shown that Psinorm significantly reduced the time required for calculation of percentiles and norms as well as for producing a draft report. The reports are prepared in .txt format and the databases are prepared in MS Excel ve CSV formats. PsiNorm is available freely at psinorm.org. PsiNorm is a free, open-source software which is available for researchers and clinicians who perform neuropsychological tests. PsiNorm provides significant time and labor-force benefits, is easy-touse and can be customized by the user.

Semi-structured data protection scheme based on robust watermarking

Semi-structured data is a widely used text format for data interchange and storage. This paper proposes a robust watermarking scheme of data protection for semi-structured data, which uses JSON format as an example for illustration. We first parse JSON file into a data structure of distinct pairs. Afterwards, we generate a transfer matrix to get the intermediate sequences, which are then encoded using error-correction codes and embedded into the pairs. A private key is shared by the data hider and the recipient to resist collusion attacks. On the recipient’s side, data extraction can be successfully carried out even the received stego data are tampered. The imperceptibility is realized by embedding data into the less significant digits of numeric data in the cover file. The proposed scheme can be extended on several other formats. The experimental results show that the proposed scheme is robust to various kinds of typical attacks such as contextual truncating, modification, and redundancy addition.

A robust computational DRM framework for protecting multimedia contents using AES and ECC

The last few years have witnessed a vast evolution in the communication and computational fields. The presence of social media makes data transferring more easier, which raises a flag of unauthorized usage and redistribution of digital contents. This copyright scenario is hardly affecting the publishing rights of authors and publishers. This paper presents a robust framework that protects the copyright property where only the authorized users legally use the data and prevents the illegal using or copying of data. The proposed framework is based on Digital Right Management (DRM) and includes two robust cryptosystems; Advanced Encryption Standard (AES) and Elliptic Curve Cryptography (ECC) algorithms. The multimedia contents are encrypted using the AES-256 algorithm, while the shared key is encrypted using the ECC algorithm. The authorized user can utilize the copyrighted contents if the authorization is not expired. The license generation is done by a lightweight data-interchange format JavaScript Object Notation (JSON), and the license is validated using the Elliptic Curve Digital Signature Algorithm. The evaluation of the system has shown that it is fulfilling all the requirements of DRM technology with low cost, availability, platform independently, reliability and scalability since it is implemented on a cloud platform.

Present Status of Neutron-, Photo-induced and Spontaneous Fission Yields Experimental Data

Nuclear reaction data collection, evaluation and dissemination have been pioneered at the Brookhaven National Laboratory since the early 50s. These activities gained popularity worldwide, and around 1970 the experimental nuclear reaction data interchange or exchange format (EXFOR) was established. The originalEXFORcompilation scope consisted only of neutron reactions and spontaneous fission data, while many other nuclear data sets were ignored. Due to the high cost of new experiments, it is very important to find and recover the previously disregarded data using scientific publications, data evaluations and nuclear databases comparisons.Fission yields play a very important role in applied and fundamental physics, and such data are essential in many applications. The comparative analysis of Nuclear Science References (NSR) and Experimental Nuclear Reaction (EXFOR) databases shows a large number of unaccounted experiments and provides a guide for the recovery of fission cross sections, yields and covariance data sets. The dedicated fission yields data compilation effort is currently underway in the Nuclear Reaction Data Centers (NRDC) network, and includes identification, compilation, storage and Web dissemination of the recovered data sets.

SAPIENS

We present SAPIENS, a software architecture designed to support engineering of interactive systems featuring peripheral interaction in the context of smart environments. SAPIENS introduces dedicated components for user and device tracking, attention detection, priority management for devices, tasks, and notifications, context-awareness inference, user interruptibility prediction, and device interchangeability that can be instantiated at will according to the needs of the application. To implement these components effectively, SAPIENS employs event-based processing by reusing the core engine of a recently introduced software architecture, Euphoria (Schipor et al., 2019), that was specifically designed for engineering interactions in smart environments with heterogeneous I/O devices, and relies entirely on web standards, protocols, and open data-interchange formats, such as JavaScript, WebSockets, HTTP, and JSON. This inheritance makes SAPIENS flexible and adaptable to support implementation of diverse application scenarios for peripheral interaction and for a wide variety of smart environments, devices, platforms, data formats, and contexts of use. We present our design criteria for SAPIENS regarding (1) event handling techniques, (2) quality, (3) contextual, and (4) attention-related properties, and describe its components and dataflows that make SAPIENS a specialized software architecture for peripheral interaction scenarios. We also demonstrate SAPIENS with a practical application, inspired and adapted from Bakker's (2013) classical example for peripheral interaction, for which we provide an online simulation tool that researchers and practitioners can readily use to consult actual JavaScript code implementing the inner logic of selected components of our architecture as well as to observe live JSON messages exchanged by the various components of SAPIENS.

Checking compliance of semantic web applications with RDFS‐semantics

Web applications calling themselves “Semantic” count in the hundreds, yet there is no clear definition about what this qualification should stand for. Semantic web applications may range from those just using Resource Description Framework (RDF) as a data interchange format, to those taking all RDF‐S consequences into account. Moreover, even with the simplest semantics, RDF data may contain blank nodes, which should be treated appropriately as existential variables—but might not. In this paper, we propose a general framework of yes/no experiments whose result tell, in a black‐box fashion, how “Semantic” can be considered a given Web application. Our experiments measure the sensitivity of the application to syntactic variations of data which are equivalent under RDF‐S‐semantics. We show how our experiments can be run on a real application, namely, RapidMiner with LODExtension. We show how RapidMiner passes most of the tests (but for blank nodes), but highlight a weakness of the workflow repository‐retrieval‐exploitation that may make the application fail in some cases.

Encompassing new use cases - level 3.0 of the HUPO-PSI format for molecular interactions

BackgroundSystems biologists study interaction data to understand the behaviour of whole cell systems, and their environment, at a molecular level. In order to effectively achieve this goal, it is critical that researchers have high quality interaction datasets available to them, in a standard data format, and also a suite of tools with which to analyse such data and form experimentally testable hypotheses from them. The PSI-MI XML standard interchange format was initially published in 2004, and expanded in 2007 to enable the download and interchange of molecular interaction data. PSI-XML2.5 was designed to describe experimental data and to date has fulfilled this basic requirement. However, new use cases have arisen that the format cannot properly accommodate. These include data abstracted from more than one publication such as allosteric/cooperative interactions and protein complexes, dynamic interactions and the need to link kinetic and affinity data to specific mutational changes.ResultsThe Molecular Interaction workgroup of the HUPO-PSI has extended the existing, well-used XML interchange format for molecular interaction data to meet new use cases and enable the capture of new data types, following extensive community consultation. PSI-MI XML3.0 expands the capabilities of the format beyond simple experimental data, with a concomitant update of the tool suite which serves this format. The format has been implemented by key data producers such as the International Molecular Exchange (IMEx) Consortium of protein interaction databases and the Complex Portal.ConclusionsPSI-MI XML3.0 has been developed by the data producers, data users, tool developers and database providers who constitute the PSI-MI workgroup. This group now actively supports PSI-MI XML2.5 as the main interchange format for experimental data, PSI-MI XML3.0 which additionally handles more complex data types, and the simpler, tab-delimited MITAB2.5, 2.6 and 2.7 for rapid parsing and download.