JavaScript Object Notation Research Articles

Efficient Pipeline for Automating Species ID in new Camera Trap Projects

Camera traps are heat- or motion-activated cameras placed in the wild to monitor and investigate animal populations and behavior. They are used to locate threatened species, identify important habitats, monitor sites of interest, and analyze wildlife activity patterns. At present, the time required to manually review images severely limits productivity. Additionally, ~70% of camera trap images are empty, due to a high rate of false triggers. Previous work has shown good results on automated species classification in camera trap data (Norouzzadeh et al. 2018), but further analysis has shown that these results do not generalize to new cameras or new geographic regions (Beery et al. 2018). Additionally, these models will fail to recognize any species they were not trained on. In theory, it is possible to re-train an existing model in order to add missing species, but in practice, this is quite difficult and requires just as much machine learning expertise as training models from scratch. Consequently, very few organizations have successfully deployed machine learning tools for accelerating camera trap image annotation. We propose a different approach to applying machine learning to camera trap projects, combining a generalizable detector with project-specific classifiers. We have trained an animal detector that is able to find and localize (but not identify) animals, even species not seen during training, in diverse ecosystems worldwide. See Fig. 1 for examples of the detector run over camera trap data covering a diverse set of regions and species, unseen at training time. By first finding and localizing animals, we are able to: drastically reduce the time spent filtering empty images, and dramatically simplify the process of training species classifiers, because we can crop images to individual animals (and thus classifiers need only worry about animal pixels, not background pixels). drastically reduce the time spent filtering empty images, and dramatically simplify the process of training species classifiers, because we can crop images to individual animals (and thus classifiers need only worry about animal pixels, not background pixels). With this detector model as a powerful new tool, we have established a modular pipeline for on-boarding new organizations and building project-specific image processing systems. We break our pipeline into four stages: 1. Data ingestion First we transfer images to the cloud, either by uploading to a drop point or by mailing an external hard drive. Data comes in a variety of formats; we convert each data set to the COCO-Camera Traps format, i.e. we create a Javascript Object Notation (JSON) file that encodes the annotations and the image locations within the organization’s file structure. 2. Animal detection We next run our (generic) animal detector on all the images to locate animals. We have developed an infrastructure for efficiently running this detector on millions of images, dividing the load over multiple nodes. We find that a single detector works for a broad range of regions and species. If the detection results (as validated by the organization) are not sufficiently accurate, it is possible to collect annotations for a small set of their images and fine-tune the detector. Typically these annotations would be fed back into a new version of the general detector, improving results for subsequent projects. 3. Species classification Using species labels provided by the organization, we train a (project-specific) classifier on the cropped-out animals. 4. Applying the system to new data We use the general detector and the project-specific classifier to power tools facilitating accelerated verification and image review, e.g. visualizing the detections, selecting images for review based on model confidence, etc. The aim of this presentation is to present a new approach to structuring camera trap projects, and to formalize discussion around the steps that are required to successfully apply machine learning to camera trap images. The work we present is available at http://github.com/microsoft/cameratraps, and we welcome new collaborating organizations.

Zenodo, an Archive and Publishing Repository: A tale of two herbarium specimen pilot projects

Zenodo (https://zenodo.org) is an open-access repository operated by CERN (European Organization for Nuclear Research), which provides researchers with an easy and stable platform to archive and publish their data and other output, such as software tools, manuals and project reports. In the context of the ICEDIG (Innovation and Consolidation for Large scale Digitisation of Natural Heritage) project, Zenodo was investigated for its usability as a platform where digitized images of collection specimens could be archived and published. In a production digitization pipeline, we foresee the automated archiving of daily image production. If Zenodo could be used for this purpose, such a process would also immediately mean that data and images are published FAIR-ly (Findable, Accessible, Interoperable and Reusable) within hours of their creation. To evaluate performance of the system, we first used a test dataset of 1800 herbarium specimen images, which was uploaded using Zenodo's API (Application Programming Interface) (Dillen et al. 2019). This dataset includes lossless TIFF images, label-segmented overlays and JSON-LD (JavaScript Object Notation for Linked Data) metadata using DwC (Darwin Core) terminology, constituting over 208 gigabytes of data. In addition, for all individual digital specimens the data about the specimen (in DwC) as well as metadata about its deposition on Zenodo (in Zenodo's internal data model) were available in multiple machine-readable formats. All data in DwC were provided as linked data with their DwC identifiers (e.g. http://rs.tdwg.org/dwc/terms/basisOfRecord). All individual specimens received minted DOIs (Digital Object Identifiers). A second upload of 280,000 herbarium JPEG images from a single institution (ca. 1 terabyte of data) with limited metadata (but using the same approach) was launched as well. In this presentation, the workflow for proper usage of the API will be described as well as some performance metrics, flexibilities and functionalities of the platform. Some issues and potential developments to tackle them will be discussed. Currently, the rate of ingestion into Zenodo seems only fast enough for small scale digitization pipelines. However, a modest improvement in transfer rate would make this a realistic proposition for large volume usage.

A Bridge between Legacy Wireless Communication Systems and Internet of Things

The software-defined radio (SDR) is a flexible platform that can adapt to various wireless telecommunication frequencies. It is able to provide a reconfigurable communication infrastructure for wireless systems. Hence, SDR is proposed here as a bridge between legacy wireless communication systems and the Internet of Things (IoT) via standard telecommunication protocols. The standard protocols are hypertext transfer protocol (HTTP), simple mail transfer protocol (SMTP), and message queuing telemetry transport (MQTT). Data collected from legacy wireless systems have been formatted via JavaScript object notation (JSON) for interoperability and categorized according to the application and the communication pattern. The extracted data are then transferred over MQTT for machine-to-machine (M2M) communication, over SMTP for machine-to-human (M2H) notification, and over HTTP for human-to-machine (H2M) communication. However, received audio signals from FM-based broadcasting stations have been transferred to the Internet servers over extensible messaging and presence protocol (XMPP), in live audio streaming. The objective is to introduce an SDR-IoT bridge that is inexpensive, scalable, and interoperable. The analyses show that the environment has good-performance, and can be used for many applications of smart city sectors, for Internet Radio, and for Internet-based monitoring of airplanes and vessel navigation.

A Bridge between Legacy Wireless Communication Systems and Internet of Things

The software-defined radio (SDR) is a flexible platform that can adapt to various wireless telecommunication frequencies. It is able to provide a reconfigurable communication infrastructure for wireless systems. Hence, SDR is proposed here as a bridge between legacy wireless communication systems and the Internet of Things (IoT) via standard telecommunication protocols. The standard protocols are hypertext transfer protocol (HTTP), simple mail transfer protocol (SMTP), and message queuing telemetry transport (MQTT). Data collected from legacy wireless systems have been formatted via JavaScript object notation (JSON) for interoperability and categorized according to the application and the communication pattern. The extracted data are then transferred over MQTT for machine-to-machine (M2M) communication, over SMTP for machine-to-human (M2H) notification, and over HTTP for human-to-machine (H2M) communication. However, received audio signals from FM-based broadcasting stations have been transferred to the Internet servers over extensible messaging and presence protocol (XMPP), in live audio streaming. The objective is to introduce an SDR-IoT bridge that is inexpensive, scalable, and interoperable. The analyses show that the environment has good-performance, and can be used for many applications of smart city sectors, for Internet Radio, and for Internet-based monitoring of airplanes and vessel navigation.

AntiSMASH 5.0: updates to the secondary metabolite genome mining pipeline.

Secondary metabolites produced by bacteria and fungi are an important source of antimicrobials and other bioactive compounds. In recent years, genome mining has seen broad applications in identifying and characterizing new compounds as well as in metabolic engineering. Since 2011, the ‘antibiotics and secondary metabolite analysis shell—antiSMASH’ (https://antismash.secondarymetabolites.org) has assisted researchers in this, both as a web server and a standalone tool. It has established itself as the most widely used tool for identifying and analysing biosynthetic gene clusters (BGCs) in bacterial and fungal genome sequences. Here, we present an entirely redesigned and extended version 5 of antiSMASH. antiSMASH 5 adds detection rules for clusters encoding the biosynthesis of acyl-amino acids, β-lactones, fungal RiPPs, RaS-RiPPs, polybrominated diphenyl ethers, C-nucleosides, PPY-like ketones and lipolanthines. For type II polyketide synthase-encoding gene clusters, antiSMASH 5 now offers more detailed predictions. The HTML output visualization has been redesigned to improve the navigation and visual representation of annotations. We have again improved the runtime of analysis steps, making it possible to deliver comprehensive annotations for bacterial genomes within a few minutes. A new output file in the standard JavaScript object notation (JSON) format is aimed at downstream tools that process antiSMASH results programmatically.

MQTT for IoT-based Applications in Smart Cities

This research presents a study on the use of the MQTT communications protocol for the Internet of Things in Smart City applications. A network model is proposed and a typical practical scenario is developed based on MQTT protocol, that cope with the requirements of some Smart City applications, mainly those which use event-based messages. Many market-available embedded electronic systems were employed for this scenario including the inexpensive Wi-Fi platform ESP8266, Arduino, Raspberry, in addition to some sensors and actuators. The network model is chosen based on the TCP/IP model, and the application layer is totally depending on the MQTT protocol that employs JavaScript Object Notation (JSON) to solve the problem of interoperability. To evaluate the protocol for small-to-medium, IoT-based, business applications of Smart Cities, some available free Open Source Software (OSS) of MQTT servers and clients were compared and tested against latency over the cloud. The protocol shows good results for cloud-based, small-to-medium business applications that depend on event-based message-oriented communication paradigms. Since the protocol defines three levels of quality of service (QoS), the simulations and the tests were conducted for QoS type zero (QoS0) to get the best results.

JSON-based Apparel Fabric Query Micro-program Application

With the continuous development of information technology, the application market has also gradually innovated. Light applications are favored by people because of its small footprint, easy to use, and fast running. JSON (JavaScript Object Notation) is considered by developers to be an ideal data exchange language because of its simplicity and ease of use, and is widely used in Web development.Compared with XML, JSON has more advantages in Web data exchange processing. JSON is used to cache data, which is convenient for storing complex data, and is more conducive to client parsing and use.

PMC text mining subset in BioC: about three million full-text articles and growing.

Interest in text mining full-text biomedical research articles is growing. To facilitate automated processing of nearly 3 million full-text articles (in PubMed Central® Open Access and Author Manuscript subsets) and to improve interoperability, we convert these articles to BioC, a community-driven simple data structure in either XML or JavaScript Object Notation format for conveniently sharing text and annotations. The resultant articles can be downloaded via both File Transfer Protocol for bulk access and a Web API for updates or a more focused collection. Since the availability of the Web API in 2017, our BioC collection has been widely used by the research community. https://www.ncbi.nlm.nih.gov/research/bionlp/APIs/BioC-PMC/.

An effective biomedical data migration tool from resource description framework to JSON.

Resource Description Framework (RDF) is widely used for representing biomedical data in practical applications. With the increases of RDF-based applications, there is an emerging requirement of novel architectures to provide effective supports for the future RDF data explosion. Inspired by the success of the new designs in National Center for Biotechnology Information dbSNP (The Single Nucleotide Polymorphism Database) for managing the increasing data volumes using JSON (JavaScript Object Notation), in this paper we present an effective mapping tool that allows data migrations from RDF to JSON for supporting future massive data explosions and releases. We firstly introduce a set of mapping rules, which transform an RDF format into the JSON format, and then present the corresponding transformation algorithm. On this basis, we develop an effective and user-friendly tool called RDF2JSON, which enables automating the process of RDF data extractions and the corresponding JSON data generations.

Translating JSON Schema logics into OWL axioms for unified data validation on a digital manufacturing platform

Abstract JSON (JavaScript Object Notation) is a prevalent data format used in cloud-based platforms that support composable digital manufacturing workflows. The current work presents a method to translate the logics found in JSON Schema into OWL axioms, in order to facilitate ontology-based unified data validation with JSON data. The specific contributions of this paper include the demonstration of using a formal ontology for the logic translation and data validation, a technique for disambiguating implicit relations found in JSON Schema as explicit OWL properties, and mapping JSON Schema validation keywords to equivalent OWL expressions.

Implementation and Comparison of Text-Based Image Retrieval Schemes

Search engines, i.e., Google, Yahoo pro-vide various libraries and API’s to assist programmers and researchers in easier and efficient access to their collected data. When a user generates a search query, the dedicated Application Programming Interface (API) returns the JavaScript Object Notation (JSON) file which contains the desired data. Scraping techniques help image descriptors to separate the image’s URL and web host’s URL in different documents for easier implementation of different algorithms. The aim of this paper is to propose a novel approach to effectively filter out the desired image(s) from the retrieved data. More specifically, this work primarily focuses on applying simple yet efficient techniques to achieve accurate image retrieval. We compare two algorithms, i.e., Cosine similarity and Sequence Matcher, to obtain the accuracy with a minimum of irrelevance. Obtained results prove Cosine similarity more accurate than its counterpart in finding the maximum relevant image(s).

Eleven quick tips to build a usable REST API for life sciences.

In recent years, technological advances have greatly expanded the range of data types generated by life sciences researchers. These span domains such as molecular structures, nucleotide and protein sequences, metabolomics, and chemogenomics, resulting in hundreds of public resources holding diverse data sets for reuse in multiple formats [1]. Most resources focus on a specific data type, yet their value for researchers is enhanced once cross-referenced and combined with expert annotation and knowledge. Cross-referencing has increasingly been achieved by implementing website application programming interfaces (web APIs), providing programming-language−agnostic methods to access online resources. Web APIs enable dynamic data exchange between resources, augment websites with additional data, and can provide access to large data sets. Web APIs also enhance adherence to FAIR data principles by making data Findable, Accessible, Interoperable, and Reusable [2], thus increasing the value of those resources. Representational state transfer (REST) [3] is a popular method for providing interoperability between a client and server [4] using the hypertext transfer protocol (HTTP), the same building block as the world wide web, [5] and a common exchange format, e.g., JavaScript Object Notation (JSON) [6]. REST APIs are considered easier to develop than previous web-service standards, e.g., Simple Object Access Protocol (SOAP). However, REST specifies a set of requirements that any implementation of a REST API must address. Although well-known resources such as the World Wide Web Consortium (W3C) (https://www.w3.org) and the Internet Engineering Task Force (IETF) (https://www.ietf.org/) provide guidance on how to implement such a service, they can be difficult to understand and may have limited documentation. We present here 11 quick tips for creating and maintaining REST web APIs that were developed while implementing various web APIs (https://www.ebi.ac.uk/services) for European Molecular Biology Laboratory, European Bioinformatics Institute (EMBL-EBI)’s data resources.

ClinGen Allele Registry links information about genetic variants.

Effective exchange of information about genetic variants is currently hampered by the lack of readily available globally unique variant identifiers that would enable aggregation of information from different sources. The ClinGen Allele Registry addresses this problem by providing (1) globally unique “canonical” variant identifiers (CAids) on demand, either individually or in large batches; (2) access to variant‐identifying information in a searchable Registry; (3) links to allele‐related records in many commonly used databases; and (4) services for adding links to information about registered variants in external sources. A core element of the Registry is a canonicalization service, implemented using in‐memory sequence alignment‐based index, which groups variant identifiers denoting the same nucleotide variant and assigns unique and dereferenceable CAids. More than 650 million distinct variants are currently registered, including those from gnomAD, ExAC, dbSNP, and ClinVar, including a small number of variants registered by Registry users. The Registry is accessible both via a web interface and programmatically via well‐documented Hypertext Transfer Protocol (HTTP) Representational State Transfer Application Programming Interface (REST‐APIs). For programmatic interoperability, the Registry content is accessible in the JavaScript Object Notation for Linked Data (JSON‐LD) format. We present several use cases and demonstrate how the linked information may provide raw material for reasoning about variant's pathogenicity.

Using a linked table-based structure to encode self-describing multiparameter spatiotemporal data

Multiparameter data with both spatial and temporal components are critical to advancing the state of environmental science. These data and data collected in the future are most useful when compared with each other and analyzed together, which is often inhibited by inconsistent data formats and a lack of structured documentation provided by researchers and (or) data repositories. In this paper we describe a linked table-based structure that encodes multiparameter spatiotemporal data and their documentation that is both flexible (able to store a wide variety of data sets) and usable (can easily be viewed, edited, and converted to plottable formats). The format is a collection of five tables (Data, Locations, Params, Data Sets, and Columns), on which restrictions are placed to ensure data are represented consistently from multiple sources. These tables can be stored in a variety of ways including spreadsheet files, comma-separated value (CSV) files, JavaScript object notation (JSON) files, databases, or objects in a software environment such as R or Python. A toolkit for users of R statistical software was also developed to facilitate converting data to and from the data format. We have used this format to combine data from multiple sources with minimal metadata loss and to effectively archive and communicate the results of spatiotemporal studies. We believe that this format and associated discussion of data and data storage will facilitate increased synergies between past, present, and future data sets in the environmental science community.

A oneM2M-Based Query Engine for Internet of Things (IoT) Data Streams.

The new standard oneM2M (one machine-to-machine) aims to standardize the architecture and protocols of Internet of Things (IoT) middleware for better interoperability. Although the standard seems promising, it lacks several features for efficiently searching and retrieving IoT data which satisfy users’ intentions. In this paper, we design and develop a oneM2M-based query engine, called OMQ, that provides a real-time processing over IoT data streams. For this purpose, we define a query language which enables users to retrieve IoT data from data sources using JavaScript Object Notation (JSON). We also propose efficient query processing algorithms which utilizes the oneM2M architecture consisting of two nodes: (1) the IoT node and (2) the infrastructure node. IoT nodes of OMQ are mainly sensor devices execute user queries the aggregate, transform and filter operators, whereas the infrastructure node handles the join operator of user queries. Since the query processing algorithms are implemented as the hybrid infrastructure-edge processing, user queries can be executed efficiently in each IoT node rather than only in the infrastructure node. Thus, our OMQ system reduces the query processing time and the network bandwidth. We conducted a comprehensive evaluation of OMQ using a real and a synthetic data set. Experimental results demonstrate the feasibility and efficiency of OMQ system for executing queries and transferring data from each IoT node.

Survey on JSON Data Modelling

JavaScript Object Notation (JSON) is a popular data format based on the data types of JavaScript programming language. As a lightweight semi-structured data format language, it has become the main data exchange format over the World Wide Web in the past several years, and gained popularity in database community research. JSON data model describes the basic data structures and semantics of the underlying JSON data, so it is the fundamental and key aspects for JSON data format. As JSON data model is also the key and foundation for other data management technologies, such as data indexing, data querying, data searching, data mapping, data integrating, and data mining, how to design an efficient and powerful model for JSON data is necessary and important to other related research topics, such as data integration, data search and query, data quality and evaluation, etc. In this paper, we investigate some approaches to modelling JOSN data and corresponding JSON schemas, which are mainly based on tree models. The advantages and disadvantages of each method are analyzed and compared.

Software Systems Approach to Multi-Scale GIS-BIM Utility Infrastructure Network Integration and Resource Flow Simulation

There is an increasing impetus for the use of digital city models and sensor network data to understand the current demand for utility resources and inform future infrastructure service planning across a range of spatial scales. Achieving this requires the ability to represent a city as a complex system of connected and interdependent components in which the topology of the electricity, water, gas, and heat demand-supply networks are modelled in an integrated manner. However, integrated modelling of these networks is hampered by the disparity between the predominant data formats and modelling processes used in the Geospatial Information Science (GIS) and Building Information Modelling (BIM) domains. This paper presents a software systems approach to scale-free, multi-format, integrated modelling of evolving cross-domain utility infrastructure network topologies, and the analysis of the spatiotemporal dynamics of their resource flows. The system uses a graph database to integrate the topology of utility network components represented in the CityGML UtilityNetwork Application Domain Extension (ADE), Industry Foundation Classes (IFC) and JavaScript Object Notation (JSON) real-time streaming messages. A message broker is used to disseminate the changing state of the integrated topology and the dynamic resource flows derived from the streaming data. The capability of the developed system is demonstrated via a case study in which internal building and local electricity distribution feeder networks are integrated, and a real-time building management sensor data stream is used to simulate and visualise the spatiotemporal dynamics of electricity flows using a dynamic web-based visualisation.

Intelligent Application Implementation Model for Automated Agent Negotiation

In the last few years, multi-agents communication has seen an increased concern. The first main driver was an intelligent agent is claimed to be the next generation model for complex and distributed system engineering. Second the agents can automate process and communication on behalf of the agent’s users for proper actions and decisions. There are many concerns about how to implement communications among different agents in term of applied technology. In this paper we propose a REST- API and JSON technology as implementation tools that can be applied to implement intelligent web-based applications. REST is stands for Representational State Transfer which is allowing communication between web-based agents. JSON (JavaScript Object Notation) is a lightweight data format makes it easy for agents to parse and interchange data. This study has merged REST-AP and JSON technology into an automated negotiation model, and then a prototype is developed to successfully apply the proposed model. This model can facilitate application in business application negotiations and even internet of things (IOT) where HTTP protocol is used.

Automated data mining of a plan-check database and example application.

PurposeThe aim of this work was to present the development and example application of an automated data mining software platform that preforms bulk analysis of results and patient data passing through the 3D plan and delivery QA system, Mobius3D.MethodsPython, matlab, and Java were used to create an interface that reads JavaScript Object Notation (JSON) created for every approved Mobius3D pre‐treatment plan‐check. The aforementioned JSON files contain all the information for every pre‐treatment QA check performed by Mobius3D, including all 3D dose, CT, structure set information, as well as all plan information and patient demographics. Two Graphical User Interfaces (GUIs) were created, the first is called Mobius3D‐Database (M3D‐DB) and presents the check results in both filterable tabular and graphical form. These data are presented for all patients and includes mean dose differences, 90% coverage, 3D gamma pass rate percentages, treatment sites, machine, beam energy, Multi‐Leaf Collimator (MLC) mode, treatment planning system (TPS), plan names, approvers, dates and times. Group statistics and statistical process control levels are then calculated based on filter settings. The second GUI, called Mobius3D organ at risk (M3DOAR), analyzes dose‐volume histogram data for all patients and all Organs‐at‐Risk (OAR). The design of the software is such that all treatment parameters and treatment site information are able to be filtered and sorted with the results, plots, and statistics updated.ResultsThe M3D‐DB software can summarize and filter large numbers of plan‐checks from Mobius3D. The M3DOAR software is also able to analyze large amounts of dose‐volume data for patient groups which may prove useful in clinical trials, where OAR doses for large numbers of patients can be compared and correlated. Target DVHs can also be analyzed en mass.ConclusionsThis work demonstrates a method to extract the large amount of treatment data for every patient that is stored by Mobius3D but not easily accessible. With scripting, it is possible to mine this data for research and clinical trials as well as patient and TPS QA.

The Biological Object Notation (BON): a structured file format for biological data

The large size and high complexity of biological data can represent a major methodological challenge for the analysis and exchange of data sets between computers and applications. There has also been a substantial increase in the amount of metadata associated with biological data sets, which is being increasingly incorporated into existing data formats. Despite the existence of structured formats based on XML, biological data sets are mainly formatted using unstructured file formats, and the incorporation of metadata results in increasingly complex parsing routines such that they become more error prone. To overcome these problems, we present the “biological object notation” (BON) format, a new way to exchange and parse nearly all biological data sets more efficiently and with less error than other currently available formats. Based on JavaScript Object Notation (JSON), BON simplifies parsing by clearly separating the biological data from its metadata and reduces complexity compared to XML based formats. The ability to selectively compress data up to 87% compared to other file formats and the reduced complexity results in improved transfer times and less error prone applications.