BioThings SDK: a toolkit for building high-performance data APIs in biomedical research

Web APIs (Application Programming Interfaces) facilitate the exchange of resources (data) between two functionally independent entities across a common programmatic interface. In more general terms, Web APIs can connect almost anything to the world wide web. Unlike traditional software, APIs are not compiled, installed, or run. Instead, data are read (or consumed in API speak) through a web-based transaction, where a client makes a request and a server responds. Web APIs can be loosely grouped into two categories within the scope of biodiversity informatics, based on purpose. First, Product APIs deliver data products to end-users. Examples include the Global Biodiversity Information Facility (GBIF) and iNaturalist APIs. Designed and built to solve specific problems, web-based Service APIs are the second type and the focus of this presentation (referred to as Service APIs). Their primary function is to provide on-demand support to existing programmatic processes. Examples of this type include Elasticsearch Suggester API and geolocation, a service that delivers geographic locations from spatial input (latitude and longitude coordinates) (Pejic et al. 2010). Many challenges lie ahead for biodiversity informatics and the sharing of global biodiversity data (e.g., Blair et al. 2020). Service-driven, standardized web-based Service APIs that adhere to best practices within the scope of biodiversity informatics can provide the transformational change needed to address many of these issues. This presentation will highlight several critical areas of interest in the biodiversity data community, describing how Service APIs can address each individually. The main topics include: standardized vocabularies, interoperability of heterogeneous data sources and data quality assessment and remediation. standardized vocabularies, interoperability of heterogeneous data sources and data quality assessment and remediation. Fundamentally, the value of any innovative technical solution can be measured by the extent of community adoption. In the context of Service APIs, adoption takes two primary forms: financial and temporal investment in the construction of clients that utilize Service APIs and willingness of the community to integrate Service APIs into their own systems and workflows. financial and temporal investment in the construction of clients that utilize Service APIs and willingness of the community to integrate Service APIs into their own systems and workflows. To achieve this, Service APIs must be simple, easy to use, pragmatic, and designed with all major stakeholder groups in mind, including users, providers, aggregators, and architects (Anderson et al. 2020Anderson et al. 2020; this study). Unfortunately, many innovative and promising technical solutions have fallen short not because of an inability to solve problems (Verner et al. 2008), rather, they were difficult to use, built in isolation, and/or designed without effective communication with stakeholders. Fortunately, projects such as Darwin Core (Wieczorek et al. 2012), the Integrated Publishing Toolkit (Robertson et al. 2014), and Megadetector (Microsoft 2021) provide the blueprint for successful community adoption of a technological solution within the biodiversity community. The final section of this presentation will examine the often overlooked non-technical aspects of this technical endeavor. Within this context, specifically how following these models can broaden community engagement and bridge the knowledge gap between the major stakeholders, resulting in the successful implementation of Service APIs.

Web Application Programming Interface (API) allows third-party and subscribed users to access data and functions of a software application through the network or the Internet. Web APIs expose data and functions to the public users, authorized users or enterprise users. Web API providers publish API documentations to help users to understand how to interact with web-based API services, and how to use the APIs in their integration systems. The exponential raise of the number of public web service APIs may cause a challenge for software engineers to choose an efficient API. The challenge may become more complicated when web APIs updated regularly by API providers. In this paper, we introduce a novel transformation-based approach which crawls the web to collect web API documentations (unstructured documents). It generates a web API Language model from API documentations, employs different machine learning algorithms to extract information and produces a structured web API specification that compliant to Open API Specification (OAS) format. The proposed approach improves information extraction patterns and learns the variety of structured and terminologies. In our experiment, we collect a sheer number of web API documentations. Our evaluation shows that the proposed approach find RESTful API documentations with 75% accuracy, constructs API endpoints with 84%, constructs endpoint attributes with 95%, and assigns endpoints to attributes with an accuracy 98%. The proposed approach were able to produces more than 2,311 OAS web API Specifications.

Web Application Programming Interfaces (APIs) specify how to access services and data over the network, typically using Web services. Web APIs are rapidly proliferating as a key element to foster reusability, integration, and innovation, enabling new consumption models such as mobile or smart TV apps. Companies such as Facebook, Twitter, Google, eBay or Netflix receive billions of API calls every day from thousands of different third-party applications and devices, which constitutes more than half of their total traffic. As Web APIs are progressively becoming the cornerstone of software integration, their validation is getting more critical. In this context, the fast detection of bugs is of utmost importance to increase the quality of internal products and third-party applications. However, testing Web APIs is challenging mainly due to the difficulty to assess whether the output of an API call is correct, i.e., the oracle problem. For instance, consider the Web API of the popular music streaming service Spotify. Suppose a search for albums with the query redhouse returning 21 total matches: Is this output correct? Do all the albums in the result set contain the keyword? Are there any albums containing the keyword not included in the result set? Answering these questions is difficult, even with small result sets, and often infeasible when the results are counted by thousands or millions. Metamorphic testing alleviates the oracle problem by providing an alternative when the expected output of a test execution is complex or unknown. Rather than checking the output of an individual program execution, metamorphic testing checks whether multiple executions of the program under test fulfil certain necessary properties called metamorphic relations. For instance, consider the following metamorphic relation in Spotify: two searches for albums with the same query should return the same number of total results regardless of the size of pagination. Suppose that a new Spotify search is performed using the exact same query as before and increasing the maximum number of results per page from 20 (default value) to 50: This search returns 27 total albums (6 more matches than in the previous search), which reveals a bug. This is an example of a real and reproducible fault detected using the approach presented in this paper and reported to Spotify. According to Spotify developers, it was a regression fault caused by a fix with undesired side effects. In this paper [1], we present a metamorphic testing approach for the automated detection of faults in RESTful Web APIs (henceforth also referred to as simply Web APIs). We introduce the concept of metamorphic relation output patterns. A Metamorphic Relation Output Pattern (MROP) defines an abstract output relation typically identified in Web APIs, regardless of their application domain. Each MROP is defined in terms of set operations among test outputs such as equality, union, subset, or intersection. MROPs provide a helpful guide for the identification of metamorphic relations, broadening the scope of our work beyond a particular Web API. Based on the notion of MROP, a methodology is proposed for the application of the approach to any Web API following the REST architectural pattern. The approach was evaluated in several steps. First, we used the proposed methodology to identify 33 metamorphic relations in four Web APIs developed by undergraduate students. All the relations are instances of the proposed MROPs. Then, we assessed the effectiveness of the identified relations at revealing 317 automatically seeded faults (i.e., mutants) in the APIs under test. As a result, 302 seeded faults were detected, achieving a mutation score of 95.3%. Second, we evaluated the approach using real Web APIs and faults. In particular, we identified 20 metamorphic relations in the Web API of Spotify and 40 metamorphic relations in the Web API of YouTube. Each metamorphic relation was implemented and automatically executed using both random and manual test data. In total, 469K metamorphic tests were generated. As a result, 21 metamorphic relations were violated, and 11 issues revealed and reported (3 issues in Spotify and 8 issues in YouTube). To date, 10 of the reported issues have been either confirmed by the API developers or reproduced by other users supporting the effectiveness of our approach.

During the last decade, Web APIs (Application Programming Interface) have gained significant traction to the extent that they have become a de-facto standard to enable HTTP-based, machine-processable data access. Despite this success, however, they still often fail in making data interoperable, insofar as they commonly rely on proprietary data models and vocabularies that lack formal semantic descriptions essential to ensure reliable data integration. In the biodiversity domain, multiple data aggregators, such as the Global Biodiversity Information Facility (GBIF) and the Encyclopedia of Life (EoL), maintain specialized Web APIs giving access to billions of records about taxonomies, occurrences, or life traits (Triebel et al. 2012). They publish data sets spanning complementary and often overlapping regions, epochs or domains, but may also report or rely on potentially conflicting perspectives, e.g. with respect to the circumscription of taxonomic concepts. It is therefore of utmost importance for biologists and collection curators to be able to confront the knowledge they have about taxa with related data coming from third-party data sources. To tackle this issue, the French National Museum of Natural History (MNHN) has developed an application to edit TAXREF, the French taxonomic register for fauna, flora and fungus (Gargominy et al. 2018). TAXREF registers all species recorded in metropolitan France and overseas territories, accounting for 260,000+ biological taxa (200,000+ species) along with 570,000+ scientific names. The TAXREF-Web application compares data available in TAXREF with corresponding data from third-party data sources, points out disagreements and allows biologists to add, remove or amend TAXREF accordingly. This requires that TAXREF-Web developers write a specific piece of code for each considered Web API to align TAXREF representation with the Web API counterpart. This task is time-consuming and makes maintenance of the web application cumbersome. In this presentation, we report on a new implementation of TAXREF-Web that harnesses the Linked Data standards: Resource Description Framework (RDF), the Semantic Web format to represent knowledge graphs, and SPARQL, the W3C standard to query RDF graphs. In addition, we leverage the SPARQL Micro-Service architecture (Michel et al. 2018), a lightweight approach to query Web APIs using SPARQL. A SPARQL micro-service is a SPARQL endpoint that wraps a Web API service; it typically produces a small, resource-centric RDF graph by invoking the Web API and transforming the response into RDF triples. We developed SPARQL micro-services to wrap the Web APIs of GBIF, World Register of Marine Species (WoRMS), FishBase, Index Fungorum, Pan-European Species directories Infrastructure (PESI), ZooBank, International Plant Names Index (IPNI), EoL, Tropicos and Sandre. These micro-services consistently translate Web APIs responses into RDF graphs utilizing mainly two well-adopted vocabularies: Schema.org (Guha et al. 2015) and Darwin Core (Baskauf et al. 2015). This approach brings about two major advantages. First, the large adoption of Schema.org and Darwin Core ensures that the services can be immediately understood and reused by a large audience within the biodiversity community. Second, wrapping all these Web APIs in SPARQL micro-services “suddenly” makes them technically and semantically interoperable, since they all represent resources (taxa, habitats, traits, etc.) in a common manner. Consequently, the integration task is simplified: confronting data from multiple sources essentially consists of writing the appropriate SPARQL queries, thus making easier web application development and maintenance. We present several concrete cases in which we use this approach to detect disagreements between TAXREF and the aforementioned data sources, with respect to taxonomic information (author, synonymy, vernacular names, classification, taxonomic rank), habitats, bibliographic references, species interactions and life traits.

Web Services (WSs) are gaining worldwide popularity due to reliable and fast intercommunication services for the development of web and mobile applications. WSs are provided to client application developers through web Application Programming Interfaces (APIs), such as YouTube API, Twitter API, Facebook API, etc. Due to the popularity of WSs, the developers frequently discuss various WSs-based application’ issues on online forums, such as Stack Overflow (SO). This study aims to highlight the problems faced by client developers in the development process of WSs-based applications using the dataset of SO. The comprehension of developers’ conversations on SO can give insight into the frequency, difficulty, and popularity of different WSs-related problems of developers. We downloaded 12,746 posts from SO relevant to WSs-related issues for this article. We used the topic modeling technique (LDA) to extract various topics from the SO dataset. The topics are labeled and organized into categories and sub-categories according to relationships among them. The difficulty and popularity of each topic have been analyzed. Our investigation yield several findings. First, developers focus on six topics related to WSs on SO: Client APIs development, Data Processing, Web services Authorization, Framework Support, Web APIs, and Mobile Applications. Secondly, the advantages and disadvantages of web applications topic (Fused_Popularity=0.39), from the Clients APIs development category have the highest prevalence, followed by Database (DB) and Data Processing in Applications topic (Fused_Popularity=0.38) from the Data Processing category. Third, most WSs-related topics in all categories are evolving promptly on SO, i.e., new questions are added daily about WSs development, deployment, and authorization. Fourth, the questions of type “how” are primarily asked in Framework support, Client APIs development, and Web APIs categories. Although, many questions in other categories are of the kind “What”. It is also observed that WSs developers not only used SO to ask How and What types of questions but they also used SO to ask information-seeking questions (i.e., in Data processing and Client APIs development categories). Fifth, the topics relevant to Web APIs (Fused_Popularity=10.8) and Client API Development ((Fused_Popularity=9.35) categories of WSs are very popular on SO. Sixth, the questions relevant to the Web APIs (Fused_Difficulty =3) & Client APIs development (Fused_Difficulty=2.25) categories are more difficult than the other four categories. The results of our research may be helpful for the following WSs stakeholders: WSs Client application developers, WSs Educators, and WSs researchers. The WSs Educators and investigators can get more knowledge of new methods and discover novel techniques to make challenging WSs topics easy to understand. WSs framework developers can utilize our extracted WSs topics and categories to know the preferences of WSs developers that may support them in upgrading existing frameworks or developing new ones.

Numerous companies create innovative software systems using Web APIs (Application Programming Interfaces). API search engines and API directory services, such as ProgrammableWeb, Rapid API Hub, APIs.guru, and API Harmony, have been developed to facilitate the utilization of various APIs. Unfortunately, most API systems provide only superficial support, with no assistance in obtaining relevant APIs or examples of code usage. To better realize the “FAIR” (Findability, Accessibility, Interoperability, and Reusability) features for the usage of Web APIs, in this study, we developed an API inspection system (referred to as API Prober) to provide a new API directory service with multiple supplemental functionalities. To facilitate the findability and accessibility of APIs, API Prober transforms OAS (OpenAPI Specifications) into a graph structure and automatically annotates the semantic concepts using LDA (Latent Dirichlet Allocation) and WordNet. To enhance interoperability, API Prober also classifies APIs by clustering OAS documents and recommends alternative services to be substituted or merged with the target service. Finally, to support reusability, API Prober makes it possible to retrieve examples of API utilization code in Java by parsing source code in GitHub. The experimental results demonstrate the effectiveness of the API Prober in recommending relevant services and providing usage examples based on real-world client code. This research contributes to providing viable methods to appropriately analyze and cluster Web APIs, and recommend APIs and client code examples.

The ever-increasing web application programming interfaces (APIs) in various service-sharing communities (e.g., ProgrammableWeb.com and Mashape.com) have enabled software developers to quickly create their interested mashups conveniently and economically. However, the big volume of candidate web APIs and their differences often make it hard for software developers to discover a set of appropriate web APIs for mashup creation by considering API functions and API quality performances (e.g., popularity, compatibility, and diversity) simultaneously. These decrease the mashup development success rate and the mashup developers’ satisfaction significantly. In view of these challenges, a novel web APIs’ recommendation method named the popularity-aware and diverse method of web API compositions’ recommendation (PD-WACR) is proposed in this article. In concrete, we model web APIs’ functions, popularity, and compatibility with an API correlation graph. Afterward, correlation graph-based web APIs’ recommendation is performed with popularity and compatibility guarantee. Moreover, a top- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> strategy is adopted in the recommendation process, so as to diversify the final recommended web APIs’ results. Finally, massive experiments are carried out on a real-world web API dataset crawled from ProgrammeableWeb.com. Experimental comparisons with related methods show the advantages and innovations of the proposed PD-WACR method.

With the ever-increasing prosperity of web Application Programming Interface (API) sharing platforms, it is becoming an economic and efficient way for software developers to design their interested mashups through web API re-use. Generally, a software developer can browse, evaluate, and select his or her preferred web APIs from the API's sharing platforms to create various mashups with rich functionality. The big volume of candidate APIs places a heavy burden on software developers’ API selection decisions. This, in turn, calls for the support of intelligent API recommender systems. However, existing API recommender systems often face two challenges. First, they focus more on the functional accuracy of APIs while neglecting the APIs’ actual compatibility. This then creates incompatible mashups. Second, they often require software developers to input a set of keywords that can accurately describe the expected functions of the mashup to be developed. This second challenge tests partial developers who have little background knowledge in the fields. To tackle the above-mentioned challenges, in this article we propose a compatibility-aware and text description-driven web API recommendation approach (named WAR text ). WAR text guarantees the compatibility among the recommended APIs by utilizing the APIs’ composition records produced by historical mashup creations. Besides, WAR text entitles a software developer to type a simple text document that describes the expected mashup functions as input. Then through textual description mining, WAR text can precisely capture the developers’ functional requirements and then return a set of APIs with the highest compatibility. Finally, through a real-world mashup dataset ProgrammableWeb, we validate the feasibility of our novel approach.

Application programming interfaces (APIs) offer a plethora of functionalities for developers to reuse without reinventing the wheel. Identifying the appropriate APIs given a project requirement is critical for the success of a project, as many functionalities can be reused to achieve faster development. However, the massive number of APIs would often hinder the developers' ability to quickly find the right APIs. In this light, we propose a new, automated approach called WebAPIRec that takes as input a project profile and outputs a ranked list of {web} APIs that can be used to implement the project. At its heart, WebAPIRec employs a personalized ranking model that ranks web APIs specific (personalized) to a project. Based on the historical data of {web} API usages, WebAPIRec learns a model that minimizes the incorrect ordering of web APIs, i.e., when a used {web} API is ranked lower than an unused (or a not-yet-used) web API. We have evaluated our approach on a dataset comprising 9,883 web APIs and 4,315 web application projects from ProgrammableWeb with promising results. For 84.0% of the projects, WebAPIRec is able to successfully return correct APIs that are used to implement the projects in the top-5 positions. This is substantially better than the recommendations provided by ProgrammableWeb's native search functionality. WebAPIRec also outperforms McMillan et al.'s application search engine and popularity-based recommendation.

In recent years, Web Application Programming Interfaces (APIs) are becoming more and more popular with the development of the Internet industry and software engineering. Many companies provide public Web APIs for their services, and developers can greatly accelerate the development of new applications by relying on such APIs to execute complex tasks without implementing the corresponding functionalities themselves. The proliferation of web APIs, however, also introduces a challenge for developers to search and discover the desired API and its endpoint. This is a practical and crucial problem because according to ProgrammableWeb, there are more than 22,000 public Web APIs each of which may have tens or hundreds of endpoints. Therefore, it is difficult and time-consuming for developers to find the desired API and its endpoint to satisfy their development needs. In this paper, we present an intelligent system for Web API searches based on natural language queries by using a two-step transfer learning. To train the model, we collect a significant amount of sentences from crowdsourcing and utilize an ensemble deep learning model to predict the correct description sentences for an API and its endpoint. A training dataset is built by synthesizing the correct description sentences and then is used to train the two-step transfer learning model for Web API search. Extensive evaluation results show that the proposed methods and system can achieve high accuracy to search a Web API and its endpoint.

Application Programming Interfaces (APIs) are means of communication between applications, hence they can be seen as user interfaces, just with different kind of users, i.e., software or computers. However, the very first consumers of the APIs are humans, namely programmers. Based on the available documentation and the ``ease of use'' perception (sometimes led by corporate decisions and/or restrictions) they decide to use or not a specific API. In this paper, we propose a data-driven approach to measure web API usability, expressed through the predicted error rate. Following the reviewed state of the art in API usability, we identify a set of usability attributes, and for each of them we propose indicators that web API providers should refer to when developing usable web APIs. Our focus in this paper is on those indicators that can be quantified using the API logs, which indeed reflect the actual behaviour of programmers. Next, we define metrics for the aforementioned indicators, and exemplify them in our use case, applying them on the logs from the web API of District Health Information System (DHIS2) used at World Health Organization (WHO). Using these metrics as features, we build a classifier model to predict the error rate of API endpoints. Besides finding usability issues, we also drill down into the usage logs and investigate the potential causes of these errors.

This article focuses on the role web application programming interfaces (APIs) play in the television (TV) industry’s social media efforts. Web APIs are coding interfaces that allow different databases of information to communicate with one another. The widespread implementation of web APIs as a standard for information sharing offers a way for TV companies to more easily cobble together a presence in connected viewing environments. In this light, web API acts as, what Joshua Braun calls, ‘transparent intermediaries’ that actively shape the range of possibilities available in these types of intermedial partnerships. I explain web APIs before showing how they relate to media studies through the TV show Psych’s forays into designing social media experiences. Additionally, I will explain how web API-connected digital environments are used to facilitate what Tiziana Terranova calls soft control as web APIs embody Web 2.0 ideologies that celebrate information sharing between businesses that track audience attention.

SummaryWith the ever‐increasing popularity of web application programming interfaces (APIs) sharing communities, it is becoming a promising way for software developers to design and create their interesting Apps through composing a set of selected web APIs that can collectively fulfill the App functions expected by the App developer. However, the App developer's web APIs selection decision‐makings are often nontrivial due to the massive candidate APIs as well as their diverse functions. Furthermore, it is difficult to guarantee that the selected web APIs are compatible enough. Moreover, traditional web APIs recommendation approaches only return a recommended APIs list, which are often not sufficient to accommodate the App developer's undetermined and fuzzy personalized preferences. Considering the above challenges, a novel keywords‐driven web APIs recommendation approach called keywords‐driven and compatibility‐aware multiple API group recommendation is proposed in this article for green and compatible software, which cannot only satisfy the App developer's functional requirements, but also return a group of web APIs recommended lists. Each returned list includes a set of compatible web APIs. Finally, we design a series of experiments based on a real‐world web APIs dataset, that is, PW dataset crawled from www.programmableWeb.com. Experimental reports compared with other competitive approaches in existing literatures indicate the effectiveness and efficiency of our proposal in this work.

This paper proposes a Web API (Application Programming Interface) design for IoT data collected from a smart community’s platform. The IoT data of a smart community should be utilized for residents, who are owners of the data, for services and applications because storing data makes cost. Therefore, a smart community would have a database that collects the IoT data, and a Web API that distributes only allowed the data for third party, which would provide smart community services to the community, is necessary. To provide IoT services and applications to a smart community, IoT data, which is specifically collected by IoT devices, should be distributed after analyzing data meaning. Thus, the Web API provides both IoT data and meaning to enhance developing applications such as high-dense temperature information. As a case study, we show the smart-community-based Web API first, and then show examples on the utilization of real IoT data collected in Urawamisono, Saitama city, which is approximately 30 km from central Tokyo, in Japan.

Application Programming Interfaces (APIs) are software tools that help different programs work together. APIs can improve an organization's presence on the Web with tools that integrate various useful, popular programs. This study aimed to identify appropriate web-based APIs used by the most popular public library websites for presentation on Iranian public libraries’ websites. For this purpose, we conducted this study in two stages: In stage one, Web APIs were identified by reviewing the websites of the top public libraries in the world. Then, in stage two, using the obtained results, important Web APIs were selected utilizing experts' opinions (the heuristic method). In stage one, the 30 Web APIs in two categories were identified: 10 public Web APIs and 20 private Web APIs. Then, in stage two, 7 public APIs and 17 private Web APIs for these websites were selected, based on expert analysis. The results of this study can be used to improve the design of public library websites and enhance the communication of such websites' presence on the Web.