API Usage Research Articles

Accelerating API-Based Program Synthesis via API Usage Pattern Mining

Program Synthesis is an exciting topic in software engineering which aims to generate programs satisfying user intent automatically. Although different approaches have been proposed in program synthesis, only small or domain-specific programs can be generated in practice, the main obstacle of which lies in the intractability of program space. With the rapid growth of reusable libraries, component-based synthesis provides a promising way, such as synthesizing Java programs that are only composed of APIs. However, the efficiency of searching for proper solutions for complex tasks is still a challenge. In certain scenarios, some API methods are frequently called together. The usage of these API methods always follows some usage patterns. Incorporating the information about API usage patterns will help to accelerate the speed of program synthesis. However, state-of-the-art synthesis tools do not capture the inherent relationships between API methods. Aiming at this problem, we propose a novel approach to accelerate the speed of API-based program synthesis via API usage pattern mining. It is a general approach that can be applied to any approach of API-based synthesis. We first collect open source projects of high quality from the Internet and use an off-the-shelf API-usage-pattern-mining tool to mine API usage patterns from these code snippets. We use two strategies to incorporate the information about API usage patterns with program synthesis, and either strategy can improve the efficiency of program synthesis. We evaluate our approach on 20 real programming tasks, which shows that our approach can accelerate the speed of program synthesis by 86% compared to the baseline.

IDENTIFICATION OF PROMOTED ECLIPSE UNSTABLE INTERFACES USING CLONE DETECTION TECHNIQUE

The Eclipse framework is a popular and widely used framework that has been evolving for over a decade. The framework provides both stable interfaces (APIs) and unstable interfaces (non-APIs). Despite being discouraged by Eclipse, client developers often use non-APIs which may cause their systems to fail when ported to new framework releases. To overcome this problem, Eclipse interface producers may promote unstable interfaces to APIs. However, client developers have no assistance to aid them to identify the promoted unstable interfaces in the Eclipse framework. We aim to help API users identify promoted unstable interfaces. We used the clone detection technique to identify promoted unstable interfaces as the framework evolves. Our empirical investigation on 16 Eclipse major releases presents the following observations. First, we have discovered that there exists over 60% non-API methods of the total interfaces in each of the analyzed 16 Eclipse releases. Second, we have discovered that the percentage of promoted non-APIs identified through clone detection ranges from 0.20% to 10.38%.

Identification of Promoted Eclipse Unstable Interfaces Using Clone Detection Technique

The Eclipse framework is a popular and widely used framework that has been evolving for over a decade. The framework provides both stable interfaces (APIs) and unstable interfaces (non-APIs). Despite being discouraged by Eclipse, client developers often use non-APIs which may cause their systems to fail when ported to new framework releases. To overcome this problem, Eclipse interface producers may promote unstable interfaces to APIs. However, client developers have no assistance to aid them to identify the promoted unstable interfaces in the Eclipse framework. We aim to help API users identify promoted unstable interfaces. We used the clone detection technique to identify promoted unstable interfaces as the framework evolves. Our empirical investigation on 16 Eclipse major releases presents the following observations. First, we have discovered that there exists over 60% non-API methods of the total interfaces in each of the analyzed 16 Eclipse releases. Second, we have discovered that the percentage of promoted non-APIs identified through clone detection ranges from 0.20% to 10.38%.

A Comment-Driven Approach to API Usage Patterns Discovery and Search

Considerable effort has gone into the discovery of API usage patterns or examples. However, how to enable programmers to search for discovered API usage examples using natural language queries is still a significant research problem. This paper presents an approach, referred to as Codepus, to facilitate the discovery of API usage examples based on mining comments in open source code while permitting searches using natural language queries. The approach includes two key features: API usage patterns as well as multiple keywords and tf-idf values are discovered by mining open source comments and code snippets; and a matchmaking function is devised for searching for API usage examples using natural language queries by aggregating scores related to semantic similarity, correctness, and the number of APIs. In a practical application, the proposed approach discovered 43,721 API usage patterns with 641,591 API usage examples from 15,814 open source projects. Experiment results revealed the following: (1) Codepus reduced the browsing time required for locating API usage examples by 46.5%, compared to the time required when using a web search engine. (2) The precision of Codepus is 91% when using eleven real-world frequently asked questions, which is superior to those of Gists and Open Hub.

Mining API usage change rules for software framework evolution

Mining API usage change rules for software framework evolution

A Dynamically Linked Library Based Indirect Call Function Analysis for Detecting Banned API Usage in Binary Code

A Dynamically Linked Library Based Indirect Call Function Analysis for Detecting Banned API Usage in Binary Code

DroidEnsemble: Detecting Android Malicious Applications With Ensemble of String and Structural Static Features

Android platform has dominated the operating system of mobile devices. However, the dramatic increase of Android malicious applications (malapps) has caused serious software failures to Android system and posed a great threat to users. The effective detection of Android malapps has thus become an emerging yet crucial issue. Characterizing the behaviors of Android applications (apps) is essential to detecting malapps. Most existing works on detecting Android malapps were mainly based on string static features, such as permissions and API usage extracted from apps. There also exists work on the detection of Android malapps with structural features, such as control flow graph and data flow graph. As Android malapps have become increasingly polymorphic and sophisticated, using only one type of static features may result in false negatives. In this paper, we propose DroidEnsemble that takes advantages of both string features and structural features to systematically and comprehensively characterize the static behaviors of Android apps and thus build a more accurate detection model for the detection of Android malapps. We extract each app’s string features, including permissions, hardware features, filter intents, restricted API calls, used permissions, code patterns, as well as structural features like function call graph. We then use three machine learning algorithms, namely, support vector machine, k-nearest neighbor, and random forest, to evaluate the performance of these two types of features and of their ensemble. In the experiments, we evaluate our methods and models with 1386 benign apps and 1296 malapps. Extensive experimental results demonstrate the effectiveness of DroidEnsemble. It achieves the detection accuracy as 95.8% with only string features and as 90.68% with only structural features. DroidEnsemble reaches the detection accuracy as 98.4% with the ensemble of both types of features, reducing 9 false positives and 12 false negatives compared to the results with only string features.

Android App behavior-intent inference based on API usage analysis

Android App behavior-intent inference based on API usage analysis

Lightweight, Obfuscation-Resilient Detection and Family Identification of Android Malware

The number of malicious Android apps is increasing rapidly. Android malware can damage or alter other files or settings, install additional applications, and so on. To determine such behaviors, a security analyst can significantly benefit from identifying the family to which an Android malware belongs rather than only detecting if an app is malicious. Techniques for detecting Android malware, and determining their families, lack the ability to handle certain obfuscations that aim to thwart detection. Moreover, some prior techniques face scalability issues, preventing them from detecting malware in a timely manner. To address these challenges, we present a novel machine-learning-based Android malware detection and family identification approach, RevealDroid, that operates without the need to perform complex program analyses or to extract large sets of features. Specifically, our selected features leverage categorized Android API usage, reflection-based features, and features from native binaries of apps. We assess RevealDroid for accuracy, efficiency, and obfuscation resilience using a large dataset consisting of more than 54,000 malicious and benign apps. Our experiments show that RevealDroid achieves an accuracy of 98% in detection of malware and an accuracy of 95% in determination of their families. We further demonstrate RevealDroid’s superiority against state-of-the-art approaches.

A fast and efficient python library for interfacing with the Biological Magnetic Resonance Data Bank

BackgroundThe Biological Magnetic Resonance Data Bank (BMRB) is a public repository of Nuclear Magnetic Resonance (NMR) spectroscopic data of biological macromolecules. It is an important resource for many researchers using NMR to study structural, biophysical, and biochemical properties of biological macromolecules. It is primarily maintained and accessed in a flat file ASCII format known as NMR-STAR. While the format is human readable, the size of most BMRB entries makes computer readability and explicit representation a practical requirement for almost any rigorous systematic analysis.ResultsTo aid in the use of this public resource, we have developed a package called nmrstarlib in the popular open-source programming language Python. The nmrstarlib’s implementation is very efficient, both in design and execution. The library has facilities for reading and writing both NMR-STAR version 2.1 and 3.1 formatted files, parsing them into usable Python dictionary- and list-based data structures, making access and manipulation of the experimental data very natural within Python programs (i.e. “saveframe” and “loop” records represented as individual Python dictionary data structures). Another major advantage of this design is that data stored in original NMR-STAR can be easily converted into its equivalent JavaScript Object Notation (JSON) format, a lightweight data interchange format, facilitating data access and manipulation using Python and any other programming language that implements a JSON parser/generator (i.e., all popular programming languages). We have also developed tools to visualize assigned chemical shift values and to convert between NMR-STAR and JSONized NMR-STAR formatted files. Full API Reference Documentation, User Guide and Tutorial with code examples are also available.We have tested this new library on all current BMRB entries: 100% of all entries are parsed without any errors for both NMR-STAR version 2.1 and version 3.1 formatted files. We also compared our software to three currently available Python libraries for parsing NMR-STAR formatted files: PyStarLib, NMRPyStar, and PyNMRSTAR.ConclusionsThe nmrstarlib package is a simple, fast, and efficient library for accessing data from the BMRB. The library provides an intuitive dictionary-based interface with which Python programs can read, edit, and write NMR-STAR formatted files and their equivalent JSONized NMR-STAR files. The nmrstarlib package can be used as a library for accessing and manipulating data stored in NMR-STAR files and as a command-line tool to convert from NMR-STAR file format into its equivalent JSON file format and vice versa, and to visualize chemical shift values. Furthermore, the nmrstarlib implementation provides a guide for effectively JSONizing other older scientific formats, improving the FAIRness of data in these formats.

Guest editor's introduction to the Special Issue on Program Comprehension (ICPC 2014)

It is our pleasure to introduce you to the papers in this Special Issue based on the 22th International Conference on Program Comprehension (ICPC 2014). Program comprehension plays a central role in most of the phases of the software development life cycle, where it helps facilitate reuse, inspection, maintenance, reverse engineering, reengineering, migration, and extension of existing software systems. The International Conference on Program Comprehension (ICPC) is the primary venue for work in the area of program comprehension. It is also one of the leading venues for work in the areas of software analysis, reverse engineering, software evolution, and software visualization. ICPC provides an opportunity for researchers and industry practitioners to present and discuss the state-of-the-art and the state-of-the-practice in program comprehension and related areas. ICPC 2014 took place during June 2–3, 2014, in Hyderabad, India, and was co-located with the International Conference on Software Engineering (ICSE 2014). ICPC 2014 received a record number of submissions (76) from 19 different countries, which allowed us to assemble an excellent program that continues ICPC's tradition of providing a high-quality venue for sharing the latest advances in program comprehension. The program included 20 full research papers, 11 short papers and 5 tool demonstration papers. Of these 20 full research papers, five were invited to submit an extended version to the Journal of Software: Evolution and Process. After a rigorous reviewing process with at least three reviewers per paper, four papers were accepted for publication in this special section. The paper entitled ‘Framing Program Comprehension as Fault Localization’ by Alexandre Perez and Rui Abreu proposes an approach, coined Spectrum-based Feature Comprehension (SFC), that borrows techniques from software-fault localization that were proven to be effective even when debugging large applications. SFC analyses the program by exploiting run-time information from test case executions to identify the components that are important for a given feature, helping software engineers to understand how a program is structured and each of the functionality's dependencies are. They present a toolset, coined PANGOLIN, that implements SFC and displays its report to the user using an intuitive visualization. The paper entitled ‘Searching Crowd Knowledge to Recommend Solutions for API Usage Tasks’ by Eduardo C. Campos, Lucas B. L. de Souza, and Marcelo de A. Maia presents an approach that makes use of ‘crowd knowledge’ in Stack Overflow to recommend information that can assist developer activities. This strategy recommends a ranked list of question-answer pairs from Stack Overflow based on a query. The criteria for ranking are based on three main aspects: the textual similarity of the pairs with respect to the query related to the developer's problem, the quality of the pairs, and a filtering mechanism that considers only ‘how-to’ posts. The paper entitled, ‘AmaLgam+: Composing Rich Information Sources for Accurate Bug Localization’ by Shaowei Wang and David Lo proposes AmaLgam+, which is a method for locating relevant buggy files that combines five sources of information, namely, version history, similar reports, structure, stack traces, and reporter information. They perform a large-scale experiment on four open source projects, namely, AspectJ, Eclipse, SWT, and ZXing to localize more than 3000 bugs and compare AmaLgam+ results with those of six state-of-the-art bug localization approaches. The study showed that the newly proposed method outperforms these existing approaches in terms of mean average precision. The paper entitled, ‘Source code analysis with LDA’ by David Binkley, Daniel Heinz, Dawn Lawrie and Justin Overfelt aims to aid software engineers in their understanding of the Latent Dirichlet allocation (LDA) tuning parameters by numerically and graphically demonstrating the relationship between the tuning parameters and the LDA output. LDA has seen increasing use in the understanding of source code and its related artifacts in part because of its impressive modeling power. However, this expressive power comes at a cost: The technique includes several tuning parameters whose impact on the resulting LDA model must be carefully considered. The aim of their work is to provide insights into the tuning parameters' impact. We hope that readers will enjoy this special issue and gain useful insights from the four papers presented. We would like to thank all the authors who submitted papers to the conference and the authors who significantly extending their work for this special issue. In addition, we would like to thank the members of the ICPC 2014 program committee and the external reviewers for their time, careful reviews, and active discussions of the submitted papers, which helped making this special issue special. Finally, we would like to thank the editorial board of the Journal of Software: Evolution and Process and the publisher Wiley for providing us with the opportunity to devote this issue to the best of ICPC 2014. We also thank Journal of Software: Evolution and Process Editor Gerardo Canfora for providing expert guidance and important advice throughout the process. Enjoy!

Fine-GRAPE: fine-grained APi usage extractor \u2013 an approach and dataset to investigate API usage

An Application Programming Interface (API) provides a set of functionalities to a developer with the aim of enabling reuse. APIs have been investigated from different angles such as popularity usage and evolution to get a better understanding of their various characteristics. For such studies, software repositories are mined for API usage examples. However, many of the mining algorithms used for such purposes do not take type information into account. Thus making the results unreliable. In this paper, we aim to rectify this by introducing fine-GRAPE, an approach that produces fine-grained API usage information by taking advantage of type information while mining API method invocations and annotation. By means of fine-GRAPE, we investigate API usages from Java projects hosted on GitHub. We select five of the most popular APIs across GitHub Java projects and collect historical API usage information by mining both the release history of these APIs and the code history of every project that uses them. We perform two case studies on the resulting dataset. The first measures the lag time of each client. The second investigates the percentage of used API features. In the first case we find that for APIs that release more frequently clients are far less likely to upgrade to a more recent version of the API as opposed to clients of APIs that release infrequently. The second case study shows us that for most APIs there is a small number of features that is actually used and most of these features relate to those that have been introduced early in the APIs lifecycle.

Searching crowd knowledge to recommend solutions for API usage tasks

AbstractStack Overflow (SO) is a question and answer service directed to issues related to software development. In SO, developers post questions related to a programming topic and other members of the site can provide answers to help them. The information available on this type of service is also known as ‘crowd knowledge’ and currently is one important trend in supporting activities related to software development.We present an approach that makes use of ‘crowd knowledge’ in SO to recommend information that can assist developer activities. This strategy recommends a ranked list of question‐answer pairs from SO based on a query. The criteria for ranking are based on three main aspects: the textual similarity of the pairs with respect to the query related to the developer's problem, the quality of the pairs, and a filtering mechanism that considers only ‘how‐to’ posts. We conducted an experiment considering programming problems on three different topics (Swing, Boost and LINQ) widely used by the software development community to evaluate the proposed recommendation strategy. The results have shown that for Lucene + Score + How-to approach, 77.14% of the assessed activities have at least one recommended pair proved to be useful concerning the target programming problem. Copyright © 2016 John Wiley & Sons, Ltd.

Stepwise API usage assistance using n -gram language models

An IDE code completion mechanism for stepwise API exploration.A method for building a language model for supporting (1).a detailed evaluation of (2), showing how n-grams perform against existing client code.Results of (3) revealing a 90% chance of getting the expected result in the 10-top-most code completion proposals. Reusing software involves learning third-party APIs, a process that is often time-consuming and error-prone. Recommendation systems for API usage assistance based on statistical models built from source code corpora are capable of assisting API users through code completion mechanisms in IDEs. A valid sequence of API calls involving different types may be regarded as a well-formed sentence of tokens from the API vocabulary. In this article we describe an approach for recommending subsequent tokens to complete API sentences using n-gram language models built from source code corpora. The provided system was integrated in the code completion facilities of the Eclipse IDE, providing contextualized completion proposals for Java taking into account the nearest lines of code. The approach was evaluated against existing client code of four widely used APIs, revealing that in more than 90% of the cases the expected subsequent token is within the 10-top-most proposals of our models. The high score provides evidence that the recommendations could help on API learning and exploration, namely through the assistance on writing valid API sentences.

Understanding the API usage in Java

ContextApplication Programming Interfaces (APIs) facilitate the use of programming languages. They define sets of rules and specifications for software programs to interact with. The design of language API is usually artistic, driven by aesthetic concerns and the intuitions of language architects. Despite recent studies on limited scope of API usage, there is a lack of comprehensive, quantitative analyses that explore and seek to understand how real-world source code uses language APIs. ObjectiveThis study aims to understand how APIs are employed in practical development and explore their potential applications based on the results of API usage analysis. MethodWe conduct a large-scale, comprehensive, empirical analysis of the actual usage of APIs on Java, a modern, mature, and widely-used programming language. Our corpus contains over 5000 open-source Java projects, totaling 150 million source lines of code (SLoC). We study the usage of both core (official) API library and third-party (unofficial) API libraries. We resolve project dependencies automatically, generate accurate resolved abstract syntax trees (ASTs), capture used API entities from over 1.5 million ASTs, and measure the usage based on our defined metrics: frequency, popularity and coverage. ResultsOur study provides detailed quantitative information and yield insight, particularly, (1) confirms the conventional wisdom that the usage of APIs obeys Zipf distribution; (2) demonstrates that core API is not fully used (many classes, methods and fields have never been used); (3) discovers that deprecated API entities (in which some were deprecated long ago) are still widely used; (4) evaluates that the use of current compact profiles is under-utilized; (5) identifies API library coldspots and hotspots. ConclusionsOur findings are suggestive of potential applications across language API design, optimization and restriction, API education, library recommendation and compact profile construction.

Improving Authentication and Authorization for Identity Based Cloud Environment Using OAUTH with Fuzzy Based Blowfish Algorithm

Cloud computing involves group of remote servers, software, networks, centralized storage, which allows different number of online services and resources to virtualized business environment, individual users and educational institutions. But the main challenges are security between the user information, trust between the user and service provider. In business environment, Private Cloud Model (PCM) provides tight security of the application data even though in cloud, security issues such as authorization, confidentiality still exist. Therefore, the paper focuses on security, authentication and authorization using OAuth protocol. OAuth protocol authorizes the user while accessing the data from one application via another application for managing identities. OAuth HTTP based protocol is used for authorization and security is implemented using Fuzzy based blowfish encryption algorithm (FBFE). In OAuth, authorization is granted in four different ways like authorization code, implicit, resource owner password credential and client credential. OAuth establishes authorization between application and API user using token and redirect URI.

Mining usage patterns for the Android API

5 API methods are not used alone, but in groups and following some patterns. However, despite being a key information for API users, most usage patterns are not explicitly described in official API documents. In this article, we report a study to evaluate the feasibility of automatically enriching API documents with information on usage patterns. For this purpose, we extract and analyze 1,952 usage patterns, from a set of 396 Android applications. As part of our findings, we report that the Android API has many undocumented and non-trivial usage patterns, which can be inferred using association rule mining algorithms. We also describe a field study where a version of the original Android documentation was instrumented with the extracted usage patterns and source code examples. During 17 months, this documentation received 77,863 visits from real Android developers. . 6

Cloud-based mapreduce workflow execution platform

With increasing demand of data-intensive applications, mapreduce technologies have become useful tools to develop large scale applications efficiently by integrating various existing mapreduce jobs. However, there are few existing researches of workflow systems which can integrates mapreduce jobs with on-demand cloud resource provisioning. In this paper, we present a new cloud-based mapreduce workflow execution platform named DIVECWM (Distributed-parallel Virtual Environment on Cloud computing for Workflow for launching Mapreduce jobs) which integrates multiple mapreduce jobs and legacy programs into a single workflow. It provides a transparent and selective job scheduling by estimating the execution time in advance for workflow to execute all its jobs. Also, it supports automatic resource provisioning scheme which offers on-demand VM resources automatically to launch a workflow onto cloud. Furthermore, it provides an agent based resource management for automatic job deployment and execution of workflow on mapreduce clusters. Additionally, service oriented architecture based on web service API and graphical user interface offers high accessibility and convenience to user and other systems. We show the experimental results which compares the different scheduling schemes for various workflows.

APIs and Researchers: The Emperor's New Clothes?

As part of the Europeana Cloud (eCloud) project, Trinity College Dublin investigated best practice in the use of web services, such as APIs, for accessing large data sets from cultural heritage collections. This research looked into the provision and use of APIs, and moreover, whether or not more customised programmatic access to datasets is what researchers want or need. In order to understand whether current patterns of API usage reflect a skills gap on the part of researchers or a mismatch of tool to purpose, we looked not only at the creators and developer/users of APIs, but also at humanists already re-using big data; approaches in cultural heritage institutions and other research infrastructures to bring API use to non-technical audiences; and the kinds of training and other support services available or emerging within the data-intensive humanities research lifecycle. We conducted both desk research and a series of 11 interviews with figures working as researchers, developers or data providers, including figures from both the API development and the data usage communities. This research, conducted under the eCloud project and supported by the European Commission’s ICT Policy and Support Programme (Grant number 325091), was begun in March 2014 and is now in its concluding validation stage. The results of the research are not yet finalised, but the contribution is already emerging of this work to the debate about APIs being either the way forward for digital cultural heritage collections, or the Emperor’s New Clothes (or maybe a bit of both).

Usage contracts: Offering immediate feedback on violations of structural source-code regularities

Developers often encode design knowledge through structural regularities such as API usage protocols, coding idioms and naming conventions. As these regularities express how the source code should be structured, they provide vital information for developers using or extending that code. Adherence to such regularities tends to deteriorate over time because they are not documented and checked explicitly. This paper introduces uContracts, an internal DSL to codify and verify such regularities as ‘usage contracts’. Our DSL aims at covering most common usage regularities, while still providing a means to express less common ones. Common regularities are identified based on regularities supported by existing approaches to detect bugs or suggest missing code fragments, techniques that mine for structural regularities, as well as on the analysis of an open-source project. We validate our DSL by documenting the structural regularities of an industrial case study, and analyse how useful the information provided by checking these regularities is for the developers of that case study.