Software Ecosystem Research Articles

The SunPy Project: An interoperable ecosystem for solar data analysis

The SunPy Project is a community of scientists and software developers creating an ecosystem of Python packages for solar physics. The project includes the sunpy core package as well as a set of affiliated packages. The sunpy core package provides general purpose tools to access data from different providers, read image and time series data, and transform between commonly used coordinate systems. Affiliated packages perform more specialized tasks that do not fall within the more general scope of the sunpy core package. In this article, we give a high-level overview of the SunPy Project, how it is broader than the sunpy core package, and how the project curates and fosters the affiliated package system. We demonstrate how components of the SunPy ecosystem, including sunpy and several affiliated packages, work together to enable multi-instrument data analysis workflows. We also describe members of the SunPy Project and how the project interacts with the wider solar physics and scientific Python communities. Finally, we discuss the future direction and priorities of the SunPy Project.

Community-wide collaboration is a must to reinstall trust in bioinformatics solutions and biomedical interpretation.

Science has become a highly competitive undertaking concerning, for example, resources, positions, students, and publications. At the same time, the number of journals presenting scientific findings skyrockets while the knowledge increase per manuscript seems to be diminishing. Science has also become ever more dependent on computational analyses. For example, virtually all biomedical applications involve computational data analysis. The science community develops many computational tools, and there are numerous alternatives for many computational tasks. The same is true for workflow management systems, leading to a tremendous duplication of efforts. Software quality is often of low concern, and typically, a small dataset is used as a proof of principle to support rapid publication. Installation and usage of such tools are complicated, so virtual machine images, containers, and package managers are employed more frequently. These simplify installation and ease of use but do not solve the software quality issue and duplication of effort. We believe that a community-wide collaboration is needed to (a) ensure software quality, (b) increase reuse of code, (c) force proper software review, (c) increase testing, and (d) make interoperability more seamless. Such a science software ecosystem will overcome current issues and increase trust in current data analyses.

Organizational Influencers in Open-Source Software Projects

Traditional software development is shifting toward the open-source development model, particularly in the current environment of competitive challenges to develop software openly. The author employs a case study approach to investigate how organizations and their affiliated developers collaborate in the open-source software (OSS) ecosystem TensorFlow (TF). The analysis of the artificial intelligence OSS library TF combines social network analysis (SNA) and an examination of archival data by mining software repositories. The study looks at the structure and evolution of code-collaboration among developers and with the ecosystem's organizational networks over the TF lifespan. These involved organizations play a particularly critical role in development. The research also looks at productivity, homophily, development, and diversity among developers. The results deepen the understanding of OSS communities' collaborative developer and organization patterns. Furthermore, the study emphasizes the importance and evolution of social networks, diversity, and productivity in ecosystems.

Plots.jl – A User Extendable Plotting API for the Julia Programming Language

There are many excellent plotting libraries. Each excels at a specific use case: one is particularly suited for creating printable 2D figures for publication, another for generating interactive 3D graphics, while a third may have excellent L<small>A</small>T<small>E</small>X integration or be ideal for creating dashboards on the web. The aim of <tt>Plots.jl</tt> is to enable the user to use the same syntax to interact with a range of different plotting libraries, making it possible to change the library that does the actual plotting (the <em>backend</em>) without needing to touch the code that creates the content – and without having to learn multiple application programming interfaces (API). This is achieved by separating the specification of the plot from the implementation of the graphical backend. This plot specification is extendable by a <em>recipe</em> system that allows package authors and users to create new types of plots, as well as to specify how to plot any type of object (e.g. a statistical model, a map, a phylogenetic tree or the solution to a system of differential equations) without depending on the <tt>Plots.jl</tt> package. This design supports a modular ecosystem structure for plotting and yields a high code reuse potential across the entire Julia package ecosystem. <tt>Plots.jl</tt> is publicly available at <a href="https://github.com/JuliaPlots/Plots.jl" target="_blank">https://github.com/JuliaPlots/Plots.jl</a>.

Connect API with Blockchain: A Survey on Blockchain Oracle Implementation

A blockchain is a form of distributed ledger technology where transactions as data state changes are permanently recorded securely and transparently without the need for third parties. Besides, introducing smart contracts to the blockchain has added programmability, revolutionizing the software ecosystem toward decentralized applications. Although promising, the usability of smart contracts is primarily limited to on-chain data without access to the external systems (i.e., off-chain) where real-world data and events reside. This connectability to off-chain data for smart contracts and blockchain is an open practical problem referred to as the “oracle problem” and is defined as how real-world data can be transferred into/from the blockchain. Hence, Blockchain oracles are introduced and implemented in the form of application programming interfaces connecting the real world to the blockchain for mitigating such a limitation. This article studies and analyzes how blockchain oracles provide final feedback (i.e., outcome) to smart contracts and survey blockchain oracle technologies and mechanisms regarding data integrity and correctness. Since the existing solutions are extensive in terms of characteristics and usage, we investigate their structure and principles by classifying the blockchain oracle implementation techniques into two major groups voting-based strategies and reputation-based ones. The former mainly relies on participants’ stakes for outcome finalization, while the latter considers reputation and performance metrics in conjunction with authenticity-proof mechanisms for data correctness and integrity. We present the result of this classification with a thorough discussion of the state of the art and provide the remaining challenges and future research directions in the end.

The Co-evolution of the WordPress Platform and Its Plugins

One can extend the features of a software system by installing a set of additional components called plugins. WordPress, as a typical example of such plugin-based software ecosystems, is used by millions of websites and has a large number (i.e., 54,777) of available plugins. These plugin-based software ecosystems are different from traditional ecosystems (e.g., NPM dependencies) in the sense that there is high coupling between a platform and its plugins compared to traditional ecosystems for which components might not necessarily depend on each other (e.g., NPM libraries do not depend on a specific version of NPM or a specific version of a client software system). The high coupling between a plugin and its platform and other plugins causes incompatibility issues that occur during the co-evolution of a plugin and its platform as well as other plugins. In fact, incompatibility issues represent a major challenge when upgrading WordPress or its plugins. According to our study of the top 500 most-released WordPress plugins, we observe that incompatibility issues represent the third major cause for bad releases, which are rapidly (within the next 24 hours) fixed via urgent releases. Thirty-two percent of these incompatibilities are between a plugin and WordPress while 19% are between peer plugins. In this article, we study how plugins co-evolve with the underlying platform as well as other plugins, in an effort to understand the practices that are related support such co-evolution and reduce incompatibility issues. In particular, we investigate how plugins support the latest available versions of WordPress, as well as how plugins are related to each other, and how they co-evolve. We observe that a plugin’s support of new versions of WordPress with a large amount of code change is risky, as the releases that declare such support have a higher chance to be followed by an urgent release compared to ordinary releases. Although plugins support the latest WordPress version, plugin developers omit important changes such as deleting the use of removed WordPress APIs, which are removed a median of 873 days after the APIs have been removed from the source code of WordPress. Plugins introduce new releases that are made according to a median of five other plugins, which we refer to as peer-triggered releases. A median of 20% of the peer-triggered releases are urgent releases that fix problems in their previous releases. The most common goal of peer-triggered releases is the fixing of incompatibility issues that a plugin detects as late as after a median of 36 days since the last release of another plugin. Our work sheds light on the co-evolution of WordPress plugins with their platform as well as peer plugins in an effort to uncover the practices of plugin evolution, so WordPress can accordingly design approaches to avoid incompatibility issues.

Improving performance for gravitational-wave parameter inference with an efficient and highly-parallelized algorithm

The rapid iterative fitting (RIFT) parameter inference algorithm provides a framework for efficient, highly parallelized parameter inference for GW sources. In this paper, we summarize essential algorithm enhancements and operating point choices for the RIFT iterative algorithm, including settings used for analysis of LIGO/Virgo O3 observations. We also describe other extensions to the RIFT algorithm and software ecosystem. Some extensions increase RIFT's flexibility to produce outputs pertinent to GW astrophysics. Other extensions increase its computational efficiency or stability. Using many randomly selected sources, we assess code robustness with two distinct code configurations, one designed to mimic settings as of LIGO/Virgo O3 and another employing several performance enhancements. We illustrate RIFT's capabilities with analysis of selected events.

Vector Extensions in COTS Processors to Increase Guaranteed Performance in Real-Time Systems

The need for increased application performance in high-integrity systems such as those in avionics is on the rise as software continues to implement more complex functionalities. The prevalent computing solution for future high-integrity embedded products is multi-processor systems-on-chip (MPSoC) processors. MPSoCs include central processing unit (CPU) multicores that enable improving performance via thread-level parallelism. MPSoCs also include generic accelerators (graphics processing units [GPUs]) and application-specific accelerators. However, the data processing approach (DPA) required to exploit each of these underlying parallel hardware blocks carries several open challenges to enable the safe deployment in high-integrity domains. The main challenges include the qualification of its associated runtime system and the difficulties in analyzing programs deploying the DPA with out-of-the-box timing analysis and code coverage tools. In this work, we perform a thorough analysis of vector extensions (VExts) in current commercial off-the-shelf (COTS) processors for high-integrity systems. We show that VExts prevent many of the challenges arising with parallel programming models and GPUs. Unlike other DPAs, VExts require no runtime support, prevent design race conditions that might arise with parallel programming models, and have minimum impact on the software ecosystem, enabling the use of existing code coverage and timing analysis tools. We develop vectorized versions of neural network kernels and show that the NVIDIA Xavier VExts provide a reasonable increase in guaranteed application performance of up to 2.7x. Our analysis contends that VExts are the DPA approach with arguably the fastest path for adoption in high-integrity systems.

MEDUSA©: A novel Python-based software ecosystem to accelerate brain-computer interface and cognitive neuroscience research

Background and objective: Neurotechnologies have great potential to transform our society in ways that are yet to be uncovered. The rate of development in this field has increased significantly in recent years, but there are still barriers that need to be overcome before bringing neurotechnologies to the general public. One of these barriers is the difficulty of performing experiments that require complex software, such as brain-computer interfaces (BCI) or cognitive neuroscience experiments. Current platforms have limitations in terms of functionality and flexibility to meet the needs of researchers, who often need to implement new experimentation settings. This work was aimed to propose a novel software ecosystem, called MEDUSA©, to overcome these limitations.Methods: We followed strict development practices to optimize MEDUSA© for research in BCI and cognitive neuroscience, making special emphasis in the modularity, flexibility and scalability of our solution. Moreover, it was implemented in Python, an open-source programming language that reduces the development cost by taking advantage from its high-level syntax and large number of community packages.Results: MEDUSA© provides a complete suite of signal processing functions, including several deep learning architectures or connectivity analysis, and ready-to-use BCI and neuroscience experiments, making it one of the most complete solutions nowadays. We also put special effort in providing tools to facilitate the development of custom experiments, which can be easily shared with the community through an app market available in our website to promote reproducibility.Conclusions: MEDUSA© is a novel software ecosystem for modern BCI and neurotechnology experimentation that provides state-of-the-art tools and encourages the participation of the community to make a difference for the progress of these fields. Visit the official website at https://www.medusabci.com/ to know more about this project.

Decentralised finance platform ecosystems: conceptualisation and outlook

ABSTRACT This work contributes to architectural and economic understanding of a largely unexplored field of blockchain-based applications referred to as Decentralised Finance (DeFi). Merging technical and economic perspectives, we define DeFi as an ecosystem of decentralised applications which is built on top of permissionless smart contract platforms to mimic and extend traditional financial services. We frame DeFi as a research object in information systems (IS) and management-related platform literature. Further, we conceptualise DeFi as a platform economy within software ecosystems and in interdependence with exogenous factors. This enables us to identify three promising avenues for future research: (i) base layer blockchains as a platform for developers, (ii) the dApp economy of platform modules, and (iii) the diffusion and adoption of DeFi.

Listening to what the system tells us: Innovative auditing for distributed systems

IntroductionIn recent years, software ecosystems have become more complex with the proliferation of distributed systems such as blockchains and distributed ledgers. Effective management of these systems requires constant monitoring to identify any potential malfunctions, anomalies, vulnerabilities, or attacks. Traditional log auditing methods can effectively monitor the health of conventional systems. Yet, they run short of handling the higher levels of complexity of distributed systems. This study aims to propose an innovative architecture for system auditing that can effectively manage the complexity of distributed systems using advanced data analytics, natural language processing, and artificial intelligence.MethodsTo develop this architecture, we considered the unique characteristics of distributed systems and the various signals that may arise within them. We also felt the need for flexibility to capture these signals effectively. The resulting architecture utilizes advanced data analytics, natural language processing, and artificial intelligence to analyze and interpret the various signals emitted by the system.ResultsWe have implemented this architecture in the DELTA (Distributed Elastic Log Text Analyzer) auditing tool and applied it to the Hyperledger Fabric platform, a widely used implementation of private blockchains.DiscussionThe proposed architecture for system auditing can effectively handle the complexity of distributed systems, and the DELTA tool provides a practical implementation of this approach. Further research could explore this approach's potential applications and effectiveness in other distributed systems.

On the dependency heaviness of CRAN/Bioconductor ecosystem

The R package ecosystem is expanding fast and dependencies among packages are becoming more complex in the ecosystem. In this study, we explored the package dependencies from a new aspect. We applied a new metric named “dependency heaviness” which measures the number of additional strong dependencies that a package uniquely contributes to its child or downstream packages. We systematically studied how the dependency heaviness spreads from parent to child packages, and how it further spreads to remote downstream packages in the CRAN/Bioconductor ecosystem. We extracted top packages and key paths that majorly transmit heavy dependencies in the ecosystem. Additionally, the dependency heaviness analysis on the ecosystem has been implemented as a web-based database that provides comprehensive tools for querying dependencies of individual R packages.

Maintenance Cost of Software Ecosystem Updates

Over the past decades, software domains have witnessed a trend towards faster software release cycles, an increase of software components, and their connectivity. Examples include products in Industry 4.0 domains such as health care, retail, or mobility, in which, technically, software components are connected to a software ecosystem. Faster release cycles and high connectivity make many software ecosystem updates an expensive, arduous, and risky maintenance task. This paper leverages software release metadata to quantify the maintenance cost of software ecosystem updates. In particular, the data is applied to model a software ecosystem graph and subsequent calculate its maintenance cost. To illustrate, the model is prototyped and evaluated in three Industry 4.0 use cases.

Developer Cooperation Relationship and Attribute Similarity Based Community Detection in Software Ecosystem

Developer Cooperation Relationship and Attribute Similarity Based Community Detection in Software Ecosystem

Vulnerability of Package Dependency Networks

Software reuse by importing packages from centralised repositories is an efficient and increasingly widespread way to develop software. Given the transitivity of dependencies, defects introduced in the repository can have extensive effects on the software ecosystem. Drawing from complex network theory, we define a model of repository vulnerability based on the statistically expected damage that the repository sustains from the random introduction of software defects. We test the model in stylized networks derived from real repositories, PyPI, Maven and npm, and show that the existence of a giant strongly connected component (SCC) explains most of the vulnerability. Indeed, we found that theoretical protection (immunization) of this entire component would remove almost all vulnerability from the network. Since repositories and their communities have limited resources to mitigate issues, we further model the problem of how to best apply these resources, finding sets much smaller than the giant SCC whose protection is nearly as good. Furthermore, we prove that the optimal selection of sets of given size is NP-Hard but can be approached with heuristics, yielding respectable results. Our model contributes to a better understanding of software package repositories and could also be applied to other systems with a similar structure.

Armenian Virtual Observatory: Status and activities

Among all astronomical data, a large spectroscopic database is especially useful for understanding the nature of all observed objects. The Astrophysical Virtual Observatories (AVOs) have been created in a number of countries using their available databases and current observing material as a collection of interoperating data archives and software tools to form a research environment for processing complex data-intensive and compute-intensive research workflows. The Armenian Virtual Observatory (ArVO) is a project of the Byurakan Astrophysical Observatory (BAO) aimed at construction of a modern system for data archiving, extraction, acquisition, reduction, use and publication. ArVO is based on the Digitized First Byurakan Survey (DFBS) and is the Armenian contribution to the International Virtual Observatories Alliance (IVOA). One of the ArVO’s main tasks is to create and utilize a global Spectroscopic Virtual Observatory, which will combine data from DFBS and other low-dispersion spectroscopic databases, as well as provide the first understanding on the nature of any object up to B=18m. Hence, ArVO will utilize the DFBS as an appropriate large spectroscopic database. In the frame of ArVO, BAO collaborates with the Institute of Informatics and Automation Problems (IIAP) of the National Academy of Sciences of Armenia to develop software ecosystem for ArVO satisfying the IVOA standards. Besides the DFBS, ArVO provides access to the Digitized Second Byurakan Survey (SBS) database, the Byurakan photographic archive, and BAO 2.6 m and 1 m Schmidt telescopes modern observations.

Qualitative Clustering of Software Repositories Based on Software Metrics

Software repositories contain a wealth of information about the aspects related to software development process. For this reason, many studies analyze software repositories using methods of data analytics with a focus on clustering. Software repository clustering has been applied in studying software ecosystems such as GitHub, defect and technical debt prediction, software remodularization. Although some interesting insights have been reported, the considered studies exhibited some limitations. The limitations are associated with the use of individual clustering methods and manifesting in the shortcomings of the obtained results. In this study, to alleviate the existing limitations we engage multiple cluster validity indices applied to multiple clustering methods and carry out consensus clustering. To our knowledge, this study is the first to apply the consensus clustering approach to analyze software repositories and one of the few to apply the consensus clustering to software metrics. Intensive experimental studies are reported for software repository metrics data consisting of a number of software repositories each described by software metrics.We revealed seven clusters of software repositories and relate them to developers’ activity. It is advocated that the proposed clustering environment could be useful for facilitating the decision making process for business investors and open-source community with the help of the Gartner’s hype cycle.

Mechanistic forecasts of species responses to climate change: The promise of biophysical ecology.

A core challenge in global change biology is to predict how species will respond to future environmental change and to manage these responses. To make such predictions and management actions robust to novel futures, we need to accurately characterize how organisms experience their environments and the biological mechanisms by which they respond. All organisms are thermodynamically connected to their environments through the exchange of heat and water at fine spatial and temporal scales and this exchange can be captured with biophysical models. Although mechanistic models based on biophysical ecology have a long history of development and application, their use in global change biology remains limited despite their enormous promise and increasingly accessible software. We contend that greater understanding and training in the theory and methods of biophysical ecology is vital to expand their application. Our review shows how biophysical models can be implemented to understand and predict climate change impacts on species' behavior, phenology, survival, distribution, and abundance. It also illustrates the types of outputs that can be generated, and the data inputs required for different implementations. Examples range from simple calculations of body temperature at a particular site and time, to more complex analyses of species' distribution limits based on projected energy and water balances, accounting for behavior and phenology. We outline challenges that currently limit the widespread application of biophysical models relating to data availability, training, and the lack of common software ecosystems. We also discuss progress and future developments that could allow these models to be applied to many species across large spatial extents and timeframes. Finally, we highlight how biophysical models are uniquely suited to solve global change biology problems that involve predicting and interpreting responses to environmental variability and extremes, multiple or shifting constraints, and novel abiotic or biotic environments.

What are the characteristics of highly-selected packages? A case study on the npm ecosystem

With the popularity of software ecosystems, the number of open source components (known as packages) has grown rapidly. Identifying high-quality and well-maintained packages from a large pool of packages to depend on is a basic and important problem, as it is beneficial for various applications, such as package recommendation and package search. However, no systematic and comprehensive work focuses on addressing this problem except in online discussions or informal literature and interviews. To fill this gap, in this paper, we conducted a mixed qualitative and quantitative analysis to understand how developers identify and select relevant open source packages. In particular, we started by surveying 118 JavaScript developers from the npm ecosystem to qualitatively understand the factors that make a package to be highly-selected within the npm ecosystem. The survey results showed that JavaScript developers believe that highly-selected packages are well-documented, receive a high number of stars on GitHub, have a large number of downloads, and do not suffer from vulnerabilities. Then, we conducted an experiment to quantitatively validate the developers’ perception of the factors that make a highly-selected package. In this analysis, we collected and mined historical data from 2,527 packages divided into highly-selected and not highly-selected packages. For each package in the dataset, we collected quantitative data to present the factors studied in the developers’ survey. Next, we used regression analysis to quantitatively investigate which of the studied factors are the most important. Our regression analysis complements our survey results about highly-selected packages. In particular, the results showed that highly-selected packages tend to be correlated by the number of downloads, stars, and how large the package’s readme file is.

Agile Practices and Organizational Agility in Software Ecosystems

As software products increasingly become part of larger ecosystems, research and development (R&D) units of software producers organize themselves around projects to become more responsive to the environment. Organizations participating in software ecosystems must continuously adapt and adjust their software development and maintenance processes to drive both medium-term and long-term innovation. Agile methods and practices are widely adopted to guide the collaboration within and between project teams in software development. Moreover, when successfully attained, agility can drive innovation by enabling software development organizations to cope with technological changes and exploit emerging opportunities in software ecosystems. In this article, we focus on how organizations attain agility in the maintenance and development of software products. To answer this question, we conduct a longitudinal case study of Agile Scrum implementation in the R&D unit of a major supplier of telecommunication equipment. We investigate the emerging tensions and highlight practices used to balance these tensions in the Agile Scrum implementation. We identify four capabilities and ten practices that support effective collaboration and coordination in the development and maintenance of software products. The study offers practical guidance for R&D managers to attain agility in software ecosystems.