Software Performance Engineering Research Articles

Mutation-based analysis of queueing network performance models

Performance models have been used in the past to understand the performance characteristics of software systems. However, the identification of performance criticalities is still an open challenge, since there might be several system components contributing to the overall system performance. This work combines two different areas of research to improve the process of interpreting model-based performance analysis results: (i) software performance engineering that provides the ground for the evaluation of the system’s performance; (ii) mutation-based techniques that nicely supports the experimentation of changes in performance models and contribute to a more systematic assessment of performance indices. We propose mutation operators for specific performance models, i.e., queueing networks, that resemble changes commonly made by designers when exploring the properties of a system’s performance. Our approach consists in introducing a mutation-based approach that generates a set of mutated queueing network models. The performance of these mutated networks is compared to that of the original network to better understand the effect of variations in the different components of the system. A set of benchmarks is adopted to show how the technique can be used to get a deeper understanding of the performance characteristics of software systems.

Cloud Function Performance: a component modeling approach

Cloud Functions are a trend in cloud computing in which developers are allowed to install code in a Function-as-a-Service (FaaS) platform able to manage provisioning, execution, monitoring and automatic scaling. The underlying infrastructure in FaaS platforms is hidden from the developers and designers and, since the inuence of the infrastructure is unknown, this makes it di_cult to apply software performance engineering approaches on cloud functions, which could lead to wrong or inaccurate performance estimations. In this study, we explore the use of component-based modeling and simulation in order to generate performance estimations of an exemplar cloud function which was exercised using a variety of workloads. A cloud function was both implemented and instrumented to record performance datain a log _le, associated with its invocations; using the log _le as an input, we extracted a performance model in a Palladio Component Model format suitable for running simulations to validate whether the generated model could explain the runtime behavior of the function. Using this approach and further tunings in the model, we were able to validate that the simulations could explain more than 95% of the function's behavior and that component-based modeling and simulation can be considered a serious option when trying to explain the behavior of a cloud function.

Early Performance Prediction in Bioinformatics Systems Using Palladio Component Modeling

Bioinformatics is a branch of science that uses computers, algorithms, and databases to solve biological problems. To achieve more accurate results, researchers need to use large and complex datasets. Sequence alignment is a well-known field of bioinformatics that allows the comparison of different genomic sequences. The comparative genomics field allows the comparison of different genomic sequences, leading to benefits in areas such as evolutionary biology, agriculture, and human health (e.g., mutation testing connects unknown genes to diseases). However, software engineering best practices, such as software performance engineering, are not taken into consideration in most bioinformatics tools and frameworks, which may lead to serious performance problems. Having an estimate of the software performance in the early phases of the Software Development Life Cycle (SDLC) is beneficial in making better decisions relating to the software design. Software performance engineering provides a reliable and observable method to build systems that can achieve their required performance goals. In this paper, we introduce the use of the Palladio Component Modeling (PCM) methodology to predict the performance of a sequence alignment system. Software performance engineering was not considered during the original system development. As a result of the performance analysis, an alternative design is proposed. Comparing the performance of the proposed design against the one already developed, a better response time is obtained. The response time of the usage scenario is reduced from 16 to 8.6 s. The study results show that using performance models at early stages in bioinformatics systems can help to achieve better software system performance.

Modeling the Negotiation of Agents in MAS and Predicting the Performance an SPE Approach

Software performance engineering(SPE) process starts at the early stages of software development life cycle which helps to develop software that meets the performance requirements on time and budget. Multi-Agent Systems(MAS) are comprised of one or more agents who coordinate each other to accomplish some task. The coordination can be achieved through cooperation and negotiation. In the early development stages measuring the negotiation workload and predicting the performance remains an important but largely unsolved problem. The problem of uncertainty regarding the negotiation workload is required to be addressed by estimation techniques. Hence, in this research we developed a probabilistic model for the negotiation scenario among the agents in a given time horizon. The negotiation workload obtained from the probabilistic model is integrated with the representative workload of the agents for predicting the performance of agents in MAS. The tool SMTQA is used for obtaining the performance metrics. Analysis of the execution environment is done by considering various configurations in the hardware resources based on the dynamic workload of the negotiation agents over a time horizon. From the sensitivity analysis, the bottleneck resources are identified and suggestions for improvement are proposed.

A systematic approach for performance assessment using process mining

Software performance engineering is a mature field that offers methods to assess system performance. Process mining is a promising research field applied to gain insight on system processes. The interplay of these two fields opens promising applications in the industry. In this work, we report our experience applying a methodology, based on process mining techniques, for the performance assessment of a commercial data-intensive software application. The methodology has successfully assessed the scalability of future versions of this system. Moreover, it has identified bottlenecks components and replication needs for fulfilling business rules. The system, an integrated port operations management system, has been developed by Prodevelop, a medium-sized software enterprise with high expertise in geospatial technologies. The performance assessment has been carried out by a team composed by practitioners and researchers. Finally, the paper offers a deep discussion on the lessons learned during the experience, that will be useful for practitioners to adopt the methodology and for researcher to find new routes.

Theme section on performance modelling and engineering of software and systems

Theme section on performance modelling and engineering of software and systems

Line: Evaluating Software Applications in Unreliable Environments

Cloud computing has paved the way to the flexible deployment of software applications. This flexibility offers service providers a number of options to tailor their deployments to the observed and foreseen customer workloads, without incurring in large capital costs. However, cloud deployments pose novel challenges regarding application reliability and performance. Examples include managing the reliability of deployments that make use of spot instances, or coping with the performance variability caused by multiple tenants in a virtualized environment. In this paper, we introduce L ine , a tool for performance and reliability analysis of software applications. L ine solves layered queueing network (LQN) models, a popular class of stochastic models in software performance engineering, by setting up and solving an associated system of ordinary differential equations. A key differentiator of L ine compared to existing solvers for LQNs is that L ine incorporates a model of the environment the application operates in. This enables the modeling of reliability and performance issues such as resource failures, server breakdowns and repairs, slow start-up times, resource interference due to multitenancy, among others. This paper describes the L ine tool, its support for performance and reliability modeling, and illustrates its potential by comparing L ine predictions against data obtained from a cloud deployment. We also illustrate the applicability of L ine with a case study on reliability-aware resource provisioning.

An Appraisal of Web Applications vs. Web Services with respect to Performance Engineering using Software Performance Engineering Approach

An Appraisal of Web Applications vs. Web Services with respect to Performance Engineering using Software Performance Engineering Approach

Guest Editorial: Automation in Software Performance Engineering

Guest Editorial: Automation in Software Performance Engineering

PERFORMANCE REQUIREMENT ANALYSIS AND EVALUATION MODEL OF WEB APPLICATIONS

One of the principal characteristic of a software system is software Performance. Software Performance analysis helps to mitigate the risk of redesign. So the software development people pay more attention one software performance. Development People has to state accurate software performance requirements, proper plan to test cases, and utilize appropriate skills to examine the software performance characteristic which helps in developing a software system of admissible performance. But insufficiency of a management schema for software performance engineering may reduce efficacy of the software performance skills. In this paper, it proposes the Performance Requirement Analysis and Evaluation Model (PRAEM) as a software performance mainframe schema settled on the particularization of quantitative analysis and loads; PREM classifies performance requirements into three steps. At each step, it shows that which information should be included in the requirement, and which techniques can be used to estimate the performance.

DWPPT: Data Warehouse Performance Prediction Tool

The increasing demands for interactive response time from the users makes query performance one of the central problems of Data warehouse systems today. Performance is an important quality aspect of Data warehouse systems. Predicting the performance of Data warehouse systems during early design stages of their development is significant. Software Performance Engineering(SPE) promotes the idea that the integration of performance analysis into the software development process, from the earliest stages to the end, can ensure that the system will meet its performance objectives. This paper describes the features and use of a prototype tool, DWPPT(Data Warehouse Performance Prediction Tool) which is designed to analyze the performance of the Data warehouse in different environmental conditions. The tool supports SPE process for Data warehouse systems. The tool is useful for Data warehouse managers in identifying critical components, diagnosing problems and hence optimizing the overall design. Our objective is to investigate the impact of Data warehouse design factors on OLAP performance for different user populations and hardware configurations. An analytical and simulation modeling approach is used for the tool to predict performance of Data warehouse systems.

Towards software performance engineering for multicore and manycore systems

In the era of multicore and manycore processors, a systematic engineering approach for software performance becomes more and more crucial to the success of modern software systems. This article argues for more software performance engineering research specifically for multicore and manycore systems, which will have a profound impact on software engineering practices.

Modeling and evaluation of information systems using coloured petri network

Article history: Received January 4, 2014 Accepted 1 June 2014 Available online June 9 2014 Nowadays with the growth of organizations and their affiliates, the importance of information systems has increased. Functional and non-functional requirements of information systems in an organization are supported. There are literally several techniques to support the functional requirements in terms of software methodologies, but support for the second set of requirements has received little attention. Software Performance Engineering (SPE) forum tries to address this issue by presenting software methodologies to support both types of requirements. In this paper, we present a formal model for the evaluation of system performance based on a pragmatic model. Because of supporting the concurrency concepts, petri net has a higher priority than queuing system. For mapping UML to colored Petri net diagram, we use an intermediate graph. The preliminary results indicate that the proposed model may save significant amount of computations. © 2014 Growing Science Ltd. All rights reserved.

Software Performance Analysis with Parallel Programming Approaches

The term software performance engineering (SPE) is a systematic and quantitative approach for constructing software systems to meet the performance objectives such as response time, throughput, scalability and resource utilization. Optimization is major concern in achieving performance parameters. Optimization is performed during run-time, or in the design phase. This paper proposes the coding practices in Open Multi Processing (OpenMP) and Open Computing Language (OpenCL) that outperforms the conventional algorithms for searching, matrix multiplication and routing tasks in terms of response time.

Database system performance evaluation models: A survey

Considerable research has been conducted into software system performance modelling leading to various software performance engineering methodologies. Most of these methodologies target the software architecture level of systems, with limited work investigating the performance of database designs and systems. In this paper, we present a categorization of queueing network performance models of database systems in the literature. These models are classified based on the level of detail at which database transactions are modelled. In addition, we present a survey and evaluation of performance evaluation methodologies proposed to map database system specifications onto queueing network models with appropriate workloads. The paper identifies future research directions that should encourage a wider application of these methodologies in mainstream industrial practice.

UML2SAN: Toward A New Software Performance Engineering Approach

Software Performance Engineering (SPE) has recently considered as an important issue in the software development process. It consists on evaluate the performance of a system during the design phase. We have recently proposed a new methodology to generate a Stochastic Automata Network (SAN) model from a UML model, to consequently obtain performance predications from UML specifications. In this paper, we expand our idea to cover more complex UML models taking in advantage the modularity of SAN in modeling large systems. A formal description of the generation process is presented. The new extension gives rise to a serious approach in SPE that we call UML2SAN.

A Framework for Bottleneck Analysis in High-Performance Software Design

The design of complex software systems is a challenging task because it involves a wide range of quality attributes such as security, performance, reliability, to name a few. Dealing with each of these attributes requires specific set of skills, which quite often, involves making various trade-offs. This paper proposes a Model-Driven Software Performance Engineering (MDSPE) process that integrates performance requirements analysis with traditional software architecture design activities. One of the most important aspects of software performance engineering is bottleneck identification and removal. We propose in this paper a new framework for performance bottleneck analysis and removal based on a simple tabular notation. A case study is presented to illustrate how the proposed framework can comply with well-known performance models to assess the performance measures and make appropriate design decisions.

An approach for modeling and detecting software performance antipatterns based on first-order logics

The problem of interpreting the results of performance analysis is quite critical in the software performance domain. Mean values, variances and probability distributions are hard to interpret for providing feedback to software architects. Instead, what architects expect are solutions to performance problems, possibly in the form of architectural alternatives (e.g. split a software component in two components and re-deploy one of them). In a software performance engineering process, the path from analysis results to software design or implementation alternatives is still based on the skills and experience of analysts. In this paper, we propose an approach for the generation of feedback based on performance antipatterns. In particular, we focus on the representation and detection of antipatterns. To this goal, we model performance antipatterns as logical predicates and we build an engine, based on such predicates, aimed at detecting performance antipatterns in an XML representation of the software system. Finally, we show the approach at work on a case study.

Software Performance Modeling using the UML: a Case Study

The performance analysis of distributed software systems is a challenging task in which the assessment of performance measures is a vital step. Due to its versatility, the concept of software performance engineering (SPE) has been advocated as a promising solution towards realizing that step. This paper illustrates how by using our recently proposed Model-Driven SPE (MDSPE) approach, one can design annotated UML performance models for the performance analysis of distributed software systems, based on the UML profile for Schedulability, Performance and Time. An outline of system performance models and metrics is provided and a case study of a business system is used to validate the stated goal.

Early Performance Prediction of Co-Operative Multi-Agent Systems

Performance is a persistent quality of any software systems. Software performance engineering (SPE) encompasses efforts to describe and improve performance of systems at the early stages of development of the system. Multi-Agent Systems (MAS) are composed of autonomous entities called agents which cooperates together to solve complex distributed problems. Whatever complex the system, the quality of the system is an important parameter to be addressed. We propose a methodology to predict the performance of agents in MAS by considering the importance of cooperative behavior of the agents. Unified Modeling Language (UML) diagrams and mathematical formulas are used to quantify the cooperation at early stages of MAS. Our methodology is illustrated with the help of a case study. The performance metrics are obtained by simulating the cooperative behavior of MAS using the tool Simulation of Multi-tier Queuing Applications (SMTQA).