Non-functional Properties Research Articles

Multi-objective improvement of Android applications

Non-functional properties, such as runtime or memory use, are important to mobile app users and developers, as they affect user experience. We propose a practical approach and the first open-source tool, GIDroid for multi-objective automated improvement of Android apps. In particular, we use Genetic Improvement, a search-based technique that navigates the space of software variants to find improved software. We use a simulation-based testing framework to greatly improve the speed of search. GIDroid contains three state-of-the-art multi-objective algorithms, and two new mutation operators, which cache the results of method calls. Genetic Improvement relies on testing to validate patches. Previous work showed that tests in open-source Android applications are scarce. We thus wrote tests for 21 versions of 7 Android apps, creating a new benchmark for performance improvements. We used GIDroid to improve versions of mobile apps where developers had previously found improvements to runtime, memory, and bandwidth use. Our technique automatically re-discovers 64% of existing improvements. We then applied our approach to current versions of software in which there were no known improvements. We were able to improve execution time by up to 35%, and memory use by up to 33% in these apps.

Emergent Model Based On Hybrid Rough Sets Systems

The development of many knowledge discovery methods provided us with a good foundation for building hybrid systems based on rough set theory for knowledge discovery in a database. The need for more effective methods of generating and maintaining global nonfunctional properties suggests an approach analogous to those of natural processes in generating emergent properties. An emergent model allows the constraints of the task to be represented more naturally and permits only pertinent task-specific knowledge to emerge in the course of solving the problem. This paper describes some basics of emergent phenomena and its implementation in the hybrid system of rough sets combined with other methods. Further, demonstrations and guidelines are presented on how to exploit emergent intelligence and extend the problem-solving capabilities of these hybrid systems.

VeriFog: A Generic Model-based Approach for Verifying Fog Systems at Design Time and Generating Deployment Configurations

Fog Computing is a paradigm decentralizing the Cloud by geographically distributing computation, storage, network resources and related services. It provides benefits such as reducing the number of bottlenecks, limiting unwanted data movements, etc. However, managing the size, complexity and heterogeneity of the Fog systems to be engineered is challenging and can quickly become costly. According to best practices in software engineering, verification tasks could be performed on a system design prior to its implementation and deployment. We propose a generic model-based approach for verifying Fog systems at design time, also enabling the automatic generation of corresponding deployment configuration files. Named VeriFog, this approach is notably based on a customizable Fog Modeling Language (FML). We experimented in practice by modeling three use cases, from three different application domains, and by considering three main types of non-functional properties to be verified. From this modeling and verification effort, we show that we are able to automatically generate usable deployment configuration files for different deployment tools. In direct collaboration with our industrial partner Smile, the approach and underlying language presented in this paper are necessary steps towards a more global model-based support for the complete life cycle of Fog systems.

Speeding up Genetic Improvement via Regression Test Selection

Genetic Improvement (GI) uses search-based optimisation algorithms to automatically improve software with respect to both functional and non-functional properties. Our previous work showed that Regression Test Selection (RTS) can help speed up the use of GI and enhance the overall results while not affecting the software system’s validity. This article expands upon our investigation by answering further questions about safety and applying a GI algorithm based on Local Search (LS) in addition to the previously explored Genetic Programming (GP) approach. Further, we extend the number of subjects to 12 by analysing five larger real-world open-source programs. We empirically compare two state-of-the-art RTS techniques combined with GP and LS for these 12 programs. The results show that both RTS techniques are safe to use and can reduce the cost of GI by up to 80% and by 31% on average across programs. We also observe that both search-based algorithms impact the effectiveness gains of GI differently, and that various RTS strategies achieve differing gains in terms of efficiency. These results serve as further evidence that RTS must be used as a core component of the GI search process to maximise its effectiveness and efficiency.

Certifying Accuracy, Privacy, and Robustness of ML-Based Malware Detection

Recent advances in artificial intelligence (AI) are radically changing how systems and applications are designed and developed. In this context, new requirements and regulations emerge, such as the AI Act, placing increasing focus on strict non-functional requirements, such as privacy and robustness, and how they are verified. Certification is considered the most suitable solution for non-functional verification of modern distributed systems, and is increasingly pushed forward in the verification of AI-based applications. In this paper, we present a novel dynamic malware detector driven by the requirements in the AI Act, which goes beyond standard support for high accuracy, and also considers privacy and robustness. Privacy aims to limit the need of malware detectors to examine the entire system in depth requiring administrator-level permissions; robustness refers to the ability to cope with malware mounting evasion attacks to escape detection. We then propose a certification scheme to evaluate non-functional properties of malware detectors, which is used to comparatively evaluate our malware detector and two representative deep-learning solutions in literature.

Investigating the Effect of m-Commerce Application’s Functional and Non-Functional Attributes on Usability and Continuance Intention

The development of mobile commerce (m-commerce) applications has indicated a shift in goals from ease-of-use to sustainable use in the future. This shift has prompted changes in the combination of attributes that constitute the usability of m-commerce applications. This study developed an m-commerce usability model that combines functional and non-functional attributes. The research data was collected using questionnaires distributed to users living in Jakarta and was processed using the Partial Least Squares Structural Equation Modeling (PLS-SEM) technique. The results demonstrate that efficiency, satisfaction, and effectiveness can explain the usability of m-commerce applications. This study proved that three non-functional usability attributes – productivity, navigation, and memorability – have a positive effect on efficiency, while two functional attributes of usability – content and security – positively influence effectiveness. The study further proved that beside non-functional attributes, functional attributes also play an important role in increasing the user experience in the m-commerce context and thereby contribute to improving the usability of m-commerce systems and the continuance intention. The overall results of the study can be used in developing strategies for m-commerce application developers to design applications that can satisfy users and help them complete their tasks correctly and efficiently.

Towards enhanced monitoring framework with smart predictions

Abstract Context: Predicting security and trust vulnerabilities and issues is crucial for IoT interconnected systems and ecosystems, especially when integrating new, third-party or open-source components. Objective: One way to ensure timely predictions is by using a smart monitoring framework to continuously verify functional and non-functional property violations during the executions of the systems and their components. Method: This paper presents a set of guidelines for the Smart Monitoring Framework definition and its application process. Results and Conclusion: The paper provides the reference architecture of the Smart Monitoring Framework and its possible implementation to promptly detect suspicious behavior or property violations. The paper also illustrates how the provided implementation satisfies the defined guidelines by design.

Predicting Nonfunctional Requirement Violations in Autonomous Systems

Autonomous systems are often used in applications where environmental and internal changes may lead to requirement violations. Adapting to these changes proactively, i.e., before the violations occur, is preferable to recovering from the failures that may be caused by such violations. However, proactive adaptation needs methods for predicting requirement violations timely, accurately, and with acceptable overheads. To address this need, we present a method that allows autonomous systems to predict violations of performance, dependability and other nonfunctional requirements, and therefore take preventative measures to avoid or otherwise mitigate them. Our method for pre dicting these autonomou s sys t em disrupti o ns (PRESTO) comprises a design time stage and a run-time stage. At design-time, we use parametric model checking to obtain algebraic expressions that formalise the relationships between the nonfunctional properties of the requirements of interest (e.g., reliability, response time, and energy use) and the parameters of the system and its environment. At run-time, we predict future changes in these parameters by applying piece-wise linear regression to online data obtained through monitoring, and we use the algebraic expressions to predict the impact of these changes on the system requirements. We demonstrate the application of PRESTO through simulation in case studies from two different domains.

Feature causality

The detection and understanding of reasons for defects and inadvertent behavior in software is challenging due to its ever increasing complexity. One major aspect contributing to this complexity is the multitude of features a user might select from in configurable systems. In this article, we tackle this challenge by introducing the notion of feature causality that identifies features and their interactions which are the reasons for a system showing certain functional and non-functional properties seen as effects. Feature causality operates at the level of system configurations and is based on counterfactual reasoning, inspired by the seminal definition of actual causality by Halpern and Pearl.Towards turning feature causality into meaningful explanations for the reasons why an effect emerges, we present various explication methods, e.g., by cause–effect covers, quantifications of causal impacts based on notions like responsibility and blame, causal reasoning with uncertainty, and feature interactions. Through a close connection of feature causality to prime implicants, we derive algorithms to effectively compute feature causes and causal explications. By means of an evaluation on a wide range of configurable software systems, including community benchmarks and real-world systems, we demonstrate the feasibility of our approach: We illustrate how our notion of causality facilitates to identify root causes, estimate the impact of features on effect properties, and detect feature interactions.

Software engineering practices for machine learning — Adoption, effects, and team assessment

Machine learning (ML) is extensively used in production-ready applications, calling for mature engineering techniques to ensure robust development, deployment and maintenance. Given the potential negative impact machine learning (ML) can have on people, society or the environment, engineering techniques that can ensure robustness against technical errors and adversarial attacks are of considerable importance. In this work, we investigate how teams of experts develop, deploy and maintain software with ML components. Moreover, we link what teams do to the effects they aim to achieve and provide means for improvement. Towards this goal, we performed a mixed-methods study with a sequential exploratory strategy. First, we performed a systematic literature review through which we mined both academic and grey literature, and compiled a catalogue of engineering practices for ML. Second, we validated this catalogue using a large-scale survey, which measured the degree of adoption of the practices and their perceived effects. Third, we ran validation interviews with practitioners to add depth to the survey results. The catalogue covers a broad range of practices for engineering software systems with ML components and for ensuring non-functional properties that fall under the umbrella of trustworthy ML, such as fairness, security or accountability. Here, we present the results of our study, which indicate, for example, that larger and more experienced teams tend to adopt more practices, but that trustworthiness practices tend to be neglected. Moreover, we show that the effects measured in our survey, such as team agility or accountability, can be predicted quite accurately from groups of practices. This allowed us to contrast the importance of the practices for these effects as well as adoption rates, revealing, for example, that widely adopted practices are, in reality, less important with respect to some effects. For instance, writing reusable scripts for data cleaning and merging is highly adopted, but has a limited impact on reproducibility. Overall, our study provides a quantitative assessment of ML engineering practices and their impact on desirable properties of software with ML components, by which we open multiple avenues for improving the adoption of useful practices.Editor’s note: Open Science material was validated by the Journal of Systems and Software Open Science Board.

A hybrid approach: Uncertain configurable QoT-IoT composition based on fuzzy logic and genetic algorithm

The combination of Quality of Thing (QoT) with Internet of Things (IoT) systems can be challenging because of the vast number of connected devices, diverse types of applications and services, and varying network conditions. During the process of composing these Things, heterogeneity arises as an uncertainty. Hence, uncertainty and imprecision emerge as a consequence of the plethora of things as well as the variety of the composition paths. One way to address these challenges is through the use of fuzzy logic to mimic uncertainty and imprecision modeling and genetic algorithm to find the optimal path. As a result, we propose a model for the Thing behaviour based on QoT non-functional properties. As well as we propose a hybrid approach for modeling the uncertainty of the configurable composition based on fuzzy logic and genetic algorithm. Our approach helps to ensure that IoT applications and services receive the resources they need to function effectively, even in the presence of varying network conditions and changing demands.

Comparing perfomance abstractions for collective adaptive systems

Non-functional properties of collective adaptive systems (CAS) are of paramount relevance practically in any application. This paper compares two recently proposed approaches to quantitative modelling that exploit different system abstractions: the first is based on generalised stochastic Petri nets, and the second is based on queueing networks. Through a case study involving autonomous robots, we analyse and discuss the relative merits of the approaches. This is done by considering three scenarios which differ on the architecture used to coordinate the distributed components. Our experimental results assess a high accuracy when comparing model-based performance analysis results derived from two different quantitative abstractions for CAS.

Automatic Synthesis of FSMs for Enforcing Non-functional Requirements on MPSoCs Using Multi-objective Evolutionary Algorithms

Embedded system applications often require guarantees regarding non-functional properties when executed on a given MPSoC platform. Examples of such requirements include real-time, energy, or safety properties on corresponding programs. One option to implement the enforcement of such requirements is by a reactive control loop, where an enforcer decides based on a system response (feedback) how to control the system, e.g., by adapting the number of cores allocated to a program or by scaling the voltage/frequency mode of involved processors. Typically, a violation of a requirement must either never happen in case of strict enforcement, or only happen temporally (in case of so-called loose enforcement). However, it is a challenge to design enforcers for which it is possible to give formal guarantees with respect to requirements, especially in the presence of typically largely varying environmental input (workload) per execution. Technically, an enforcement strategy can be formally modeled by a finite state machine (FSM) and the uncertain environment determining the workload by a discrete-time Markov chain. It has been shown in previous work that this formalization allows the formal verification of temporal properties (verification goals) regarding the fulfillment of requirements for a given enforcement strategy. In this article, we consider the so-far-unsolved problem of design space exploration and automatic synthesis of enforcement automata that maximize a number of deterministic and probabilistic verification goals formulated on a given set of non-functional requirements. For the design space exploration (DSE), an approach based on multi-objective evolutionary algorithms is proposed in which enforcement automata are encoded as genes of states and state transition conditions. For each individual, the verification goals are evaluated using probabilistic model checking. At the end, the DSE returns a set of efficient FSMs in terms of probabilities of meeting given requirements. As experimental results, we present three use cases while considering requirements on latency and energy consumption.

A Petri net oriented approach for advanced building energy management systems

Sustainability is one of the main goals to pursue in several aspects of everyday life; the recent energy shortage and the price raise worsen this problem, especially in the management of energy in buildings. As the Internet of Things (IoT) is an assessed computing paradigm able to capture meaningful data from the field and send them to cloud infrastructures, other approaches are also enabled, namely model-based approaches. These methods can be used to predict functional and non-functional properties of Building Energy Management Systems (BEMS) before setting up them. This paper aims at bridging the gap between model-based approaches and physical realizations of sensing and small computing devices. Through an integrated approach, able to exploit the power of different dialects of Petri Nets, this paper proposes a methodology for the early evaluation of BEMS properties as well as the automatic generation of IoT controllers.

Re-Scheduling IoT Services in Edge Networks

With the explosive growth of the Internet of Things (IoT) devices deployed in edge networks, the functionalities of IoT devices are typically encapsulated as IoT services, and user requests can be achieved through the composition of data and/or computation-intensive IoT services. Considering the prediction-uncertainty of forthcoming requests, certain IoT services may (i) not be hosted currently by appropriate IoT devices, or (ii) such an IoT service exists, but its non-functional properties may hardly be satisfied with respect to certain constraints prescribed by requests. To address this challenge, this paper proposes an efficiency-aware service Migration Scheduling (denoted eMS) mechanism in edge networks, in order to migrate IoT services on-demand, and thus, to optimally settle non-satisfiable constraints. Specifically, IoT services are re-scheduled, such that certain IoT services are migrated from their hosting IoT devices to neighboring ones, while minimizing the energy consumption and average delay caused by this service re-scheduling operation. We formulate this service re-scheduling as a multi-objective and multi-constraint optimization problem, which is solved through integrating the greedy algorithm into the fast non-dominated sorting and crowded-comparison operators as the hybrid genetic algorithm (G-NSGA-II). Based on real-life datasets provided by an oil pipeline monitoring project, extensive experiments are conducted, and evaluation results show that our eMS is promising in reducing the energy consumption and average delay of service re-scheduling in comparison with the state-of-art’s techniques.

Performance Rubrics for Robustness Evaluation of Web Mutation Operators

Web Applications are the predominant medium for not only business enterprises but also for service-based sector to establish and continue their online presence. However, the robustness of web application is mandatory in seamless interaction with customers for achieving sustainable business. Intruders and unethical hackers keep trying to gain unauthentic access to the web applications and hence it is more necessary for the web application to be resistant against any such attacks. The strength of a web application is indirectly responsible for gaining customer confidence leading to repeat business as well as attracting new customers for profitable longer run. Once the web application gains credibility it is bound to run successfully. In the current work, an attempt has been made to assess the robustness of mutation operators used to test web applications is made. A few rubrics have been proposed to ascertain the strength of projected mutation operators verified on some sample open-source web applications. The functional attributes of a web application are the functionalities offered by the web application. The non-functional attributes of a typical web application are security, performance, availability. Here, web applications are challenged against the afore mentioned non-functional attributes using rubrics like uniformity, uniqueness, reliability, unpredictability, and entropy. A comprehensive analysis has been made for the robustness of the projected web operators against the designed and formulated rubrics.

FPGA Design of Transposed Convolutions for Deep Learning Using High-Level Synthesis

Deep Learning (DL) is pervasive across a wide variety of domains. Convolutional Neural Networks (CNNs) are often used for image processing DL applications. Modern CNN models are growing to meet the needs of more sophisticated tasks, e.g. using Transposed Convolutions (TCONVs) for image decompression and image generation. Such state-of-the-art DL models often target GPU-based high-performance architectures, due to the high computational and hardware resource needs of TCONV layers. To avoid prohibitive GPU energy costs, CNNs are increasingly deployed to decentralized embedded autonomous devices, such as Field Programmable Gate Arrays (FPGAs). However, this poses challenges for designing efficient hardware implementations of TCONV layers. This paper presents a parameterized design and implementation of a new TCONV module, which is synthesizable onto FPGAs. It is implemented using the High-Level Synthesis (HLS), through a C++ template to parameterize its functional and non-functional properties. These parameters allow kernel sizes, image sizes, quantization and parallelism to be varied by users. With a systematic exploration in this design space, we find an optimal instance of this TCONV module that achieves 6.25 Giga Outputs per Second (Gout/s) using just 1.53 W of power. We then use our TCONV layer in two neural networks for image decompression and image generation. Image decompression achieves a speed throughput of more than 30K frames-per-second (fps) using only the 16% of resources on average, image generation achieves an energy efficiency of 324 fps/W and outperforms comparable state-of-the-art models by at least 7.3×.

Towards smart contract distributed directory based on the uniform description language

A Smart Contract (SC) is a piece of code executed on the blockchain to automatically trigger transactions upon the occurrence of predefined events. Due to the intrinsic features regarding traceability and data immutability, many companies started using blockchain Smart Contracts to perform collaborative processes. Despite their promising features, there is a lack of Smart Contacts management platforms that enable blockchain participants to describe and publish their smart contacts or “search and match” already deployed ones. In this paper, a new Distributed Smart Directory (DSD) where providers can publish their SCs description is proposed. The SCs descriptions include metadata covering functional, and non-functional properties of the SC. Hence, users can find SCs according to their non-functional preferences, needs, and constraints. The proposed DSD is an extension of the ebXML directory. It was fully implemented on-chain. The SCs descriptions are generated based on the Uniform Description language for SC (UDL-SC). The proposed solution is implemented on the Ethereum blockchain. It was then tested and evaluated.

Verification Column

Many verification problems can be formulated as a language inclusion problem where the task is to decide whether the language of the system model (given by the runs of the system) is contained in the language induced by a logical specification. The corresponding verification problem for non-functional properties is the quantitative inclusion problem where both the system model and the specification are given by weighted automata and the task is to decide whether the weight for each input string in the automaton for the system is less than that in the automaton for the specification. Here, the weight of a string is defined as an aggregate value of the weights assigned to the states and/or transitions in that string. So, the essential task of the quantitative inclusion problem is to compare the aggregate weight of two input strings. Comparator automata yield an elegant automata-based approach to reason about multiple instances of the quantitative inclusion problem and other algorithmic problems for weighted structures. Speaking roughly, they are defined as automata that take as input two infinite weighted sequences and relate their aggregate values. Among others, they have used to introduce generic algorithms for quantitative inclusion problems and other verification problems for weighted system models. The article by Sugumal Bansal presents an introduction to comparator automata and a summary of their application areas.

Many-objective optimization of non-functional attributes based on refactoring of software models

Software quality estimation is a challenging and time-consuming activity, and models are crucial to face the complexity of such activity on modern software applications. In this context, software refactoring is a crucial activity within development life-cycles where requirements and functionalities rapidly evolve. One main challenge is that the improvement of distinctive quality attributes may require contrasting refactoring actions on software, as for trade-off between performance and reliability (or other non-functional attributes). In such cases, multi-objective optimization can provide the designer with a wider view on these trade-offs and, consequently, can lead to identify suitable refactoring actions that take into account independent or even competing objectives. In this paper, we present an approach that exploits the NSGA-II as the genetic algorithm to search optimal Pareto frontiers for software refactoring while considering many objectives. We consider performance and reliability variations of a model alternative with respect to an initial model, the amount of performance antipatterns detected on the model alternative, and the architectural distance, which quantifies the effort to obtain a model alternative from the initial one. We applied our approach on two case studies: a Train Ticket Booking Service, and CoCoME. We observed that our approach is able to improve performance (by up to 42%) while preserving or even improving the reliability (by up to 32%) of generated model alternatives. We also observed that there exists an order of preference of refactoring actions among model alternatives. Based on our analysis, we can state that performance antipatterns confirmed their ability to improve performance of a subject model in the context of many-objective optimization. In addition, the metric that we adopted for the architectural distance seems to be suitable for estimating the refactoring effort.