Complex Software-intensive Systems Research Articles

Report on the 11th ACM/IEEE International Workshop on Software Engineering for Systems-of-Systems and Software Ecosystems - SESoS@ICSE 2023

This article reports on the results of the 11th ACM/IEEE International Workshop on Software Engineering for Systems-of-Systems and Software Ecosystems (SESoS 2023) in which researchers and practitioners discussed ideas and experiences on the research and practice for the development and evolution of complex softwareintensive systems, more specifically systems-of-systems (SoS) and software ecosystems (SECO). SESoS 2023 was co-located with the 45th IEEE/ACM International Conference on Software Engineering (ICSE 2023). After a decade running this workshop, the SESoS community is advancing on how to cope with the different dimensions that should be considered in the engineering of those classes of systems (i.e. technological, organizational, and social). In addition, benchmarks for conducting research on the areas as well as approaches for investigating emerging domains (smart ecosystems) and non-functional requirements on those systems were also pointed out as relevant challenges.

Boosting the visibility of services in microservice architecture

AbstractMonolithic software architectures are no longer sufficient for the highly complex software-intensive systems, which modern society depends on. Service Oriented Architecture (SOA) surpassed monolithic architecture due to its reusability, platform independency, ease of maintenance, and scalability. Recent SOA implementations made use of cloud-native architectural approaches such as microservice architecture, which has resulted in a new challenge: the discovery difficulties of services. One way to dynamically discover and route traffic to service instances is to use a service discovery tool to locate the Internet Protocol (IP) address and port number of a microservice. In the event that replicated microservice instances are found to provide the same function, it is crucial to select the right microservice that provides the best overall experience for the end-user. Parameters including success rate, efficiency, delay time, and response time play a vital role in establishing a microservice’s Quality of Service (QoS). These assessments can be performed by means of a live health-check service, or, alternatively, by making a prediction of the current state of affairs with the application of machine learning-based approaches. In this research, we evaluate the performance of several classification algorithms for estimating the quality of microservices using the QWS dataset containing traffic data of 2505 microservices. Our research also analyzed the boosting algorithms, namely Gradient Boost, XGBoost, LightGBM, and CatBoost to improve the overall performance. We utilized parameter optimization techniques, namely Grid Search, Random Search, Bayes Search, Halvin Grid Search, and Halvin Random Search to fine-tune the hyperparameters of our classifier models. Experimental results demonstrated that the CatBoost algorithm achieved the highest level of accuracy (90.42%) in predicting microservice quality.

Reporting on a Decade of Promoting Software Engineering for Systems-of-Systems and Software Ecosystems

Systems-of-systems (SoS) have become increasingly complex and frequently used in highly distributed, dynamic, and open environments. SoS refer to evolving software systems in which constituent systems (themselves systems in their own right) cooperatively work to fulfill specific, complex missions, facing Software Engineering researchers and practitioners with substantial challenges. Software ecosystems (SECO) have also become an important research topic, addressing social issues along with technical aspects of software development. SoS and SECO are closely related and naturally distributed in development teams. In both fields, distributed resources aim at reducing cost and reaching new Information Technology markets. Software Engineering-related problems and challenges are amplified and become more critical when SoS and SECO are analyzed together. The 10th IEEE/ACM International Workshop on Software Engineering for Systems-of-Systems and Software Ecosystems (SESoS 2022) celebrates a decade of promoting a forum to exchange ideas and experiences, analyze research and development issues, discuss promising solutions, and propose theoretical foundations for the development and evolution of complex software-intensive systems, inspiring visions for the future of Software Engineering for SoS and SECO, as well as paving the way for a more structured community effort. Based on the community feedback, topics of interest should address systems thinking and complexity of those types of systems. As such, the SESoS community is reflecting on how the different dimensions of those types of systems - technological, organizational, and social - should be addressed in research and practice.

MADneSs: A Multi-Layer Anomaly Detection Framework for Complex Dynamic Systems

Anomaly detection can infer the presence of errors without observing the target services, but detecting variations in the observable parts of the system on which the services reside. This is a promising technique in complex software-intensive systems, because either instrumenting the services' internals is exceedingly time-consuming, or encapsulation makes them not accessible. Unfortunately, in such systems anomaly detection is often ineffective due to their dynamicity, which implies changes in the services or their expected workload. Here we present our approach to enhance the efficacy of anomaly detection in complex, dynamic software-intensive systems. After discussing the related challenges, we present MADneSs, an anomaly detection framework tailored for the above systems that includes an adaptive multi-layer monitoring module. Monitored data are then processed by the anomaly detector, which adapts its parameters depending on the current system behavior. An anomaly alert is provided if the analysis conducted by the anomaly detector identify unexpected trends in the data. MADneSs is evaluated through an experimental campaign on two service-oriented architectures; software faults are injected in the application layer, and detected through monitoring of underlying system layers. Lastly, we quantitatively and qualitatively discuss our results with respect to state-of-the-art solutions, highlighting the key contributions of MADneSs.

AMPL: aspect multiple product lines

The Software Product Line approach is well-known for the advantages it brings to the software-development field. However, single SPLs had been found to be inadequate in managing the variability of large scale and complex software-intensive systems. For this reason, although the Multiple Software Product Lines have been proposed as another solution to tame the complexity of SPLs, the adoption of an MPL has given rise to several challenges such as facilitating the reuse between SPLs, the complexity of the model as well as distributed-derivation. In this paper, we have presented a new methodology for MPLs engineering, which is called the AMPL. Our approach had helped the reuse management of MPLs through the two key concepts of separation of concerns and partial derivation. The various steps of the AMPL approach are illustrated in this paper and were validated in the context of an e-Government MPL. Apart from the systemization of reuse within the MPLs as well as in the structuring of the MPL model, our approach on the early integration of the SPLs during the development process had also been proven to be effective in the prevention of the distributed-derivation issue.

Developing and evolving a DSL-based approach for runtime monitoring of systems of systems

Complex software-intensive systems are often described as systems of systems (SoS) due to their heterogeneous architectural elements. As SoS behavior is often only understandable during operation, runtime monitoring is needed to detect deviations from requirements. Today, while diverse monitoring approaches exist, most do not provide what is needed to monitor SoS, e.g., support for dynamically defining and deploying diverse checks across multiple systems. In this paper we report on our experiences of developing, applying, and evolving an approach for monitoring an SoS in the domain of industrial automation software, that is based on a domain-specific language (DSL). We first describe our initial approach to dynamically define and check constraints in SoS at runtime and then motivate and describe its evolution based on requirements elicited in an industry collaboration project. We furthermore describe solutions we have developed to support the evolution of our approach, i.e., a code generation approach and a framework to automate testing the DSL after changes. We evaluate the expressiveness and scalability of our new DSL-based approach using an industrial SoS. We also discuss lessons we learned. Our results show that while developing a DSL-based approach is a good solution to support industrial users, one must prepare the approach for evolution, by making it extensible and adaptable to future scenarios. Particularly, support for automated (re-)generation of tools and code after changes and automated testing are essential.

Characterization of trade-off preferences between non-functional properties

Efficient design and evolution of complex software intensive systems rely on the ability to make informed decisions as early as possible in the life cycle. Such informed decisions should take both the intended functional and non-functional properties into account. Especially regarding the latter, it is both necessary to be able to predict properties and to prioritize them according to well-defined criteria. In this paper we focus on the latter problem, that is to say how to make trade-offs between non-functional properties of software intensive systems. We provide an approach based on the elicitation of utility functions from stake-holders and subsequent checks for consistency among these functions. The approach is exploitable through an easy-to-use GUI, which is also presented. Moreover, we describe the setup and the outcome of our two-fold validation based on exploratory elicitations with students and practitioners.

Guest Editorial

Guest Editorial

Early Concept Development and Safety Analysis of Future Transportation Systems

As transportation systems become increasingly complex and the roles of human operators and autonomous software continue to evolve, traditional safety-related analytical methods are becoming inadequate. Traditional hazard analysis tools are based on an accident causality model that does not capture many of the complex behaviors found in modern engineered systems. Additionally, these traditional approaches are most effective during the late stages of system development, when detailed design information is available. However, system safety cannot be cost-effectively assured by discovering problems at these late stages and adding expensive updates to the design. Rather, safety should be designed into complex intelligent transportation systems from their very conception, which can be achieved by integrating powerful hazard analysis techniques into the general systems engineering process. The primary barrier to achieving this objective is the lack of effectiveness of the existing analytical tools during early concept development. This paper introduces a new technique, which is based on a systems- and control-theoretic model of accident causality that can capture behaviors that are prevalent in these complex software-intensive systems. The goals are to 1) develop rigorous systematic tools for the analysis of future concepts to identify potentially hazardous scenarios and undocumented assumptions and to 2) extend these tools to assist stakeholders in the development of concepts using a safety-driven approach. Current work focuses on air transportation, but future goals of this research are to extend to and generalize all modes of transportation.

Incorporating architecture-based self-adaptation into an adaptive industrial software system

Complex software-intensive systems are increasingly relied upon for all kinds of activities in society, leading to the requirement that these systems should be resilient to changes that may occur to the system, its environment, or its goals. Traditionally, resilience has been achieved either through: (i) low-level mechanisms embedded in the implementation (e.g., exception handling, timeouts, redundancies), which are unable to detect subtle but important anomalies (e.g., progressive performance degradation); or (ii) human oversight, which is costly and unreliable. Architecture-based self-adaptation (ABSA) is regarded as a promising approach to improve the resilience and reduce the development/operation costs of such systems. Although researchers have illustrated the benefits of ABSA through a number of small-scale case studies, it remains to be seen whether ABSA is truly effective in handling changes at run-time in industrial-scale systems. In this paper, we report on our experience applying an ABSA framework (Rainbow) to a large-scale commercial software system, called Data Acquisition and Control Service (DCAS), which is used to monitor and manage highly populated networks of devices in renewable energy production plants. In the approach followed, we have replaced some of the existing adaptive mechanisms embedded in DCAS by those advocated by ABSA proponents. This has allowed us to assess the development costs associated with the reengineering of adaptive mechanisms when using an ABSA solution, and to make effective comparisons, in terms of operational performance, between a baseline industrial system and one that uses ABSA. Our results show that using the ABSA concepts as embodied in Rainbow enabled an independent team of developers to: (i) effectively implement the adaptation behavior required from such industrial systems; and (ii) obtain important benefits in terms of maintainability and extensibility of adaptation mechanisms.

Assessing the benefits of search-based approaches when designing self-adaptive systems: a controlled experiment

The well-orchestrated use of distilled experience, domain-specific knowledge, and well-informed trade-off decisions is imperative if we are to design effective architectures for complex software-intensive systems. In particular, designing modern self-adaptive systems requires intricate decision-making over a remarkably complex problem space and a vast array of solution mechanisms. Nowadays, a large number of approaches tackle the issue of endowing software systems with self-adaptive behavior from different perspectives and under diverse assumptions, making it harder for architects to make judicious decisions about design alternatives and quality attributes trade-offs. It has currently been claimed that search-based software design approaches may improve the quality of resulting artifacts and the productivity of design processes, as a consequence of promoting a more comprehensive and systematic representation of design knowledge and preventing design bias and false intuition. To the best of our knowledge, no empirical studies have been performed to provide sound evidence of such claim in the self-adaptive systems domain. This paper reports the results of a quasi-experiment performed with 24 students of a graduate program in Distributed and Ubiquitous Computing. The experiment evaluated the design of self-adaptive systems using a search-based approach proposed by us, in contrast to the use of a non-automated approach based on architectural styles catalogs. The goal was to investigate to which extent the adoption of search-based design approaches impacts on the effectiveness and complexity of resulting architectures. In addition, we also analyzed the approach’s potential for leveraging the acquisition of distilled design knowledge. Our findings show that search-based approaches can improve the effectiveness of resulting self-adaptive systems architectures and reduce their design complexity. We found no evidence regarding the approach’s potential for leveraging the acquisition of distilled design knowledge by novice software architects. This study contributes to reveal empirical evidence on the benefits of search-based approaches when designing self-adaptive systems architectures. The results presented herein increase our belief that the systematic representation of distilled design knowledge and the adoption of search-based design approaches indeed lead to improved architectures.

Query-driven soft traceability links for models

Model repositories play a central role in the model driven development of complex software-intensive systems by offering means to persist and manipulate models obtained from heterogeneous languages and tools. Complex models can be assembled by interconnecting model fragments by hard links, i.e., regular references, where the target end points to external resources using storage-specific identifiers. This approach, in certain application scenarios, may prove to be a too rigid and error prone way of interlinking models. As a flexible alternative, we propose to combine derived features with advanced incremental model queries as means for soft interlinking of model elements residing in different model resources. These soft links can be calculated on-demand with graceful handling for temporarily unresolved references. In the background, the links are maintained efficiently and flexibly by using incremental model query evaluation. The approach is applicable to modeling environments or even property graphs for representing query results as first-class relations, which also allows the chaining of soft links that is useful for modular applications. The approach is evaluated using the Eclipse Modeling Framework (EMF) and EMF-IncQuery in two complex industrial case studies. The first case study is motivated by a knowledge management project from the financial domain, involving a complex interlinked structure of concept and business process models. The second case study is set in the avionics domain with strict traceability requirements enforced by certification standards (DO-178b). It consists of multiple domain models describing the allocation scenario of software functions to hardware components.

Model-based cross-layer monitoring and adaptation of multilayer systems

With the vision of “Internet as a computer”, complex software-intensive systems running on the Internet, or the “Internetwares”, can be also divided into multiple layers. Each layer has a different focus, implementation technique, and stakeholders. Monitoring and adaptation of such multilayer systems are challenging, because the mismatches and adaptations are interrelated across the layers. This interrelation makes it difficult to find out: 1) When a system change causes mismatches in one layer, how to identify all the cascaded mismatches on the other layers? 2) When an adaptation is performed at one layer, how to find out all the complementary adaptations required in other layers? This paper presents a model-based approach towards cross-layer monitoring and adaptation of multilayer systems. We provide standard meta-modelling languages for system experts to specify the concepts and constraints separately for each layer, as well as the relations among the concepts from different layers. Within each individual layer, we use run-time models to represent the system state specific to this layer, monitor the systems by evaluating model changes according to specified constraints, and support manual or semi-automated adaption by modifying the models. When a change happens in the run-time model for one layer, either caused by system changes or by the adaptation, we synchronize the models for other layers to identify cascaded mismatches and complementary adaptations across the layers. We illustrate the approach on a simulated crisis management system, and are using it on a number of ongoing projects.

The evolution of modeling research challenges

In 2007 ICSE hosted a track called “Future of SoftwareDevelopment”(FOSD).Wewereinvitedtowriteandpresenta paper on the future of modeling for the track. The result-ing paper [1] described the state of modeling research, iden-tified some major challenges and proposed a research roadmap.Partsofthisroadmaparecurrentlybeingexplored,andprogresshasbeenmadeinaddressingsomeofthechallengeswe identified. However, there is still significant research “tobedone”withrespecttothechallengesoutlinedinthatpaper.Itisnotourintenttodiscusstheprogressthecommunityhasmadewithrespecttotheroadmapinthiseditorial(ourapolo-gies for deflating expectations in this regard; an editorial issimplynottheplaceforsuchdiscussions).Rather,wewouldliketousethiseditorialtostimulatediscussionsaroundsomeof the challenges that have arisen since we wrote that paper.Before we get into identifying specific challenges, letus first take a step back and ask “What is the grand chal-lenge addressed by researchers in the SoSyM community?”The answer has typically been “To significantly reduce thetime, cost, and effort required to develop complex software-intensive systems that meet stringent quality requirementsthrough use of models that are fit-for-purpose.” This is stillthe focus of many research programs, but we are also see-ing a broadening of the grand challenge to include not onlydevelopment problems, but problems that occur throughoutthe lifetime of a complex system, from conception to retire-ment. For example, in the FOSD paper we made referenceto emerging work on the use of models to manage software

Construct Aspectual Models from Requirement Documents for Model-driven Development of Automotive Software

In the model-driven development of complex software-intensive systems, it is the first class issue how to capture the software models in the very beginning of development. In this paper, we focus on the domain of automotive software, explore how to capture software models from requirement documents. To this end, we mainly investigated these three closely related problems: what requirement information should be elicited from requirement documents; how to organize these information as aspectual models; and how to integrate the aspectual models together to form a complete specification of requirements. This work will help software analyzers to decompose complicated requirement documents into separated concerns, and organize essential requirement information as rigorous models, which will both facilitate simulation and verification of requirements and set up the starting point for the model-driven development.

A framework for evolution of modelling languages

In model-driven engineering, evolution is inevitable over the course of the complete life cycle of complex software-intensive systems and more importantly of entire product families. Not only instance models, but also entire modelling languages are subject to change. This is in particular true for domain-specific languages, whose language constructs are tightly coupled to an application domain. The most popular approach to evolution in the modelling domain is a manual process, with tedious and error-prone migration of artefacts such as instance models as a result. This paper provides a taxonomy for evolution of modelling languages and discusses the different evolution scenarios for various kinds of modelling artefacts, such as instance models, meta-models, and transformation models. Subsequently, the consequences of evolution and the required remedial actions are decomposed into primitive scenarios such that all possible evolutions can be covered exhaustively. These primitives are then used in a high-level framework for the evolution of modelling languages. We suggest that our structured approach enables the design of (semi-)automatic modelling language evolution solutions.

Measuring Architectural Complexity

Without refactoring, complex software-intensive systems become increasingly irregular and thus increasingly chaotic over time. We can understand complex software systems only when they're nearly decomposable and hierarchic. One measure the author uses is lines of source code: the greater the SLOC, the more inertia to change the system will have, the more people it will take to keep it fed, the more stakeholders who will be crawling all over it. The author describes the more complex measures he uses; these are tuned to Philippe Kruchten's 4+1 view model of architecture. He also counts the number of identifiable design patterns at work. These metrics can generally be gathered automatically via clever mining of configuration management and testing data.

THE ART AND SCIENCE OF SOFTWARE ARCHITECTURE

Experience in all aspects of software engineering has confirmed the pivotal role of focusing on architectural concerns in the development of complex software-intensive systems. Consequently, the past 20 years has seen significant investments in the theory and practice of software architecture. However, architectural deficiencies are frequently cited as a key factor in the shortcomings and failures that lead to unpredictable delivery of complex operational systems. Here, we consider the art and science of software architecture: we explore the current state of software architecture, identify key architectural trends, and directions in academia and industry, and highlight some of the architectural research challenges which need to be addressed. The paper proposes a detailed agenda of research activities to be carried out by a partnership between academia and industry. While challenges exist in many domains, for this paper we draw examples from one area of particular concern: safety-critical systems.

Teaching disciplined software development

Discipline is an essential prerequisite for the development of large and complex software-intensive systems. However, discipline is also important on the level of individual development activities. A major challenge for teaching disciplined software development is to enable students to experience the benefits of discipline and to overcome the gap between real professional scenarios and scenarios used in software engineering university courses. Students often do not have the chance to internalize what disciplined software development means at both the individual and collaborative level. Therefore, students often feel overwhelmed by the complexity of disciplined development and later on tend to avoid applying the underlying principles. The Personal Software Process (PSP) and the Team Software Process (TSP) are tools designed to help software engineers control, manage, and improve the way they work at both the individual and collaborative level. Both tools have been considered effective means for introducing discipline into the conscience of professional developers. In this paper, we address the meaning of disciplined software development, its benefits, and the challenges of teaching it. We present a quantitative study that demonstrates the benefits of disciplined software development on the individual level and provides further experience and recommendations with PSP and TSP as teaching tools.

The Open Software Intensive System Strategy: An Integrative Approach for Building Software Intensive Systems

Full spectrum dominance in markets, timely reaction to evolving business and customer needs, and system and process flexibility and cost effectiveness can only be achieved through steady infusion of affordable new technology and continuing modernization of systems and equipment. The open system strategy is an effective business and technical strategy to maintain the superiority of modern complex systems, including software intensive systems within constantly changing system requirements, growing resource constraints, and unprecedented rate of technological change. By following an open system strategy in design and development of complex software intensive systems, an organization will be in a position to continually leverage the investments made in commercial products, practices, and technologies by the global market place. Moreover, the organizations can also field superior system capabilities more quickly and more affordably. This paper proposes an open systems based strategy for developing complex software intensive systems. The proposed strategy can enhance adaptation of software intensive systems to the unprecedented changes brought about by the contemporary technical and business paradigm shifts.