Systems Engineering Community Research Articles

Early Validation and Verification of System Behaviour in Model-based Systems Engineering: A Systematic Literature Review

In the Systems Engineering (SE) domain there has been a paradigm shift from document-based to model-based system development artefacts; in fact, new methodologies are emerging to meet the increasing complexity of current systems and the corresponding growing need of digital workflows. In this regard, Model-Based Systems Engineering (MBSE) is considered as a key enabler by many central players of the SE community. MBSE has reached an adequate level of maturity, and there exist documented success stories in its adoption in industry. In particular, one significant benefit of utilising MBSE when compared to the traditional manual and document-centric workflows is that models are available from early phases of systems development; these enable a multitude of analyses prior any implementation effort together with other relevant capabilities, like the automation of development tasks. Nonetheless, it is noticeable there is a lack of a common understanding for how formal analyses for the verification and validation (V&V) of systems behaviour, specifically in the early phases of development, could be placed in an MBSE setting. In this article, we report on the planning, execution, and results of a systematic literature review regarding the early V&V of systems behaviour in the context of model-based systems engineering. The review aims to provide a structured representation of the state of the art with respect to motivations, proposed solutions, and limitations. From an initial set of potentially relevant 701 peer-reviewed publications we selected 149 primary studies, which we analysed according to a rigorous data extraction, analysis, and synthesis process. Based on our results, early V&V has usually the goal of checking the quality of a system design to avoid discovering flaws when parts are being concretely realised; SysML is a de facto standard for describing the system under study, while the solutions for the analyses tend to be varied; also V&V analyses tend to target varied properties with a slight predominance of functional concerns, and following the variation mentioned so far the proposed solutions are largely context specific; the proposed approaches are usually presented without explicit limitations, while when limitations are discussed, readiness of the solutions, handling of analyses simplifications/assumptions, and languages/tools integration are among the most frequently mentioned issues. Based on the survey results and the standard SE practices, we discuss how the current state-of-the-art MBSE supports early V&V of systems behaviour with a special focus on industrial adoption and identify relevant challenges to be researched further.

The Effects of Organizational Structure on MBSE Adoption in Industry: Insights from Practitioners

Despite a general increase of use across the industry, Model-Based Systems Engineering (MBSE) continues to be reported as difficult to adopt in organizations. Data related to adoption is largely taken from surveys asking respondents to share what they think are barriers or challenges to adopting MBSE. Consistently, organizational-related factors such as culture and structure are often one of the top reported barriers to adopting MBSE. Despite this, there has been little research exploring how these organizational factors actually affect MBSE adoption. This study attempts to address that void by comparing data on organizational structure characteristics with adoption and implementation metrics. A survey was created and distributed amongst the Systems Engineering community to collect this data. Characteristics of organizational structure that were measured as variables include: Size, Formalization, Centralization, Specialization, Vertical Differentiation, Flexibility, and Interconnectedness. All of these variables except for Specialization were found to have some level of significant correlation with adoption and/or implementation variables as perceived by the respondents. Flexibility and Interconnectedness especially had strong correlations with both adoption and implementation variables. This study serves as an exploratory report of this type of organizational-related data as it relates to MBSE, seeding future work that enables Engineering Managers to derive strategies to adopt and implement MBSE customized to their organization.

Model‐based systems engineering: Evaluating perceived value, metrics, and evidence through literature

AbstractAlthough Model‐Based Systems Engineering (MBSE) is quickly becoming adopted in Systems Engineering (SE), there have not been many studies evaluating literature to determine the perceived value of implementing MBSE. This research first identifies and discusses previous studies on the justification or rejection of MBSE. This study investigates how the SE community perceives the value of MBSE by coding and analyzing positive and negative descriptions of MBSE; perceived benefits and drawbacks of implementing MBSE; and the evidence and metrics used to substantiate and measure each claim about MBSE. From 60 academic sources, this study collected and coded over 2900 claims on MBSE. Our findings determine the most positive attributes of MBSE to be Verification & Validation Capability, Consistency, Reasoning, and Risk & Error Manageability, while the most negative attributes were Approach Understandability, Acceptability, Familiarity, and Approach Complexity. The most‐stated benefits were Reduced Time, Better Communication/Information Sharing, Reduced Costs, and Better Analysis Capability. The most claimed drawbacks were Increased Costs, Increased Time, Increased Effort, and Worsened Capability. A large share of claims (47%) about MBSE was based on author opinions. Most claims (86%) were not substantiated by a metric.

Towards Developing Metrics to Evaluate Digital Engineering

AbstractModel‐based systems engineering (MBSE) is an increasingly accepted practice in the Systems Engineering (SE) community, however, little has been done to empirically show that MBSE provides value. Furthermore, as the industry continues in the direction of digital transformation, MBSE will become a critical component of the larger Digital Engineering (DE) approach. This paper presents a measurement framework for selecting and developing appropriate metrics to assess the value/benefits of MBSE and subsequently DE. Utilizing expected benefits identified in a review of MBSE literature, a causal map was hypothesized to show how expected benefits (potential metrics) influence and relate to each other. This was done in order to systematically determine which benefits would be the most impactful to measure. The hypothesized causal model was presented for feedback to subject‐matter experts from a working group developing the first DE measurement framework. This group is a joint effort with industry, academia, and the USA government to develop DE metric standards. Once the causal map was finalized, a case study was used to partially validate the causal model. Based on the causal map and subsequent analysis, we can recommend the first metrics to be employed for DE/MBSE based on the most influential nodes of the causal model. The potential metric candidates include: system quality, defects, time, rework, ease of making changes, system understanding, Effort, accessibility of information, collaboration, project methods/processes, and use of DE/MBSE tools. We believe a concerted effort across the industry to focus on measuring these variables is the most effective way to establish proof of the value of MBSE and DE.

Metrics for Human-Robot Team Design: A Teamwork Perspective on Evaluation of Human-Robot Teams

Metrics for human-robot teaming should extend to teams consisting of multiple human and robotic agents, and to teams working in complex, dynamic work domains. This work proposes that to comprehensively analyze and evaluate multi-human, multi-robot teams, traditional HRI metrics of performance, and efficiency must be expanded upon to incorporate metrics of teamwork. We develop five distinct metrics to capture both ecological and cognitive aspects of teamwork found to be important in human-automation interaction, inspired by research in the cognitive systems engineering (CSE) community. We demonstrate the application of these metrics in a spacecraft maintenance case study comparing multiple human-robot team architectures. The case study demonstrates that the teamwork metrics capture aspects of human-robot interaction (HRI) not apparent when using only traditional performance and efficiency metrics. The article concludes that the proposed teamwork metrics are complementary to existing metrics in HRI and should be included in the evaluation of human-robot teams.

Fit As a Diagnostic Tool: An Analytic Review of Approaches to Measure Correspondence Between Technical and Organizational Architectures

Abstract Standardized design approaches such as those embodied by concurrent design facilities have many benefits, such as increased efficiency of the design process, but may also have hidden costs. Specifically, when their standardized organizational decomposition is a poor fit for the particular design problem, important design trades might be missed or poor decisions made. Before we can understand how this lack of fit impacts the design process, we must be able to empirically observe and measure it. To that end, this paper identifies measures of “fit” from the literature along with attributes likely to impact design process performance, then evaluates the measures to determine how well the measures can detect and diagnose potential issues. The results provide comparative insights into the capabilities of existing fit measures, and also build guidance for how the systems engineering and design community can use insights from the “fit” literature to inform process improvement.

Multivariable Control Structure Design for Voltage Regulation in Active Distribution Networks

The distributed generation (DG) paradigm has driven the growth of small and medium-scale renewable generation systems located close to demand centers. As a consequence, traditional distribution networks are in a transition towards the so-called Active Distribution Networks (ADNs). However, the intermittent nature of renewable generation causes rapid voltage fluctuations that are difficult to manage and deteriorate power quality. In the literature, the tasks corresponding to the design of voltage control in ADNs are addressed in a heuristic and sequential manner. In this work, a systematic strategy is proposed for the analysis, design and implementation of controllers for voltage regulation in ADNs with DG, based on the Plant Wide Control (PWC) theory of the Process Systems Engineering (PSE) community. This strategy allows us to find control structures that optimize the hardware necessary for its implementation and have good dynamic performance. The general PWC design procedure is formulated as a mixed integer two-level nonlinear programming with cost functions that only depend on the steady state models of the network. To evaluate the proposed methodology, the IEEE 33 distribution network model is used, to which 6 DGs are incorporated. The application of the proposed methodology to identify and evaluate different control structures contributes to the generation of systematic tools to approach the ADN control design process in a comprehensive manner, based on quantitative information.

Importing participatory practices of the socio-environmental systems community to the process system engineering community: An application to supply chain

The Process System Engineering community has an extensive knowledge and skills on supply chain design: from the time dimension (production and flow planning) to the space dimension (geographic position of facilities). Nevertheless, supply chains are also social networks where multiple stakeholders have to collaborate while they have different, and sometimes, diverging objectives. For this reason, having a more realistic model representing collaboration between the various stakeholders involved is necessary and new methods that facilitate the development of a shared representation of the system must be introduced. We propose to import a participatory method, PARDI (Problematic, Actors, Resources, Dynamics and Interactions), from the Socio-Environmental System community to the practices of the Process System Engineering community. Based on this method, we develop a participatory process in order to collect the necessary knowledge on the supply chain and its context. Following this participatory process, we then develop an Agent-Based Model as a simulation and decision making tool to support collective scenario analyses and collectively draw solutions with stakeholders. Our participatory modeling approach necessarily imposes a multi stakeholders vision (within the modeling but also in the result analyses) and therefore the search for a modeling consensus. Thus, it brings a better inclusion of social aspects in problem solving which are usually poorly considered leading to implementation failure sometimes. By comparing our approach with the classic one of the Process Systems Engineering community, we highlight the strengths and weaknesses of both and how complementary they can be. A case study on the already existing supply chain of the chestnut wood in Cévennes area (France) illustrates the capabilities of our participatory methodology. It focuses on the socio-economic model design of the first two steps (forestry activities to harvest) in the supply chain as the latter is locked because of economic and social organisation issues. The objective is to find the best action levers to unlock the resistance that forest plot owners have to remove declining wood from their land.

An Assessment of the Adequacy of Common Definitions of the Concept of System

AbstractIn all areas of scientific study, definitions are used to describe the meaning of terms. Thus, a good set of definitions aids the scientific process by enabling researchers to communicate in a common language. In this regard, the Systems Engineering community has allocated significant effort to understanding the nature and scope of common definitions of a system. However, less attention has been given to formally examining whether these common definitions of a system are adequate. In this paper, we argue that the common definitions of a system are limited in their ability to adequately define a system's boundary. Furthermore, we argue that the common definitions of a system rely on context and prior understanding to communicate the boundary of a system. Finally, by using concepts from philosophy and mathematical logic, we show that the common definitions of a system are nominal in their ability to define a system's boundary.

An interdisciplinary strategy to advance systems engineering theory: The case of abstraction and elaboration

AbstractPrior advances in systems engineering (SE) theory were instrumental in defining the discipline and its tools, but are limited in perspective. The SE community needs new theoretical advances to address its existing and emerging sociotechnical challenges. This communication paper is a product of an NSF/SERC/INCOSE funded workshop on theory building in SE with a focus on the use of abstraction and elaboration. The overarching goals of the workshop were twofold. First, to illustrate the nuances and complexities of the theory building process with an emphasis on developing theory about the SE discipline rather than about a particular system of interest. The second goal was to stimulate a new wave of SE theory by providing a set of frames that could be used for formal theory articulation in follow‐up research. The workshop focused on abstraction and elaboration due to their richness and pervasiveness in SE and investigated the concept in two interrelated contexts: (a) the social coordination of design teams and (b) its strategic use during system design. For each research context, we offer a structured guideline for theorizing, and provide an understanding and framing of some of the generalized abstraction and elaboration uses in SE. Research frames articulated in this workshop could serve as the first step for future research on SE theory, which necessitates hypotheses generation, data collection, validation, and refinement.

110th Anniversary: On the Departure from Heuristics and Simplified Models toward Globally Optimal Design of Process Equipment

Despite the large advances in computational tools attained by the Process Systems Engineering community, industry for the most part still performs basic design of process equipment using trial-and-verification procedures guided by heuristic rules. In this opinion article, we present a discussion on how to depart from the use of these heuristics-based procedures, most step-by-step, sometimes computer-aided. We believe that there are direction changes, some incipient and some in full development already, toward the use of optimization tools for the task. The academic literature is dominated by mixed-integer nonlinear models, solved using different techniques (mostly stochastic or mathematical programming-based). These procedures have practical limitations that have hindered the migration of practitioners away from current heuristics and simplified model-based tools. We discuss these drawbacks and propose solutions. We first show how the use of commercially available discrete values of the design geometrical...

Model‐based systems engineering: Motivation, current status, and research opportunities

AbstractAs systems continue to grow in scale and complexity, the Systems Engineering community has turned to Model‐Based Systems Engineering (MBSE) to manage complexity, maintain consistency, and assure traceability during system development. It is different from “engineering with models,” which has been a common practice in the engineering profession for decades. MBSE is a holistic, systems engineering approach centered on the evolving system model, which serves as the “sole source of truth” about the system. It comprises system specification, design, validation, and configuration management. Even though MBSE is beginning to see a fair amount of use in multiple industries, specific advances are needed on multiple fronts to realize its full benefits. This paper discusses the motivation for MBSE, and its current state of maturity. It presents systems modeling methodologies and the role of ontologies and metamodels in MBSE. It presents model‐based verification and validation (V&V) as an example of MBSE use. An illustrative example of the use of MBSE for design synthesis is presented to demonstrate an important MBSE capability. The paper concludes with a discussion of challenges to widescale adoption and offers promising research directions to fully realize the potential benefits of MBSE.

Digitalization: Opportunity and Challenge for the Business and Information Systems Engineering Community

Digitalization: Opportunity and Challenge for the Business and Information Systems Engineering Community

Data-driven multi-stage scenario tree generation via statistical property and distribution matching

This paper brings systematic methods for scenario tree generation to the attention of the Process Systems Engineering community. We focus on a general, data-driven optimization-based method for generating scenario trees that does not require strict assumptions on the probability distributions of the uncertain parameters. Using as a basis the Moment Matching Problem (MMP), originally proposed by Høyland and Wallace (2001), we propose matching marginal (Empirical) Cumulative Distribution Function information of the uncertain parameters in order to cope with potentially under-specified MMP formulations. The new method gives rise to a Distribution Matching Problem (DMP) that is aided by predictive analytics. We present two approaches for generating multi-stage scenario trees by considering time series modeling and forecasting. The aforementioned techniques are illustrated with a production planning problem with uncertainty in production yield and correlated product demands.

The Agile Enterprise: Systems Engineering Agility at the Enterprise Level

AbstractAgility is all the rage. Across the landscape, businesses request agility as a deliverable, make claims that they are agile, and (some) go so far as to assert they can provide agility to other corporate entities outside of their own sphere of influence. The challenge before us as a Systems Engineering Community is that (Enterprise) Agility, although well documented throughout engineering literature since the early 1990s, is often misinterpreted. Agility means many things to many people, and heretofore, has been used as a synonym for rapid development and delivery. Although speed is a byproduct of agility, speed in and of itself is not robust enough to encompass all that agility provides an organization, its customers, and partners. The goals of this paper are to shed light on agility in context to businesses, especially with Enterprises engaged in Systems Engineering, and provide guidance as to how agility is best achieved regardless of the Enterprise's business or mission objectives.

1.6.1 xFFBD: towards a formal yet functional modeling language for system designers

AbstractAlthough the eFFBD formalism dates back to the 1990s (or even, in a simplified form, the 1950s), it seems that it is still not as much used by the Systems Engineering community as it could. Indeed, eFFBD is a modeling language focusing on functional paradigm i.e. allowing functional and behavioral modeling and reasoning about a system. Currently, it is often confronted or compared to other languages such as SysML for activity modeling (activity diagrams) based on object paradigm. This paper aims to demonstrate the interest and the potential advantages for systems designers, like most of the discipline‐oriented designers to dispose of an enriched (conceptually and semantically) eFFBD modeling language called here xFFBD. This has to be a credible framework for modeling, communicating and reasoning about complex systems. After shortly recalling the history, the key concepts and capabilities of eFFBD, this paper compares eFFBD with other formalisms considered here as relevant for the study, Petri nets and SysML. Several leads are then identified and discussed in order to improve the eFFBD language and to provide a first draft version of xFFBD specification.

7.2.2 An Introduction to Applying Systems Engineering to In‐Service Systems

AbstractThere is emerging consensus within the Systems Engineering (SE) community that, while the principles underpinning SE remain the same across the lifecycle: Some of the issues concerned with sustaining existing systems are more problematic than when realizing new systems; and The existing SE Body of Knowledge and the competences of SE practitioners tend to focus on the issues that are more important when realizing new systems than on the issues that are more important when changing existing systems.A UK working group on Applying SE to In‐Service Systems looked into difficulties in this area and produced a report containing draft guidance and recommendations for further action. These recommendations were recognized by INCOSE and consequently the International In‐Service Systems Working Group (“the ISSWG”) was set up in 2009 with the following objectives: To consider the work of the UK Working Group and extend and adjust it as necessary in order to provide improved advice on the difficulties encountered in practice in applying authoritative guidance on SE, including the INCOSE SE Handbook, to systems which are in service and best current practice in adapting SE guidance to overcome these difficulties; and To consider the current channels by which INCOSE promulgates advice on good SE practice, including the SE Handbook and the SE Book of Knowledge and make recommendations on how additional guidance on performing SE on in‐service systems should be integrated into these channels.

Business and Information Systems Engineering: A Complementary Approach to Information Systems – What We Can Learn from the Past and May Conclude from Present Reflection on the Future

Although both communities share a common object of research, the Business and Information Systems Engineering (BISE) community from the German-speaking countries and the North American Information Systems (NAIS) community have developed quite differently. In our opinion, each community has the opportunity to learn from the other’s strengths to mitigate its own weaknesses. The BISE community promotes connections with industry and draws substantial funding from there. BISE researchers’ topics are attractive to students and ensure the practical relevance of publications. Due to various reasons, numerous BISE researchers struggle with strong contributions to theory, research quality, and publications in top-ranked journals. While this obviously is a strength of the NAIS community, we observe that the NAIS community struggles with its industry connections and enrollment numbers. What the global IS/BISE community needs is a more intense discourse that increases mutual understanding, creates awareness for the need to complement one another, and ensures that this opportunity is seized. Organized along the history of the BISE community’s main publication outlet, this paper offers insights into the community’s ability to fully engage with industry and how this ability was maintained over time. Based thereon, we as BISE insiders would like to give recommendations on how the NAIS community can mitigate some of its weaknesses. These recommendations are intended to complement the valuable hints already provided by NAIS scholars. They also intend to make insights into the traditional strength of the BISE community available when discussing the global IS/BISE community’s future.

Why Affordability Is A Systems Engineering Metric

The Systems Engineering approach to Affordability is equal to CAIV + LCC + TOC + Systems Effectiveness. The purpose of this presentation is to apprise the Systems Engineering community of Affordability as a critical metric in the total organization. Affordability will be defined in terms of the most “bang” for the available bucks from the Government and commercial perspectives. Illustrations of how the design process can affect the affordability and determine the “profit” and maximize sales. The flow of Affordability will define at a top level the definition of Cost As an Independent Variable (CAIV) with program examples and real life scenarios. Affordability is dependent on mission requirements, i.e. quantity, performance, supportability. Life Cycle Cost (LCC) will be defined at a top level and include the elements of LCC with the appropriate process of “How an LCC is developed” along with the customer requirements and models to support. Total Ownership Cost (TOC) is defined and illustrations of various TOC's will be provided at a top level. Systems Effectiveness will be discussed as mission utilities that are inherent in the Systems Engineering processes. Additionally, there is discussion of IPT responsibility, when this process is implemented and other Economic Analysis aspects pertinent to the Affordability Program.

1.3.3 Old Dog, New Tricks

AbstractEnterprises are increasingly adopting a capability based approach to optimise their effectiveness in achieving their outcomes and this paradigm is attracting significant interest amongst the Systems Engineering community. In the UK, the Ministry of Defence (MOD) has adopted Through Life Capability Management (TLCM) as its acquisition model and is transforming itself to more effectively deliver affordable military capability. BAE Systems has been supporting the UK MOD in its implementation of TLCM and early work has identified the essential role of Systems Engineering in defining and addressing the challenges associated with effective capability management.