Behavior-Driven Development Research Articles

AAT4IRS: automated acceptance testing for industrial robotic systems.

Industrial robotic systems (IRS) consist of industrial robots that automate industrial processes. They accurately perform repetitive tasks, replacing or assisting with dangerous jobs like assembly in the automotive and chemical industries. Failures in these systems can be catastrophic, so it is important to ensure their quality and safety before using them. One way to do this is by applying a software testing process to find faults before they become failures. However, software testing in industrial robotic systems has some challenges. These include differences in perspectives on software testing from people with diverse backgrounds, coordinating and collaborating with diverse teams, and performing software testing within the complex integration inherent in industrial environments. In traditional systems, a well-known development process uses simple, structured sentences in English to facilitate communication between project team members and business stakeholders. This process is called behavior-driven development (BDD), and one of its pillars is the use of templates to write user stories, scenarios, and automated acceptance tests. We propose a software testing (ST) approach called automated acceptance testing for industrial robotic systems (AAT4IRS) that uses natural language to write the features and scenarios to be tested. We evaluated our ST approach through a proof-of-concept, performing a pick-and-place process and applying mutation testing to measure its effectiveness. The results show that the test suites implemented using AAT4IRS were highly effective, with 79% of the generated mutants detected, thus instilling confidence in the robustness of our approach.

Modeling contextual goals and detecting context conflicts from BDD-based user stories

Behavior-Driven Development (BDD) user stories are widely used in agile methods for capturing user requirements and acceptance criteria due to their simplicity and clarity. However, the concise structure of BDD-based user stories prevents capturing contextual information and connections between them, which help stakeholders understand requirements and uncover potential issues. The contextual goal models (CGMs) can provide explicit relationships between goals and contexts, but manually constructing models is effort-intensive. This paper proposes an automated approach called BUS2C to model contextual goals and detect context conflicts from BDD-based user stories. The aim is to improve requirements analysis by systematically eliciting goals and dependencies. BUS2C approach involves: (1) mapping BDD-based user story elements to CGMs, and (2) merging related models based on similarity. The context conflict detection algorithm checks if contexts associated with different goals are compatible using natural language metrics. We evaluated BUS2C approach through three experiments. First, we showed the effectiveness in merging scattered CGMs with common features. Second, we demonstrated the capability to construct CGMs from BDD-based user stories quickly, assisting modelers. Our merging method also produced models closer to manually built ones versus without merging. Third, we validated that the context conflict detection can successfully identify inconsistencies in small datasets, enhancing requirements quality. Further validation on larger datasets is needed. In conclusion, this work contributes automated techniques to systematically model and analyze contextual goals from BDD-based user stories. By providing unified goal understanding and detecting issues like conflicts, BUS2C approach aims to improve requirements analysis and support adaptive systems development.

Organizing Graphical User Interface tests from behavior‐driven development as videos to obtain stakeholders' feedback

AbstractDemonstrating software early and responding to feedback is crucial in agile development. However, it is difficult for stakeholders who are not on‐site customers but end users, marketing people, or designers, and so forth to give feedback in an agile development environment. Successful graphical user interface (GUI) test executions can be documented and then demonstrated for feedback. In our new concept, GUI tests from behavior‐driven development (BDD) are recorded, augmented, and demonstrated as videos. A GUI test is divided into several GUI unit tests, which are specified in Gherkin, a semi‐structured natural language. For each GUI unit test, a video is generated during test execution. Test steps specified in Gherkin are traced and highlighted in the video. Stakeholders review these generated videos and provide feedback, for example, on misunderstandings of requirements or on inconsistencies. To evaluate the impact of videos in identifying inconsistencies, we asked 22 participants to identify inconsistencies between (1) given requirements in regular sentences and (2) demonstrated behaviors from videos with Gherkin specifications or from Gherkin specifications alone. Our results show that participants tend to identify more inconsistencies from demonstrated behaviors, which are not in accordance with given requirements. They tend to recognize inconsistencies more easily through videos than through Gherkin specifications alone. The types of inconsistency are threefold: The mentioned feature can be incorrectly implemented, not implemented, or an unspecified new feature. We use a fictitious example showing how this feedback helps a product owner and her team manage requirements. We conclude that GUI test videos can help stakeholders give feedback more effectively. By obtaining early feedback, inconsistencies can be resolved, thus contributing to higher stakeholder satisfaction.

Using Behavior-Driven Development (BDD) for Non-Functional Requirements

Using Behavior-Driven Development (BDD) for Non-Functional Requirements

Design Goals for End-User Development of Robot-Assisted Physical Training Activities: A Participatory Design Study

Programming robots presents significant challenges, including high costs, extensive time commitments and steep learning curves, particularly for individuals lacking technical background in engineering. These barriers have been partially mitigated by the emergence of end-user development methodologies. Yet existing approaches often fall short in equipping users with the necessary software engineering competencies to develop comprehensive robot behaviors or to effectively maintain and re-purpose their creations. In this paper, we introduce a novel end-user development approach designed to empower physical therapists to independently specify robot-assisted physical training exercises, eliminating the need for robotics experts' intervention. Our approach is based on a set of design goals obtained through a participatory design study with experts in the field. It utilizes a textual domain-specific language (DSL) that enables users to define expected robot behaviors through Behaviour-Driven Development (BDD) scenarios. This paper discusses key themes, design objectives, and the evolution of requirements that emerged from an evaluative workshop.

Behaviour-driven development and metrics framework for enhanced agile practices in scrum teams

ContextAgile methodologies highlight collaborative efforts among software engineering groups for iterative, high-quality product delivery within short timeframes. However, Scrum teams face persistent challenges in achieving these objectives, stemming from difficulties in seamless collaboration and effective communication among various roles, such as developers and testers. To address these issues, Scrum teams are increasingly adopting Behaviour-Driven Development (BDD), a testing technique fostering collaboration and shared understanding through test scenarios. ObjectivesThis research investigates the adoption of BDD practices in Scrum teams and the formulation of a metrics framework tailored for optimizing Scrum practices and product quality. MethodsEmploying action research, this study extends over two and half years, actively engaging Scrum team members and stakeholders to encompass their collaborative contributions, insights, and perspectives. It commences with defining a metrics framework through exploration within agile teams to measure and evaluate Scrum team performance. Subsequently, the focus shifts to implementing BDD practices systematically, employing training sessions, workshops, and iterative refinements. ResultsThe results of the study emphasize the substantial role of Behaviour-Driven Development (BDD) in improving collaboration, communication, and the comprehension of requirements within the Scrum team. Concurrently, the tailored metrics framework bolsters quality assurance practices, enhancing software quality and customer satisfaction. BDD adoption expedites automation and product delivery, while the metrics framework enables informed decision-making. ConclusionsCombining BDD practices with a custom metrics framework offers a holistic strategy for addressing Scrum challenges. Enhanced collaboration, communication, and requirements comprehension, resulting from BDD, synergize with the metrics framework to elevate Scrum teams' performance, software quality, and customer value. This research underlines the importance of adopting BDD as a testing methodology to achieve these improvements in Scrum teams.

Comprehensive Evaluation and Insights Into the Use of Large Language Models in the Automation of Behavior-Driven Development Acceptance Test Formulation

Comprehensive Evaluation and Insights Into the Use of Large Language Models in the Automation of Behavior-Driven Development Acceptance Test Formulation

Software Quality Assurace

Abstract: Software Quality Assurance (SQA) plays a pivotal role in the software development lifecycle, ensuring the delivery of high-quality software products. This abstract provides a comprehensive overview of the latest advancements in Software Quality Assurance methodologies, practices, and tools. The evolving landscape of software development, characterized by agile methodologies, DevOps practices, and continuous integration, has necessitated a reevaluation and enhancement of SQA processes. The abstract begins by elucidating the fundamental principles of SQA, emphasizing its critical role in detecting and preventing defects throughout the software development process. It explores the shift-left approach, where quality assurance is integrated early in the development cycle, and the benefits of this approach in terms of cost reduction and accelerated time-tomarket. Furthermore, the abstract delves into the integration of artificial intelligence (AI) and machine learning (ML) techniques in SQA processes. The use of AI-driven testing tools for automated test case generation, anomaly detection, and predictive analysis is explored as a means to enhance the efficiency and effectiveness of quality assurance activities. The abstract also discusses the challenges and ethical considerations associated with the use of AI in SQA. Additionally, the abstract highlights the importance of comprehensive test automation strategies in modern SQA. It outlines the advantages of continuous testing, test-driven development (TDD), and behavior-driven development (BDD) methodologies in ensuring robust software quality. The abstract also addresses the challenges of maintaining test automation suites and advocates for a balanced approach that combines automated and manual testing. The abstract concludes by emphasizing the need for a holistic and adaptive SQA framework that aligns with the dynamic nature of software development. It calls for ongoing research and collaboration in the SQA domain to address emerging challenges and capitalize on new opportunities for improving software quality

Behaviour driven development: A systematic mapping study

Context:Behaviour Driven Development (BDD) uses scenarios written in semi-structured natural language to express software requirements in a way that can be understood by all stakeholders. The resulting natural language specifications can also be executed to reveal correct and problematic parts of a software. Although BDD was introduced about two decades ago, there is a lack of secondary studies in peer-reviewed scientific literature, making it difficult to understand the state of BDD research and existing gaps. Objective:To understand the current state of BDD research by conducting a systematic mapping study that covers studies published from 2006 (when BDD was introduced) to 2021. Method:By following the guidelines for conducting systematic mapping studies in software engineering, we sought to answer research questions on types of venues in which BDD papers have been published, research types, contribution types, studied topics and their evolution, as well as evaluation methods used in published BDD research. Results:The study identified 166 papers which were mapped. Key results include the following: the dominance of conference papers; scarcity of research with insights from the industry; shortage of philosophical papers on BDD; acute shortage of metrics for measuring various aspects of BDD specifications and the processes for producing BDD specifications; the dominance of studies on using BDD for facilitating various software development endeavours, improving the BDD process and associated artefacts, and applying BDD in different contexts; scarcity of studies on using BDD alongside other software techniques and technologies; increase in diversity of studied BDD topics; and notable use of case studies and experiments to study different BDD aspects. Conclusion:The paper improves our understanding of the state of the art of BDD, and highlights important areas of focus for future BDD research.

Automation of Software and Hardware Systems Acceptance Testing in the Paradigm of Behavior-Driven Development

The paper is devoted to the problem of software and hardware systems acceptance testing according to the Agile Testing methodology. The Agile approach is widely used by software developers, however, developers of software and hardware solutions rarely use this approach, not believing in its effectiveness. The paper assumes that for software and hardware complexes, the practice of continuous integration should be applicable and, as a result, test interaction with hardware at the level ofphysical interfaces performs automatically. The article's authors presented Accepta, a system for automating acceptance testing of software and hardware complexes, explicitly designed for the Agile Testing methodology in the context of continuous integration. The main component of Accepta is an interface block based on the Nucleo-F767ZI debug board manufactured by ST. Test actions that run within the framework of Accepta supplement by expanding the command system of the interface block. The software part, which implements the functions of test describing and execution, is based on the Cucumber framework for automating software systems acceptance testing. The requirements for the object under test and the scenarios for checking the requirements are described in the Gherkin language, which is close to the natural description. The test script steps are described programmatically in the Ruby language. The actual execution of test actions is provided by sending commands through the COM port to the interface unit and analyzing the received responses. As the practice of using Accepta in working projects has shown, this approach allows us to successfully apply the Agile development methodology for software and hardware systems. Due to the automotive interaction with the device under the test interface, high intensity of testing in the development process, including regression, is ensured. The regular testing consequence is fast feedback: as soon as any functionality stops working correctly, developers find out about it fast. At the same time, due to the use of test automation tools, the reproducibility of test action sequences led to the detection of a defect is ensured.

Generating multiple conceptual models from behavior-driven development scenarios

Researchers have proposed that generating conceptual models automatically from user stories might be useful for agile software development. It is, however, unclear from the state-of-the-art what a consistent and complementary set of models to generate is, how these models can be generated such that relationships and dependencies in a set of related user stories are unveiled, and why these models are useful in agile software development projects. In this paper, we address these questions through a Design Science research study. First, we define four stylized versions of Unified Modeling Language (UML) diagrams (i.e., use case diagram, class diagram, activity diagram, state machine diagram) that will be the target of the model generation. Although these stylized UML diagrams have a reduced abstract syntax, they offer different perspectives on the software system in focus with potential usefulness for requirements and software engineering. Second, we develop an automated model generation approach based on different design artifacts including a Natural Language Processing (NLP) tool that implements our approach. Key to our solution is the use of the Behavior-Driven Development (BDD) scenario template to document user stories. Using an example set of BDD scenarios as source of the model generation, we demonstrate the feasibility of our approach via the NLP tool that implements our approach. Third, we conduct an empirical study with experts in agile software development involving the researcher-guided interactive use of our tool to explore the use of the generated models. This study shows the perceived usefulness of the models that our tool can generate and identifies different uses and benefits of the models for requirements analysis, system design, software implementation, and testing in projects that employ agile methods.

Specifying Internet of Things Behaviors in Behavior-Driven Development: Concurrency Enhancement and Tool Support

The Internet of Things (IoT) systems are inherently distributed with many concurrent behaviors. In order to apply behavior-driven development (BDD), a proven agile practice of software development that brings many benefits, we must ensure that the specification of sequential and concurrent behaviors is supported at the specification level and that tool support is in place to execute the specification. This study proposes a minimal semantic enhancement to the Gherkin language, the most popular specification language in BDD, to distinguish sequential and concurrent behaviors. At the same time, a tool called concurrentSpec is developed to support the correct execution of specifications written in the enhanced Gherkin language. With two IoT examples involving both sequential and concurrent behaviors, it is shown that the enhanced Gherkin with concurrentSpec can correctly specify and execute the specifications, while the original Gherkin with existing tools is unable to do so. Hence, the contribution of this study is to eliminate a technical impediment for the IoT development community to adopt BDD and receive its benefits.

Behavior Driven Development using Gauge Framework

Abstract: Behavior Driven Development (BDD) focuses on the behavior of the system or the business point of view and captures the business specific requirements. Many BDD frameworks implement a strict syntax like Gherkin, this study proposes a solution to remove this by using Gauge Framework. The BDD Gauge Framework will be analyzed and studied in-depth. The drawback of this study is that a big industry level testing isn’t feasible hence will need to be dependent on external research papers. The emerging opportunity here is that Gauge Framework can be considered as a BDD tool with its unique inbuilt capability of parallelization or scaling of tests, data driven testing, no strict restricted syntax and also enhances agile development. Key findings of this study will be how Gauge Framework improves testing with better defect detection while maintaining good method coverage for various business scenarios and also how it can be used as a BDD framework by reducing manual effort. This paper will also determine the performance with respect to other competitive frameworks in the market like Cucumber, RSpec, SpecFlow.

Supporting refactoring of BDD specifications—An empirical study

Behavior-driven development (BDD) is a variant of test-driven development where specifications are described in a structured domain-specific natural language. Although refactoring is a crucial activity of BDD, little research is available on the topic. To support practitioners in refactoring BDD specifications by (1) proposing semi-automated approaches to identify refactoring candidates; (2) defining refactoring techniques for BDD specifications; and (3) evaluating the proposed identification approaches in an industry context. Using Action Research, we have developed an approach for identifying refactoring candidates in BDD specifications based on two measures of similarity and applied the approach in two projects of a large software organization. The accuracy of the measures for identifying refactoring candidates was then evaluated against an approach based on machine learning and a manual approach based on practitioner perception. We proposed two measures of similarity to support the identification of refactoring candidates in a BDD specification base; (1) normalized compression similarity (NCS) and (2) similarity ratio (SR). A semi-automated approach based on NCS and SR was developed and applied to two industrial cases to identify refactoring candidates. Our results show that our approach can identify candidates for refactoring 6o times faster than a manual approach. Our results furthermore showed that our measures accurately identified refactoring candidates compared with a manual identification by software practitioners and outperformed an ML-based text classification approach. We also described four types of refactoring techniques applicable to BDD specifications; merging candidates, restructuring candidates, deleting duplicates, and renaming specification titles. Our results show that NCS and SR can help practitioners in accurately identifying BDD specifications that are suitable candidates for refactoring, which also decreases the time for identifying refactoring candidates.

Behavior-Driven Development of Microservice Applications

As the number of microservice applications rises, different development methodologies for them are under consideration. In this manuscript, we propose a behavior-driven development method for microservice applications. The proposed method starts with writing end-to-end tests at the system or application level and then moves down to the microservice level, where component and unit tests are written. Next, code that passes these tests is developed one by one for each level. Once user stories are covered, our method loops again to integrate negative tests to achieve holistic testing for the microservices and the application. Finally, the proposed method is validated with an application with five microservices. Results confirm that the proposed method matches with the generally accepted test pyramid.

The transformation framework The role security in the global education system

This article analyses the role of Global Education System (GES) and proposes the Applied Holistic Mathematical Model for GES (AHMM4GES). The AHMM4GES is based on many years of research on business & educational transformations, Artificial Intelligence (AI), applied mathematics, software modelling, business & organizational engineering, skills & educational systems, financial analysis, security and enterprise architecture. The used research methodology is based on the author’s authentic mixed research method that is supported by a mainly qualitative reasoning module. AHMM4GES’s formalism mimics the human brain, by using empirical processes that are based on heuristics. The AHMM4GES is used to implement a decision-making system (or an expert system) to support a GES and uses a behaviour-driven development environment that can be easily adopted by any organization. The development environment can be used by any team member without any prior computer sciences qualifications. The AHMM4GES is used to estimate the Role of AI based Security in GES’s (RAISGES) context and tries to estimate the roles of the giants in this domain, like the USA, China, and India; and what would be the real role of the European Union and France. The uniqueness and originality of this research is that the AHMM4GES promotes a holistic unbundling process, the alignment of transformation strategies to support GES’ evolution. For a successful integration of AHMM4GES in projects, the manager’s profile, education, skills and role are crucial, where his decisions are supported by the selection, implementation and processing of critical success factors. The main implication is a systemic approach that is the optimal to integrate an RAI4GES.

ScrumOntoBDD: Agile software development based on scrum, ontologies and behaviour-driven development

When developing a Learning Management System (LMS) using Scrum, we noticed that it was quite often necessary to redefine some system behaviour scenarios, due to ambiguities in the requirement specifications, or due to misinterpretations of stories reported by the Product Owners (POs). The definition of test suites was also cumbersome, resulting in test suites that were incomplete or did not at all comply with the system requirements. Based on this experience and to deal with these problems, in this paper, we propose the ScrumOntoBDD approach to agile software development, which combines Scrum, ontologies and Behaviour-Driven Development (BDD). This approach is centred on the concepts and techniques of Scrum and BDD and focuses on the planning and analysis phases of the software life cycle, since the BDD tools currently provide little support to these phases, while most of the problems during the LMS development were found exactly there. We claim that our approach improves the software development practices in this respect. Furthermore, ScrumOntoBDD employs ontologies in order to reduce ambiguities intrinsic to the use of a natural language as a BDD ubiquitous language. In this paper, we illustrate and systematically evaluate our approach, showing that it is beneficial since it improves the communication between members of an agile development team.

The business transformation enterprise architecture framework for innovation The role of artificial intelligence in the global business education (RAIGBE)

This article analyses the role of Global Business Education (GBE) and proposes the Applied Holistic Mathematical Model for GBE (AHMM4GBE). The AHMM4GBE is based on a lifetime long research on business transformations, Artificial Intelligence (AI), applied mathematics, software modelling, business engineering, educational systems, financial analysis, security and enterprise architecture. The used research methodology is based on the author’s authentic mixed research method that is supported by a mainly qualitative reasoning module. AHMM4GBE’s formalism mimics the human brain, by using empirical processes that are mainly based on heuristics. The AHMM4GBE is used to implement a decision-making system (or an expert system) to support a GBE and uses a behaviour-driven development environment that can be easily adopted by any organization. The development environment can be used by any team member without any prior computer sciences qualifications. The AHMM4GBE is used to estimate the Role of AI in GEB’s (RAIGBE) context and tries to estimate the roles of the giants in this domain, like USA, China, and India; and what would be the real role of the European Union and France. The uniqueness of this research is that the AHMM4GBE promotes a holistic unbundling process, the alignment of transformation strategies to support GBE’s evolution. For a successful integration of the AHMM4GBE in projects, the manager’s profile and role are crucial, where his decisions are supported by the selection, implementation and processing of critical success factors. A holistic systemic system approach is the optimal choice to integrate an RAI4GBE.

Adapting Behavior Driven Development (BDD) for large-scale software systems

Context:Large-scale software projects require interaction between many stakeholders. Behavior-driven development (BDD) facilitates collaboration between stakeholders, and an adapted BDD process can help improve cooperation in a large-scale project. Objective:The objective of this study is to propose and empirically evaluate a BDD based process adapted for large-scale projects. Method:A technology transfer model was used to propose a BDD based process for large-scale projects. We conducted six workshop sessions to understand the challenges and benefits of BDD. Later, an industrial evaluation was performed for the process with the help of practitioners. Results:From our investigations, understanding of a business aspect of requirements, their improved quality, a guide to system-level use-cases, reuse of artifacts, and help for test organization are found as benefits of BDD. Practitioners identified the following challenges: specification and ownership of behaviors, adoption of new tools, the software projects’ scale, and versioning of behaviors. We proposed a process to address these challenges and evaluated the process with the help of practitioners. Conclusion:The evaluation proved that BDD could be adapted and used to facilitate interaction in large-scale software projects in the software industry. The feedback from the practitioners helped in improving the proposed process.

Behavior-Driven Quality First Agile Testing for Cloud Service

In a recent testing software system, with cloud implementation a cloud service abstraction allows providing high-level ubiquitous language (UL) compositions through behavior-driven step-wise agile cycles. By adapting this UL the instantiated behaviors simplify the demand in software quality through mechanisms to cope with complex digital transformation and evolution. However, in such a context, testing becomes challenging quality engineering. As a result, it poses a threat to the services on a cloud as the access to its source codes rely on these abstractions. The aim is to introduce a way by strictly focusing on a black box approach. One way is to realize the client-side continuous Quality First (QF)-Test behavior-driven development (BDD). On this point, a meta-model helps to transform the RESTful cloud specification through domain specific language (DSL) while accommodating a low-level details on a test coverage report. By using features from the user stories the Gherkin BDD styles enable the meta-model which creates the instances of DSL to implement the runnable test steps on a cucumber framework. Each step is designed in a GraphWalker through modeling a context finite machine via a model visual editor, and generates the dependency test model paths. As an evaluation, the QF-Test executes the required steps given by data-driven elements for creating run-log trace analysis. As a comparison, the Jenkins framework is configured to build the QF-Test node of test suites for generating the behavior-driven and continuous integration test report. Moreover, the steps with the keywords are automated to verify the GraphWalker test cases through traversing the paths generated. As a case application, a sample REST API mobility service instance is considered.