Testing Software Systems Research Articles

АРХІТЕКТУРА ПРОГРАМНОЇ СИСТЕМИ ДЛЯ ВИРІШЕННЯ ЗАДАЧІ КЛАСИФІКАЦІЇ НА ОСНОВІ ПРИВАТНИХ ДАНИХ

Data analysis and artificial intelligence systems are becoming widely used in various spheres of human life. This is confirmed by more typical cases of their use, in particular, the selection of recommendations for the user in e-commerce, the detection of spam in e-mail services, and the moderation of user comments; as well as cases of personal use of such tools (for example, chatbots ChatGPT, Google Bard, Microsoft Copilot have appeared and gained significant popularity in the last two years). One of the key elements of such systems is data, which is necessary for training and testing software systems of intelligent data analysis. A significant amount of diverse data contributes to the construction of a software system with high accuracy. Considering this, the task of choosing and preparation of datasets that can be used in the construction of such systems is important. One of the difficulties in this task is the presence of private information in the datasets, which limits their use for systems of intelligent data analysis. The paper is devoted to the development of the software system architecture for solving the classification problem based on private data. The existing methods and architectural approaches for privacy-preserving in machine learning were considered. The architecture of the software system was proposed, the characteristic feature of which is the protection of private datasets by using of functional encryption, which allows to increase the number of datasets for training publicly available data analysis and artificial intelligence systems. The proposed architecture of the software system is based on the client-server architecture and functional encryption. The components are a classifier, a generator of encryption keys, and modules of functional encryption and decryption. Prospects for further research were discussed.

Four-Dimensional ACC Analysis

The article discusses the issues of planning and resource management in the process of testing software systems. The paper presents the ACC analysis method used at Google to optimize the distribution of efforts for testing different parts of the system. Extending the method by adding a fourth characteristic - actors (roles of system users) – allows for a more flexible assessment of action requirements and user skill levels. Illustrative examples of system attributes and components help understand the principles of the method. The work proposes a new approach to risk management and process improvement in testing software systems in a multidimensional space. The effectiveness of applying the enhanced ACC analysis method using a risk-oriented approach was demonstrated using the example of a control system for technological operations in the repair of electric motors, for which attributes, components, actors were identified, opportunities at their intersection were analyzed, and testing was conducted, which helped improve the system's quality.

Design and self-diagnostics of cyberphysical control devices on SOC platform

The subject of research in this article is models, methods, and procedures for designing and self-diagnosing automated models of logic control devices implemented in SoCs. The object of work is the procedures for automated design and diagnosis of digital devices on the SoC technology platform. The aim of the study is to develop models and procedures for designing and self-testing in the cycle of automated design of automatic logic control systems on the SoC technology platform, which will significantly increase the reliability of their operation. The article solves the following tasks: consideration of the procedures for interacting the processor core with programmable logic as part of the SoC; improvement of the procedures for designing and testing software and hardware systems based on SoC; further development of procedures for automated design, verification, and diagnosis of cyber-physical logic control systems using programming languages and hardware description languages; implementation of the procedure for hardware self-testing of control automata on the SoC technology platform. The following methods are implemented: synthesis of control automata based on graph models, implementation of control automata models in the C programming language using an automata template, diagnostic experiment by traversing the automata transition graph. Results achieved. Based on the analysis of the procedures for the interaction of the processor core and programmable logic on the selected SoC platform, a model of a cyber-physical logic control system was designed. The practical implementation was carried out on the basis of the Vivado/Vitis/Vitis HLS CAD toolkit. The method of hardware self-testing of control automata on the technological platform of SoC ZYNQ-7000 was implemented. Conclusions. The article analyzes the principles of designing embedded cyber-physical systems implemented in system-on-chip. The principles of building verification systems and embedded self-diagnostics of system-on-chip systems containing software and hardware are considered. The developed methods are tested on a model of a traffic light logic control device on the SoC FPGA platform of the ZYNQ-7000 family by Xilinx. The Moore's control automaton is implemented in the PL block in the C programming language, and the operational automaton is implemented in the PS block. During the organization of the self-diagnosis process, a non-destructive diagnostic experiment was performed by traversing all arcs of the transition graph, starting from the initial vertex. In this case, the tester was an operational automaton, the reference logic and time values of which were stored in the memory of the PS unit. Visual observation of the diagnostic experiment was carried out using the LED panel of the ZedBoard board.

Detecting system defects of digital devices in the course of malfunction imitation using fuzzing

Aim. The paper proposes approaches to the organisation of testing of digital systems through malfunction imitation for the purpose of ensuring compliance of international and Russian failure and fault resistance standards for the purpose of efficient (in terms of time) detection of software defects as part of mass production of products. The paper proposes a structure and operating algorithm of a hardware and software test bed for malfunction simulation intended for testing a system’s devices. The test bed collects and processes data for fuzzing, hardware error identification, as well as defines the scope of testing.Methods. The paper used basic systems approach, classical methods of the probability theory and mathematical statistics, decision theory, methods of hardware and software testing and development, mathematical theory of fuzzy sets and fuzzy logic.Results. Malfunction simulation algorithms were developed for the purpose of testing hardware and software systems using fuzzing that ensure probabilistic estimation of the termination time of testing with a specified accuracy.Conclusions. The above set of algorithms allows detecting system defects in the process of software and hardware integration into a single system that cause new malfunctions (emergence) that cannot be taken into consideration at the design stage.

The Method of Forming a Digital Shadow of the Human Movement Process Based on the Combination of Motion Capture Systems

The article deals with the problem of forming a digital shadow of the process of moving a person. An analysis of the subject area was carried out, which showed the need to formalize the process of creating digital shadows to simulate human movements in virtual space, testing software and hardware systems that operate on the basis of human actions, as well as in various systems of musculoskeletal rehabilitation. It was revealed that among the existing approaches to the capture of human movements, it is impossible to single out a universal and stable method under various environmental conditions. A method for forming a digital shadow has been developed based on combining and synchronizing data from three motion capture systems (virtual reality trackers, a motion capture suit, and cameras using computer vision technologies). Combining the above systems makes it possible to obtain a comprehensive assessment of the position and condition of a person regardless of environmental conditions (electromagnetic interference, illumination). To implement the proposed method, a formalization of the digital shadow of the human movement process was carried out, including a description of the mechanisms for collecting and processing data from various motion capture systems, as well as the stages of combining, filtering, and synchronizing data. The scientific novelty of the method lies in the formalization of the process of collecting data on the movement of a person, combining and synchronizing the hardware of the motion capture systems to create digital shadows of the process of moving a person. The obtained theoretical results will be used as a basis for software abstraction of a digital shadow in information systems to solve the problems of testing, simulating a person, and modeling his reaction to external stimuli by generalizing the collected data arrays about his movement.

Comparing fine-grained source code changes and code churn for bug prediction - A retrospective

More than two decades ago, researchers started to mine the data stored in software repositories to help software developers in making informed decisions for developing and testing software systems. Bug prediction was one of the most promising and popular research directions that uses the data stored in software repositories to predict the bug-proneness or number of bugs in source files. On that topic and as part of Emanuel's PhD studies, we submitted a paper with the title Comparing fine-grained source code changes and code churn for bug prediction [8] to the 8th Working Conference on Mining Software Engineering, held 2011 in beautiful Honolulu, Hawaii. Ten years later, it got selected as one of the finalists to receive the MSR 2021 Most Influential Paper Award. In the following, we provide a retrospective on our work, describing the road to publishing this paper, its impact in the field of bug prediction, and the road ahead.

Improving the quality of corporate information systems development based on a risk-based approach

In relation to improving the quality of corporate information systems development, we consider risk management methods based on testing software systems, using automatic production controls, and ensuring the production accuracy of product manufacturing. Development models and the set of inherent risks are presented. A comprehensive risk assessment indicator is proposed that takes into account the probability and consequences of their occurrence. It is shown that to assess the degree of destabilisation of production, it is advisable to use a point scale of risks.

Behavior-Driven Quality First Agile Testing for Cloud Service

In a recent testing software system, with cloud implementation a cloud service abstraction allows providing high-level ubiquitous language (UL) compositions through behavior-driven step-wise agile cycles. By adapting this UL the instantiated behaviors simplify the demand in software quality through mechanisms to cope with complex digital transformation and evolution. However, in such a context, testing becomes challenging quality engineering. As a result, it poses a threat to the services on a cloud as the access to its source codes rely on these abstractions. The aim is to introduce a way by strictly focusing on a black box approach. One way is to realize the client-side continuous Quality First (QF)-Test behavior-driven development (BDD). On this point, a meta-model helps to transform the RESTful cloud specification through domain specific language (DSL) while accommodating a low-level details on a test coverage report. By using features from the user stories the Gherkin BDD styles enable the meta-model which creates the instances of DSL to implement the runnable test steps on a cucumber framework. Each step is designed in a GraphWalker through modeling a context finite machine via a model visual editor, and generates the dependency test model paths. As an evaluation, the QF-Test executes the required steps given by data-driven elements for creating run-log trace analysis. As a comparison, the Jenkins framework is configured to build the QF-Test node of test suites for generating the behavior-driven and continuous integration test report. Moreover, the steps with the keywords are automated to verify the GraphWalker test cases through traversing the paths generated. As a case application, a sample REST API mobility service instance is considered.

Высокотемпературное воздействие газовой струи реагирующих компонентов на фронтальную пластину

The paper introduces the results of measuring and predicting the heat and force effect of jets of high-temperature reacting mixtures on the oxygen-methane, oxygen-alcohol components when acting on the front plate in the near field of the jet. A high-temperature supersonic gas jet flows out of a model chamber with a Laval nozzle into a medium with atmospheric pressure at a degree of off-design ratio of about unity. In the chamber, ignition and stable combustion of a mixture of selected substances occur, the ratio of these substances providing a stagnation temperature in the range of 1900 ... 3400 K. The pressure distribution function on the front plate obtained in the experiment is used. The proposed model of the high-temperature flow effect on the frontal surface can be used to test software systems and determine the levels of thermal effect during sample tests.

SpecMiner: Heuristic-based mining of service behavioral models from interaction traces

Software behavioral models have proven useful for emulating and testing software systems. Many techniques have been proposed to infer behavioral models of software systems from their interaction traces. The quality of the inferred/mined model is critical to their successful use. While generalization is necessary to deduce concise behavioral models, existing techniques of inferring models, in general, overgeneralize what behavior is valid. Imprecise models include many spurious behaviors, and thus compromise the effectiveness of their use. In this paper, we propose a novel approach, named SpecMiner, that increases the precision of the behavioral model inferred from interaction traces. The essence of our approach is a heuristic-based generalization and truthful minimization. The set of heuristics include patterns to match input traces and generalize them towards concise model representations. Furthermore, we adopt a truthful minimization technique to merge these generalized traces. The key insight of our approach is to infer a concise behavioral model without compromising its precision. We present an empirical evaluation of how our approach improves upon the state-of-the-art specification inference techniques. The results show that our approach mines model with 100% precision and recall with a limited computation overhead.

Discussion of cloud CAD software testing approach

In the life cycle of software systems, the testing phase is one of the most time-consuming and expensive. One of the fundamental issues in test planning is the choice between automated and manual methods for testing a software system or its individual modules. Since the choice can be different for individual modules, then in fact we are talking about a combination of these methods, and this is an important feature of CAD quality assessment. When choosing a testing method, it is recommended to take into account the specific characteristics of the tested software system. In this article, we propose to use a list of questions that can be used to determine the qualitative properties of such a test system. A list of questions is created using the ISO / IEC 25010: 2011 model. Developing a method to support the software testing process is an urgent scientific task.

МОДЕЛЬ ИНФОРМАЦИОННО-СОВЕТУЮЩЕЙ СИСТЕМЫ ПОДДЕРЖКИ ПРИНЯТИЯ РЕШЕНИЯ ПРИ ВЫБОРЕ СПОСОБА ТЕСТИРОВАНИЯ ПРО-ГРАММНОГО ОБЕСПЕЧЕНИЯ

High quality software systems is an urgent task for any enterprise. Improving quality increases the competitiveness and relevance of software products. One of the tools to ensure quality is testing software systems. This article discusses the basic principles of building an information-advisory system (IAS) of decision support when choosing a method for testing software. The conceptual approaches to testing based on the functional characteristics of software classes are analyzed. When choosing a testing method, it is recommended to take into account not only the financial component, but also the specific characteristics of the tested software system. A list of questions has been developed with the help of which the qualitative properties of software systems are formalized. The list of questions is based on the ISO / IEC 25010: 2011 model. Questionnaires and tables of pairwise comparisons of questions are invited to fill out a group of experts. Questions have equivalent significance with respect to the criterion of “Importance for Automation”. When selecting experts, it is not always possible to create a homogeneous group, therefore, it is proposed to calculate the weighting coefficients of competence of each expert. Based on the data obtained, weights are calculated for answers to questions. Next, the convolution is calculated and, depending on the result, a recommendation is made on choosing the type of testing. The analytical and structural model of IAS is constructed. In the developed software system, a data input subsystem, a subsystem for ranking questions and expert competencies, and a unit for generating the resulting data are allocated. The proposed IAS will make it possible to calculate the competency coefficients of experts, to rank questions according to the degree of importance for testing automation, and to choose the optimal approach to testing software systems.

Features of software testing in the development of geographic information systems

In this article, the method of selecting a method of testing geographic information systems is proposed. Nowadays, thanks to advances in information technology new methods of data exchange, positioning technologies, improvements in spatial databases, there has been a significant leap in development of geographic information system. When choosing a testing method, it is recommended to take into account not only the financial component, but also the specific characteristics of the tested software system. A list of questions is developed by which qualitative properties of software systems are formalized. The list of questions is based on the ISO/IEC 25010:2011 model. The relevance of this work follows from the fact that new technologies of software development require new approaches to testing software systems. Properly developed testing technique improves the quality of geographic information systems created. Improving quality increases the competitiveness and demand of software products.

Concept Selection and Neutronic Characteristics of Nuclear Icebreaker Cores

The developmental history of the series-produced core for nuclear icebreakers and its conceptual and structural designs and the computational-methodological base of the physical design are presented. The fundamental differences from VVER-type power reactor cores, which are due to the specific operating conditions of ice breakers, are discussed. Measurements of the basic neutronic characteristics of a series-produced icebreaker core are presented and compared with calculations performed with the tested software system VKN-02. Experimental data confirm that the accuracy of the VKN-02 system in calculating the neutronic characteristics determining nuclear safety and the thermal engineering reliability of an icebreaker core is acceptable.

Energy-Saving Extremal Control of an Electrical Mode for Electric Arc Units

The aim of this research is to determine a rationale and build a system for electrical mode control to ensure the maximum possible hourly capacity of an electric arc unit. To achieve this objective, a double-circuit control structure with independent stabilizing and optimizing control circuits is proposed. To build the optimizing circuit, the principles of extremal control and fuzzy logic are used to choose the optimum current and arc power of the processing unit. Simulations of production revealed that it took 28 s to achieve an optimum mode from the moment of arc ignition, and when increasing the number of taps on the furnace transformer from 14 to 23, the system achieved the optimum mode of power consumption in 12 s. To transfer this control approach to electric arc units in production, the developed and tested software system based on the optimized circuit for electrical mode control would require only adaptation to relevant conditions without major changes.

Search‐based software engineering for constructing covering arrays

Search-based software engineering involves the application of optimisation methods to solve software engineering problems. One of the most significant difficulties in testing software systems is the effort needed to build the test suites required to validate a software system, which efficiently exposes faults. Given the importance of the software testing stage, a specific sub-area known as search-based software testing has become relevant in recent years. In this work, a search-based software testing algorithm for constructing covering arrays is proposed. A covering array is a combinatorial structure that can be used as a set of test cases. By utilising this algorithm, the authors reduce the size of 893 test suites.

Relationship Maintenance in Software Language Repositories

The context of this research is testing and building software systems and, specifically, software language repositories (SLRs), i.e., repositories with components for language processing (interpreters, translators, analyzers, transformers, pretty printers, etc.). SLRs are typically set up for developing and using metaprogramming systems, language workbenches, language definition frameworks, executable semantic frameworks, and modeling frameworks. This work is an inquiry into testing and building SLRs in a manner that the repository is seen as a collection of language-typed artifacts being related by the applications of language-typed functions or relations which serve language processing. The notion of language is used in a broad sense to include text-, tree-, graph-based languages as well as representations based on interchange formats and also proprietary formats for serialization. The overall approach underlying this research is one of language design driven by a complex case study, i.e., a specific SLR with a significant number of processed languages and language processors as well as a noteworthy heterogeneity in terms of representation types and implementation languages. The knowledge gained by our research is best understood as a declarative language design for regression testing and build management, we introduce a corresponding language Ueber with an executable semantics which maintains relationships between language-typed artifacts in an SLR. The grounding of the reported research is based on the comprehensive, formal, executable (logic programming-based) definition of the Ueber language and its systematic application to the management of the SLR YAS which consists of hundreds of language definition and processing components (such as interpreters and transformations) for more than thirty languages (not counting different representation types) with Prolog, Haskell, Java, and Python being used as implementation languages. The importance of this work follows from the significant costs implied by regression testing and build management and also from the complexity of SLRs which calls for means to help with understanding.

A Systematic Review of Use Cases based Software Testing Techniques

Use case based Software testing concerned with testing an entire system’s functionality and its constraints. User’s requirements can be drawn in form of use cases to show the internal and external behaviour of a system. The use case has been used to test software system for different levels as well as through many ways according to nature of software. Rather than proposing a new technique, it is necessary to synthesize the existing techniques of Use case based software testing which is available in the literature. A Systematic Literature Review has been performed to investigate all the current approaches of Use case based software testing techniques. A question-wise analysis has also been presented for the researchers to investigate the effectiveness of these techniques. In results, we have discussed all the existing approaches of Use case based software testing along with its strengths & weaknesses and measure the characteristics of these approaches based on some key parameters. We have concluded the current state of the art of all the existing approaches of Use case based testing. Results are based on both types of analysis; Qualitative as well as Quantitative.

Paradigms of Model-Based and Symbolic Testing of Software Systems

Model-based testing (MBT) of software systems is considered and well-known MBT methods and tools are reviewed. These methods make it possible to automate procedures performed by software test engineers and significantly increase the quality of testing software systems. The MBT technology, its components, and main tools are described. A list of problems arising in using MBT is compiled. A symbolic approach that makes it possible to solve some of existing MBT problems and the Generic Trace Generator based on the symbolic approach are presented. This tool was developed at the V. M. Glushkov Institute of Cybernetics. The symbolic approach is extended for the first time as a result of addition of the test execution stage that considerably improves the MBT technology.

METRIC: METamorphic Relation Identification based on the Category-choice framework

Metamorphic testing is a promising technique for testing software systems when the oracle problem exists, and has been successfully applied to various application domains and paradigms. An important and essential task in metamorphic testing is the identification of metamorphic relations, which, due to the absence of a systematic and specification-based methodology, has often been done in an ad hoc manner—something which has hindered the applicability and effectiveness of metamorphic testing. To address this, a systematic methodology for identifying metamorphic relations based on the category-choice framework, called metric, is introduced in this paper. A tool implementing this methodology has been developed and examined in an experiment to determine the viability and effectiveness of metric, with the results of the experiment confirming that metric is both effective and efficient at identifying metamorphic relations.