Natural Language Specifications Research Articles

The Genesis of AIbyAI Integrated Circuit: Where AI Creates AI

The typical Integrated Circuit (IC) development process commences with formulating specifications in natural language and subsequently proceeds to Register Transfer Level (RTL) implementation. RTL code is traditionally generated through manual efforts, using Hardware Description Languages (HDL) such as VHDL or Verilog. High-Level Synthesis (HLS), on the other hand, converts programming languages to HDL; these methods aim to streamline the engineering process, minimizing human effort and errors. Currently, Electronic Design Automation (EDA) algorithms have been improved with the use of AI, with new advancements in commercial (such as ChatGPT, Bard, among others) Large Language Models (LLM) and open-source tools presenting an opportunity to automate the chip design process. This paper centers on the creation of AIbyAI, a Convolutional Neural Network (CNN) IC entirely developed by an LLM (ChatGPT-4), and its manufacturing with the first fabricable open-source Process Design Kit (PDK), SKY130A. The challenges, opportunities, advantages, disadvantages, conversation flow, and workflow involved in CNN IC development are presented in this work, culminating in the manufacturing process of AIbyAI using a 130 nm technology, marking a groundbreaking achievement as possibly the world’s first CNN entirely written by AI for its IC manufacturing with a free PDK, being a benchmark for systems that can be generated today with LLMs.

Extraction of UML class diagrams using deep learning: Comparative study and critical analysis

Several approaches and tools have been proposed to facilitate the automatic generation of Unified Modeling Language (UML) class diagrams from natural language specifications, based on advances in Natural Language Processing (NLP). However, these tools suffer from difficulties due to the inherent imprecision and ambiguity commonly found in natural language expressions. In this article, we present an overview and a study of approaches and tools designed to extract UML diagrams from textual requirements using NLP and computational linguistics techniques. Furthermore, we introduce approaches employing deep learning techniques. Next, we provide a descriptive study and comparative analysis of the limitations and contributions of these automatic and semiautomatic tools, as well as solutions for improving the existing state of the art.

Automated programming, symbolic computation, machine learning: my personal view

In this note, I present my personal view on the interaction of the three areas Automated Programming, Symbolic Computation, and Machine Learning. Programming is the activity of finding a (hopefully) correct program (algorithm) for a given problem. Programming is central to automation in all areas and is considered one of the most creative human activities. However, already very early in the history of programming, people started to “jump to the meta-level” of programming, i.e., started to develop procedures that automate, or semi-automate, (various aspects or parts of) the process of programming. This area has various names like “Automated Programming”, “Automated Algorithm Synthesis”, etc. Developing compilers can be considered an early example of a problem in automated programming. Automated reasoners for proving the correctness of programs with respect to a specification is an advanced example of a topic in automated programming. ChatGPT producing (amazingly good) programs from problem specifications in natural language is a recent example of automated programming. Programming tends to become the most important activity as the level of technological sophistication increases. Therefore, automating programming is maybe the most exciting and relevant technological endeavor today. It also will have enormous impact on the global job market in the software industry. Roughly, I see two main approaches to automated programming:symbolic computationand machine learning. In this note, I explain how the two approaches work and that they are fundamentally different because they address two completely different ways of how problems are specified. Together, the two approaches constitute (part of) what some people like to call “artificial intelligence”. In my analysis, both approaches are just part of (algorithmic) mathematics. The approaches, like all non-trivial mathematical methods, need quite some intelligence on the side of the human inventors of the methods. However, applying the methods is just “machine execution” of algorithms. It is misleading to call the application “machine intelligence” or “artificial intelligence”. The analysis of the two approaches to automated programming also suggests that the two approaches, in the future, should be combined to achieve even higher levels of sophistication. At the end of this note, I propose some research questions for this new direction.

USE CASE TOOLS FOR IDENTIFICATION SOFTWARE REQUIREMENTS: STATE OF ART

The requirement analysis step of the Software Engineering Life Cycle (SELC) is the most crucial in order to ensure excellent quality. Besides, requirements specifications must be compared to the consistency, completeness, and correctness (3C's). Thus, dealing with Natural Language (NL) requirements makes this more challenging than usual., The problems that appear at this stage, if they are not detected or proceed, will lead to errors in the system and sometimes to the failure of the entire system; in addition to the effort required and the cost of maintenance will be large. Accordingly,to overcome these problems, differenttools have been developed and implemented to analyse the requirements through Natural Language Processing (NLP). These tools facilitate the process of eliciting requirements through Unified Modelling Language (UML) diagrams. Additionally, creating UML diagrams from NL specifications is a very difficult process. In particular, few efforts have been made in the work of extracting NL specifications, so these diagrams facilitate understanding and analysis of requirements and reduce the time of completion very significantly compared to the manual method. Using automated software engineering tools presents high-quality software that can be produced to help software developers and engineers. The main aim of this research is to concentrate onanalysing requirements based on the Use Case Diagram, which is a more popular one in UML diagram. Thus, this research will support other researchers in understanding and specifying the useful tools and mechanisms that can help them to analyse the requirements through Natural Language Processing (NLP) based UML diagrams

WebUAV-3M: A Benchmark for Unveiling the Power of Million-Scale Deep UAV Tracking.

Unmanned aerial vehicle (UAV) tracking is of great significance for a wide range of applications, such as delivery and agriculture. Previous benchmarks in this area mainly focused on small-scale tracking problems while ignoring the amounts of data, types of data modalities, diversities of target categories and scenarios, and evaluation protocols involved, greatly hiding the massive power of deep UAV tracking. In this article, we propose WebUAV-3M, the largest public UAV tracking benchmark to date, to facilitate both the development and evaluation of deep UAV trackers. WebUAV-3M contains over 3.3 million frames across 4,500 videos and offers 223 highly diverse target categories. Each video is densely annotated with bounding boxes by an efficient and scalable semi-automatic target annotation (SATA) pipeline. Importantly, to take advantage of the complementary superiority of language and audio, we enrich WebUAV-3M by innovatively providing both natural language specifications and audio descriptions. We believe that such additions will greatly boost future research in terms of exploring language features and audio cues for multi-modal UAV tracking. In addition, a fine-grained UAV tracking-under-scenario constraint (UTUSC) evaluation protocol and seven challenging scenario subtest sets are constructed to enable the community to develop, adapt and evaluate various types of advanced trackers. We provide extensive evaluations and detailed analyses of 43 representative trackers and envision future research directions in the field of deep UAV tracking and beyond. The dataset, toolkits, and baseline results are available at https://github.com/983632847/WebUAV-3M.

Natural Language Specification of Reinforcement Learning Policies Through Differentiable Decision Trees

Human-AI policy specification is a novel procedure we define in which humans can collaboratively warm-start a robot's reinforcement learning policy. This procedure is comprised of two steps; (1) Policy Specification, i.e. humans specifying the behavior they would like their companion robot to accomplish, and (2) Policy Optimization, i.e. the robot applying reinforcement learning to improve the initial policy. Existing approaches to enabling collaborative policy specification are often unintelligible black-box methods, and are not catered towards making the autonomous system accessible to a novice end-user. In this paper, we develop a novel collaborative framework to allow humans to initialize and interpret an autonomous agent's behavior. Through our framework, we enable humans to specify an initial behavior model via unstructured, natural language (NL), which we convert to lexical decision trees. Next, we leverage these translated specifications, to warm-start reinforcement learning and allow the agent to further optimize these potentially suboptimal policies. Our approach warm-starts an RL agent by utilizing non-expert natural language specifications without incurring the additional domain exploration costs. We validate our approach by showing that our model is able to produce <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$&gt;$</tex-math></inline-formula> 80% translation accuracy, and that policies initialized by a human can match the performance of relevant RL baselines in two domains.

Behaviour driven development: A systematic mapping study

Context:Behaviour Driven Development (BDD) uses scenarios written in semi-structured natural language to express software requirements in a way that can be understood by all stakeholders. The resulting natural language specifications can also be executed to reveal correct and problematic parts of a software. Although BDD was introduced about two decades ago, there is a lack of secondary studies in peer-reviewed scientific literature, making it difficult to understand the state of BDD research and existing gaps. Objective:To understand the current state of BDD research by conducting a systematic mapping study that covers studies published from 2006 (when BDD was introduced) to 2021. Method:By following the guidelines for conducting systematic mapping studies in software engineering, we sought to answer research questions on types of venues in which BDD papers have been published, research types, contribution types, studied topics and their evolution, as well as evaluation methods used in published BDD research. Results:The study identified 166 papers which were mapped. Key results include the following: the dominance of conference papers; scarcity of research with insights from the industry; shortage of philosophical papers on BDD; acute shortage of metrics for measuring various aspects of BDD specifications and the processes for producing BDD specifications; the dominance of studies on using BDD for facilitating various software development endeavours, improving the BDD process and associated artefacts, and applying BDD in different contexts; scarcity of studies on using BDD alongside other software techniques and technologies; increase in diversity of studied BDD topics; and notable use of case studies and experiments to study different BDD aspects. Conclusion:The paper improves our understanding of the state of the art of BDD, and highlights important areas of focus for future BDD research.

LanCon-Learn: Learning With Language to Enable Generalization in Multi-Task Manipulation

Robots must be capable of learning from previously solved tasks and generalizing that knowledge to quickly perform new tasks to realize the vision of ubiquitous and useful robot assistance in the real world. While multi-task learning research has produced agents capable of performing multiple tasks, these tasks are often encoded as one-hot goals. In contrast, natural language specifications offer an accessible means both for (1) users to describe a set of new tasks to the robot and (2) robots to reason about the similarities and differences among tasks through language-based task embeddings. Until now, multi-task learning with language has been limited to navigation based tasks and has not been applied to continuous manipulation tasks, requiring precision to grasp and move objects. We present LanCon-Learn, a novel attention-based approach to language-conditioned multi-task learning in manipulation domains to enable learning agents to reason about relationships between skills and task objectives through natural language and interaction. We evaluate LanCon-Learn for both reinforcement learning and imitation learning, across multiple virtual robot domains along with a demonstration on a physical robot. LanCon-Learn achieves up to a 200% improvement in zero-shot task success rate and transfers known skills to novel tasks faster than non-language-based baselines, demonstrating the utility of language for goal specification.

Automatic Generation of Acceptance Test Cases From Use Case Specifications: An NLP-Based Approach

Acceptance testing is a validation activity performed to ensure the conformance of software systems with respect to their functional requirements. In safety critical systems, it plays a crucial role since it is enforced by software standards, which mandate that each requirement be validated by such testing in a clearly traceable manner. Test engineers need to identify all the representative test execution scenarios from requirements, determine the runtime conditions that trigger these scenarios, and finally provide the input data that satisfy these conditions. Given that requirements specifications are typically large and often provided in natural language (e.g., use case specifications), the generation of acceptance test cases tends to be expensive and error-prone. In this paper, we present Use Case Modeling for System-level, Acceptance Tests Generation (UMTG), an approach that supports the generation of executable, system-level, acceptance test cases from requirements specifications in natural language, with the goal of reducing the manual effort required to generate test cases and ensuring requirements coverage. More specifically, UMTG automates the generation of acceptance test cases based on use case specifications and a domain model for the system under test, which are commonly produced in many development environments. Unlike existing approaches, it does not impose strong restrictions on the expressiveness of use case specifications. We rely on recent advances in natural language processing to automatically identify test scenarios and to generate formal constraints that capture conditions triggering the execution of the scenarios, thus enabling the generation of test data. In two industrial case studies, UMTG automatically and correctly translated 95 percent of the use case specification steps into formal constraints required for test data generation; furthermore, it generated test cases that exercise not only all the test scenarios manually implemented by experts, but also some critical scenarios not previously considered.

Proposal of a Method to Generate Classes and Instance Variable Definitions in the VDM++ Specification from Natural Language Specification

Proposal of a Method to Generate Classes and Instance Variable Definitions in the VDM++ Specification from Natural Language Specification

An algorithm of test generation from functional specification using Open IE model and clustering

Automated test coverage is a widespread practice in long-live software development projects for now. According to the test development approach, each automated test should reuse functions implemented in test framework. The provided research is aimed at improving the test framework development approach using natural language processing methods. The algorithm includes the following steps: preparation of test scenarios; transformation of scenario paragraphs to syntax tree using pretrained OpenIE model; test steps comparison with test framework interfaces using GloVe model; transformation of the given semantic tree to the Kotlin language code. The paper contains the description of protype of system automatically generating Kotlin language tests from natural language specification.

Semantic Modeling and Analysis of Natural Language System Requirements

System requirements specify how a system meets stakeholder needs. They are a partial definition of the system under design in natural language that may be restricted in syntax terms. Any natural language specification inevitably lacks a unique interpretation and includes underspecified terms and inconsistencies. If the requirements are not validated early in the system development cycle and refined, as needed, specification flaws may cause costly cycles of corrections in design, implementation and testing. However, validation should be based on a consistent interpretation with respect to a rigorously defined semantic context of the domain of the system. We propose a specification approach that, while sufficiently expressive, it restricts the requirements definition to terms from an ontology with precisely defined concepts and semantic relationships in the domain of the system under design. This enables a series of semantic analyses, which guide the engineer towards improving the requirement specification as well as eliciting tacit knowledge. The problems addressed are prerequisites to enable the derivation of verifiable specifications. We present the results from a case study of modest size from the space system domain, as well as an evaluation of our approach from the user&#x2019;s point of view. The requirement types that have been covered demonstrate the applicability of the approach in an industrial context, although the effectiveness of the analysis depends on pre-existing domain ontologies.

The Specification of Air-to-Air Combat Tactics Using UML Sequence Diagram

Air force air-to-air combat tactics are occurring at a high speed in three-dimensional space. The specification of the tactics requires dealing with a quite amount of information, which makes it a challenge to accurately describe the maneuvering procedure of the tactics. The specification of air-to-air tactics using natural languages is not suitable because of the intrinsic ambiguity of natural languages. Therefore, this paper proposes an approach of using UML Sequence Diagram to describe air-to-air combat tactics. Since the current Sequence Diagram notation is not sufficient to express all aspects of the tactics, we extend the syntax of the Sequence Diagram to accommodate the required features of air-to-air combat tactics. We evaluate the applicability of the extended Sequence Diagram to air-to-air combat tactics using a case example, that is the manned-unmanned teaming combat tactic. The result shows that Sequence Diagram specification is more advantageous than natural language specification in terms of readability, conciseness, and accuracy. However, the expressiveness of the Sequence Diagram is evaluated to be less powerful than natural language, requiring further study to address this issue.

How to bridge natural language specifications to formal modeling: Evaluation and Selection of a set of relevant tools

How to bridge natural language specifications to formal modeling: Evaluation and Selection of a set of relevant tools

Defining behaviorizeable relations to enable inference in semi-automatic program synthesis

Developing and using automatic program synthesis mechanisms within Domain Specific Languages (DSLs) requires transcribing empirical knowledge and specifications to formal models. Since the related expertise is time-consuming to build, software engineers are discouraged from adopting automated synthesis as a development paradigm. New approaches promise a significant reduction of human training effort by synthesizing programs from lax specifications, such as unstructured or semi-structured natural language. However, they do not introduce semantics of logical inference that would help users interact with the synthesis mechanism, for example to treat misinterpreted specifications. To cover this shortcoming, in this work we introduce the concept of behaviorizeable relations, which can be less rigorous than behavioral equivalence –and therefore allow laxer specifications– but still perform inference under implicit logical interpretations of the synthesis domain. Using these relations, we develop a generalized greedy framework that iteratively replaces parts of specifications with equivalent implementations; any behaviorizeable relation can be plugged in this framework to synthesize programs while respecting inference under a corresponding logical system. We finally perform a case study, where we define an example predicate-based behaviorizeable relation that lets our framework linearly combine blocks of Python code based on their comment similarity to natural language specifications. We use the resulting mechanism to demonstrate that implicit inference pertains to understanding how to adjust or state new specifications when lax behaviorizeable relations are employed.

OpenBuildingControl: Digitizing the control delivery from building energy modeling to specification, implementation and formal verification

The current process for specifying, installing and commissioning building control sequences is largely manual and based on ambiguous natural language specifications. It lacks a formal end-to-end quality control and it has been shown not to deliver high performance sequences at scale. While high-performance HVAC control sequences enable significant reductions in energy consumption, errors in implementing the control logic are common even for less advanced sequences. To improve this situation, we present a digitized building control delivery workflow with formal end-to-end verification, a Control Description Language for the digital specification of building control sequences within this workflow, and software tools that enable digitization of this process. Using the process and tools introduced here, mechanical designers can customize, test and improve these sequences within annual energy simulation, store them in a library for use in other projects, and export them for bidding. Control providers can implement the sequences on existing control product lines through code generation. Commissioning providers can formally verify whether as-installed sequences conform to the digital design specification that was exported by the mechanical designer. Moreover, control product development teams can use the reference implementations of these libraries within their product testing to ensure that their products reproduce the behavior of the reference implementations. This paper presents this process, the language and the supporting software, together with examples of all of the above steps. The presented work has given rise to a new proposed standard, ASHRAE 231P, that will allow digitizing the building control delivery process through the standardization of a control-vendor independent format for exchanging control logic that we pioneered through the here presented work.

Specification governor for fault tolerant control of large-scale manufacturing systems

In this paper, a novel approach for fault tolerant control of large-scale manufacturing systems is presented. A specification governor is introduced, which is inspired by the concept of ”reference governor” in feedback control systems. The goal is to modify an infeasible sequence of specifications, which can be used to generate a distributed logic controller, in such a way that the desired product requirements are satisfied in case of faults. The procedure considers redundancies in the manufacturing system and has the capability of keeping the modification of the infeasible specification sequence as less as possible. A modeling process is proposed that automatically adapts fault-free models in form of signal interpreted Petri nets (SIPNs) according to the occurring faults. Based on these faulty models, it is possible to find out which of the natural language specifications in the specification sequence cannot be executed.

A review of the generation of requirements specification in natural language using objects UML models and domain ontology

In the software development life cycle, requirements engineering is the main process that is derived from users by informal interviews written in natural language by requirements engineers (analysts). The requirements may suffer from incompleteness and ambiguity when transformed into formal or semi-formal models that are not well understood by stakeholders. Hence, the stakeholder cannot verify if the formal or semi-formal models satisfy their needs and requirements. Another problem faced by requirements is that when code and/or designs are updated, it is often the case that requirements and specifically the requirements document are not updated. Hence ending with a requirements document not reflecting the implemented software.Generating requirements from the design and/or implementation document is seen by many researchers as a way to address the latter issue. This paper presents a survey of some works undertaken in the field of generation natural language specifications from object UML model using the support of an ontology. and analyzing the robustness and limitations of these existing approaches. This includes studying the generation of natural language from a formal model, review the generation of natural language from ontologies, and finally reviews studies about check to generate natural language from OntoUML.

Proposal of an Algorithm to Generate VDM++ Specification Based on its Grammar by Using Word Lists Extracted from the Natural Language Specification

The natural language includes ambiguous expressions. Vienna Development Method (VDM) is one of methodology on the formal methods to write the specification without ambiguity. Because VDM++ is written by strict grammar, it is difficult to write a VDM++ specification. This research attempts to generate a VDM++ specification automatically from a natural language specification by machine learning. To generate a VDM++ specification, it is necessary to extract words that consist of predicate corresponding to the function and nouns corresponding to variable from the natural language specification. This paper proposes an approach to generate a VDM++ specification based on its grammar from the classified word list. Identifiers are generated from the classified word list, and then the VDM++ specification can be generated by converting them into VDM++ grammar.

METHOD OF FUNCTIONING AND REALIZATION OF ONTOLOGY-BASED INTELLIGENT AGENT FOR PARSING THE NATURAL-LANGUAGE SOFTWARE REQUIREMENTS SPECIFICATIONS

The paper is devoted to development of ontology-based intelligent agent for parsing the natural-language software requirements specifications. The paper develops a method of functioning, structure and realization of ontology-based intelligent agent for parsing the natural-language software requirements specifications. The simplicity of the method of functioning the intelligent agent provides high-speed parsing of natural-language specifications. The developed intelligent agent makes it possible to perform the analysis of natural-language specifications to determine the presence or absence of attributes needed to determine the non-functional characteristics-components of software quality. As a result of such parsing, the proposed intelligent agent determines the number and percentage of missing attributes, displays which attributes are missing for a particular subcharacteristics of non-functional characteristics, and also forms a real ontology for non-functional characteristics. The results of the operation of the intelligent agent are then used to assess the sufficiency of information (attributes) to determine the non-functional characteristics-components of software quality, and can also be used as recommendations for developers of specifications (requirements’ engineers) with the purpose of providing the maximum quantity of attributes, which are necessary to determine the non-functional characteristics, in the specifications of software requirements.