Software Test Bed Research Articles

Detecting system defects of digital devices in the course of malfunction imitation using fuzzing

Aim. The paper proposes approaches to the organisation of testing of digital systems through malfunction imitation for the purpose of ensuring compliance of international and Russian failure and fault resistance standards for the purpose of efficient (in terms of time) detection of software defects as part of mass production of products. The paper proposes a structure and operating algorithm of a hardware and software test bed for malfunction simulation intended for testing a system’s devices. The test bed collects and processes data for fuzzing, hardware error identification, as well as defines the scope of testing.Methods. The paper used basic systems approach, classical methods of the probability theory and mathematical statistics, decision theory, methods of hardware and software testing and development, mathematical theory of fuzzy sets and fuzzy logic.Results. Malfunction simulation algorithms were developed for the purpose of testing hardware and software systems using fuzzing that ensure probabilistic estimation of the termination time of testing with a specified accuracy.Conclusions. The above set of algorithms allows detecting system defects in the process of software and hardware integration into a single system that cause new malfunctions (emergence) that cannot be taken into consideration at the design stage.

Accelerating the Verification of Forward Error Correction Decoders by PCIe FPGA Cards

Pre-silicon forward error correction (FEC) decoding hardware is typically designed using hardware description languages (HDL). Its verification is a hard task due to its intrinsic tendency to correct errors. The generation and injection of millions of random inputs as well as the cross-checking of the corresponding outputs is highly recommended. Using HDL simulations for such work leads to prohibitive execution times. This letter proposes a verification strategy in which the software testbed is executed on a multi-core host and the hardware under verification is prototyped on a PCIe accelerator card. Data are transferred in big blocks of codewords over a high-bandwidth PCIe channel and applied to the decoder using a pipeline management to maximize the use of computational resources and to minimize the verification time. The decoder is replicated with parallel access to DDRs. OpenMP is used to leverage the parallel capabilities of the host and OpenCL, together with Xilinx Runtime Library (XRT), to manage the PCIe FPGA card execution. The results show an important speed-up with respect to HDL simulation and to other prototyping approaches.

The Evryscope Fast Transient Engine: Real-time Detection for Rapidly Evolving Transients

Astrophysical transients with rapid developments on subhour timescales are intrinsically rare. Due to their short durations, events like stellar superflares, optical flashes from gamma-ray bursts, and shock breakouts from young supernovae are difficult to identify on timescales that enable spectroscopic follow-up. This paper presents the Evryscope Fast Transient Engine (EFTE), a new data reduction pipeline that is designed to provide low-latency transient alerts from the Evryscopes—a north–south pair of ultra-wide-field telescopes with an instantaneous footprint covering 38% of the entire sky—and tools for building long-term light curves from Evryscope data. EFTE leverages the optical stability of the Evryscopes by using a simple direct image subtraction routine that is suited to continuously monitoring the transient sky at a cadence of a minute. Candidates are produced within the base Evryscope 2 minute cadence for 98.5% of images, and internally filtered using vetnet, a convolutional neural network real–bogus classifier. EFTE provides an extensible and robust architecture for transient surveys probing similar timescales, and serves as the software test bed for the real-time analysis pipelines and public data distribution systems for the Argus Array, a next-generation all-sky observatory with a data rate 62 times higher than that of Evryscope.

SOGI-FLL-WDCRC Filter for Seamless Control of Microgrid for Optimal Coordination of Conventional and Renewable Energy Resources

An efficient control strategy is presented in this paper to obtain seamless synchronization and improved power quality (PQ) for an AC microgrid feeding highly distorted and unpredictable loads. This paper proposes an improved second-order generalized integrator-frequency locked loop with DC-offset rejection capacity (SOGI-FLL-WDCRC) filter in an AC microgrid to mitigate the grid pollution and the PQ issues caused by the nonlinear loads. The SOGI-FLL-WDCRC filter accurately estimates the positive sequence components (PSCs), phase angle and frequency from the abnormal grid voltages. Thereby, high-quality grid currents and seamless synchronization are realized even at abnormal grid voltages. Moreover, it accurately extracts the fundamental active weight component of the nonlinear load currents, even during dynamic conditions. This results in the generation of accurate reference source currents, thus allowing efficient functioning of the voltage source converter (VSC) and delivering high-quality power to the system by mitigating the load's harmonics, reactive power and unbalanced components. The proposed control is designed in MATLAB/Simulink platform and its performance is studied. A 3.2kW solar PV array integrated to a 220V (line), 50Hz three-phase utility grid/DEG set is designed and developed as a laboratory prototype to validate the simulated responses. Furthermore, the effectiveness of the proposed control is verified and compared with conventional controls using dSPACE-1202 MicroLabBox on a laboratory test bed. The results obtained from the simulation software and laboratory test bed confirm the effectiveness of the proposed control in estimating the accurate PSCs of grid voltages and oscillations free weight components with higher convergence and better stability under abnormal grid and load conditions.

Development of simulation-based testing environment for safety-critical software

Abstract Recently, a software program has been used in nuclear power plants (NPPs) to digitalize many instrumentation and control systems. To guarantee NPP safety, the reliability of the software used in safety-critical instrumentation and control systems must be quantified and verified with proper test cases and test environment. In this study, a software testing method using a simulation-based software test bed is proposed. The test bed is developed by emulating the microprocessor architecture of the programmable logic controller used in NPP safety-critical applications and capturing its behavior at each machine instruction. The effectiveness of the proposed method is demonstrated via a case study. To represent the possible states of software input and the internal variables that contribute to generating a dedicated safety signal, the software test cases are developed in consideration of the digital characteristics of the target system and the plant dynamics. The method provides a practical way to conduct exhaustive software testing, which can prove the software to be error free and minimize the uncertainty in software reliability quantification. Compared with existing testing methods, it can effectively reduce the software testing effort by emulating the programmable logic controller behavior at the machine level.

A Game Theoretic Software Test-bed for Cyber Security Analysis of Critical Infrastructure

<p class="p1">National critical infrastructures are vital to the functioning of modern societies and economies. The dependence on these infrastructures is so succinct that their incapacitation or destruction has a debilitating and cascading effect on national security. Critical infrastructure sectors ranging from financial services to power and transportation to communications and health care, all depend on massive information communication technology networks. Cyberspace is composed of numerous interconnected computers, servers and databases that hold critical data and allow critical infrastructures to function. Securing critical data in a cyberspace that holds against growing and evolving cyber threats is an important focus area for most countries across the world. A novel approach is proposed to assess the vulnerabilities of own networks against adversarial attackers, where the adversary’s perception of strengths and vulnerabilities are modelled using game theoretic techniques. The proposed game theoretic framework models the uncertainties of information with the players (attackers and defenders) in terms of their information sets and their behaviour is modelled and assessed using a probability and belief function framework. The attack-defence scenarios are exercised on a virtual cyber warfare test-bed to assess and evaluate vulnerability of cyber systems. Optimal strategies for attack and defence are computed for the players which are validated using simulation experiments on the cyber war-games testbed, the results of which are used for security analyses.</p>

Evaluation of visual feedback techniques for virtual grasping with bare hands using Leap Motion and Oculus Rift

Bare hand interaction (BHI) allows users to use their hands and fingers to interact with digital content without any attached devices or accessories. For BHI to realize widespread adoption, interaction techniques for fundamental operations, like grasp-and-release, need to be identified and optimized. This paper presents a controlled usability evaluation of four common visual feedback techniques in grasp-and-release tasks using bare hand interaction (BHI). The techniques are ‘object coloring,’ ‘connecting line,’ ‘shadow’ and ‘object halo.’ The usability was examined in terms of task time, accuracy, errors and user satisfaction. A software test bed was developed for two interface configurations: using the Leap Motion controller alone (desktop configuration) and using the Leap with Oculus Rift (virtual reality (VR) configuration). Participants (n 32) performed four trials × five feedback techniques × two UI (user interface) configurations, i.e., a total of 1280 trials. The results can be summarized into: (a) user performance is significantly better in the VR configuration compared to the desktop; (b) coloring techniques for visual feedback (‘object coloring’ and ‘object halo’) are more usable than ‘connecting line’ regardless of UI; (c) in the VR, coloring techniques remain more usable, while in the desktop interface the ‘shadow’ technique is also usable and preferred by users, (d) the ‘connecting line’ technique often distracts users from grasp-and-release tasks on static targets. (e) Some visual feedback is always preferred by users than none in both VR and desktop. We discuss these findings in terms of design recommendations for bare hands interactions that involve grasp-and-release tasks.

Usage of Software Test Beds for Testing the New Technologies

Usage of Software Test Beds for Testing the New Technologies

Autonomous aerobraking for low-cost interplanetary missions

Aerobraking has previously been used to reduce the propellant required to deliver an orbiter to its desired final orbit. In principle, aerobraking should be possible around any target planet or moon having sufficient atmosphere to permit atmospheric drag to provide a portion of the mission ΔV, in lieu of supplying all of the required ΔV propulsively. The spacecraft is flown through the upper atmosphere of the target using multiple passes, ensuring that the dynamic pressure and thermal loads remain within the spacecraft's design parameters. NASA has successfully conducted aerobraking operations four times, once at Venus and three times at Mars. While aerobraking reduces the fuel required, it does so at the expense of time (typically 3–6 months), continuous Deep Space Network (DSN) coverage, and a large ground staff. These factors can result in aerobraking being a very expensive operational phase of the mission. However, aerobraking has matured to the point that much of the daily operation could potentially be performed autonomously onboard the spacecraft, thereby reducing the required ground support and attendant aerobraking related costs. To facilitate a lower-risk transition from ground processing to an autonomous capability, the NASA Engineering and Safety Center (NESC) has assembled a team of experts in aerobraking and interplanetary guidance and control to develop a high-fidelity, flight-like simulation. This simulation will be used to demonstrate the overall feasibility while exploring the potential for staff and DSN coverage reductions that autonomous aerobraking might provide. This paper reviews the various elements of autonomous aerobraking and presents an overview of the various models and algorithms that must be transformed from the current ground processing methodology to a flight-like environment. Additionally the high-fidelity flight software test bed, being developed from models used in a recent interplanetary mission, will be summarized.

Real‐time, decentralized and bio‐inspired topology control for holonomic autonomous vehicles

PurposeThis paper aims to present an approach for a bio‐inspired decentralization topology control mechanism, called force‐based genetic algorithm (FGA), where a genetic algorithm (GA) is run by each holonomic autonomous vehicle (HAV) in a mobile ad hoc network (MANET) as software agent to achieve a uniform spread of HAVs and to provide a fully connected network over an unknown geographical terrain. An HAV runs its own FGA to decide its next movement direction and speed based on local neighborhood information, such as obstacles and the number of neighbors, without a centralized control unit or global knowledge.Design/methodology/approachThe objective function used in FGA is inspired by the equilibrium of the molecules in physics where each molecule tries to be in the balanced position to spend minimum energy to maintain its position. In this approach, a virtual force is assumed to be applied by the neighboring HAVs to a given HAV. At equilibrium, the aggregate virtual force applied to an HAV by its neighbors should sum up to zero. If the aggregate virtual force is not zero, it is used as a fitness value for the HAV. The value of this virtual force depends on the number of neighbors within the communication range of Rcom and the distance among them. Each chromosome in our GA‐based framework is composed of speed and movement direction. The FGA is independently run by each HAV as a topology control mechanism and only utilizes information from neighbors and local terrain to make movement and speed decisions to converge towards a uniform distribution of HAVs. The authors developed an analytical model, simulation software and several testbeds to study the convergence properties of the FGA.FindingsThe paper finds that coverage‐centric, bio‐inspired, mobile node deployment algorithm ensures effective sensing coverage for each mobile node after initial deployment. The FGA is also an energy‐aware self‐organization framework since it reduces energy consumption by eliminating unnecessary excessive movements. Fault‐tolerance is another important feature of the GA‐based approach since the FGA is resilient to losses and malfunctions of HAVs. Furthermore, the analytical results show that the authors' bio‐inspired approach is effective in terms of convergence speed and area coverage uniformity. As seen from the experimental results, the FGA delivers promising results for uniform autonomous mobile node distribution over an unknown geographical terrain.Originality/valueThe proposed decentralized and bio‐inspired approach for autonomous mobile nodes can be used as a real‐time topology control mechanism for commercial and military applications since it adapts to local environment rapidly but does not require global network knowledge.

A conjugate gradient algorithm for the astrometric core solution ofGaia

The ESA space astrometry mission Gaia, planned to be launched in 2013, has been designed to make angular measurements on a global scale with micro-arcsecond accuracy. A key component of the data processing for Gaia is the astrometric core solution, which must implement an efficient and accurate numerical algorithm to solve the resulting, extremely large least-squares problem. The Astrometric Global Iterative Solution (AGIS) is a framework that allows to implement a range of different iterative solution schemes suitable for a scanning astrometric satellite. In order to find a computationally efficient and numerically accurate iteration scheme for the astrometric solution, compatible with the AGIS framework, we study an adaptation of the classical conjugate gradient (CG) algorithm, and compare it to the so-called simple iteration (SI) scheme that was previously known to converge for this problem, although very slowly. The different schemes are implemented within a software test bed for AGIS known as AGISLab, which allows to define, simulate and study scaled astrometric core solutions. After successful testing in AGISLab, the CG scheme has been implemented also in AGIS. The two algorithms CG and SI eventually converge to identical solutions, to within the numerical noise (of the order of 0.00001 micro-arcsec). These solutions are independent of the starting values (initial star catalogue), and we conclude that they are equivalent to a rigorous least-squares estimation of the astrometric parameters. The CG scheme converges up to a factor four faster than SI in the tested cases, and in particular spatially correlated truncation errors are much more efficiently damped out with the CG scheme.

Industry and Government Space Weather Experts Meet

The fifth annual NOAA Space Weather Prediction Center (SWPC)-Commercial Space Weather Interest Group (CSWIG) Summit was held on 28 April 2011 in Boulder, Colo., in association with the 2011 Space Weather Workshop. Interest was high, in the United States and internationally, in potential space weather impacts on many aspects of everyday life because of the increased vulnerability of technological systems and the possibility that a major space weather event may occur as the twenty-fourth solar cycle begins to progress toward solar maximum. Industry and government space weather experts participated in the summit. Devrie Intriligator (Carmel Research Center, Inc.) and W. Kent Tobiska (Space Environment Technologies (SET)) served as cochairs. Tom Bogdan, Director of SWPC, summarized SWPC’s current fiscal situation and future plans. SWPC has been successful in educating members of the executive branch and other government officials about space weather. President Obama’s proposed fiscal year (FY) 2012 budget includes Department of Defense and NOAA funding for the Deep Space Climate Observatory (DSCOVR), the replacement for the Advanced Composition Explorer (ACE) spacecraft, which warns of approaching interplanetary disturbances from the Sun. It is crucial for Congress to appropriate this funding if DSCOVR is to be launched on schedule in 2014. The Constellation Observing System for Meteorology, Ionosphere, and Climate (COSMIC-2) mission is also in the proposed FY 2012 budget. Starting in early fall 2011, SWPC is planning to have two forecasters on duty 24 hours a day, 7 days a week. SWPC’s top operational priorities include an improved WangSheeleyArge (WSA) model of the Sun/solar wind interface and a Ground Induced Current regional model. A high priority is improved forecasting of the interplanetary magnetic field characteristics of a coronal mass ejection that hits Earth. David Bouwer (SET) summarized SWPC’s External Space Weather Data Store (E-SWDS) and other data/model sources. E-SWDS is the most reliable data source with the fewest changes/interruptions in service, format, and parameters. CSWIG looks forward to when E-SWDS also will include improved model outputs (e.g., ENLIL, WSA, etc.). Intriligator summarized concerns on the quality and quantity of future data including those from ACE and National Polar-Orbiting Operational Environmental Satellite System (NPOESS) replacements, the Geostationary Operational Environmental Satellite (GOES), etc. Space weather research, operations, and forecasts are data limited, and longterm monitoring of solar, interplanetary, and near-Earth regions is crucial. NOAA is tasked with this monitoring. Stable funding of long-term spacecraft lines (e.g., GOES) is needed. Michael Bonadonna (Office of the Federal Coordinator for Meteorology (OFCM)) updated the group on the National Space Weather Program (NSWP) and OFCM activities. One of the near-term priorities of OFCM is to improve its ability to work with the commercial sector. At present, there is no mechanism for the commercial sector to participate in the NSWP Council or other high-level NSWP activities. The Office of Science and Technology Policy has requested that NSWP assess U.S. space weather capabilities now and 10 years in the future. Tobiska discussed CSWIG participation in software test beds and Small Business Innovative Research opportunities as ways of leveraging funding. He also discussed moving toward space weather common practices and a quality assurance program including standards as developed by the American Institute of Aeronautics and Astronautics (http://spacewx.com, see the “Standards” link under “Resources”). Geoff Crowley (Atmospheric and Space Technology Research Associates) reported on the American Commercial Space Weather Association (ACSWA), the space weather association that was unanimously formed last year at the SWPCCSWIG Summit. The ACSWA executive committee (Crowley, Intriligator, Robert Schunk, and Tobiska) had earlier drafted the mission statement for ACSWA and established a Web site (http://sw01.spaceweather.usu.edu/ACSWA/ landing.html), which was unveiled at the summit meeting. Bonadonna and Bogdan enthusiastically supported ACSWA. Genene Fisher

Seismic FRP retrofit of circular single-column bents using a ductility wrap envelope to alter failure modes

The target displacement ductility requirements for circular reinforced concrete (R/C) single-column bridge bents are considered using a proposed multi-failure mode algorithm to determine the required thickness of fiber reinforced polymer wraps (FRPs). The procedure is developed using two in-house computer algorithms, PACCC (Plastic Analysis of Circular Concrete Columns) and PACCC-FRP, to generate a moment-curvature analysis using circular segments and subsequent failure mode predictions in single-column bents for both FRP-wrapped and unwrapped circular R/C sections. The results of the study were verified against published experimental tests at the ultimate force-deflection states of R/C sections and also against three commercial “software test beds”. The study uses PACCC-FRP to show that single-columns experiencing a brittle failure may be retrofitted with FRP wraps in order to increase the displacement ductility and satisfy target ductility values within the Ductility Wrap Envelope (DWE), or wrap-saturation level, as established herein.

Analytical Models and Guidelines for the Ductility Enhancement of Circular Reinforced Concrete Single-Column Bents Using Fiber-Reinforced Polymers

The target displacement ductility requirements for circular RC single-column bridge bents are considered using a proposed multifailure mode algorithm to determine the required thickness of fiber-reinforced polymer wraps FRPs. The procedure is developed using two in-house computer algorithms, PACCC plastic analysis of circular concrete columns and PACCC-FRP, to generate a moment- curvature analysis using circular segment slices and subsequent failure mode predictions in single-column bents for both FRP-wrapped and unwrapped circular RC sections. The results of the study showed good comparison to published experimental tests at the ultimate force-deflection states of RC sections and against three commercial software test beds. The study uses PACCC-FRP to show that single columns experiencing a brittle failure may be retrofitted with FRP wraps in order to increase the displacement ductility and satisfy target ductility values within the ductility wrap envelope, or wrap-saturation level, as established herein. DOI: 10.1061/ASCEBE.1943-5592.0000136 CE Database subject headings: Brittle failures; Failure modes; Concrete columns; Column beams; Fiber reinforced polymer; Lateral displacement; Ductility. Author keywords: Brittle failure; Failure modes; Concrete columns; Fiber-reinforced polymers; Lateral displacement; Ductility.

Experimenting with software testbeds for evaluating new technologies

The evolution of a new technology depends upon a good theoretical basis for developing the technology, as well as upon its experimental validation. In order to provide for this experimentation, we have investigated the creation of a software testbed and the feasibility of using the same testbed for experimenting with a broad set of technologies. The testbed is a set of programs, data, and supporting documentation that allows researchers to test their new technology on a standard software platform. An important component of this testbed is the Unified Model of Dependability (UMD), which was used to elicit dependability requirements for the testbed software. With a collection of seeded faults and known issues of the target system, we are able to determine if a new technology is adept at uncovering defects or providing other aids proposed by its developers. In this paper, we present the Tactical Separation Assisted Flight Environment (TSAFE) testbed environment for which we modeled and evaluated dependability requirements and defined faults to be seeded for experimentation. We describe two completed experiments that we conducted on the testbed. The first experiment studies a technology that identifies architectural violations and evaluates its ability to detect the violations. The second experiment studies model checking as part of design for verification. We conclude by describing ongoing experimental work studying testing, using the same testbed. Our conclusion is that even though these three experiments are very different in terms of the studied technology, using and re-using the same testbed is beneficial and cost effective.

Kansei: a high-fidelity sensing testbed

Hardware and software testbeds are becoming the preferred basis for experimenting with embedded wireless sensor network applications. The Kansei testbed at the Ohio State University features a heterogeneous hardware infrastructure, with dedicated node resources for local computation, storage, data retrieval, and back-channel communication. Kansei includes a time-accurate hybrid simulation engine that uses testbed hardware resources to simulate large arrays. It supports high-fidelity sensor data generation as well as real-time data and event injection. The testbed also includes software components and an associated job-control language for complex multi-tier experiments.

Design of autonomous photovoltaic power plant for telecommunication relay station

The design and prototyping of an autonomous photovoltaic power generation system using the Virtual Test Bed software environment is presented. The comprehensive design of the photovoltaic power plant is very important for achieving the best performance, durability and reliability of the system. The goal of the design is to optimise the system performance by accounting for various data available at the design time. These data may include statistical data of weather conditions at particular geographic locations, the statistics of the load profiles and, finally, the projected outlook of the system use. The design of the autonomous photovoltaic power system presented in this work includes establishing physics-based models for components, prototyping system architectures, implementing control algorithms, testing and evaluating the system performance using the actual weather data at a particular geographical location. The dynamic performance of the photovoltaic power system and its components are revealed in great detail through the system-level simulation. This knowledge is used for verification and improvements of the system design in order to achieve the best system power capability, efficiency and size.

VTB-Based Design of a Standalone Photovoltaic Power System

Design and prototyping of a standalone photovoltaic power generation system using the Virtual Test Bed software environment is presented. The design includes establishing physics-based models for components, prototyping system architectures, implementing control algorithms, testing and evaluating the system performance using the actual weather data at a particular geographical location. The dynamic performance of the photovoltaic power system and its components are revealed in great details through the system level simulation. This knowledge is used for verification and improvements of the system design in order to achieve the best system power capability, efficiency and size.

VTB-Based Design of a Standalone Photovoltaic Power System

Abstract Design and prototyping of a standalone photovoltaic power generation system using the Virtual Test Bed software environment is presented. The design includes establishing physics-based models for components, prototyping system architectures, implementing control algorithms, testing and evaluating the system performance using the actual weather data at a particular geographical location. The dynamic performance of the photovoltaic power system and its components are revealed in great details through the system level simulation. This knowledge is used for verification and improvements of the system design in order to achieve the best system power capability, efficiency and size.

CAD-based integrated tolerancing system

Assembly tolerance analysis involves the determination of tolerances of critical dimensions. This paper addresses two basic areas in tolerance analysis and allocation. (1) Dimension normalization, which identifies the relevant functional dimensions. A new algorithm called the dimension block diagram (DBD) algorithm for dimension normalization is presented. (2) Dimension representation, which provides a robust framework for representing the tolerance chain. A new dimension representation framework using Assembly Dimensional Tolerance (ADT) tree is presented. The two methodologies are automated and integrated within the framework of a CAD system using an Expert System. A computer test bed software has been developed demonstrating the viability of this approach.