Hardware Testbeds Research Articles

Demonstrating autonomous controls on hardware test beds is a necessity for successful missions to Mars and beyond

NASA and the Department of Defense are planning for a mission to Mars in the 2030s–2040s using nuclear thermal propulsion (NTP). NTP uses a nuclear reactor to heat flowing hydrogen and create thrust. A serious concern for crewed and uncrewed missions to Mars is the loss of reactor control. The reactor startup and initial rocket impulse are initiated in cislunar or near-earth orbital regions; therefore, radio communications between ground control and the NTP engine should occur in real time. However, radio communications can take more than 20 min, depending on planet positions, to reach Mars orbiters from ground control. To address this delay, local autonomous controls are implemented onboard the NTP engine to ensure acceptable operation. However, autonomous controls have not been demonstrated or implemented in research or power reactor contexts because of safety and reliability concerns. To enable autonomous controls development, demonstration, and validation, Oak Ridge National Laboratory has created a nonnuclear hardware-in-the-loop test bed. Sensors throughout the test bed relay system status and hardware response to the user control algorithm, including measurements of temperature, flow, pressure of a loop, control drum position, and drum speed. This paper discusses the development of this facility and user accessibility.

Open-source emulation-based test environment to settle O-RAN-compliant trials

Experimental tools are a key factor in both academic and industrial research communities to create design evaluations of new networking technologies that involve troubleshooting or changing the planning of deployed networks. Physical Software-Defined Radio (SDR) experimental platforms enable a design solution for the quick prototyping of wireless communication systems. However, SDR-based experimental platforms incur high costs, which leads to scalability limitations in the experimental settings. Having said this, network simulators, emulators, and new testbeds have attracted increasing attention. Emulation-based research prototyping can be distinguished from real communication networks and SDR-based platforms by allowing a tradeoff between cost and flexibility. This paper examines the Mininet-RAN emulation tool, which, as well as Radio Access Network (RAN) modeling, provides a way to test Open RAN Intelligent Controller (RIC) services without the need to deploy an entire RAN infrastructure. The Mininet-RAN creates virtual network elements, such as hosts, L2/L3 devices, controllers, and links, by combining some of the best emulator features, hardware testbeds, and simulators. By running the current code of standard practice Unix/Linux network applications and network stack, the Mininet-RAN enables real-world network data traffic patterns to be delivered to the RIC, regarding the most significant aspect of the dynamic generation of wireless system's KPIs. We provide the basic code of Mininet-RAN for the first two O-RAN Alliance-defined use cases involving V2X and UAV. The xApps are being implemented in O-RAN SC near-RT RIC, with Mininet-RAN which provides a closed-loop validation environment.

Alternating-Current Microgrid Testbed Built with Low-Cost Modular Hardware

With the growing popularity of microgrids for alternative energy management, there is demand for tools that allow us to study the effect of microgrids in distributed power systems. Popular methods involve software simulation and prototype validation with physical hardware. Simulations often do not capture the complex interactions, and combinations of software simulations with hardware testbeds promise to give a more accurate picture. These testbeds, however, usually aim at the validation of hardware for industrial-scale use, which makes them expensive and not readily accessible. To fill the gap between full-scale hardware and software simulation, we propose a modular lab-scale grid model at a 1:100 power scale over residential single-phase networks with 12 V AC and 60 Hz grid voltage. We present different modules-power sources, inverters, demanders, grid monitors, and grid-to-grid bridges-that can be assembled into distributed grids of almost arbitrary complexity. The model voltage poses no electrical hazards, and microgrids can readily be assembled with an open power line model. Unlike a prior DC-based grid testbed, the proposed AC model allows us to examine additional aspects, such as frequency, phase, active and apparent power, and reactive loads. Grid metrics, including the discretely sampled voltage and current waveforms, can be collected and sent to higher-tier grid management systems. We integrated the modules with Beagle Bone micro-PCs, which in turn connect any such microgrid with an emulation platform built on CORE (Common Open Research Emulator) and the Gridlab-D power simulator, thereby allowing hybrid software/hardware simulations. Our grid modules were shown to fully operate in this environment. Through the CORE system, multitiered control and even remote grid management is possible. However, we also found that the AC waveform poses design challenges that require us to balance accurate emulation (most notably with respect to harmonic distortion) with per-module costs.

Modeling the Training Iteration Time for Heterogeneous Distributed Deep Learning Systems

Distributed deep learning systems effectively respond to the increasing demand for large-scale data processing in recent years. However, the significant investment in building distributed learning systems with powerful computing nodes places a huge financial burden on developers and researchers. It will be good to predict the precise benefit, i.e., how many times of speedup it can get compared with training on single machine (or a few), before actually building such big learning systems. To address this problem, this paper presents a novel performance model on training iteration time for heterogeneous distributed deep learning systems based on the characteristics of the parameter server (PS) system with bulk synchronous parallel (BSP) synchronization style. The accuracy of our performance model is demonstrated by comparing real measurement results on TensorFlow when training different neural networks with various kinds of hardware testbeds: the prediction accuracy is higher than 90% in most cases.

From the Lab to People's Home: Lessons from Accessing Blind Participants' Interactions via Smart Glasses in Remote Studies.

Researchers have adopted remote methods, such as online surveys and video conferencing, to overcome challenges in conducting in-person usability testing, such as participation, user representation, and safety. However, remote user evaluation on hardware testbeds is limited, especially for blind participants, as such methods restrict access to observations of user interactions. We employ smart glasses in usability testing with blind people and share our lessons from a case study conducted in blind participants' homes (N = 12), where the experimenter can access participants' activities via dual video conferencing: a third-person view via a laptop camera and a first-person view via smart glasses worn by the participant. We show that smart glasses hold potential for observing participants' interactions with smartphone testbeds remotely; on average 58.7% of the interactions were fully captured via the first-person view compared to 3.7% via the third-person. However, this gain is not uniform across participants as it is susceptible to head movements orienting the ear towards a sound source, which highlights the need for a more inclusive camera form factor. We also share our lessons learned when it comes to dealing with lack of screen reader support in smart glasses, a rapidly draining battery, and Internet connectivity in remote studies with blind participants.

Shape of the Cloak: Formal Analysis of Clock Skew-Based Intrusion Detection System in Controller Area Networks

This paper presents a new masquerade attack called the cloaking attack and provides formal analyses for clock skew-based intrusion detection systems (IDSs) that detect masquerade attacks in the controller area network (CAN) in automobiles. In the cloaking attack, the adversary manipulates the message inter-transmission times of spoofed messages by adding delays so as to emulate a desired clock skew and avoid detection. In order to predict and characterize the impact of the cloaking attack in terms of the attack success probability on a given CAN bus and IDS, we develop formal models for two clock skew-based IDSs, i.e., the state-of-the-art (SOTA) IDS and its adaptation to the widely used network time protocol (NTP), using parameters of the attacker, the detector, and the hardware platform. To the best of our knowledge, this is the first paper that provides formal analyses of clock skew-based IDSs in automotive CAN. We implement the cloaking attack on two hardware testbeds, a prototype and a real vehicle (the University of Washington EcoCAR), and demonstrate its effectiveness against both the SOTA and NTP-based IDSs. By comparing each predicted attack success probability curve against its experimental curve, we find that the average prediction error is within 3.0% for the SOTA IDS and 5.7% for the NTP-based IDS.

Real-Time Testing Approaches for Microgrids

Microgrids (MGs) are seen today as the solution to the challenge of meeting the ever-increasing demand of energy which is clean. A variety of challenges need to be overcome for these MGs to be successful varying from stability, power quality to protection, and management. Since the various control strategies are actively being researched and emerging challenges still being addressed, there has been a lot of interest related to the aspect of testing. The ever-increasing complexity of control mechanisms invokes the need for safer, faster yet reliable testing methods, which do not compromise on the degree of detail and at the same time are economical and require the minimum testing time. Off-line simulation, real-time (RT) simulation, hardware in the loop simulation, RT emulation, hardware test beds, pilot plants, and other integrated approaches are the major methods for testing in MGs. This paper gives an overview of the testing techniques and discusses the trends in the various technologies employed for testing MGs. First, the various methods are briefly discussed and a comparison is provided. Next, the various research efforts carried out on the aspect of MG testing are categorized and presented one-by-one. Furthermore, new methodologies and emerging approaches for the testing have also been highlighted, and the challenges and future research avenues have been presented.

Design verification enhancement of field programmable gate array-based safety-critical I&C system of nuclear power plant

Safety-critical instrumentation and control (I&C) system in nuclear power plant (NPP) implemented on programmable logic controllers (PLCs) plays a vital role in safe operation of the plant. The challenges such as fast obsolescence, the vulnerability to cyber-attack, and other related issues of software systems have currently led to the consideration of field programmable gate arrays (FPGAs) as an alternative to PLCs because of their advantages and hardware related benefits. However, safety analysis for FPGA-based I&C systems, and verification and validation (V&V) assessments still remain important issues to be resolved, which are now become a global research point of interests. In this work, we proposed a systematic design and verification strategies from start to ready-to-use in form of model-based approaches for FPGA-based reactor protection system (RPS) that can lead to the enhancement of the design verification and validation processes. The proposed methodology stages are requirement analysis, enhanced functional flow block diagram (EFFBD) models, finite state machine with data path (FSMD) models, hardware description language (HDL) code development, and design verifications. The design verification stage includes unit test – Very high speed integrated circuit Hardware Description Language (VHDL) test and modified condition decision coverage (MC/DC) test, module test – MATLAB/Simulink Co-simulation test, and integration test – FPGA hardware test beds. To prove the adequacy of the proposed approaches, the design architect that focused on the RPS bistable trip logics which are the safety-critical functions of RPS are designed, analyzed, verified and discussed, using bistable fixed setpoint trip logic algorithms as case study. The results showed that the proposed approaches can enhance the design verification processes alongside the reduction in rigorous V&V tasks of FPGA-based safety-critical I&C system for NPP.

ScorePlus

We present ScorePlus, a software-hardware hybrid and federated experiment environment for Smart Grid. ScorePlus incorporates both a software emulator and hardware testbed, such that they all follow the same architecture, and the same Smart Grid application program can be tested on either of them without any modification; ScorePlus provides a federated environment such that multiple software emulators and hardware testbeds at different locations are able to connect and form a unified Smart Grid system; ScorePlus software is encapsulated as a resource plugin in the OpenStack cloud computing platform, such that it supports massive deployments with large-scale test cases in cloud infrastructure.

Energy conservation for GPU–CPU architectures with dynamic workload division and frequency scaling

In recent years, GPU–CPU heterogeneous architectures have been increasingly adopted in high performance computing, because of their capabilities of providing high computational throughput. However, the energy consumption is a major concern due to the large scale of such kind of systems. There are a few existing efforts that try to lower the energy consumption of GPU–CPU architectures, but they address either GPU or CPU in an isolated manner and thus cannot achieve maximized energy savings. In this paper, we propose GreenGPU, a holistic energy management framework for GPU–CPU heterogeneous architectures. Our solution features a two-tier design. In the first tier, GreenGPU dynamically splits and distributes workloads to GPU and CPU based on the workload characteristics, such that both sides can finish approximately at the same time. We comparatively discuss four dynamic workload allocation algorithms: a Simple Heuristic with fixed step size, an Improved Heuristic with adaptive step size, and two binary search-style algorithms. As a result, the energy wasted on idling and waiting for the slower side to finish is minimized. In the second tier, GreenGPU dynamically throttles the frequencies of GPU cores and memory in a coordinated manner, based on their utilizations, for maximized energy savings with only marginal performance degradation. Likewise, the frequency and voltage of the CPU are scaled similarly. We implement GreenGPU using the CUDA framework on two real hardware testbeds. Experiment results show that GreenGPU achieves 21.04% average energy savings and outperforms several well-designed baselines.

Models for wireless H.264 video-on-demand services using self-similarity and heavy-tails

In this work two video-on-demand (VoD) capacity models for H.264 video traces transmitted using 802.11g are proposed, one based on a self-similar traffic distribution and the other one based in the summation of a large number of Pareto distributed random variables. To ascertain the validity of using such modeling techniques a statistical analysis was performed where it was found that H.264 video traces exhibit self-similarity and heavy-tailed properties, as previous video formats that also use variable bit rate encoding. The models were evaluated against trace based simulations using ns-3 and results from hardware testbeds from other works. The model based on Pareto distributions gives a lower bound on a wide range of buffer sizes, while the model based on self-similarity provides a closer approximation to the user load when buffer size is high. The results show that the models can approximate the maximum user load for H.264 transmission on a local area VoD system and that they depend on the access point buffer size and the desired quality of service expressed as packet-loss probability.

A Scalable Context-Aware Objective Function (SCAOF) of Routing Protocol for Agricultural Low-Power and Lossy Networks (RPAL).

In recent years, IoT (Internet of Things) technologies have seen great advances, particularly, the IPv6 Routing Protocol for Low-power and Lossy Networks (RPL), which provides a powerful and flexible routing framework that can be applied in a variety of application scenarios. In this context, as an important role of IoT, Wireless Sensor Networks (WSNs) can utilize RPL to design efficient routing protocols for a specific application to increase the ubiquity of networks with resource-constrained WSN nodes that are low-cost and easy to deploy. In this article, our work starts with the description of Agricultural Low-power and Lossy Networks (A-LLNs) complying with the LLN framework, and to clarify the requirements of this application-oriented routing solution. After a brief review of existing optimization techniques for RPL, our contribution is dedicated to a Scalable Context-Aware Objective Function (SCAOF) that can adapt RPL to the environmental monitoring of A-LLNs, through combining energy-aware, reliability-aware, robustness-aware and resource-aware contexts according to the composite routing metrics approach. The correct behavior of this enhanced RPL version (RPAL) was verified by performance evaluations on both simulation and field tests. The obtained experimental results confirm that SCAOF can deliver the desired advantages on network lifetime extension, and high reliability and efficiency in different simulation scenarios and hardware testbeds.

CarSpeak

This paper introduces CarSpeak, a communication system for autonomous driving. CarSpeak enables a car to query and access sensory information captured by other cars in a manner similar to how it accesses information from its local sensors. CarSpeak adopts a content-centric approach where information objects -- i.e., regions along the road -- are first class citizens. It names and accesses road regions using a multi-resolution system, which allows it to scale the amount of transmitted data with the available bandwidth. CarSpeak also changes the MAC protocol so that, instead of having nodes contend for the medium, contention is between road regions, and the medium share assigned to any region depends on the number of cars interested in that region. CarSpeak is implemented in a state-of-the-art autonomous driving system and tested on indoor and outdoor hardware testbeds including an autonomous golf car and 10 iRobot Create robots. In comparison with a baseline that directly uses 802.11, CarSpeak reduces the time for navigating around obstacles by 2.4x, and reduces the probability of a collision due to limited visibility by 14x.

A Survey of Communications and Networking Technologies for Energy Management in Buildings and Home Automation

With the exploding power consumption in private households and increasing environmental and regulatory restraints, the need to improve the overall efficiency of electrical networks has never been greater. That being said, the most efficient way to minimize the power consumption is by voluntary mitigation of home electric energy consumption, based on energy-awareness and automatic or manual reduction of standby power of idling home appliances. Deploying bi-directional smart meters and home energy management (HEM) agents that provision real-time usage monitoring and remote control, will enable HEM in “smart households.” Furthermore, the traditionally inelastic demand curve has began to change, and these emerging HEM technologies enable consumers (industrial to residential) to respond to the energy market behavior to reduce their consumption at peak prices, to supply reserves on a as-needed basis, and to reduce demand on the electric grid. Because the development of smart grid-related activities has resulted in an increased interest in demand response (DR) and demand side management (DSM) programs, this paper presents some popular DR and DSM initiatives that include planning, implementation and evaluation techniques for reducing energy consumption and peak electricity demand. The paper then focuses on reviewing and distinguishing the various state-of-the-art HEM control and networking technologies, and outlines directions for promoting the shift towards a society with low energy demand and low greenhouse gas emissions. The paper also surveys the existing software and hardware tools, platforms, and test beds for evaluating the performance of the information and communications technologies that are at the core of future smart grids. It is envisioned that this paper will inspire future research and design efforts in developing standardized and user-friendly smart energy monitoring systems that are suitable for wide scale deployment in homes.

Testbed‐assisted learning for digital communications courses

AbstractWe introduce testbed‐assisted learning as an effective means for teaching digital communications. Laboratory teaching activities of digital communications courses benefit very much from utilizing a hardware testbed, since it greatly facilitates the understanding of very important effects introduced by real‐world transceivers. We overcome the main drawback of communications hardware, that is, the cumbersome low‐level programming interfaces provided by hardware manufacturers, by introducing a distributed multilayer software architecture. This architecture provides different abstraction levels to access hardware testbeds, releasing students from the low‐level interaction with the hardware. Also, the distributed nature of this architecture results in a high flexibility of operation. This way, students can focus on learning communications topics without devoting any time to low‐level programming, that is usually out of the scope of digital communications courses. Thanks to testbed‐assisted learning, they are able to perform illustrative experiments to understand digital communications concepts (e.g., source coding, modulation, space‐time coding, etc.) and to test algorithms without developing a new program from scratch, speeding up both the implementation and the debugging tasks. However, those students interested in hardware implementations can use the software architecture to access and interact with lower programming levels until they are as close as possible to the hardware. © 2010 Wiley Periodicals, Inc. Comput Appl Eng Educ 21: 539–549, 2013

Performance Characterization of a Rover Navigation Algorithm Using Large‐Scale Simulation

Autonomous rover navigation is a critical technology for robotic exploration of Mars. Simulation allows more extensive testing of such technologies than would be possible with hardware test beds alone. A large number of simulations, running in parallel, can test an algorithm under many different operating conditions to quickly identify the operational envelope of the technology and identify failure modes that were not discovered in more limited testing. GESTALT is the autonomous navigation algorithm developed for NASA′s Mars rovers. ROAMS is a rover simulator developed to support the Mars program. We have integrated GESTALT into ROAMS to test closed‐loop, autonomous navigation in simulation. We have developed a prototype capability to run many copies of ROAMS in parallel on a supercomputer, varying input parameters to rapidly explore GESTALT′s performance across a parameter space. Using these tools, we have demonstrated that large scale simulation can identify performance limits and unexpected behaviors in an algorithm. Such parallel simulation was able to test approximately 500 parameter combinations in the time required for a single test on a hardware test bed.

Experimental demonstration of coordinated control for multi-vehicle teams

This paper discusses several unique hardware testbeds that were developed to evaluate coordination and control algorithms for missions with multiple unmanned aerial vehicles (UAVs). The first testbed uses eight rovers and two blimps operated indoors to emulate a heterogeneous fleet of aircraft. The second testbed has eight UAVs flown autonomously with commercial autopilots, which are also used to perform sophisticated hardware-in-the-loop simulations. Future missions will require that teams of UAVs cooperate in dynamic and uncertain environments, and these testbeds can be used to compare various control approaches. A hierarchical configuration of task assignment, trajectory design, and low-level waypoint following uses a receding horizon framework to control the team of vehicles. The resulting demonstrations of the high-level planning algorithms on scaled vehicles operating in uncertain environments represent key steps towards transitioning them to future UAV missions.

Current Trends in Parallel Computing

The main emphasis of the author's work has been directed towards achieving efficient parallel solutions to the complex computing problems encountered in scientific and engineering applications, by developing high performance parallel architectures and efficient mapping of algorithms onto these. Besides investigating the theoretical aspects of algorithms and architectures and carrying out simulation studies on them, he has also been able to build hardware testbeds for many of the proposed parallel architectures.