Security Testbed Research Articles

Reinforcement Learning as a Path to Autonomous Intelligent Cyber-Defense Agents in Vehicle Platforms

Technological advancement of vehicle platforms exposes opportunities for new attack paths and vulnerabilities. Static cyber defenses can help mitigate certain attacks, but those attacks must generally be known ahead of time, and the cyber defenses must be hand-crafted by experts. This research explores reinforcement learning (RL) as a path to achieve autonomous, intelligent cyber defense of vehicle control networks—namely, the controller area network (CAN) bus. We train an RL agent for the CAN bus using Toyota’s Portable Automotive Security Testbed with Adaptability (PASTA). We then apply the U.S. Army Combat Capabilities Development Command (DEVCOM) Army Research Laboratory’s methodology for quantitative measurement of cyber resilience to assess the agent’s effect on the vehicle testbed in a contested cyberspace environment. Despite all defenses having similar traditional performance measures, our RL agent averaged a 90% cyber resilience measurement during drive cycles executed on hardware versus 41% for a naïve static timing defense and 98% for the bespoke timing-based defense. Our results also show that an RL-based agent can detect and block injection attacks on a vehicle CAN bus in a laboratory environment with greater cyber resilience than prior learning approaches (1% for convolutional networks and 0% for recurrent networks). With further research, we believe there is potential for using RL in the autonomous intelligent cyber defense agent concept.

A CAN Bus Security Testbed Framework for Automotive Cyber-Physical Systems

The popularization of electronic devices and the enrichment of external interfaces have increased the attack surface of the automotive cyber-physical system (CPS). As a vital part of the CPS, the controller area network (CAN) is more vulnerable to security threats due to the lack of corresponding security protection mechanisms. This kind of security problem has also attracted extensive attention from academia and industry. Researchers have proposed numerous intrusion detection models for the in-vehicle CAN bus, solving some security problems to a certain extent. However, vehicle manufacturers seldom disclose the internal details of vehicle ECUs due to safety concerns. Thus, it is difficult for researchers to investigate the operation mechanism of the bus. Meanwhile, there is a risk of personal safety in completing attack experiments on real vehicles, which can also lead to the lack and diversification of in-vehicle network data, especially the data of attack behavior. Based on real vehicle data, an open, adaptable, and low-risk CAN bus security testbed framework in the automotive CPS is proposed in this study, aiming to enrich the operation data of the CAN bus and enhance the personal safety of researchers. Besides, the delay of the testbed sending and receiving periodic and aperiodic CAN messages is theoretically explored. The results demonstrate that the generated timestamp in the dataset is mainly associated with the timestamp of the real vehicle data and that the transmission and collection of time series data are completed by Algorithm 1 and Algorithm 2. In the evaluation of the security testbed, the stability of the security testbed is studied from the two indicators of delay and packet loss rate. The experiment reveals that the testbed has a small relative delay difference and a low packet loss rate. Moreover, the DTW algorithm is employed to calculate the distance between the real vehicle and the testbed, and the experimental results demonstrate that the testbed is featured with high similarity and simulation.

Design Considerations for Building Credible Security Testbeds: Perspectives from Industrial Control System Use Cases

ABSTRACT This paper presents a mapping framework for design factors and an implementation process for building credible Industrial Control Systems (ICS) security testbeds. The security and resilience of ICSs has become a critical concern to operators and governments following widely publicised cyber security events. The inability to apply conventional Information Technology security practice to ICSs further compounds challenges in adequately securing critical systems. To overcome these challenges, and do so without impacting live environments, testbeds are widely used for the exploration, development, and evaluation of security controls. However, how a testbed is designed and its attributes, can directly impact not only its viability but also its credibility. Combining systematic and thematic analysis, and the mapping of identified ICS security testbed design attributes, we propose a novel relationship map of credibility-supporting design factors (and their associated attributes) and a process implementation flow structure for ICS security testbeds. The framework and implementation process highlight the significance of demonstrating some design factors such as user/experimenter expertise, clearly defined testbed design objectives, simulation implementation approach, covered architectural components, core structural and functional characteristics covered, and evaluations to enhance confidence, trustworthiness and acceptance of ICS security testbeds as credible. These can streamline testbed requirement definition, improve design consistency and quality while reducing implementation costs.

Design and implementation of automated IoT security testbed

The emergence of technology associated with the Internet of Things (IoT) is reshaping our lives, while simultaneously raising many issues due to their low level of security, which attackers can exploit for malicious purposes. This research paper conducts a comprehensive analysis of previous studies on IoT device security with a focus on the various tools used to test IoT devices and the vulnerabilities that were found. Additionally, the paper contains a survey of IoT-based security testbeds in the research literature. In this research study, we introduce an open source platform for identifying weaknesses in IoT networks and communications. The platform is easily modifiable and extendible to enable the addition of new security assessment tests and functionalities. It automates security evaluation, allowing for testing without human intervention. The testbed reports the security problems of the tested devices and can detect all attacks made against the devices. It is also designed to monitor communications within the testbed and with connected devices, enabling the system to abort if malicious activity is detected. To demonstrate the capabilities of the proposed IoT security testbed, it is used to examine the vulnerabilities of two IoT devices: a wireless camera and a smart bulb.

Full‐Scale Drinking Water System Decontamination at the Water Security Test Bed

The EPA's Water Security Test Bed (WSTB) facility is a full-scale representation of a drinking water distribution system. In collaboration with the Idaho National Laboratory (INL), EPA designed the WSTB facility to support full-scale evaluations of water infrastructure decontamination, real-time sensors, mobile water treatment systems, and decontamination of premise plumbing and appliances. The EPA research focused on decontamination of 1) Bacillus globigii (BG) spores, a non-pathogenic surrogate for Bacillus anthracis and 2) Bakken crude oil. Flushing and chlorination effectively removed most BG spores from the bulk water but BG spores still remained on the pipe wall coupons. Soluble oil components of Bakken crude oil were removed by flushing although oil components persisted in the dishwasher and refrigerator water dispenser. Using this full-scale distribution system allows EPA to 1) test contaminants without any human health or ecological risk and 2) inform water systems on effective methodologies responding to possible contamination incidents.

Advanced Security Testbed Framework for Wearable IoT Devices

Analyzing the security of Wearable Internet-of-Things (WIoT) devices is considered a complex task due to their heterogeneous nature. In addition, there is currently no mechanism that performs security testing for WIoT devices in different contexts. In this article, we propose an innovative security testbed framework targeted at wearable devices, where a set of security tests are conducted, and a dynamic analysis is performed by realistically simulating environmental conditions in which WIoT devices operate. The architectural design of the proposed testbed and a proof-of-concept, demonstrating a preliminary analysis and the detection of context-based attacks executed by smartwatch devices, are presented.

Research on Information Security Test Bed and Evaluation Framework for Distribution Automation System

Analyzing the characters and abstracting the system model of the distribution automation system, information security test-bed of distribution automation is designed. Meanwhile, information security test and evaluation framework is proposed based on the lifecycle of distribution automation system. The evaluation activities and the testing methods are analyzed and key problems and solutions of information security evaluation and reinforcement requirements for distribution automation system are introduced. The work in this paper can effectively support the information security protection of distribution automation system, and can provide reference for other power industrial control systems.

Taxonomy of SCADA systems security testbeds

Considerable research has focused on developing simulation environments to study the effects of cyber-attacks on supervisory control and data acquisition (SCADA) systems although a methodology to develop a SCADA security testbed is not yet available. This paper attempts to address the issue by presenting a reference model taken from the literature and classifying specific contributions on the development of SCADA security testbed according to the reference model. Specific techniques and experiments used to recreate cyber-attacks on the de facto standard communication protocol Modbus are also presented. This paper also explores the possibility of integrating specific simulation models into a SCADA security testbed to assess the impact of cyber-attacks on the physical system.

An Information Security Assessments Framework for Power Control Systems

Information and cyber security of Industrial Control Systems (ICS) faces severe challenges and has gained considerable importance. Information security assessment is an essential component of information security assurance infrastructure mechanisms. First, a hierarchical model of smart grid was abstracted. Based on the proposed model and the information security risks and information security protection demands of power control systems, an information security assessments framework for power control systems was proposed in dimensions of system layers and life cycle to guide the security assessment contents of power control systems. Finally, a test bed function design for power control system security assessment was proposed. The power control system security test bed may include four parts, such as power control system security assessment management platform, power control system simulation environment, security assessment tools, and security policy. The proposed security assessment framework and test bed functional design can be used to guide the electric power utilities in their power control system information security efforts.

Research and Construction of Information Security Simulation Test Bed for Electric Power System

Electric power system relies heavily on common information technologies and platforms, so all common information security issues have been experienced in the electric power information system. Because of attacks destructivity and uncontrollability, it is inappropriate to conduct security analysis in a real Power system directly. Therefore, the paper uses simulation and emulation technology to build a information security test bed for power system at a lower cost and no affect the normal operation and the premise of the aimed system. Simulation contents in the test bed are arranged in four different layers including firmware layer, system layer, network layer and application layer. The test bed also provides a network attack and defense combat system. The test bed can be used for Power system information and network system security testing, engineering validation and evaluation.

Cyber-Physical Security Testbeds: Architecture, Application, and Evaluation for Smart Grid

The development of a smarter electric grid will depend on increased deployments of information and communication technology (ICT) to support novel communication and control functions. Unfortunately, this additional dependency also expands the risk from cyber attacks. Designing systems with adequate cyber security depends heavily on the availability of representative environments, such as testbeds, where current issues and future ideas can be evaluated. This paper provides an overview of a smart grid security testbed, including the set of control, communication, and physical system components required to provide an accurate cyber-physical environment. It then identifies various testbed research applications and also identifies how various components support these applications. The PowerCyber testbed at Iowa State University is then introduced, including the architecture, applications, and novel capabilities, such as virtualization, Real Time Digital Simulators (RTDS), and ISEAGE WAN emulation. Finally, several attack scenarios are evaluated using the testbed to explore cyber-physical impacts. In particular, availability and integrity attacks are demonstrated with both isolated and coordinated approaches, these attacks are then evaluated based on the physical system's voltage and rotor angle stability.