Wargame Simulation Research Articles

An Efficient Multi-Agent Policy Self-Play Learning Method Aiming at Seize-Control Scenarios

Aiming at the problem of multi-agent cooperative confrontation in seize-control scenarios, we design an efficient multi-agent policy self-play (EMAP-SP) learning method. First, a multi-agent centralized policy model is constructed to command the agents to perform tasks cooperatively. Considering that the policy being trained and its historical policies usually have poor exploration capability under incomplete information in self-play trainings, the intrinsic reward mechanism based on random network distillation (RND) is introduced in the self-play learning method. In addition, we propose a multi-step on-policy deep reinforcement learning (DRL) algorithm assisted by off-policy policy evaluation (MSOAO) to learn the best response policy in the self-play. Compared with DRL algorithms commonly used in complex decision problems, MSOAO has more efficient policy evaluation capability, and efficient policy evaluation further improves the policy learning capability. The effectiveness of EMAP-SP is fully verified in MiaoSuan wargame simulation system, and the evaluation results show that EMAP-SP can learn the cooperative policy of effectively defeating the Blue side’s knowledge-based policy under incomplete information. Moreover, the evaluations results in DRL benchmark environments also show that the best response policy learning algorithm MSOAO can promote the agent to learn approximately optimal policies.

Modeling and Design Methodology for the Development of Electro-Optical and Laser Systems

Modeling and simulation (M&S) plays a key role in the development of all systems, but especially so in the development and integration of electro-optical (EO) subsystems and lasers for precision strike applications or counter measures that require precise pointing control. Since the development of the first EO and laser systems, the directed energy community has pursued an evolving modeling and design (M&D) approach to support the development of these systems to provide the data and insights needed for the acquisition process. M&D is the lower-level detailed engineering models that are used to define and perfect the design (mechanical, optical, electrical, software), whereas M&S is typically the higher-level engagement models embedded in the mission and war game simulations. When the M&S is closely tied to the M&D, the M&S will accurately represent the system capabilities and limitations. This paper presents an M&D methodology that has matured over the past years and continues to be applied on current EO and laser programs. Similar to other development methodologies utilized by government agencies and industry, the M&D methodology is critical for airborne EO-laser systems, where the operating environment is especially hostile to achieving microradian pointing accuracy and jitter stability in the optical line of sight. Applying the M&D methodology early and often throughout the development process in conjunction with the model-based systems engineering approach, the M&S can fully integrate into the life cycle of EO-laser programs from the development of requirements, the verification of design, the prediction and verification of performance, and the defining of system capabilities and limitations, to the maintenance and upgrades to the system. This disciplined connectivity of M&S to the M&D will lead to cost-effective EO-laser systems.

Development of standard loss analysis model using big data

The Republic of Korea Army conducts simulations during peacetime using ground operation resource requirements analysis model (GORRAM) to determine potential losses when at war based on the latest operation plan. Although war-game simulation can yield reliable results, it takes considerable amount of time and effort to build a database and generate scenarios. Therefore, a study is required to supplement the detailed war-game simulation method to quickly determine expected losses. Using data built-in GORRAM, we tested the significance of four factors using beta regression analysis. While multiple regression is most commonly used to model the causality, beta regression is a powerful method for modeling response variables in the (0,1) range, such as the loss ratio. We verified that three factors, namely ‘topography’, ‘operational posture’, and ‘friend/foe power ratio’ were related to loss. This study proposes a new method for calculating the expected loss in real-time, overcoming a limitation of existing war-game simulation methods.

A Study on Reinforcement Learning Method for the Deception Behavior : Focusing on Marine Corps Amphibious Demonstrations

Military deception is an action executed to deliberately mislead enemy’s decision by deceiving friendly forces intention. In the lessons learned from war history, deception appears to be a critical factor in the battlefield for successful operations. As training using war-game simulation is growing more important, it is become necessary to implement military deception in war-game model. However, there is no logics or rules proven to be effective for CGF(Computer Generated Forces) to conduct deception behavior automatically. In this study, we investigate methodologies for CGF to learn and conduct military deception using Reinforcement Learning. The key idea of the research is to define a new criterion called a “deception index” which defines how agent learn the action of deception considering both their own combat objectives and deception objectives. We choose Korea Marine Corps Amphibious Demonstrations to show applicability of our methods. The study has an unique contribution as the first research that describes method of implementing deception behavior.

Optimization of Unmanned Air Vehicle Tactical Formation in War Games

War game simulations are decision-making tools that may provide quantitative data about the scenario analyzed by stakeholders. They are widely used to develop tactics and doctrines in the military context. Recently, unmanned air vehicles (UAVs) have become a relevant element in these simulations because of their prominent role in contemporary conflicts, surveillance missions, and search and rescue missions. For instance, it is possible to admit aircraft losses from a tactical formation in favor of the victory of a squadron in a given combat scenario. The optimization of the position of UAVs in beyond visual range (BVR) combat has attracted attention in the literature, considering that the distribution of UAVs can be a determining factor in this scenario. This work aims to optimize UAV tactical formations considering enemy uncertainties such as firing distance and position using six metaheuristics and a high-fidelity simulator. A tactical formation often employed by air forces called line abreast was chosen for the RED swarm for a case study. The objective of the optimization is to obtain a tactical formation of the BLUE swarm that wins the BVR combat against the RED swarm. A procedure to confirm the robustness of the optimization is employed, varying the position of each UAV of the RED swarm up to 8 km from its initial configuration and using the war game approach. A tactical analysis is performed to confirm whether the formations found in the optimization are applicable.

The effects of aircraft use and available repair spares on aircraft sortie generation: a long-duration logistical wargaming simulation tool

Long-duration logistical wargames within the Air domain are complex and highly dynamic events that are driven by aircraft availability. In order to gain insight into the impact of aircraft use, this research developed a simulation tool that uses a stepwise approach for adjudication and provides the user many capabilities including, but not limited to, the ability to have multiple bases and types of aircraft. Daily aircraft availability and missions accomplished are two critical metrics of interest. Within the simulation, the user has the ability to control types of part failures, control parts availability, control maintenance capabilities, and control number of mission scheduled. Finally, the user can account for the possibility of attrition along with the effects of numerous major events present in real-life scenarios. This tool is validated through application of a space covering design along with regression modeling and shows that the tool is well-behaved, functions as expected, and can quickly provide meaningful insights into operational scenarios.

Shared Parameter Database of War Gaming Software: Case Study on Commercial Simulation Databases

Background: The current research brings up the perspective of security-critical information systems in shared parameter databases in the context of processing sensitive data at Finnish Naval Warfare Centre. It refers to the environment of isolated military war gaming simulation and modeling systems. The research problem is: How to make an optimal solution for data distribution in different military war gaming simulation and modeling software? Objectives: The objective is to create a single shared database usable with different detail level software, e.g. high-level scenario simulation, technical system-of-system simulations, and system-level physical simulations. Methods: The methods are modeling, simulation and operation analysis. The approach is inductive, the strategy is a qualitative case study and the data collection was implemented by exploring database models and their combinations. The integration was implemented in an object-relational database management system (ORDBMS), PostgreSQL. Results: The shared database led to efficient access to simulation parameters, more straightforward system integration and improved scalability. Conclusions: The results of modeling and simulation indicated that the integration is possible to implement.

Recurrent MADDPG for Object Detection and Assignment in Combat Tasks

With the development of artificial intelligence, multiagent algorithms have been applied to many real-time strategy games. Making plans on the human being is gradually passing away, especially in combat scenarios. Cognitive electronic warfare (CEW) is a complex and challenging work due to the sensitivity of the data sources. There are few studies on CEW. In the past, wargame simulations depended on differential equations and war theory, which resulted in high time and human resource costs. In the future, as other artificial intelligence theories are developed, artificial intelligence will play a more critical role in wargames. The capabilities of multiagent modeling to describe complex systems and predict actions in dynamic environments are superior to those of traditional methods. In this paper, we use a 3D wargame engine from China Aerospace System Simulation Technology Co., Ltd. (Beijing) named All Domain Simulation (ACS), which supports land, sea, and air combat scenarios, to simulate combat. In the simulations, there are several unmanned air vehicles (UAVs) as attackers and several radar stations as defenders, and both have the ability to detect the others. In the game, several UAVs need to learn to detect targets and track targets separately, and we train the UAV's behavior by well-designed reward shaping and multiagent reinforcement learning (MARL) with LSTM. We improved the RDPG algorithm and merged the MADDPG and RDPG algorithms. From the experimental results, we can see that the effectiveness and accuracy of the algorithm have been greatly improved.

Distributed Data Processing for Bigdata Analysis in War Game Simulation Environment

Distributed Data Processing for Bigdata Analysis in War Game Simulation Environment

Modeling and simulation of future capabilities with an automated computer-aided wargame

This article explores the development and application of an automated computer-aided wargame to establish high-level capability requirements and concepts of operations for future Navy unmanned aerial vehicles and unmanned underwater vehicles. The Joint Theater Level Simulation-Global Operations serves as the modeling environment, in which a computer-aided exercise models the impact of future intelligence, surveillance, and reconnaissance assets. Automating wargame simulations permits the replication of a large-scale exercise without the continued investment of support personnel and operating units. The environment enables experimentation that provides force planners with pertinent metrics to inform decision-making.

통합 교전 시뮬레이터 환경에서 다기능 레이다 탐지/추적 성능 모의를 위한 고해상도 레이다 모델

In this paper, modeling of a high-resolution multi-function radar is proposed to simulate radar performance in a war game simulator, called AddSIM. To incorporate the multi-function radar model into the AddSIM, the modeling must comprise a component-based structure consisting of physics, logics, and information blocks. Therefore, we assign the RF hardware of a RADAR as the physic block, a controller as the logics block, and the RF specifications of the RADAR as the information block. Detailed modeling of the physics and logics blocks are addressed, and data structure is also presented on an engineering level. On a multi-target engaged scenario, the performance of the multi-function radar is numerically analyzed and its validation is examined.

Research on Emergency Rescue of Urban Flood Disaster Based on Wargame Simulation

In view of the great impact of urban flooding disasters on residents, traffic and urban infrastructure in recent years, we aim at finding high-risk events and loopholes in emergency plans at low cost, and simulate the emergency rescue process. According to the principles of wargame and the disaster theory, combining GIS technology and decision tree, we have established dynamic models for game pieces, rainstorm rules, visual maps and models of wargame, and demonstrated the process of urban flooding emergency rescue. Results showed that, under a 5-year rainstorm scenario and 18 deduction rounds, among the two emergency rescue action plans, our action 1 could complete the emergency rescue, but with a very high probability of loss, while action 2 could complete the emergency flooding rescue, with a small probability of loss. The application of wargame method in rescuing during urban flooding can precisely rollout the practicability of emergency plans. The results of such studies are of great significance to the training of emergency department in commanding ability, emergency management ability and decision-making ability in emergency response.

Unified learning to enhance adaptive behavior of simulation objects

Modeling and simulation are methods of validating new systems that are risky to be directly deployed in the real world. During the simulation, the simulation environment continuously changes and simulation objects correspondingly behave according to the changing situations. In general, modeling the behavior for all possible situations is extremely difficult when the rationale is unknown. Therefore, in order to adapt to the changing situation, it is important to recognize the rationale behind the behaviors of the simulation object. However, in many cases, even though the rationale is unknown or difficult to recognize, the simulation requires reasonable behaviors such as a commander’s decision in a war game simulation and a driver’s behavior in rush hours. In this study, we propose a new approach to determine the behavior of simulation objects under changing situations. The proposal is a unified learning approach that integrates two methods, data-driven and knowledge-driven approaches, which allow simulation objects to learn behavioral knowledge from experience as well as from domain experts performing the simulation and reuse verified knowledge. By combining both approaches, we supplement the shortcomings of one method with the strengths of the other. To verify our method, we apply the proposed approach to a military training simulation.

Wargame Simulation Theory and Evaluation Method for Emergency Evacuation of Residents from Urban Waterlogging Disaster Area.

Urban waterlogging seriously threatens the safety of urban residents and properties. Wargame simulation research on resident emergency evacuation from waterlogged areas can determine the effectiveness of emergency response plans for high risk events at low cost. Based on wargame theory and emergency evacuation plans, we used a wargame exercise method, incorporating qualitative and quantitative aspects, to build an urban waterlogging disaster emergency shelter using a wargame exercise and evaluation model. The simulation was empirically tested in Daoli District of Harbin. The results showed that the wargame simulation scored 96.40 points, evaluated as good. From the simulation results, wargame simulation of urban waterlogging emergency procedures for disaster response can improve the flexibility and capacity for command, management and decision-making in emergency management departments.

Special Issue: Modeling & Simulation for Cyber Security of Autonomous Vehicle Systems

The transportation and information gathering needs of our world are changing. Transportation systems are becoming more vital for our society and also more congested. Networking holds out the hope for more efficient use of multiple autonomous systems but also opens the door for increased risk exposure to failures and malicious attacks. Developments in internet technologies, online auctions and games, social networks, high-technology engineered systems with multiple dynamic components, and elsewhere motivate new advancements for decision, interaction, and control in cyberphysical systems (CPS) with autonomous dynamical subsystems. Examples of vehicle CPS can be found in areas as diverse as automobiles, air transportation, civil infrastructure, trains, and surface and subsurface water vehicles. Information Assurance must provide authentic, accurate, secure, reliable, and timely information to vehicle decision and control systems in order to achieve information dominance, mission performance, and survivability in risky changing environments. Computing and information processes must be carried out over distributed and heterogeneous networked systems for resilience to minimize single point of failure to malicious attacks. Recent advances in autonomous cyberphysical systems show promise for improved reconnaissance, surveillance, and security in foreign and domestic conflicts. However, CPS also present new and unevaluated sets of attack surfaces in communications and controls. As such, the modeling and simulation of cyber security in autonomous systems is a critical research area that will ultimately dictate the ways these systems should interact with people and each other. Thus, the goal of this Special Issue is collect a comprehensive body of descriptive and prescriptive approaches to modeling and simulating cyber security threats, defense techniques, and overall frameworks in highly dynamic and hostile environments. This special issue is composed of five papers that show the current state of modeling, simulation, decision, and control in some important areas of cyberphysical systems. The paper by Yagdereli, Gemci, and Aktas x presents a study on cyber security of unmanned vehicles, including cars, civilian unmanned aircraft, trains, and boats. Types of cyber-attacks are characterized, including passive and active attacks. Potential vulnerabilities of autonomous vehicles are detailed. Metrics for cybersecurity are proposed. A framework for mitigation strategies based on modeling and simulation is presented. The paper by Bergin presents a cyber-attack and simulation framework based on a live-virtual-constructive (LVC) environment. The focus is on wireless mobile network interfaces for military autonomous vehicles. LVC offers opportunities for training and assessment of humanin-the loop performance. Security threats and vulnerabilities are discussed, and attack techniques are outlined including denial of service, passive attack, and malicious agents. A cyber-attack and simulation framework is detailed including methods of data modeling, storage, and transfer. The paper by Cavagnaro and Tiller presents a framework for risk assessment of force allocation and countermeasures in autonomous vehicle air operations. A probability theory approach is used to define tipping points to quantify game outcomes that represent mission success. Probability distributions are presented for risk assessment, and random walks are employed to yield a distribution of outcome results. Based on these tools, the Tiller software wargame simulation Modern Air Power has been developed and is applied in this paper to force allocation and countermeasures based on game-changing capabilities.

HLA/RTI를 이용한 워게임 시뮬레이터와 통신 효과 시뮬레이터의 연동 시뮬레이션 연구

Wargame simulators are widely used in the field of defence modeling and simulation. Because of increasing importance of communication effects on the warfare, the war game simulator is also required to reflect communication effects. One way to satisfy the requirement is the interoperation simulation between war game simulator and communication effect simulators. This paper shows the application of interoperation simulation between war game and communication effect simulators using HLA(High-Level Architecture)/RTI(RunTime Infrastructure). The war game simulator mainly deal with the engagement of troops and the troops communicate each other at the mission execution level. In the other hand, The communication effect simulator perform communication actions between the troops in the engineering level. Using the interoperation simulation, we can reflect the communication effects on the war game simulation. We show various applications of the interoperation simulation with the point of the war game and communication effect simulator. with a case study, we explain the interoperation simulation improves the reality and fidelity of the war game simulator and how the interoperation simulation can be applied to developing doctrines and real communication system.

Practical Integration of Positive and Normative Approaches to Cost-Benefit Analysis of National Defense Spending

Cost-benefit analyses for national defense spending can suffer from industry trade association biases evident in inflated supposed gains from industrial multiplier effects. At the same time, ideological presumption of normative values to either increasing or decreasing defense spending (and therefore presence) by the US ignores a more effective analytic framework: first quantifying the purely financial costs of defense spending: immediate cash expense, and increased borrowing costs and slower growth due to increased debt; and the purely financial benefits: the most accurate multiplier effect.This is still only the first part of the story. Quantifying the above first allows us to evaluate the premium we may be paying for defense as a public good. This positive approach could be complemented by a normative one that ignores rhetoric, but simply attempts to assign particular defense spending line items to the particular percent reduction in the chance of a possible conflict (with an identifiable cost) to which it contributes, and use this expected value to determine the return-on-investment of defense as public good. While this framework is tested on the sequestration cuts (for the positivist component) and the costs of potential conflict in Sudan (for the normative one, where ideally defense spending items could be traced to the expected value of their ability to reduce the chance of that conflict occurring), it can be extended to any defense investment decision. This framework is hopefully preferable to anything short of combining sophisticated input-output models with a wargaming simulation.

복합체계 개념에 기반한 국방체계 모델링 시뮬레이션 방법론

Tag Gon Kim․Se Jung Kwon․Bonggu KangDepartment of Electrical Engineering, KAISTA complex system such as defense systems is in a form of System of Systems (SoS) in which each component is a system being independent of other component systems. Dynamical behavior of SoS is represented by a composition of behaviors of component systems. Thus, a M&S tool/environment would not be efficient for deve-lopment of heterogeneous models nor for simulation in a centralized environment. Moreover, such an envi-ronment restricts reusability and composability. This paper presents an interoperation method for M&S of defense systems as SoS. The approach first develops component system models using tools, each specialized to M&S of each component system. It then interoperates such simulations together to simulate a whole system as SoS. HLA/RTI is employed for such interoperation, which is a DoD/IEEE standard to support interoperation. We will introduce a case study for interoperable simulation of defense systems, which consists of a wargame simulator and a communication simulator.

Scenario development: using geopolitical wargames and strategic simulations

The art of scenario development is the core of successful and effective wargaming and conflict simulation which includes some fairly scientific aspects combined to form a proving ground for decision making, leadership and public policy. Paying attention to situational detail, estimating the relative strength, disposition, leadership and logistic determinants of contending opponents is key. Military wargames, in particular, have been effective in developing future leaders, assessing critical decisions, analyzing tactical pitfalls and helping to devise strategy for many centuries. The value of geopolitical wargames and strategic simulations is profoundly influential and valuable in helping aspiring leaders, executives and military commanders understand the obvious and hidden dynamics of contending armies and contextual influences. Likewise, the principles involved include substantial insights and benefits to those wishing to know more about the power of scenarios and wargaming on the development of leaders, the assessment of challenging pressures and the resolution of complex problems.

통신 효과를 반영한 네트워크 중심전의 전투 효과도 분석 방법

ABSTRACTAs a future warfare is changed to network-centric warfare (NCW), importance of research on the relation between combat effectiveness and communication effect has increased. The combat effectiveness has been analyzed using war game simulator and the communication effect has been analyzed using communication simulator. However, the relation of them has not been researched so far. In this paper, we describe a methodology of analyzing the measure of effectiveness reflecting communication effects in NCW. In the methodology, firstly, simulation interoperation environment between the war game simulator and the communication simulator is developed. Especially, we propose an interface model and interoperation interface to support the simulators for simulation interoperation. Then, we statistically analyze the output data of the simulation interoperation environment and develop a meta-model which illustrates the relation between war game and communication simulator directly. With this methodology, we can figure out the relation between the war game and communication simulator directly. Moreover, we expect that this relation effects on developing doctrines, communication operation and communication devices.