Game AI Research Articles

Hierarchical Reinforcement Learning: Structuring Decision-Making for Complex AI Tasks

Hierarchical Reinforcement Learning (abbreviated as HRL) represents a progress in the realm of artificial intelligence (AI) focusing on handling the increasing complexity of decision-making processes effectively. By breaking down tasks into smaller sub tasks that are easier to manage and comprehend; HRL plays a key role in enhancing the learning efficiency of AI models. The hierarchical system mirrors decision making approaches by breaking down complex goals into smaller achievable steps; this allows reinforcement learning agents to effectively learn and adjust to dynamic environments, with limited rewards and long-term objectives. In contrast to the limitations faced by flat reinforcement learning techniques in terms of scalability and efficiency Hierarchical Reinforcement Learning (HRL) introduces a higher-level policy that controls the series and implementation of sub tasks. This study delves into the evolution, application, and influence of reinforcement learning models in artificial intelligence. We explore HRL frameworks, such, as the Options framework and temporal adaptive models that offer a systematic decision-making method. Furthermore, the study analyzes the role HRL plays in fields including robotics, self-governing systems, and game artificial intelligence. HRL is on the brink of transforming how AI tackles challenges by shaping decision making processes in intricate settings that mimic real world scenarios. This piece explores the issue at hand along, with its solutions and applications while discussing the influence and extent of HRL in enhancing AI technologies. Keywords: Hierarchical Reinforcement Learning, AI, Reinforcement Learning, Decision-Making, Sub-tasks, Options Framework, Temporal-Adaptive Models, Robotics, Autonomous Systems, Game AI, Artificial Intelligence.

‘Journeys in the Dark’ - Towards Game Master AI in Complex Board Games

The Game Master is a player role synonymous with many tabletop games. The asymmetric gameplay of the role provides different opportunities compared to other players, and can be both cooperative and competitive with the other players in the same game. Though complex environments for exploring human and Artificial Intelligence collaboration exist, few focus on the Game Master role's semi-cooperative play. Here, we propose a new complex environment based on the board game `Descent: Journeys in the Dark (Second Edition)', as part of the Tabletop Games Framework, showcasing one-versus-many play, tactical combat, and large, dynamic action and state spaces. We include baseline AI player performance of Monte Carlo Tree Search agents in this game, finding them to be well-adept at considering multiple possible end-game conditions compared to the greedy One Step Look Ahead agents. In-depth analysis reveals interesting behaviours and Hero synergies, with the aim of informing the design of games and AI models to enhance human experience in semi-cooperative environments.

Research on the integrated application of machine learning in Unity

Abstract. In recent years, the introduction of Machine Learning Agents has enabled more individuals to apply machine learning within the renowned Unity game engine. As application development has become increasingly complex, traditional artificial intelligence (AI) struggles to keep pace with the growing demands of modern applications. However, machine learning offers new avenues for developing smarter, more dynamic game characters. This paper aims to provide a comprehensive review of the various applications of machine learning within the Unity ecosystem, including game AI, intelligent character behavior, and reinforcement learning. Through a literature review and case studies, this paper explores the specific applications of machine learning algorithms in the Unity engine and analyzes the current state of development in different fields. The research findings indicate that the application of machine learning in Unity exhibits numerous developmental trends and leaves ample room for further application development in its strong areas. This technology can be innovatively applied across many fields, from implementing advanced NPCs and third-person game agents to anti-cheat mechanisms, from image recognition-based health systems to the design of complex constrained environments. The integrated application model of machine learning and Unity3D will bring more innovation and possibilities to future application development.

КОНДИЦІЙНЕ ТРЕНУВАННЯ КІБЕРСПОРТСМЕНІВ З ВИКОРИСТАННЯМ МОБІЛЬНИХ ДОДАТКІВ

The issue of insufficient physical activity among computer game players remains relevant. This study proposes a solution based on the comprehensive use of gaming-oriented mobile applications for the conditioning training of esports athletes. A sequential pedagogical experiment was conducted to determine the effectiveness of esports athletes applying an aerobic-focused conditioning training program (slow running). The program’s unique feature is its implementation in a remote format using mobile applications for planning, execution, control, and performance evaluation. It is based on an original concept of implementing running training in the form of a mobile running application (the so-called “exergame”) with a game-like nature (“Zombies, Run!”). The program was designed for a six-week implementation. Its effectiveness was determined by evaluating the dynamics of indicators: 1600 m run time; calculated maximal oxygen consumption (VO2 max); and heart rate (HR). Subjective attitudes towards the program’s physical loads were assessed using the Rating of Perceived Exertion (RPE) scale and the Physical Activity Satisfaction Needs scale. It was found that esports athletes significantly (p<0.05) improved their 1600 m run time, VO2 max, and HR indicators under the influence of the slow running program. Along with the volume and intensity increase of the running load, a decrease in RPE scale indicators was observed. The obtained data confirmed the advantages of the comprehensive use of mobile applications for normalizing the motor activity of esports athletes in the form of conditioning training. Both the remote format of planning and accounting and the game format of activities contributed to increasing the engagement of esports athletes in physical activity. The effectiveness of the slow running training program for esports athletes using mobile applications was established in terms of increasing physical activity (regular, organized physical activity), improving physical fitness (duration and volume of slow running), enhancing physical condition (VO2 max, HR), and interest in physical activity (adherence).

Challengers are Coming for Nvidia's Crown: In AI's Game of Thrones, Don't Count Out the Upstarts

Challengers are Coming for Nvidia's Crown: In AI's Game of Thrones, Don't Count Out the Upstarts

AI 게임 경험의 설계 및 몰입적 평가: 게임 치료 효과 분석

AI 게임 경험의 설계 및 몰입적 평가: 게임 치료 효과 분석

AI Strategy Formulation Across Various Game Genres - A Literature Review

Game AI is a very active area of study in recent times. These advancements appear to coincide with advancements in processing power of both computers and gaming consoles, allowing for deploying more computationally intensive algorithm. This paper samples a small selection of papers from two periods, the late 2000s to early 2010s and the beginnings of the 2020s to current time in order to demonstrate this transition toward higher level of AI development.

A crossword solving system based on Monte Carlo tree search

Although the development of AI in games is remarkable, intelligent machines still lag behind humans in games that require the ability of language understanding. In this paper, we focus on the crossword puzzle resolution task. Solving crossword puzzles is a challenging task since it requires the ability to answer natural language questions with knowledge and the ability to execute a search over possible answers to find an optimal set of solutions for the grid. Previous solutions are devoted to exploiting heuristic strategies in search to find solutions while having limited ability to explore the search space. We build a comprehensive system for crossword puzzle resolution based on Monte Carlo Tree Search (MCTS). As far as we know, we are the first to model the crossword puzzle resolution problem as a Markov Decision Process and apply the MCTS to solve it. We construct a dataset for crossword puzzle resolution based on daily puzzles from The New York Times with detailed specifications of both the puzzle and clue database selection. Our method achieves state-of-the-art performance on the dataset. The code of the system and experiments in this paper is publicly available: https://www.github.com/lhlclhl/CP.

Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domain.

Deep reinforcement learning (DRL) and deep multiagent reinforcement learning (MARL) have achieved significant success across a wide range of domains, including game artificial intelligence (AI), autonomous vehicles, and robotics. However, DRL and deep MARL agents are widely known to be sample inefficient that millions of interactions are usually needed even for relatively simple problem settings, thus preventing the wide application and deployment in real-industry scenarios. One bottleneck challenge behind is the well-known exploration problem, i.e., how efficiently exploring the environment and collecting informative experiences that could benefit policy learning toward the optimal ones. This problem becomes more challenging in complex environments with sparse rewards, noisy distractions, long horizons, and nonstationary co-learners. In this article, we conduct a comprehensive survey on existing exploration methods for both single-agent RL and multiagent RL. We start the survey by identifying several key challenges to efficient exploration. Then, we provide a systematic survey of existing approaches by classifying them into two major categories: uncertainty-oriented exploration and intrinsic motivation-oriented exploration. Beyond the above two main branches, we also include other notable exploration methods with different ideas and techniques. In addition to algorithmic analysis, we provide a comprehensive and unified empirical comparison of different exploration methods for DRL on a set of commonly used benchmarks. According to our algorithmic and empirical investigation, we finally summarize the open problems of exploration in DRL and deep MARL and point out a few future directions.

Revisiting of AlphaStar

Research on StarCraft II (SC2) is considered important due to its similarity to real-life tasks and its potential to inspire game AI design. However, the complexity of SC2 presents considerable challenges. In 2019, DeepMind proposed AlphaStar (AS), an agent that achieved Grandmaster level in SC2. Nevertheless, the reasons for AS's success remain unclear. In this paper, we revisit AS by analyzing its technical details, implementation codes, and replays. We also propose the open-sourced mini-scaled AS's new versions to do ablation studies. We classify SC2 problems by difficulty level and suggest a research path for tackling them. We then identify several limitations of AS, such as its lack of strategic view, reasoning, scouting, changes in tactics, and planning. Our paper also presents the first analysis of AS's replays. In conclusion, we emphasize that there is still a long way to solve the final SC2 problem.

Artificial Intelligence in MOBA Games: A Multivocal Literature Mapping

Esports - games played competitively - comprise a major sector of the global games industry. Esports has been used as a testbed for game AI and game analytics for two decades. This article presents a multivocal literature mapping of available research that focuses strictly on the use of artificial intelligence approaches in Multiplayer Online Battle Arena (MOBA) games, one of the most popular esports genres and the one most widely used for game AI and game analytics research. A mapping is performed on relevant publications published between 2011 and 2022 and systematically examines them to extract similarities, gaps, and main findings. We analyzed 124 publications to identify the most studied topics, the most commonly used techniques, and the most commonly applied evaluation methods. The results show that League of Legends and DOTA are the most studied games, with outcome prediction being the most popular research topic. Finally, we provide an analysis of the potential future flagship areas for research in the domain, considering the gaps found in the white and grey literature.

Application of AI in Video Games

Application of AI in Video Games

Efficient Visibility Approximation for Game AI using Neural Omnidirectional Distance Fields

Visibility information is critical in game AI applications, but the computational cost of raycasting-based methods poses a challenge for real-time systems. To address this challenge, we propose a novel method that represents a partitioned game scene as neural Omnidirectional Distance Fields (ODFs), allowing scalable and efficient visibility approximation between positions without raycasting. For each position of interest, we map its omnidirectional distance data from the spherical surface onto a UV plane. We then use multi-resolution grids and bilinearly interpolated features to encode directions. This allows us to use a compact multi-layer perceptron (MLP) to reconstruct the high-frequency directional distance data at these positions, ensuring fast inference speed. We demonstrate the effectiveness of our method through offline experiments and in-game evaluation. For in-game evaluation, we conduct a side-by-side comparison with raycasting-based visibility tests in three different scenes. Using a compact MLP (128 neurons and 2 layers), our method achieves an average cold start speedup of 9.35 times and warm start speedup of 4.8 times across these scenes. In addition, unlike the raycasting-based method, whose evaluation time is affected by the characteristics of the scenes, our method's evaluation time remains constant.

Optimization and Performance Evaluation of Artificial Intelligence Algorithms in Game Design

This article aims to optimize artificial intelligence algorithms in game design to enhance game performance and player experience. Therefore, an innovative method called Dynamic Game AI Fitness Optimization Algorithm (DGAFOA) was proposed in the article, and its effectiveness was verified through experiments. In the experimental section, this article applies the DGAFOA algorithm to a typical role-playing game and compares it with traditional fixed parameter AI algorithms. The experimental results show that the DGAFOA algorithm exhibits significant advantages in key indicators such as game task completion rate and player satisfaction. Specifically, game AI using the DGAFOA algorithm can respond more quickly and accurately to player behavior, improving the overall smoothness and fun of the game. In addition, the DGAFOA algorithm also introduces ε- The greedy strategy balances exploration and utilization, effectively avoiding the problem of AI getting stuck in local optima. This enables game AI to maintain stable performance while still possessing the ability to explore new strategies, bringing players more surprises and challenges.

Emotional Simulation in Game AI and Its Impact on Player Experience

In modern video games, the role of artificial intelligence (AI) cannot be overlooked. Game AI is not only responsible for the internal logic and character behavior within games but also serves as a crucial tool meticulously crafted by game designers to enhance the gaming experience. The objective of AI extends beyond simply achieving the highest efficiency or optimal decision-making; it aims to provide players with an engaging and challenging experience. To this end, emotional simulation has emerged as a popular research direction in the field of game AI. Emotional simulation enables virtual characters in games to have more realistic and rich emotional responses, thereby enhancing the immersion of players. When players interact with these virtual characters and perceive their genuine emotions, such as happiness, anger, sadness, and joy, they become more invested in the game and gain a deeper emotional experience. Hence, emotional simulation represents not only a technical challenge but also a crucial area of research for improving the overall quality of video games. With ongoing technological advancements, future video games will offer even richer and more realistic emotional experiences.

The use of artificial intelligence in games : based on the perspective of service players and ethical principles

The use of artificial intelligence in games : based on the perspective of service players and ethical principles

Perceived Foolishness: How Does the Saltybet Community Construct AI vs AI Spectatorship?

The spectatorship of games has become a topic of growing interest with the parallel rise of esports and livestreaming platforms. Taking Saltybet.com as its primary case study, this paper examines cases where zero-player games played by artificial intelligence-controlled characters are the focus of spectatorship. A discourse analysis identifies trends and themes in the recorded chat transcripts of 15 livestreamed tournaments from Saltybet.com where players bet fake money on the outcome of fighting game matches between AI opponents. Several themes are identified that guide discussion on how spectators discuss AI players as well as their own and the community's behaviour. These insights may be applicable to understanding the broad appeal of the entertainment people derive from AI generally whether they were meant to entertain or not. The discussion explores how the absence of human players and the scale of Saltybet's niche audience contribute to a unique, but foolish space.

A review of artificial intelligence in video games: From preset scripts to self-learning

It is now the 21st century, with the progressive development of various science and technology, such as artificial intelligence, big data, and so on, and these ever-evolving technologies have also greatly contributed to the development of today's flourishing video game field. This paper focuses on the development of artificial intelligence applications in video games over the past two decades, from preset scripts to self-learning processes, and adopts the research method of literature review. The paper concludes that the shift from pre-scripted to self-learning AI marks a shift in video games from experiences with clear rules and controlled processes to complex, dynamic, personalized experiences. This shift brings not only new opportunities but also new challenges. In the future, we can expect to see more research and practice to explore and take advantage of more of the possibilities of self-learning AI in video games.

Playing an AI game to drive adoption of regulated employee identity

Purpose This study aims to answer the key questions about the role of digital identities in organisations and within the HR function, the role of regulation in the digital identity space as it catches up with innovators and the vast potential of artificial intelligence (AI) in supporting digital identity. Design/methodology/approach Developed by using insight from the organisation’s extensive experience in digital identities and knowledge of the regulatory environment, alongside experience with the HR industry and relevant customers. Findings The digitalisation of business processes and the reality of an increasingly geographically distributed workforce have made digital identities for employees an increasingly important element of modern organisational and human resources functions. The benefits of using digital identities for employees are clear. With the growth of remote working and borderless company operations, digital identities provide employers with enhanced security, improved efficiency and cost savings. As organisations embark on their digital transformation journeys, the delicate balance between facilitating employees’ access to technology and safeguarding the organisation against cyber threats becomes clear. This intricate compromise requires the precise orchestration of certain processes, governance and technology. Originality/value In the UK, it is especially important for HR directors to consider the role of AI-empowered employee digital identities. The UK is taking a lead in digitising employee processes, with 68% of respondents in a 2023 poll by SD Worx reporting their company is investing in digital HR and training offerings, compared to a 60% average across Europe.

Cognitive Load and Mixed Strategies: On Brains and Minimax

It is well known that laboratory subjects often do not play mixed strategy equilibria games according to the theoretical predictions. However, little is known about the role of cognition in these strategic settings. We therefore conduct an experiment where subjects play a repeated hide and seek game against a computer opponent. Subjects play with either fewer available cognitive resources (high cognitive load treatment) or with more available cognitive resources (low cognitive load treatment). Surprisingly, we find some evidence that subjects in the high load treatment earn more than subjects in the low treatment. However, we also find that subjects in the low treatment exhibit a greater rate of increase in earnings across rounds, thus suggesting more learning. Our evidence is consistent with subjects in the low load treatment over-experimenting. Further, while we observe that subjects do not mix in the predicted proportions and that their actions exhibit serial correlation, we do not find strong evidence these are related to their available cognitive resources. This suggests that the standard laboratory deviations from the theoretical predictions are not associated with the availability of cognitive resources. Our results shed light on the extent to which cognitive resources affect (and do not affect) behavior in games with mixed strategy equilibria.