Machine Learning Agents Research Articles

Research on the integrated application of machine learning in Unity

Abstract. In recent years, the introduction of Machine Learning Agents has enabled more individuals to apply machine learning within the renowned Unity game engine. As application development has become increasingly complex, traditional artificial intelligence (AI) struggles to keep pace with the growing demands of modern applications. However, machine learning offers new avenues for developing smarter, more dynamic game characters. This paper aims to provide a comprehensive review of the various applications of machine learning within the Unity ecosystem, including game AI, intelligent character behavior, and reinforcement learning. Through a literature review and case studies, this paper explores the specific applications of machine learning algorithms in the Unity engine and analyzes the current state of development in different fields. The research findings indicate that the application of machine learning in Unity exhibits numerous developmental trends and leaves ample room for further application development in its strong areas. This technology can be innovatively applied across many fields, from implementing advanced NPCs and third-person game agents to anti-cheat mechanisms, from image recognition-based health systems to the design of complex constrained environments. The integrated application model of machine learning and Unity3D will bring more innovation and possibilities to future application development.

Forecasting Future Behavior: Agents in Board Game Strategy

This paper presents findings on machine learning agent behavior prediction in a board game application developed by a group of students. The goal of this research is to create a model facilitating collaboration between a user and an AI to play together in the board game using a Human-in-the-Loop architecture. By injecting explainability, the aim is to enhance communication and understanding between the user and the AI agent. Featuring a competitive Artificial Intelligence (AI) based on the Proximal Policy Optimization model, this research explores methods to make AI decisions transparent for enhanced player understanding. Two predictive models, a Decision Tree (DT) and a Deep Learning (DL) classifier, were developed and compared. The results show that the DT model is effective for short-term predictions but limited in broader applications, while the DL classifier shows potential for long-term prediction without requiring direct access to the game's AI. This study contributes to understanding human-AI interaction in gaming and offers insights into AI decision-making processes.

Load frequency control strategy for islanded multimicrogrids with V2G dependent on learning‐based model predictive control

AbstractSystem reliability and stability can be significantly improved by the interconnected operation of multimicrogrids, and electric vehicles (EVs) provide a more flexible solution for frequency control, which also present challenges for frequency control. Therefore, a load frequency control (LFC) strategy for multimicrogrids with vehicle to grid (V2G) dependent on learning‐based model predictive control (MPC) is proposed. First, a controller‐interconnected multimicrogrid topology is proposed; thus, a multimicrogrid consisting of microturbines (MTs), distributed power sources, and EVs and their random power constraints is established. Second, a control parameter adaptive algorithm based on learning‐based MPC is designed. The real‐time frequency offset and EV station output power boundary are used as the state set, adjustable parameters of the MPC controller are used as the action set, and reward function is set with frequency deviation so that the adaptive adjustment of the weight parameters of the MPC controller is realised. Additionally, the improved MPC controller designed in this paper can transform the frequency control process into an optimization problem, which is well adapted to various random constraints in the control process. In addition, the deep deterministic policy gradient (DDPG)‐MPC double‐layer controller can prevent machine learning controller failure. Finally, the simulation results show that, compared with traditional control and MPC algorithms, the learning‐based MPC controller applied to the controller interconnection structure can exchange information between submicrogrids. Moreover, based on the experience accumulated in the prelearning process, the controller parameters can be updated according to the environmental state in real time, thereby significantly improving the robustness and rapidity of the multimicrogrid frequency control process. Meanwhile, compared with a traditional DDPG controller, the proposed controller with double‐layer coupling structure can better ensure the safe operation of the multimicrogrid system when the machine learning agent fails and cannot output actions normally.

Real-time hand interaction and self-directed machine learning agents in immersive learning environments

Integration of extended reality (XR) in education is becoming popular to transform the traditional classroom with immersive learning environments. The adoption of immersive learning is accelerating as an innovative approach for science and engineering subjects. With new powerful interaction techniques in XR and the latest developments in artificial intelligence, interactive and self-directed learning are becoming important. However, there is a lack of research exploring these emerging technologies research with kinesthetic learning or “hands-one learning" as a pedagogical approach using real-time hand interaction and agent-guided learning in immersive environments. This paper proposes a novel approach that uses machine learning agents to facilitate interactive kinesthetic learning in science and engineering education through real-time hand interaction in the virtual world. To implement the following approach, this paper uses a chemistry-related case study and presents a usability evaluation conducted with 15 expert reviewers and 2 subject experts. NASA task load index is used for cognitive workload measurement, and the technology acceptance model is used for measuring perceived ease of use and perceived usefulness in the evaluations. The evaluation with expert reviewers proposed self-directed learning using trained agents can help in the end-user training in learning technical topics and controller-free hand interaction for kinesthetic tasks can improve hands-on learning motivation in virtual laboratories. This success points to a novel research area where agents embodied in an immersive environment using machine learning techniques can forge a new pedagogical approach where they can act as both teacher and assessor.

Reinforcement Learning Applied to AI Bots in First-Person Shooters: A Systematic Review

Reinforcement Learning is one of the many machine learning paradigms. With no labelled data, it is concerned with balancing the exploration and exploitation of an environment with one or more agents present in it. Recently, many breakthroughs have been made in the creation of these agents for video game machine learning development, especially in first-person shooters with platforms such as ViZDoom, DeepMind Lab, and Unity’s ML-Agents. In this paper, we review the state-of-the-art of creation of Reinforcement Learning agents for use in multiplayer deathmatch first-person shooters. We selected various platforms, frameworks, and training architectures from various papers and examined each of them, analysing their uses. We compared each platform and training architecture, and then concluded whether machine learning agents can now face off against humans and whether they make for better gameplay than traditional Artificial Intelligence. In the end, we thought about future research and what researchers should keep in mind when exploring and testing this area.

Trajectory control of autonomous mobile robots considering disturbance with machine learning agents

Trajectory control of autonomous mobile robots considering disturbance with machine learning agents

Active learning for optimum experimental design—insight into perovskite oxides

Finding the optimum material with improved properties for a given application is challenging because data acquisition in materials science and chemistry is time consuming and expensive. Therefore, dealing with small datasets is a reality in chemistry, whether the data are obtained from synthesis or computational experiments. In this work, we propose a new artificial intelligence method based on active learning (AL) to guide new experiments with as little data as possible, for optimum experimental design. The AL method is applied to ABO3 perovskites, where a descriptor based on atomic properties was developed. Several regressor algorithms were employed: artificial neural network, Gaussian process, and support vector regressor. The developed AL method was applied in the experimental design of two important materials: non-stoichiometric perovskites (Ba(1− x)A x Ti(1− y)B y O3) due to substituting ionic sites with different concentrations and elements (A = Ca, Sr, Cd; B = Zr, Sn, Hf), aiming at the maximization of the energy storage density, and stoichiometric ABO3 perovskites where different elements are changed in the A and B sites for the minimization of the formation energy. AL for experimental design is implemented in the machine learning agent for chemistry and design (MLChem4D) software, which has the potential to be applied in inorganic and organic synthesis (e.g., search for the optimum concentrations, catalysts, reactants, temperatures, and pH to improve the yield) and materials science (e.g., search the periodic table for the proper elements and their concentrations to improve the materials properties). The latter marks the first MLChem4D application for the design of perovskites.

Application of Artificial Intelligence and Audit Quality in Nigeria

This study examined the effect of artificial intelligence on the quality of audit practice in Nigeria. Specifically, examined how expert systems, machine learning, and intelligent agents affect audit quality in Nigeria. The study employed a survey research design. The population of this study comprised 178 practicing accounting firms in Nigeria with the application of artificial intelligence. A sample size of 125 was selected using the purposive sampling technique. Data was obtained from primary sources using a well-structured questionnaire. Data were analysed using descriptive statistics and OLS regression analysis. The result of the study indicated that expert systems, machine learning, and intelligent agents exhibited a significant positive relationship to audit quality in Nigeria. Thus, the study concluded that the application of artificial intelligence positively influences the quality of auditors’ reports. With the positive impact made by artificial intelligence, in the coming decades, intelligent systems will take over more and more decision-making tasks from humans. The study recommended constant training of accountants and audit personnel on the use of AI techniques for the improvement of audit quality. Audit firms in Nigeria should invest in machine learning tools to further improve audit quality in the country. Also, the use of intelligent agents that assist in the classification of detecting objects into different categories should be increased among audit firms in Nigeria. Keywords: Artificial Intelligence, Expert Systems, Machine Learning, Intelligent Agents, Audit Quality. JEL Classification: M42, M49, C451. Journal Reference Format: Dagunduro, M.E., Falana, G.A., Adewara, Y.M. & Busayo, T.O. (2023): Application of Artificial Intelligence and Audit Quality in Nigeria. Humanities, Management, Arts, Education & the Social Sciences Journal. Vol. 11. No. 1, Pp 39-56. www.isteams.net/humanitiesjournal . dx.doi.org/10.22624/AIMS/HUMANITIES/V11N1P4

Online fault classification in Connected Autonomous Vehicles using output-only measurements

Different health-monitoring techniques were considered in the literature to enhance the safety and stability of Connected Autonomous Vehicle (CAV) platoons. Mitigating these faults is faster and more reliable if the fault structure is known. In this paper, we propose using transmissibility operators, which are relationships that relate a set of velocities with another in the platoon, to classify the faults. Transmissibility operators were shown to be exceptional in signal estimation. However, it is also shown to be noncausal and thus can only be used offline. To this end, we propose using Data Aggregation (DAgger), an extension to imitation learning, to transfer the classification experience from transmissibility operators to a novice machine learning agent to be used online. The integration of transmissibility-DAgger gives the ability to adapt to new fault classes without the need to re-train the diagnosis model from the beginning. A heterogeneous CAV platoon was modeled with three different faults separately. These faults are actuator disturbances, false data injection attacks, and communication time delays. The classification scheme depends on estimating the faulty signal in the case of each fault class. Next, the measured faulty signal is compared with the three estimations, and the closer fault estimation to the measured one is considered the actual fault on the platoon. The proposed algorithm is then tested on the platoon model and then applied to an experimental setup that consists of three autonomous robots. The proposed results are compared with six different machine learning classification models. The overall classification accuracy achieved was 95.8% for the experiment using the proposed approach.

HUMANISE: Human-Inspired Smart Management, towards a Healthy and Safe Industrial Collaborative Robotics.

The workplace is evolving towards scenarios where humans are acquiring a more active and dynamic role alongside increasingly intelligent machines. Moreover, the active population is ageing and consequently emerging risks could appear due to health disorders of workers, which requires intelligent intervention both for production management and workers' support. In this sense, the innovative and smart systems oriented towards monitoring and regulating workers' well-being will become essential. This work presents HUMANISE, a novel proposal of an intelligent system for risk management, oriented to workers suffering from disease conditions. The developed support system is based on Computer Vision, Machine Learning and Intelligent Agents. Results: The system was applied to a two-arm Cobot scenario during a Learning from Demonstration task for collaborative parts transportation, where risk management is critical. In this environment with a worker suffering from a mental disorder, safety is successfully controlled by means of human/robot coordination, and risk levels are managed through the integration of human/robot behaviour models and worker's models based on the workplace model of the World Health Organization. The results show a promising real-time support tool to coordinate and monitoring these scenarios by integrating workers' health information towards a successful risk management strategy for safe industrial Cobot environments.

Machine Learning Agents Augmented by Digital Twinning for Smart Production Scheduling

Digital manufacturing tools aim to provide intelligent solutions that can support manufacturing industry to adapt to the volatile operational environment. The successful implementation of such tools highly depends on the capabilities of the digital frameworks or platforms they are deployed upon as well as the quality of their intelligence. The objective of this work is to develop and discuss a framework for training and deploying Machine Learning (ML) agents for production scheduling with the augmentation of Digital Twin (DT) technologies. Two types of ML production scheduling agents have been developed and integrated with the DT framework: a Deep Learning agent and a Deep Reinforcement Learning agent. In order to increase interoperability, Asset Administration Shell Industry4.0 standard has been utilized for the integration and deployment of the proposed DT framework into industrial practice. The proposed framework is tested and validated upon an industrial case study from the bicycles’ production industry.

Industrial Robot Training in the Simulation Using the Machine Learning Agent

Continuous change in manufacturing requires robotization, requiring a skilled workforce with robotic skills. It is also important for a manufacturing company to be able to transform its production process quickly. But now it is a long and complex process. The paper presents the simulation of the movement of an industrial robot in a digital environment, to which implemented the inverse kinematics functionality and machine learning model have been applied. The use of machine learning reduces the time required to develop the process and the investment in finding the path of the robot. The results obtained in the application of Bio-ik inverse kinematics and machine learning have been observed and analyzed as a simulation in the created research.

Harnessing Machine Learning and Multi Agent Systems for Health Crisis Analysis in North Africa

The COVID-19 pandemic has presented a significant global health challenge, including in Morocco. These actions had direct repercussions on the economy as well as essential institutions in society; however, there were also indirect effects from these changes. This article focuses on these indirect consequences on the environment’s sustainability. It demonstrates that the net effect has been good in terms of reduced carbon gases, oil exploration operations, and pollution. This study introduces a novel approach to predicting and simulating the pandemic’s dynamics in Morocco using machine learning and multi-agent system models. We collected and processed daily data on COVID-19 cases, deaths, and interventions in Morocco from March 2, 2020, to June 30, 2021. We developed and validated several machine learning models, including decision trees, random forests, and support vector machines, to predict daily COVID-19 cases and deaths. Additionally, we designed a multi-agent system model to simulate the interactions among individuals, social groups, and the government in response to the pandemic, using agent-based modelling and game theory. Our results indicate that the machine learning models achieved high accuracy and generalization performance, with an average R-squared value of 0.83 for the cases and 0.90 for the deaths. The multi-agent simulations reveal the complex dynamics and trade-offs among pandemic control measures, economic activity, and social welfare in Morocco, suggesting that a coordinated and adaptive approach is necessary to balance these factors. Our study contributes to the growing literature on using machine learning and multiagent systems for pandemic prediction and management, providing valuable insights and recommendations for policymakers and public health officials in Morocco and beyond.

An iterative regulatory process for robot governance

Abstract There is an increasing gap between the policy cycle’s speed and that of technological and social change. This gap is becoming broader and more prominent in robotics, that is, movable machines that perform tasks either automatically or with a degree of autonomy. This is because current legislation was unprepared for machine learning and autonomous agents. As a result, the law often lags behind and does not adequately frame robot technologies. This state of affairs inevitably increases legal uncertainty. It is unclear what regulatory frameworks developers have to follow to comply, often resulting in technology that does not perform well in the wild, is unsafe, and can exacerbate biases and lead to discrimination. This paper explores these issues and considers the background, key findings, and lessons learned of the LIAISON project, which stands for “Liaising robot development and policymaking,” and aims to ideate an alignment model for robots’ legal appraisal channeling robot policy development from a hybrid top-down/bottom-up perspective to solve this mismatch. As such, LIAISON seeks to uncover to what extent compliance tools could be used as data generators for robot policy purposes to unravel an optimal regulatory framing for existing and emerging robot technologies.

AGILEST approach: Using machine learning agents to facilitate kinesthetic learning in STEM education through real-time touchless hand interaction

There is an increasing interest in creating interactive learning applications using innovative interaction technologies, especially in STEM (Science, technology, engineering, and mathematics) subjects. Recent developments in machine learning have allowed for nearly perfect hand-tracking recognition, introducing a touchless modality for interaction within Augmented Reality (AR) environments. However, the research community has not explored the pedagogical approach of Kinesthetic Learning or “Learning by Doing”, hand tracking, and machine learning agents combined with Augmented Reality technology. Fundamentally, this exploration of touchless interaction technologies has taken on new importance in the new post-COVID world. Meanwhile, machine learning has gained attention for its ability to enhance personalized learning and play a vital new role as a virtual instructor. This paper proposes a novel approach called the AGILEST approach, which uses machine learning Agents to facilitate interactive kinesthetic learning in STEM education through touchless interaction. The first case study for this approach will be an AR learning application for chemistry. This application uses real-time touchless hand interaction for kinesthetic learning and uses a machine learning agent to act as both trainer and assessor of the user. The evaluation of this research has been conducted remotely through a usability study with expert reviewers, which includes 15 young researchers with peer-reviewed work in Human-Computer Interaction & AR and 2 subject experts STEM teachers at the secondary school level. The usability evaluation through NASA Task Load Index (NASA-TLX), Perceived Ease of Use (PUEU), and Perceived Usefulness (PU) with expert reviewers provide positive feedback about this approach for productive learning gain, engagement and interactiveness in learning STEM subjects.

Current Challenges and Future Research Directions in Augmented Reality for Education

The progression and adoption of innovative learning methodologies signify that a respective part of society is open to new technologies and ideas and thus is advancing. The latest innovation in teaching is the use of Augmented Reality (AR). Applications using this technology have been deployed successfully in STEM (Science, Technology, Engineering, and Mathematics) education for delivering the practical and creative parts of teaching. Since AR technology already has a large volume of published studies about education that reports advantages, limitations, effectiveness, and challenges, classifying these projects will allow for a review of the success in the different educational settings and discover current challenges and future research areas. Due to COVID-19, the landscape of technology-enhanced learning has shifted more toward blended learning, personalized learning spaces and user-centered approach with safety measures. The main findings of this paper include a review of the current literature, investigating the challenges, identifying future research areas, and finally, reporting on the development of two case studies that can highlight the first steps needed to address these research areas. The result of this research ultimately details the research gap required to facilitate real-time touchless hand interaction, kinesthetic learning, and machine learning agents with a remote learning pedagogy.

Multiple Types of AI and Their Performance in Video Games

"In this article, we present a comparative study of Artificial Intelligence training methods, in the context of a racing video game. The algorithms Proximal Policy Policy Optimization (PPO), Generative Adversarial Imitation Learning (GAIL) and Behavioral Cloning (BC), present in the Machine Learning Agents (ML-Agents) toolkit have been used in several scenarios. We measured their learning capability and performance in terms of speed, correct level traversal, number of training steps required and we explored ways to improve their performance. These algorithms prove to be suitable for racing games and the toolkit is highly accessible within the ML-Agents toolkit. Keywords and phrases: racing game, PPO, GAIL, behavioral cloning, AI in games. "

Empirical Study of Adaptive Serious Games in Enhancing Learning Outcome

Use of serious games to teach concepts of various important topics including Cybersecurity is growing. A figure of merit for the serious games could be learning outcome and user experience(UX). With enhanced learning outcome and user experience, the player is likely to favourably rate a game. The organisation supporting such games would also benefit from such efficient training process.We report an empirical comparison of two cybersecurity games namely ; Use of Firewalls for network protection and concepts of Structured Query Language (SQL) injections to get unauthorised access to online databases. We have designed these games in two versions. The version without using adaptive features provide a baseline to compare efficacy of the machine learning based adaptive game while comparing the learning outcomes and user experience (UX). The efficacy of the Machine Learning (ML) agent in providing the adaptability to the game play is based on classification of player to two categories viz. Beginner and Expert using historical player data on three relevant attributes. The game dynamics is modulated based on the player classification to ensure that game challenge is optimally suited to player type and the player continues to experience playful flow in different stages of the game.

Anticheat System Based on Reinforcement Learning Agents in Unity

Game cheating is a common occurrence that may degrade the experience of “honest” players. It can be hindered by using appropriate anticheat systems, which are being considered as a subset of security-related issues. In this paper, we implement and test an anticheat system whose main goal is to help differentiate human players from AI players. For this purpose, we first developed a multiplayer game inside game engine Unity that would serve as a framework for training the reinforcement learning agent. This agent would thus learn to differentiate human players from bots within the game. We implemented the Machine Learning Agents Toolkit library, which uses the proximal policy optimization algorithm. AI players are implemented using state machines, and perform certain actions depending on which condition is satisfied. Two experiments were carried out for testing the agent and showed promising results for identifying artificial players.

Optimisation of Operator Support Systems through Artificial Intelligence for the Cast Steel Industry: A Case for Optimisation of the Oxygen Blowing Process Based on Machine Learning Algorithms

The processes involved in the metallurgical industry consume significant amounts of energy and materials, so improving their control would result in considerable improvements in the efficient use of these resources. This study is part of the MORSE H2020 Project, and it aims to implement an operator support system that improves the efficiency of the oxygen blowing process of a real cast steel foundry. For this purpose, a machine learning agent is developed according to a reinforcement learning method suitable for the dynamics of the oxygen blowing process in the cast steel factory. This reinforcement learning agent is trained with both historical data provided by the company and data generated by an external model. The trained agent will be the basis of the operator support system that will be integrated into the factory, allowing the agent to continue improving with new and real experience. The results show that the suggestions of the agent improve as it gains experience, and consequently the efficiency of the process also improves. As a result, the success rate of the process increases by 12%.