Decisions Of Agents Research Articles

Adaptive and transparent decision-making in autonomous robots through graph-structured world models

ABSTRACT The growing ubiquity of autonomous robots across different fields necessitates that agents adapt to diverse tasks and ensure transparency and intelligibility in their decision-making processes. This study presents a novel framework that combines a graph-structured world model with large language models (LLMs) to address these requirements. First, a latent space is created to capture the reachability between distinct states. Next, a graph is constructed within this latent space by clustering an offline dataset that effectively captures the complex dynamics of the environment. Subsequently, LLMs are employed to redefine the reward function and relabel the dataset, thereby establishing a well-defined Markov decision process based on the previously learned graph. This relabeling process ensures that the agent's decision space aligns with user intentions. Consequently, a predictive world model is obtained, offering insights into potential future states and facilitating graph-based planning. Moreover, by inputting the planned path of the agent in the graph-structured world model into LLMs, natural language explanations can be generated to provide transparency in the decision-making process. Experiments on the D4RL benchmark validated the effectiveness of our approach in long-horizon planning, its adaptability to different user tasks, and its inherent explainability.

Bounded Rational Decision Networks with Belief Propagation.

Complex information processing systems that are capable of a wide variety of tasks, such as the human brain, are composed of specialized units that collaborate and communicate with each other. An important property of such information processing networks is locality: there is no single global unit controlling the modules, but information is exchanged locally. Here, we consider a decision-theoretic approach to study networks of bounded rational decision makers that are allowed to specialize and communicate with each other. In contrast to previous work that has focused on feedforward communication between decision-making agents, we consider cyclical information processing paths allowing for back-and-forth communication. We adapt message-passing algorithms to suit this purpose, essentially allowing for local information flow between units and thus enabling circular dependency structures. We provide examples that show how repeated communication can increase performance given that each unit's information processing capability is limited and that decision-making systems with too few or too many connections and feedback loops achieve suboptimal utility.

The itinerary of children in search of healthcare: A scoping review and proposal of an explanatory model.

This scoping review mapped the academic literature focused on the therapeutic itinerary of children who seek care in health services and proposed an explanatory model to expand the concept and classification of these health itineraries. A total of 789 articles were reviewed, of which 28 were eligible for inclusion. In these 28 it was possible to observe that the child's therapeutic itinerary is more than a physical path, but also encompasses all choices within a specific social and cultural environment in which the child is inserted. Our proposal is to expand the concept beyond the therapeutic, classifying the itinerary also according to the objective, the decision-making agent, respect for the presence of company, the health subsystem used, according to the physical continuity of the itinerary, the perception of efficacy of the patient, the nature of the illness, the administration of healthcare, the means of transport used, the person providing information about the itinerary, the planning of the itinerary and its completeness. Knowing the child's itineraries toward healthcare allows the development of innovative discourses and practices for future public policies, through which the principles of comprehensiveness and resoluteness in children's health would be strengthened. There is still a need to deepen knowledge about the meanings and feelings regarding their interpretations of the events suffered in childhood.

Present-biased preferences and the effect of illiquid assets

This paper incorporates liquidity constraints into a discrete three-stage model to investigate the consumption-savings decisions of present-biased agents. We derive the analytical solution for consumption choices of sophisticated and naive agents who hold illiquid assets that are untradable until the last period. Then, we investigate how the proportion of illiquid assets, liquidity premium and agents’ preferences affect the consumption of both agents. Our findings reveal that naive agents not always consume more than their sophisticated counterparts. Moreover, we theoretically find that illiquid assets may improve the welfare of agents and have a pre-commitment effect on both agents.

Learning in Inverse Optimization: Incenter Cost, Augmented Suboptimality Loss, and Algorithms

Enhancing the Efficiency and Accuracy of Inverse Optimization Inverse optimization (IO) is used to model the behavior of decision-making agents who solve optimization problems in response to external signals. Inspired by the geometry of IO problems, in “Learning in Inverse Optimization: Incenter Cost, Augmented Suboptimality Loss, and Algorithms,” Zattoni Scroccaro, Atasoy, and Mohajerin Esfahani propose the “incenter” concept to solve IO problems, which unlike previously proposed approaches, can be used to derive computationally tractable solutions to this modeling problem. Moreover, they also propose a novel loss function for IO problems and a tailored optimization algorithm to optimize it. Extensive numerical experiments showcase the improved efficiency and accuracy of the proposed IO formulations and algorithm.

Norm Emergence through Conflict-Blocking Interactions in Industrial Internet of Things Environments.

Norms have been effectively utilized to facilitate smooth interactions among agents. Norms are usually the global data that agents cannot directly access in complex environments; instead, norms can only be indirectly accessed by agents via maintaining their own beliefs about norms. Establishing norms using decentralized interaction-based methods has attracted much attention. However, the current methods overlook Industrial Internet of Things (IIoT) environments. In IIoT, there is a prevalent feature called "conflict-blocking", where agents' conflicting action strategies can block an interaction from being completed or even cause danger. To facilitate norm emergence in IIoT, we propose a framework to support agent decisions in conflict-blocking interactions. The framework aids in achieving system scalability by integrating the fusion of agent beliefs about norms. We prove that the proposed framework guarantees norm emergence. We also theoretically and experimentally analyze the time required for norm emergence under the influence of various factors, such as the number of agents. A vehicle movement simulator is also developed to vividly illustrate the process of norm emergence.

Active Inference and Reinforcement Learning: A Unified Inference on Continuous State and Action Spaces Under Partial Observability.

Reinforcement learning (RL) has garnered significant attention for developing decision-making agents that aim to maximize rewards, specified by an external supervisor, within fully observable environments. However, many real-world problems involve partial or noisy observations, where agents cannot access complete and accurate information about the environment. These problems are commonly formulated as partially observable Markov decision processes (POMDPs). Previous studies have tackled RL in POMDPs by either incorporating the memory of past actions and observations or by inferring the true state of the environment from observed data. Nevertheless, aggregating observations and actions over time becomes impractical in problems with large decision-making time horizons and high-dimensional spaces. Furthermore, inference-based RL approaches often require many environmental samples to perform well, as they focus solely on reward maximization and neglect uncertainty in the inferred state. Active inference (AIF) is a framework naturally formulated in POMDPs and directs agents to select actions by minimizing a function called expected free energy (EFE). This supplies reward-maximizing (or exploitative) behavior, as in RL, with information-seeking (or exploratory) behavior. Despite this exploratory behavior of AIF, its use is limited to problems with small time horizons and discrete spaces due to the computational challenges associated with EFE. In this article, we propose a unified principle that establishes a theoretical connection between AIF and RL, enabling seamless integration of these two approaches and overcoming their limitations in continuous space POMDP settings. We substantiate our findings with rigorous theoretical analysis, providing novel perspectives for using AIF in designing and implementing artificial agents. Experimental results demonstrate the superior learning capabilities of our method compared to other alternative RL approaches in solving partially observable tasks with continuous spaces. Notably, our approach harnesses information-seeking exploration, enabling it to effectively solve reward-free problems and rendering explicit task reward design by an external supervisor optional.

Adaptive Control for Underwater Simultaneous Lightwave Information and Power Transfer: A Hierarchical Deep-Reinforcement Approach

In this work, we consider a point-to-point underwater optical wireless communication scenario where an underwater sensor (US) transmits its sensing data to a remotely operated vehicle (ROV). Before the US transmits its data to the ROV, the ROV performs simultaneous lightwave information and power transfer (SLIPT), delivering both control data and lightwave power to the US. Under the considered scenario, our objective is to maximize energy harvesting at the US while supporting predetermined communication performance between the two nodes. To achieve this objective, we develop a hierarchical deep Q-network (DQN)–deep deterministic policy gradient (DDPG)-based online algorithm. This algorithm involves two reinforcement learning agents: the ROV and US. The role of the ROV agent is to determine an optimal beam-divergence angle that maximizes the received optical signal power at the US while ensuring a seamless optical link. Meanwhile, the US agent, which is influenced by the decision of the ROV agent, is responsible for determining the time-switching and power-splitting ratios to maximize energy harvesting without compromising the required communication performance. Unlike existing studies that do not account for adaptive parameter control in underwater SLIPT, the proposed algorithm’s adaptive nature allows for the dynamic fine-tuning of optimization parameters in response to varying underwater environmental conditions and diverse user requirements.

Özgür ÖZDAMAR and Sercan CANBOLAT, Leaders in the Middle East and North Africa: How Ideology Shapes Foreign Policy

It is almost always the case that any Introduction to International Relations course teaches the contributions of Kenneth Waltz to the field, firstly through his grandiose formulation of a theory: it should be simple, parsimonious, abstract, and accordingly, one needs to move away from reality as much as possible to increase the theory’s explanatory and predictive capacity. Moving the individual and state level of analyses aside, Waltz simplifies his theory of international politics at the system/structural level and, with his neorealist theory becoming the dominant approach for a large part of the twentieth century, it put the state in a sort of black box, purposefully ignored individual actors, and targeted the maximum degree of abstraction as possible. In Leaders in the Middle East and North Africa: How Ideology Shapes Foreign Policy, which consists of seven chapters (one introduction chapter, four empirical chapters, one theoretical conclusions chapter and one policy implications chapter), Özdamar and Canbolat aim to shed light on foreign policy belief patterns of leaders in the Middle East and North Africa (MENA), highlighting divergences across them and comparing them to the average world leadership. In the process, and in total opposition to Waltz’s claim, the authors argue that advancing “the actor-specific empirical studies zeroing in on agent behaviors and decisions in the future” is “the only way for IR to establish itself as a scientific discipline” (p. 148). In other words, as opposed to the structural approaches, the authors propound that less abstraction and a more actor-specific, nuanced, and tailored approach would provide significant opportunities for the IR discipline to be scientific.

Supply responsive scheduling for ethylene cracking furnace systems based on deep reinforcement learning

AbstractEthylene is one of the most important chemicals, and scheduling optimization is crucial for the profitability of ethylene cracking furnace systems. With the diversification of feedstocks and the high variability in prices, supply chain fluctuations pose significant challenges to the scheduling decisions. Dynamically responding to these fluctuations has become crucial. Traditional mixed integer nonlinear programming (MINLP) models lack the capability of supply chain response, while receding horizon optimization (RHO) models require parameter prediction and repeated optimization solving. To address this challenge, we propose a deep reinforcement learning‐based framework that includes an ethylene dynamic scheduling environment and a decision agent based on deep Q‐network. Across three test cases, compared to the MINLP and RHO models, this framework significantly minimizes losses caused by supply chain fluctuations, thereby increasing daily average net profits by 9%–27%, demonstrating its significant potential for application in responsive scheduling in the presence of supply chain fluctuations.

Strategic behaviour in the mean field Ising game

The properties of the mean field version of the Ising game of far-sighted agents on a complete graph are considered. General evolution equation for arbitrary yield of simultaneously decision-making agents is derived. Analytical expressions demonstrating strategic stiffening/cooling in the limits of small and large yield of simultaneously decision-making agents are derived.

Optimal investment with insurable background risk and nonlinear portfolio allocation frictions

We study optimal investment and insurance demand in a continuous-time model that combines risky assets with an insurable background risk. This risk takes the form of a jump-diffusion process that reduces the return rate of the agent's wealth. The main distinctive feature of our model is that the agent's decision on portfolio choice and insurance demand causes nonlinear friction in the dynamics of the wealth process. We use the HJB equation to find the optimal conditions for the agent to fully, partially, or totally insure against the background risk. We consider different types of friction, such as differential borrowing and lending rates. We also show a mutual-fund separation result and provide numerical examples.

The way out for urban household solid waste classification in China: From the perspective of multi-agent based behavior simulation

Rapid industrialization and urbanization drive China's massive urban household solid waste (HSW) generation. Classification and resource recycling of HSW is an effective way to address the “garbage-sieged city” dilemma and promote “urban mineral resources” exploitation. Despite efforts since 2000, HSW sorting and recycling remain at 30%. This study develops a multi-agent based behavior simulation model for HSW classification and recycling, incorporating the interactions and decision-making of key agents, including residents, enterprises, and the government. Taking Suzhou, China as a case study, field surveys were conducted, and behavioral decision-making functions were constructed to simulate micro-behavioral responses and macro-system evolution under different policy incentives, and to predict the implementation effect of domestic waste management approaches, provides the basis for decision-making. The results demonstrate that a combination of waste metering charges, recycling subsidies, and enhanced education program can effectively regulate agent behavior and optimize the system. One year after the implementation of the simulation, the number of “unclassification” residents has dropped from 1.85 million to 250,000 decreased by 86%; the number of “classification deposition” residents has declined from 1.28 million to 1.08 million, reduced by 16%; the number of “selling after classification” residents has risen from 1.05 million to 2.81 million, increased by 63%; the daily waste output per capita has decreased from 1.21 kg to 1.04 kg, reduced by 14%; the recovery rate of resources has been raised from 33.4% to 66.5%. The implementation effects of other policy scenarios are ranked as follows: the metering charge > the recycling subsides > the enhanced education program. The findings provide a basis for decision-making in HSW management. Concerning the metering charge and the recycling subsidies, they can benefit positive interactions encouraged by such economic stimulus. At the same time, it is suggested to strengthen classification and recycling infrastructure, and push forward the integration of HSW classification recycling.

Challenges in Accounting for Carbon: Analysis of Accounting Policies of the World's Largest Polluters

The study's focus was on examining the accounting policies related to carbon credits of the top 100 global GHG emitters, as mentioned by Kouloukoui et al. (2019). Of these 100, only 36 gave access to the CDP questionnaire. We used the model proposed by Lovell et al. (2010) to examine carbon credit accounting choices, employing content analysis with NVivo software and statistical analysis with Stata. The financial statements (FS) and explanatory notes of 247 companies from different countries and sectors, such as oil and gas, mining, coal, transport and marine logistics, and the chemical sector, were analyzed from 2015 to 2021. Finally, nine companies provided information on the accounting treatment for carbon credits. The results indicate a predominance of classification as inventories, followed by intangible assets, measurement at cost, and liabilities accounted for by the excess carbon credits owed about existing assets at market value. We observed significant heterogeneity in accounting policies, which remained over time. The lack of standardization and consistency in reporting methods and the concealment of accounting records in this context make it challenging to compare practices, compromising the evaluation of companies' performance, the decision-making of investors and other agents, and the transparency and accountability of this market. These findings highlight the need for a more coherent and rigorous approach to accounting for carbon credits, especially considering the urgent challenges of climate change.

Participating together in CP-ACHIEVE: Experiences, opportunities and reflections from a collaborative research team of people with lived experience of cerebral palsy and health care professionals

The Australian Centre for Health, Independence, Economic participation and Value Enhanced care for adolescents and young adults with Cerebral Palsy (CP-ACHIEVE) is a Centre of Research Excellence (CRE), funded for 5-year by the National Health and Medical Research Council of Australia. The vision of CP-ACHIEVE is an Australia where people with cerebral palsy receive excellent healthcare throughout their lives and live in, and contribute to, supportive communities that welcome and enable their participation. CP-ACHIEVE began with the ethical commitment to bring together people with lived experience of cerebral palsy, researchers, and health professionals to develop and conduct research informed by, and relevant to, people with cerebral palsy and their allies. From inception, co-research and collaboration with (not ‘to’ or ‘about’) young people with cerebral palsy (10 to 30 years of age) and their families has been central to our work. In this paper, we describe the CP-ACHIEVE values, structure and strategy for this approach, and its implementation at each stage of the research process. We then provide an example of the strategy in action, using a qualitative exploration of CP-ACHIEVE’s Participation Theme team’s experiences of collaboration and involvement as co-researchers. Active participation in research for young people with lived experience of cerebral palsy and their families is a fundamental human right, based on their right to be active agents in decisions that affect them. In this paper we explore how our collaborative approach, and the integration of diverse views, has enhanced the relevance, quality, usefulness, and translation of our research. We also describe (i) the structural elements of our research group that have facilitated our work together, (ii) our challenges, and (iii) how the ownership of our research by people with cerebral palsy is driving future research directions and empowering involvement of people with lived experience beyond CP-ACHIEVE. We offer this knowledge and our experiences to assist other research teams in their journeys towards collaborative research alongside people with lived experience of disability.

Renewable Energy, Economic Policy Uncertainty and Climate Policy Uncertainty: New Evidence for Environmental Kuznets Curve from Emerging and Developed Countries

Recent events, such as the financial crisis, oil price shocks or fluctuations, Brexit, the US–China trade war, the COVID-19 pandemic, the Russia–Ukraine conflict and the subsequent energy crisis, have surged global economic policy uncertainty. As climate change has recently been more pronounced around the globe, discussions about climate policies and related uncertainties have also become a major concern. This study investigates the role of economic policy uncertainty (EPU) and climate policy uncertainty (CPU) on climate change (environmental degradation) for selected emerging and developed economies, expanding the IPAT framework and merging it with the Environmental Kuznets Curve (EKC) hypothesis. The IPAT framework examines the impact (I) of population (P), affluence (A), and technology (T) on the environment, whereas the EKC hypothesis proposes an inverted U-shaped curve between affluence and environmental degradation. Two models were created and tested for emerging and developed countries, namely Model 1 with EPU and Model 2 with CPU. A Pooled Mean Group (PMG) estimator is employed to investigate the interrelation between carbon dioxide (CO2) emissions and selected variables; namely the real Gross Domestic Product (GDP) per capita, squared real GDP per capita, renewable share in consumption, the EPU, the CPU and population. Test results indicate that the EKC hypothesis is verified only in Model 1 and for emerging countries, whereas population escalates climate change in both country groups. Furthermore, in line with the consumption effect theorized earlier in the literature, EPU is negatively related to carbon emissions in emerging countries. Thus, the EPU leads to a decrease in the use of energy and pollution-intensive commodities and mitigates climate change in EMEs. Compatible with our ex-ante expectations, renewable energy consumption alleviates climate change in both country groups in the short term. In Model 2, with CPU, we find no evidence supporting the EKC hypothesis for any country groups. However, we reaffirm that renewable energy consumption decreases CO2 emissions in developed countries, which is in support of the argument that energy transition holds the key to tackling climate change. Finally, CPU is associated with a decrease in CO2 emissions in emerging countries in the short term, potentially leading to a reduction in overall economic activity and alleviating climate change. This might also be attributable to the fact that the decisions of economic agents substantially rely on current and future policy (both economic and climate) expectations. Overall, verifying the EKC hypothesis for emerging countries in Model 1, we might argue that there is good potential for emerging countries to save money and time on environmental costs via the adoption of clean technologies and related policies. Last but not least, on a global scale, energy transition with better utilization of renewable sources holds the key to tackling climate change and reducing emissions.

Hierarchical Coordinated Predictive Control of Multiagent Systems for Process Industries

Focusing on the requirements for efficient and accurate control in large-scale process industries with integrated “distributed-decentralized” characteristics, a novel hierarchical coordinated predictive control strategy for process industries is proposed with a multiagent system as the computational paradigm. This approach comprehensively considers the overall state of the system, the interactions of control actions among agents, the constraints of processes and energy consumption to solve the problems of poor flexibility of agent decision-making in the narrow consensus strategy and strong interaction of parts of the system. The proposed hierarchical control strategy requires each agent to perform three tasks at each time step. First, each agent iteratively obtains a consistent basis for closed-loop prediction in a distributed way. Then, each agent independently proposes a control scheme and determines its own priority by playing games based on the economic performance of the scheme. Next, each agent calculates its own optimal dynamic predictive control sequence in order of priority based on the system’s dynamic process model. Finally, by considering the temperature-control process of heating an alumina ceramic block in a high-power microwave reactor with six microwave sources, the effectiveness of the proposed hierarchical coordinated predictive control strategy is verified under different communication topologies by comparing it with the centralized model predictive control strategy.

Quantum-inspired modeling of distributed intelligence systems with artificial intelligent agents self-organization

Distributed intelligence systems (DIS) containing natural and artificial intelligence agents (NIA and AIA) for decision making (DM) belong to promising interdisciplinary studies aimed at digitalization of routine processes in industry, economy, management, and everyday life. In this work, we suggest a novel quantum-inspired approach to investigate the crucial features of DIS consisting of NIAs (users) and AIAs (digital assistants, or avatars). We suppose that N users and their avatars are located in N nodes of a complex avatar - avatar network. The avatars can receive information from and transmit it to each other within this network, while the users obtain information from the outside. The users are associated with their digital assistants and cannot communicate with each other directly. Depending on the meaningfulness/uselessness of the information presented by avatars, users show their attitude making emotional binary “like”/“dislike” responses. To characterize NIA cognitive abilities in a simple DM process, we propose a mapping procedure for the Russell’s valence-arousal circumplex model onto an effective quantum-like two-level system. The DIS aims to maximize the average satisfaction of users via AIAs long-term adaptation to their users. In this regard, we examine the opinion formation and social impact as a result of the collective emotional state evolution occurring in the DIS. We show that generalized cooperativity parameters Gi\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$G_i$$\\end{document}, i=1,⋯,N\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$i=1,\\dots,N$$\\end{document} introduced in this work play a significant role in DIS features reflecting the users activity in possible cooperation and responses to their avatar suggestions. These parameters reveal how frequently AIAs and NIAs communicate with each other accounting the cognitive abilities of NIAs and information losses within the network. We demonstrate that conditions for opinion formation and social impact in the DIS are relevant to the second-order non-equilibrium phase transition. The transition establishes a non-vanishing average information field inherent to information diffusion and long-term avatar adaptation to their users. It occurs above the phase transition threshold, i.e. at Gi>1\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$G_i>1$$\\end{document}, which implies small (residual) social polarization of the NIAs community. Below the threshold, at weak AIA–NIA coupling (Gi≤1\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$G_i\\le 1$$\\end{document}), many uncertainties in the DIS inhibit opinion formation and social impact for the DM agents due to the information diffusion suppression; the AIAs self-organization within the avatar–avatar network is elucidated in this limit. To increase the users’ impact, we suggest an adaptive approach by establishing a network-dependent coupling rate with their digital assistants. In this case, the mechanism of AIA control helps resolve the DM process in the presence of some uncertainties resulting from the variety of users’ preferences. Our findings open new perspectives in different areas where AIAs become effective teammates for humans to solve common routine problems in network organizations.

Autonomous Small Body Science Operations Using Reinforcement Learning

This work studies the feasibility of using reinforcement learning for small body science operations subject to resource constraints. Two mission scenarios are considered. In the first scenario, a spacecraft autonomously maneuvers between waypoints about a small body while performing science activities, such as mapping and imaging, and periodically downlinking data and managing on-board resources like battery charge, data buffer storage, and fuel usage. In the second scenario, the spacecraft periodically performs navigation updates to improve its state estimate, ensuring that the collected science is within the specified requirements. A Markov decision process formulation of the mission scenarios is formulated, and reinforcement learning is applied to solve the problem. A range of noisy observation types are tested, demonstrating that a fully observable formulation of the problem trained on direct observations of the state is robust to noisy measurements or a filtered state estimate. A decision-making agent is then trained to manage the state estimate by choosing when to take measurements, demonstrating that near-equivalent policies, in comparison to nominal problem formulation, can be trained with an optional navigation update. Finally, a demonstration is performed in which a ground station outage is simulated. The decision-making agent is shown to be robust to this outage, rapidly adjusting its plan to continue nominal operations.

Developing an eco-driving strategy in a hybrid traffic network using reinforcement learning.

Eco-driving has garnered considerable research attention owing to its potential socio-economic impact, including enhanced public health and mitigated climate change effects through the reduction of greenhouse gas emissions. With an expectation of more autonomous vehicles (AVs) on the road, an eco-driving strategy in hybrid traffic networks encompassing AV and human-driven vehicles (HDVs) with the coordination of traffic lights is a challenging task. The challenge is partially due to the insufficient infrastructure for collecting, transmitting, and sharing real-time traffic data among vehicles, facilities, and traffic control centers, and the following decision-making of agents involved in traffic control. Additionally, the intricate nature of the existing traffic network, with its diverse array of vehicles and facilities, contributes to the challenge by hindering the development of a mathematical model for accurately characterizing the traffic network. In this study, we utilized the Simulation of Urban Mobility (SUMO) simulator to tackle the first challenge through computational analysis. To address the second challenge, we employed a model-free reinforcement learning (RL) algorithm, proximal policy optimization, to decide the actions of AV and traffic light signals in a traffic network. A novel eco-driving strategy was proposed by introducing different percentages of AV into the traffic flow and collaborating with traffic light signals using RL to control the overall speed of the vehicles, resulting in improved fuel consumption efficiency. Average rewards with different penetration rates of AV (5%, 10%, and 20% of total vehicles) were compared to the situation without any AV in the traffic flow (0% penetration rate). The 10% penetration rate of AV showed a minimum time of convergence to achieve average reward, leading to a significant reduction in fuel consumption and total delay of all vehicles.