Multi-agent Scheduling System Research Articles

Integration of deep reinforcement learning and multi-agent system for dynamic scheduling of re-entrant hybrid flow shop considering worker fatigue and skill levels

In real-life manufacturing systems, production management is often affected by urgent demands and unexpected interruptions, such as new job insertions, machine breakdowns and operator unavailability. In this context, agent-based techniques are useful and able to respond quickly to dynamic disturbances. The ability of agents to recognize their environment and make decisions can be further enhanced by deep reinforcement learning (DRL). This paper investigates a novel dynamic re-entrant hybrid flow shop scheduling problem (DRHFSP) considering worker fatigue and skill levels to minimize the total tardiness of all production tasks. An integrated architecture of DRL and MAS (DRL-MAS) is proposed for real-time scheduling in dynamic environments. Two DRL models are proposed for different sub-decisions, where a reward-shaping technique combining long-term and short-term returns is proposed for the job sequence and machine selection sub-decisions, and an attention-based network is proposed for the worker assignment sub-decision for efficient feature extraction and decision making. Numerical experiments and case studies demonstrate the superior performance of the proposed DRL models compared with existing scheduling strategies.

Online scheduling of dependent tasks of cloud’s workflows to enhance resource utilization and reduce the makespan using multiple reinforcement learning-based agents

Due to different heterogeneous cloud resources and diverse and complex applications of the users, an optimal task scheduling, which can satisfy users and cloud service providers with energy-saving and cost-effective use of resources, is a major issue in cloud computing. On the one hand, network users are demanding the quality assurance of their requested services, minimizing their costs, and their own data security, and on the other hand, the service providers consider less power consumption, more efficient use of resources, and optimal utilization. In dependent tasks dealing with massive data, resource scheduling is considered as an important challenge. Due to the time limitation of online scheduling process of dependent tasks, many existing methods of the literature are not able to guarantee the best resource utilization. In this paper, a reinforcement learning approach is exploited in a multi-agent system for task scheduling and resource provisioning, in order to reduce the makespan, minimize the required power, optimize the cost of using the resources, and maximize the utilization of the resources (considering their expiration time), simultaneously. The proposed algorithm has two phases. In the first phase, the tasks are scheduled using reinforcement learning techniques, and in the second one, considering the information obtained from the scheduling phase, resources are allocated in a multi-agent environment. The results of experiments show that this method improves the efficiency of the use of resources and reduces their costs. Moreover, the expiration time of the tasks is observed and the total execution time and energy consumption will be significantly reduced.

Distributed multi-agent scheduling and control system for robotic flexible assembly cells

This paper deals with the development of a distributed multi-agent system (DMAS) for scheduling and controlling Robotic Flexible Assembly Cells (RFAC). In the proposed system, an approach for solving one of the most challenging decisional problems in RFAC is proposed and implemented. This problem is related to the products operations scheduling which requires their allocation and sequencing on the robots, while satisfying products and robots constraints under makespan minimization. The proposed DMAS addresses this challenge by using a cooperative approach supported by three kinds of autonomous control agents: supervisory agent, local agents, and remote agents. These agents interact by a negotiation protocol based on common dispatching rules for coordinating their individual decisions, satisfying their local objective and providing an optimized global solution. Moreover, because of the dynamic nature of the assembly systems, it is imperative to consider external disturbances on production scheduling and to solve the related issues. Consequently, DMAS has the ability to respond and manage some dynamic events that may occur in the cells such as unexpected robot breakdown or dynamic products arrivals. Computational results on benchmarks show the effectiveness and the robustness of the proposed system.

Особенности реализации интерактивной мультиагентной системы построения программы полета, грузопотока и расчета ресурсов Российского сегмента Международной космической станции

Рассматривается реализация интерактивной мультиагентной системы построения программы полета, грузопотока и расчета ресурсов Российского сегмента Международной космической станции. Система решает проблемы поддержки жизнедеятельности экипажа и обеспечения нормального функционирования Российского сегмента Международной космической станции (PC МКС), а также повышения эффективности принятия решений специалистами РКК "Энергия" при построении программы полета, грузопотока и расчета ресурсов PC МКС. Система реализует метод адаптивного планирования программы полета и грузопотока PC МКС в реальном времени, учитывающий приоритеты, правила, ограничения грузов и транспортных кораблей. Показано, что система способна гибко и эффективно адаптировать программу полета и план грузопотока PC МКС по событиям в реальном времени.

Применение мультиагентной платформы для создания интеллектуальных систем управления ресурсами в реальном времени

Problem statement: The paper considers a task of multi-agent platform development for designing smart resource management systems in real time. Multi-agent platform is based on main classes of interacting agents with competing interests represented by satisfaction and penalty functions. Methods: A method of multi-criteria planning of production and transport resources is proposed, based on a modified model of demand and resource network of enterprises, where various criteria are alternately improved depending on the situation reflected by the state of resources and incoming event flow. Every cycle of replanning has two stages: conflict-free planning and proactive schedule improving with conflicts resolutions. Enterprise production orders and industrial/human resources serve as demands and resources, correspondingly. Demand and resource network agents have their own goals but are capable of making a compromise to achieve a common interest of the whole system. Results: The paper experimentally demonstrates the opportunity of solving complex tasks of resource management, which cannot be solved by traditional methods and known heuristics with required quality and efficiency. Practical value: The developed method and platform can be applied when designing industrial systems of resource management in various domains, providing a 20-40 % increase of enterprise efficiency. The following systems has been developed and put into operation: a multi-agent system for scheduling of flight program, cargo flow and resources of the Russian Segment of the International Space Station; a system of railway transportation management; a system for workshop management of a machine-building enterprise; a system of transportation logistics and some others.

Flexible testing platform for employment of RFID-enabled multi-agent system on flexible assembly line

The success of a flexible assembly line (FAL) depends on efficacious scheduling and control architecture. However, the scheduling and control architectures currently employed in FALs lack the flexibility and reconfiguration capacity to manage disturbances when they occur. Consequently, the system performance rapidly degrades when the system operation is interrupted. The objective of this study is to examine the potential enhancement of FAL performance through the use of a radio-frequency-identification-enabled multi-agent scheduling and control system (RFID-enabled MASCS). A simulation test platform is developed for the examination of an RFID-enabled MASCS in a FAL, and several system performance measures are considered in the simulation test platform. The results indicate that the RFID-enabled MASCS can increase the uptime productivity and production rate of a FAL. A real case-study test is performed, and a 22% decrease in lead time along with significant improvements in other system performance indicators are observed, especially when a series of disturbances occur within the examined assembly machine.

A new immune multi-agent system for the flexible job shop scheduling problem

Scheduling for the flexible job shop is very important and challenging in manufacturing field. Multi-agent-based approaches have been used to solve the flexible job shop scheduling problem (FJSP), in order to reduce complexity and cost, increase flexibility, and enhance robustness. However, the quality of solution obtained by the multi-agent approach is always worse than the centralized meta-heuristic algorithms. The immune system is a distributed and complicated information processing system, which can protect body from foreign antigens by immune responses. In this paper, we analyze the similarities between the FJSP and humoral immunity, which is one of the immune responses. Based on the similarities, we develop a new immune multi-agent scheduling system (NIMASS) to solve the FJSP with the objective of minimizing the maximal completion time (makespan). In order to acquire the higher-quality solution of the FJSP, we simulate humoral immunity to establish the architecture of NIMASS and the negotiation strategies of NIMASS, which are proposed for negotiation among agents. NIMASS was tested on different benchmark instances of the FJSP. In comparison with the multi-agent approaches and the centralized heuristic algorithms, the computational results indicate that NIMASS can effectively improve the quality of solution in very short time. And the computational time of NIMASS is superior to that of the centralized meta-heuristic algorithms, especially for the complex FJSPs. These results indicate that NIMASS can be very useful in applications that deal with real-time FJSPs.

Multi-agent job shop scheduling system based on co-operative approach of idle time minimisation

In this paper, a distributed multi-agent scheduling system (MASS) based on co-operative approach is proposed to solve static and dynamic job shop scheduling problems (JSSP). The proposed system is composed of two kinds of agents, Supervisor agents and Resource agents. The Supervisor agent decomposes JSSP into interrelated sub-problems and the Resource agents co-operate, through a distributed approach of local idle time minimisation, to solve this problem which is known as one of the most difficult NP-hard problems. Computational results are presented to show the efficiency of MASS in static job shop scheduling. Then, a comparison of the computational results between MASS and some common dispatching rules, on dynamic job arrivals, is studied in terms of effectiveness and stability. Finally, the developed system is validated within an illustrative example, to demonstrate the feasibility of MASS.

Dynamic flexible job shop scheduling with alternative process plans: an agent-based approach

This paper studies a flexible job shop problem considering dynamic events such as stochastic job arrivals, uncertain processing times, and unexpected machine breakdowns. Also, the considered job shop problem has routing flexibility and process flexibility. A multi-agent scheduling system has been developed for solution with good quality and robustness. A pheromone-based approach is proposed for coordination among agents. The proposed multi-agent approach is compared with five dispatching rules from literature via simulation experiments to statistical analysis. The simulation experiments are performed under various experimental settings such as shop utilization level, due date tightness, breakdown level, and mean time to repair. The results show that the proposed agent-based approach performs well under all problem settings.

Multi-agent system for energy resource scheduling of integrated microgrids in a distributed system

This paper proposes a multi-agent system for energy resource scheduling of an islanded power system with distributed resources, which consists of integrated microgrids and lumped loads. Distributed intelligent multi-agent technology is applied to make the power system more reliable, efficient and capable of exploiting and integrating alternative sources of energy. The algorithm behind the proposed energy resource scheduling has three stages. The first stage is to schedule each microgrid individually to satisfy its internal demand. The next stage involves finding the best possible bids for exporting power to the network and compete in a whole sale energy market. The final stage is to reschedule each microgrid individually to satisfy the total demand, which is the addition of internal demand and the demand from the results of the whole sale energy market simulation. The simulation results of a power system with distributed resources comprising three microgrids and five lumped loads show that the proposed multi-agent system allows efficient management of micro-sources with minimum operational cost. The case studies demonstrate that the system is successfully monitored, controlled and operated by means of the developed multi-agent system.

The Design and Dev elopment of a Multi-agent Agile Scheduling System

The Design and Dev elopment of a Multi-agent Agile Scheduling System

MAS_UP-UCT: A Multi-Agent System for University Course Timetable Scheduling

Many real-world applications are mapped into combinatorial problems. An example of such problem is timetable scheduling. In this case, the two basic characteristics can be defined by its distributed and dynamic environment. One efficient solution to solve this problem could be provided by an agent-based approach. A timetable scheduling problem can be modelled as a multi-agent system that provides the final schedule by taken into account all the restrictions. In this paper it is presented a preliminary research work that involves the development of a multi-agent system for university course timetable scheduling, named MAS_UP-UCT. We focus on the architecture of the multi-agent system, and on the evaluation of the communication process by using the interaction diagrams.

Bio-inspired scheduling for dynamic job shops with flexible routing and sequence-dependent setups

Flexible routing requires scheduling to be responsive and robust. Multi-agent systems have the potential to achieve robustness and provide a means for real-time planning and scheduling. The objective of this paper is to propose a multi-agent scheduling system with a good solution quality and robustness. The proposed multi-agent approach is designed for dynamic job shops with routing flexibility and sequence-dependent setup. A bio-inspired strategy based on division of labour in insect societies is presented for coordination among agents. The strategy is accomplished using a computational model which is composed of response threshold, response intention, and machine-centred reinforcement learning. The bio-inspired scheduling is compared with an agent-based approach and a dispatching rule-based approach. The experiments were performed using simulation and statistical analysis. Results show that the proposed bio-inspired scheduling model performs better than the other two methods on all eight common scheduling metrics.

The theory and experiments of designing cooperative intelligent systems

In this paper, we identify the business problems that lend themselves to the design of cooperative intelligent systems and empirically demonstrate the design and application of a multi-agent intelligent system for production scheduling. Our experiments suggest that a multi-agent system where agents coordinate their actions generally performs better than a multi-agent system where agents do not coordinate their actions.

Performance analysis of a multi-agent scheduling and control system under manufacturing disturbances

This paper is concerned with evaluating the performance of alternative multi-agent approaches to manufacturing planning and control. In order to separate the influence of ‘control algorithm’ from that of ‘control architecture’, a simple local control algorithm is chosen as a common starting point for developing the agent-based system. Two sets of experiments are then reported to evaluate how changes in individual agent characteristics can affect the control system's flexibility and adaptability against disturbances. These experiments show that the manufacturing and control system performance is not affected by architecture if the control algorithm remains fixed, however, altering the characteristics of the control system decision-makers influences both systems’ performance.

Interoperating standards in multiagent agile manufacturing scheduling systems

This paper presents an implemented framework that deals with the high-level of heterogeneity that a multi-agent scheduling system should tackle when the legacy systems and component-based approaches are considered. Perspectives from different natures of information sources to the scheduling system are taken into account, namely the information that comes from the supply chain and from the internal/heterogeneous planning and shop floor supervision systems. The utilisation of standards for inter-tools communication, information exchange and information modelling ascends in significance within this context. They are seen as a base for an agile and lean reaction of the enterprise both in its normal operation and in the presence of unexpected events. The framework and the system prototype developed called HOLOS comprise what can be called an interoperation among standards, and they are discussed and illustrated with some examples.

Communication infrastructure in distributed scheduling

The emergence of distributed artificial intelligent (DAI) introduced a new approach to solve scheduling problems by a set of scheduling systems that interact with each other in the problem-solving process. In this paper, we describe a communication infrastructure to handle connection and communication between distributed Internet scheduling systems for distributed applications. First, we present an agent model of distributed scheduling systems where agents can communicate and coordinate activities with each other via an agent communication language. Then, we define the syntax and semantics for the agent communication languages, and negotiation mechanism. Following that, we discuss the design and development of the prototype for the multi-agent scheduling systems. We conclude with a discussion of communication issues for heterogeneous agent-based scheduling systems to solve distributed scheduling problems.

Using a multi-agent approach to optimise the train coupling and sharing system

This paper discusses a multi-agent scheduling system using the example of the railway transport system train coupling and sharing (TCS). The scheduling system includes the planning of travel units and the optimisation of solutions. The planning process is implemented as an incremental anytime algorithm that is capable of integrating new task specifications into the ongoing planning process. The initial solution is then optimised by a simulated trading approach. Furthermore, the multi-agent approach is tested on a practical example. The operational parameters and boundaries of the TCS-system are changed to get a comparable rail operation schedule.

Generic Framework for Conflict Resolution in Negotiation-Based Agile Scheduling Systems

This paper describes a general framework for conflict resolution being implemented on the HOLOS multi-agent scheduling system. The framework is integrated with the planning and execution supervision activities, emphasizing the importance of local supervision actions. Conflicts are classified in levels and types, and the specific actions for their resolution, carried out dming scheduling genemtion, after its generation and during its execution, are also presented. A significant part ofthe conflict resolution process is supported by using the user guide negotiation approach. Finally, the decentralized scheduling supervision process is shown.