Social learning in self-organizing systems for complex assembly tasks

Abstract

Self-organizing systems have the potential to adapt to situation changes and gracefully deal with system degradations. Designing such systems, however, requires devising self-organizing knowledge into the individual agents of the system through either explicitly modelling the knowledge or allowing the agents to learn it by themselves through reinforcement learning. For tasks that are complex to a certain extent, the research has shown that developing proper reward functions with the help of reward shaping is an effective approach for successful multiagent reinforcement learning and hence for self-organizing systems design. In this case, agents often are individuals who learn by themselves through directly interacting with the task environment with little sense of the existence of other agents. As the task becomes more complex, explicitly considering the interactions among the agents during their learning process has evolved as a rich resource to explore for further eliciting the power of self-organizing systems. In this paper, social learning in its limited form—i.e., observing the actions of other agents—has been introduced to the multiagent reinforcement learning process, and a multiagent social deep Q-learning (MASo-DQL) has been developed and tested in the context of an assembly task complicated with obstacles. The experiment results have demonstrated that social learning can be effectively integrated into multiagent reinforcement learning frameworks through MASo-DQL; it has a positive impact on the training process and the task performance of the trained teams when the task complexity becomes high; and it entails costs and may lead to the inferior system performance for low complexity tasks.