Robust Training in Multiagent Deep Reinforcement Learning Against Optimal Adversary

TY - JOUR

T1 - Robust Training in Multiagent Deep Reinforcement Learning Against Optimal Adversary

AU - Guo, Weiran

AU - Liu, Guanjun

AU - Zhou, Ziyuan

AU - Wang, Jiacun

AU - Tang, Ying

AU - Wang, Miaomiao

N1 - Publisher Copyright: © 2013 IEEE.

PY - 2025

Y1 - 2025

N2 - Industry 5.0 enhances manufacturing ability through efficient human-machine interaction, combining human resources and robots to complete tasks more accurately and effectively. Artificial intelligence (AI) plays an essential role in Industry 5.0. As a branch in AI, multiagent deep reinforcement learning (MADRL) attracts vast attention in both academia and industry. However, there is a gap between virtual and physical environments in terms of how clean an observed state is. In addition, state adversarial attacks can seriously impact the performance of MADRL. Hence, how to improve the robustness of MADRL algorithms is an important research topic. In this article, we propose an optimal policy-based state adversary attack method that would make the MADRL algorithm more robust when it is applied in the training process of agents. Two case studies related to Industry 5.0 and a general case study are presented in which robustness training against the optimal adversarial attack is tested. The MADRL algorithms involved in the experiments include centralized training and decentralized execution (CTDE) framework and shared experience actor-critic (SEAC) to demonstrate the universality of our method.

AB - Industry 5.0 enhances manufacturing ability through efficient human-machine interaction, combining human resources and robots to complete tasks more accurately and effectively. Artificial intelligence (AI) plays an essential role in Industry 5.0. As a branch in AI, multiagent deep reinforcement learning (MADRL) attracts vast attention in both academia and industry. However, there is a gap between virtual and physical environments in terms of how clean an observed state is. In addition, state adversarial attacks can seriously impact the performance of MADRL. Hence, how to improve the robustness of MADRL algorithms is an important research topic. In this article, we propose an optimal policy-based state adversary attack method that would make the MADRL algorithm more robust when it is applied in the training process of agents. Two case studies related to Industry 5.0 and a general case study are presented in which robustness training against the optimal adversarial attack is tested. The MADRL algorithms involved in the experiments include centralized training and decentralized execution (CTDE) framework and shared experience actor-critic (SEAC) to demonstrate the universality of our method.

UR - https://www.scopus.com/pages/publications/105003648370

UR - https://www.scopus.com/pages/publications/105003648370#tab=citedBy

U2 - 10.1109/TSMC.2025.3561276

DO - 10.1109/TSMC.2025.3561276

M3 - Article

SN - 2168-2216

VL - 55

SP - 4957

EP - 4968

JO - IEEE Transactions on Systems, Man, and Cybernetics: Systems

JF - IEEE Transactions on Systems, Man, and Cybernetics: Systems

IS - 7

ER -