Defending Observation Attacks in Deep Reinforcement Learning via Detection and Denoising

TY - GEN

T1 - Defending Observation Attacks in Deep Reinforcement Learning via Detection and Denoising

AU - Xiong, Zikang

AU - Eappen, Joe

AU - Zhu, He

AU - Jagannathan, Suresh

N1 - Publisher Copyright: © 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.

PY - 2023

Y1 - 2023

N2 - Neural network policies trained using Deep Reinforcement Learning (DRL) are well-known to be susceptible to adversarial attacks. In this paper, we consider attacks manifesting as perturbations in the observation space managed by the external environment. These attacks have been shown to downgrade policy performance significantly. We focus our attention on well-trained deterministic and stochastic neural network policies in the context of continuous control benchmarks subject to four well-studied observation space adversarial attacks. To defend against these attacks, we propose a novel defense strategy using a detect-and-denoise schema. Unlike previous adversarial training approaches that sample data in adversarial scenarios, our solution does not require sampling data in an environment under attack, thereby greatly reducing risk during training. Detailed experimental results show that our technique is comparable with state-of-the-art adversarial training approaches.

AB - Neural network policies trained using Deep Reinforcement Learning (DRL) are well-known to be susceptible to adversarial attacks. In this paper, we consider attacks manifesting as perturbations in the observation space managed by the external environment. These attacks have been shown to downgrade policy performance significantly. We focus our attention on well-trained deterministic and stochastic neural network policies in the context of continuous control benchmarks subject to four well-studied observation space adversarial attacks. To defend against these attacks, we propose a novel defense strategy using a detect-and-denoise schema. Unlike previous adversarial training approaches that sample data in adversarial scenarios, our solution does not require sampling data in an environment under attack, thereby greatly reducing risk during training. Detailed experimental results show that our technique is comparable with state-of-the-art adversarial training approaches.

UR - http://www.scopus.com/inward/record.url?scp=85151060277&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85151060277&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-26409-2_15

DO - 10.1007/978-3-031-26409-2_15

M3 - Conference contribution

SN - 9783031264085

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 235

EP - 250

BT - Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2022, Proceedings

A2 - Amini, Massih-Reza

A2 - Canu, Stéphane

A2 - Fischer, Asja

A2 - Guns, Tias

A2 - Kralj Novak, Petra

A2 - Tsoumakas, Grigorios

PB - Springer Science and Business Media Deutschland GmbH

T2 - 22nd Joint European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2022

Y2 - 19 September 2022 through 23 September 2022

ER -