Adversary Aware Continual Learning

TY - JOUR

T1 - Adversary Aware Continual Learning

AU - Umer, Muhammad

AU - Polikar, Robi

N1 - Publisher Copyright: © 2013 IEEE.

PY - 2024

Y1 - 2024

N2 - Continual learning approaches are useful to help a model learn new information or new tasks sequentially, while also retaining the previously acquired information. However, such approaches are known to be extremely vulnerable to the adversarial backdoor attacks, where an intelligent adversary can introduce small amount of misinformation in the form of imperceptible backdoor pattern during training to cause deliberate forgetting of a specific task or class at test time. In this work, we propose a novel defensive framework to counter such an attack and use the attacker's primary strength - hiding the backdoor pattern by making it imperceptible to humans - against itself. To do so we train the model to learn a random perceptible pattern as decoy data during training with the deliberate intent to overpower the attacker's imperceptible pattern. We demonstrate the effectiveness of the proposed defensive mechanism through various commonly used replay-based (both generative and exact replay-based) continual learning algorithms using continual learning benchmark variants of CIFAR-10, CIFAR-100, and MNIST datasets. We show that the proposed defensive framework considerably improves the robustness of continual learning algorithms without having any knowledge of the attacker's target task, attacker's target class, shape, size, and location of the attacker's backdoor pattern. Moreover, our defensive framework does not depend on the underlying continual learning algorithm and does not rely on detecting the attack samples (to subsequently remove them from further consideration), but instead attempts to correctly classify even the attack samples and thus ensuring robustness in continual learning models. We refer to the proposed defensive framework as adversary aware continual learning (AACL).

AB - Continual learning approaches are useful to help a model learn new information or new tasks sequentially, while also retaining the previously acquired information. However, such approaches are known to be extremely vulnerable to the adversarial backdoor attacks, where an intelligent adversary can introduce small amount of misinformation in the form of imperceptible backdoor pattern during training to cause deliberate forgetting of a specific task or class at test time. In this work, we propose a novel defensive framework to counter such an attack and use the attacker's primary strength - hiding the backdoor pattern by making it imperceptible to humans - against itself. To do so we train the model to learn a random perceptible pattern as decoy data during training with the deliberate intent to overpower the attacker's imperceptible pattern. We demonstrate the effectiveness of the proposed defensive mechanism through various commonly used replay-based (both generative and exact replay-based) continual learning algorithms using continual learning benchmark variants of CIFAR-10, CIFAR-100, and MNIST datasets. We show that the proposed defensive framework considerably improves the robustness of continual learning algorithms without having any knowledge of the attacker's target task, attacker's target class, shape, size, and location of the attacker's backdoor pattern. Moreover, our defensive framework does not depend on the underlying continual learning algorithm and does not rely on detecting the attack samples (to subsequently remove them from further consideration), but instead attempts to correctly classify even the attack samples and thus ensuring robustness in continual learning models. We refer to the proposed defensive framework as adversary aware continual learning (AACL).

UR - https://www.scopus.com/pages/publications/85203404576

UR - https://www.scopus.com/pages/publications/85203404576#tab=citedBy

U2 - 10.1109/ACCESS.2024.3455090

DO - 10.1109/ACCESS.2024.3455090

M3 - Article

SN - 2169-3536

VL - 12

SP - 126108

EP - 126121

JO - IEEE Access

JF - IEEE Access

ER -