HUMAN INDUCTIVE BIASES FOR AVERSIVE CONTINUAL LEARNING — A HIERARCHICAL BAYESIAN NONPARAMETRIC MODEL

TY - JOUR

T1 - HUMAN INDUCTIVE BIASES FOR AVERSIVE CONTINUAL LEARNING — A HIERARCHICAL BAYESIAN NONPARAMETRIC MODEL

AU - Pisupati, Sashank

AU - Berwian, Isabel M.

AU - Chiu, Jamie

AU - Ren, Yongjing

AU - Niv, Yael

N1 - Publisher Copyright: © 2023 Proceedings of Machine Learning Research. All rights reserved.

PY - 2023

Y1 - 2023

N2 - Humans and animals often display remarkable continual learning abilities, adapting quickly to changing environments while retaining, reusing, and accumulating old knowledge over a lifetime. Unfortunately, in environments with adverse outcomes, the inductive biases supporting such forms of learning can turn maladaptive, yielding persistent negative beliefs that are hard to extinguish, such as those prevalent in anxiety disorders. Here, we present and model human behavioral data from a fear-conditioning task with changing latent contexts, in which participants had to predict whether visual stimuli would be followed by an aversive scream. We show that participants’ learning in our task spans three different regimes — with old knowledge either being updated, discarded (forgotten) or retained and reused in new contexts (remembered) by different participants. The latter regime corresponds to (maladaptive) spontaneous recovery of fear. We demonstrate using simulations that these behavioral regimes can be captured by varying inductive biases in Bayesian non-parametric models of contextual learning. In particular, we show that the “remembering” regime can be produced by “persistent” variants of hierarchical Dirichlet process priors over contexts and negatively biased “deterministic” beta distribution priors over outcomes. Such inductive biases correspond well to widely observed “core beliefs” that may have adaptive value in some lifelong-learning environments, at the cost of being maladaptive in other environments and tasks such as ours. Our work offers a tractable window into human inductive biases for continual learning algorithms, and could potentially help identify individual differences in learning strategies relevant for response to psychotherapy.

AB - Humans and animals often display remarkable continual learning abilities, adapting quickly to changing environments while retaining, reusing, and accumulating old knowledge over a lifetime. Unfortunately, in environments with adverse outcomes, the inductive biases supporting such forms of learning can turn maladaptive, yielding persistent negative beliefs that are hard to extinguish, such as those prevalent in anxiety disorders. Here, we present and model human behavioral data from a fear-conditioning task with changing latent contexts, in which participants had to predict whether visual stimuli would be followed by an aversive scream. We show that participants’ learning in our task spans three different regimes — with old knowledge either being updated, discarded (forgotten) or retained and reused in new contexts (remembered) by different participants. The latter regime corresponds to (maladaptive) spontaneous recovery of fear. We demonstrate using simulations that these behavioral regimes can be captured by varying inductive biases in Bayesian non-parametric models of contextual learning. In particular, we show that the “remembering” regime can be produced by “persistent” variants of hierarchical Dirichlet process priors over contexts and negatively biased “deterministic” beta distribution priors over outcomes. Such inductive biases correspond well to widely observed “core beliefs” that may have adaptive value in some lifelong-learning environments, at the cost of being maladaptive in other environments and tasks such as ours. Our work offers a tractable window into human inductive biases for continual learning algorithms, and could potentially help identify individual differences in learning strategies relevant for response to psychotherapy.

UR - http://www.scopus.com/inward/record.url?scp=85184104113&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85184104113&partnerID=8YFLogxK

M3 - Conference article

SN - 2640-3498

VL - 232

SP - 337

EP - 346

JO - Proceedings of Machine Learning Research

JF - Proceedings of Machine Learning Research

T2 - 2nd Conference on Lifelong Learning Agents, CoLLA 2023

Y2 - 22 August 2023 through 25 August 2023

ER -