Understanding curiosity: a combined behavioral, biological and computational theory

Abstract

Curiosity, defined as the intrinsic desire to know, is among the last unexplored frontiers of higher cognition, and we know very little about its neural mechanisms. We propose to address this question by developing a program for studying curiosity using an integrated empirical/computational approach in humans and non-human primates. We examine the hypothesis that curiosity is a family of mechanisms that evolved to allow animals to maximize their knowledge of the useful properties of the world – i.e., the regularities that exist in the world - using active, targeted investigations. In two experiments, we probe two processes that contribute to curiosity-based exploration. In experiment 1 we examine how agents ascribe value (“interest”) to surprising events, by (1) developing a new behavioral task where children and monkeys make tradeoffs between exploring for sources of reward versus exploring a surprising item, and (2) investigating single-neuron responses related to curiosity-based exploration in cortical areas implicated in the control of attention. In experiment 2 we ask whether subjects show a more sophisticated form of curiosity guided by learning progress – a meta-cognitive tracking of the amount of learning that the individual can make in a task - using new behavioral paradigms where children or non-human primates see a set of symbolic learning problems and freely choose which ones to explore. To quantitatively model curiosity we use the framework of Bayesian Reinforcement Learning, which allows us to infer the agent’s beliefs about the probabilities of various events (rewards, surprises, or learning), the value they place on sampling different events, and how this value depends on the task context. The studies closely integrate the expertise of the 3 member teams in developmental psychology (Kidd), neurophysiology of non-human primates (Gottlieb) and computational modeling of active learning in robotic systems (Oudeyer). Our goal is to develop an integrated theory that (1) incorporates curiosity in established quantitative frameworks of learning and decision making, (2) links it with core cognitive functions such as selective attention, (3) compares its expression in humans and non-human primates, and (4) begins to elucidate its neural mechanisms.