Revealing the mechanisms of primate face recognition with synthetic stimulus sets optimized to compare computational models

Project Summary Neuroscience is entering a new era, where large-scale neural network models can be tested with unprecedent- edly rich measurements of neural activity. This proposal develops a general methodology for linking theory to experiment in this new era and applies the methodology to the problem of primate face recognition. Face recogni- tion is an important problem at the intersection of neuroscience and AI, and provides an ideal domain in which to tackle the more general problem of object recognition: the problem of face recognition is confined to a particular stimulus class (faces) and constrained by a known network of face areas in the brain. The project is a collabo- ration between two laboratories with complementary strengths in computational modeling and neurophysiology in fMRI-identified face areas, whose shared focus and past work provide a strong foundation to build on. To link theory to experiment, we will implement computational theories in neural network models and use optimization techniques to create sets of synthetic face stimuli that elicit strongly divergent predictions from the models. We refer to such stimuli as controversial stimuli since they are optimized to make models disagree. Controversial stimuli provide out-of-distribution probes of the models and increase our power to distinguish between alternative computational hypotheses. We will test feedforward and recurrent computational mechanisms of face recognition by implementing them in neural network models simultaneously constrained by biology (anatomical connectivity and neurophysiology) and cognitive function (recognition objective and computational constraints). Aim 1 will implement computational theories of face recognition in feedforward and recurrent neural network models, so as to render the theories testable in terms of both their ability to account for successful recognition and their ability to explain neural population codes in primate face patches. Aim 2 will compare the models by recording neural responses in face patches elicited by synthetic face stimuli that are optimized for the models to make contrasting predictions. Aim 3 is to reveal the remaining limitations of the best models for each face patch in recording experiments where the stimuli are adapted in a closed loop, so as to maximize the empirical prediction error of the models. The expected outcomes of this work include the identification of the computational mecha- nisms of primate face recognition, the development of novel computational architectures, and the development of the method of controversial stimuli as a general experimental methodology for neural recordings that enables powerful direct tests of computational theories implemented in neural network models. The computational and methodological insights are expected to contribute to the development of new diagnostic and treatment meth- ods for face blindness (prosopagnosia) and other perceptual disorders and could lead to new approaches for decision-making in neurology and psychiatry.