Reinforcement learning from post-ingestive rewards

The ability to obtain food from the environment is fundamental for animal survival. To achieve this, animals must choose which food sources are appropriate for their homeostatic needs and select the actions that lead to them. To solve this problem, animals rely not only on the sensory information generally available in the world but also on specific information obtained from food. This information is composed of pre-ingestive sensory signals like food flavor, but also of post-ingestive signals like nutrient detection in the intestine and metabolic changes resulting from nutrient absorption. Flavor-nutrient conditioning experiments have demonstrated that animals develop a preference for flavors that have been previously paired with the infusion of high-calorie solutions directly in the gut. Hence, food flavor can provide a fast estimate of the potential nutritional impact of food through learning. At the same time, this highlights the importance of post-ingestive signaling in the shaping of food preference. In our previous work, we found that post-ingestive stimulation by high-calorie solutions is sufficient to elicit a response of dopamine neurons that project to the ventral striatum, and are known to be involved in reward-based learning. Moreover, we showed that post-ingestive delivery of sucrose is sufficient to support, and robustly modulate, previously established food-seeking behaviors, and that NMDA-dependent bursting/plasticity of ventral tegmental area VTA dopamine neurons is necessary for this modulation. Importantly, when animals underwent a hepatic branch vagotomy, the modulation of food-seeking behavior and the response of VTA DANs to post-ingestive sucrose was largely impaired. This argues for a role of gut-brain communication via the vagus nerve in the regulation of food-seeking behavior. Nonetheless, we found it very challenging to have mice learn a new action-based exclusively on post-ingestive reinforcement, and the introduction of a pre-ingestive stimulus water was needed early in training. However, later in training, there was no need to maintain the pre-ingestive stimulus and the manipulations of the post-ingestive stimulus had a clear effect on food-seeking behavior. Hence, in spite of the ability of both pre and post-ingestive stimuli to elicit responses from midbrain DANs, they seem to have different effects in learning and modulation of food-seeking behavior. How specifically the pre and post-ingestive properties of food contribute to shape food-seeking behavior remains unclear. In this project, we propose to understand the individual contributions of pre and post-ingestive stimulus for reinforcement learning and its neural underpinnings. To achieve this we will develop a novel probabilistic instrumental task that overcomes limitations of previous tasks e.g. flavor-nutrient conditioning. In this task, mice can choose between two possible actions that, with a certain probability, will give them access to rewards. However, while one action leads to intragastric infusion of a high-calorie solution, the other leads to the intragastric administration of a non-caloric solution. Furthermore, we will develop a computational model based on reinforcement learning RL theory, using both pre and post-ingestive components to predict animal’s choices. RL models provide an opportunity to correlate latent behavioral variables, with neural activity, providing insight into how brain circuits implement the computations that drive animal behavior. To take advantage of this we will record the activity of vagal afferents and DANs while mice are performing the 2-action task. Finally, in light of our model, we will explore how the manipulation of afferent vagal fibers, with origin in the gut, shapes food-seeking behavior. Understanding what is the specific contribution of pre and post-ingestive stimulus to feeding decisions has become increasingly important in the current ‘obesogenic’ food environment, where post-ingestive signaling of the high sugar and fat content of processed foods may tap into reward circuits with deleterious changes in food-seeking behavior.