SCH: Human-Centered Reinforcement Learning for Personalized Nutritional Coaching

Chronic diseases, such as type 2 diabetes, hypertension, and obesity, place an ever-increasing burden on individuals and society at large. Health coaching has emerged as an effective approach to promoting self management. However, there are not enough coaching professionals to accommodate the growing population of individuals with chronic diseases. Conversational agents have the potential to overcome these barriers and make health coaching available to a more diverse population. One promising data-driven approach employs reinforcement learning (RL), a machine-learning approach that learns from past interactions and prescribes sequences of actions for reaching a predetermined goal. However, RL-based dialogs can be perceived as unintuitive to users, and there is a need for new approaches to aligning RL-based conversational agents with human reasoning and expectations. In addition, RL algorithms are opaque and there is a need for new approaches to generating explanations for RL inferences and actions.To address these gaps, this project develops a new approach to providing health coaching with RL-based conversational agents, while at the same time addressing more general challenges of designing human-centered RL-based conversational agents. To achieve these goals, this project includes a user study of health coaching in the context of type 2 diabetes, in which human health coaches will be asked to provide guidance to individuals with type 2 diabetes via text messages. The corpus of dialogs collected during this study provides a foundation for developing data driven computational representation of textual meal descriptions and for the development of a chatbot that uses RL to produce conversational structures appropriate for nutritional coaching. Furthermore, this project uses learned representations of meals to provide individuals with feedback on their nutritional choices and explanations for this feedback. Finally, it integrates the human perspective into the RL policy to generate dialog structures that are perceived as intuitive by humans. The evaluation study examines the impact of the RL-based health coach on individuals’ ability to achieve their nutritional goals as compared to other, non-RL-based coaching techniques. This research is consequential to society at large in several ways. First, conversational interfaces can lower entry barriers for engaging with technological interventions in health and wellness for diverse communities and reduce “intervention-generated inequalities” in health. Furthermore, new techniques for aligning RL with human reasoning and explaining its inferences and choices to users can increase its applicability to a broader set of problems and domains. On a broader level, this research and educational plan take important steps towards further promoting human-centered approaches to data science, machine learning, and artificial intelligence education that can have broader impact on future research in this field.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.