Functional and computational characterization of the human auditory cortex

According to NIDCD, 6 to 8 million people in the United States have some form of speech or communication disorder. Speech perception requires a listener to map variable acoustic signals onto a finite set of phonological categories known as phonemes, and to integrate those categories over time to form larger linguistic units such as syllables and words. It remains speculative where these different speech features are encoded and what cortical computations are needed for their calculation from an acoustic signal. A better understanding of what neural circuits are involved, how they are organized, and what computations they perform to support speech comprehension is critical for developing a detailed neurobiological model of speech perception. The major aim of this proposal is to use a joint framework to study the encoding of acoustic and linguistic features and the computational underpinnings of natural speech processing, using invasive surface and depth electrodes implanted in human neurosurgical patients. To study the cortical organization of acoustic features, we will characterize the encoding and anatomical organization of acoustic features in auditory cortical regions. To study the cortical organization of linguistic features, we will measure the encoding of phonetic, phonotactic, and semantic information using multivariate linear regression. To understand the underlying computational mechanisms, we will train convolutional neural network models to predict the neural responses to speech and use a novel method to express their computation as a set of linear transforms. By interpreting these models, we will uncover nonlinear computations used in different auditory areas and relate them to the encoding of acoustic and linguistic features. These complementary analyses will extend our knowledge of speech processing in the human auditory cortex and lead to new hypotheses about the mechanisms of various speech and language disorders. Together, the proposed research will greatly improve the current models of cortical speech processing, which are of great interest in many disciplines including neurolinguistics, speech pathology, speech prostheses, and speech technologies.