COMPUTATION AND PSYCHOPHYSICS OF MOTION AND STEREOPSIS

A complete understanding of visual information processing requires not only experimental investigations of the visual system but also computational theories which specify how the signals carried by a large number of neurons in the brain can be combined to accomplish a given perceptual task. Most existing theories of vision, however, either ignore neural constraints or consider them as secondary. Our long-term goal is therefore to construct computational models of vision based solely on physiological mechanisms. We believe that this goal can only be achieved by directly analyzing response properties of real visual cells instead of treating them as important implementational details. As a step towards this direction, we propose to investigate problems centered around motion analysis and stereo vision in this application. In addition to our strong emphasis on physiological reality, our work will also represent one of the first computational studies that integrates motion and stereopsis into a common framework. We have recently developed a physiologically realistic model for stereo vision based on known receptive field profiles of real binocular cells in the brain. We will use our model to explain the characteristic disparities of cortical cells reported by Wagner and Frost and the psychophysically observed disparity attraction and repulsion. This work will help clarify the mechanisms of disparity sensitivity in the brain. We will also determine whether the model can be extended to solve the difficult problem of stereo transparency. We will then combine our stereo model with physiologically plausible models of motion into a unified framework using the spatiotemporal receptive field properties of real cells found in the primary visual cortex. The resulting integrated model will be used to explain interesting psychophysical observations of motion-stereo interaction such as the perceived depth in Pulfrich's pendulum, the generalized Pulfrich effect to dynamic noise patterns and to stroboscopic stimuli, and the dichoptic motion phenomenon reported by Shadlen and Carney. We believe that this work will provide one of the most comprehensive, quantitative and physiologically plausible explanations of the Pulfrich phenomenon and its major variations. Interactions among our model cells will be introduced in the next level of processing according o the physiological properties of MT cells. The resulting model will be used to explain several phenomena of motion transparency from disparity cues. We will also develop a physiologically plausible model of structure-from- motion and test the hypothesis that the problem can be solved with a distributed velocity coding scheme. Finally, we will test several new predictions from our models at the behavioral level with psychophysical experiments on human subjects.