Minimalist Sensing for Computer Vision

Today, computer vision technologies are being used in many aspects of our everyday lives. The data used by any computer vision system are images produced by cameras designed for photography and television. These images are rich in terms of visual information and in most case include much more information than needed to solve the task of interest. This has several detrimental effects. First, due to the large number of pixels (millions) in a typical image, vision applications tend to be resource intensive, requiring significant hardware, bandwidth, computations and power. Second, since every captured image has more information within it than needed to solve the problem, it exposes vision systems to a wide range of privacy issues. Society is decidedly leery of cameras in public environments, fearing that their personal information (identity, location) may be compromised and misused. For all these reasons, webelieve it is time to explore a new paradigm for designing vision systems. Our goal is to lay the foundation for the design and fabrication of a minimalist camera, one that captures only the information needed to solve the application at hand. The question we seek to answer in this project is the following: What are the minimal resources needed to solve a given vision task. Examples of tasks can vary from simple ones such as intrusion detection, velocity estimation, and depth estimation to more complex ones such as crowd flow estimation, activity detection, object inspection, etc. We believe that each of these applications can be solved with a remarkably small number of measurements (in contrast to themillions made by a traditional camera), without compromising robustness and accuracy. Our key idea would redefine the most elementary construct used to make a visual measurement the pixel. Rather than measuring a conventional image using a lens and an image sensor with millions of photodetectors, we use a small number of photodetectors that each view the world through an attenuation mask. Each of these measurement devices, called a mixel (masked pixel), essentially measures a linear projection of the scene. Our research is geared towards answering two important questions: (a) What is the minimum number of mixels needed to solve a task? (b) What are the mask functions (projections) that should be used for a given task? The great advantage of this approach is that the vision system only needs to measure and process a very small number of projections. This greatly mitigates the privacy issues mentioned above, while significantly reducing the resources (including power) needed to perform the task. Our past experience informs us that such a low-powered vision system can be designed to be entirely selfpowered requiring no external energy or cabling.