Vision Through Rain and Snow

Bad weather can severely limit the usefulness of computer vision in the outdoors. The algorithms (segmentation, tracking, recognition, etc.) used in today's vision systems are not equipped to handle the adverse effects of weather. For outdoor vision systems to perform well in all conditions, they must explicitly account for the diverse effects of weather. The visual effects produced by different weather conditions can be broadly classified into two categories: steady (fog, mist and haze) and dynamic (rain, hail and snow). This research project is devoted to the modeling and analysis of dynamic weather effects. The first stage of the study is to develop compact and yet descriptive photometric and geometric models of raindrops and snowflakes. These models will be used to develop advanced models for the dynamical and statistical properties of rain and snow. Using these models, efficient and robust algorithms will be developed for detecting and removing rain and snow from videos. The proposed models will also be used to develop algorithms for fast and realistic rendering of rain and snow. In addition, the developed models will be used to design and implement real-time rain and snow gauges based on video cameras. Such gauges will be very inexpensive compared to ones currently used in meteorological sciences, and yet will provide high accuracy in the estimation of weather parameters. The results of the proposed research are anticipated to have a broad and tangible impact on both research and education. From the perspective of research, the results are expected to impact the robustness of outdoor vision systems and the realism that computer graphics can achieve. The models, algorithms and databases developed during the project will be broadly disseminated in a timely fashion. With regards to education, the PI is currently developing a course titled ``Computer Vision through the Atmosphere,'' and the results of this investigation will contribute significantly to the content of the course.