Real-time processing of depth and color video streams to improve the reliability of depth maps

Depth is a useful information in vision to understand the geometrical
properties of an environment. Depth is traditionally computed in
terms of a disparity map acquired by a stereoscopic system but, over
the last few years, several manufacturers have released single-lens
cameras that directly capture depth information (also called range).
This is an important technological breakthrough although range signals
remain difficult to handle in practice, due to many reasons (low
resolution, noise, low framerate, . . . ). Practitioners still struggle to
use range data in their applications. The purpose of this paper is
to give a brief introduction to range data (captured with a camera),
discuss common limitations, and propose techniques to cope with
difficulties typically encountered with range cameras. These techniques
are based on a simultaneous view of the scene by a color and
a depth camera that are combined to improve their interpretation in
real time.