ViBe: A universal background subtraction algorithm for video sequences

2011 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

All documents in ORBi are protected by a user license.

Send to

RIS BibTex APA Chicago Permalink X Linkedin

Details

Keywords :

Background; Background subtraction; Foreground; Video processing; Learning; Segmentation; Computer vision; Embedded systems; Real-time processing; Motion; Motion detection; Motion analysis; Pixel classification; Mixture of Gaussians; Benchmarking; Exponential filtering; Random sampling; Initialization; Background model; Fast algorithm; Algorithm; Comparision; Code; Pseudo-code; C code; C/C++; Smart camera; Kernel density estimation; Ghost; OpenCV; ViBe

Abstract :

[en] This paper presents a technique for motion detection that incorporates several innovative mechanisms. For example, our proposed technique stores, for each pixel, a set of values taken in the past at the same location or in the neighborhood. It then compares this set to the current pixel value in order to determine whether that pixel belongs to the background, and adapts the model by choosing randomly which values to substitute from the background model. This approach differs from those based on the classical belief that the oldest values should be replaced first. Finally, when the pixel is found to be part of the background, its value is propagated into the background model of a neighboring pixel. We describe our method in full details (including pseudocode and the parameter values used) and compare it to other background subtraction techniques. Efficiency figures show that our method outperforms recent and proven state-of-the-art methods in terms of both computation speed and detection rate. We also analyze the performance of a downscaled version of our algorithm to the absolute minimum of one comparison and one byte of memory per pixel. It appears that even such a simplified version of our algorithm performs better than mainstream techniques. There is a dedicated web page for ViBe at http://www.telecom.ulg.ac.be/research/vibe/

Research Center/Unit :

Intelsig
Telim

Disciplines :

Electrical & electronics engineering

Author, co-author :

Barnich, Olivier

Van Droogenbroeck, Marc ; Université de Liège - ULiège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Télécommunications

Language :

English

Title :

ViBe: A universal background subtraction algorithm for video sequences

Publication date :

June 2011

Journal title :

IEEE Transactions on Image Processing

ISSN :

1057-7149

eISSN :

1941-0042

Publisher :

IEEE

Volume :

Issue :

Pages :

1709-1724

Peer reviewed :

Peer Reviewed verified by ORBi

Additional URL :

http://ieeexplore.ieee.org/search/freesrchabstract.jsp?tp=&arnumber=5672785

Funders :

DGTRE - Région wallonne. Direction générale des Technologies, de la Recherche et de l'Énergie

Commentary :

Source code in C/C++ available. This code is copyrighted; please do not copy it to another site.

Available on ORBi :

since 07 January 2011

Statistics

Number of views

38704 (211 by ULiège)

Number of downloads

67348 (133 by ULiège)

More statistics

Scopus citations^®

1833

Scopus citations^®
without self-citations

1815

OpenCitations

1275

OpenAlex citations

1774

Bibliography

O. Barnich and M. Van Droogenbroeck, "ViBe: A powerful random technique to estimate the background in video sequences," in Proc. Int. Conf. Acoust., Speech Signal Process., Apr. 2009, pp. 945-948.
M. Van Droogenbroeck and O. Barnich, Visual Background Extractor p. 36, Jan. 2009, World Intellectual Property Organization, WO 2009/ 007198.
A. McIvor, "Background subtraction techniques," in Proc. Image Vis. Comput., Auckland, New Zealand, Nov. 2000.
R. Radke, S. Andra, O. Al-Kofahi, and B. Roysam, "Image change detection algorithms: A systematic survey," IEEE Trans. Image Process., vol. 14, pp. 294-307, Mar. 2005. (Pubitemid 40351907)
Y. Benezeth, P. Jodoin, B. Emile, H. Laurent, and C. Rosenberger, "Review and evaluation of commonly-implemented background subtraction algorithms," in Proc. IEEE Int. Conf. Pattern Recognit., Dec. 2008, pp. 1-4.
S. Elhabian, K. El-Sayed, and S. Ahmed, "Moving object detection in spatial domain using background removal techniques-State-of-art," Recent Pat. Comput. Sci., vol. 1, pp. 32-54, Jan. 2008.
M. Piccardi, "Background subtraction techniques: A review," in Proc. IEEE Int. Conf. Syst., Man Cybern., The Hague, The Netherlands, Oct. 2004, vol. 4, pp. 3099-3104. (Pubitemid 40408308)
D. Parks and S. Fels, "Evaluation of background subtraction algorithms with post-processing," in Proc. IEEE Int. Conf. Adv. Video Signal Based Surveillance, Santa Fe, New Mexico, Sep. 2008, pp. 192-199.
T. Bouwmans, F. El Baf, and B. Vachon, "Statistical background modeling for foreground detection: A survey," in Handbook of Pattern Recognition and Computer Vision (Volume 4). Singapore: World Scientific, Jan. 2010, ch. 3, pp. 181-199.
M. Seki, T.Wada, H. Fujiwara, and K. Sumi, "Background subtraction based on cooccurrence of image variations," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Los Alamitos, CA, Jun. 2003, vol. 2, pp. 65-72.
P. Power and J. Schoonees, "Understanding background mixture models for foreground segmentation," in Proc. Image Vis. Comput., Auckland, New Zealand, Nov. 2002, pp. 267-271.
N. Oliver, B. Rosario, and A. Pentland, "A Bayesian computer vision system for modeling human interactions," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 8, pp. 831-843, Aug. 2000.
D.-M. Tsai and S.-C. Lai, "Independent component analysis-based background subtraction for indoor surveillance," IEEE Trans. Image Process., vol. 18, no. 1, pp. 158-167, Jan. 2009.
H.-H. Lin, T.-L. Liu, and J.-C. Chuang, "Learning a scene background model via classification," IEEE Signal Process. Mag., vol. 57, no. 5, pp. 1641-1654, May 2009.
L. Maddalena and A. Petrosino, "A self-organizing approach to background subtraction for visual surveillance applications," IEEE Trans. Image Process., vol. 17, no. 7, pp. 1168-1177, Jul. 2008.
V. Cevher, A. Sankaranarayanan, M. Duarte, D. Reddy, R. Baraniuk, and R. Chellappa, "Compressive sensing for background subtraction," in Proc. Eur. Conf. Comput. Vis., Oct. 2008, pp. 155-168.
M. Dikmen and T. Huang, "Robust estimation of foreground in surveillance videos by sparse error estimation," in Proc. IEEE Int. Conf. Pattern Recognit., Tampa, FL, Dec. 2008, pp. 1-4.
S. Cohen, "Background estimation as a labeling problem," in Proc. Int. Conf. Comput. Vis., Beijing, China, Oct. 2005, vol. 2, pp. 1034-1041. (Pubitemid 44042294)
V. Mahadevan and N. Vasconcelos, "Spatiotemporal saliency in dynamic scenes," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 1, pp. 171-177, Jan. 2010.
M. Sivabalakrishnan and D. Manjula, "An efficient foreground detection algorithm for visual surveillance system," Int. J. Comput. Sci. Network Sec., vol. 9, pp. 221-227, May 2009.
A. Cavallaro and T. Ebrahimi, "Video object extraction based on adaptive background and statistical change detection," in Proc. Vis. Commun. Image Process., Jan. 2001, pp. 465-475. (Pubitemid 32327063)
A. El Maadi and X. Maldague, "Outdoor infrared video surveillance: A novel dynamic technique for the subtraction of a changing background of IR images," Infrared Phys. Technol., vol. 49, pp. 261-265, Jan. 2007. (Pubitemid 46049086)
R. Abbott and L. Williams, "Multiple target tracking with lazy background subtraction and connected components analysis," Mach. Vis. Appl., vol. 20, pp. 93-101, Feb. 2009.
B. Shoushtarian and H. Bez, "A practical adaptive approach for dynamic background subtraction using an invariant colour model and object tracking," Pattern Recognit. Lett., vol. 26, pp. 5-26, Jan. 2005.
J. Cezar, C. Rosito, and S. Musse, "A background subtraction model adapted to illumination changes," in Proc. IEEE Int. Conf. Image Process., Oct. 2006, pp. 1817-1820.
J. Davis and V. Sharma, "Robust background-subtraction for person detection in thermal imagery," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Washington, DC, Jun. 2004, vol. 8, p. 128.
C. Wren, A. Azarbayejani, T. Darrell, and A. Pentland, "Pfinder: Realtime tracking of the human body," IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 7, pp. 780-785, Jul. 1997. (Pubitemid 127762271)
K. Toyama, J. Krumm, B. Brumitt, and M. Meyers, "Wallflower: Principles and practice of background maintenance," in Proc. Int. Conf. Comput. Vis., Kerkyra, Greece, Sep. 1999, pp. 255-261. (Pubitemid 32080048)
D. Koller, J. Weber, and J. Malik, "Robust multiple car tracking with occlusion reasoning," in Proc. Eur. Conf. Comput. Vis., Stockholm, Sweden, May 1994, pp. 189-196.
J. Davis and V. Sharma, "Background-subtraction in thermal imagery using contour saliency," Int. J. Comput. Vis., vol. 71, pp. 161-181, Feb. 2007. (Pubitemid 44359119)
C. Jung, "Efficient background subtraction and shadow removal for monochromatic video sequences," IEEE Trans. Multimedia, vol. 11, no. 3, pp. 571-577, Apr. 2009.
D. Gutchess, M. Trajkovics, E. Cohen-Solal, D. Lyons, and A. Jain, "A background model initialization algorithm for video surveillance," in Proc. Int. Conf. Comput. Vis., Vancouver, BC, Jul. 2001, vol. 1, pp. 733-740. (Pubitemid 32795027)
I. Haritaoglu, D. Harwood, and L. Davis, "W4 : Real-time surveillance of people and their activities," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 8, pp. 809-830, Aug. 2000.
J. Jacques, C. Jung, and S. Musse, "Background subtraction and shadow detection in grayscale video sequences," in Proc. Brazilian Symp. Comput. Graph. Image Process., Natal, Brazil, Oct. 2005, pp. 189-196.
A. Manzanera and J. Richefeu, "A robust and computationally efficient motion detection algorithm based on sigma-delta background estimation," in Proc. Indian Conf. Comput. Vis., Graph. Image Process., Kolkata, India, Dec. 2004, pp. 46-51.
A. Manzanera and J. Richefeu, "A new motion detection algorithm based on σ- Δ background estimation," Pattern Recognit. Lett., vol. 28, pp. 320-328, Feb. 2007. (Pubitemid 44821434)
A. Manzanera, "σ- Δ background subtraction and the Zipf law," in Proc. Progr. Pattern Recognit., Image Anal. Appl., Nov. 2007, pp. 42-51.
L. Lacassagne, A. Manzanera, J. Denoulet, and A. Mérigot, "High performance motion detection: Some trends toward new embedded architectures for vision systems," J. Real-Time Image Process., vol. 4, pp. 127-146, Jun. 2009.
L. Lacassagne, A. Manzanera, and A. Dupret, "Motion detection: Fast and robust algorithms for embedded systems," in Proc. IEEE Int. Conf. Image Process., Cairo, Egypt, Nov. 2009, pp. 3265-3268.
C. Stauffer and E. Grimson, "Adaptive background mixture models for real-time tracking," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Ft. Collins, CO, Jun. 1999, vol. 2, pp. 246-252.
C. Stauffer and E. Grimson, "Learning patterns of activity using realtime tracking," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 8, pp. 747-757, Aug. 2000.
B. Lei and L. Xu, "Real-time outdoor video surveillance with robust foreground extraction and object tracking via multi-state transition management," Pattern Recognit. Lett., vol. 27, pp. 1816-1825, Nov. 2006. (Pubitemid 44382700)
Y. Wang, T. Tan, K. Loe, and J. Wu, "A probabilistic approach for foreground and shadow segmentation in monocular image sequences," Pattern Recognit., vol. 38, pp. 1937-1946, Nov. 2005. (Pubitemid 41214636)
Y. Wang, K. Loe, and J. Wu, "A dynamic conditional random field model for foreground and shadow segmentation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 2, pp. 279-289, Feb. 2006. (Pubitemid 46395294)
O. Barnich, S. Jodogne, and M. Van Droogenbroeck, "Robust analysis of silhouettes by morphological size distributions," in Advanced Concepts for Intelligent Vision Systems (ACIVS 2006), Vol. 4179 of Lecture Notes on Computer Science. New York: Springer-Verlag, Sep. 2006, pp. 734-745.
J.-S. Hu and T.-M. Su, "Robust background subtraction with shadow and highlight removal for indoor surveillance," EURASIP J. Appl. Signal Process., vol. 2007, pp. 108-108, Jan. 2007.
P. KaewTraKulPong and R. Bowden, "An improved adaptive background mixture model for real-time tracking with shadow detection," in Proc. Eur. Workshop Adv. Video Based Surveillance Syst., London, U.K., Sep. 2001.
D. Lee, "Effective Gaussian mixture learning for video background subtraction," IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 5, pp. 827-832, May 2005. (Pubitemid 40608416)
Q. Zang and R. Klette, "Robust background subtraction and maintenance," in Proc. IEEE Int. Conf. Pattern Recognit., Washington, DC, Aug. 2004, vol. 2, pp. 90-93.
Z. Zivkovic, "Improved adaptive gausian mixture model for background subtraction," in Proc. IEEE Int. Conf. Pattern Recognit., Cambridge, U.K., Aug. 2004, vol. 2, pp. 28-31.
B. White and M. Shah, "Automatically tuning background subtraction parameters using particle swarm optimization," in Proc. IEEE Int. Conf. Multimedia Expo, Beijing, China, Jul. 2007, pp. 1826-1829.
P. Varcheie, M. Sills-Lavoie, and G.-A. Bilodeau, "A multiscale region- based motion detection and background subtraction algorithm," Sensors, vol. 10, pp. 1041-1061, Jan. 2010.
A. Elgammal, D. Harwood, and L. Davis, "Non-parametric model for background subtraction," in Proc. 6th Eur. Conf. Comput. Vis., London, U.K., Jun.-Jul. 2000, pp. 751-767.
A. Srivastava, A. Lee, E. Simoncelli, and S.-C. Zhu, "On advances in statistical modeling of natural images," J. Math. Imag. Vis., vol. 18, pp. 17-33, Jan. 2003.
A. Elgammal, R. Duraiswami, D. Harwood, and L. Davis, "Background and foreground modeling using nonparametric kernel density estimation for visual surveillance," Proc. IEEE, vol. 90, no. 7, pp. 1151-1163, Jul. 2002.
A. Mittal and N. Paragios, "Motion-based background subtraction using adaptive kernel density estimation," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Los Alamitos, CA, Jun.-Jul. 2004, vol. 2, pp. 302-309.
Y. Sheikh and M. Shah, "Bayesian modeling of dynamic scenes for object detection," IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 11, pp. 1778-1792, Nov. 2005. (Pubitemid 41685113)
Z. Zivkovic and F. van der Heijden, "Efficient adaptive density estimation per image pixel for the task of background subtraction," Pattern Recognit. Lett., vol. 27, pp. 773-780, May 2006. (Pubitemid 43374860)
A. Tavakkoli, M. Nicolescu, G. Bebis, and M. Nicolescu, "Non-parametric statistical background modeling for efficient foreground region detection," Mach. Vis. Appl., vol. 20, pp. 395-409, Oct. 2008.
E. Monari and C. Pasqual, "Fusion of background estimation approaches for motion detection in non-static backgrounds," in Proc. IEEE Int. Conf. Adv. Video Signal Based Surveillance, London, U.K., Sep. 2007, pp. 347-352.
K. Kim, T. Chalidabhongse, D. Harwood, and L. Davis, "Background modeling and subtraction by codebook construction," in Proc. IEEE Int. Conf. Image Process., Singapore, Oct. 2004, vol. 5, pp. 3061-3064. (Pubitemid 40820720)
K. Kim, T. Chalidabhongse, D. Harwood, and L. Davis, "Real-time foreground-background segmentation using codebook model," Real- Time Imag., vol. 11, Special Issue on Video Object Processing, pp. 172-185, Jun. 2005. (Pubitemid 40929458)
M.Wu and X. Peng, "Spatio-temporal context for codebook-based dynamic background subtraction," Int. J. Electron. Commun., vol. 64, no. 8, pp. 739-747, 2010.
H. Wang and D. Suter, "Background subtraction based on a robust consensus method," in Proc. IEEE Int. Conf. Pattern Recognit.,Washington, DC, Aug. 2006, pp. 223-226. (Pubitemid 46531883)
H. Wang and D. Suter, "A consensus-based method for tracking: Modelling background scenario and foreground appearance," Pattern Recognit., vol. 40, pp. 1091-1105, Mar. 2007. (Pubitemid 44667747)
P.-M. Jodoin, M. Mignotte, and J. Konrad, "Statistical background subtraction using spatial cues," IEEE Trans. Circuits Syst. Video Technol., vol. 17, no. 12, pp. 1758-1763, Dec. 2007. (Pubitemid 350183927)
A. Criminisi, P. Pérez, and K. Toyama, "Region filling and object removal by exemplar-based image inpainting," IEEE Trans. Image Process., vol. 13, no. 9, pp. 1200-1212, Sep. 2004.
A. Papoulis, Probability, Random Variables, and Stochastic Processes. New York: McGraw-Hill, 1984.
R. Cucchiara, C. Grana, M. Piccardi, and A. Prati, "Detecting moving objects, ghosts, and shadows in video streams," IEEE Trans. Pattern Anal. Mach. Intell., vol. 25, no. 10, pp. 1337-1342, Oct. 2003.
L. Li, W. Huang, I. Gu, and Q. Tian, "Foreground object detection from videos containing complex background," in Proc. ACMInt. Conf. Multimedia, Berkeley, CA, Nov. 2003, pp. 2-10.
C.-C. Chiu, M.-Y. Ku, and L.-W. Liang, "A robust object segmentation system using a probability-based background extraction algorithm," IEEE Trans. Circuits Syst. Video Technol., vol. 20, no. 4, pp. 518-528, Apr. 2010.
B. Lucas and T. Kanade, "An iterative image registration technique with an application to stereo vision," in Proc. Int. Joint Conf. Artif. Intell., Vancouver, BC, Apr. 1981, pp. 674-679. (Pubitemid 14478744)