[en] Semantic background subtraction (SBS) has been shown to improve the performance of most background subtraction algorithms by combining them with semantic information, derived from a semantic segmentation network. However, SBS requires high-quality semantic segmentation masks for all frames, which are slow to compute. In addition, most state-of-the-art background subtraction algorithms are not real-time, which makes them unsuitable for real-world applications. In this paper, we present a novel background subtraction algorithm called Real-Time Semantic Background Subtraction (denoted RT-SBS) which extends SBS for real-time constrained applications while keeping similar performances. RT-SBS effectively combines a real-time background subtraction algorithm with high-quality semantic information which can be provided at a slower pace, independently for each pixel. We show that RT-SBS coupled with ViBe sets a new state of the art for real-time background subtraction algorithms and even competes with the non real-time state-of-the-art ones. Note that we provide python CPU and GPU implementations of RT-SBS at https://github.com/cioppaanthony/rt-sbs
Centre/Unité de recherche :
Montefiore Institute - Montefiore Institute of Electrical Engineering and Computer Science - ULiège Telim
Disciplines :
Ingénierie électrique & électronique
Auteur, co-auteur :
Cioppa, Anthony ; Université de Liège - ULiège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Télécommunications
Van Droogenbroeck, Marc ; Université de Liège - ULiège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Télécommunications
Braham, Marc ; Université de Liège - ULiège > Dép. d'électric., électron. et informat. (Inst.Montefiore) > Télécommunications
Langue du document :
Anglais
Titre :
Real-Time Semantic Background Subtraction
Date de publication/diffusion :
octobre 2020
Nom de la manifestation :
IEEE International Conference on Image Processing (ICIP)
Organisateur de la manifestation :
IEEE
Lieu de la manifestation :
Abu Dhabi, Emirats Arabes Unis
Date de la manifestation :
from 25-10-2020 to 28-10-2020
Manifestation à portée :
International
Titre de l'ouvrage principal :
Proceedings of the IEEE International Conference on Image Processing (ICIP)
T. Bouwmans, "Traditional and recent approaches in background modeling for foreground detection: An overview, " Computer Science Review, vol. 11-12, pp. 31-66, May 2014.
C. Stauffer and E. Grimson, "Adaptive background mixture models for real-time tracking, " in IEEE Int. Conf. Comput. Vision and Pattern Recogn. (CVPR), Fort Collins, Colorado, USA, June 1999, vol. 2, pp. 246-252.
A. Elgammal, D. Harwood, and L. Davis, "Nonparametric model for background subtraction, " in Eur. Conf. Comput. Vision (ECCV). June 2000, vol. 1843 of Lecture Notes Comp. Sci., pp. 751-767, Springer.
A. Vacavant, T. Chateau, A. Wilhelm, and L. Lequièvre, "A benchmark dataset for outdoor foreground/background extraction, " in Asian Conf. Comput. Vision (ACCV). Nov. 2012, vol. 7728 of Lecture Notes Comp. Sci., pp. 291-300, Springer.
Y. Wang, P.-M. Jodoin, F. Porikli, J. Konrad, Y. Benezeth, and P. Ishwar, "CDnet 2014: An expanded change detection benchmark dataset, " in IEEE Int. Conf. Comput. Vision and Pattern Recogn. Workshops (CVPRW), Columbus, Ohio, USA, June 2014, pp. 393-400.
C. Cuevas, E. Yanez, and N. Garcia, "Labeled dataset for integral evaluation of moving object detection algorithms: LASIESTA, " Comp. Vision and Image Understanding, vol. 152, pp. 103-117, Nov. 2016.
O. Barnich and M. Van Droogenbroeck, "ViBe: A universal background subtraction algorithm for video sequences, " IEEE Trans. Image Process., vol. 20, no. 6, pp. 1709-1724, June 2011.
P.-L. St-Charles, G.-A. Bilodeau, and R. Bergevin, "Universal background subtraction using word consensus models, " IEEE Trans. Image Process., vol. 25, no. 10, pp. 4768-4781, Oct. 2016.
S. Bianco, G. Ciocca, and R. Schettini, "Combination of video change detection algorithms by genetic programming, " IEEE Trans. Evolutionary Computation, vol. 21, no. 6, pp. 914-928, Dec. 2017.
S. Roy and A. Ghosh, "Real-time adaptive Histogram Min-Max Bucket (HMMB) model for background subtraction, " IEEE Trans. Circuits and Syst. for Video Technol., vol. 28, no. 7, pp. 1513-1525, July 2018.
M. Braham and M. Van Droogenbroeck, "Deep background subtraction with scene-specific convolutional neural networks, " in IEEE Int. Conf. Syst., Signals and Image Process. (IWSSIP), May 2016, pp. 1-4.
B. Garcia-Garcia, T. Bouwmans, and A. J. Rosales Silva, "Background subtraction in real applications: Challenges, current models and future directions, " Comp. Sci. Review, vol. 35, pp. 1-42, Feb. 2020.
W.-B. Zheng, K.-F. Wang, and F.-Y. Wang, "Background subtraction algorithm with Bayesian generative adversarial networks, " Acta Automatica Sinica, vol. 44, no. 5, pp. 878-890, May 2018.
M. Braham, S. Piérard, and M. Van Droogenbroeck, "Semantic background subtraction, " in IEEE Int. Conf. Image Process. (ICIP), Beijing, China, Sept. 2017, pp. 4552-4556.
H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia, "Pyramid scene parsing network, " in IEEE Int. Conf. Comput. Vision and Pattern Recogn. (CVPR), Honolulu, HI, USA, July 2017, pp. 6230-6239.
M. Tezcan, P. Ishwar, and J. Konrad, "BSUV-Net: A fully-convolutional neural network for background subtraction of unseen videos, " in IEEE Winter Conf. Applicat. Comp. Vision (WACV), Snowmass village, Colorado, USA, Mar. 2020, pp. 2774-2783.
S. Minaee, Y. Boykov, F. Porikli, A. Plaza, N. Kehtarnavaz, and D. Terzopoulos, "Image segmentation using deep learning: A survey, " CoRR, vol. abs/2001. 05566v1, 2020.
O. Barnich and M. Van Droogenbroeck, "ViBe: a powerful random technique to estimate the background in video sequences, " in Int. Conf. Acoust., Speech and Signal Process. (ICASSP), Apr. 2009, pp. 945-948.
"Bayesian optimization, " https://github. com/ fmfn/BayesianOptimization, Last accessed: 2019-10-20.
S. h. Lee, G. c. Lee, J. Yoo, and S. Kwon, "WisenetMD: Motion detection using dynamic background region analysis, " Symmetry, vol. 11, no. 5, pp. 1-15, May 2019.
S. Jiang and X. Lu, "WeSamBE: A weight-samplebased method for background subtraction, " IEEE Trans. Circuits and Syst. for Video Technol., vol. 28, no. 9, pp. 2105-2115, Sept. 2018.
P.-L. St-Charles, G.-A. Bilodeau, and R. Bergevin, "A self-adjusting approach to change detection based on background word consensus, " in IEEE Winter Conf. Applicat. Comp. Vision (WACV), Waikoloa Beach, Hawaii, USA, Jan. 2015, pp. 990-997.
P-M. Jodoin and J. Konrad, "Change detection website, " http://changedetection. net/.
