References of "Laugraud, Benjamin"
     in
Bookmark and Share    
Full Text
See detailMotion-aware temporal median filtering for robust background estimation
Laugraud, Benjamin ULiege

Doctoral thesis (2020)

Given an input video sequence, whose frames depict the same scene at different times, the background estimation problem consists in generating a model of the scene background, free of the foreground ... [more ▼]

Given an input video sequence, whose frames depict the same scene at different times, the background estimation problem consists in generating a model of the scene background, free of the foreground elements occluding it. In this thesis, we are interested in a unimodal variant of this problem, whose resulting background model is a single image. To date, background estimation, that is often confused with background subtraction, has been marginally explored. The simplest method, called the temporal median filter, consists in computing the median pixel value in each pixel position. While it produces excellent results for basic scenes, it relies on the strong assumption that the background is observed more than half of the time in each pixel position. As this assumption is rarely met in complex video sequences, such as the ones containing a large amount of foreground elements, or subject to background motion and/or intermittent motion, the temporal median filter usually fails to generate a clean background image for realistic scenes. In this thesis, we propose LaBGen, a new background estimation method built upon the temporal median filter while improving its robustness. It is mainly based on the idea that, if we had an information indicating for a given frame which pixels are in motion, we could filter out foreground pixel values considered during median computations, and relax the need to observe the background more than half of the time. After describing and justifying the design of LaBGen, we test the relevance of the motion detection performed by different popular background subtraction algorithms for our task. It turns out that the simple frame difference algorithm enables LaBGen to achieve its best performance. For this reason, we integrate this algorithm in LaBGen-P, another of our methods that improves LaBGen by avoiding some artifacts that it sometimes introduces into the generated background images. In addition to achieving a better performance than many sophisticated state-of-the-art methods, while having a much lower run time, LaBGen and LaBGen-P were ranked first in the international IEEE Scene Background Modeling Contest organized in 2016. Thereafter, we study the relationship between the performance of motion detection, and the performance of our methods. Although we do not find an obvious correlation between both, we make the assumption upon previous experimental evidence that a temporally memoryless motion detection is the most relevant for LaBGen. Unlike a temporally aware motion detection that does not ignore the temporal information history, a temporally memoryless approach detects motion between two frames without relying on additional past frames. Based on this hypothesis, we design LaBGen-OF, a variant of LaBGen that leverages temporally memoryless optical flow algorithms (i.e. that determine the displacement of each pixel from one frame to another). A consecutive performance study highlights that LaBGen-OF always performs better than LaBGen embedded with different temporally aware motion detection algorithms. Even better, LaBGen-OF is ranked number 2 over 30 on the popular SBMnet background estimation dataset, and takes the lead in 2 categories over 8. These promising results lead us to push the temporally memoryless even further. For this purpose, we propose two homemade intra-frame motion detection algorithms that leverage semantic segmentation (i.e. a segmentation indicating which object is currently depicted in a given pixel) to determine the possibility of observing motion from spatial information only. Afterwards, we integrate those algorithms into a new variant of LaBGen-P, called LaBGen-P-Semantic, and determine their relevance to our task. In addition to validate the use of intra-frame motion detection algorithms, a performance evaluation shows that LaBGen-P-Semantic performs better than LaBGen and LaBGen-P, and takes the lead in 3 other SBMnet categories. Finally, an additional contribution of this thesis lies in the performance evaluation subfield. Indeed, as most evaluation methodologies and datasets are used blindly without ever being questioned, we describe and analyze them in depth in order to determine whether such trust is justified. It turns out that some evaluation tools are mathematically inaccurate, and/or redundant, and/or not well correlated with what the visual perception of a human considers as an acceptable background image. In addition, the public implementations of some of those tools return erroneous results. We thus revisit the performance evaluation paradigms used in background estimation, review the problems, and provide possible solutions. Furthermore, as no methodology to assess online background estimation methods (i.e. generating a background image after each frame of the input video sequence) has been proposed to date, we provide insights into how to evaluate such methods. Our proposal is based on paradigms borrowed from the video quality assessment field, and a proof of concept shows that it is able to discriminate the performance of two different online methods that traditional evaluation tools consider to be identical. [less ▲]

Detailed reference viewed: 334 (113 ULiège)
Full Text
See detailChallenges and Benchmarks: Evaluation of Background Subtraction Algorithms
Laugraud, Benjamin ULiege; Bouwmans, Thierry

Conference (2018, August 28)

Detailed reference viewed: 66 (20 ULiège)
Full Text
Peer Reviewed
See detailLaBGen-P-Semantic: A First Step for Leveraging Semantic Segmentation in Background Generation
Laugraud, Benjamin ULiege; Pierard, Sébastien ULiege; Van Droogenbroeck, Marc ULiege

in Journal of Imaging (2018), 4(7), 86

Given a video sequence acquired from a fixed camera, the stationary background generation problem consists of generating a unique image estimating the stationary background of the sequence. During the ... [more ▼]

Given a video sequence acquired from a fixed camera, the stationary background generation problem consists of generating a unique image estimating the stationary background of the sequence. During the IEEE Scene Background Modeling Contest (SBMC) organized in 2016, we presented the LaBGen-P method. In short, this method relies on a motion detection algorithm for selecting, for each pixel location, a given amount of pixel intensities that are most likely static by keeping the ones with the smallest quantities of motion. These quantities are estimated by aggregating the motion scores returned by the motion detection algorithm in the spatial neighborhood of the pixel. After this selection process, the background image is then generated by blending the selected intensities with a median filter. In our previous works, we showed that using a temporally-memoryless motion detection, detecting motion between two frames without relying on additional temporal information, leads our method to achieve the best performance. In this work, we go one step further by developing LaBGen-P-Semantic, a variant of LaBGen-P, the motion detection step of which is built on the current frame only by using semantic segmentation. For this purpose, two intra-frame motion detection algorithms, detecting motion from a unique frame, are presented and compared. Our experiments, carried out on the Scene Background Initialization (SBI) and SceneBackgroundModeling.NET (SBMnet) datasets, show that leveraging semantic segmentation improves the robustness against intermittent motions, background motions and very short video sequences, which are among the main challenges in the background generation field. Moreover, our results confirm that using an intra-frame motion detection is an appropriate choice for our method and paves the way for more techniques based on semantic segmentation. [less ▲]

Detailed reference viewed: 198 (63 ULiège)
Full Text
See detailTutorial on Evaluation of Background Subtraction Algorithms
Laugraud, Benjamin ULiege; Bouwmans, Thierry

Conference (2018, April 03)

The tutorial focused on the BGSLibrary (https://github.com/andrewssobral/bgslibrary) and the CDnet 2014 dataset (http://www.changedetection.net), and answered three questions: how to install the ... [more ▼]

The tutorial focused on the BGSLibrary (https://github.com/andrewssobral/bgslibrary) and the CDnet 2014 dataset (http://www.changedetection.net), and answered three questions: how to install the BGSLibrary? How to add your own code in the BGSLibrary? How to use the CDnet 2014 dataset and its metrics? [less ▲]

Detailed reference viewed: 220 (18 ULiège)
Full Text
See detailAn Overview of Background Initialization and LaBGen
Laugraud, Benjamin ULiege

Conference (2018, April 03)

Given a video sequence captured from a static viewpoint, the stationary background initialization problem consists in generating a unique image estimating the stationary background of the sequence (i.e ... [more ▼]

Given a video sequence captured from a static viewpoint, the stationary background initialization problem consists in generating a unique image estimating the stationary background of the sequence (i.e. the set of elements which are motionless throughout the sequence). Generating an estimation of the background is helpful, and sometimes crucial for many applications including video surveillance, segmentation, compression, inpainting, privacy protection, and computational photography. The aim of this talk is to first introduce the background initialization field by presenting the main challenges, some important methods, and the evaluation framework. Second, LaBGen, which emerged as the best method during the Scene Background Modeling and Initialization (SBMI 2015) workshop and IEEE Scene Background Modeling Contest (SBMC 2016), will be presented in depth. [less ▲]

Detailed reference viewed: 176 (44 ULiège)
Full Text
Peer Reviewed
See detailIs a Memoryless Motion Detection Truly Relevant for Background Generation with LaBGen?
Laugraud, Benjamin ULiege; Van Droogenbroeck, Marc ULiege

in Advanced Concepts for Intelligent Vision Systems (2017, September)

The stationary background generation problem consists in generating a unique image representing the stationary background of a given video sequence. The LaBGen background generation method combines a ... [more ▼]

The stationary background generation problem consists in generating a unique image representing the stationary background of a given video sequence. The LaBGen background generation method combines a pixel-wise median filter and a patch selection mechanism based on a motion detection performed by a background subtraction algorithm. In our previous works related to LaBGen, we have shown that, surprisingly, the frame difference algorithm provides the most effective motion detection on average. Compared to other background subtraction algorithms, it detects motion between two frames without relying on additional past frames, and is therefore memoryless. In this paper, we experimentally check whether the memoryless property is truly relevant for LaBGen, and whether the effective motion detection provided by the frame difference is not an isolated case. For this purpose, we introduce LaBGen-OF, a variant of LaBGen leverages memoryless dense optical flow algorithms for motion detection. Our experiments show that using a memoryless motion detector is an adequate choice for our background generation framework, and that LaBGen-OF outperforms LaBGen on the SBMnet dataset. We further provide an open-source C++ implementation of both methods at http://www.telecom.ulg.ac.be/labgen. [less ▲]

Detailed reference viewed: 369 (57 ULiège)
Full Text
Peer Reviewed
See detailLaBGen: A method based on motion detection for generating the background of a scene
Laugraud, Benjamin ULiege; Pierard, Sébastien ULiege; Van Droogenbroeck, Marc ULiege

in Pattern Recognition Letters (2017), 96

Given a video sequence acquired with a fixed camera, the generation of the stationary background of the scene is a challenging problem which aims at computing a reference image for a motionless background ... [more ▼]

Given a video sequence acquired with a fixed camera, the generation of the stationary background of the scene is a challenging problem which aims at computing a reference image for a motionless background. For that purpose, we developed our method named LaBGen, which emerged as the best one during the Scene Background Modeling and Initialization (SBMI) workshop organized in 2015, and the IEEE Scene Background Modeling Contest (SBMC) organized in 2016. LaBGen combines a pixel-wise temporal median filter and a patch selection mechanism based on motion detection. To detect motion, a background subtraction algorithm decides, for each frame, which pixels belong to the background. In this paper, we describe the LaBGen method extensively, evaluate it on the SBI 2016 dataset and compare its performance with other background generation methods. We also study its computational complexity, the performance sensitivity with respect to its parameters, and the stability of the predicted background image over time with respect to the chosen background subtraction algorithm. We provide an open source C++ implementation at http://www.telecom.ulg.ac.be/labgen. [less ▲]

Detailed reference viewed: 588 (90 ULiège)
Full Text
Peer Reviewed
See detailLaBGen-P: A Pixel-Level Stationary Background Generation Method Based on LaBGen
Laugraud, Benjamin ULiege; Pierard, Sébastien ULiege; Van Droogenbroeck, Marc ULiege

in 2016 International Conference on Pattern Recognition Contest Proceedings (2016, December)

Estimating the stationary background of a video sequence is useful in many applications like surveillance, segmentation, compression, inpainting, privacy protection, and computational photography. To ... [more ▼]

Estimating the stationary background of a video sequence is useful in many applications like surveillance, segmentation, compression, inpainting, privacy protection, and computational photography. To perform this task, we introduce the LaBGen-P method based on the principles of LaBGen and the conclusions drawn in the corresponding paper. It combines a pixel-wise median filter and a pixel selection mechanism based on a motion detection performed by the frame difference algorithm. By working with pixels instead of patches, as originally done in LaBGen, it avoids some discontinuities between different spatial areas and generates better visual results. In this paper, we describe the LaBGen-P method, study its performance on the sequences of the SBMnet dataset, and compare it to that of LaBGen and other methods on the same dataset. Both algorithms emerged as the best ones during the IEEE Scene Background Modeling Contest (SBMC) organized in 2016. However, as there is not yet a good understanding of the recommended metrics, and due to the small amount of video sequences provided with the corresponding ground truth, we have performed a subjective evaluation. More precisely, 35 human experts were asked to compare background images estimated by LaBGen-P and LaBGen, and select the best one. From these experiments, it turns out that the results of LaBGen-P are preferred for about two thirds of the video sequences. Note that we provide an open-source C++ implementation at http://www.telecom.ulg.ac.be/labgen. [less ▲]

Detailed reference viewed: 349 (75 ULiège)
Full Text
Peer Reviewed
See detailTime Ordering Shuffling for Improving Background Subtraction
Laugraud, Benjamin ULiege; Latour, Philippe ULiege; Van Droogenbroeck, Marc ULiege

in Advanced Concepts for Intelligent Vision Systems (2015, October)

By construction, a video is a series of ordered frames, whose order is defined at the time of the acquisition process. Background subtraction methods then take this input video and produce a series of ... [more ▼]

By construction, a video is a series of ordered frames, whose order is defined at the time of the acquisition process. Background subtraction methods then take this input video and produce a series of segmentation maps expressed in terms of foreground objects and scene background. To our knowledge, this natural ordering of frames has never been questioned or challenged. In this paper, we propose to challenge, in a prospective view, the natural ordering of video frames in the context of background subtraction, and examine alternative time orderings. The idea consists in changing the order before background subtraction is applied, by means of shuffling strategies, and re-ordering the segmentation maps afterwards. For this purpose, we propose several shuffling strategies and show that, for some background subtraction methods, results are preserved or even improved. The practical advantage of time shuffling is that it can been applied to any existing background subtraction seamlessly. [less ▲]

Detailed reference viewed: 318 (114 ULiège)
Full Text
Peer Reviewed
See detailSimple Median-Based Method for Stationary Background Generation Using Background Subtraction Algorithms
Laugraud, Benjamin ULiege; Pierard, Sébastien ULiege; Braham, Marc ULiege et al

in New Trends in Image Analysis and Processing - ICIAP 2015 Workshops (2015, September)

The estimation of the background image from a video sequence is necessary in some applications. Computing the median for each pixel over time is effective, but it fails when the background is visible for ... [more ▼]

The estimation of the background image from a video sequence is necessary in some applications. Computing the median for each pixel over time is effective, but it fails when the background is visible for less than half of the time. In this paper, we propose a new method leveraging the segmentation performed by a background subtraction algorithm, which reduces the set of color candidates, for each pixel, before the median is applied. Our method is simple and fully generic as any background subtraction algorithm can be used. While recent background subtraction algorithms are excellent in detecting moving objects, our experiments show that the frame difference algorithm is a technique that compare advantageously to more advanced ones. Finally, we present the background images obtained on the SBI dataset, which appear to be almost perfect. The source code of our method can be downloaded at http://www.ulg.ac.be/telecom/research/sbg. [less ▲]

Detailed reference viewed: 550 (210 ULiège)