SoccerNet-v3; Soccer; Football; Artificial intelligence; Deep learning; Dataset; Human tracking; Player tracking; Camera calibration; Player detection; Sports; Sport
Abstract :
[en] AbstractSoccer videos are a rich playground for computer vision, involving many elements, such as players, lines, and specific objects. Hence, to capture the richness of this sport and allow for fine automated analyses, we release SoccerNet-v3, a major extension of the SoccerNet dataset, providing a wide variety of spatial annotations and cross-view correspondences. SoccerNet’s broadcast videos contain replays of important actions, allowing us to retrieve a same action from different viewpoints. We annotate those live and replay action frames showing same moments with exhaustive local information. Specifically, we label lines, goal parts, players, referees, teams, salient objects, jersey numbers, and we establish player correspondences between the views. This yields 1,324,732 annotations on 33,986 soccer images, making SoccerNet-v3 the largest dataset for multi-view soccer analysis. Derived tasks may benefit from these annotations, like camera calibration, player localization, team discrimination and multi-view re-identification, which can further sustain practical applications in augmented reality and soccer analytics. Finally, we provide Python codes to easily download our data and access our annotations.
Research Center/Unit :
Montefiore Institute - Montefiore Institute of Electrical Engineering and Computer Science - ULiège Telim
Disciplines :
Electrical & electronics engineering
Author, co-author :
Cioppa, Anthony ✱; Université de Liège - ULiège > Montefiore Institute of Electrical Engineering and Computer Science
Deliège, Adrien ✱; Université de Liège - ULiège > Département d'électricité, électronique et informatique (Institut Montefiore) > Télécommunications
Giancola, Silvio ✱
Ghanem, Bernard
Van Droogenbroeck, Marc ; Université de Liège - ULiège > Montefiore Institute of Electrical Engineering and Computer Science
✱ These authors have contributed equally to this work.
Language :
English
Title :
Scaling up SoccerNet with multi-view spatial localization and re-identification
Applications et Recherche pour une Intelligence Artificielle de Confiance (ARIAC)
Funders :
F.R.S.-FNRS - Fonds de la Recherche Scientifique SPW - Service Public de Wallonie KAUST - King Abdullah University of Science and Technology
Funding number :
2010235
Funding text :
This work was supported by the Service Public de Wallonie (SPW) Recherche under the DeepSport project and Grant N°. 2010235 (ARIAC by https://DigitalWallonia4.ai), the FRIA, and KAUST Office of Sponsored Research through the Visual Computing Center (VCC) funding.
scite shows how a scientific paper has been cited by providing the context of the citation, a classification describing whether it supports, mentions, or contrasts the cited claim, and a label indicating in which section the citation was made.
Bibliography
Lange, D. Market size of the European professional soccer market from 2006/07 to 2019/20. https://www.statista.com/statistics/261223/european-soccer-market-total-revenue/ (2021).
Moeslund, T. B., Thomas, G. & Hilton, A. Computer vision in sports (Springer, 2014).
Thomas, G., Gade, R., Moeslund, T. B., Carr, P. & Hilton, A. Computer vision for sports: current applications and research topics. Comp. Vision and Image Understanding 159, 3–18, 10.1016/j.cviu.2017.04.011 (2017). DOI: 10.1016/j.cviu.2017.04.011
Cioppa, A. et al. A context-aware loss function for action spotting in soccer videos. In IEEE Int. Conf. Comput. Vis. Pattern Recogn. (CVPR), 13126–13136, https://doi.org/10.1109/CVPR42600.2020.01314 (2020).
Deliège, A. et al. SoccerNet-v2: a dataset and benchmarks for holistic understanding of broadcast soccer videos. In IEEE Int. Conf. Comput. Vis. Pattern Recogn. Work. (CVPRW), 4508–4519, https://doi.org/10.1109/CVPRW53098.2021.00508 (2021).
Giancola, S. & Ghanem, B. Temporally-aware feature pooling for action spotting in video broadcasts. In IEEE Int. Conf. Comput. Vis. Pattern Recogn. Work. (CVPRW), 4485–4494, https://doi.org/10.1109/CVPRW53098.2021.00506 (2021).5
Richly, K., Moritz, F. & Schwarz, C. Utilizing artificial neural networks to detect compound events in spatio-temporal soccer data. In Proc. SIGKDD Work. MiLeTS, 1–7 (2017).
Tomei, M., Baraldi, L., Calderara, S., Bronzin, S. & Cucchiara, R. RMS-Net: regression and masking for soccer event spotting. In IEEE Int. Conf. Pattern Recogn. (ICPR), 7699–7706, https://doi.org/10.1109/ICPR48806.2021.9412268 (2020).
Khaustov, V. & Mozgovoy, M. Recognizing events in spatiotemporal soccer data. Applied Sciences 10, 1–12, 10.3390/app10228046 (2020). DOI: 10.3390/app10228046
Zhou, X., Kang, L., Cheng, Z., He, B. & Xin, J. Feature combination meets attention: Baidu soccer embeddings and transformer based temporal detection. Preprint at https://doi.org/10.48550/arXiv.2106.14447 (2021).
Cioppa, A., Deliège, A., Istasse, M., De Vleeschouwer, C. & Van Droogenbroeck, M. ARTHuS: adaptive real-time human segmentation in sports through online distillation. In IEEE Int. Conf. Comput. Vis. Pattern Recogn. Work. (CVPRW), 2505–2514, https://doi.org/10.1109/CVPRW.2019.00306 (2019).
Cioppa, A. et al. Multimodal and multiview distillation for real-time player detection on a football field. In IEEE Int. Conf. Comput. Vis. Pattern Recogn. Work. (CVPRW), 3846–3855, https://doi.org/10.1109/CVPRW50498.2020.00448 (2020).
Hurault, S., Ballester, C. & Haro, G. Self-supervised small soccer player detection and tracking. In Int. Work. Multimedia Content Analysis in Sports, 9–18, https://doi.org/10.1145/3422844.3423054 (2020).
Manafifard, M., Ebadi, H. & Abrishami Moghaddam, H. A survey on player tracking in soccer videos. Comp. Vision and Image Understanding 159, 19–46, 10.1016/j.cviu.2017.02.002 (2017). DOI: 10.1016/j.cviu.2017.02.002
Kamble, P. R., Keskar, A. G. & Bhurchandi, K. M. A deep learning ball tracking system in soccer videos. Opto-Electronics Review 27, 58–69, 10.1016/j.opelre.2019.02.003 (2019). DOI: 10.1016/j.opelre.2019.02.003
Suzuki, G., Takahashi, S., Ogawa, T. & Haseyama, M. Team tactics estimation in soccer videos based on a deep extreme learning machine and characteristics of the tactics. IEEE Access 7, 153238–153248, 10.1109/ACCESS.2019.2946378 (2019). DOI: 10.1109/ACCESS.2019.2946378
Arbués Sangüesa, A., Martín, A., Fernández, J., Ballester, C. & Haro, G. Using player’s body-orientation to model pass feasibility in soccer. In IEEE Int. Conf. Comput. Vis. Pattern Recogn. Work. (CVPRW), 3875–3884, https://doi.org/10.1109/CVPRW50498.2020.00451 (2020).
Decroos, T., Bransen, L., Van Haaren, J. & Davis, J. Actions speak louder than goals: valuing player actions in soccer. In ACM Int. Conf. Knowl. Disc. and Data Mining (KDD), 1851–1861, https://doi.org/10.1145/3292500.3330758 (2019).
Cioppa, A., Deliège, A. & Van Droogenbroeck, M. A bottom-up approach based on semantics for the interpretation of the main camera stream in soccer games. In IEEE Int. Conf. Comput. Vis. Pattern Recogn. Work. (CVPRW), 1846–1855, https://doi.org/10.1109/CVPRW.2018.00229 (2018).
Agyeman, R., Muhammad, R. & Choi, G. S. Soccer video summarization using deep learning. In IEEE Conf. Multimedia Inf. Process. Retr. (MIPR), 270–273, https://doi.org/10.1109/MIPR.2019.00055 (2019).
Sanabria, M., Sherly, Precioso, F. & Menguy, T. A deep architecture for multimodal summarization of soccer games. In Int. Work. Multimedia Content Anal. Sports (MMSports), 16–24, https://doi.org/10.1145/3347318.3355524 (2019).
He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In IEEE Int. Conf. Comput. Vis. Pattern Recogn. (CVPR), 770–778, https://doi.org/10.1109/CVPR.2016.90 (2016).
Tan, M. & Le, Q. V. EfficientNet: rethinking model scaling for convolutional neural networks. In Int. Conf. Mach. Learn. (ICML), 6105–6114 (2019).
Deng, J. et al. ImageNet: a large-scale hierarchical image database. In IEEE Int. Conf. Comput. Vis. Pattern Recogn. (CVPR), 248–255, https://doi.org/10.1109/CVPR.2009.5206848 (2009).
Lin, T.-Y. et al. Microsoft COCO: common objects in context. In Eur. Conf. Comput. Vision (ECCV), vol. 8693 of Lect. Notes Comput. Sci. 740–755, https://doi.org/10.1007/978-3-319-10602-1_48 (Springer, 2014).
Homayounfar, N., Fidler, S. & Urtasun, R. Sports field localization via deep structured models. In IEEE Int. Conf. Comput. Vis. Pattern Recogn. (CVPR), 4012–4020, https://doi.org/10.1109/CVPR.2017.427 (2017).
Biermann, H. et al. A unified taxonomy and multimodal dataset for events in invasion games. Preprint at https://doi.org/10.48550/arXiv.2108.11149 (2021).
Giancola, S., Amine, M., Dghaily, T. & Ghanem, B. SoccerNet: a scalable dataset for action spotting in soccer videos. In IEEE Int. Conf. Comput. Vis. Pattern Recogn. Work. (CVPRW), 1711–1721, https://doi.org/10.1109/CVPRW.2018.00223 (2018).
Pappalardo, L. et al. A public data set of spatio-temporal match events in soccer competitions. Scientific Data 6, 1–15, 10.1038/s41597-019-0247-7 (2019). DOI: 10.1038/s41597-019-0247-7
Pappalardo, L. et al. Metadata record for: a public data set of spatio-temporal match events in soccer competitions, figshare, 10.6084/m9.figshare.9711164.v2 (2020).
Yu, J. et al. Comprehensive dataset of broadcast soccer videos. In IEEE Conf. Multimedia Inf. Process. Retr. (MIPR), 418–423, https://doi.org/10.1109/MIPR.2018.00090 (2018).
Jiang, Y., Cui, K., Chen, L., Wang, C. & Xu, C. SoccerDB: A large-scale database for comprehensive video understanding. In Int. Work. Multimedia Content Anal. Sports (MMSports), 1–8, https://doi.org/10.1145/3422844.3423051 (2020).
He, K., Gkioxari, G., Dollár, P. & Girshick, R. Mask R-CNN. In IEEE Int. Conf. Comput. Vision (ICCV), 2980–2988, https://doi.org/10.1109/ICCV.2017.322 (2017).
Sha, L. et al. End-to-end camera calibration for broadcast videos. In IEEE Int. Conf. Comput. Vis. Pattern Recogn. (CVPR), 13627–13636, https://doi.org/10.1109/CVPR42600.2020 (2020).
Cioppa, A. et al. Camera calibration and player localization in SoccerNet-v2 and investigation of their representations for action spotting. In IEEE Int. Conf. Comput. Vis. Pattern Recogn. Work. (CVPRW), 4537–4546, https://doi.org/10.1109/CVPRW53098.2021.00511 (2021).
Kurach, K. et al. Google research football: a novel reinforcement learning environment. AAAI Conf. Artificial Intell. 34, 4501–4510, 10.1609/aaai.v34i04.5878 (2020). DOI: 10.1609/aaai.v34i04.5878
Rematas, K., Kemelmacher-Shlizerman, I., Curless, B. & Seitz, S. Soccer on your tabletop. In IEEE Int. Conf. Comput. Vis. Pattern Recogn. (CVPR), 4738–4747, https://doi.org/10.1109/CVPR.2018.00498 (2018).
Morra, L. et al. Slicing and dicing soccer: automatic detection of complex events from spatio-temporal data. In Int. Conf. Image Anal. and Recognit. (ICIAR), vol. 12131 of Lect. Notes Comput. Sci. 107–121, https://doi.org/10.1007/978-3-030-50347-5_11 (2020).
Cioppa, A. et al. SoccerNet-v3: scaling up SoccerNet with multi-view spatial localization and re-identification, figshare, 10.6084/m9.figshare.c.5668645 (2022).
European Commission. Proposal for a regulation of the European parliament and of the council laying down harmonised rules on artificial intelligence (artificial intelligence ACT) and amending certain union legislative ACTs. https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:52021PC0206&from=EN (2021).
Similar publications
Sorry the service is unavailable at the moment. Please try again later.
This website uses cookies to improve user experience. Read more
Save & Close
Accept all
Decline all
Show detailsHide details
Cookie declaration
About cookies
Strictly necessary
Performance
Strictly necessary cookies allow core website functionality such as user login and account management. The website cannot be used properly without strictly necessary cookies.
This cookie is used by Cookie-Script.com service to remember visitor cookie consent preferences. It is necessary for Cookie-Script.com cookie banner to work properly.
Performance cookies are used to see how visitors use the website, eg. analytics cookies. Those cookies cannot be used to directly identify a certain visitor.
Used to store the attribution information, the referrer initially used to visit the website
Cookies are small text files that are placed on your computer by websites that you visit. Websites use cookies to help users navigate efficiently and perform certain functions. Cookies that are required for the website to operate properly are allowed to be set without your permission. All other cookies need to be approved before they can be set in the browser.
You can change your consent to cookie usage at any time on our Privacy Policy page.
