VARS: Video Assistant Referee System for Automated Soccer Decision Making from Multiple Views

The Video Assistant Referee (VAR) has revolutionized association football,
enabling referees to review incidents on the pitch, make informed decisions,
and ensure fairness. However, due to the lack of referees in many countries and
the high cost of the VAR infrastructure, only professional leagues can benefit
from it. In this paper, we propose a Video Assistant Referee System (VARS) that
can automate soccer decision-making. VARS leverages the latest findings in
multi-view video analysis, to provide real-time feedback to the referee, and
help them make informed decisions that can impact the outcome of a game. To
validate VARS, we introduce SoccerNet-MVFoul, a novel video dataset of soccer
fouls from multiple camera views, annotated with extensive foul descriptions by
a professional soccer referee, and we benchmark our VARS to automatically
recognize the characteristics of these fouls. We believe that VARS has the
potential to revolutionize soccer refereeing and take the game to new heights
of fairness and accuracy across all levels of professional and amateur
federations.