A Comprehensive Framework for Evaluating Football Player Actions
Written on
This article delves into a groundbreaking research paper titled “Actions Speak Louder than Goals: Valuing Player Actions in Soccer,” which presents a fresh perspective on how to evaluate soccer players' contributions during matches. It highlights a new framework designed to quantify player actions by considering their impact on game outcomes, while also addressing the limitations of conventional metrics that primarily focus on goals and shots.
Introduction
The primary goal of this research is to understand how individual actions by football players influence their team's overall performance. Despite the importance of this analysis for player recruitment, evaluation, and fan engagement, measuring the effect of specific actions can be challenging due to the dynamic and low-scoring nature of football. The authors identify three main limitations of existing evaluation methods: they tend to concentrate on goals and shots, assign static values to actions irrespective of context, and primarily consider immediate effects rather than longer-term implications.
In response to these challenges, the authors propose a novel data-driven framework that evaluates all types of player actions, taking into account the circumstances surrounding each action and its potential longer-term effects. This framework assigns values to actions based on their expected contributions to scoring, with positive values indicating beneficial actions for the team and negative values suggesting a disadvantage.
The research outlines five key contributions: 1. A standardized language for describing player actions. 2. A valuation framework for assessing player actions and generating player ratings based on their game impact. 3. A predictive model for estimating scoring and conceding probabilities at any point in a match. 4. Practical applications showcasing insightful results and findings. 5. A Python package that converts existing event data into the proposed language, implements the valuation framework, and models scoring probabilities.
Ultimately, this research aims to provide a clearer, objective method for quantifying player performances, offering valuable insights for clubs, media, and fans alike.
SPADL: A Language for Describing Player Actions
This section outlines the data sources available for football analysis, specifically focusing on event stream data and optical tracking data. While tracking data can be costly and is typically only accessible to affluent leagues or clubs, event stream data is more readily available and cost-effective. Thus, this study emphasizes the use of event stream data, which records the timing and locations of various events like passes, shots, and fouls during a match. However, analyzing this data comes with its own set of challenges. To tackle these issues, the authors introduce SPADL (Soccer Player Action Description Language), a standardized method for detailing player actions, which enables deeper insights from the data. Although SPADL was primarily developed for event stream data, it can also be adapted for tracking data with minor adjustments.
Five Data Science Challenges Posed by Current Event Stream Data
The authors identify five significant challenges related to current event stream data: 1. The design of event data for multiple objectives can result in missing critical information and the presence of irrelevant details. 2. Each data vendor employs distinct terminology and definitions for events, complicating cross-vendor data analysis. 3. Compatibility issues with older data formats can lead to inefficient design choices. 4. Optional information snippets provided by vendors complicate automated analysis. 5. Many machine learning algorithms require fixed-length feature vectors, making it difficult to handle variable-sized vectors, which necessitates complex preprocessing.
Language Description
SPADL serves as a comprehensive language for describing player actions, aiming to unify various event stream formats into a common vocabulary for enhanced data analysis. It is designed to be user-friendly, straightforward, and comprehensive, accurately defining and detailing in-game actions. SPADL differentiates between actions and events, depicting a match as a sequence of on-the-ball actions characterized by nine attributes, including player identity, team, action type, body part used, and the outcome.
VAEP: A Framework for Valuing Player Actions
This section introduces the VAEP (Valuing Actions by Estimating Probabilities) framework, designed to objectively assess the value of actions performed by football players. This framework translates action values into player ratings that reflect their total contributions—both offensive and defensive—to their team.
Converting Scoring and Conceding Probabilities to Action Values
The authors explain that most actions in football aim to either enhance the likelihood of scoring or reduce the risk of conceding. The framework quantifies an action's impact by measuring its effect on the probabilities of both scoring and conceding in the immediate future.
The offensive value of an action is defined as the change in the likelihood of the team scoring as a result of that action, while the defensive value measures the change in the probability of conceding. The overall action value combines both offensive and defensive contributions, transforming the subjective nature of action valuation into an objective prediction of future events.
Converting Action Values to Player Ratings
The authors elaborate on how to derive player ratings from previously calculated action values, reflecting a player's overall contributions. This rating can be computed for various time frames, including specific game segments or entire seasons.
To determine a player’s rating, the authors aggregate individual action values per 90 minutes of play, effectively capturing the net goal difference attributed to the player. Ratings can also be computed for specific action types, allowing for the creation of player profiles that highlight different playing styles.
Estimating Scoring and Conceding Probabilities
The authors further describe their methodology for estimating the probabilities of teams scoring or conceding in the near future. They define the home and visiting teams' goals as goal(h) and goal(v), respectively. The estimation process simplifies to two binary classification problems focused on the team currently in possession of the ball.
To estimate these probabilities, the authors train a probabilistic classifier using machine learning models, ensuring that the probability estimates are well-calibrated. They convert the series of actions throughout a game into examples in a feature-vector format.
Constructing Labels
This section explains how labels are created for the two binary classification tasks of estimating scoring and conceding probabilities. A game state receives a positive label (= 1) if the team with possession scores or concedes within a specified number of subsequent actions.
Constructing Features
Features for each example are constructed based on the previous three actions rather than the entire game state. This approach allows for a fixed number of features, essential for most machine learning algorithms, and emphasizes the most relevant aspects of the current context.
The authors categorize features into three groups based on the SPADL representation: 1. SPADL features include action type, result, body part used, and spatial coordinates. 2. Complex features combine information from individual actions and consecutive actions. 3. Game context features capture the current scoreline and goals scored by each team.
Experiments
In this section, the authors outline the experiments conducted to evaluate the VAEP framework. Given the lack of objective ground truth for action values or player ratings, the evaluation is challenging. They focus on three primary questions: 1. Understanding how the framework compares to other metrics. 2. Presenting use cases related to player acquisition and characterization. 3. Assessing various design choices.
The authors employ Wyscout data from several top European leagues, applying the VAEP framework to over 11,000 games spanning multiple seasons.
Intuition Behind the Action Values
This section analyzes the actions leading to a goal scored by Barcelona against Real Madrid, illustrating the action values generated by the framework.
Comparing VAEP Player Ratings to Traditional Player Performance Metrics
The authors compare their VAEP ratings against conventional metrics, producing rankings for the top players based on goals, assists, and the VAEP framework. Notably, the VAEP framework identifies impactful players who may not feature prominently in traditional rankings.
Identifying Promising Young Players and Minor League Talent
The authors highlight the potential of the VAEP framework in discovering promising young talents from smaller leagues, noting discrepancies in player valuations based on traditional metrics.
Characterizing Playing Style
The VAEP framework can also assist clubs in identifying players who align with their preferred playing style during recruitment.
Trading Off Action Quality and Quantity
The authors discuss the balance between the quality and quantity of actions, using VAEP to illustrate the performances of various players across leagues.
Evaluating Design Choices
The authors evaluate the performance of their system by comparing their approach with alternative methods, employing metrics like the Brier score and ROC AUC.
Discussion of Remaining Challenges
The authors acknowledge the limitations of the VAEP framework, including its focus on on-the-ball actions and challenges in cross-league comparisons.
Conclusion
In summary, the authors present SPADL as a language for event stream data and VAEP as a framework for valuing player actions in football. They highlight the advantages of their approach, which offers a comprehensive method for evaluating player contributions by considering the game context and a variety of action types.