Blab Sports
New Preprint Proposes Transformer-Based Dense Representation for Football Event Data
A preprint submitted to arXiv on 8 June 2026 introduces a transformer‑based approach for encoding football event data. The paper, titled A Universal Dense Football Event Representation Based on TabTransformer, is authored by Weiran Yang, Daniel Memmert, and Maximilian Klemp‑Weins and is slated for presentation at the 13th Workshop on Machine Learning and Data Mining for Sports Analytics (MLSA 2026).
Football event data are a rich source of spatiotemporal information that combine continuous location coordinates with categorical variables such as action type, action outcome, and body part. Existing methods for preparing these data for machine learning typically rely on one‑hot or ordinal embeddings for categorical features. The authors argue that such representations ignore the intrinsic semantics of action descriptors and therefore limit the quality of downstream analyses.
The proposed method applies the TabTransformer architecture—a transformer model adapted for tabular data—to learn latent dependencies among categorical event features. In the TabTransformer, each categorical feature is first mapped to a learned embedding vector. A stack of self‑attention layers then contextualises these embeddings, allowing the model to capture relationships between different categorical attributes within a single event record. Continuous features, such as player coordinates, are fed directly into the network without discretisation.
After training, the transformer produces a dense vector representation for each football event. These representations can be used as inputs to a variety of downstream tasks. The authors evaluated the approach on two representative problems: action value estimation and play‑style recognition. In both cases, the transformer‑derived embeddings yielded superior probability calibration compared with task‑specific baselines. Calibration was measured using the Brier score, a proper scoring rule that quantifies the mean squared difference between predicted probabilities and actual outcomes. Lower Brier scores indicate better calibration.
The paper reports that the dense embeddings outperform baselines across multiple metrics, suggesting that the transformer captures meaningful semantic structure in the categorical data. This improvement has practical implications for sports analytics, where accurate probabilistic predictions are essential for match‑outcome forecasting, player evaluation, and tactical pattern recognition.
The preprint is 12 pages long and includes a single figure illustrating the model architecture. It is available on arXiv under the identifier arXiv:2606.09327 and can be accessed via the DOI 10.48550/arXiv.2606.09327. The submission history shows that version 1 was uploaded on 8 June 2026.
While the paper focuses on football, the underlying methodology is applicable to other sports that generate event‑level tabular data. By replacing one‑hot encodings with learned embeddings and leveraging self‑attention, the approach offers a general framework for generating dense, semantically rich representations of heterogeneous event data.
The authors do not report any external funding or conflicts of interest. They also do not provide code or dataset links in the preprint, but the methodology can be replicated using publicly available implementations of the TabTransformer.
In summary, the paper presents a novel application of transformer‑based self‑attention to the domain of football event analytics. By learning dense representations that preserve categorical semantics, the approach improves probability calibration for downstream predictive tasks, offering a promising direction for future research in sports analytics and machine learning.
Football event data are a rich source of spatiotemporal information that combine continuous location coordinates with categorical variables such as action type, action outcome, and body part. Existing methods for preparing these data for machine learning typically rely on one‑hot or ordinal embeddings for categorical features. The authors argue that such representations ignore the intrinsic semantics of action descriptors and therefore limit the quality of downstream analyses.
The proposed method applies the TabTransformer architecture—a transformer model adapted for tabular data—to learn latent dependencies among categorical event features. In the TabTransformer, each categorical feature is first mapped to a learned embedding vector. A stack of self‑attention layers then contextualises these embeddings, allowing the model to capture relationships between different categorical attributes within a single event record. Continuous features, such as player coordinates, are fed directly into the network without discretisation.
After training, the transformer produces a dense vector representation for each football event. These representations can be used as inputs to a variety of downstream tasks. The authors evaluated the approach on two representative problems: action value estimation and play‑style recognition. In both cases, the transformer‑derived embeddings yielded superior probability calibration compared with task‑specific baselines. Calibration was measured using the Brier score, a proper scoring rule that quantifies the mean squared difference between predicted probabilities and actual outcomes. Lower Brier scores indicate better calibration.
The paper reports that the dense embeddings outperform baselines across multiple metrics, suggesting that the transformer captures meaningful semantic structure in the categorical data. This improvement has practical implications for sports analytics, where accurate probabilistic predictions are essential for match‑outcome forecasting, player evaluation, and tactical pattern recognition.
The preprint is 12 pages long and includes a single figure illustrating the model architecture. It is available on arXiv under the identifier arXiv:2606.09327 and can be accessed via the DOI 10.48550/arXiv.2606.09327. The submission history shows that version 1 was uploaded on 8 June 2026.
While the paper focuses on football, the underlying methodology is applicable to other sports that generate event‑level tabular data. By replacing one‑hot encodings with learned embeddings and leveraging self‑attention, the approach offers a general framework for generating dense, semantically rich representations of heterogeneous event data.
The authors do not report any external funding or conflicts of interest. They also do not provide code or dataset links in the preprint, but the methodology can be replicated using publicly available implementations of the TabTransformer.
In summary, the paper presents a novel application of transformer‑based self‑attention to the domain of football event analytics. By learning dense representations that preserve categorical semantics, the approach improves probability calibration for downstream predictive tasks, offering a promising direction for future research in sports analytics and machine learning.
