Every year, just after the Super Bowl, America's top college football players head to Indianapolis. It's a rite of spring, like the migration of migratory birds. Their destination is the Combine, a week-long event where NFL teams evaluate player talent and decide who to select in the next NFL draft.
Not far from the stadium, in a convention center ballroom, another “combine” is taking place, where the centerpiece isn't a 40-yard sprint but six-minute research presentations. The participants aren't athletes, but data scientists competing in the final round of the Big Data Bowl, a competition launched by the NFL in 2018 that pits teams of researchers against each other applying analytics and AI tools to football data.
Over the past few years, analytics have allowed NFL teams to evaluate players in ways never before possible — not just tackle success, but also a defender's ability to create tackle opportunities. Coaches use these metrics to streamline game preparation, and fans, as well as bettors and bookmakers, are eager for the insights provided by what the NFL is calling Next Gen Stats.
Big Data Bowl participants, like players, have the potential to be recruited by football teams. About 40 have been recruited by about 20 teams, according to Mike Lopez, the NFL's senior director of football data and analytics. Other participants are joining companies like Zellus Analytics, StatsBomb and Telemetry Sports, which provide data and services to NFL teams and other sports teams. (Stephanie Kowalczyk, a data scientist at Zellus Analytics, explained how the same technique could be applied to a variety of sports in 2023.) Annual Review of Statistics and Its Applications.
The 2024 Big Data Bowl had more than 300 entries, narrowed down to five finalists invited to Indianapolis. “We have academics, industry professionals, students, and student-coach collaborations,” says Ron Yurko, a statistician at Carnegie Mellon University in Pittsburgh and one of this year's finalists. The goal is to gain insights that are “meaningful to football.”
Track your every move
Since 2014, NFL players have been wearing computer chips in their shoulder pads. The chips record the player's position, direction, speed and acceleration 10 times per second. “The next generation of statistics in football means tracking the player,” Lopez said. Since 2017, similar chips have been embedded in balls, and since 2018, all the data has been accessible to all teams.
But that's only a small part of the story. What really makes today's statistics stand out is how they're analyzed. The goal is to understand not just what happened, but also why: Why did this run only gain three yards, but that one go for 88 yards and a touchdown? In the process, Next Gen Stats can, for the first time, quantify the contributions of unsung players who never touch the ball, like the blocker who freed the runner for an 88-yard touchdown.
Katherine Dai, one of this year's finalists, said the research presented at the 2024 Big Data Bowl featured two complementary approaches: Analytics typically uses human-devised mathematical formulas to extract meaningful metrics from data; in contrast, machine learning (the approach that has resulted in generative AI like ChatGPT) trains computers to find the most predictive features.
If a metric only captures what happened, it's probably analytics. If it relies on predictions or probabilities of what could have happened, it's probably machine learning, Dai says.
When the NFL hired Lopez, a former statistics professor at Skidmore College in New York and former college football player, he pitched the idea of a Big Data Bowl in his interview, saying it would be like the 1989 film: Field of Dreams“If you make your data public, analysts will come,” he said. But when the first contest received just three submissions three hours before the submission deadline, he became worried. “Then the submissions started pouring in,” he says — 100 between 9 p.m. and midnight. “This was a lesson in how data scientists do their jobs.”
Since then, the competition has had a specific theme each year: in 2020, for example, it used tracking data to predict the expected yards gained on a running play at any moment in time based on the position and speed of 22 players, a task specifically created for machine learning.
The winners were Austrian data scientists Philipp Singer and Dmitry Gordeev, who had only a rudimentary knowledge of American football. They were both computer-based “grandmasters” and had developed a neural network, a common machine learning algorithm, that outshone their opponents.
Singer and Gordeev's algorithm was used to power several new Next Generation statistics: expected rushing yards, rushing yards over expected (the difference between actual yards gained and expected), first down probability, and touchdown probability, which debuted on national television just six months later.
Ensure victory
If you were betting on a team to win in 2024, you might be wise to pick Yurko's team. He was working on football analytics before the NFL got interested. In 2017, Yurko and his colleagues published a method to estimate a football player's WAR (Wins Above Replacement), which is defined as the partial number of wins a particular player produces compared to the average replacement. (It's “partial” because only a portion of the credit for the wins is given to the player.)
While WAR has been the go-to metric in baseball for over 20 years, it's not as easy to generalize to football. Journal of Quantitative Sports Analysisinspired Nate Sterken, winner of the first Big Data Bowl and now chief data scientist for the Cleveland Browns, to pursue a career in football analytics.
Yurko was a judge for the Big Data Bowl, but after joining the faculty at Carnegie Mellon University, he stopped being a judge because, he says, “I wanted the students to win.” In fact, his students were on two of the five teams that made it to the finals this year, and one student, Quang Nguyen, made it to the finals two years in a row.
The theme for 2024 was tackling, and Yurko's team used tracking data to calculate a physics-based partial tackle metric. After identifying when a runner's forward momentum was significantly reduced, the computer would identify nearby defenders and split the credit accordingly. For example, if two defenders were nearby when the runner's momentum was reduced by 50 percent, they would each get 25 percent credit for the final tackle.
The tackle rate metric highlights the contributions of defensive linemen who often slow down runners but make fewer tackles, and these linemen (or their agents) can use this statistic during salary negotiations, for example.
But Yurko's team didn't win. Instead, it was Dai, Matthew Zhang, Daniel Zhang, and Harvey Chen who took home the first place and the $25,000 prize. Three of the data scientists met as graduate students at Princeton University. None had ever participated in a coding competition before. “We joked that it would be a good excuse to watch football,” Dai says. None had worked in sports analytics, but “you're welcome to join,” she adds.
The team initially tried to predict when a tackle would occur within the next second, but their three neural network algorithms weren't accurate enough. So the team turned to another well-known machine learning technique, decision trees, and had great success: they were able to predict tackles better and even identify near misses.
After Chang graphed the probability of multiple defenders making tackles on the same play over time, he noticed peaks and valleys. Comparing that to video of the play, it became clear that the peaks corresponded to times when someone missed a tackle. “All credit to Matt,” Dye said.
As a result, the team came up with a quantitative definition of a missed tackle: It occurs when a defensive player's probability of making a tackle is above 75 percent for at least half a second, then drops below 75 percent, and within the next second, neither the player nor his teammates make the tackle. It's a simple definition, but the key is the probability calculation, which relies on machine learning.
All of these metrics still have room to evolve. Matt Edwards, head of American football analytics at StatsBomb, points out that both teams graded tackles based on distance, not actual contact with the runner. That's a limitation of tracking data: the chip can't tell if a player is making contact, something the old-fashioned way of doing it – having humans watch game video – can do.
While the chip-based data isn't available to college players, several teams will be taking video tracking data and new analytics into account when the next NFL draft begins April 25.
Edwards points to the Los Angeles Rams as an example. Rather than relying on how players perform in the 40-yard dash and other combine events, which don't replicate what happens in an actual game, the Rams focus exclusively on tracking data. “We want to know how quickly players get off the ball,” Edwards says. “What's their approach speed and reaction time when the ball is in the air? Those are football-specific skills.”
