Summer School 2020: What does “running backs don’t matter” really mean?

Feel like starting an argument?

Go to pretty much any gathering of football fans and utter the phrase “running backs don’t matter.”

Like pretty much any slogan that can find a home on a bumper sticker, “Running Backs Don’t Matter” is catchy, full of nuance and context for the group that originated it, and it elicits an emotional response from people who might not get the full context that the shorthand represents — enter the arguments.

There were a few ways I could go as we dip our toes into the analytics pool, but I thought decoding a phrase that has become something of a slogan among analytics wonks is a good place to start. I won’t be weighing in on either side, but rather trying to explain what “running backs don’t matter” actually means in practice.

In the beginning ...

Back in the day, running backs ruled the NFL. Jim Brown, Walter Payton, Bo Jackson, Herschel Walker, Eric Dickerson, Earl Campbell, Gale Sayers. The list of iconic running backs from yesteryear goes on and on. And up until 1978, running the ball well WAS more valuable than passing the ball.

But in the 1980s things began to change and offensive coaches began to really explore the potential of passing. And perhaps nobody distilled the shift in mentality more than Bill Walsh, who once said, “When you gain four yards on the ground against a defense, they think you’re kicking their ass. When you gain four yards in the passing game against a defense, they think they’re kicking your ass. Four yards is four yards. I’ll take it any way I can get it.”

Walsh’s philosophic shift — that yardage is yardage, points are points, and it doesn’t matter how you get them as long as you’re advancing the ball and scoring — flies in the face of received wisdom. After all, the dogma is that “you throw to score, but run to win.”

But Walsh didn’t have that option as a young coach. He didn’t have the personnel for a powerful running game, or (despite Joe Montana’s greatness) the quarterback to execute a vertical passing game. So he decided to take his yardage any way that he could get it — with quick timing passes that played to Jerry Rice’s precision route running and incredible endurance and Montana’s quick release and accuracy. In doing so, he won three Super Bowls, became a two-time coach of the year, a Hall of Famer, and created one of he foundational philosophies for modern football offense.

Fast forward about 30 years and the digital revolution has created another paradigm shift in how some coaches think about the game of football.

The seeds of modern football analytics were sewn back in 1971 when Bengals QB Virgil Carter and Northwestern professor Robert Machol examined more than 8,000 plays from the 1969 season to create the idea of “expected points added” (EPA). But while many of the concepts discussed today trace back to the 1970s, the study of mass data in football didn’t really mature until the last few years.

A series of recent technological advances are responsible for that maturation and the widespread use of analytics we see today. The Internet making film widely available for analysis and communication between stats-minded analysts allowed for a rapid acceleration that just wasn’t possible in the pre-digital age. As well, things like the small and accurate GPS trackers used by the NFL’s NextGenStats have also played important roles as well.

In short, with film and data — as well as sources of data that aren’t accounted for in a traditional box score — becoming available to people who love football but didn’t necessarily grow up in the culture of football (i.e. stats nerds), the door was opened for new insight and perspective. And some of the conclusions seem downright heretical.

They start to say things like “running backs don’t matter.” So now that we have some background as to how we got to this point, let’s take apart the threads of the argument and find out why the analytics community has come to that conclusion and what they’re actually saying.

What goes into running the ball?

On the field, in the moment, a running play seems simple. The ball is snapped, the quarterback hands it off, the running back finds a hole, and gains yardage. Or he doesn’t find a hole and he doesn’t gain yardage. But either way, as the guy carrying the ball, the running back is obviously the most important part of the play, right?

Over the course of studying thousands of running plays, the analytics community has come to view the results of any given running play not as the efforts of a running back. Instead, the view is that other factors at play are permissive to the running back’s success.

On any given running play there is a lot of context that doesn’t show up in the box score. Studies have shown that things like field position, offensive personnel package, and box count (how many defenders are in the tackle box) all play vital roles in determining the success — or failure — of a running play.

Field position

We’ll start with the most basic piece of context you can have on a play: Where it is. It’s often overlooked, but field position is one of the most basic factors taken into account when an offense or defense is calling plays. And, generally speaking, it is easier to run the ball the further away from the end zone you are.

Note: This doesn’t count plays when the offense is backed up on its own goal line. The realistic possibility of a safety changes the calculus considerably.

League-wide, running plays called against a neutral box (seven defenders in the tackle box) from the offense’s 10-yard line (90 yards from the goal line) to the opponent’s 30-yard line (30 yards from the goal line), average 4.43 yards per carry. The high end is 4.6 yards per carry on plays called from 90 to 80 yards from the goal line, to a low end of 4.3 yards per carry on plays called 40-30 yards from the end zone. And really, that makes sense. With 30 to 90 yards of field to defend, defenses are forced to spread out. The game is a bit slower and “deep” players truly line up in the deep part of the field.

But in the red zone, things contract. With just 20 (or fewer) yards of field to defend, defenses become denser. Running plays called in the red zone average just 3.5 yards per carry at the high end and just over 2 yards per carry inside the 10-yard line.

Personnel grouping

The failure of Ben McAdoo has made personnel packages something of a sore point among Giants fans. After the 11-personnel package came to dominate McAdoo’s play calls and the team failed to score 30 points in any one game for two years, fans were sick of seeing three receivers on the field.

However, a look at at the advanced stats show that McAdoo was actually on to something.

By and large, defenses base their decisions on what personnel package to field based on what personnel the offense decides to use. This might seem obvious, but if a defense sees an offense using a three-receiver set, they are very likely to field a nickel package with a third cornerback. From there it follows that the offense will likely have an easier time running against a defense which is fielding six linemen and linebackers than a defense which is fielding seven linemen and linebackers.

For an example of this, let’s take a look at the New York Giants and Dallas Cowboys in 2019.

Last year the Giants and Cowboys both used the 11-personnel package as their “base” offensive personnel group. The Giants used 11-personnel on 74 percent of their offensive plays, while the Cowboys used it on 67 percent of their plays, per SharpFootballStats.

Both offenses were more successful running the ball than throwing it. The Giants had a success rate of 53 percent when running and 42 percent when throwing out of 11 personnel, and the Cowboys had a success rate of 58 percent running and 53 percent throwing when in 11 personnel.

Note: A play is successful when it gains at least 40 percent of yards-to-go on first down, 60% percent of yards-to-go on second down and 100 percent of yards-to-go on third or fourth down.

However, both teams threw much more than passed when in three-receiver sets. Dallas ran on 32 percent of their plays in 11-personnel and the Giants only ran the ball on 28 percent of those plays. Adding weight to the hypothesis, both teams saw the tendencies flip when they fielded 12-personnel. Dallas ran 63 percent of the time with a 50 percent success rate in 12-personnel, and threw 37 percent of the time with a 59 percent success rate. The Giants ran on 55 percent of those plays with a 38 percent success rate, and threw 45 percent of the time with a 43 percent success rate.

With a second tight end on the field, both teams ran more than they threw the ball, but both teams also saw lower success rates running the ball than throwing it.

So it would seem that forcing the defense into smaller packages has a greater effect than using an additional blocking option.

Box count

The final facet is related to personnel packages, and that’s the box count. Commonly abbreviated “MIB” or “Men In Box,” this simply refers to the number of players the defense commits to the tackle box.

The box count is determined, in part, by the offensive personnel, as a defense is much less likely to stack the box against a three- or four-receiver set than against a two tight end or two running back set. While that is in part due to defensive personnel choices — a linebacker is much more likely to line up in the tackle box than a cornerback — it’s also due to offensive alignment.

A tight end, H-back, or fullback in a 12 or 21-personnel package is probably going to be be close to the offensive line. Either attached, detached but close, or in the backfield. That gives the defense motivation to concentrate players around the tackle box, giving blockers more to account for and increasing the odds of an unblocked defender.

Watching the tape, it’s generally easy to get an idea whether a running play is going to be successful simply based on the number of defenders in the tackle box at the start of the play. If you count 8 or more, it’s likely that the play will be stopped for little to no gain, while plays against six or fewer defenders tend to end with sizable gains.

Josh Hermsmeyer, writing for FiveThirtyEight, looked at every NFL run from 2009 to 2018 — a full decade worth of running plays. In one graphic he shows how field position and box count and field position combine to affect rushing efficiency.

https://fivethirtyeight.com/features/the-secret-to-the-rams-blocking-success-isnt-the-linemen-its-sean-mcvay/

Almost across the board, teams ran the ball better when the defense had more field to defend, and were forced into lighter personnel packages.

To go back to our original “simple” running play, the analytic studies found that the offense could strongly influence whether their running back would be able to find a hole and how many yards he would gain based a variety of things independent of the running back.

• Where the offense is on the field when they call the running play.
• Whether they use a personnel package that forces the defense into a favorable personnel match-up.
• Whether the offense uses an alignment that forces the defense to take players out of the tackle box.

All of those factors can help determine whether or not the play will be successful before the ball is even snapped.

The value of a running back

Next we get to the studies of the running back position in particular. Just how much value does running the ball hold for an offense?

Going back to a statement made earlier in this piece, I said that running the ball was roughly more valuable than throwing the ball until 1978.

Writing for Football Perspective, Chase Stuart looked at the league-wide efficiency of passing plays and running plays from 1970 to 2016. He found that up until 1977, running the ball and throwing the ball were roughly equivalent, with running plays edging out passing plays in terms of efficiency. However, after 1978 — when the NFL made several rule changes to open up the passing game, including limiting pass interference to 5 yards beyond the line of scrimmage and allowing offensive linemen to extend their arms and open their hands when blocking — passing the ball became much more efficient.

Since then have seen a steady improvement in the efficiency of passing while running efficiency has stayed about the same.

That’s efficiency, but what about value? We always see the analytics community talking about expected points and the value of plays and players. How do running plays stack up to receiving plays?

The answer is “Not well.”

Perhaps unsurprisingly, the Baltimore Ravens, powered by an unconventional offense and Lamar Jackson’s unique skill set, were the best running team in the NFL by a wide margin. Per Pro Football Reference, the Ravens’ running game added 100.56 expected points over the course of the season (0.17 EPA per running play). The next best running game was fielded by the Dallas Cowboys who totaled just 37.08 expected points on the season, or .082 EPA per play. Only ten teams had positive expected points running the ball.

For comparison, the best passing game in the league last year was fielded (unsurprisingly) by the Super Bowl Champion Kansas City Chiefs. They finished the year with 247.8 expected points, or 0.43 EPA per play. The New Orleans Saints fielded the second-best passing offense with 180.79 expected points (0.31 EPA per play).

Overall, average NFL teams had 73.6 expected points passing (0.132 per play). On running plays, the NFL averaged -11.1 expected points (-0.027 per play). Even if we exclude the Ravens as the obvious outliers they are, the average expected points drops to -11.5 expected points.

For every team in the NFL, running the ball made it less likely that they would score points, and for 22 of them, it potentially took points off the board.

But does that mean the running game is useless? A vestigial play left over from a bygone era of football which we have evolved past?

No.

For the (very rare) teams like the 2019 Ravens, analytics say that running the ball has value on a play-in, play-out basis — still not as much as throwing the ball, but not negligible and certainly not harmful. And for every team there are cases when running the ball isn’t just valuable, but the preferred play.

The first situation in which running increases the likelihood of a team winning is at the end of the game. Running the ball has one undeniable advantage over throwing it for a team that has built a lead by the end of the game: Running plays keep the clock moving.

While throwing the ball averages more yards per play than running it, there is always the possibility that a passing play will result in an incompletion. And as we well know, if the pass falls incomplete, the clock stops. So in this case, offensive plays are less about trying to advance the offense than advancing the clock and deny opportunities to the opposing offense.

The next two situations are in the red zone and in short-yardage situations.

We saw earlier how running gets more difficult close to the goal line and teams average less than 3 yards per carry regardless of the box count. However, those two or three yards get to be very valuable once you get within 10 yards of the goal line. That’s because the field shrinks and the game speeds up. There is less room for receivers to work and quarterbacks have to be even quicker, more decisive, and more accurate than they are in the middle of the field. Being able to run well in these situations (and short-yardage situations are very similar) is a definite advantage.

We saw that for most teams, most running plays were at best a wash and were often prohibitive to scoring.

But in red zone situations, the top 25 running backs all had a positive EPA when rushing from inside the 5-yard line, and 22 of the top 25 running backs gave their team a positive EPA on plays of 3-yards to go (or less) when outside the red zone.

Big Data Bowl 2020

You can be forgiven if you don’t know what the Big Data Bowl is, but for all intents and purposes, it is the Super Bowl for statisticians. Starting in 2019, the NFL held a contest for the analytics community to drive the study forward. Entrants are given a data set generated by NFL NextGenStats and challenged to come up with the most accurate model they can given the available data.

In October of 2019 they announced the subject and the data points. Teams were challenged to predict the outcome of running plays over the last few weeks of the 2019 season.

The 2020 Big Data Bowl was ultimately won by Dmitry Gordeev and Philipp Singer, a pair of data scientists who work for an insurance company in Vienna — about as far from traditional “Football Guys” as you can get.

This is the kind of snapshot they used to train their model:

The play is a handoff to Tennessee Titans running back DerrickHenry against the New England Patriots in 2018.

You might expect that they would use the historic data on Henry’s runs, based on down, distance, personnel, formation, as well as the tendencies of the defense, including against the opponent they’re facing.

That wasn’t what Gordeev and Singer did. Instead they completely disregarded the players’ identity and instead focused on the data coming from NextGenStats, namely position and velocity at the time of handoff.

Basically, Gordeev and Singer used the GPS data from NextGenStats to turn the players into 21 (the quarterback was excluded) individual vectors* and use a neural network to compute the probable outcomes of the play. They turned a football play into a (very complicated) physics problem, and had a program tell them the probable results.

*Note: In physics, a vector is an entity with both magnitude and direction.

As you can guess from the fact that they won the competition (and the $50,000 grand prize), Gordeev and Singer’s solution worked and was predictive. Well, it wasn’t just successful, it was wildly successful and they couldn’t resist flexing on the rest of their competition.

Or, as Gordeev said after the Big Data Bowl was decided, “Judging at the moment of handoff, it is not statistically important who the ball carrier is. It is important what is the situation on the field, driven by starting formations and movements of the players prior to handoff, including rusher’s movement.”

Final thoughts

Looking at the various lines of logic coming leading from the data analysts have examined, you can start to see how and why they could come to the conclusions they have about the game of football.

After all, every football field is the same size and every game is the same length. Every team can anticipate having, roughly, the same number of plays over the course of a game. So given those basics, it would make sense to advise teams to make the best use of those opportunities that they can. Use every play as efficiently as possible and get as much value out of your time with the ball as you can.

So what do proponents of analytics really mean when they say that “running backs don’t matter”?

Well, if we take a step back and look at the various lines of logic that came out of the analysis of massive data sets over the last several years, the statement starts to look a bit more like this:

“Individual running backs don’t matter to the outcome of a running play nearly as much as other factors beyond a running back’s control, such as field position, offensive and defensive personnel packages, and personnel alignment do. Likewise, throwing the ball is much more efficient and likely to result in points than running the ball in most — but not all — situations.”

But that’s probably just a bit wordy for a bumper sticker.