The Current QB Rating Formula Has Issues

I did a little research this weekend.  I looked at the calculation for the current QB rating system and, you know what?  It kind of sucks. 

Explanation of the Formula
Here’s the formula – it’s based on 4 general categories.  The specific category for each part of the calculation is in the brackets:



a = ((Comp/Att) * 100) -30) / 20  [Completion percentage]
b = ((TDs/Att) * 100) / 5  [Touchdown Pass pct]
c = (9.5 – (Int/Att) * 100))/4  [Interception pct]
d = ((Yards/Att) – 3) / 4  [Yards per attempt]

The final formula is (a + b + c + d)/.06



a, b, c and d can not be greater than 2.375 or less than zero.




So that you don’t have to go too nuts digging into what the formula is doing, I’ll try to explain as best I can.

The intent of the formula is to give, essentially, equal weighting to each of these categories.  Since some of the numbers are percentages and others are integers, some data manipulation needs to take place (e.g., multiply the percentages by 100). 

Also, not all stats are on the same scale – even if they are both percentages.  For example, 10% is a very good touchdown percentage, but it is a horrible completion percentage.  So, each of the categories is divided by a different amount.  There are a few other pieces in place to “normalize” the numbers and try to ensure equal weighting.

The Formula Applied
To finish up the explanation of the calculation, let me give an example and show you how the categories factor into the final rating.

A player with a 50% completion percentage, 5% TDs, 5.5% interception percentage, and 7 yards per attempt will have a QB rating of 66.67.

This is how each category contributed to the rating:

Completion Percentage = 16.67
Touchdown Percentage + 16.67
Interception Percentage + 16.67
Yards Per Attempt + 16.67

Final QB Rating = 66.67.

So maybe I cheated a little with the numbers chosen.  But, to me, it appears that, when the formula was written, the statistics I laid out above (50% comp, 5% TD, 5.5% Int, 7 Y/A) is what the NFL considered an average or midpoint quarterback.  So, a rating of 66.67 was considered the midpoint of the rating.


I know you’re saying that, today, a rating of 67 gets you a one way ticket to the CFL.  But that’s likely a result of changes in the game – including more restrictions on defensive backs and the growth of efficiency oriented (for lack of a better term) pass offenses like the west coast offense.  Some of these offenses actually replace elements of the running game (for an example see:  Patriots, New England).  This leads to significantly higher completion percentages and fewer interceptions.


Why it Sucks
After reviewing the calculation, I came to the following conclusion.  It sucks.  I have a few reasons why:

·             The formula doesn’t give enough weight to yards per attempt.
·             The formula gives far too much weight to completion and TD %

Despite looking like a very fancy calculation, there is no real science behind it.  I understand that nobody wants to overcomplicate things.  But, if you’re going to have something that looks as horrible as this formula (thereby making things look complicated), there should be some rationale behind it.

I have run a regression model and found that yards per attempt is by far the biggest driver of points scored.  (By no means do I claim to be the first to discover this.  Just check out ColdHardFootballFacts.com or advancednflstats.com to see some pretty in-depth analysis on this topic.  Or you could stay on nflstatanalysis.com, which also has some brilliant stuff – just saying). 

At the end of the day, a quarterback’s job is to score points.  The model also suggested that interception percentage plays a role, but not nearly as great as Y/A.

As far as the other two categories?  Let’s address them in order:

Touchdown percentage, in my mind, is what Brian Burke at AdvancedNFLStats.com would call an “intermediate outcome” (my apologies if I’m mis-using the term).  This means that a player who’s good at other things (e.g. moving the ball down the field) is likely to get more TD opportunities and, therefore, more TD passes.  While better quarterbacks tend to throw more TD passes, it’s probably because the quarterback has more yards and completions and really, a TD pass is really just a completion on a different part of the field.  With that said, I don’t have a huge issue with TD % being a part of the calculation. I just don’t think it adds much.

However…….

I have several issues with completion percentage being a part of model.  Evidence suggests that completion percentage, in and of itself, has no real bearing on points or outcome.  So, it doesn’t deserve equal billing with the other categories – especially Y/A. 

But, here’s the thing.  Although it's not immediately apparent, completion percentage actually has a higher weight than each of the other factors. 

Think about it.  On the surface, completion percentage and Y/A (as well as Int and TD %) get equal billing in the formula.  But, the formula to calculate Y/A could be restated as yards per completion times completion percentage.  This means that completion percentage actually accounts for somewhere around one-third of the model…….and, it doesn’t add anything that Y/A doesn’t.

This would be the equivalent of including yards per completion as well as Y/A.  The two metrics are, essentially, functions of each other.  And, like completion percentage, a player could have averaged 40 yards per completion – but, if he only completed 10% of his throws, he didn’t have a very good game.

Yards per attempt is all you need.

The Extreme Example of Why the QB Rating Sucks
Take the examples below – and yes, I’m intentionally using extremes.

Quarterback 1 – let’s call him Chad Mennington – dinks and dunks his way through a Sunday afternoon.  He finishes 30 for 40 for 100 yards and 1 TD with 0 interceptions.  I don’t know whether he won the game.  And, frankly, it doesn’t matter as anyone watching on TV or in person has long since killed themselves out of boredom.

Quarterback 2 – let’s call him Jay Cutler….I’m not even trying anymore – completes 10 passes on 40 attempts for 350 yards and 2 TDs with 0 interceptions.

What would their QB ratings be?  Quarterback 1 would have a rating of 85.4 while quarterback 2 would have a rating of 80.2.  (See Table 1 to the right)

 
Quarterback 1 probably did nothing to move his team down the field.  But, because his harmless passes were not incomplete, he is given a high rating.



What Next?
I’m just some schlep who is using a geeky website as a way to work through an early mid-life crisis – so, I’ll never have any impact on the NFL changing their QB rating calculation.

But, I have taken a stab at it anyway.  You could actually rank QBs simply by their Y/A and get a better indication of a player relative worth.  Again, go to ColdHardFootballFacts.com to see someone do just that.

I’ve taken it a step further and calculated a rating based on Y/A and interception percentage – with a much higher weight being placed on Y/A (per the regression model).  Additionally, the actual metric used was not simply Y/A, but Y/A adjusted for the quality of the opponent.  The metric was Y/A minus the average Y/A of the opponent (the same metric was used for interception percentage).  My adjusted QB rating is currently called (Opponent Adjusted QB Rating – or OA QB Rating – or OAQBR – but I’ll think of something better) See the results here

Most of the ratings were in line between my rankings and the QB rating system.  But, in my mind, OAQBR adjusted certain anomalies that gave too much credit to completion percentage (and, to a lesser degree, TD %).  Some examples:

Philip Rivers
After week 3 (not including Monday night’s game), Philip Rivers ranked 18th in the league with a rating of 86.1.  He had an OABQR of 6.36 – good for 2nd in the league.

What drove the difference?  Rivers is has averaged almost 9 yards per attempt – which gave him a good score in my ratings.  However, his 59% completions brought him down in the traditional ratings system.  (His TD % is not exceptional – this statistic is not part of my calculation).

With that said, Rivers has his team at 2-1 and averaging 24 points per game.  He is also leading the league with 991 yards passing. 

Essentially, Rivers is being penalized for taking shots down the field.  His production (in terms of yards) is higher than any other quarterback.

The system favors the dink/dunkers.

Brett Favre

This week's great finish aside, Brett Favre has done very little to help his team score points.  He's averaging just over 6 yards per attempt - which is not even in the top 20.  But, he's completing 70% of his passes - albeit for almost no yardage - so he has a qb rating of 94.5.  This makes him the 8th highest rated passer in football.


In my ratings, Favre ranks 26th with a -3.39.  That's not to say he sucks or hasn't played well.  He just hasn't done anything to merit a lofty rating at the game's most important position when all he's done is play pitch and catch with a couple of guys 5 yards away from him.




7 comments:

Thomas said...

Uh Oh, better be careful there with the Favre bashing - I know there are some Packer "fans" out there who care more for Favre than for the Packers and they will bad-mouth you from here till eternity!

I agree with you that the QB rating formula sucks and always has. And you're right that the NFL will never admit they're wrong about it. But I have to believe that there has got to be more to it than just Y/A, although I'll grant you that it probably is the major driver.

In my mind so much of what makes a good NFL QB is not really quantifiable, as much as we may try. The rating both you and the NFL are creating is really a measure of how good a passer a player is, not how good a QB he is.

But saying this brings to mind that one factor that can make a HUGE difference in both completion %-age and Y/A is not referenced in either rating - receiver drops. Granted it has not been tracked very well, if at all, until recently. But just imagine if you are a QB who has a starting receiver who can't catch with his hands (reference former Steeler Dwight "Hands Of" Stone).

I don't know if there would be any realistic way of incorporating this into the passer rating - you would be going into the realm of home-field scorekeeper that is the bane of baseball statistics, especially hit vs error. And do we really want to take football there?

matt said...

I understand that to debase stats you have to use extremes, but a QB going 10 for 40 is terrible. I am in a fantasy league that gives points for completions and I hate it, but just imagine how crazy it would be to throw 40 passes and only complete 10. That would be seven 3 and outs, and then a few drives where you end up throwing incompletions on 1st and 2nd only to convert on 3rd. Not to mention a QB who throws 30 incompletions is more than likely going to throw a few INTs.

George said...

One interesting fact to figure in is the "quality" of the receivers (though tough to measure). YAC could at least be a good indicator.

Nutria said...

What in the world does "opponent's average yards per attempt" have to do with a QB's rating?

IOW, a bad defense pulls down (and a good defense lifts up a QB's rating, and that's WRONG.

JG said...

"The system favors the dink/dunkers."

Right, as I noted in a comment I just left elsewhere, the system is so biased in favor of completions that a QB can increase his rating by completing passes that lose yards.

Say a QB loses 10 yards per completion, but goes 10 for 10 doing so.

His rating for losing 100 yards on 10 passes is 79.2.

That's almost the league average rating! Right now 14 QBs have ratings lower than that. Those 14 guys could all raise their ratings by going out this week and dedicating themselves to throwing nothing but dumpoffs that lose yards.

That shows how worthless this stat is.

As you say, it is highly biased in favor of the checkdown kings and against the long-throwers.

That's bad enough in the year-by-year rankings. But when you go back to the all-time rankings that count the era before the '79 rule changes that opened up the high-% passing attack and that slashed interception rates ... yowza! Brian Griese is ranked ahead of John Unitas as one example.

Packer Pete said...

Regarding Thomas' comment on receiver drops that negatively impact a QB's rating, what about the seven yard slant that the receiver carries another 40 yards to give the QB a huge boost? Often a well thrown ball gives the receiver an opportunity to run, so I think it all comes out in the wash.

I question as to why the metric has to be set with a maximum ceiling such as 158.3? Why should a 15 for 20 for 200 yards and 1 TD rate the same as 30 for 40 for 400 yards and 2 TDs? Why is the rating not cumulative?

Let's go back to a weighted, maximum ceiling system. I'd like to see third down conversions added in some manner, weighted to greater distance for conversion. I'd also like to see TD percentage replaced by percent of yardage in a TD drive. For instance, a TD drive of 80 yards in which the QB throws for 60 yards would be rated higher than a TD drive of 80 yards in which the QB threw for 20 yards. In either case, I have no interest in whether the QB actually threw a TD; I'm interested only that the team generated a TD. I want to measure the QB's contribution to that TD drive, not that he actually threw a TD.

I'll leave the details to you wizards. My slide rule can't keep up.

John John said...

I disagree with your premise that yards per attempt is more important than TDs. I know you're not the only one who believes this. What about situational football? For example, you're not taking into account that it is harder to complete passes the closer you get to the endzone because the field gets smaller and the defenses tighten up. Plus, as QB's are closer to the endzone, their yards per attempt will go down, but the difficulty of the pass has increased. There can be no perfect formula, epsecially on a small scale like per pass attempt or even per game, but at the end of the season, the QB with the most TD passes probably had the best season.

Lets look at the 2010 stats of 3 QB's who are considered the best in the league; Tom Brady, Peyton Manning, and Drew Brees. They finished 1, 2, and 3, respectively, in TD passes. Yet, they only rank 5th(Tom Brady), 16th(Drew Brees), and 20th(Peyton Manning) in yards per attempt.

I'm not sure if this was just an anomaly for this season, but you can't dispute my original premise that the most important(and toughest) throws, are the ones inside the redzone where 2 of the possible 3 outcomes of throwing a pass can have a huge impact and momentum swing on the game; INT or TD.

Theoretically speaking, there could be a situation where 2 QBs have equal stats between the 20's, but 1 guy has more TD passes which come from inside the 10. That would leave the guy, who got his team in the endzone more, with a lower yards per attempt.

Post a Comment