You may have read in my review of the current QB rating that I think the current QB rating system could use a few tweaks.

I have created a ranking that better reflects a quarterback's performance. Below is the background and explanation of my new rating - as well as the formula itself and an example from week 6.

Background and Explanation

I recently put together a regression model to determine the quarterback statistics that most drive the number of points an offense scores in any given game. The model showed that yards per attempt is by far the biggest driver of points scored.

At the end of the day, a quarterback’s job is to score points. The model also suggested that interception percentage plays a role, but not nearly as great as Y/A.

TD % also correlates, but, in my mind, TD % is what Brian Burke at AdvancedNFLStats.com would call an “intermediate outcome”. This means that a player who’s good at other things (e.g. moving the ball down the field) is likely to get more TD opportunities and, therefore, more TD passes.

Using the aforementioned regression model as a base, I have create a new rating system that I believe is a better indicator of a quarterback's success.

The rating takes into account four factors - with different weights being applied to each:

1. The quarterback's yards per attempt (adjusted for sack yardage)

2. The opponent's average yards per attempt in other games played in the current season (adjusted for sack yardage)

3. Interception percentage

4. Touchdown percentage

The Y/A statistics were given higher weights than Int %.

Because of the my statements above about TD %, this statistic was given only a nominal weight to reward exceptional games.

The biggest differences between this rating and the "traditional" rating are:

1. Completion % has been removed as a factor

2. Significantly different weights were given to the factors that were included

3. Quality of opponent was factored into the calculation

4. Sack yardage was included in the Y/A calculation

The Formula

* Y/A Differential - Y/A minus the opponent's average Y/A yielded (times a factor of 2.5 - as determined by the regression model) MINUS

* Int % TIMES 50 PLUS

* TD % TIMES 20

It can also be stated as:

((Y/A - OPP Avg Y/A) * 2.5) - (Int % * 50) + (TD % * 20)

Notes: The opponent's Y/A is bounded at a maximum of 7.2 and a minimum of 5.2. This is done because because a team's average Y/A yielded could be greatly affected by a single game (I call this the "Peyton manning" effect) - especially early in the season.

The Formula Applied

To explain the formula, let's use the examples of Tom Brady and Drew Brees (see table below) in week 6.

The calculation went as follows

Rating = ((Y/A - OPP Avg Y/A) * 2.5) - (Int % * 50) + (TD % * 20)

Brees = ((12.3 - 5.2) * 2.5) - (0 * 50) + (.133 * 20) = 20.41

Brady = ((10.4 - 6.9) * 2.5) - (0 * 50) + (.176 * 20) = 12.27

Brees superior rating was largely driven by the quality of the opponent. The Giants were the top ranked defense and actually had a Y/A against - when adjusted for sacks - of 3.4. As discussed earlier, this figure was bound at 5.2. This gave Brees a Y/A differential of 7.1.

Brady, on the other hand, was playing a team with an average Y/A yielded of 6.9. Combine his slightly lower Y/A with an opponent who was significantly inferior to the Giants, and you get a Y/A differential of 3.45 - about half of Brees' differential.

**Week 6 Stat Lines: Tom Brady vs. Drew Brees**

Name | Att. | Yds | TDs | TD % | Ints | Int % | Sacked | Sack Yards | Net Yards (MINUS Sack Yards) | Y/A (Adj For Sacks) |
---|---|---|---|---|---|---|---|---|---|---|

Drew Brees | 30 | 369 | 4 | 13.3% | 0 | 0% | 0 | 0 | 369 | 12.3 |

Tom Brady | 34 | 380 | 6 | 17.6% | 0 | 0% | 2 | 6 | 374 | 10.4 |

**Week 6 Stat Lines: Opponents (As of Week 6)**

Name | Opponent | OPP Avg Y/A Yielded | OPP Avg Y/A Yielded (Adj) |
---|---|---|---|

Drew Brees | NY Giants | 3.4 | 5.2 |

Tom Brady | Tennessee | 6.9 | 6.9 |

## 12 comments:

I like that you're using a regression model. This is a big improvement over the NFL passer rating. However, I think there are a couple of things worth investigating that might improve your rating. First, the number of points that an offense scores is also influenced by field position, which is a function of the performance of the defense and special teams v. the other team's offense. Therefore, it would be interesting to look at scoring per drive, correcting for starting field position. Second, you might want to use median yards gained on passing plays rather than the average. The median corrects for very long gains that can skew an average upwards and gives a better indication of the consistency of the passing offense. While you're at it you might want to look at consistency explicitly by calculating the standard deviation. Thanks for your great site and I hope you find these comments useful.

I'm confused. In the formula you state TD% is multiplied by 20, but in the application you multiply it by 50. Which is it?

I think you're wrong to weigh TD% less as I don't think it's an "intermediate outcome". It's much easer to throw for yards in the middle of the field than throw for TDs.

Otherwise I think you have a great formula. Much better than the trad QB rating system.

Why did you cap the opponents Y/A against instead of using Switzer's trimmed mean? Just take out the highest and lowest data points and then calculate an average.

Also, were the 50 and 20 found via regression? If not, why not? Could ANOVA easily find the sources of variation and thus come up with coefficients to use?

One possible weakness with your formula is that it's all percentage or per-attempt based. There's no reward for tossing 50-for-45 instead of 10-for-9, of 50 passes for 5 TDs rather than 10 for 1, or 450 net yards instead of 90. Those two QB scenariosl yield identical scores in your formula. Do you think a QB who completes 9 passes for 90 yards and 1 TD with 0 int should be rated the same as 1 who completes 45 passes for 450 yards and 5 TDs (assuming equal opponent YPA)? Worse, if the 450-yd passer had *one* int to his 5 TDs he'd be ranked *lower& than the 90-yd passer with 1 TD!?! And both would be eclipsed by a halfback throwing one 20yd pass for a touchdown on a razzle-dazzle play.

That's what your formula does.

I think you need to add some sort of weighting on total tosses or yards or TDs or NPY - (Opp Ave NPY) --- not just percentage or Y/A. You can use a rank transform to make the regression prettier :-). Production. Not percent.

Week 6 example Brady 374/34 is 11. How did you get 10.4?

Syed - the 50 was found using regression. The 20 was not. Here's why. TD passes lead directly to points - therefore, any model that includes TD passes is immediately skewed.

Syed, also, removing the high and the low would leave a very small sample. As the season has progressed, the bounding has become less of a factor.

I like this approach very much, this seems to me the right way to rate a quarterback.

I'm not sure I agree with the premise that good quaterbacking is about scoring the most points. Quarterbacks also have an influence on their team's ability to prevent points from being scored.

Quarterbacks who throw more interceptions give opponents better field position from which to score on the QB's defense. Quarterbacks that score with long passes but do not sustain drives may "tire out" their defense (though I'm not sure if this effect is substantiated by evidence).

I suggest you choose variables that best represent the independent tasks asked of a quarterback: to score points while putting his defense at minimum risk. Maybe adjust "points scored" to a risk adjusted figure, like points scored per interception thrown. This risk adjustment captures the role of intercept % and TD % I think. Then regress Y/A and other figures on this. Just a suggestion.

Great work!

I agree that scoring points is the QB's ultimate job. I would remove the TD% and replace it with a number that reflects the percentage of yards a QB contributes to a scoring drive. For example, on an 80 yard TD drive, a QB who throws for 60 yards would be rewarded with a higher rating that a QB who throws for only 20 yards in an 80-yard TD drive. In neither case am I interested as to whether the QB actually threw a TD pass, only that the team scored a TD. This rating could be further weighted by the distance of the drive as well as third down conversions by passing.

I think third down conversions are very important in QB play and they are absent in the NFL rating. That's what keeps drives alive.

In the mid 90s when Favre won his 3 consecutive MVP awards, he threw in the mid to upper 30 TDs each season. In the season that he tossed 39 TDs, I think about 13 were from within 3 yards of the end zone. Those short TDs shouldn't be weighted so heavily, no more so than a 12 yard pass on 3rd and 10 that continued the drive that ended in a 1 yard TD pass. That 12 yard completion to continue the drive was the more important throw. That's why I'd like to see the QB rating include a weighted percentage of contribution to a scoring drive, and not just a reward for throwing a short TD.

Second comment is right about the formula being inconsistent with the example. Also, if you run the numbers, you will see that the stated results for Brees and Brady are both wrong. Pretty sloppy, when you are trying to fundamentally trying to explain what you are doing...

Jim - you're right. I can't believe I did that. I wrote this up awhile back and I never went back to it. Anyway, I've corrected it.

Thanks for checking out the site.

Since you are placing a great weight on Y/A, what are you doing to account for YAC? Just like people who point out down and distance factors.

Which is indicative of a QB doing a better job and should, by this formula, be considered the better passer (by rating):

1) A QB throwing a 2-yard dumpoff to his RB, who runs about 80 yards with it.

2) A QB throwing a 40-yard TD to a WR, caught in the endzone.

Both would have equal TD%, suggesting that the points resulting it were equally due to the equal quality of the QB play.

The QB in scenario 1 would have a Y/A twice that of the QB in scenario 2. He'll get that advantage weighted into many other mediocre plays to bring up the average for many lackluster ones to follow.

I'm not saying I have a better answer, and it's easier to fault someone else's system than come up with one's own. So don't take this as overt criticism. But while YAC often take into account a QB hitting a receiver in stride as opposed to forcing him to break stride or even stop, YAC also takes into account great plays by ballcarriers that have absolutely nothing to do with the efficiency or effectiveness of the QB.

Maybe cap YAC so they can't exceed the distance of the pass itself. It would negate some of an 80-yard TD that traveled 30 yards in the air, but it would negate a lot MORE of a dumpoff pass that someone runs 60-80 yards with. That's more like a handoff given to a RB when he's already gotten past the DL than a true pass play.

They both put the same number on the scoreboard, and points are points, but we all know they aren't remotely equivalent plays in terms of QB skill and contribution.

One other thing. Though this is "passer" rating rather than "QB" rating, and I've always been a pocket-passer fan myself, the reality is that QB's who make plays and score points with their feet are penalized by this system. Great as he is, Peyton Manning throwing a 1-yard TD pass to an uncovered, open receiver is not nearly as impressive of a play as a far lesser passer like Vick getting a 50-yard TD with his feet. They put the same amount of points on the board, one due to a 1-yard play and one due to a 50-yard play. Yet Vick is penalized here, as it's a drive where passer rating suggests he didn't score a TD, as though he handed the ball off to someone else (who then fumbled the ball away, to make the statistic fail even worse). So Vick's 50 yard TD run is weighted the same as him handing off to his RB who fumbles the ball and the defense scores points. Statistically, they are the same play in this model.

So while it may be purely a "passer" rating, but it clearly favors certain "types" of passers over others in the same way that the current passer rating system greatly favors WCO types, or those like a Chad Pennington who completed like 70% of his passes by gutlessly dumping the ball off to checkdeown receivers 85% of the time.

The point of any passer rating, at its core, should be to determine answer to the question: "Who is the best QB?"

By negating (and indirectly, penalizing) the ability of QB's who make plays with their feet, it suggests that points scored off those plays are lesser in some way. Also dumpoff TD passes with 80 YAC to a 40 yard TD pass all in the air, it suggest that the QB throwing the dumpoff did a far more impressive job of passing, and of QB'ing in general.

If a QB at on the opponent's 9 yard line gets blitzed heavily and throws two incomplete passes to wide open receivers, then the defense commits an inadvertant facemask penalty after a 1 yard pass on 3rd down gives them an automatic first, THEN the QB throws a TD pass, he's rewarded greatly even though his own play should have led to a FG rather than a TD. The QB who just runs it in himself on the initial first down is given no credit whatsoever for putting points on the board.

## Post a Comment