Case Study: Romo vs Favre (Through Week 7)

My Rankings vs. "Traditional" Ratings:
In my methodology, I point out the differences between my ratings and the "traditions" passer ratings.  Specifically, the differences can be summed up as:

1. Completion % has been removed as an independent factor
2. Significantly different weights assigned to the factors that were included
3. Quality of opponent was factored into the calculation
4. Sacks are measured.

 (A quick note on completion percentage - since I've gotten a good amount of feedback that suggest I shouldn't ignore it altogether.  The formula for Yards per Attempt (ignoring sack yardage for now) is Yards/Attempts.  This can be re-written as Yards/ Completion X Completion/Attempts.  Completion/Attempts is completion percentage.  Since it is inherent in Y/A, it has not actually been removed from the model.  The "traditional" rankings give it too much weight by counting completion percentage and Y/A)

In the ratings table below, I have highlighted in yellow five players where there is some deviation between my rankings and the traditional rankings.  Below I will dig into why two of them (Tony Romo and Brett Favre) differ from the "traditional" ratings. 

First, the Methodologies:

Traditional:

The formula for the "traditional" rating is:

a = ((Comp/Att) * 100) -30) / 20  [Completion percentage]

b = ((TDs/Att) * 100) / 5  [Touchdown Pass pct]

c = (9.5 – (Int/Att) * 100))/4  [Interception pct]

d = ((Yards/Att) – 3) / 4  [Yards per attempt]

The final formula is (a + b + c + d)/.06


My Rating

•  Y/A Differential - Y/A minus the opponent’s average Y/A yielded (times a factor of 2.5 - as determined by the regression model) MINUS
•  Int % TIMES 50 PLUS
•  TD % TIMES 20

It can also be stated as:
((Y/A - OPP Avg Y/A) * 2.5) - (Int % * 50) + (TD % * 20)

Notes: The opponent’s Y/A is bounded at a maximum of 7.2 and a minimum of 5.2. This is done because a team’s average Y/A yielded could be greatly affected by a single game (I call this the “Peyton Manning” effect) — especially early in the season.


Let's compare Tony Romo and Brett Favre.

Tony Romo:  Traditional Rating: 94.7 (10th)  My Rating: 4.54 (3rd)
Brett Favre:  Traditional Rating: 102.2 (6th)  My Rating: 0.52 (15th)


Here are the statistics needed to calculate either rating (except for the opposition adjustment in my ratings).

Player	Comp %	Y/A (Unadj for Sacks)	TD %	INT %	Sacks	Sack Yards	Y/A Adj For Sacks
Brett Favre	69%	7.3	5.2%	1.3%	18	127	6.3
Tony Romo	60%	8.5	4.6%	2.1%	11	56	7.8

The "traditional" rating would break down as follows (based on application of the above formula):

Value Towards Rating	Favre	Romo	Difference (Favre - Romo)
Comp %	32.5	25.3	7.2
TD %	17.5	15.5	2.0
Int %	34.1	31.0	3.1
Y/A	18.1	23.0	(4.9)
Rating	102.2	94.7	7.5

My rating would break down as follows (the actual calculation needs to take place on a weekly basis.  Below is an aggregation of results):

Value Towards Rating	Favre	Romo	Difference (Favre - Romo)
Y/A Differential (Adj For Sacks)	0.12	4.6	(4.5)
TD %	1.0	0.9	0.1
Int %	(0.7)	(1.0)	0.4
Rating	0.52	4.5	(4.0)

In the "traditional" ratings (I'm going to call them TDR from now on), you can see that Favre wins by 7.5 points - a fairly significant amount. The two are close to even in TD and INT - Favre comes out slightly ahead in each.  However, Romo comes out significantly ahead in Y/A and, more importantly, Favre comes out well ahead in completion percentage.

I did the calculation and, if Romo had completed 70% of his passes (like Favre) his rating would be 102.8 - almost identical to Favre.  But, they have the same number of yards and Romo has 35 fewer attempts.  Favre is being rewarded for being conservative - throwing mostly checkdown passes (especially in the early games) with an average completion going for just over 10 yards.


As Peter King said in hi 10/16 article: "The best single statistical barometer of quarterback proficiency is probably yards-per-attempt because it measures how far downfield a quarterback is getting the ball on his average pass-drop."
 

Further, Favre's high completion percentage is already factored into his Y/A.  Again, Y/A = Y/Completion X Comp %.  The fact that Romo manages to have a higher Y/A with a -10% differential in completion percentage indicates that completion percentage is a meaningless statistic when taken by itself. 

If you look at my ratings, Romo wins big in the Y/A.  There are a couple of reasons for that.

1. Romo's Y/A (8.5) (unadjusted for sacks) is significantly higher than Favre's (7.3). 
2. Favre has been sacked seven more times than Romo (18 vs 11) for 71 more sack yards.  This brings the Y/A (adjusted for sacks) to 7.8 for Romo and 6.3 for Favre.  A differential of 1.5 yards (and remember, Favre has a better comp %).  This cost Favre 2.5 points in the ratings.
   
3. The opposition factor had a negigible effect.  Favre's rating dropped by 0.66 points while Romo's dropped by 0.62 points.

The two are fairly even in INT and TDs.  I believe TDR suffers from, among other things, the over-reliance on completion percentage.  And I believe my methodology corrects for that.