Ready for some Football

There are three types of teams (vaguely) in the NFL. There are those that overchieve, those that underachieve and of course, those that just achieve.

In Statistics, we like to predict the outcomes of events. I've talked earlier in the blog about win rates and how to vaguely calculate them. Over the past 56 seasons, that formula has never been off by more than one game. So i will say, it is a pretty good estimator. So I'll go over the three types of teams below.

The Achievers : These are most the teams
Team                      W            L

Baltimore	3	1
Tennessee	3	1
Houston	3	1
Buffalo	3	1
New England	3	1
Washington	3	1
New Orleans	3	1
San Francisco	3	1
Cincinnati	2	2
NY Jets	2	2
Oakland	2	2
Dallas	2	2
Chicago	2	2
Pittsburgh	2	2
Atlanta	2	2
Denver	1	3
Seattle	1	3
Jacksonville	1	3
St. Louis	0	4

These teams do things like most teams. They either score a lot, and prevent a lot of points, or they just allow a lot of points. The St. Louis Rams have that problem. The problem is , the Rams just really cant score, and give up a ton of points. They are tied for the second most amount of points given up, while their offense has only scored 46 points. That is just over 11 point average per game. Now.. i wouldnt say they dont have hope, but what they need is offense. Sure, offense and defense are highly correlated (meaning if your offense isnt of the field, your defense is out there defending it), BUT they give up an acceptable amount of points to win a few more games. They need more offense. I would suggest adding a wide out. The Rams only have 748 passing yards putting them in 26th ; but have 370 running yards getting them 23rd. All other teams, well only if I had the time!

The Over-Achievers : Teams that win, despite us not predicting that.
Team                    Wins         Loss

Detroit	4	0
Green Bay	4	0
NY Giants	3	1
Tampa Bay	3	1
San Diego	3	1
Cleveland	2	2
Kansas City	1	3

There are an array of teams here. Detriot - Not many people will complain with this one, since everyone things they are overrated and beat some "easy teams" What they have to worry about... not much actually.They have the second best offense tied with New England and the 8th best Defense. It is their defense that has expected them to lose a game. I mean, every team has to lose (well minus the 1974 Miami Dolphins). The Packers  are easy to pick apart. They are terrible at defense and there isnt much hiding that, ranked 18th on points allowed.
Kansas City - is just terrible, and got lucky on their one win against the vikings.

The Under-Achievers : Teams that win, despite us not predicting that.

Philadelphia	1	3
Arizona	1	3
Carolina	1	3
Minnesota	0	4
Miami	0	4
Indianapolis	0	4

All of these teams should be  one game higher. The Eagles have given up as many points as they have scored; and that not shockingly in their problem. Sure Vick could throw for more points and they could win games. But when your defence blows the lead they had on sunday, there is only so much you can expect from your offnese. That is what the Eagles need, an Defense. Ranks D- 23 Offense - 10. Give up less points and expect more wins. Think of this in baseball as having heavy bats, but no pitching, you'll pull a red soxs blow the season quickly.

Updates to come

America | Where the weak surprise

In America, the poor and the underdog are always the fan favourite. This really only happens in America. As the Atlanta Braves and Red Sox continue to ruin their play-off hopes it got me thinking about how American (and ESPN for that matter) it is.

In Europe, they have mixed systems. The Barclay's Premier League, La Liga and the Bundesliga League all work on a "best" record system. Meaning, the team that ends the season with the most amount of points wins the league. Sure, it becomes less exciting when teams like Manchester United sweep the league and just crush their way to the top. But that is the World's way. The team that fought all year, shouldn't be suppressed to have to prove they are the best in some random series at the end of the year.

The World and It's Argument :
Formula 1, European Football, Golf and Rugby all have this style of awarding a winner. The team at the end of the season with the most points wins the League. While I was in England, I would discuss with the English about this. They always had solid arguments for this method. Think about if Aaron Rodgers got a concussion the last week of the season, and lets say the team had the most amount of points. Is it fair that the packers, who clearly showed they were the best should not be awarded the Super Bowl. The argument works really well in baseball where the sample size of games in much larger (162). The Phillies who are likely to go to the World Series should at least be given a direct path there. Instead, their very successful season will be tested unnecessarily against some random events. Randomness could kill a season, where the Phillies who have less likely Randomly got the most amount of points.

But this is America, and we love hope:
That is right, this is America, and like our founding, we want the small dog to overcome the evil empire [The British]. We want the Phillies to be upset by the struggling Braves, or to have the Tamp Bay Rays win on the final day of the season and make their way to the World Series where they win for the small town that barely has actual fans. This is something greater than sports!

Wild Card Sucess:
MLB : 3-World Series Won 8-World Series Appearances [Since 1981]
NFL :  5 - Super Bowls Won 9-Super Bowl Appearances [Since 1970]

The NHL and the NBA (assuming it isnt locked out forever) do not have wildcards per say. Instead, 8 teams from the league make the playoffs. Then they continue to play 7 game series for 4 series, making the playoffs boring long. 8/15 teams make the playoffs in both leagues, thats more than 50% of the league. It almost makes one wonder why a team would try for so long. A good example of this is the Washington Capitals. They get a lot of heat for being in the southeastern division, one of the weaker divisions, but continue to produce 100+ point seasons, quite the impressive feat. However, they haven't been to a Stanley Cup in years, in fact its been since 1998., yet  they finished first in the league 4 years in a row. Shouldn't they be rewarded for this?

What Does Australia have anything to do with this?
 The Australians may be up to something here. A friend of mine brought this up to me when were discussing this exact topic. In Australian, the better teams do have an advantage in the playoff structure. The first round works a lot like the American Structure. 1vs8 2vs7 3vs6 4vs5 : Standard. But what is interesting is the second round. All winning teams advance, but the two teams with the higher seed get a bye week, and the two teams with the lower seed of the losing side are eliminated. So for example, lets say the winners are 8,2,6,4 thus the losers are 1,7,3,5. So 2,4 would get the bye week, while 1 and 3 would get another chance to win. They are being rewarded somewhat for their season long success, but still have to win some important games.

Baseball :
If  baseball does add teams to its post season play, i would like to see them try this system. It would be unique. I would like to see good teams be rewarded for having a season long of success, instead of losing to a fluke home-run, or having a pitcher injured for a game.

Expected Win Ratio: New Statistic

Baseball is a game of numbers, but aren't most sports. Bill James invented a metric called the Pythagorean Expectation. It derives its explanation from the Pythagorean Theorem. What the equation says is, if we take how many runs a team scores, square it, then divide that number by how many runs a team scored squared plus how many runs a team gave up (squared) we will get that team's winning (or losing) percentage.
 So all in all it looks a little like this [Runs for]^2/([Runs for]^2+[Runs Against]^2).
Or more neatly

Now sabermetricans have perfected the equation, and realized that actual exponent should be around 1.86. There are many sabermetric articles about how well this is at calibrating the win expectation of a baseball team. Some teams are really good at beating the odds, meaning they win more games than their expected wins. In 2011,The San Francisco Gaints are the best team at this, they have won 6 more games than their expected winning percentage would say. Now all of this has been examined, so what does this really mean, why I am blogging about this. That is because, what if we borrowed this idea for other sports. Could we predict the winning percentage of your favorite football team?

Using a similar equation, but with Points For and Points against, allow us to start :

[Points for]^2/ ([Points For]^2 + [Points against]^2) .
Now if we optimize the Standard Error equation, being [Expected Winning - Actual winning]^2 . Our goal is to make the sum of those numbers the lowest, by adjusting the Expected winning number. And of course, the only thing that isnt constant is the exponent value. So if we optimize that, we should aquire the value of 2.50 .

So using the last 51 years of NFL data, a teams overall record can be determined by using the above equation.

This is a plot of all the data. As we would expect, it is very close to a straight line. That means that, our Estimated winning percentage is very close (in error) to the Actual winning percentage. There are a few minor flaws. The projection will rarely give a 0-16 season, but teams actually do that. But as you can see, the projection only predicted they would win 10% of their games, so still a small margin.

 So what, why does this matter you may ask.  How does this equation effect anything. It actually means a lot As I continue to examine predictors and estimates, we now can value a teams outcome. If we can predict (with in a certain error) a team's Points For, and a team's Points Against, we could predict how many games the team is going to win that season.

Now if we can use points to get wins, we could other stats to get points. How many points is a particular position responsible for, so if we add Quarterback B, who is expected to throw X amount of yards, we can contribute his X yards to Y wins. Almost in a linear fashion. So as I continue to expand the Football statistics I will refer to this expected winning ratio.

Free Market Failures : Sort of

I, like most humans, get bored from time to time. I get bored writing columns about sports, I get bored watching particular television shows or reading news columns. Then every once in a while, a little spark stimulates my mind and that boredom goes away. For me, that spark was Top Gear. As I watched these nutjobs guys ride across Africa in their £1500 cars, I honestly thought, if this can be done with higher taxes please raise mine. Then it got me thinking, does the free market fail us. Please note future news broadcasters, this is my political waiver. This has nothing to honestly do with politics or whether or not a free market is good or not. That is up for colleges and or the New York Times to debate. I will simply express my opinions on things that are free markets in themselves, and why they have failed us.

Top Gear:
Top Gear is produced by the British Broadcasting Corporation, as many know it as the BBC. The BBC is funded by this fee applied to all British residence with a television. There are different rates such as black and white or colour.Now whether you watch the television or not, you still have to pay the annual tax. So this funds shows like Top Gear, the Inbetweeners and the Weakest Link. The Americans have recently copied this programme with one on the world war II History channel. Now the budget for the British show must be massive, because they buy all these cars and then trash them. The show is such a hit that when they travel the world they are often greeted by some political ambassador. The show is now on for its 9th year and 17th (series, the British ones are smaller than our regular season).

So what does this great show have anything to do with Free Markets. Well, the BBC is far from a free market. Shows are given money from the British tax payers and are rarely dependent on rankings. The advertisement sure adds some, but the BBC still funds the show. It was also born as a concept not in a Free Market. Shows are allowed to fail for a lot longer on the BBC before they are taken off.  In America's Free Market television we have the reverse effect! We have shows on that cost very little and are shoved down our throat.

Why you should be angry
Of the top 10 shows in America by rating, only two are written, and its actually the same show twice!(NCIS). American Idol, The Voice, and the NFL all come in front of Modern Family, How I Met Your Mother and The Office. American's are watching the cheapest shows! How is this possible? Well lets assume that a tv channel is a Business (unlike the BBC). In order to raise profits you need to do two things, 1 get more customers(which is somewhat already capped) or lower costs. The TV companies have lowered cost by getting rid of writers (or in our Top Gear case, giving a lot of money to blow things up). Even MTV has bought into this. This is truly why you should be angry. The Jersey Shore has more reasons (the fifth one is finishing production) than Arrested Development! Sure what my blog is saying is that if we had the BBC, Arrested Development would still be around. But the sad part is, the Jersey Shore is unlikely to go away soon. People watch a bunch of drunk idiots get in fights. How much does that cost? It costs one really good film editor and few lawyers to fight off potential lawsuits. Anyone with a home camcorder could do this [disclaimer:: do not attempt]. Other than that, it costs MTV nothing! Think about all the money they are saving from props, to filming time to actors to writers and all the things that go into quality television. The advertising  pushes us to shows like this, the TV companies like to save money. The BBC on the other hand, would rather send a bunch of blokes to America to race cars, who cares how much it cost!

Sports:
I read a great article over at grantland (linked here) by Malcolmn Gladwell that said sports are not a Business.Despite me agreeing, they do have a market and it is free. This is another free market that is failing us.

Baseball :  I don't think I say this enough on this blog. I hate the Yankees, here, I'll say it again, I hate the Yankees.However, the Yankee's do not hate their Free Market. The Yankees have been to the World Series 40 times, more than any other team in history. They have won 26 27 World Series, again more than any other team. The Yankees have been to double the amount of World Series than the second place team (the LA Dodgers) and have won nearly there times more than the second place team (St.Louis Cardinals).They have spent more money than any other team.So sure we cant blame them for spending all that cash because they do get wins. Now not too long ago, the MLB decided to have some profit sharing, where teams like the Yankees, the Red Soxs and the Phillies would pay a lot of money to beat up on the poor teams. Think of it this way, its like building your neighbour a house, to only watch it burn to the ground so your house is the nicest looking on the block. Baseball is partially a free market. Teams share profits, and there is a draft system so that some teams can pick good players for sucking for so long. However, as Ron Swanson once said, Capitalism separates the rich from the stupid. But in sports, we dont want that. To me, and I believe most sports fan, the fun of going to a game is that any team can win! On a day to day basis, over 162 games, sure any team can win any day. But not every team really has the chance of making the play-offs. There has been a recent argument going around sports, that a team has to suck (and be poor) for 10 years in order to make a run at a championship for two years. Then of course their players will be bought off of Free Agency. That is probably where free markets fail! A team is given a lot of sucky players for losing its good player because it wasnt rich enough to buy the good player. And a poor team, cant risk signing a long term contract because if the player is a bust, they lost a good portion of money. Sure sabermetrics helped the A's ... for a while. Since the book Moneyball(Micheal Lewis 2003) came out, the A's have only made the playoffs once, losing to the Twins in a straight sweep.That is the thing about Free Markets, once your intellectual intelligence leaves you, the teams that can afford it, buy it. Before starting the season, without even playing a game, the Yankees had a 11/2 odds of winning the World Series. When sports are predictable, the season is not as fun.

Gaps in statistics : NFL

People always tell us, ::insert sport:: is a team sport. My biggest arguement is, why are the statistics individual, and why do you draft a player for fantasy and not the team. A team sure has a "portion" of sucess, but come on, we are humans we can calculate that addition and remove it! However, as I continue my quest to give every player a particular metric, I get stuck at certain parts. I have listed a few of them below.

Missed Field Goals:
A touchdown is simple, a team scores 7 points and that is the end of it. If they fail, no one is really at fault. The better question is, who is to blame for a missed field goal. Is it the kicker? Well sure if the ball is on the goal line and the kick is missed than naturally it is the kickers fault. This of course, assuming the ball is snapped properly the wind is normal and the weather is without snowfall/rain. So given normal conditions we expect an NFL kicker to kick from a range. A dome, well throw out all the previous remarks because we assume they will be able to make a kick from any reasonable distance. What is reasonable though? The Average Longest kick is 53 yards. And the Average kick distance is 36.3 yards. Now lets set 36.3 yards as a threshold. We would assume that an NFL kicker would be able to score from that range. Now lets assume the ball is at the 37 yard line, and the coach calls for a field goal. The kicker shanks the ball left. Here in lies the question, whose fault is it? In sports we always blame the person at hand. Why dont we blame the Quarterback/offense. Maybe those few extra yards would have allowed the kicker to get a better kick, more breathing space. Is it the coaches fault, he should have known that the kicker was in his average range. That nothing is promised, and maybe they should have known that.

my thoughts : Now given all of the nfl drive/statistics, it is shown that the expected score depends on yards to go. 6.5 - (.081235)*(Yardstogo) = Expected Points. Now we want to know when it is better to go for it rather than to kick a field goal. So we set our expected points to 3 and reverse solve. This leaves us with 46 yards. An average kicker can make a 46 yard field goal. Thus it should be a quarerbacks second goal (obviously the first to score a touchdown) to get his team safely to the 46 yard line. Now with more research we can calculate field conditions such as rain, wind and snow. There is most likely an Expected Points for any weather condition. So I propose to use the method above until a weather additive model is created. On the otherhand, a Quarterback should be given the portion he is responsible for divided by the total * 3. REGARDLESS of if the kicker actually makes the kick (assuming it is withing the 46 yardline). This will eliminate any errors caused by the kicker.

Sacks or Interceptions:
When I always watch a game at my local watering hole, i always hear "WHY DID YOU GET SACKED". That is normally in the direction of the quarterback. I take sacks differently. I applaud the sack! What the quarterback did was realize that a fumble or interception is worse than a sack. Sure a thrown away ball is better than a sack, but without an intended receiver it becomes tricky. Possession is like gold in football. It is the ultimate prize, and losing it turns the game quickly. Not only did you go from being able to score, you now have to prevent the other team from scoring. As in Moneyball (Micheal Lewis 2003), outs came at a premium and so does possession in football. It is ok to lose a down as long as you keep possession. Sacks should be treated as a lost of yards rather than a "Sack". The sack is really the offensive line's fault. This is two fold in my "I haven't played a game of football in my life" mind. This is because not only did the offensive line not provide him enough time (even if it is a long time) they didnt give the Quarterback another option. They didnt give the quarterback a running hole, or even break the tackles so that he could throw it away. The sack should rarely be considered the fault of the Quarterback. The only time it should be considered is if a better outcome (being a throw away) is possible.

New Statistic : Quarterback Value

The Quarterback is often considered the most valuable player in the NFL. In fact, a Quarterback has won 65% of the Most Valuable Player awards, more than any other position. However, is there really a value to a Quarterback? He is often credited with Wins or Points or various other metrics that really aren’t contributed to him. There are several statistics (none that I fully disagree with), that measure the quality of a Quarterback



Quarterback Rating (Passer Rating)

This statistic was created in 1971 by Don Smith, and is often quoted by a lot of football enthusiast. My favorite part is, most people don’t know how to calculate it. If you know how to calculate the passer rating than you surely are in the minority.

ATT = Number of passing attempts

COMP = Number of completions

YARDS = Passing yards

TD = Touchdown passes

INT = Interceptions

A perfect passer rating in the NFL is 158.3. the highest ever recorded within a season is 121.4 by Peyton Manning.



Now although I normally hate bad statistics, I’m willing to keep this around until something better comes along.

Pseudo-sabermetricans came a long and created the EPA (or expected points added). There isn’t a full definition of how this is created, or is it really explained. But if I ever figure it out I’ll be sure to comment on it.

Quarterback Value (QV)

This is a statistic I roughly created the other day. However, the data doesn’t present its self to show you a cool neat graph, instead it will be a hypothetical statistic until I get some more data or find a source of this information. There are three sections of this statistic. 1. Gained Points, and credits towards those 2.Expected Points loss. 3. Expected Score of Opponent



1. Gained Points
GP = 6 * [Passing Yards + Running yards that he ran - PTyards / Total Yards Gained]*[1 if TD] + 3*[% of Passing Yards]*[1 if within the NFL average Field Goal range for that stadium] + 6*[% of Running Yards in which he ran]*[1 if TD]

There are a few things to note here, I’ll work the equation from left to right. The 6 points is for the touchdown, if a team scores a touchdown, they get six, and it should be independent of the extra point attempt. The quarterback should be awarded the percentage of that in which he is solely responsible for, the passing yards. He should be given Points for driving down the field, and having a running back run the ball in. It was part of the quarterback that got the team down the field with his % of passing yards on that drive. The same applies to the third part of this equation, the Running Yards in which he ran. Some Quarterbacks are just at good at running the ball as throwing it, and they should be rewarded for that (well minus the injury risk). So they get the percent they helped the team make it down the field.

Now PT yards are penalty yards, a Quarterback should be penalized for actually committing a penalty. If he is bailed out by a running back, he should be awarded more than his share. So if he is given a penalty for Intentional Grounding, False Start or Delay of Game, than that should be subtracted from his total yards, because he put his team is danger, and forced them to have a harder time to score.

2. Expected Points Loss [EPL]

EPL = 6*[The times the team has scored 6 points / All the Times they were at this situation] + 3[The times the team has scored 3 / All the times they were at this situation] + 0 [ The times they didn’t score / all the times they were at this situation].

This is the true addition to this metric. The Expected Points Loss accounts for turnovers. I was watching a pre-season game when I thought, not all Interceptions and fumbles are created equal. There are two parts to a turnover, this portion of the statistic deals with the loss the quarterback inflicted on his team. This really works off of odds. Almost a “What would have happened if the Quarterback didn’t turn over the ball” Think about this as being 1st and goal, and the Quarterback throws an interception. Everyone this surely they would have scored had he not done that.

This is clearly a declining statistic. The farther you are away from your end zone, the less likely you are going to score, thus your team didn’t really lose that much from your turn over besides field position which in accounted for by the other half of this statistic.

3. Expected Score of Opponent [ESO]

ESO = 6*[The Probability of the other team scoring a touchdown]+3*[The Probability of the other team scoring a field goal]+6[If returned for touchdown].

Mistakes have consequences, and some mistakes are worse than others. The quarterback who fumbles the ball on his goal line, hands the ball over in the red zone, which the statistics are higher than if say he fumbled the ball at the opponents 1 yard line, where they would have to travel the length of the field to score. If the fumble or interception leads to a direct score, the quarterback should be penalized the full portion of the score.

Examples : (And Fake Numbers) |

This Game is where I will get some of the statistics
http://espn.go.com/nfl/boxscore?gameId=310206009

A rough estimate for the NFL’s expected points per yard is a negative slope of
Expected Points = 6.5 - .08125*Yards.

So allow us to begin.
The first scoring / turnover drive was with the Packers
Rodgers threw for 79% of the yards, so his + value will be 4.78
Ben Roethlisberger on the other hand threw an interception that was returned for a touchdown, so his – value would be -6
On his next Drive Roethlisberger drove the field either by running or passing and was involved for 100% of the drive, giving him +3 for the field goal. Updated score -3.
Unfortunately, he threw an interception, giving the packers the ball with 53 yards to go, costing him an opponent score of -2.45 bringing him back down to -5.45
Rodgers’s next drive he threw for 41/53 of the yards, gaining him 4.61 and moving his total points to 9.42
Roethlisberger then threw for 90% and earned a 5.37+ bringing him to .07
Roethlisberger then ran for 50% of the drive, giving him 3 points, and a total +2.93
Rodgers threw/ran for 81% of the next drive to get him 4.90 bringing him to 14.32
Roethlisberger then ran for 100% of the drive, giving him 6 points, and a total +8.93
Rodgers threw/ran for 80% of the next drive to get him 2.40 bringing him to +16.72

Thus by the end of the Game, Roethlisberger’s score was +8.93 and Rodgers was +16.72 

The previous week, Mark Sanchez scored a +6.6

The expected opponents points was only used once because he ball was only turned over once, but you can see that if you are Mark Sanchez or Carson Palmer, your score would be much. Lower.

Broken Statistics : The Yards

I am in the works of creating some new edge statistics that take a look at a Quarterback's rating on the field. Sure the offensive line has a large part to do with the Quarterback's performance, along with the running back. However, the Quarterback often receives more credit than he deserves. That brings me to the broken statistic, Yards.

A Quarterback often tosses guns the ball to a wide receiver who somehow catches the ball and runs down the field. Now the Quarterback is given the Yards statistic if the player catches the ball and continues to run. The statistic is really inaccurate. The Quarterback is not helping the wide receiver run down the field, but is given the credit as though he did. Now there are two sides to this argument. One is that the Quarterback looks for the best option when passing the ball. He looks for the best receiver that will cash him out the most amount of yards, and thus selects the most profitable receiver. Think about it this way, a Quarterback selects a stock (hopefully not in this economy) and hopes that it pays off more money(yards) than he paid for it(yards thrown). The findings are actually really cool. You can predict how many yards a quarter will receive by his receivers alone. With that, you can can see which Quarterback is the most profitable. Now the statistics are somewhat granulated, because there is no way to find out which yards a receiver caught with a particular Quarterback, but one can do this on the team level. A graph below demonstrates the teams Profit Rates.

Profit Rate = (Total Yards -Yards After Catch)/(Total Yards).

So what does this Patchstat - Profit Rate mean. What it calculates is really two things. The first is how many yards is the Quarterback actually throwing the ball, the raw value of yards. The second is how much is he being rewarded for choosing that particular receiver. A higher profit rate means two things, #1 the quarterback is smart and choosing the receiver that will get him a lot more yards, #2 is that the receivers are helping the Quarterback get more yards. Since a lot of what the Quarterback is doing depends on the receiver, you cant put full responsibility on the Quarterback, but you can make an assessment of how he chooses his receivers. Dallas had the lowest profit rate! Meaning the Cowboys do not earn additional yards after the throw. And New England had the highest, see the chart below.

All in All, the Quarterback gets a lot of "free" yards, and this will hopefully give us a better understanding of a Quarterbacks's ability.

And just for the Record, Michael Vick has a profit rate of 54%, which means he doesn't always pick the most profitable receiver.

Team	Rate
DAL	46%
STL	47%
OAK	47%
DET	48%
SF	50%
CAR	51%
KC	51%
JAC	51%
SD	52%
GB	52%
WSH	53%
HOU	54%
BUF	54%
PHI	54%
MIN	54%
CHI	54%
TB	54%
BAL	55%
CLE	58%
NO	58%
CIN	58%
DEN	59%
IND	59%
NYJ	60%
NYG	60%
PIT	61%
MIA	61%
SEA	62%
ATL	64%
TEN	67%
ARI	68%
NE	68%