As a history major during my university days, I spent plenty of years buried in such minute details as the Byzantine Empire's education system and the social importance of the invention of the printing press. One of the things that's always interested me about the past is that seemingly minor changes can produce far-reaching effects; this is where you get ideas like the butterfly effect, and it's an area that's seen plenty of impressive work (including that by Harry Turtledove, who, among other things, wrote an excellent and very historically-plausible 12-volume series based solely on the premise that the Union Army never found a Confederate order at a critical moment of the Civil War). You can apply those principles to any sporting event as well, but they're particularly useful in football, where many minor changes to any particular play can result in significantly different outcomes of a game. However, it definitely helps if you have some evidence to support your case; just saying "What if they ran up the middle instead of throwing that interception?" doesn't work entirely, as there's a chance that a run could have caused a fumble, a tackle for a loss or some other equally unfortunate outcome. That's where Rob Pettapiece comes in; the editor of The CIS Blog is also a brilliant statistician whose work should be well-known to long-time 55-Yard Line readers from his RPI/SRS calculations and third-down studies, and he's concluded after an extensive analysis of third-down gambles in the playoffs that Saskatchewan likely could have won the 2010 Grey Cup simply by making different decisions on third down.
In an e-mail interview earlier this month about Saskatchewan's decisions in that Grey Cup game (play-by-play data available here), Pettapiece told me there was one decision that was particularly bad. The one that stood out to him was in the third quarter, where Saskatchewan faced third-and-eight on the Alouettes' 38-yard line with 12:42 left and opted to punt. According to Pettapiece's extensive third-down study (based on data from the entire 2009 regular-season), punting there generates about 0.3 points (from forcing the opponent in to bad starting field position), while kicking a field goal would give you 1.9 points on average (including the chances of a made field goal, the chances of a single, the chances of a missed field goal returned and the other team's effective field position following a return or a conceded single). Thus, by expected point values, that's a loss of 1.6 points (significant, as Saskatchewan only lost the game by three points). You might modify those odds down somewhat for the Riders in that particular situation, as their kicker was the largely-untested Warren Kean (thanks to Luca Congi's mid-season injury), but considering all possible outcomes, going for a field goal would appear likely to lead to better results than opting to punt. (By the way, Eddie Johnson's punt sailed out at the 12, so while it certainly wasn't bad, it definitely didn't lead to particular success and it wasn't punting to win; Montreal went on to drive 83 yards for a field goal, tied the game and never trailed again).
There was also a questionable decision with 12:07 left in the second quarter, where Saskatchewan faced third-and-two on the Montreal 20. There, they opted to kick a field goal (the one Kean is pictured making above) instead of going for it. Pettapiece's third-down study indicates that trying a field goal there is worth 2.1 points on average (again counting the chances of a made field goal, a miss that leads to a single, a returned miss and the expected field positions resulting from each), while trying to convert a third-and-two produces 3.7 points (taking into account the chances of a successful conversion, its chances in turn of leading to a touchdown or a field goal, the chances of failure and the resulting field positions from each outcome). Thus, that's an expected loss of 2.6 points.
If you combine that swing with the 1.6 from earlier (and a 0.2 point loss from another third-down decision), Saskatchewan left 4.4 points on the board, more than enough to win the game. (The Riders did in fact make that field goal, but even if you give them a full three points for that instead of the 2.1 expected, that's still a loss of 1.7 points, and that's enough in combination with the other decisions (3.5 points overall) to give them a Grey Cup victory). As Pettapiece pointed out, though, it's impossible to say Saskatchewan definitely would have won with different decisions (as we don't for sure know what outcomes those decisions would have produced, or how Montreal would have responded); what this really indicates is that their decisions were not solid ones from a probability standpoint:
"Would the Riders have lost if they kicked a FG themselves?" Pettapiece asked. "Who knows. The idea here is that in the long run, the decision they made will work out worse for them than the decision they should have made. In the short term, it hurt them too, which makes it easier to tell a story about why they were wrong."
Pettapiece ran this kind of analysis for the entire 2010 playoffs, and the results are quite interesting. He focused on the first and third quarters, as that precludes any clock-related issues due to the end of halves (the second-quarter example above is mentioned because it came so early in the quarter that the clock wasn't a particular factor). Here's the table he came up with after analyzing every first- and third-quarter third-down decision from each 2010 playoff game and comparing the expected points value of the decisions made to the expected point values of the other options available:
The most interesting element there may be Toronto's 4.9 points lost in the East semifinal. Of course, the Argonauts went on to win the game 16-14, but it might not have been as close if they'd been a little more aggressive at times. (You can make a convincing argument that the expected points value of going for it might be lower if your quarterback is Cleo Lemon, but it's still probably worth taking shots when the probabilites are in your favour.) One particularly bad move they made there was kicking a field goal on third-and-two from the 25; it worked, but much like the Roughriders' Grey Cup move, it probably cost them several points. Jim Barker was a deserving winner of the Coach of the Year award in my mind, but his third-down decisions in this game weren't particularly great from a probability standpoint.
Here's Pettapiece's summary of how the first- and third-quarter third-down decisions broke down over the course of the standpoint:
On average, teams' wrong decisions on third down cost them 0.3 to 0.4 points per third-down. Given that there were 11.4 third-down decisions made in the first and third quarters in these playoff games, that's 4.2 points per team per playoff game. (Again, I remove the other quarters because of clock issues, and to make a more conservative guess as to how bad these teams are at third-down decisions.)
As it happens, one point per game in the CFL is worth about 3% of a win, so we have about an eighth of a win lost every season by the average team in making poor decisions. That's not a lot, but remember it's a conservative estimate; if you face more third downs in a game, and do what these teams did, you'd lose even more points. Plus we're only counting half the quarters.
Also, considering how low the costs are (print out my "when to go for it" chart and affix it to the back of the Argos' OC's clipboard), the benefits don't seem so shabby in comparison.
On the whole, I think this study fits in the general trend of alternate-history scenarios. We can't definitively state that borrowing Doc Brown's DeLorean, going back and changing the third-down decisions on these specific plays would lead to the expected swing in game outcome, but we can still learn a substantial amount about both the past and the present from considering how different things might have been after a couple of small changes. Perhaps even more importantly, those past decisions give us new information to consider in future when evaluating third-down decisions. Each call on its own may seem small and inconsequential, but when viewed in a larger context, those decisions may alter who winds up hoisting the Grey Cup.
Maggie Gyllenhaal Foxy Brown Ivana Bozilovic Cristina Dumitru Cat Power
No comments:
Post a Comment