One of the most interesting points of debate during CFL games revolves around third-down situations (like the 2006 one pictured above, where Edmonton quarterback Ricky Ray scrambled to try and convert a third-and-short against Montreal). On broadcasts, live blogs, Twitter and even just in the stands, everyone has an opinion on whether it's best to go for it, punt or attempt a field goal. The problem is that most of these comments are based on personal preferences rather than empirical data, so the debate's pretty limited; if one person thinks the team should punt and another thinks they should go for it, who's to say which is the better decision?
One way to get around this is to assume whichever decision the coach ends up making is the correct one and focus your efforts on trying to predict what move they'll make, but I'm rather skeptical of the "coaches always know best" argument (especially on game-management decisions like whether to go for it, as they don't actually have as much specific experience there as you'd think). Fortunately, there's a way around this; my co-editor at The CIS Blog, Rob Pettapiece (who I've interviewed before about his RPI and SRS calculations), ran an exhaustive study looking at the outcomes of each possible third-down decision from every point on the CFL field and published his results Tuesday. Armed with the results of his study, we can have a much more productive discussion around third-down options and what coaches should do when faced with any given situation.
Before we dive into the meat of the study, there are a few important things to note. One, many studies along these lines have been done south of the border for the NFL game, but while I've always suspected their general conclusion (NFL teams don't go for it enough) to also be true in the three-down game, it wouldn't work well to simply try and translate their data. The CFL game is just too different (more of a focus on passing, 12 men on the field, a bigger field, different goalpost locations). Moreover, it wouldn't really work well to do this with simulations or to try and apply data from all plays (just because a team averages 5.8 yards per play doesn't mean they'll automatically pick up a third-and-five). Pettapiece's approach is much better; he's taken the raw data from the CFL's live play-by-play records for the entire 2009 season and has put it into a database. That database, which includes every third-down situation (field position), decision (punt, field goal, going for it) and result, is then used to produce expected point values for a first down at any given point on the field, and those values are then used to produce the rest of the data.
What can we learn from the data itself? Perhaps the most important part is the table and graph shown at right, which provide the 2009 success rate on third down conversions. Basically, everything up to and including third-and-six conversions succeeded upwards of 50 per cent of the time, with third-and-one and third-and-two attempts succeeding over 70 per cent of the time. Even the success rates for everything down to third and 10 aren't all that bad. As Pettapiece writes, these are smoothed rather than raw values, but the empirical results align with them quite nicely.
We don't have to stop there, though. Knowing the likely success rate of a third-down conversion is only part of the problem; other issues include how much you're likely to gain if you pick up the first down, how much you're likely to lose if you don't, and what you stand to gain from other options (punting or kicking a field goal). That's what Pettapiece focuses on next, looking at the available options on one key play in last year's Grey Cup. In their first drive in the first quarter, the Alouettes had a third-and-one on Saskatchewan's 47. If they go for it, that gives them a first down at the Riders' 46.
Over the course of the 2009 season, teams with a first down in that position gained an average of +2.8 points (some would have went on to score touchdowns, some would have kicked field goals, some would have gone on to punt or turn the ball over via fumble, interception or failed third-down conversion). On touchdowns or field goals, that also factors in the opponents' expected starting position after the kickoff (near the 35, on average). Making a field goal is worth +2.2 points (3 minus 0.8 for expected starting positon), while a good punt (net punting average minus net return average) would expect the Riders to start at their own eight and be worth +1 point on average. Pettapiece then factors in the probabilities of success and failure for each circumstance, and comes to the conclusion that a third-down conversion attempt in that situation is worth +1.8 points, a field-goal attempt is worth +1.2 points and a punt is worth +1. Each decision is expected to benefit the Alouettes to some extent, thanks to their starting in enemy territory, but even with probabilities included, trying to pick up the first down appears to be the best option.
More generally, Pettapiece created the following chart to demonstrate the break-even points for going for it on third down at any given distance on any given part of the field. If your distance for a first down is above the line, you should either kick a field goal (left half) or punt (right half), depending on how far you are away from the opponent's end zone. If your distance is below the line, you should go for it. This obviously varies depending on teams' strategies, schemes and personnel, but by and large, going for it appears to always be the best strategy in everything from third-and-four on down, and it's also frequently the best strategy even at higher distances if you're close to the opposing end zone:
Unfortunately, the reality is that CFL coaches are currently nowhere near this aggressive. Pettapiece told me that his data suggests coaches only currently go for it around 48 per cent of the time on third and one, which seems about right from my observations. That figure trails off sharply, dropping to just over 20 per cent of the time for third and two and then to around 12 per cent for third and three. With four to seven yards to go, they go for it only around eight per cent of the time, and with eight to 10 yards to go, conversion attempts are only made about six per cent of the time.
It's easy to understand why many coaches take a cautious approach to third downs, though. The reality is, in our media and fan landscape, it's always the unusual decisions that get vilified. Very few will question a punt, even if it leads to the other team's winning touchdown drive, while everyone will jump all over a coach if his third-down gamble fails. The ultimate example of this came last year, when Saskatchewan coach Ken Miller made a decision to attempt a punt single instead of a field goal; the odds favoured him, but the plan didn't work out, and Miller was blasted about it for days on end. If he'd made the conventional decision to try the field goal, the effects of failure would have been even worse from a field-position standpoint and they would have been more likely to occur, but few would have criticized his decision. It's the same thing on third downs; even though all the data indicates that going for it is often the right move from a probability standpoint, many coaches aren't going to try it unless they have to thanks to the grilling that would accompany a failed conversion attempt. There aren't many out there who criticize a decision to punt, so that's the safer move from a coaching standpoint, even if it might damage the team's chances of success.
One disclaimer this study does come with is that it is centered on the 2009 season, so it isn't a huge sample size. Also, if there was some peculiar anomaly that affected only that year, the conclusions might not be broadly applicable. Every team also handles third-down decisions differently based on coach, scheme and personnel. However, the amount of data included means the team-specific issues should balance out into a more general picture, and league-wide third-down conversion rates are generally something that don't change enormously from season to season. This is great information to have, and it's further ammunition for those of us who believe that CFL coaches are often too cautious. Moreover, it allows us to add some statistical seasoning to the differing opinions out there. That should only enhance the debate, and that's a good thing in my mind.
Jennifer Gareis Ashlee Simpson Donna Feldman Jodi Lyn OKeefe Emma Watson
No comments:
Post a Comment