STolen bases & Variable change

The data used in analyzing the value of a stolen base are dramatically flawed.

The classic variable change brain teaser, the Monty Hall problem, goes something like this: given a free choice between three doors (behind one of which contains a grand prize), your probability of choosing the grand prize is 1/3. No dispute. The variable change (and mathematical controversy) comes after the contestant has selected a door, but before it is opened. The host (who knows which door the grand prize is behind) opens one of the doors which does not have the grand prize. Now, two doors remain, one which contains the grand prize and one which does not.

The host offers you the chance to switch doors. Should you? What are your chances of winning in each scenario?

If you keep your original choice, you have a 1/3 chance of winning. The new information exposing one of the “incorrect” doors doesn’t impact the conditions under which you selected originally (1/3): regardless of whether you picked the correct door, an incorrect door would be exposed.

So what are your chances of winning if you switch doors? The answer, though perhaps difficult to understand intuitively, is actually 2/3. Why? Your original selection was made with a 1/3 chance, and again, the new information doesn’t change the conditions of your original choice. The exposed door carries a probability of 0, which leaves the remaining door with a 2/3 chance. One simple action, opening an incorrect door and allowing you the chance to switch, allows you to double your chances of winning, from 1/3 to 2/3.

as it pertains to baseball

Baseball is a game packed with variable change and statistical variation. It’s perfectly fine to generalize, but we understand intuitively that generalizations are just that. For example:

The Yankees scored 5.77 runs/game in 2019. Does that mean they will “usually” score 5-6 runs in a game?

It’s no surprise that a team’s lineup, home field advantage, strength of schedule, weather, how close a game is, and certainly the opponent’s pitching for any given game all play a role in how many runs their team scores. No one would dispute that the Yankees averaged 5.77 runs/game in 2019 - but wouldn’t it be silly to “manage” every game, regardless of variables, with the expectation of scoring 5-6 runs?

In fact, the Yankees had more games where they scored 5 or fewer runs (51.9%) than they did scoring 6 or more, even while their average runs per game was 5.77. They were more likely to score exactly 3 or 4 runs (24.1%) than they were to score exactly 5 or 6 (22.2%).

If the Yankees scored 2 runs in the first inning of a game, you’d rightfully say their runs-scored expectancy has increased over when the game started. That doesn’t mean the expectancy was “wrong” it means that averages are inherently highly variable - and the variables have changed!

The point is this: even if it might be counter-intuitive, using generalizations or averages and attempting to apply them to specific situations doesn’t make sense (and doesn’t work, statistically).

What this has to do with stolen bases

Folks who are far better at math than myself (Tom Tango and many others) have compiled a lot of research regarding the value of a stolen base. It generally starts - and often ends - with this chart:

Put simply, you can predict how many runs a team will score depending on two factors: how many outs there are and which base(s) they’re on. This is often called the base outs state.

From here, you can calculate how many more runs a team would score if successful, and how many fewer runs they’d score if unsuccessful. The “breakeven” point, or expected runs scored, is somewhere between 67%-75% (varies from year to year). As a rule of thumb, 75% is often cited as the necessary success rate, which is why many analytics-minded folks dislike the stolen base.

The math behind this “break-even” calculation for stolen bases isn’t just oversimplified, it’s enormously flawed.

That’s not to say the math is incorrect. The math is absolutely correct. But like using a team’s average runs scored/game to try and figure out how many runs they’ll score in any given game you’ll end up ignoring so many variables that your conclusion can end up flat out wrong.

The run expectancy calculation is an average of averages.

When you know all of the variables for a given calculation, why would you disregard them in favor of generalized chart of averages?

The run expectancy chart is great for generalizing about a season’s worth of data. But that’s not how people use it! They’re using it to say that the break even of stolen bases (in any context) is approximately 75% - and that’s not the case!

If Brett Gardner reaches base to lead off a game for the Yankees - to be followed by Judge, Stanton, Voit, etc. - is the run expectancy 0.86?

If Jose Iglesias reaches base to lead off the second inning for the Reds - to be followed by Barnhart, Farmer, Pitcher, etc. - is the run expectancy 0.86?

Of course not! Treating these very different situations as comparable is just one problem with using the generalized run expectancy chart to find the “break even” value of a stolen base.

When it comes to evaluating the value of a stolen base (or the “cost” of an out) we should input all of the variables we can.