I’d like to share a sample of recent text messages from my mom:
“What is going on?!”
“Am I insane?”
“How can they be so bad at their jobs??”
The target of her ire? The MLB umpires calling balls and strikes during Washington Nationals games.
While I don’t see quite as many games as her, I still catch a bunch on MLB.TV, and completely share her sense that the rate of “bad calls” by umpires was up a ton in the first half of the 2023 season. To add insult to injury, it also has felt that these calls often came during pivotal moments of games, where it’s most critical for umpires to spot balls and strikes accurately.
Since we both felt so strongly about it, I was excited to dig into the data to see if our feelings were truly merited, or if our anger at the umpires was misplaced.
The Data
Fortunately for me, there is an amazing group called @UmpScorecards which uses Statcast pitch location data to “grade” umpires, and then automatically posts the scorecard data publicly to Twitter for fans!
Here’s an example scorecard in which Laz Diaz had a very bad day behind the plate… you can see that he incorrectly classified 10% of pitches, which “tilted” the outcome of the game by +1.24 runs to the Giants, and away from the Yankees.
#OpeningDay
— Umpire Scorecards (@UmpScorecards) March 31, 2023
Umpire: Laz Diaz
Final: Giants 0, Yankees 5#SFGameUp // #RepBX#SFvsNYY // #NYYvsSF
More stats for this game 👇https://t.co/4sqr4nQTVr pic.twitter.com/LCrLh6AiXq
In addition to their single game tweets, @UmpScorecards is very generous to publish summarized season-level data to their website in easy tabular form, which is precisely the data we want to study.
Below, I’ll discuss the impact of umpiring on each team by aggregating the total “run impact” of missed calls. This uses some situational modeling (details on the @UmpScores Information page) to quantify the cost of missed balls or strikes in terms of runs. Importantly, this measure helps capture the proper weight of a missed call in a high leverage situation (ie. 2-2 count, bases loaded) vs low leverage situations (0-0, bases empty).
The Findings
2023 First-Half Umpiring Data
To start, I looked at the Total Favor metric per team in the first half of the 2023 season. This combines the impact of missed calls to a team’s pitchers and batters, for a holistic total that answers the question “how many runs have missed calls helped or hurt this team’s batters and pitchers?”
Fortunately (or unfortunately) for my mum and I, the data absolutely bears out our suspicions about the impact of umpiring on the Washington Nationals - they’re getting screwed!
Missed calls cost the Nats 27 runs in the first half of the 2023 season - twice as many as the second-place Oakland A’s have lost.
To dig a bit deeper, we can decompose the total run impact into it’s primary batting and pitching subtotals. This data continues to show that the umpires are doing the Nats dirty - hurting us with missed balls and strikes while pitching, and then continuing to do the same in the other half of the inning when at the plate. No other team in the league scores so poorly along both dimensions, which is crazy to see.
Longtudinal Umpiring Data
Given the data above, I’m sure you can understand that I was a little shook! As suspected, the Umps have absolutely been shafting our Nats.
To make myself feel better, I also took a peek at the data for the past five seasons, trying to find a reason for optimism on a go-forward basis, and found two trends which simmered my anger down a bit.
Umpire Favorability and Team Winning Percentage
First, the data tells us that the linkage between team winning percentage and average run impact of missed calls is weak - just because a team is really good, it doesn’t mean that they are always going to get favorable calls (and vice versa). Because of this, we wouldn’t expect the Nats to be getting systematically worse calls than their opponents purely on the basis of team record.
It’s nice to see this trend borne out in the data for baseball - I’m skeptical that basketball or football would see this same trend, as I think those sports have more subjectivity, where the quality of the teams involved is more likely to skew umpire perceptions.
Year-over-Year Stability of Favorability
Secondly, the aggregate impact of missed calls does seem to settle around zero in the long run, albeit with a reasonable amount of noise. This tells me that while certain teams might have a season or two of unfavorable outcomes in a row, that we should generally expect the impact of umpiring decisions to regress to the mean.
For the sake of my Nats, let’s hope they do!
