Now that the Mariners have just signed Fernando Rodney to a 2-year, $14M contract, I feel this article I wrote in September is especially relevant to fans with opinions on these sorts of things.
The closer role and its importance in baseball is still an issue of controversy between the “old school” and the “new school,” or whatever you want to call them. I would probably say that the conventional wisdom argues that a pitcher must have the proper mindset to make it as a closer, and that only so many players can handle it. That group of players does not grow on trees, according to this old school. The new school concedes that not everyone can be a closer, but thinks that the pool of capable candidates is quite a bit larger. Can you throw strikes? Do you have stuff? You can close for me, say the sabermetricians.
As per usual, I got some data to help parse through the two sides. Note that it was important not to mess around too much with the sample, here. Adding the Angels’ Ernesto Frieri to the data set, for instance, would be cheating unless we also added in every other reliever that a manager tried out mid-season for his closer role. No picking and choosing who goes in! So let’s be inclusive and get everyone in here—everybody that had at least one save opportunity in 2012.
Here’s my first question:is experience required to fulfill the duties of closer? Perhaps not required, but it seems to help. Using relief innings pitched and total saves from the 2007-through-2011 seasons as proxies for closing experience, I found that both were good predictors of save success during 2012. Each additional 50 saves (in the previous five seasons) correlated to a 3-or-4-percentage-point increase in saves conversions probability during 2012. More innings correlated with save success, too, though not as well as past saves.
A new school pupil might shun the use of saves to measure effectiveness, and so I have Fangraphs’ shutdowns-to-meltdown ratio for you. The results, however, were virtually the same. Having experience with recording saves and shutdowns prior to 2012 correlated to a higher probability of converting such chances during one’s 2012 season.
Before we praise experience as the one thing a good closer needs, the reader should be keen to my use of the word “correlation.” Putting these variables into a fancy regression program doesn’t mean there is a causal link. Let’s ask ourselves another question then:is raw skill required to fulfill the duties of closer? Of course it is!
When I controlled for pitcher skill using the 2012 preseason ZiPS projected ERA, that took over as the best predictor of relieving success. For example, when we control for a pitcher’s projected ERA (perceived skill), that same additional 50 saves in his record book correlates to only a 1.5-percentage-point increase on the saves rate, and the margin of error includes “no improved rate” as a possibility (p-value = 6.9%). It seems a combination of experience and “stuff” is required. Below is the best model (logistic model, that is) that I could come up with to explain save conversion probability:
Notice that any measure of past saves recorded is not actually in the model. Instead, a combination of innings pitched and the leverage (LI) of those innings was the most predictive experience proxy. Here are some ways we could interpret these findings in everyday language:
1) Consider two pitchers that each have each pitched 200 relief innings at an average leverage of 1.4, but one is projected to have an ERA of 2.2 versus the other’s 3.1. The pitcher with the better ERA projection (likely due to better strikeout rates and the like) has an 85% chance of getting a save as compared to 77% for the other guy. By the way, Sergio Romo and Luke Gregerson were basically those two guys. Romo and his better ERA projection helped to take over when Brian Wilson went down, and Romo finished 14-for-15 in save opportunities (93.3%). Gregerson’s experience (209 IP in previous five years) and ability to handle pressure (1.41 LI) helped him to a respectable 9-for-13 figure in save opportunities (69.2%).
2) Consider two pitchers that are projected to record a 3.60 ERA, but one has 325 innings at a 1.7 LI (higher pressure) while the other has just 225 innings at a 1.0 LI (average pressure). The pitcher that has more innings under his belt at higher leverage has an 82% chance of converting his next save, while the “less-seasoned” reliever has just a 62% chance. Those two players are essentially Jose Valverde, who saved 35 of 40 chances that season (87.5%) and Javier Lopez, another Giant who saved 7 of 9 (77.8%).
Obviously the examples weren’t meant to prove the point—the regression analysis made the argument for itself—but the examples provided an idea of what these types of players were like. It very well could be that, despite losing his stuff and walking more batters, Valverde’s high-leverage experience allowed his saves percentage to outperform his ZiPS ERA and WAR expectations. Of course, with just one player, the variance in save success could always just be unexplained dumb luck.
If you’re of the ilk that believes a 7th-inning appearance in a close game with runners in scoring position is just as pressure-packed and important—if not more so—than a ninth-inning save with a three-run lead, then get this. When I created the best shutdown-to-meltdown ratio regression that I could, I got that the best proxy of experience was simply a pitcher’s past shutdown-to-meltdown ratio. Skill, as measured by ZiPS ERA was still the best predictor. In other words, guys that have gotten clutch outs before will get clutch outs in the future, even if those outs came in the 7th inning.
In both regressions, the main takeaway is this: raw talent (as measured by ZiPS projections) is probably the most important part of a good relievers, as just about any baseball fan and analyst would guess. But some amount of successful experiences in pressure-packed situations plays a role in projecting future relieving success, too, so it shouldn’t be totally discounted. It’s just that those pressure-packed instances don’t have to be in the ninth inning.
This means a few things for Mariners fans preparing for 2014. We don’t necessarily need a closer with a proven track record of getting saves, nor would we want to pay for that sort of track record. We need to find a closer with both talent and a proven track record of getting outs in pressure-packed situations. There is apparent talent in the bullpen right now, where Charlie Furbush, Danny Farquhar and Tom Wilhelmsen all have projected ERAs below 3.5. What this bullpen lacks is guys that have gotten lots of outs in high-leverage situations. Often, those guys can be found for relatively inexpensive contracts on the free-agent market.