Duncan's proponents will cite a few statistics supporting their claims, including his staffs' ERAs (they've led the league 4 times) and his 4 Cy Young winners. However, the answers here are not that simple. What we have just shown is mostly that over his long career, Dave Duncan has coached some very good pitchers. We have no idea how much of that is due to Dave Duncan. We can't use that as evidence much more than we can say Mike Piazza or Javy Lopez were brilliant catchers because their staffs had such success in L.A. and Atlanta. Additionally, we have anecdotal evidence of pitchers like Woody Williams or Jeff Suppan who were perceived to have grown much more successful thanks to pitching under Duncan. This approach also has it's problems and does not necessarily prove anything about Duncan.
None of this means, of course, that the claims about Duncan are unfounded. It just means we need to formulate a better approach if we are going to attempt to find evidence for or against the claims about Duncan. The primary issue here is that we have a premise (Dave Duncan is a genius), but no good way to go about seeing if that might be true. Most fans, broadcasters, analysts, writers, etc. have a relatively weak grasp on statistical meaning and, consequentially, return essentially meaningless statistical evidence when they feel such evidence is needed. It doesn't invalidate their claims, since they are usually based on some sort of intuitive or anecdotal understanding rather than the statistics, but it does fail to validate them.
In Duncan's case, we need to find a way to isolate as best we can his influence on pitchers. If we just look at everyone he coaches and compare them to the rest of the league, we can't tell what Duncan is changing in those pitchers. Instead, we take an approach similar to what Tom Tango calls WOWY (with or without you), meaning we look at what pitchers did under Dave Duncan and then compare that to what they did without him.
I decided to look at starting pitchers who had pitched three straight years away from Duncan before joining his staff, and then compare their performance in those three years to their performance in their first year with Duncan (for players traded for midseason, their first full year was used). This gives us a good sample to evaluate what we can expect from these pitchers without Duncan's influence as well as lets us focus on Duncan's perceived specialty: veteran starters. For this study, a starting pitcher was anyone who made at least 10 starts that year. There were 12 pitchers who were starters for all 3 years before joining the Cardinals and then starters in their first year with the Cardinals, plus 8 more who were relievers at some point in the three previous years who started for the Cardinals in their first year with the team.
The simplest approach would be to simply take each pitcher's ERA over the years leading up to learning from Duncan and compare that to his ERA under Duncan. This approach has several problems, which will be discussed presently, but it is a good place to start. Starting pitchers going to St. Louis during Duncan's tenure certainly have improved their ERAs. In the three years before joining Duncan, the 12 starters dropped their ERA from 4.48 to 4.12 in a bit over 2000 innings. Adding in the other 8 pitchers, their ERA dropped from 4.58 to 4.29 in about 3200 innings. This lends credence to the anecdotal evidence that pitchers do improve significantly under Duncan.
There are four major problems with this simple approach. One of them works against Duncan, two work in his favour, and the final one is mostly neutral. The first is that an aggregate of the past 3 years is not always a very good estimate of what a pitcher should be expected to do. We are looking mostly at veterans here (the average age of the 12 starters when they joined the Cardinals was 29.4, and Mark Mulder at 27 was the only one under 28). A pitcher hitting his 30s, in general, is not supposed to be as good as he was 3 years earlier, but our aggregate counts what he did 3 years ago as just as important to what he should be expected to do as last year. To illustrate this, look at how the ERAs of our sample rise in each successive year prior to joining the Cardinals:
- 3 years prior: 3.84 ERA
- 2 years prior: 4.68 ERA
- 1 year prior: 4.97 ERA
Looking at the trend over the previous three years rather than lumping them all together shows how much our initial method diminished the Duncan Effect. Going just by the previous year, these pitchers dropped from a 4.97 ERA to a 4.12 ERA. Adding in the other 8 pitchers again, the ERA drops from 4.88 to 4.29.
This seems like a huge difference, but we still have to account for the other three differences. One, pitchers going to the Cardinals are always moving to the NL. Sometimes they are coming from the NL, and sometimes from the AL; in the former case, it doesn't matter, but in the latter, a league adjustment becomes necessary because ERAs are lower in the NL. A pitcher going from the AL to the NL should see his ERA drop even if he doesn't pitch any better. Two, the Cardinals have fielded very good defenses, on the whole, during Duncan's tenure. Again, a pitcher moving in front of a good defense should see his ERA drop even if he does not pitch any better. Three, we need to account for park factors. A pitcher coming from Coors should see his ERA drop, while a pitcher coming from Petco should see it rise. Both Busches have been pretty neutral, so this mostly matters when we look at pitchers coming from more extreme parks to Busch.
There is an additional problem that only applies to looking at the sample with the 8 converted relievers: a pitcher's ERA will generally be lower when he is used as a reliever than when he is used as a starter. So converting a pitcher to a starter is likely to boost his ERA a bit. There is also the issue of having fewer innings to project these pitchers' expected ERAs, which will create a minor issue in the next step (namely more regression when we project them). For these reasons, counting these pitchers will underrate the Duncan Effect to some degree, but they do increase our sample to a more comfortable level, and these pitchers are some of Duncan's most famous projects, so we can still look at them, keeping in mind that we are not exactly comparing apples to apples.
Now that we have our main problems outlined, we can refine our approach. The issues will be addressed in the following ways:
- Use Marcel projections instead of a simple 3-year aggregate or the previous year's ERA alone to determine each pitcher's expected performance level
- Park adjust each pitcher's ERA for each year we look at
- Determine a league adjustment to apply for pitchers in the AL to put their stats on par with NL pitchers
- Calculate FIP as well as ERA to account for the impact of fielders on ERA
Then I determined my league adjustments. This was done in traditional fashion, by looking at all pitchers who switched leagues from one year to the next and comparing how they did. To smooth out some of the noise, I looked at 5-year samples (2 years before and after each season), giving more weight to the season I was measuring in my aggregate. In recent years, this adjustment is about .92 (meaning you multiply a pitcher's ERA by .92 when he goes from the AL to the NL). In the mid-90s, it got as low as .80. Separate adjusments were done for FIP and ERA; the two adjustments are similar, but the FIP adjustments seem to be a bit more stable from year to year and didn't go quite as low at their lowest as the ERA adjustments.
The league adjustments were applied to all AL seasons. I did this because I ultimately want to look at what pitchers would do in a neutral environment, so I converted everyone's stats to a neutral NL park (hey, that sounds a lot like Busch Stadium). Once the adjustment was applied, I combined separate stints into full seasons (i.e. our previous Baltimore/Houston example is now simply counted as 1 season for the pitcher in a neutral environment rather than 2 stints in separate environments). These are the figures I plugged into the Marcel projections. I projected both ERA and FIP using this method.
This gives us a much better idea of how each pitcher should be expected to pitch free from any of the above influences. Our group of 12 starters now projects to a neutralized 4.61 ERA and 4.64 FIP entering Duncan's care and ends up with a 4.16 ERA and 4.44 FIP. With the other 8 pitchers, they project to 4.63/4.64 and end up at 4.29/4.53. As we would expect, the resulting ERAs are much lower than the FIPs. The Cardinals' team ERA over Duncan's tenure has been .25 points lower than its FIP due to consistently good defenses.
What about Oakland?
The same thing can be done with Duncan's pitchers in Oakland. This time, the group of pitchers dropped their ERAs by about .2 points, but again, that was in front of mostly good defenses. The FIPs stayed about the same.
What does this all say about the Duncan Effect? To be honest, nothing definitive. We aren't looking at all at young pitchers Duncan might have to develop who came up through his own organization. It doesn't include all pitchers who joined the Cardinals, only ones with a three year track record elsewhere. This means pitchers like Chris Carpenter, who was coming off missing extended time from injuries when he joined the Cardinals, are not counted here because our methods don't do much to isolate Duncan's effect. We don't have nearly as large a sample as we'd like to decide anything for sure. Our use of FIP downplays Duncan's pitch-to-contact philosophy where utilizing those good defenses was more effective than FIP can account for. We can, however, tell a couple important things. One, a pretty good chunk of the percieved Duncan Effect is due to other factors, probably most notably the defenses his teams have had. Two, those other effects don't cover all of the improvement we see in pitchers Duncan has coached, and they still did noticeably better than expected as a group. This does not prove a Duncan Effect, nor does it assign a real value to it, but it does support the claims and suggest that there is a good chance it does exist.
Continue Reading...
