FAQ and Forum on Advanced Stats

[Eminor3rd]

Why WAR doesn't always add up.

 

http://sports.yahoo.com/news/10-degrees--w...-030133203.html

 

Dave Cameron's response to the article, complete with massive comment thread: http://www.fangraphs.com/blogs/a-discussio...-improving-war/

[cabiness42]

Dave Cameron's response to the article, complete with massive comment thread: http://www.fangraphs.com/blogs/a-discussio...-improving-war/

 

While I have the statistical background to do a very serious analysis of WAR, I've never really taken the time to do it.

 

I think one of the problems is that we are basing metrics on "replacement level" instead of "league average". I get that replacement level comparisons create better marginal difference when comparing players, but I think it runs into some issues when comparing across positions. If you are a league average 1B, you receive negative credit for your defense. If you are a league average SS, you receive positive credit for your defense. In one way it makes sense because SS defense is much more important than 1B defense, but at the same time you can really distort the value of the two positions. Maybe instead of having predetermined offsets for defensive value, just compare each player to the positional norms for both offense and defense to determine value.

 

 

[witesoxfan]

While I have the statistical background to do a very serious analysis of WAR, I've never really taken the time to do it.

 

I think one of the problems is that we are basing metrics on "replacement level" instead of "league average". I get that replacement level comparisons create better marginal difference when comparing players, but I think it runs into some issues when comparing across positions. If you are a league average 1B, you receive negative credit for your defense. If you are a league average SS, you receive positive credit for your defense. In one way it makes sense because SS defense is much more important than 1B defense, but at the same time you can really distort the value of the two positions. Maybe instead of having predetermined offsets for defensive value, just compare each player to the positional norms for both offense and defense to determine value.

 

Players that play those positions receive those +/- adjustments based on the position itself, not the defense. Because 1B is easier to play, there is a larger negative adjustment in play to compare players between 1B and SS to try and determine total value. DH receives the largest negative adjustment because there is literally no skill involved in being a DH defensive except for knowing to stay off the field. Those adjustments have been determined based largely upon how teams have constructed their lineups historically and the numbers those positions have posted. It might be time to recalculate some of those numbers, but based solely upon numbers, the DH position would receive a higher adjustment than 1B due to the way teams use the DH in today's game, which is not logical.

 

You won't see a 1B defensive ability being degraded because he plays 1B - you can absolutely provide positive defensive value at 1B and Pujols is a guy who's done it before. Instead, I'd say it's because they have less of an opportunity and they are simply not asked to make a lot of plays that would vastly improve their contributions defensively. Along the same lines, a SS isn't given credit for his defense because he plays SS but is given credit solely because he plays SS and the idea is that someone like Jose Abreu would absolutely cost his teams more wins at SS than would be worthwhile. Compare the likely ranges of Andrelton Simmons and Jose Abreu at SS - is it far-fetched to think that Simmons will save you 60 more runs than Abreu at SS? I don't think it is, and the positive positional adjustment that Abreu got for playing SS (+7.5) is easily negated by his incredibly poor defense (say -27.5, and that's being generous).

[cabiness42]

Players that play those positions receive those +/- adjustments based on the position itself, not the defense. Because 1B is easier to play, there is a larger negative adjustment in play to compare players between 1B and SS to try and determine total value. DH receives the largest negative adjustment because there is literally no skill involved in being a DH defensive except for knowing to stay off the field. Those adjustments have been determined based largely upon how teams have constructed their lineups historically and the numbers those positions have posted. It might be time to recalculate some of those numbers, but based solely upon numbers, the DH position would receive a higher adjustment than 1B due to the way teams use the DH in today's game, which is not logical.

 

You won't see a 1B defensive ability being degraded because he plays 1B - you can absolutely provide positive defensive value at 1B and Pujols is a guy who's done it before. Instead, I'd say it's because they have less of an opportunity and they are simply not asked to make a lot of plays that would vastly improve their contributions defensively. Along the same lines, a SS isn't given credit for his defense because he plays SS but is given credit solely because he plays SS and the idea is that someone like Jose Abreu would absolutely cost his teams more wins at SS than would be worthwhile. Compare the likely ranges of Andrelton Simmons and Jose Abreu at SS - is it far-fetched to think that Simmons will save you 60 more runs than Abreu at SS? I don't think it is, and the positive positional adjustment that Abreu got for playing SS (+7.5) is easily negated by his incredibly poor defense (say -27.5, and that's being generous).

 

OK, but I guess my point is that nobody is ever going to put Abreu at SS, so saying that Andrelton would save 60 more runs than Abreu at SS is really meaningless. If you replaced both players with guys who were replacement level players at those positions, there is not going to be a 60 run swing between the two teams.

 

Teams allow an average of 665 runs per year. I think that article stated that 86% of run prevention is pitching and 14% is defense, so defenses are responsible for allowing 93 runs per year as a team. No single player is 60 runs better defensively than another player. Something is out of whack, here. I just haven't taken the time to figure out exactly what that is.

[Eminor3rd]

OK, but I guess my point is that nobody is ever going to put Abreu at SS, so saying that Andrelton would save 60 more runs than Abreu at SS is really meaningless. If you replaced both players with guys who were replacement level players at those positions, there is not going to be a 60 run swing between the two teams.

 

Teams allow an average of 665 runs per year. I think that article stated that 86% of run prevention is pitching and 14% is defense, so defenses are responsible for allowing 93 runs per year as a team. No single player is 60 runs better defensively than another player. Something is out of whack, here. I just haven't taken the time to figure out exactly what that is.

 

The idea of positional adjustment is for an objective frame of reference for comparison. Without it, it's difficult to lend context to the difference between a SS with a 98 wRC+ and a 1B with a 112 wRC+. Which is better for your team? The answer is, of course, that it depends on who you're replacing. If you had to go with a run of the mill replacement at each position, the position adjustment give you a common denominator of pure production. A replacement level 1B is a better hitter than a replacement level SS, so a good hitting SS's production plays up more than a good hitting 1B's does.

[cabiness42]

The idea of positional adjustment is for an objective frame of reference for comparison. Without it, it's difficult to lend context to the difference between a SS with a 98 wRC+ and a 1B with a 112 wRC+. Which is better for your team? The answer is, of course, that it depends on who you're replacing. If you had to go with a run of the mill replacement at each position, the position adjustment give you a common denominator of pure production. A replacement level 1B is a better hitter than a replacement level SS, so a good hitting SS's production plays up more than a good hitting 1B's does.

 

OK, but that seems to be a different issue. One one hand, you are saying that a replacement level 1B hits better than a replacement level SS, so a 1B needs a better wRC+ than the SS in order to generate the same amount of value. Yet on the other hand, the baseline for defensive runs saved is being adjusted because an average SS saves more runs on defense than an average 1B. It seems like the 1B is getting dinged twice for the same issue.

[Eminor3rd]

OK, but that seems to be a different issue. One one hand, you are saying that a replacement level 1B hits better than a replacement level SS, so a 1B needs a better wRC+ than the SS in order to generate the same amount of value. Yet on the other hand, the baseline for defensive runs saved is being adjusted because an average SS saves more runs on defense than an average 1B. It seems like the 1B is getting dinged twice for the same issue.

 

I'm not familiar enough with how positional adjustment factors into DRS to argue otherwise. I would agree that, on the surface, it makes no sense to make an adjustment for a counting at any point but the final stage. But does WAR use DRS? I know fWAR uses UZR.

[cabiness42]

I'm not familiar enough with how positional adjustment factors into DRS to argue otherwise. I would agree that, on the surface, it makes no sense to make an adjustment for a counting at any point but the final stage. But does WAR use DRS? I know fWAR uses UZR.

 

I'm not familiar enough with it either, but it certainly seems like somehow, defense is getting overvalued in the calculation of WAR.

[LDF]

 

I thought this might be good reading, esp for the advance stat people. 1

of the sox players made the list. I don't know if it was good or bad.

 

http://www.hardballtimes.com/improvements-...-but-effective/

[nickofypres]

I *think* I have a general understanding of what IN/FB%, GB/FB%, and IFFB% mean, but the point of these is that the higher they go up the worse, right? I would assume that more infield flies, more ground balls, and less fly balls leaving the infield are a sign of losing power and strength, no?

 

Anyone want to confirm/deny/re-teach me?

 

 

Thank you.

Edited October 20, 2014 by woods of ypres

[witesoxfan]

I *think* I have a general understanding of what IN/FB%, GB/FB%, and IFFB% mean, but the point of these is that the higher they go up the worse, right? I would assume that more infield flies, more ground balls, and less fly balls leaving the infield are a sign of losing power and strength, no?

 

Anyone want to confirm/deny/re-teach me?

 

 

Thank you.

 

For the most part, yes, unless there was some underlying cause. If there were an injury that affected a player's talent level but did not ultimately change a player's talent level, you can see certain traits like that "pop up" (pun intended ).

 

However, it also depends on the type of player. For a player as fast as Adam Eaton, I'd rather not see him trying to hit the ball in the air because you will have better batting averages on ground balls versus fly balls, but hits via fly balls will go for more bases. Obviously infield fly balls are always bad for hitters and always good for pitchers, but GB/FB% will depend on the context and the player you're looking at.

[caulfield12]

http://www.beyondtheboxscore.com/2014/10/4...rics-revolution

 

AJ Hinch and the marriage between SABER front offices and on-field staff decision-making

[nickofypres]

For the most part, yes, unless there was some underlying cause. If there were an injury that affected a player's talent level but did not ultimately change a player's talent level, you can see certain traits like that "pop up" (pun intended ).

 

However, it also depends on the type of player. For a player as fast as Adam Eaton, I'd rather not see him trying to hit the ball in the air because you will have better batting averages on ground balls versus fly balls, but hits via fly balls will go for more bases. Obviously infield fly balls are always bad for hitters and always good for pitchers, but GB/FB% will depend on the context and the player you're looking at.

 

I see what you're saying.

 

 

Thank you.

 

[QuickJones81]

I was thinking about the bullpen issues, and ways to approach fixing it. One idea was to look at SPs who could be good candidates for a secone life as a closer/reliever. Thinking of it briefly one thought was to look at the 1st inning splits for all mlb and milb SP that could potentially be buy low candidates. I have no idea how to easily pull together that data. Is there a good site availbale that you guys know of that could help?

[Eminor3rd]

I was thinking about the bullpen issues, and ways to approach fixing it. One idea was to look at SPs who could be good candidates for a secone life as a closer/reliever. Thinking of it briefly one thought was to look at the 1st inning splits for all mlb and milb SP that could potentially be buy low candidates. I have no idea how to easily pull together that data. Is there a good site availbale that you guys know of that could help?

 

Interesting thought. I don't know of any site that has the ability for you to get a list with those splits, but if you wanted the data, you might try shooting David Appelman at FanGraphs an email. They have a ton more data than they actually display on the site.

 

That said, I'm not sure this is a good way to identify RP candidates, simply because SPs don't tend to throw anywhere near their hardest in the 1st inning and that they often purposely withhold some of their pitches to save for the second and third times through the order.

 

I think you may be onto something in principle, though. There probably are some consistent indicators of what characteristics of starters make them good candidates for the bullpen. I'd bet that teams have their own criteria internally, but it would be cool to develop something for the public.

[witesoxfan]

Interesting thought. I don't know of any site that has the ability for you to get a list with those splits, but if you wanted the data, you might try shooting David Appelman at FanGraphs an email. They have a ton more data than they actually display on the site.

 

That said, I'm not sure this is a good way to identify RP candidates, simply because SPs don't tend to throw anywhere near their hardest in the 1st inning and that they often purposely withhold some of their pitches to save for the second and third times through the order.

 

I think you may be onto something in principle, though. There probably are some consistent indicators of what characteristics of starters make them good candidates for the bullpen. I'd bet that teams have their own criteria internally, but it would be cool to develop something for the public.

 

I think looking at the first time through the order compared to the second and third times would be a better indicator of success. I think the first inning itself lends itself to a lot of sample bias, particularly in that you are going to face at least the 1-2-3 hitters who should be among the 4 or 5 best hitters on the team. You'll also see how well teams adjust to a certain pitcher based on that. It's safe to say that Chris Sale would make an excellent reliever - he already did at one point in his career - but teams can't figure him out the second and third times through, so he also makes a fine starting pitcher.

 

That would show both how teams adjust to a guy as they continue to see him as well as possible deterioration of stuff.

[QuickJones81]

I think looking at the first time through the order compared to the second and third times would be a better indicator of success. I think the first inning itself lends itself to a lot of sample bias, particularly in that you are going to face at least the 1-2-3 hitters who should be among the 4 or 5 best hitters on the team. You'll also see how well teams adjust to a certain pitcher based on that. It's safe to say that Chris Sale would make an excellent reliever - he already did at one point in his career - but teams can't figure him out the second and third times through, so he also makes a fine starting pitcher.

 

That would show both how teams adjust to a guy as they continue to see him as well as possible deterioration of stuff.

 

It would be interesting to look at a guy like Eric Gagne as a case study, to understand why he wasn't successful as a starter, but ultra successful as a reliever. What indicators existed from his time as a starter that could have predicted his eventual success. As you guys mentioned internally teams probably have run through these scenarios but it feels like the best chance to get effective relievers at a low cost. On top of that, it's a great way for teams to create value buying low on relievers then flipping them when they meet success.

[witesoxfan]

It would be interesting to look at a guy like Eric Gagne as a case study, to understand why he wasn't successful as a starter, but ultra successful as a reliever. What indicators existed from his time as a starter that could have predicted his eventual success. As you guys mentioned internally teams probably have run through these scenarios but it feels like the best chance to get effective relievers at a low cost. On top of that, it's a great way for teams to create value buying low on relievers then flipping them when they meet success.

 

I think part of Gagne's success was found in syringes. Even the best closers in the game right now - Holland, Kimbrel, Jansen - blow saves. He converted 82 saves in a row. That's crazy.

 

Anyways, Gagne actually seemed to have more success his 2nd time through the order in 2001 and in the 4th inning. My gut tells me that has to do with him getting hit around enough in the first 3 innings that he was facing the back end of lineups again already by the 4th inning.

[Eminor3rd]

Yeah, just intuitively, it's probably something like:

 

1. Has two good pitches, but not three, leading to a bigger than normal split for "times through the order"

2. Has fringey control, meaning that he has probably dialed his velo back to limit walks, but has more in the tank if he could get away walking 4.5 per 9

3. Has trouble with runners on base, suggesting that he has trouble repeating his mechanics when shifting from the windup to the stretch and may benefit from just getting used to going out of the stretch all the time

[Rowand44]

Steroids or not when Gagne was in the middle of that streak he was one of my favorite pitchers ever to watch. He just made hitters look silly.

[witesoxfan]

I posted this in our previous discussion of projections for the Sox, but this needs to go here too.

 

http://www.fangraphs.com/blogs/does-projec...thing/

 

When we talk about projections, and you have disagreements with them, this will be referenced often. Projections are not trying to be perfect. Metaphorically speaking, they are painting a picture of what you look like in October of year "x" based on how you have looked in every year prior to that. Given that it's based in the future, we have absolutely no idea what you will look like, but given the circumstances, we can probably make an educated guess. That's all this is, and if it's incredibly wrong on a few samples, it doesn't mean the system or the process is wrong.

[Balta1701]

I posted this in our previous discussion of projections for the Sox, but this needs to go here too.

 

http://www.fangraphs.com/blogs/does-projec...thing/

 

When we talk about projections, and you have disagreements with them, this will be referenced often. Projections are not trying to be perfect. Metaphorically speaking, they are painting a picture of what you look like in October of year "x" based on how you have looked in every year prior to that. Given that it's based in the future, we have absolutely no idea what you will look like, but given the circumstances, we can probably make an educated guess. That's all this is, and if it's incredibly wrong on a few samples, it doesn't mean the system or the process is wrong.

So the 2 sigma SE prediction appears to be about plus/minus 8-10.

[witesoxfan]

quick edited the post so it doesn't immediately go to my brown-nosing of a 1st comment of "THANKS JEFF U R SO AWESOME SIGN MAH BOOBIES"

[shysocks]

I posted this in our previous discussion of projections for the Sox, but this needs to go here too.

 

http://www.fangraphs.com/blogs/does-projec...thing/

 

When we talk about projections, and you have disagreements with them, this will be referenced often. Projections are not trying to be perfect. Metaphorically speaking, they are painting a picture of what you look like in October of year "x" based on how you have looked in every year prior to that. Given that it's based in the future, we have absolutely no idea what you will look like, but given the circumstances, we can probably make an educated guess. That's all this is, and if it's incredibly wrong on a few samples, it doesn't mean the system or the process is wrong.

An R-squared of .43 doesn't strike me as all that great. Give it more than two years of data and that would maybe go up, but this shouldn't really convince us of anything we didn't already know. The projections have the right idea but there are enough big misses to seriously caution against using them as the gospel. I mean, just eyeballing, it looks like there are maybe ten teams that exceeded their projection by 10 WAR or more. That's a sixth of the sample.

 

The most valuable paragraph is easily the one below. If your projection isn't favorable, you're a longshot:

Over the two seasons, 20 teams have made the playoffs. I’m counting the wild-card games as the playoffs. They’ve averaged 37 projected WAR. Of the 20, 14 had at least 35 projected WAR. A total of 19 had at least 30 projected WAR, where the actual minimum in there was 32.7. But, the 2013 Indians played an extra game after entering the year with 27.6 projected WAR. So there’s the floor, so far. The Indians have had the lowest projected WAR of a playoff team. Then there’s a gap of more than five wins until the next-lowest projection for a playoff team, but this just proves you don’t have to project that well to actually do something. This is why the White Sox have a real shot. Maybe not so much the Padres, but, who knows? They’re not done.

[witesoxfan]

An R-squared of .43 doesn't strike me as all that great. Give it more than two years of data and that would maybe go up, but this shouldn't really convince us of anything we didn't already know. The projections have the right idea but there are enough big misses to seriously caution against using them as the gospel. I mean, just eyeballing, it looks like there are maybe ten teams that exceeded their projection by 10 WAR or more. That's a sixth of the sample.

 

The most valuable paragraph is easily the one below. If your projection isn't favorable, you're a longshot:

 

I think that's the point of these projections - it's to give us some sort of grasp of what these teams might look like. And, considering they've only been doing this for 3 years or so, they're bound to improve and refine their equations and techniques.

 

The article states that these things do matter and that they are worthwhile because they do show some correlation between zWAR and sWAR and fWAR, even if it's not perfect. If the R-squared were way, way worse, I think we could throw it out the window.

 

Also, I find the final paragraph to be the most valuable and important (emphasis mine):

 

Ultimately this teaches you nothing you couldn’t have already guessed: the projections we have are fine, and they can generally identify good and bad players. Teams with more good players project as better teams. Teams with more good players end up as better teams. There’s also a lot of noise, such that we don’t actually ever know who’s going to have the best record when we’re looking at things in December or March. Yet people have been requesting something like this, so I think it’s worth the occasional reminder that what we have is functional, as a starting point if nothing else. Look at the projections, and go from there. They’re not trying to mislead you. But there are reasons they play the actual games, and it’s not just to make people money. Although it is in large part to make people money. But that’s a different conversation.

 

I've kind of been preaching this for a while when looking at Steamer and ZiPS projections...these things aren't going to look favorably upon the White Sox for a ton of reasons, but this isn't cause for getting upset, because these aren't the gospel and no one has ever claimed they should be seen as such. They are a baseline as to how we should look and react to teams going into the season, and these projections are saying maybe we should temper expectations a little because, even with a lot of holes replaced, this was still a team that lost 89 games last year. Improving by 12 games and winning 85 games would mean that, in 3 years on the job, Rick Hahn took a terrible team with little talented up and down the system and he's transformed it into a competent MLB team with some semblance of talent up and down the minor league system.

 

The Orioles were projected for 32 zWAR last year and then lost 3 of their top 5 players, yet they still won 96 games. ZiPS and Steamer would have suggested that they would have lost 90 games, but that's why they play the games.