It depends on how much he falls off, obviously. Removing 40 walks still gives him a BB% of 12.5%~. That's still above average. Give him an 9.75% walk rate, which would rank 3rd best for a team average BB%, and it drops the Sox to 15th. Give him a league average BB% of 8.7% and it drops another spot I'd imagine.
15% of a teams walks by one player is pretty significant. If you replace him with a player with the league average walk rate, it drops the Sox from 4th to 15th or 16th. That's a lot.