Spacers become a problem when the suspension is used to it's full travel capabilities because the you are changing the parts meant to bear the weight and stress and transferring it to parts that aren't designed for it. I've broken enough parts that were the correct fitment off-roading that I'm not going to chance lowering that threshold even more. It's not that I care about the cost of breaking parts, but being stranded up in the woods just plain sucks.
For most trucks, when you go over 3", you really need to start changing other parts to correct the geometry regardless of the type of lift.