Personally I would bump the shoulder more-- the most you could be sizing would be .003 and many of them wouldn't be getting any shoulder bump....

As far as the .004 difference- how accurate are you at measuring? If you measure the same piece of brass (without looking at the readout until you are done) 5 or 10 times do you get the exact same reading? I never did with the Hornady comparators....
The .004 difference is not causing bad groups--