I personally doubt it is the barrel. My 10PC shot PMC 55gr and 75gr Hornady Match ammo all into the same 1/2" groups.

I think you need to be sure about the bedding. The aluminum block is very unforgiving if not set up right. If you get a little something trapped in there so it doesn't seat right it can be a real issue. I have seen this kind of issue time and again with the Rem 700 SPS with the crap plastic stock with the recoil lug bouncing around or intermittent contact with the barrel. You need to check your action/stock fitup really well. Maybe even blue the action, then torque the action screws then separate to see where it is touching.