If your rifle is shooting that well now with factory ammo, I would not change the barrel yet - but a 20" bbl is a handicap for long range competition.

Those are good groups for factory ammo, but with hand loads and the typical accuracy mods which you should do anyway you can probably cut the size of your groups in half. To be competitive the competition you at least have to be shooting 1/2 moa.

Go here and look for known accuracy loads...
and work with someone at first to show you how to handload properly.

http://www.6mmbr.com/308Win.html

Check to make sure the tang and entire barrel is free floated.
One piece scope base if you dont already have one, check and bed the scope base if necessary.
Good rings and hone them if not perfectly aligned. Takes a hone kit to do that but its easy.
Do the entire normal savage accuracy routine - Properly bed your stock (or bed a different stock. If choate with the V block you dont need to bed), machined recoil lug, SSS trigger or other, make sure the tang and entire barrel is free floated. I know little about working the accustock, but what I have read is its better out of the box than a non bedded Tupperware or other factory stock but not better than a properly pillar bedded stock. The idea is to make sure the action has full and even support, everything is free floated, the recoil lug is in 100% contact and the action screws take no recoil.

What scope do you have now ?