On a 100 yard deer gun, I wouldn't worry about it. Just put the factory stuff back in it.

On a 600-1000 yard prairie dog gun, yeah, I'd replace the lug, the nut and bed the action.

Depends on what you're building it for.