Did you work the load at 100 yards? It is not uncommon for the best load at 100 not work well at longer distaces. Before you do anything drastic, do your load work at 300 yards. Everything should have settled down at that point.

If that doesn't work then I would look at subtle things to see if you have anything causing bullet deformity.