I think it depends on whether you push the velocity of your loads up near Pmax for the 6.5mm CM (63,091 psi) or you stay around 51,000 to 54,000 psi where I find my best accuracy.
At those lighter loads, I am getting 22 to 25 reloads with Lapua brass. The primer pockets wear out before the necks so I don't bother annealing.

If you are wearing brass out in five reloads (I think that was what the five meant in your post), you might have a tight chamber that is increasing the pressure and wearing out your brass.
Have you checked your brass for any telltale signs of pressure?
I had the problem on my 12 LRP when I got it, and recorded pressure signs on the brass (via pictures) at different loads from 57,000 psi down to 38,000 psi and was still getting pressure signs.

I checked with Hornady, who invented the cartridge, and they told me that 6.5mm CM chambers are often bored tight to improve accuracy and that would increase pressure.
The higher pressure from the tight chamber was wearing out the expensive Norma brass primer pockets after 4 reloads.
I sent the pictures to Savage and they had me send the rifle back to them. They bored out the chamber a few thousandths and the pressure signs went away, even at 57,000 psi
The accuracy didn't get any worse, in fact it got better, but I was also really paying attention to my technique and reducing shooter induced variations too so that might have been part of it.

I have had two other 6.5mm Creedmoor Savages since that 12 LRP and have had no chamber issues with either of them at up to 57,000 psi, so
I concluded that the chamber reamer may have been at the end of its useful life when they used it on the12 LRP rifle.

Before you give up on the Hornady brass, I suggest that you check for pressure signs.
If there are, take some pictures and then reduce the loads and see if the reduced loads also show the same high pressure signs. That would indicate to me that you have a tight chamber.
It there are none, and you aren't pushing velocity out of the rifle's accuracy zone, then it might be worth trying different brass.

I believe, Starline also makes 6.5mm CM brass with small rifle primers. That might be a lower cost alternative.
Personally, I use Lapua on almost all of my accurate rifles because I get so many reloads with Lapua compared to what I got with other brass that the price per reload is lower than most other brass I have used.
If I had been able to find Lapua at the time I got my 12 LRP, I never would have bought Norma and Hornady brass with the LRPs to begin with, whether or not Lapua used small rifle primers.

I have recorded some improvement in accuracy since switching to Lapua brass with small rifle primers, but I also made the change from Hornady and Norma to Lapua right after Savage bored out the chambers.

If you do decide to make the change, I found that CCI BR-4 and 400 small rifle primers perform better than Federal 205M small rifle primers in my two 6.5mm Creedmoor Savages.
I have yet to try Remington 7 1/5 Benchrest primers but I probably won't because I just bought another 5,000 CCI BR-4s.