It's not unsafe to use unfired brass, it's just a hassle.

I did that once with a 220 Swift. Set the barrel with tape on an unfired piece of brass. Shot all of the brass I had 1x. FL sized them and and checked them in the rifle. Some would chamber with some bolt drag, some would not chamber at all. So, I reset the barrel with one of the freshly FL sized. Shot them all again. FL sized them again and checked them in the rifle. Again, most would drag and the rest would not chamber. Reset the barrel again and shot. FL sized this time they all chambered fine.

The problem was that the unfired brass was so undersized that the barrel was set with quite a bit short headspace. So much so, when I FL sized the first two times, the die was only sizing the neck and not touching the shoulder or body.

I'll use brass to set barrel, but only fired and FL sized brass from a properly headspaced barrel. I actually prefer it to proper gauges.