I have really only damaged one barrel crown- bad enough that it could be easily seen. I cleaned it up the best i could and went and shot it. I was expecting it to not shoot as good but it didn't have any measurable effect on groups. I figured I had gotten lucky and kind of forgot about it. The next time I shot the rifle I used my other load (one was for varmint hunting and the other was a heavier target bullet) and my groups were pretty poor. The rifle had shot both groups very good before- so the damaged crown bothered one bullet but not the other. I had the crown re-cut and it went back to normal- so it definitely can be a issue. I'm still not to the point that i worry about a brush getting pulled back into the barrel every so often....

In my case the barrel still shot flat base bullets excellent but the boat-tailed bullets shot poor. I don't know if it was the gas escaping around them that did it- or the way the bullet shape had it last contacting the barrel. Have you shot different bullet shapes through yours after you re-crowned it?