I have found that when to anneal depends on the brand of brass. If you start getting spring-back when sizing your case necks and bumping the shoulder it is time to anneal. I use a Wilson seater die and arbor press. I have noticed another indication annealing is needed when seating the bullet it is not smooth and takes more than usual pressure to seat the bullet.

Following is some interesting info on annealing.

http://www.kenlightmfg.com/cartridgecaseannealer.htm