Look at Eric Cortina's method for setting FL size dies. I would not be concerned with how many thousandths it is set back, just that it is 'enough' to fit your chamber and allows for smooth action. Why would you remove the mandrel? Just curious.

Why can't you use the proper mandrel with your FL or neck size die?