I have never really noticed it on any of my rifles.
Curious as to what could cause the POI to change if it was reinstalled at the exact same clocking/torque?

Now when breaking in a barrel without a muzzle brake, I have had a POI change when I install the brake. Some shot higher, some shot lower, one did not change at all. But never had it change left/right, only vertical.