I have had a similar experience with a few rifles. The sporter contour barrels tend to heat up quickly and that has had a tendency to change my point of impact.

How far were the shots stringing? I would call it normal if the changes in poi were not too large. Also, exactly how warm is your barrel? How many shots can you fire before you notice a poi? Did you run a boresnake through the barrel or clean your rifle in your cold bore trials? I have also had some issues with clean vs. fouled bore... Keep shooting, if this is a new barrel it may change how it "behaves" after the barrel is broken in.

Not much help maybe, but a couple of things to think about.

-Scott