Sounds like shooter technique... You have proven the rifle shoots with your groups.

A few years back, I worked with a young man that was having the same issue. I had him on the range for a few sessions, only I had him shoot a 10/22 that I had rebuilt. Great trigger and would shoot bug holes at 50 with the Remington Green label. Working on his trigger control, breath control, etc... he began tearing out the 10X.

Switched him back to his rifle, he applied what he learned, and he became quite proficient with it.

Just an idea...