You are just making alot more work for yourself. However, if it bothers you that much, mark the sear and only remove enough material from the pillar till it fits and functions correctly. I wouldn't venture into any notching of the sear.

Just my $0.02 worth...

DK