I was a bit worried about removing metal in places I shouldn't. I started with sander drum in the Dremel, but quickly changed to the polishing cone.

It seems like it was just popping free (before the mods mentioned above) once it gets all the way over to the side... beyond the point where it would be hitting that corner. Not sure.