So I ended up breaking the retaining pin and that was while compressing the plunger. I tried 2 separate brands of 1/16 punches. Both would not go in the hole. I turned the punches down slightly but i ended up breaking one off in the bolt head. I finally got it removed. I have a feeling that this is not normal. I have never had any AR Roll pins ever give me this much trouble.So now I'm waiting for a new retainer to arrive.