How to replace a broken handle

The parts department said that the handle on our Samsung SMH19265 just pops out. However when it pops out it leave a square plastic piece that is too large to slide the new part onto. Any suggestions on removing the extra plastic layer. Needle nose pliers don't seem to work.