I'm guessing you are referring to the rubber membrane that the shoulder button clicks onto. If you have had your psp for a long time and if it has seen a lot of use it could be possible that the rubber membrane is starting to become warn out or possibly breaking. I would recommend (if you are comfortable with it) inspecting the rubber membrane for the right shoulder button and see if there is any damage to it.
I would recommend getting a replacement battery. If the battery is heating up it could also be a possibility that there is gas forming in your battery which will cause bloating. For your safety I would recommend removing the existing battery and try purchasing a replacement one.