To be honest you got lucky being able to remove it out of the port, if not that port would've been unusable.
Generally I would avoid using the dongle again as it is bound to break off again however I know mice like these do not have the ability to be used wired and would result in the mouse being junk.
So here is what you could do (but i wouldn't do myself),
If the shielding around the dongle broke off you could try and re-attach it back to the dongle then use a strong adhesive to try and bond the two halves together.
Avoid getting any of it inside the part where the dongle would theoretically mate with the port this is as it would make plugging it in impossible.
In addition to this I would buy a small USB-A extension cable in which you can permanently plug this dongle into, this would provide a safer way of plugging into different devices without risking the port.
However as much as I hate to say this you are probably better of buying a new mouse from a more reputable brand such as Logitech in which you wont encounter this issue.
Also whenever anything USB is too tight it's always best to try another port.