I think you have a corrupted install and you need to repair your firmware.
So if I were you, I'd put the old battery back and restore the phone.
The easy way, try it first:
- Download and install the latest version of iTunes on your laptop (new version just out now);
- Turn off your iPhone;
- Press and hold Home Button;
- While pressing Home Button, connect your iPhone to your computer using a known good sync/charge cable;
- Your phone should turn ON and show you "connect to iTunes," while iTunes will tell you that it "detected an iPhone in restore mode;"
- You will be prompted to download and restore your firmware. Accept and leave your iPhone undisturbed until the process is over and your iPhone starts fresh.
NOTES:
- If you iPhone's Wi-Fi was not working properly before the initial update to 8.3 that caused this issue, then there is a high probability that the new restore will fail as well and your phone will remain stuck on restore. Let us know of that's the case; there could be a remedy;
- All your data will be erased, hope you have a back-up somewhere;
- You'll need your Apple ID and password to unlock your iPhone, if activation lock was ON (Find My iPhone ON) before the restore.