I had the same problem. After months of letting it sit, and replacing with a fitbit inspire, I tried a more extreme solution. I jb-welded the removal tool to the back of the fitbit, let it set for almost a day, and then was able to twist the back off normally. I used a razor blade to gently pry under the jb-weld until I could pry it up with my thumb and break it off. Then used the razor blade to further disconnect the jb-weld from the lid until it came loose. Now the tool fits better into the worn out back cover, and I'm back in business. To more directly answer your question: yes, you would break it by forcing it out without twisting. Then you'd have to come up with another way to hold the battery in place tightly so it would make a reliable connection.