Starter replacement or other problem

I went to get into my jeep yesterday and the engine tries to start but does not turn on. When I turn the key I hear a buzzing type noise which is coming from my fuel tank. So I'm assuming that noise is my fuel pump. When I try to start the car the lights inside also dim a little bit.

I tried to clean the battery terminal connectors and when I disconnected them the first time nothing would happen. The car wouldn't even try to start. I had to jump the battery and then it would at least try but not turn over. When I use my multimeter to test the battery it comes up as 12.3 V.

Does this sound like a starter? If so, how the heck do you get to it? I believe I found it but it is in a ridiculous spot. I have pictures if anyone needs them!!