Hi Jason,
Need to clarify a bit. There are two speakers in your phone, the earpiece speaker that's used when you're on a call, and the lower speaker, AKA the loudspeaker at the bottom of the phone that's used for most other audio.
I'm afraid you confused me a bit when you said it's fuzzy when you're on the phone and use the speaker. Do you mean you put it in speakerphone mode while on a call and it gets fuzzy? Because when you do that, you're transferring the audio from the earpiece speaker to the loudspeaker.
Either way, it wouldn't hurt to try and clean whichever speaker is giving you problems. Take a toothbrush and some 90% or higher concentration isopropyl (rubbing) alcohol and see if you can clean out the speaker grille. It's not uncommon for those speakers to sound bad when they're gummed up, so a good cleaning can sometimes do wonders. If you have some canned air, a good spray with that can also help blow out any dust or dirt that's collected in it.
This part's a little gross, but a lot of people report good results at getting the speaker cleaned out by putting it in your mouth and giving it a good suck. If nothing else helps and you're desperate, I suppose you could give it a try.
Anyway, if all that fails then yes, the next step would be to replace the speaker, whichever one it is that's giving you problems. They're both about the same level of difficulty to replace, which isn't particularly hard, fortunately. Here's the guide you'll need for the earpiece speaker.
iPhone 7 Earpiece Speaker Replacement - iFixit Repair Guide
And here's what you need to replace the lower speaker.
iPhone 7 Speaker Replacement - iFixit Repair Guide
You can buy all the parts and tools you need to do the job right here on iFixit and you'll be supporting their repair efforts at the same time, or you can find everything you need online on sites like Amazon, eBay and AliExpress, among others.
Good luck and let us know what you find!