Sounds like your microphone may be faulty/upon reassembly you have not installed it correctly or damaged it.
In the iPhone 4 there are Two microphones
- One on the lower left (iPhone facing Frontwards)
- One in the top left (iPhone facing Frontwards)
The other port on the bottom right of the iPhone is for the speaker. Now the speaker also acts as a microphone however only activates when the phone is put on loud speaker, hence people being able to hear you when activated.
Circled in Red is the Main Microphone, circled in Blue is the connection ribbon responsible for the iPhone recognising the microphone along with the Speaker and several other parts.
So now that we know that the ribbon (circled in blue) is working as the speaker produces sound, the problem now lies with the microphone itself.
When you reassembled your iPhone, when replacing the speaker assembly you may have pushed the microphone out of place, try taking a look at it to see if the small rubber piece covering the microphone is situated correctly.
If it is then take a very close look at the ribbon itself to see if any small tears are visible. Let me know how you go.
Hope this helps!