After removing backplate, Wireless charging is spotty

I opened up my N5 just to see what was inside, removed the battery and cleaned out the speaker. Nothing intense. I put everything back and now the wireless charging is kinda spotty. Pressing the phone together firmly with my hands will let it charge more reliably for a few days. I have taken the back off and put it back on a few times, and even tried to adjust the contact points on the phone to make better contact, all solutions so far have been temporary. Is there a trick to making sure this connection is solid when putting the phone back together?