Adding memory is by far the easiest upgrade in this series! I would start there.
Servicing 'Thin' series systems is not as easy as the older models as just taking off the display is a very delicate task as if you mess up you damage it! Which can be a very expensive mistake! So make sure you have the correct tools and follow the IFIXIT guides to the letter!
OK, what can you upgrade inside this series sadly not much!
If you have a SSD only model adding a traditional HD or even a second SSD is possible. Here is the IFIXIT guide: iMac Intel 27" Retina 5K Display Hard Drive Replacement. You will need to add this: OWC In-line Digital Thermal Sensor for Hard Drive Upgrade for 27" iMacs 2012 and Later as the system requires a thermal sensor to manage the heat. You'll need it if you swap out the HD for another, add either a HD or SSD to the system in the 3.5" drive bay.
Adding in the blade SSD if you have a HD only model is possible but much more work and you do need to have the right custom blade SSD and so far Apple is the only source for this custom part. To add to this the price of this part is at a premium and not easy to find! So don't be surprised its double what you expect and you might get the wrong part (scam'ed) as people will try to take advantage of the desire of people like you. Here's the IFIXIT guide to get to it: iMac Intel 27" Retina 5K Display SSD Replacement
If you have a Fusion Drive you face some added complications as a fusion drive is nothing more than logically linking two discrete drives (HD & SSD) into one logical drive. You'll need to break the set before you can take either drive out. Otherwise, the guides above will step you though the process of replacing the discreet drive.
I would look at external Thunderbolt RAID drives as a safer and better solution if you need more storage. They are a lot less work and surprisingly can be faster than anything you put into the 3.5" SATA III (6.0 Gb/s) bay!
As to the CPU it can be replaced with the i7 model but that won't gain you much. Here's the IFIXIT guide to do this: iMac Intel 27" Retina 5K Display CPU Replacement. Really all you gain is hyper-threading so a multi threaded app can take advantage of these additional CPU's threads. Must user based apps are not heavy threaded so given the work you might not see any improvements.
The GPU is soldered to the logic board and not replaceable!
Frankly, given the risks of just opening this system if you've never opened one up should stop you.
Doing the HD swap out is the easiest if you're brave enough to pop the display. Adding the HD is a lot harder! As you need to pop the logic board out to gain access to the header on it to plug in the custom SATA/power cable for the drive. I would forget the SSD or CPU upgrade. It would be wiser at that point to sell your system to get a replacement with the bigger SSD or better CPU/GPU config.