Your system has two storage connections 1 SATA 3.5" HD & 1 PCIe SSD blade drive.
To start with the newer 'Thin Series' systems are much harder to open and if you are not careful you can easily damage the display which would be an expensive mistake! Carefully review the IFIXIT guides and make sure you use the correct tools (don't substitute).
As to your choices here, yes you could replace the 1TB HD for a SSD. You will need to get a 2.5" to 3.5" frame and you'll need this adapter cable: OWC In-line Digital Thermal Sensor for Hard Drive Upgrade for 27" iMacs 2012 and Later as you need to replace the thermal sensor for the SMC services from running wild without the HD input. You see, the SSD does not have the needed sensor, the in-line adapter cable replaces the missing sensor.
If you just bought the system you may want to see if you can exchange it for a SSD model, then add in a HD configuring the system as a Fusion Drive. That way you gain the space of the HD but also have the speed of the SSD.
Update >> 8/7/15
The Blade SSD is mounted on the other side of the logic board and requires a lot more work to put in. So far no one offers the custom Apple part. Which is why I was recommending you exchange out your system if you can with a SSD model.
Review these IFIXIT guides to see the differences:
We suspect the Apple iMac 5K SSD is a custom 2 lane PCIe-3 SSD blade drive (Apple hasn't disclosed the SSD it uses spec's).
Here's a good reference on what you could get with a SSD PCIe drive Vs SATA drive. Intel’s 100-series chipsets: DDR4, PCIe 3.0 SSDs, and other Skylake supplements, review the section: PCI Express 3.0, and what it means for SSDs