Want to upgrade to have a faster computer

Sorry Joel your options are very limited here.

The system just can't support more memory as the RAM controller chip and the logic board just don't have the extra lanes for more memory modules.

As for upgrading the HD to a larger drive sadly the newer drives we have today are unable to run in your older system. Your systems SATA port is the first generation - SATA I (1.5 Gb/s) todays ultra large hard drives (4 - 12 TB) are only able to run in SATA III (6.0 Gb/s) systems. They just won't run inside your system.

Maybe you need to rethink things a bit here. Instead of trying to put the drive internally use an external drive. While hard to find I would look for a 3.5" FireWire external drive enclosure and then put the bigger drive within it. Do make sure the enclosure can support a SATA III drive some don't.

So what to do within the system? I would swap out your current HD with a SSD drive. There are a few SSD drives which offer SATA port sense technology which will work in your older SATA I system. As an example: Samsung 750 EVO will work. Here's its spec sheet: Samsung 750 EVO Specs. Note the interface line states clearly it will work in your older SATA I (1.5 Gb/s) system. With the external HD I would recommend you get a 128/256 GB drive as you don't want to spend to much here given the age of the system. Here is the IFIXIT guide you'll need to follow: iMac Intel 24" EMC 2134 and 2211 Hard Drive Replacement. In addition you'll need an 2.5" to 3.5" adapter frame to hold the smaller SSD and don't forget to remove the thermal sensor from the HD and place it onto the SSD!

Now the rub... Sadly your system is so old it just won't run the newer MacOS as its memory is so limited. In addition Apple is no longer maintaining the older OS's either. Maybe its time to think about upgrading to a newer system. Even used! Which will offer more options. I would recommend looking for a 2011 model.