- I would assume a SW problem before a HW problem. Sometimes the issue is as simple as a corrupted kextcache, and you can fix that by booting with the shift key down. (This boot will take a LOT longer than usual. That's normal.) If that doesn't work and you are running Lion or Mountain Lion, can you successfully boot of the recovery partition (boot holding down command-R)? Or can you successfully boot off the CD that came with the iMac? If booting off a foreign source works, then the obvious first thing to do is run DIsk Utility on the boot partition. If that reports a problem and fixes it, try to boot again. If it tells you there is a file system problem that can't be fixed, Disk Warrior (which costs some money but is worth it) may solve the problem. Otherwise, worst case, try to re-install the OS (I assume you have a Time Machine backup) and maybe that will help.
I know you're joking, but calling QAM256 "four times the modulation" is deeply misleading.
Speaking very simply, QAM64 packs 6 bits (ie log in base 2 of 64) into a "single Hz" of bandwidth, QAM256 packs 8 bits into the same "single Hz". So the jump in performance from this particular change is a factor 8/6=1.33 --- nice but not 4x.