Well I don't know what you are referencing on the bottom and top LEDs. But if it's not power on at all, it is most likely a failing power supply from the way it acted. It does sound like you may have changed the hard drive but not taken care of the thermal sensor issues. What are the diagnostic LEDs on the logic board telling you?
LED 1 - Indicates that the trickle voltage from the power supply is detected by the main logic board. This LED will remain ON while the iMac is connected to the AC power. The LED will remain on even when the computer has been shut down or put to sleep. The LED will turn off only if the AC power is disconnected or the power supply is faulty.
LED 2 - Indicates that the main logic board has detected proper power from the power supply when the computer is turned on. This LED will be ON when the computer is turned on and the power supply is working correctly.
LED 3 - Indicates that the computer and the video card are communicating. This LED will be ON when the computer is communicating properly with the video card. If LEDs 1 and 2 are ON and you heard the startup sound, but LED 3 is OFF, then the video card might be installed incorrectly or need replacement.
LED 4 - Indicates that the computer and the LCD display panel are communicating. This LED will be ON when the computer is turned on and video signal is being generated. If the LED is ON and there is no image on the LCD display panel, the LCD display panel or inverter might be installed incorrectly or need replacement.
Don't bother with the thermal sensor until you have your other problems solved.