I have an old G5 iMac. The momentary push button power switch is mounted on the back of the case….the removable cover. It is pretty imbedded in a metal shrouding that lines much of that case cover along the inside. The momentary switch produces a "relay" function while it is depressed to send low power to the high power, "On" circuitry. In the symptoms I experienced, it seemed to be getting longer and longer of a press to get the power to kick on and then it wouldn't power on at all.
My fix is not necessarily for the faint of heart but is pretty cheap:
Clean the contact plates and pins of the momentary switch/motherboard transmission switch..
Details. I pulled all of the cables and the power supply cord. I opened up the case by backing out the 3 screws along the bottom and pulled the back cover off. The momentary switch is partly revealed on the inside of the back cover. What you see in my machine is two gold colored (and presumably gold plated) contact plates about 0.5-cm square side by side, separated by about 1-mm of space in an exposed, circular cutout portion of that metal shroud. This is aligned with the transmitting switch position which consists of two spring-loaded pins coming out of the plastic housing at the aligned position coming off the motherboard and other interior components. Those pins are also gold colored (and presumably either brass or gold plated).
Those pins individually make separate contact with the respective contact plates of the momentary switch as the back cover is put in place and "transmits" the completed circuit condition of the depressed momentary onto the motherboard power logic circuit. This configuration seems to allow some wiggle room in the momentary switch alignment with the motherboard foregoing direct wiring of the cover-mounted momentary to the mother board with a long set of lead wires that would be needed to allow for case/cover removal on this rear-mounted momentary on the removable case cover. While installing the back cover, that placement action aligns and presses the momentary switch contact plates onto the contact pins, simultaneously loading the springs of the pins to help ensure good contact.
Check the momentary switch at those contact plates using a volt-ohm meter to test whether it is shot from the inside-out. If you can't make a circuit complete between those contact plates when pressing the button then the momentary switch is probably a goner and you'll need to consider a replacement momentary switch. If you can make a circuit, I'd optimistically suspect the contact condition with the pins is poor. Over time, the plates and pins can oxidize, get dusty, coated with airborne pollutants and vaporized plasticizers that recondense, etc, making a progressively poorer contact.
If you are really bold and comfortable with working around exposed powered up microcircuits and are sure you've correctly identified those pins, you can confirm this further: plug the power cord back in with the back cover off and use something metallic to bridge those spring-loaded contact pins. (I've used a paper clip successfully and alligator clip leads, too.) DONT TOUCH ANYTHING ELSE!!!! If the machine then starts to power up normally, you can be pretty certain you have isolated the problem to the contact condition between the momentary switch and the pressure loaded contact pins which might afford a simple solution. At that point, as the machine powers up, you can also inspect the LED series as well to ensure all the other pieces are functioning up to the self-test standards of those LEDs. If those LEDs indicate function then you have probably further isolated the trouble to the contact condition.
(You might need to plug in the mouse and keyboard cables at that point to finish that inspection. Power the machine back down in the usual way via the software shutdown instruction in the OS menu. (yeah, it's a gyration with that puppy open on your desk but it can be done.)
Subsequently, A good cleaning of both the contact plates of the momentary switch and the pins of the board mounted transmission switch contacts with a Q-tip and Isopropyl alcohol or even electric contact cleaner should help the situation if that is the problem. You probably don't want to use any "real" abrasive like superfine sandpaper to super-shine them since those plates don't have lot of metal and you'd leave residual grit. I've done this contact clean on a couple of G5s and it seems to have put them back to regular function.