The short answer, is that there may be an issue with your scissor clip, or the silicone dome underneath.
The longer answer...
All clips are designed to slide on one side, and will clip on the other.
On the Magic keyboard, they are upside-down when comparing with most earlier models. So when removing the key, to prevent any damage to the scissor clip pegs, (when removing the alphanumeric keys) you will need to pull the bottom side of the key up before the top.
As an example, here is a guide for an earlier version of keycap, so you have to be aware that your key is upside down in comparison, but this illustration will help understand how the key works:
Have a close look at the scissor clip underneath and make sure that the top 2 corners have the pegs intact still, and the bottom edge has 2 square holes for the key to clip onto. Then make sure the rubber dome (known as the silicone dome) in the centre is not damaged and inverts properly.
Then inspect the underside of the keycap to make sure the plastic pegs and clips are not bent or damaged.
The larger keys are slightly more complicated, as they often have retaining bars (metal brackets) that help keep the key straight when being pressed from different angles, but the alignment of these is crucial to getting a good response from it.
The space bar has multiple scissor clips, but only one silicone dome, so when resetting, the process is similar, you just have to be very carful when lifting it that you dont damage the pegs on the clips.
Here is an image of an intact clip and keycap for the magic keyboard. Align the orange section first, making sure the scissor clip tabs are under the loops, then press the bottom of the key (the red marked pegs) until it clicks into place.
Hopefully this will help.