Use one hand to grip the back of the controller piece with the joysticks. With the other hand, grip the rubber cap of the joystick and carefully pull it off of the joystick.

You may need to apply a bit of pressure to pull off the rubber cap.