Why does my headphone jack sound intermitten?

Recently, my headphone jack for my 2nd gen iPod touch only wants to work on the right side. I use it at the gym and in my car, so 2 different things plug into it, and I have to mess with it until I get good sound on both speakers or headphones. Ive already taken it to the apple store and the guy pulled a big fur ball out of the headphone jack, but its still doing the same thing. Please help.