I'm not sure what you're talking about when you say "the whole thing". The removable battery is here: MacBook Battery
Here's how it's replaced: Installing MacBook Core Duo Battery
Here's the internal PRAM battery: MacBook PRAM Battery
Here's how the PRAM is replaced: Installing MacBook Core Duo PRAM Battery
If you are talking about the Backup Battery (CMOS battery if you are familiar with PC's), yes. On your model, it's under the logic board.
Apple part number for hte replacement is 922-7369.
If you are talking about something in the main battery, i do not recommend taking apart any Li-ION batteries as it can be dangerous.