Your system will not accept or work with DDR3, it works with DDR2. The specifications are totally different. The notch placement on the chip and RAM slot are different. So the DDR3 chip will not properly fit into a DDR2 slot. The chip handling memory management on your logic board wasn't programmed with how to deal with it. So even changing slots on the board will not work. Click here and click here for verification of that.
Officially Apple says this machine supports up to 4GB of PC2-5300 DDR2 RAM. However many people have been able to install 6GB of that RAM. Click here for verification of that.