Here's what I can piece together from what I've learned about Lithium Polymer batteries in the RC world.
The 3GS battery is a nominal 3.7V battery. This means peak voltage should be about 4.2V and minimum should be 3V (though 2.8V-2.7V is sometimes ok).
If you wish to test a lithium battery, or any battery for that matter, a load test would be best. WHAT load is a bit of a tough question, as different Lithium battery constructions have different load limitations. However, knowing that the battery capacity is 4.8 WHrs, nominal voltage is 3.7V and that these phones can usually last 24 hrs under a heavy load, my guess would be that a load of .05A may be good for testing. What you would do is put a fully charged battery under a .05A load and observe battery voltage. Lithium batteries hold their voltage very well under load, so any major fluctuations at the application of the load could indicate a bad battery. Voltage should slowly drop as time passes with the final voltage being absolutely no less than 2.7V.
A word of caution, do not attempt to draw too much amperage from the battery and be cautious of trying to bring dead batteries back to life. This can cause damage to the cells and they make rather nasty fires that spew toxic smoke.
Hope this helped. Let us know if you need to know more!