A couple of points here from a design engineer of 30 years. On charge (and fully charged) a LiPo (Lithium Polymer) battery should have about 4.20 - 4.30V across its terminals. With the charger freshly switched off the voltage will be about 4.1V. When fully discharged the battery voltage can be as low as 2.7V. Any lower than that and the battery is permanently dead! That's why most LiPo batteries have overdischarge cutout built in.
As for Jonathan's Amps, if you short out the iPhone battery (or connect your multimeter across it set to measure current) I would expect TENS of Amps to flow. This is a bad idea! Never connect a multimeter across a battery when set to measure Amps. The ability of a battery to supply current is not related to the capacity of the charger.
Measuring voltage is fine, but as Jonathan correctly points out it tells you nothing about the Amp-Hour capacity of the battery. To measure this you need to take a known current out of the battery and measure the time taken for the battery to reach end voltage (2.7-2-9V). Amp - Hours is exactly that, Amps(load) x Hours (Time).