Alternating screen from blank, Apple logo, very low battery charging.

Have iPhone 4 that only came on when plugged in to charger and then it only showed the Apple logo that would come on then go off then come on then go off ..... Read on the site that I should try replacing battery. Replaced battery with new one then it would alternate (when plugged into wall charger) Blank screen, Apple logo, picture of low battery (red) charging over and over. Left it plugged in cause it looked like it was charging, after 13 minutes it went completely blank and will not respond at all now