I didn't use a pump for the longest time, but my chilling process took quite a while with merely an immersion chiller - I started using the whirlpool immersion chiller by simply adding a little copper and some silicone high temp hoses and haven't looked back...so you might want to consider that. But I brew 10 gallons typically, so YMMV. If your burner seems adequate, then you may save more time in your brew day by getting a pump. Also, the pump can be used to save your back, if lifting is an issue.
That all assumes your burner is relatively adequate, of course, because a poor burner can add a lot of time to your brew day.