I use a 5 gal. Rubbermaid round cooler for 2.5 gal. batches. My last batch I started with 5.125 gallons water and mashed 7# of grain with just a bit over half of that water. I heated it to 161°, poured it in the cooler and added my grains. A few minutes later, i measured the temperature and I was at 152°, right where I wanted to be, and at that point I started timing my mash. Not only does the cooler absorb some of the heat, but the grains also are absorbing the heat, too. More grain, more heat loss. Less grain, less heat loss.
I like JT's idea of pre-heating the cooler. Might have to give that a try. And one thing you can do to prevent a bit of heat loss is to fill the lid, if it's hollow like mine was, with some foam insulation. Might save you a degree or two over the course of the hour.