Also, it has an impact on hop utilization, and I figure my hopping based on the wort gravity with the sugar in there.
I tend to do the opposite here Denny and try to get more hop components into the unladen water
before I laden the water with additional sugar...
As long as that's how you calculate your hop additions, no problems. I use Promash, which calcs things based on boil gravity, so I want the sugar in there. In either case, I don't think it makes a huge difference.