I prefer to add honey to beers 2-3 days after fermentation has been well under way. There is already alcohol present which will deter any infection from wild yeast or bacteria, and still plenty of yeast rarin' to go to ferment fresh (and simpler) sugars, as long as your chosen yeast is tolerant of higher alcohol %.
No problems after many years of brewing with honey this way. I use the least-processed honey I can find (i.e. not super-market pasteurized honey). Local beekeepers' honey is the best and most aromatic, but at least in my neck of the woods, the only local beekeeper honey that is available is usually wildflower. For varietal honey, I have to drive a bit, or order online.